Plot hole: The Kaiju's EMP fries all electrical circuits next to him, and affects the base as well. If that's true, how did the helicopters carrying Gypsy survive the blast without having their electronics fried? EMP pulses induce current in all electronics regardless of whether they're powered on or not. (01:18:00)
Plot hole: Gypsy Danger grabs the wounded Kaiju and falls into the interdimensional rift, which opens in response to the presence of Kaiju DNA. That's how Gypsy Danger entered the Kaiju dimension, but Mako and Raleigh somehow escape the Kaiju dimension in their life pods without using Kaiju DNA. Obviously, the rift allows physical objects and energy to pass unobstructed from the Kaiju dimension into our dimension, but not the other way 'round. If this is the case, then Gypsy Danger's thermonuclear detonation in the Kaiju dimension should have burst through the interdimensional rift, as well, at least for a second or so, at roughly the speed of light. Given that Raleigh and Mako had only a 16 to 20 second head start ahead of the blast, they should have been vaporized.
Chosen answer: Most likely they only use it when they know they have a guaranteed shot and won't waste it. It can't be reloaded that easily. Like having one bullet to kill a zombie vs a sword so go for the sword save the bullet it's more valuable. That's typical military tactics. Possible firing whatever weapons could cost a ridiculous amount of money just like missiles do. It's not exactly like Yeager weapons were mass produced they are all different. Last reason is reducing extra casualties by making sure they don't miss when they fire or making firing weapons the last resort. And just for the movie's sake it would be boring as crap if they just blew the heads off in seconds.