1) They fail to reach a decent pressure. Either the physical effort is too much, and your arm falls off first. Or, the valve seal cannot take the pressure and as you pump the air leaks out.
2) You have to hold the pump directly onto the valve. With the pump acting as a perfect lever, you can snap the valve core off a presta valve. And the more effort you put it, the more likely you will snap it.
The one time I did snap a valve core, I had no spare tube and I ended up walking down one junction of the A14 to a rendezvous point for rescue to save a very long walk. With endless trucks thundering past I say never again and I have always carried a spare tube since!
It's taken me a good while to find it, but I think I have found the right pump for me. It's a pump with a hose, and will fill fat tyres on the MTB, and thin road tyres to a good pressure too. The pump is a Topeak Mini Morph. The quality is ok, could be better, but what I am raving about is the design - it's a mini track pump. There are other brands too.
You put the pump on the ground, with one foot on the little flip out piece of plastic. You then pump with one hand. I cannot stress enough how much easier it is to pump towards the ground than in the traditional way.
I bought this pump just before a week long cycle tour. I wanted to be able to get the tyre pressure on a 32mm tyre back up to a high enough pressure to carry me and my 20kg's of kit. For that purpose it works.
I did a test of this pump on my touring bike tyre. 40psi comes reasonably easy after after about 60 easy and quick sub-second pumps. All other pumps I have owned struggle to get 40psi, mainly because you give up due to exhaustion. Another 60 pumps, starting to get slower perhaps once per second now, gets you to 60psi. I would normally give up there as it is enough air to continue a ride with. I have tested to 80psi, that's where you need to put a fair amount of effort really leaning on the pump now and I think the realistic maximum. To get more, you need a pump that puts less air in with each stroke.
With the pressure at 60psi+, you do have a hard time removing the head from the valve. This seems to be a problem on a number of pumps I have tried. My full sized track pump does not have that problem.
Folded up, the pump is a little large. It's just a bit too big to fit in an underseat pack. On the road bike I use the frame attachment it came with. On other bikes it goes in the back-pack or panniers.
The pump is also not ideal for mountain bike tyres with their large volume. You need to do a lot of pumping. It's easy and takes a couple of minutes but you do feel like a pump pushing more air per stroke would take less time. But, it does work and is still much easier to get 30-40psi than a traditional pump.
[update]. A 29x2.1 tyre takes around 250 pumps and 2 minutes to a rideable 20psi. The first 200 pumps are low effort and fast. 200+ is where you start to slow. 20psi is fine for a front tyre but I would go nearer 30psi on the rear and would happily spend another 2 minute to avoid a pinch flat.
If you only ride bikes with one size of tyre, or you are prepared to buy more than one pump then you might be able to get a better pump, but as an all rounder the Mini Morph is excellent. I rarely say that about kit after a year. I would definitely buy another mini track pump if it broke, even at £25.
Worth a mention are CO2 inflators. I have not used one, but have watched three people use them. They are impressive, very quick to inflate a mountain bike tyre with zero effort. I think you get, but nobody seemed to be able to answer precisely, about two inflations per cannister. But, that comes at a price - £1.50 or more per cannister.
[update] I now have a CO2 inflator for racing. They are fast when they work but the Weldtite JetValve is not foolproof. The first three attempts, I lost the CO2 before I got it in the tyre whilst trying to thread the cannister onto the adapter. The second issue is that because it is so fast and without a flow adjuster, you will have no chance getting the amount of air required to find a hole in a tube - replacing with a fresh tube is the only realistic option. These are risks I'm prepared to for racing, but on day rides I'll stick with the pump - it works every time.