I use a 5 volt charger and they charge a lot faster then if I use a 2 volt charger ?
All wall chargers are not the same
Not sure what you mean there. USB is 5 volts. That's it. There's no variation there. The Li-ion batteries used in vapor devices actually charge to 4.2 volts though they run at about 3.7 (the chemistry makes the numbers kinda oddball).
But the amperage matters. The "cig-a-like" batteries (like a Blu) are charged at around 150mA (milliamps). Bigger "stick" style batteries (like a Twist or Spinner) are
usually charged
around 420mA. The 18650s I use in my VTR are charged at 1000mA (1 amp).
Some batteries can be charged at higher amperages than others. Which is faster. Larger batteries
can be charged at lower amperage but it'll take forever. Smaller batteries
can't be charged at the higher amperages unless you like starting fires in your house.
(Mismatched chargers are the cause of almost every battery "explosion"--which is not what happens actually--that the "news" likes to yap about. They find doing actual research to be boring but screeching about E-CIGS WILL BLOW UP YOUR HOUSE is entertaining and sells ads.)
So since the OP has batteries that are taking forever to charge and the "stick" style batteries have two different chargers that can be hard to tell apart (I have a 150mA and a 420mA, they look exactly alike), it's possible it's the lower powered charger.
The other way around (higher powered charger on smaller batteries) would be, ahem, blazingly obvious. In the sense of the battery would overheat, vent, and burst explosively...