Rovers were the better side in the first half and deservedly led at the interval through Gael Givet's stunning strike.
But Emile Heskey brought the sides level after just coming on in the 59th minute and then Ashley Young struck twice to finish the job off.
Allardyce said: "We've gone in front in six games out of seven and haven't been able to hold onto things. A game lasts 90 minutes, not 45 and it's frustrating for us.
"We couldn't cope with Villa's change in shape. We must be more resilient defensively and just keep working away at things."