Smaller size bullets almost ignored compared to large scale ones?

[Neikke]

Hi guys, please help! I want to achieve the look of a thick metal plate being pierced by small 5-6cm bullets but if I make them this small - they don't seem to pierce the metal plate at all, just barely bending it. So here's for comparison 30cm bullets which is fine but the problem is that the holes are too big which I don't want: 

 

And here's what happens if I make them small (5-6cm) - so basically no piercing occurs here:

 

My metal wall is fractured to 5000 pieces by Voronoi Fracture, but even if I crank it up to 10K pieces it's still almost won't detect any collisions: 

 

Please help and thanks in advance!