Actually, I think smaller is better. When we drive rallies it is very noticeable how much nimble, lightness, and narrowness helps in travelling quickly. The Alfa Romeo Giulia Sprint GT we use is very, very narrow at 1580mm, and can squeeze through spaces...
When you see European cars that in our youth we used to think normal sized, or even on the larger side (e.g. a BMW 2002, or Alfa Romeo Giulia (below), or a Saab 96) next to a modern Mini, you'll notice that the Mini is taller, wider and longer than these old cars. The VW Polo of today is larger, and certainly heavier than a VW Golf of the first generation.
On Top Gear yesterday Richard Hammond tested the new, lightweight Alfa Romeo 4C which is very light indeed at ~ 850kg and with a 1741 cc engine, delivers 240 hp and a top speed of 258km/h (160mph). He was horrified at the width of the car which is over 2010mm including door mirrors. That's over 2 meters... and makes making progress through Italian villages quite hard.