I did the opposite last year and removed the bumper guards. There was no need to remove the bumpers to do this, I plugged the holes with chrome bolts.
Do your bumpers not have the holes drilled for the bumper guards? If you've got to drill the holes, then yes, it would be prudent to remove the bumpers.
I can measure the distance along the bumpers for the holes and let you have the measurements tomorrow.
Best,
Mike