Maybe you shouldn't, but you can, as I have had to do a couple of times. New cam bearings are precision made but as they wear you can sometimes sneak a used one in to replace a broken or damaged one. As long as the cam will turn freely after everything is torqued down you should be OK.
I had an engine in a while back ( 190SL ) and both end bearings were broken off at their flanges. Closer inspection showed where a previous shop had hooked chains under the head bolts and used that to lift engine in and out of the engine bay! This clever approach broke the bearings off about an inch above the top of the head. I replaced the cam bearings with good used ones and it still runs fine several years later.
