I've never done this, but I wonder if you could remove the inner panel (vinyl covered) at least on the right (left would be tricky with the lever to release the lid) enough to tap the roll pin out that holds the handle on. Can't remember how it looks right now . . . .