Yes, you need to remove the grill because the light assembly won't fit through the opening. Best to just put dishwahing soap or something in through the bulb opening, add some water, shake, and rinse. Then use a hairdryer or something to dry. You'd probably break the lens if you tried to remove it and clean them that way. Usually they just have dust in them anyway and just shaking the water and detergent around will get them reasonably clean.
And if that doesn't work, you can find them on eBay for cheap. For that matter, I have two that are in very good shape that I would be willing to sell. They have some dirt and scuffs, but not bad.
_________________________
ChuckH
"Every morning when I wake up I know it's not going to get any better 'til I go back to sleep again!" Al Bundy