A thought to consider.
Singer, a car modifier who creates some of wildest 911's out there, remove all of the exterior body panels that are made of steel and replace them with carbon fiber...EXCEPT for the doors.
Their cars are well into the 6 figures, so it's not like they just didn't get around to it.
I do not know the reason for them not do it. It could be a safety thing, or possibly a legal issue, But even they don't.
Personally, I wouldn't. There's a lot of other areas to lose weight on the car if that's what you're after.