Yes get rid of all that sound deadener. If there was rust under what you removed, odds are there's more. If there are no rust through places sand off as much as you can and then treat it with POR-15. IMHO I wouldn't rely on regular primer and paint. Then you can install sound deadener over the POR-15. The stuff is expensive but it is awesome stuff to stop further rusting. http://www.por15.com/ You can get it on EBay too, but it will get to you faster if you order straight from POR15.com. You can brush this stuff on or spray it, but trust me don't get any on you!