4 Ways to Remove a Paper Label from a Plastic Prescription Bottle … 12.05.2021 · Peel or scrub off the disintegrating labels. By this time, the baking soda will have broken down the adhesive to the point where they may come off on their own. If not, you’ll need to help them along by hand, or with a soft sponge. You shouldn’t take much to get your bottles cleaned up and ready for their new use.