Keep that stuff on a shelf! Someday you may regret removal and need it. It's also a good thing to at least have any of the parts removed(not just in regards to the heater), when it comes time to sell the vehicle, because this will probably be a concern of the majority of interested buyers(and even if it's not a concern of theirs, they may use it to their advantage anyway to try to undermine the cars value). You'll never regret holding onto the stuff, but you may regret getting rid of it.
Just wait until you get caught in the rain! 🙁 You will eventually find yourself in need of defrost, on cool spring morning...you'll be on your way to work with you widow's down freezing your nuts off just to see were you're going.
Be ready to ride in her car on cool fall nights now, girls learn fast that cold cars don't make them hot, and they usually think a working heater is more impressive than any Dyno results.
I have been down this road before, the weight saved is not worth it and If you want to clean it up under the hood and do it right, get yourself an A/C Delete heater swap.