First we really need to know the year, make, model, and sub model of your car and engine, stock or highly modified, etc.
It could either be something wrong with the engine giving it a slighty weak idle that shows up under the increased load of the A/C system or something is wrong with the A/C system causing excessive drag. I would first check the idle speed in drive with A/C system on and off. Most engines have a method for automatically increasing idle speed when the A/C is on but it varies with each engine brand, Chevy, Olds, and Buick. Chevy uses a simple kicker solenoid, Olds and Buick use more complex methods for kicking up the idle. Its likely you either have a vacuum leak or late ignition timing.