A compressor is basically a variable gain device that is used to either try to smooth out the dynamic range of a signal, or is also sometimes used as an effect. Different compressors have different types of characteristics and sounds that they impart to the source signal as well... the really expensive compressors are often expensive because of the character of their sound that is desirable for a lot of applications.
Basically, the way a compressor works is that there is a threshold control that determines at what signal level the compressor starts working it's magic. Any signal that exceeds this threshold level is attenuated (reduced in volume) in proportion to whatever the compression ratio is set at. For example, if the ratio is set at 2:1, that means that for every 2db change of signal level above the threshold from the source input, the compressor will only output a 1db change. If you set the compressor at very high ratios or 20:1 or more, then you are getting into what is more commonly termed limiting, because basically at that point the compressor/limiter is not allowing the signal to rise at all above the threshold level, thus "limiting" the output to a certain level. In digital recording, a limiter is sometimes handy to prevent clipping the signal in the digital domain by not allowing it to exceed a certain threshold.
Other controls you will see on some compressors are "attack", "release" and "knee" settings. The attack time determines how quickly the compressor reacts to a signal once it crosses the threshold level. The release time determines how much time it takes for the compressor to return to it's zero gain state once the signal passes back below the threshold. The knee setting determines whether the compressor works only on signals that are above the exact threshold level, or if it starts kicking in gradually as the signal level approaches the threshold. "Soft Knee" settings will allow the compressor to start kicking in gradually as the level approaches the threshold, and is desirable for certain types of material, while "hard knee" settings will not affect the audio at all until it reaches/passes the threshold level, and is also desirable for certain types of material. Some compressors also have input and output gain controls. Many compressor have automatic settings for attack and release times that are based on the incoming signal itself.
Learning how to use a compressor properly takes a lot of practice and experimentation. If you set the compressor wrong, you can get a "pumping" effect where you actually here the compressor reducing the gain and then bringing the gain back up, repeatedly. If you aren't going for a special effect, then the best compressors and settings are when it does its job but you can't hear it working. If you can hear it, then you've gone too far (usually). For special effects, sometime people will purposely overdrive a compressor to either make it distort (popular with many tube compressor where you get a certain type of tube distortion) or to simply impart a certain unnatural sound on a source. A common application of this is to set up some room microphones for a drum kit (in a good sounding room) and then run the room microphones through a compressor set to really extreme settings. This really over emphasizes the reflections and ambience from the room and can make a drum set sound really big when blended in properly with the rest of the drum tracks. Other people have used compressors set to extreme settings to purposely distort vocals or other instruments. Sometimes tracking or mixing engineers will use several compressors one after another to really flatten out a vocal or instrument performance, without trying to go for an extreme effect. The first compressor is usually set to mild settings with a lower ratio to do a little bit of smoothing out, and then the next compressor will be set to act as more of a limiter to catch the bigger peaks that make it through the first compressor. Some compressors have a second limiting stage built into them that emulates this type of set up.
There is no right or wrong way to use a compressor, but it takes a some practice and experience to get the sounds you are after. Pop and rock music uses a lot of compression.... the individual tracks are usually compressed during tracking, and then often again during mixing, then the overall mix is usually compressed, and then it is compressed again during the mastering stage, plus if it is broadcast over radio, the radio station has its own compressors that the signal runs through first. Classical and Jazz music, plus a lot of acoustic type of music, is usually not compressed, or at least not nearly as much. Again, there are no rules, and many people do things many different ways.
The loudness of a signal to our ears is mostly determined by the average, or rms, level of the signal. Large, short-duration peaks in a signal usually will not make a noticeable difference in our perceived loudness of a signal. The purpose of a compressor, then, is to help smooth out the dynamics of the rms level of a signal to make it appear to have a more consistent loudness to our ears. Using a compressor, you can actually make a signal appear to be louder since you can reduce the overall dynamic range (the difference between the softest parts and the loudest parts) and then bring up the overall average level (using the make up or output gain control) without overdriving your recording or playback medium. This is the most common use for a compressor, especially during the tracking stage. When recording to analog tape, the engineers want to keep the signal level as hot as possible without overdriving the tape and causing distortion. If the signal level is too low, then you begin to hear the tape hiss. Thus, engineers employed the use of compressors to allow them to decrease the overall dynamic range of a source and then record it to tape at a much hotter level without having to worry so much about possibly overdriving the tape. In essence, by making the louder parts quieter, the average level to tape can be increased without distorting or clippling, and the softer parts are thus raised up even further away from the noise floor (tape hiss). The use of compressors while tracking has carried over into the digital age as well. In the digital world, you don't have to worry about tape hiss anymore, but you still want to print as hot a level as possible to get the highest sound quality, but you definitely don't want to overdrive a digital recording device since digital clipping is very harsh and unpleasant (you can get away with some tape distortion since it is actually a desirable sound in many styles of production). Thus, compressors and limiters are just as important, if not more important in the digital age.
Compressors are also quite handy for tracking certain instruments like bass guitar or horns where notes played in some registers (or on different strings for bass guitars) are inherently louder than others. In this situation, a compressor will help to smooth out these differences in volume.
Using compressors during mixing is extremely common in most modern musical styles. By using a compressor on a track during mixing, you can level out the dynamic range even more and make that track sit better in the mix. The more dense the mix is (more tracks playing at the same time) the harder it will be to hear all the subtle details in a certain part without constantly riding the levels. By adding a compressor, you can smooth out the dynamics of the part and raise the average level of the part until it is not getting lost amongst all the other parts. On very aggressive mixes, it is not uncommon to have a compressor inserted on almost every channel and then a compressor inserted across the stereo bus as well.
As much as compressors can be a good thing, they can also be a bad thing as well. In general, you are always going to lose some of the transients and detail of an instrument by running it through a compressor. The more aggressive your compressor settings, generally the more damage you are going to do to the signal. If a compressor is overused, you can actually end up having the opposite affect of what you intended. Instead of making the signal appear to be louder overall, by over using a compressor, you can actually make it seem dull and lifeless without any punch. As with all good things, generally when used in moderation, compression can be a good thing, but it can easily be abused as well. Most pro engineers are usually more cautious and conservative with their compression settings during recording because although you can always add more compression later during mixing, you can not take away the negative affects of too much compression after it has been printed to tape.
As already mentioned, the attack setting on a compressor determines how quickly the compressor will react to a signal the crosses the threshold level. The setting can be very crucial to getting the sound you are after. If the attack setting is too fast, then the initial peak transients of the source signal will trigger the compressor, as opposed to having the compressor act on the average (rms) level of the signal. If the compressor is constantly being triggered by the loud fast peak transients, it will end up reducing the volume of the overall signal instead of helping us to make it louder and smoother. By increasing the attack time, you allow the initial transients to pass through uncompressed, and the compressor then acts on the average rms level rather than the peak levels. For vocals and sustaining instruments you may want to make the attack time even longer to make the compressor kick in a bit more gently over a longer period of time (so that it is not noticeable). Again, you have to listen very carefully to the source and choose your settings based on what your ears are telling you. If you can hear the compressor working, then you probably need to back off on it, or readjust your settings.
The release time on the compressor is probably a more critical setting than the attack time, but they do work in conjunction with each other. A good way to go about it is to try to time the release settings to the rhythm of the part that is being compressed. Try to make it so that the compressor returns to a zero gain setting just before the next note is played, otherwise the attack of the next note is going to be softer since the gain hasn't returned back to zero yet. Of course, if the part has a combination of very fast patterns and slow patterns, you'll just have to experiment with the settings until you find a setting that works for all the different sections. If an instrumental part is very rhythmic with a lot of quick notes, you would probably not want to set a very fast release time or you would probably hear the compressor pumping along with the rhythm of the track, and in those situations the compressor is set so that it is triggered just once by the first note, and then gradually releases over time as the signal gets quiter. Many engineers just try to go with as fast of a release time as they can get away with without the compression being obvious. Setting the combination of the attack and release times correctly will give you smooth, almost transparent compression, while setting them incorrectly will give a noticeable pumping effect to the sound.
There is an additional feature on many compressors called a "side chain" or "key input". This is a seperate input from the source input, and can be used to control the compressor from a signal other than the source signal. One common use of this input is to turn your compressor into a de-esser. A de-esser is a device that attempts to remove excessive sibilance from a vocal part. The way to create a de-esser with a compressor that has a key input is to make a mult of the source signal and run it through an equalizer with a high pass filter set to filter out most of the signal except for the range where the sibilance is the worst (usually somewhere between 4000 hz and 10000 hz). Then this equalized signal is sent to the key input. Now, whenever there are some strong sibilant sounds, this key input will cause the compressor to kick in and reduce the gain of the signal, but will not affect the signal when there is no excessive sibilance. Setting a de-esser properly has a lot to do with setting the equalizer properly, in addition to all the regular compressor controls. The key input can also be used creatively in other ways as well. You could take a sustaining pad or bass part and make it kind of pulsate in time to a rhythm track by feeding something like a hi-hat part into the key input (actually, this would be even cooler using a noise gate and triggering the noise gate with the hi-hat part so that the gate opens up with every hi-hat hit.... with a compressor, it would be the opposite, the level of the part would be reduced with every hi-hat hit).
Although the basic operation of compressors is all the same, there are many different ways to implement the design. There are tube compressors and solid state compressors, compressors that work with a photo-optical circuit, software based compressors (which many times try to emulate the characteristics of certain types of hardware compressors), and probably many other designs as well. Plus, all of these designs can have many variations amonst themselves. One thing is for certain, each and every device that you pass your signal through is going to impart its own sonic characteristics onto that signal. That's why there isn't just one type of compressor, or even one type of microphone for that matter. They all have different sounds to them that work differently with different types of source signals. Some compressors have a very clean and transparent sound to them and those types are used when you want to smooth out a signal without changing its characteristics too much. Other compressors have a very noticeable sound quality to them that people like to use to make things sound "fatter" or "warmer" or "edgier" or whatever. In those cases, the compressor is not just being used for its dynamic properties but also because the engineer or producer wants to impart that sound onto their signal (kind of like adding EQ). Many engineers and producers have certain compressors that they instinctively reach for in particular situations, but it's very hard to give specific recommendations for generalized applications. The bottom line, as always, is to trust your ears and use whatever sounds best out of what is available to you. Plus, don't forget, that some times the best sounds are achieved without the use of any compression at all... so don't always just patch a compressor in right away without listening to the source first to determine if you even need one or not. If you are not an experienced engineer who really knows how to set compressor properly, play it on the safe side while tracking and use less compression than you think you need, or don't use any at all and just be a lot more careful with your recording levels. You can always add the compression later when you are mixing.