This page will help you to understand guitar geometry as related to guitar setup and intonation. It explains the causes of intonation error that necessitate both nut compensation and saddle compensation.
Before getting into any details or mathematics, let’s bring to mind what we may already know about guitars (or other stringed instruments).
Regarding mathematics: with a little effort and exploration, we can understand the guitar pretty well without math, just as we are able to understand a lot of other complex things without math, if we apply ourselves to the challenge, and take a closer look. However, there are some readers who would like to see the applicable math, for deeper understanding, and for verification of my claims. Therefore we have a separate page, Stringed Instrument Math, with the pertinent equations and notes.
The guitar is fretted to produce musical notes in equal temperament. Equal temperament means that the 12 notes within each octave sound evenly spaced. Accurate equal temperament enables us to play in tune in all keys.
The frets are placed closer together as the notes get higher, such that the frequency doubles in the span of an octave. Each space between frets is smaller than the previous one, by an exact percentage. Even so, the notes sound evenly spaced to our ears. An octave occurs at the 12th fret, which is half way between the nut and the bridge saddle (the vibrating string length is halved).
Raising the tension of the string raises the pitch (while keeping the length and weight of the string the same) The heavier the string, the lower the pitch produced (while keeping the same tension and length)
While playing the instrument, we naturally think in terms of the distance from the nut to the neck position that our hand is moving to. But, while evaluating intonation we need to keep in mind that it is the vibrating portion of the string, the length between the selected fret and the saddle, that actually produces the sound and pitch. (Or between nut and saddle, if it’s an open string.)
Fretted notes sound out from the length of a string from the selected fret to the saddle. The frets are distanced so that the correct length of string, from each fret to the saddle, will produce the correct pitch. The fret layout plus accurate saddle compensation results in all of the fretted notes playing in good tune.
Keep in mind that, for all fretted notes, finger pressure tension has been added (increasing pitch) whereas open strings are played at lower tension. That's why intonation for opens needs to remain separate from fretboard intonation.
The fretboard layout (distance between frets) is mathematically analogous to the 12-tone equal temperament musical scale. Bridge saddle compensation actually aligns the fretboard, compensating for some of the effects discussed below, such that fretted notes over a wide range are in very good tune with each-other (but not with the opens until nut compensation is done).
Let’s start with the physical guitar strings.
The material used in a guitar string has a resistance to being stretched (elongated). (Nylon strings are pretty stretchy compared to steel strings, but similar strings of different manufacturers generally have similar characteristics). Pulling a string down to a fret requires force. The string is forced to lengthen, increasing the tension on the string, which raises the pitch of the length of the string from the fret to the saddle.
Thin strings are easy to stretch compared to fat strings. Though the strings are likely to be of the same material, a fat string has more material to pull against. Therefore, fatter strings are harder to lengthen, requiring more force and more compensation (Proportional to the area of the cross section of the string.)
Now consider wound strings. The windings are used to add mass (or weight) to give the string a lower pitch, but being wrapped around the core, they do not contribute to the resistance to lengthening. So it’s only the core of a wound string that resists lengthening.
(We can now see why both the first string and the smallest wound sting need little or no compensation.)
Lateral stiffness, resistance to bending, prevents the strings from freely bending at the edge (or release point) of the nut and of the saddle. To compensate for that, a small amount of length must be added to the vibrating string length to the saddle, which allows for a small arc, rather than a sharp bend of the string.
Though the material may be the same, the amount of stiffness resistance here again depends on the string size. As for wound strings, stiffness mainly comes from the core, and the windings contribute very little.
Saddle compensation is necessary primarily because of the additional tension created from the strings being fretted and from string stiffness.
Intonation correction should be made at the saddle - not to make the 12th fret note in tune with the open string, but to make, as much as possible, all of the fretted notes in tune with each other. To do this, we need to move the point of suspension of the string on the saddle away from the nut, to increase the vibrating length of the string until the note fretted at 12 or 14 is exactly in tune with the note fretted at fret 2. Then, assuming that your fretboard is accurate, all of the notes from fret 1 to fret 12 and beyond will be in good tune.
Why are we not using the open strings to adjust the saddle? Because, if the nut has not been compensated, the open strings are out of tune with the fretboard! (I know, this conflicts with what you have always been taught.)
We have noticed that some strings require more compensation than others.
Why do we need to compensate at the saddle? The following are main elements of intonation error, requiring compensation at the saddle:
Saddle compensation takes care of string stiffness and other effects that vary over the length of the fretboard.
When we fret a string, we add tension to the string, raising the pitch. Lengthening the string at the saddle compensates for that added tension by resulting in lower pitch for any fretted note, compared to the open string. We increase the string length by changing the point of suspension on the saddle.
Generally, as we move up the fretboard, the tension becomes greater, because the strings are higher off the fretboard, but this is somewhat countered by what I call the Clothesline effect - the string is much easier tp push downward near the center than near the ends. Compensation at the bridge takes care of this nicely because the added distance to the saddle becomes a greater proportion of the vibrating length as you move up the fretboard.
Because of the stiffness of a string, the string does not bend perfectly at the edge of the saddle. The effective point of suspension, as far as vibration is concerned, is beyond the actual point of suspension. The effective point of suspension at the bridge is just a bit in front of the saddle. Also, the effective point of suspension of any fretted note will be ahead of the top of the fret. Both of these contribute to the need of saddle compensation.
We may not have thought of the stiffness effect before, because we usually compensate by trial and error, and make the required adjustments irrespective of the specific causes of intonation error.
We can easily see now, why the smallest wound string needs less compensation than the biggest unwound string: the stiffness of a string is largly determined by the string’s core size, not its overall size, and the core size of the smallest wound string is small compared to its overall size. It is also smaller than the biggest unwound string.
Compensation at the nut is needed in a similar pattern as that of the saddle - strings with a bigger core need more compensation. Compensation is done by effectively moving the nut toward the first fret, either by shortening the fretboard and moving the nut forward, by extending portions of the nut over the fretboard, or by a combination of the two.
Why do we need to compensate at the nut?
We need compensation at the nut to make the open string in tune with the notes at the lower frets.
For bridge compensation, I cited (above) pulling the string to the top of the fret, however, the added tension from pressing the string on to the depth of your pull is compensated at the nut. That added tension is approximately the same at any fret. (If there is any variable component of the added tension, it will be included in the saddle adjustment.)
We don’t normally pull the string all the way to the fretboard, and we don’t all pull to the same depth, so it’s necessary to be mindful in testing, to approximate the force that the guitar owner will use.
We often set the nut slots slightly higher than the level of the frets. This allows for wear and tear and a safety margin. However, some people set the nut a lot higher than the first fret. In any case, pulling from the added height of the nut requires some nut compensation.
Because of the stiffness of a string, the string does not bend perfectly at the edge of the nut. The effective point of suspension, as far as string vibration is concerned, is beyond the actual point of suspension on the nut. However, the same thing happens at the first fret and each subsequent fret: the effective point of suspension is beyond the actual crown. Therefore, the effective distance from the nut to the effective point of suspension of the first fret is equal to the actual distance from the nut to the first fret, as far as string stiffness is concerned. Therefore, string stiffness is not a significant part of nut compensation.
(Note: One luthier has advised me that the string actually moves more freely at the fret crowns than at the nut. I don't know if this is true, but it could mean that nut compensation would be slightly affected by string stiffness, and that it would be in the reverse direction as other components, ie, tending to shorten the effective distance to the first fret.)