Modes of limited transposition
Major scales have twelve different transpositions. This means that a major scale can start on any of the twelve notes (C, C#, D, D# etc.). There are also twelve different transpositions of the minor scale. Each transposition (“each scale”) is a different combination of notes (in this case: all of them).
A chromatic scale (one which uses every note, i.e. using all white and black notes of a keyboard) only has one transposition. This means that a chromatic scale can start on any note: each time it is the same combination of notes.
Messiaen was fascinated by scales that only had a few (usually two or three) transpositions. For example: the whole tone scale, which rises by a whole tone each time, has two transpositions. It can start, for example, on a C, so that the notes are C, D, E, F#, G#, A#, C. It can start on a C# so that the notes are C#, D#, F, G, A, B, C#. Transposing this up another semitone, starting on D, would give D, E, F#, G#, A#, C, D which is exactly the same combination of notes as the first one (the starting note is not important). The whole-tone scale was used by many composers including Glinka, Liszt and, especially, Debussy. Messiaen called it the first mode of transposition.
Messiaen’s second mode, also called the “octatonic scale”, rises by alternating semitone, tone, semitone, tone etc. Messiaen used this scale a great deal, not just in his tunes but in the chords that he used (i.e. melodically and harmonically.
The third mode rises by a pattern of tone, semitone, semitone. It has four transpositions.
The other four modes each have a total of six transpositions.
Messiaen liked these modes because there is no note which sounds like the starting note. All the notes sound equal. He described them as having "the charm of impossibilities."