Booting is what happens when a computer starts up. When you boot a computer, your processor looks in system ROM (the BIOS) for instructions and does them. They normally 'wake up' add-in cards and searches for the boot device. The boot device either loads the operating system or gets the operating system from someplace else.
People use the word "boot" to mean "to start a computer" or other device with electronics built in. For example, if a person wants to ask a friend to turn on a satellite phone, they would say "could you boot up the satellite phone?".
Most operating systems call the first data storage device it uses a boot device. This is because the computer is making itself go, as in the idiom. When we start a computer, we can often see the simple instructions the computer uses to start, then more complicated pictures or software.
The phrase "to boot" in this meaning is short for "to bootstrap". This use is part of net jargon along with similar multi-use words like Internet or web. Often the computer is just called a box, so a phrase like "to boot the box" means "to start the computer".