Voltage is the change in electric potential (meaning potential energy per unit charge) between two positions. The voltage is always measured between two points, for example between the positive and negative ends of a battery, or between a wire and ground. It is measured in volts.
The voltage can be seen as the pressure on the electrons to move out of the source. It is directly proportional to the pressure exerted on the electrons. In other words, the higher the voltage, the higher the pressure. For example, a battery of 3 volt will exert pressure on the electrons twice as hard as a battery of 1.5 volt.
The voltage can push the electrons into a component, like a resistor, creating a current. Usually, the voltage and the current are related by a formula (see impedance), Voltage is directly proportional to the current i.e. if the voltage increases, the current also increases by the same amount.
Note that there must be both voltage and current to get power (energy). For example, a wire can have a high voltage on it, but unless it's connected, nothing will happen (that's why birds can land on high voltage lines without problems).
There are two types of voltage, DC voltage and AC voltage. The DC voltage (Direct Current voltage) always has the same polarity (positive or negative), such as in a battery. The AC voltage (Alternating Current voltage) alternates between positive and negative. For example, the voltage from the wall socket changes polarity 60 times per second (in America). The DC is typically used for electronics and the AC for motors.
Where V=Voltage, I=Current, R=Total resistance.
The voltmeter measures the voltage between two points and can be set to the DC mode or the AC mode. The voltmeter can measure the DC voltage of a battery for example (typically 1.5V or 9V), or the AC voltage from the power socket on the wall (typically 120V).
For more complex signals, an oscilloscope can be used the measured the DC and/or AC voltage, for example to measure the voltage across a speaker.