The question "What is voltage?" seems more appropriate for a physics forum than it is for electronics. So I dug out my old physics textbook (Sears & Zemansky, 1947) and found the relevant answer. But first one must understand the concepts of electric potential and electromotive force.
The electrical potential energy of a test charge at a point in any electric field is the work required to bring the test charge from infinity to the point. It must be assumed that the original charge distribution which creates the electric field is not altered by the introduction of the test charge. (This will be true if the test charge is sufficiently small.) The 'potential' at a point in an electric field is defined as the ratio of the potential energy of a test charge to the magnitude of the charge, or as the potential energy per unit charge. A potential of one joule per coulomb is called one volt.
The difference between the potential at two points in an electrostatic field is called the potential difference between the points. The concept of potential difference is an extremely important one, both in electrostatics and in electric circuits. Electrical engineers commonly refer to it as "voltage".
In order to have a current in a conductor, an electric field must be maintained in the conductor; furthermore, continuous energy input is necessary to maintain the current, the energy being converted to heat in the conductor. The development of heat in a conductor is an irreversible process in the thermodynamic sense. That is, while energy is conserved in the process, the heat developed can not be reconverted to electrical energy except under the restrictions imposed by the second law of thermodynamics.
There is another type of energy transformation which is reversible in the thermodynamic sense. Devices in which reversible transformations occur are the storage battery, the generator, and the motor. As contrasted with the irreversible conversion of electrical energy into heat, the conversion of electrical energy into electro-chemical or electro-mechanical energy is reversible in the sense that this energy can be completely recovered and converted back to electrical form. The fact that recovery is not 100% in actual devices under the conditions in which they are used does not vitiate the possibility of complete recovery under the proper conditions. There is no theoretical lower bound to friction or electrical resistance in the laws of thermodynamics.
Any device in which a reversible transformation between electrical energy and some other form of energy can take place, is called a 'seat of electromotive force'. Electromotive force (emf) may be defined as work per unit charge, the same as the measure for electrical potential. It should be noted, however, that although emf and potential are expressible in the same unit (volt), they relate to different concepts. Also, the use of the word "force" in connection with this concept is unfortunate, since an emf is not a force but work per unit charge. At one time it was suggested that electromotance would be a better term.
So the simple answer to the question, "What is voltage?", is that voltage is a potential difference or emf that is measured in volts.
