The output of the high pass filter is controlled by the change of voltage that appears at the input.
The values of the components determine the time constant of the effect, but in essence the output voltage is proportional to the change in input voltage over some period of time -- similar to the difference between the current (instantaneous) voltage compared to some average voltage over some recent time period. (it's more complex than that, but stick with the simple explanation).
Imagine that the input has remained at 0V for a long time. The input voltage has not changed (the average = the instantaneous voltage), so the output voltage is zero.
Now let's assume the input voltage suddenly jumps to 5V. The instantaneous voltage is 5V, but the average over some period is still 0V. The output is the difference (5V).
As time goes on, with the input voltage no longer rising, the average voltage over time becomes closer and closer to the instantaneous voltage so the output falls (and the component values determine how fast this occurs).
Imagine that the input has now remained at 5V for a long time. The input voltage has not changed (the average = the instantaneous voltage), so the output voltage is zero.
Now the input voltage suddenly jumps to back to 0V. The instantaneous voltage is 0V, but the average over some period is still 5V. The output is the difference (-5V).
As time goes on, with the input voltage no longer falling, the average voltage over time becomes closer and closer to the instantaneous voltage so the output rises (the component values determine how fast this occurs).
Increasing the capacitance or the resistance in this circuit will make the circuit look at what in this simple analogy is a longer period of time.
The nature of the "average", and how the components affect this require some real maths, but in essence the average is an exponential function that places more weight on recent values, but which theoretically is affected by all voltages back to the beginning of the universe!
So in summary, the output goes negative because the output is related to the change of the input voltage, and when that change is from a more positive voltage to a less positive voltage (i.e. in the negative direction) the output goes negative.