OK, you are assuming that the differential amplifier has a gain of 1.
The differential amplifier will operate correctly as long as the input voltages are within the range of the supply voltages (the output might get clipped to the supply voltage, but that's expected).
If you want negative input voltages then you must ensure that your negative supply voltage is more negative than the most negative input.
You might, for example have a supply that is +/- 18V. That will allow (for example) one input to be -6V and the other -1V and get an output of -7V.