Yes, reducing the voltage to the lights by removing a battery or two will reduce the temperature.
This will also reduce the brightness of the bulbs, so you may have to play to find a happy medium.
The twisted wire method you have done should not increase heat to the bulbs, if anything the connection where the wires are twisted together would get warm/hot if not connected firmly enough and would reduce the power to the bulbs.
If this continues to be a problem with heat, you may need to settle for a lower voltage and bulbs that are too dim, or you may need to invest some more time into swapping out the bulbs for LEDs...
I'm sorry to hear the heat is an issue already. It does look good though