You are correct, the electrolytic capacitor does see reverse voltage for part of the time during each cycle. The maximum reverse voltage across the capacitor will be limited by the 2N3906 Vbe as this transistor begins to turn on. Under 0.7V at room temperature.
During the time the NPN transistor is on, the capacitor will see a normal polarity voltage across it, at least as soon as it has discharged back through zero volts.
The reverse voltage on the capacitor is not the only thing about this circuit that's not so good. Yet it might work well enough, depending on what you want it to do.
The saving grace is that the power supply voltage is so low.
Have you built one up? Does it work?
What color is your LED?
The circuit's operation is very dependent on the characteristics of the transistors, the LED, the capacitor, and even the battery. The characteristics of these devices all vary from unit to unit and over temperature.
If the circuit needs to work reliably, or many of these are to be built, perhaps some redesign is in order.
For sure, use an aluminum, and not a tantalum electrolytic. A film cap might be a better choice, but we may be trying to make a silk purse...
Ted