Can I recharge AA batteries directly from a solar panel?

Finding an old Victorian lamp post in fantastic condition but long buried in a hedge just over the wall of our garden on an old abandoned right of way seems to have developed into a "public" art project (it does not belong to me, after all)! After I cleaned and painted it, a friend made a Victorian lamp top frame in which I set some stained glass. That looks REALLY good, and even better when I put a remote control LED camping light in it (I perhaps got over-excited and bought three more - seemed great value at £10 each but now realise that was perhaps a bit OTT and they are still in the package.) However the four rechargeable AA's only last six to eight hours before I have to be up the ladder to replace them. But a 12V 5watt solar panel only costs £15 and just nicely fits on the roof of the lamp top - that is on its way.
My question is - can I just wire the solar panel directly to the rechargeable AA's in the LED camping light, or does it have to be more subtle than that and operate via a charger?

 

Yes, after looking around other places for advice I was rather expecting that you would say this - I now I have to show my ignorance of electronic issues.
How would I practically do this without baby-sitting the batteries ...,which would rather defeat the object? I want to just wire the solar panel, charger and light/batteries so that I do not need to use the ladder to get up there and just switch it on remotely from the ground.

 

Yes, I can understand this. But if I was in the habit of switching the thing on at night then wouldn't this help deal with this? Perhaps I could experiment with the panel and a meter on sunny / dull days to get some timings? And if I forgot and they exploded then they are well out of the way!
Re the other comments, thanks for these. I had thought of a commercial solar AA charger but a) they are comparatively expensive b) they are very slow and c) there is still this problem of taking the cells out and putting them back in the LED light - which defeats the whole purpose.
I suppose I could use the panel to trickle charge an old 12V motorcycle battery and put that on the charger. That deals with a) and b) ... but not c)
It is c) which bugs me but it must have been well-sorted somewhere out there. Can I wire the LED light straight from the charger? But that would need a diode wouldn't it?
This all sounds as though I know more about the subject than I do. It is only 'cos it has been on my mind over the weekend and done some exploring around - any help would be gratefully appreciated.