Having speakers rated at higher wattage than the amp is no problem, in fact it is desirable since it means you cannot destroy the speakers.
You would not want to regulate the voltage down. All that does is wastes power and limits the power you can get out of the amp.
The LM386 is a popular amp that can drive 8 Ohm speakers directly. It will get you about 1/2 W at 9V (although that is listed at 10% distortion, whereas the amp I posted has < 1%, so it can probably also put out 1/2 W at 10% distortion).
As Steve suggested, there are a lot of pre-built modules out there, on Ebay for instance, that are cheap and have great specs. If you look for a class D it will get you the most power at a given voltage and the most efficiency, so your batteries last longer.
Have fun.
Bob