Unless the modem is marked to say that it can accept that voltage, then
yes, there is the possibility of damage. It might not happen all at
once, it might destroy the modem over time. I certainly wouldn't do it
unless I was planning to buy a new modem soon anyway.
In all likelihood, what's happening internally is that there is a voltage
regulator which is running noticeably hotter than before, and is coping
OK for now. Probably the whole thing will age faster than before, or it
might just go pop one day.
OTOH I've had some cheap far eastern network gear which was sometiumes
delivered with 9V and sometimes with 12V supplies, and the 12V versions
ran far more stably for a long time.
If it works, and it's a cheap router, use it. If it goes pop, buy another
one.