It's not that easy to answer but generally larger bits do need to spin more slowly.
There are charts, calculators, and formulas on the web and especially on manufacturers sites that are worth checking out.
There are several factors that come into play and these may vary widely for each router, bit profile, wood species, feed rate, and number of flutes.
Some say start fast and slow down until the vibration stops, some say start slow and speed up till it starts to vibrate then back off.
Either too slow or too fast can burn the wood, burn the bit, dull the bit, or even cause a catastrophic failure of the bit.
I tend to err on the safe side by going a little slow since I'm not a high production shop. I want a good surface finish without burning or chatter.