Yes. Three solutions.
1. Just remove "Customize this page" and "Finish Customizing" buttons from .html file. This way nobody will see the button, and therefore nobody will be able to get to this page ( unless they know the direct url)
As site admin you will still be able to get to the customize page using direct url:
www.yoursitename.com/forum/customize
Good place to start with is customize_index.html under ow_plugins/forum/views/controllers.
2. You can also customize .php code so that "customize this page" button is not accessible for moderators at all but they can still edit/delete forum topics.
Good place to start with is customize.php under ow_plugins/forum/controllers and you might need to edit customize_index.html as well.
3. Just take away ALL "powers" from moderators to do anything in forums.
This can be done in admin panel>>privacy&permissions>>moderators>>untick "forums".
Note that if you choose this option, moderators won't be able to see "customize this page" button AND they won't be able to edit/delete forum topics.