I had this problem with a site built with SMF. The page load time is about 18 secs. But the page load is normal (less than a second) when logged in as admin user.
If enabled and depends on the theme, the forum will show somewhere on the page the time to load the current page.
In this particular scenerio, you need to change an admin option. In Layout and Options, look for ‘Disable hostname lookups?‘. Check it.
I don’t understand why it works, but it works!