It's three problems combined

The second problem is that although you got a stable framerate of 30 fps while it was happening, the timer measured large flucuations.
The third problem is that the camera code does not get along too well with fluctuating framerates.
The first problem, root of the second problem, MAY be that we set the frame swap mode to glFlush on nvidia cards because so far, no problems were reported, but fluctuating framerates are exactly the thing using glFlush instead of glFinish can cause.
You can do something against that: go to "System Setup"/"Display Settings"/"Performance Tweals" and change "Swap Mode" from Flush to Finsh.
These framerate flucuations have other bad consequences: your keypresses are handled with a tiny, fluctuating delay. So we're going to have to detect unusual framerate fluctuations automatically and change the swap mode.
Furthermore, fluctuating framerates should not mess up the camera.
Flush puts more strain on the CPU than Finish (The NVidia driver puts on 100% CPU load, no matter how much the game uses), that's why we're not always using Flush.