No you shouldn't keep debug on at all times.
Best practice for enabling is to use the else at line 40 as follows:
else
{
/**
* Make changes in this block if you want to enable DEV mode and DEBUG mode
*/
define('OW_DEBUG_MODE', !false);
define('OW_DEV_MODE', !false);
define('OW_PROFILER_ENABLE', !false);
}
When finished simply remove the ! marks you see set at per false, save changes, now your completely out of debug mode.