According to Google Webmasters this seriously increases the bounce rate for your site as users just cant find what they want.
Is there a way to use mobile version but automatically take users to desktop version when they click a link or find a link from searching.
They can always switch back to mobile version afterward.