Googlebot encountered 1 errors while attempting to access your robots.txt how to fix it ?
We build. You grow.
Get best community software hereStart a social network, a fan-site, an education project with oxwall - free opensource community software
how to allow googlebot | Forum
Tawhid
Jun 6 '14
this is what i have in htaccess and robot txt am i doing something wrong :?
RewriteCond %{REQUEST_URI} !/robots.txt
RewriteCond %{REQUEST_URI} !^/index\.php
RewriteCond %{REQUEST_URI} !/ow_updates/index\.php
RewriteCond %{REQUEST_URI} !/ow_updates/
RewriteCond %{REQUEST_URI} !/ow_cron/run\.php
RewriteCond %{REQUEST_URI} (/|\.php|\.html|\.htm|\.xml|\.feed|robots\.txt|\.raw|/[^.]*)$ [NC]
RewriteRule (.*) index.php
#FilesDisallow: ow_version.xmlDisallow: INSTALL.txtDisallow: LICENSE.txtDisallow: README.txtDisallow: UPDATE.txtDisallow: CHANGES.txt
# URLsDisallow: /admin/
User-agent: Mediapartners-Google*Disallow:
ben
Jun 7 '14
by default oxwall installation allow google bot, dont change it. If google bot cant access your site for 1 or 2 times nothing to worry about, your hosting sometimes doing some maintenance and restrart their server and google bot cant access it because it's down. that's normal
Tawhid
Jun 7 '14
ok i figured it out it need to include www insted of was getting a redirect issue because of that
