We build. You grow.
Get best community software hereStart a social network, a fan-site, an education project with oxwall - free opensource community software
its geting reidicules all this spam | Forum
Pete
May 24 '14
Its getting a nightmare all this spam ,every time I come on this forum there seems to be more and more of it ,people well stop using this forum if something is not done about it most bots tend to use gmail,maybe oxwall should think about banning these gmail accounts ,people can use yahoo ,etc instead,i don't think bots can sign up to yahoo mail .
The Forum post is edited by Pete May 24 '14
bobbi
May 24 '14
the spam is out of control here the need to sort it out, i use gmail and i am not a spammer so dont tarnish all gmail with the same brush, i usually find that the spam i get which is not that often is yahoo, hotmail or aol sometimes yes gmail but i find hotmail or yahoo are more so
Pete
May 24 '14
sorry bobbi I did not mean to offend you , it just seems the bots that join my site tend to be using gmail .
Pete
May 24 '14
Step 1:
Open your favorite text editor and create a file called robots.txt.
Step 2:
Place the following code in this file.
Code:
# Deny all robots that we do not specifically want to allow User-agent: * Disallow: / # Allow these robots only User-agent: googlebot Allow: /The code above will block all bots from accessing your website, with the exception of Google (googlebot).**See the end of this post for more search engines / robots that are safe to add to your robots.txt file.
Step 3:
Save the file and upload it to your public_html directory. You can upload it via FTP or through the cPanel file manager.
More Good Bots to allow
The example above only uses Googlebot. There are others that you may want to add to your robots.txt file. Here are a few.
- Googlebot-News - Google News
- Googlebot-Image - Google Images
- Googlebot-Mobile - Google Mobile
- MSNBot - Microsoft MSN
- Teoma - Teoma Search
- bingbot - Bing Search
- Slurp - yahoo! Search
- Scooter - AltaVista Search
- Scrubby - Scrub the Web
You can add them into the robots.txt file in the following format:
Code:
User-agent: BOTNAME Allow: /Where BOTNAME is the name of the bot listed above.So one example of a robots.txt file which bans all robots except yahoo, bing, and google might look like this:
Code:
# Deny all robots that we do not specifically want to allow User-agent: * Disallow: / # Allow these robots only User-agent: slurp Allow: / User-agent: bingbot Allow: / User-agent: googlebot Allow: /
.
Pete
May 24 '14
I think its getting to the point were we can only allow a few bots ,like google ,etc ,I think this is the best way forward ,just to allow bots like google ,etc .
