Results 1 to 9 of 9

Thread: robots.txt

Threaded View

Previous Post Previous Post   Next Post Next Post
  1. #1

    Default robots.txt

    I recently added a robots.txt file to my site sofar the only thing in it is

    Code:
    User-agent: *
    Disallow: *.swf
    Should I be stopping bots crawling anything else? Will this be fine?

    Anything and everything that may help with ranking and protecting my arcade please.



    For anyone not using a robots.txt file it is simply to prevent the crawling and indexing of certain parts of your site, this:

    Code:
    User-agent: *
    Disallow: *.swf
    Will hopefully stop them crawling my swf files since google bots got clever and can now read them lol

    to create one simple make a file called robots.txt and upload it to the root of your site, Example: yourarcade.com/robots.txt

    Thanks to the wonderful people at TA for teaching me this

    Thanks all
    Last edited by Badger; 04-27-2012 at 11:41 PM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •