Results 1 to 9 of 9

Thread: robots.txt

Hybrid View

Previous Post Previous Post   Next Post Next Post
  1. #1

    Default robots.txt

    I recently added a robots.txt file to my site sofar the only thing in it is

    Code:
    User-agent: *
    Disallow: *.swf
    Should I be stopping bots crawling anything else? Will this be fine?

    Anything and everything that may help with ranking and protecting my arcade please.



    For anyone not using a robots.txt file it is simply to prevent the crawling and indexing of certain parts of your site, this:

    Code:
    User-agent: *
    Disallow: *.swf
    Will hopefully stop them crawling my swf files since google bots got clever and can now read them lol

    to create one simple make a file called robots.txt and upload it to the root of your site, Example: yourarcade.com/robots.txt

    Thanks to the wonderful people at TA for teaching me this

    Thanks all
    Last edited by Badger; 04-27-2012 at 11:41 PM.

  2. #2
    License owner
    Join Date
    Jul 2010
    Location
    Sunny South Carolina
    Posts
    1,004

    Default

    User-agent: *
    Disallow: /cgi-bin/
    Disallow: /admin/
    Disallow: /resources/
    Sitemap: your sites sitemap url

    Not sure why you would want Google to stop crawling your swf files but each to his own I guess.

    Put this in a google search and look at my flash files...

    site:bored-wife-arcade.com bored wife arcade

    Why on earth would you not want Google to index your flash files? If you click on one of the urls for one of my games where are you going? That's right my site. That is where I want people to go. Blocking google from crawling your games so no one can find your games does not sound like a good plan to me....Just saying.
    Last edited by mystical; 04-28-2012 at 04:49 AM.
    My Redneck YouTube Channel

    Need a logo or banner? PM Me!

    Just an old dude with his hobbies......

  3. #3

    Default

    I was getting some strange keyword rankings in webmaster tools, someone suggested it could be swf content.

    Thanks for the others

  4. #4

    Default

    mystical is right don't block your games
    User-agent: *
    Disallow: /uploads/
    Disallow: /admin/
    Disallow: /includes/
    Sitemap: sitemap.xml

  5. #5

    Default

    Quote Originally Posted by GalaxyGamer85 View Post
    mystical is right don't block your games
    Well I took it out, I'm not sure it was flash causing my strange keywords anyway, Hmmy lol.

    Ty

    Quote Originally Posted by Giorgi View Post
    what is default robots.txt for av arcade pro
    It doesn’t have one.

  6. #6

    Default

    what is default robots.txt for av arcade pro

  7. #7

    Default

    pls say your robots.txt

  8. #8
    License owner
    Join Date
    Feb 2012
    Location
    England
    Posts
    784

    Default

    You need to write your own. it is simple.. http://www.robotstxt.org/ more information
    No longer creating mods nor supporting any of my work.

  9. #9
    Junior Member
    Join Date
    Aug 2017
    Posts
    1

    Default

    Thanks for the help.

    It is important to configure the robots.txt file correctly so that your blog is crawled correctly and everything works fine. In my personal blog http://uid.me/webcamporno I tried this file to be configured as optimally as possible.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •