Page 1 of 2 12 LastLast
Results 1 to 15 of 17

Thread: Robots

  1. #1
    5 Star Lounger
    Join Date
    May 2001
    Location
    Allen, Texas, USA
    Posts
    727
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Robots

    I am trying to get a robot to index my page, but it keeps coming up with a "fobidden" error. See attached image file.

    I have deleted the robots.txt file and even remarked out any and all meta tags related to robots. Shouldn't this open it wide open?

    Web URL is: www.grace-efc.org.

    Thanks
    --KZ
    Attached Images Attached Images

  2. #2
    Platinum Lounger
    Join Date
    Feb 2002
    Location
    A Magic Forest in Deepest, Darkest Kent
    Posts
    5,681
    Thanks
    0
    Thanked 1 Time in 1 Post

    Re: Robots

    Hi there

    There is a lot of confusion around about the usage of the robots.txt file I have had this link stored away in an old file which is a great tutorial which may help you

    Robots tutorial
    Jerry

  3. #3
    5 Star Lounger
    Join Date
    May 2001
    Location
    Allen, Texas, USA
    Posts
    727
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Re: Robots

    Thanks. I've read through that and even created a brand new robots.txt file and still is shows as forbidden. Hmmm

    --KZ

  4. #4
    Platinum Lounger
    Join Date
    Feb 2002
    Location
    A Magic Forest in Deepest, Darkest Kent
    Posts
    5,681
    Thanks
    0
    Thanked 1 Time in 1 Post

    Re: Robots

    Right let's go back....where are you getting the error message ...in your browser? <img src=/S/shrug.gif border=0 alt=shrug width=39 height=15>
    Jerry

  5. #5
    Platinum Lounger
    Join Date
    Feb 2002
    Location
    A Magic Forest in Deepest, Darkest Kent
    Posts
    5,681
    Thanks
    0
    Thanked 1 Time in 1 Post

    Re: Robots

    Ken

    Your robots.txt states:

    User-agent: *
    Disallow:

    How about if you just delete this text and make it an empty file?
    Jerry

  6. #6
    5 Star Lounger
    Join Date
    May 2001
    Location
    Allen, Texas, USA
    Posts
    727
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Re: Robots

    No, I'm trying to use www.websidestory.com. When I run their indexing feature it shows forbidden in the main page and in the robots.txt file and stops.

    --KZ

  7. #7
    5 Star Lounger
    Join Date
    May 2001
    Location
    Allen, Texas, USA
    Posts
    727
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Re: Robots

    I'll try that. It didn't work when I didn't even have a robots.txt file though.

  8. #8
    5 Star Lounger
    Join Date
    May 2001
    Location
    Allen, Texas, USA
    Posts
    727
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Re: Robots

    Same thing with it blank too. [img]/forums/images/smilies/sad.gif[/img]

  9. #9
    Platinum Lounger
    Join Date
    Feb 2002
    Location
    A Magic Forest in Deepest, Darkest Kent
    Posts
    5,681
    Thanks
    0
    Thanked 1 Time in 1 Post

    Re: Robots

    Right time for a new analyser. Place this code in yout index page <meta name="robots" content="index,follow"> and then try analysing it using this link

    Meta tag Analyser Tool

    Comes up with some interesting info about your site <img src=/S/grin.gif border=0 alt=grin width=15 height=15>
    Jerry

  10. #10
    5 Star Lounger
    Join Date
    May 2001
    Location
    Allen, Texas, USA
    Posts
    727
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Re: Robots

    Interesting information! Still doesn't allow that index to run though, not sure why.

    --KZ

  11. #11
    Platinum Lounger
    Join Date
    Feb 2002
    Location
    A Magic Forest in Deepest, Darkest Kent
    Posts
    5,681
    Thanks
    0
    Thanked 1 Time in 1 Post

    Re: Robots

    I am running out of ideas so will hand over the baton...but I have a feeling that you ISP server administrator does not allow this
    Jerry

  12. #12
    5 Star Lounger
    Join Date
    May 2001
    Location
    Allen, Texas, USA
    Posts
    727
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Re: Robots

    That's probably it. I'll check on that. Thanks!

  13. #13
    5 Star Lounger
    Join Date
    May 2001
    Location
    Allen, Texas, USA
    Posts
    727
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Re: Robots

    The IP of that company was blocked in my config. Works now. Thanks!

  14. #14
    Platinum Lounger
    Join Date
    Feb 2002
    Location
    A Magic Forest in Deepest, Darkest Kent
    Posts
    5,681
    Thanks
    0
    Thanked 1 Time in 1 Post

    Re: Robots

    <img src=/S/bananas.gif border=0 alt=bananas width=33 height=35> I thought I was going mad....glad it is fixed
    Jerry

  15. #15
    5 Star Lounger
    Join Date
    May 2001
    Location
    Allen, Texas, USA
    Posts
    727
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Re: Robots

    ME TOO. <img src=/S/dizzy.gif border=0 alt=dizzy width=15 height=15>

    Check it out:

    http://www.grace-efc.org/

    Type in Youth or something in the search.

    --KZ

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •