Google Bot Activity

Discussion in 'General Webmaster Chat' started by MoneyMan, Jun 24, 2008.

  1. MoneyMan Not Active

    MoneyMan
    Joined:
    May 15, 2008
    Messages:
    87
    Likes Received:
    1
    In google webmaster tools on of my sites allows me to speed up the crawl rate of the bots on my site, which is suppose to help indexing etc, it says that server usage can make the site slow if i do this ! is it true and is it worth it ?
  2. Similar Threads: Google Activity
    Forum Title Date
    General Webmaster Chat Google owned by Mafia Aug 19, 2009
    General Webmaster Chat Google API Dec 16, 2008
    General Webmaster Chat Google Webmaster Tools Oct 18, 2008
    General Webmaster Chat Google Chrome For Mac and Linux | Available For Download Before Release!!! Sep 16, 2008
    General Webmaster Chat My Blog Has 74 Backlinks in Google!!! Sep 15, 2008
    General Webmaster Chat Google in the Black Hole?? Sep 10, 2008
    General Webmaster Chat 2 Bugs Found in Google Chrome! Sep 10, 2008
    General Webmaster Chat Lol new google brower on form :) Sep 8, 2008
    General Webmaster Chat Happy Birthday Google Sep 7, 2008
    General Webmaster Chat The New Google Browser!! --> Google Chrome Sep 3, 2008

  3. Fuzzer Qdoos Contributor

    Fuzzer
    Joined:
    Aug 10, 2007
    Messages:
    1,223
    Likes Received:
    5
    I have that setting on my forums, it does increase bot activity and indexed pages and to be honest i haven't seen any difference in performance or bandwidth
  4. Iearn-Money Qdoos Contributor

    Iearn-Money
    Joined:
    Jun 10, 2008
    Messages:
    1,147
    Likes Received:
    8
    Location:
    Iearn-Money.Com
    You cannot get any useful things from bots.. But I heard that the bots has a relation to google sitemap !
    If you create a sitemap on a website you will notice that the bots activity is decreasing ! From the other hand if you didn't create a google sitemap on your website the bots activity is very active ! Anyone has an idea ?
  5. Fuzzer Qdoos Contributor

    Fuzzer
    Joined:
    Aug 10, 2007
    Messages:
    1,223
    Likes Received:
    5
    Iearn i appreciate you posting but please make sure your correct in what you say and its not a personal opinion, as this statement is totally incorrect.
  6. hitman Qdoos Contributor

    hitman
    Joined:
    May 27, 2008
    Messages:
    1,235
    Likes Received:
    6
    i tried a similar thing, but this gave me more bots rather than making crawl any quicker, so just chewed up more bandwidth
  7. Iearn-Money Qdoos Contributor

    Iearn-Money
    Joined:
    Jun 10, 2008
    Messages:
    1,147
    Likes Received:
    8
    Location:
    Iearn-Money.Com
    Well we can say they come from google to index our website.. But how often will Googlebot access your web pages!
    I was googling and found that How often Googlebot crawls your site depends on several things, including how often your site is updated. If your site is regularly updated, but Google haven't crawled it recently, it may be because Googlebot cannot easily discover it. In this case, Google recommend telling them about the pages on your site by creating and submitting a Sitemap!

    But returning to my previous post.
    This is a conclusion from the people who already installed s sitemap on their website, and reported Googlebots activity..

    Maybe it's usefull.. But everything has consequences !
  8. Fuzzer Qdoos Contributor

    Fuzzer
    Joined:
    Aug 10, 2007
    Messages:
    1,223
    Likes Received:
    5
    Sitemaps are excellent and do allow google to crawl site alot better, especially sites with Dynamic URL's, However google is very active on on most sites unless the site does do inactive for a while, most of my forums have got google and yahoo bots on it 24 hours a day.
  9. Iearn-Money Qdoos Contributor

    Iearn-Money
    Joined:
    Jun 10, 2008
    Messages:
    1,147
    Likes Received:
    8
    Location:
    Iearn-Money.Com
    Well take a look at this:
    http://www.webmasterworld.com/forum30/33342.htm

    That guy tried google sitemap.. And wasn't useful for him
  10. masterweb Qdoos Contributor

    masterweb
    Joined:
    Jun 20, 2008
    Messages:
    552
    Likes Received:
    4
    Location:
    right now?
    I'm not a good SEO expert but to be honest if someone runs a forum (Vbulletin) and needs a sitemap, a helping hand for robots and bots and a ping to yahoo and Google sending your updates then must use Vbseo for that (i'm not promoting it for money due i haven't ant contact with this brand, but when i installed it into my forum the organic traffic from Google as well as the frequency of bots and robots searching my pages duplicates hour after hour).

    Just an off-topic opinion.-
  11. hitman Qdoos Contributor

    hitman
    Joined:
    May 27, 2008
    Messages:
    1,235
    Likes Received:
    6
    crawl really isnt important so long as it doesnt take more than 24 hours for new content on your site to be crawled, i find mine is instantly crawled and i dont do anything to optimize the crawl time
  12. Fuzzer Qdoos Contributor

    Fuzzer
    Joined:
    Aug 10, 2007
    Messages:
    1,223
    Likes Received:
    5
    Yeah i think the crawl time only makes a diffrence on sites like forums where there is lots of new posts every couple of hours
  13. hitman Qdoos Contributor

    hitman
    Joined:
    May 27, 2008
    Messages:
    1,235
    Likes Received:
    6
    or maybe if your a very big blog, in which case you will gave a lot of people on it all the time so they can quickly steal content before it is crawled

Share This Page