Tool to diagnose crawl issues.

1
Diagnosing Crawl Issues Using SEO
2
How Search Engines Work ?
3
Crawling

• Crawling means the visit of Google Bot (spiders)
  to track your page.
• In SEO words, crawling means following your links
  and crawling your website.
• When bots come to your website it follows other
  linked page and crawls your site.
• Crawling is done by search engine crawlers.
• A search engine crawler is an automated script
  that browses the WWW to provide up to date data
  to the search engine

www.yourseopick.com
4
Indexing

• After crawling, the search engine processes each
  of the page it crawls.
• It compiles massive index of all the words it
  sees and their location on each page.
• This content is then stored, with the information
  and organized by the search engines.
• Storing and sorting this informations requires
  space and both Yahoo and Google have over a
  million server each.

www.yourseopick.com
5
Crawl Issues
6
Crawl Log Example
www.yourseopick.com
7
Crawl Issue Example
www.yourseopick.com
www.yourseopick.com
8
Diagnosis - Webmaster Tool

• Youll see three possible types of crawl errors.
• HTTP errors These arent common, and they
  usually occur in small numbers. This happens when
  Google gets a server-side response that doesnt
  yield an ok or found status.
• Not Found In succeeding crawls, Googlebot will
  revisit these pages and compare them with what it
  has in its cache. When it doesnt find the pages
  that it wants to revisit, it lists the URLs as
  404 Not Found.
• Restricted by Robots.txt Your robots.txt
  file is a part of
• your site where you tell search engine
  spiders which
• pages in your site they can crawl and which
  ones are off-
• limits.

www.yourseopick.com
www.yourseopick.com
9

• Not Found URLs from old pages that have been
  replaced by new ones 301 redirect them to the
  URL of the new page that replaced them.
• HTTP errors Ask your developer to diagnose what
  type of HTTP error it is, since theres a lot of
  these.
• Private/Secure Pages restricted by robots.txt
  No action
• Public pages restricted by robots.txt Edit
  robots.txt file and remove the URL from the list.
  -

www.yourseopick.com
10
www.yourseopick.com
11
Crawl Efficiency Checklist

• Ensure page load time aren't slow as to reduce
  the number of pages crawled.
• Ensure server is responsive.
• Return a 303 for unchanged content.
• Use a compression.
• Return a 404 for not found content.
• Ensure each page has at least one link.
• Avoid infinite redirects and redirect loops.
• Ensure most important pages are linked from
  homepage.
• Reasonable crawl delay setting (if used at all).
• Reasonable use of Google Webmaster tool crawl
  setting.

www.yourseopick.com
12
Who we Are ?

• YourSEOPick is an initiative by Smartinfosys.net,
  a brand owned by WebConnect Private
  Limited. We are committed to provide the best SEO
  services to all businesses to make the most out
  of their websites.
• Our goal is to help our clients maximize their
  return on investment by generating high
  quality traffic from the major search engines.
• Unlike other SEO companies that are just focused
  on hiking ranks on Google, YourSEOPick is focused
  on leads and sales.
• All type of SEO services can be found under one
  roof YourSeoPick.

www.yourseopick.com
13
Our Services

• Local SEO Optimize your website for local
  searches. Get found locally and maximize your
  earning through people in your targeted
  geographical Areas.
• Mobile SEO Optimize your website for mobile
  searches and get maximum exposure from more than
  60 of total Internet traffic that comes from
  mobile device visitors.
• Comprehensive SEO audit A website is audited
  for various on-page and off page factors that
  affect visibility of a website on various search
  engines.

www.yourseopick.com
14
Reach Us
E-mail support_at_yourseopick.com URL
www.yourseopick.com
Address 401/604/801, Meridian Tower,
Nr. Apple Hospital, Udhna
Darwaja,Surat. Gujarat (INDIA) -
395002
Contact 91-9426157202
www.yourseopick.com