![]() It could be interesting to limit the crawl to one parameter to avoid crawling tens of thousands of URLs. This option will allow you to limit the crawl to URLs containing a certain number of parameters.Įxample: This URL contains two parameters age= and format=. Some sites have parameters of type ?x= in their URLs. In this case, it is the maximum folder depth to which you want the crawler to have access. If it is too high, we will have to take action to reduce it. We rarely modify this parameter because it is interesting to look at the maximum depth of a site in an SEO audit. Level 1 is the page located one click away from the homepage and so on. This is the maximum depth you want the crawler to access. That’s why in general we don’t touch this parameter. In general, when we analyze a site it does not necessarily make sense to limit it to several URLs. This corresponds to the number of URLs that Screaming Frog can crawl. This can be particularly useful when you want to analyze sites with many pages. In this third tab, it is possible to define limits for the crawler. That is to select the information related to Structured data: JSON-LD, Microdata, RDFa, Validation. You will be able to get precious information on the implementation of microdata on your site. If you want to do an in-depth SEO audit, one thing can be interesting. Normally you should not change this part too much because in most cases the information extracted from your site that is selected as default is the most important. If you uncheck Word Count, you will no longer have the number of words on each of your crawled pages. In this part, you can choose which data you want Screaming Frog to extract from your site and bring up in its interface. Here you can tell Screaming Frog where your Sitemap is so that it can analyze it. By selecting Crawl All Subdomains before launching our Crawl, the application brings out all the URLs of this subdomain.ĭo you have a Sitemap on your site? Then it might be interesting to analyze the gaps between what you have in your Sitemap and the URLs that Screaming Frog finds. By checking Crawl All Subdomains, Screaming Frog will be able to bring them up in its crawler.īelow you can see a site that has a subdomain of type: blog. You have pages in a subdomain and want to crawl them all the same way to analyze them. ![]() In this part, you can give instructions to the tool to crawl some specific pages of your site. Again, here it is up to you to choose what you want to analyze.Ĭrawl with external links and without external links: You can for example say that you do not want to crawl canonicals or external links. In this part, you can also decide what you want to crawl in terms of links. We recommend you select the crawl and storage of images, CSS, javascript, and SWF files for an in-depth analysis. On most of your crawls, you will have a very basic configuration in this window. In the screenshot above, yellow will not be crawled, and the link of the image with the red cross will be crawled. Good to know: It may still happen that you have some images that come up if they are in the form of a href and do not have an img src It is up to you to choose what you need to analyze. Your crawl will be faster but you will not be able to analyze the weight of images, the attributes of images … Screaming frog will exclude all the elements IMG: (IMG src=”image.jpg”). You can, for example, decide not to crawl the images of the site. Usually, you should have a window that looks like this: Here, you can decide which resources you want to crawl Spider is the first option that is available when you enter the configuration menu. To do this go to the top menu and select Configuration This can be very useful when you crawl sites with a lot of pages or when you want to analyze a subdomain. There are many ways to configure your crawler so that it only returns information that you are interested in. When it is launched and you want to stop it, you just have to click on Pause and then Clear to reset your crawl Configure your first crawl on Screaming frog: You just have to enter your URL and select start to launch your first crawl. When you get to the tool, the first thing you will see is a bar where you can enter the URL of the site you want to analyze: Find out how to configure it step by step Launch your first crawl on Screaming Frog: A good configuration of your tool will be very important to analyze a site in-depth and get all the data you need.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |