INTRODUCTION

There are many tools inside of an SEO engineer’s toolbox. And as you already know, SEO for Google is not easy, but when you have the right tools great job, it can make all the difference. And that’s where  crawler website comes in.

In my many years consulting for many small, medium and even Fortune 500 companies, there seems to be one sounding question I have to solve before I get a contract signed. And that is what needs to be solved on my website as a client.

If you haven’t learned already, you will soon, but as soon as you start talking about building links and all the technical details that go into ranking a website, the clients’ eyes will immediately gloss over.

Technical details are what make you different. Some of you will focus on creating backlinks, others on content and article writing, and yet others will look to create a social influence within their industry. There’s truly no wrong way to go about in the advertising space. What you analyze technically as a marketer, this is what allows you to keep the contract for years or so you can wow your boss before you ask for a raise. 

That’s where the right tool comes in. More or less, that’s where screaming frog SEO spider comes in.

No matter what size the website, it needs to be scanned regularly for broken links, redirects and server errors. 

And as your marketing becomes more refined, you’ll naturally focus on different keywords then you are currently optimized for. on-page optimization and having the ability to quickly see all of the titles, meta tags, meta descriptions, keywords, URLs of every page and so much more – all at once! That makes your job go 10x faster. And in today’s world we need every spare minute.

Free vs Paid SEO Softwares

Believe it or not, there is a big difference when it comes to purchasing software to help your work move along faster – and using free tools.

Oftentimes, the free tools will come with limitations or even worse – won’t complete a job fully. Meaning it may scan your website for sitemaps, but it won’t tell you where there are broken backlinks. In the end, this is a huge issue when you aim to do your job well (and keep it).

Furthermore, paid softwares are updated quite often as the developers are being PAID for their time. The better their software performs, the more customers they are going to get. When a software developer works for scraps (like in the freeware business) you end up with a shoddy product that may or may not work.

That’s why I’m willing to invest money into purchasing software, rather than my time to get the job done right, the first time. Remember, money is something you can get more of – time IS NOT.

Accessibility in an SEO Audit

When it comes to Screaming Frog SEO Spider, the biggest advantage you’ll receive is the ability to access all of your data quickly. Want to see what needs to be optimized? Need to know what URLs have redirection errors or 500 codes? Need to see what page has a Meta description, and which ones don’t? All of this is available in the dashboard, in one easy-to-read place.

Keywords in an SEO Audit

Choosing the right keywords is absolutely necessary BEFORE you go onto optimizing any websites. The best way I’ve found to do this, is with Google Keyword Planner and KWFinder. The first, Keyword Planner, being used to get the best keyword variations from Google’s own data centers. This lets you know all the potential words, phrases and variations you can rank for.

The second tool, KWFinder, is only there to give you a more accurate idea of keyword search volume. See, currently Google only allows you to see search volume “estimates”. These estimates can range widely from 10 to 100 searches, 100 to 1,000 searches, to 1,000 to 10,000 searches. So you don’t know if a keyword is receiving 1,000 searches a month or 9,000 searches a month (and that can make a big difference). So utilization of KWFinder will help you bridge the gap when finding keyword volumes.

Screaming Frog Free Trial Limitations

Ok, so let’s imagine you have your keywords chosen already. Great! So now you’re ready to move onto TRYING Screaming Frog SEO Spider, right?

Well there’s some good news! First, Screaming Frog SEO Spider offers a FREE TRIAL for their software. So you don’t have to make a buying decision… just yet! Instead you get to use virtually 90% of the software without any limitations. In fact, the only real limitations are the fact that you can only crawl 500 links per domain. This is enough crawling ability for your average website to be optimized inside a FREE TRIAL VERSION of Screaming Frog SEO Spider.

Cool, huh?

What You’ll Have in Your SEO Audit Reports

Inside you’re audit reports you’re going to have A LOT of different options, and it can seem overwhelming at first to look at… and I agree with you! But, we’re just going to go over what each part means, and you’ll see it’s really easier than it looks.

Let’s break down each tab you’ll see inside Screaming Frog SEO Spider, shall we?

Internal (Internal Links Tab)

This tab in the software deals with all the links that are found within your website’s address (i.e. http://YOURSITE.com) – and this can relate to pages, posts, images, JavaScript, txt files, etc.

Now my personal tip is, focus just on your pages. You can’t really optimize an image once it’s been uploaded (except giving it an ALT Tag), so work on the parts of the website you can improve.

Often the Titles will be under-optimized, with less than 70 characters. The Meta Descriptions will often be found to have less than 160 characters, allowing more keywords to be placed in the page description. All of these things are the most important part of on-page optimization. So focus on this, FIRST!

External (External Links Tab)

The external tab refers to all the properties on your website that ARE NOT yours. This often includes Google Fonts, Payment Processor API codes, Facebook Pixels, External Advertising Scripts & Trackers, etc.

These things you really cannot control; however, it’s always important to make sure there aren’t any malicious codes inside your website OR that the codes are still in use. If you have pages that load these external scripts AND DOES NOT USE IT, then it’s slowing your page load speeds down! And that’s yet another important factor in, not only ranking but in making conversions.

A little known fact that Amazon shares with its advertisers is this. For every extra 0.1 second it takes to load your page, you begin to lose 3% of your visitors after the very first second. That means that a page that takes 4-5 seconds to load will lose more than 90% of its visitors, due to slow load times.

Protocol

This tab has everything to do with the canonicalization of your web address.

And now you’re asking, what is canonicalization? Well it’s the idea of whether your website shows as (http://yoursite.com) OR (http://www.yoursite.com). It also has to do with whether your website has an SSL Certificate as well making it either (http://yoursite.com) OR (https://yoursite.com).

Just a quick tip. You want to have an SSL Certificate (HTTPS) as it improves site security and global rankings + you want to have your site as (https://yoursite.com) as this will allow you to fit more characters in your website URL when found on Google’s search engine.

Why? Well, that extra (www.) adds 4 characters and can be the difference of having your website show up as:

https://yoursite.com/blog/really-cool-article-you-must-read

OR

https://yoursite.com/blog/really-cool-article-you…

In your eyes, which looks better? With the periods OR with the full title for the visitor to read?

Response Codes

Ok, this is where Screaming Frog SEO Spider begins to focus more acutely on each portion of the optimization you’ll be performing. Each tab hereafter will refer to SOMETHING found within the first 3 tabs.

In this case, the response codes has to do with whether the page/script is 200 (live), 301 (permanent redirect), 302 (temporary redirect), 307 (technical redirect), 404 (missing) or 500 (dead/no response).

If you have 302/307 redirects & 404 missing pages, change them to 301’s where possible.

If you have a 500 response error, make sure your hosting isn’t the issue.

URI

This tab deals with the coding of each page on your website, is it a script, ASCII, image, etc. This tab is relevant for finding & organizing the parts of your website into these categories quickly.

Page Titles/Meta Descriptions/Meta Keywords/H1 & H2 Headers

Believe it or not, this is just a repeat of the very first tab we looked at together. Only in this section, it allows you to see the Titles OR Description OR Keywords all separately. This makes optimizing 100+ page website much easier, as you won’t get lost in the 25+ tabs of data (like when you export the report), these tabs break it down into just one.

Image

View all the images within your website and see if there are any size optimization you can make with this tab.

Hreflang

The hreflang attribute (also referred to as rel=”alternate” hreflang=”x” ) tells Google which language you are using on a specific page, so the search engine can display it to the correct countries.

AJAX

This tab is for HTML encoded website. If you have a CMS (Content Management System) then you won’t use this part of Screaming Frog SEO Spider. This shows you the ugly URLs and pretty URLs that your server fixes for web delivery to visitors.

Link Metrics

Inside this tab, you can analyze the links coming in & going out of the website. This allows you see any toxic links which may be affecting your rankings negatively.

Exporting Your Reports

I’ve found the best way to show a client what their website needs, is to give them the report and go over it for 5 minutes with them (or a copy/paste email does just as well). This allows you to export the entire SEO Audit with Screaming Frog Spider SEO and let them see (all on one page) what is going on inside their website.

Once you let them know you intend to: optimize titles, tags, meta, H1 & H2 headers, images, sitemaps, analyze link metrics, etc. The client will begin to realize the scope of your work and enjoy the help and expertise you bring (and it only took ONE SOFTWARE to do).

Sitemaps

If your website doesn’t have a sitemap, it’s highly invaluable to have Screaming Frog SEO Spider do this part for you. You’ll then upload it to the root folder of your website (usually under “public_html”) and name the file (sitemap.xml) as this is the place where EVERY search engine spider goes to check for your sitemap.

When you’ve uploaded your sitemap, you’ll notice within 3-5 days that all your site’s links will be indexed inside Google.

In Conclusion

Does it seem like I’m promoting Screaming Frog SEO Spider? If so – I AM NOT!

Screaming Frog SEO Spider doesn’t have an affiliate program, they don’t have partnerships, joint ventures – anything! I am literally telling you all about Screaming Frog SEO Spider simply because how great of a tool they are and what they can do for an online marketer.

So whether you have a boss OR are your own boss… getting your website, and conversely your sales machine, in tip-top shape is your FIRST PRIORITY as a marketer. Everything else is secondary.

When you have marketing on the forefront of your mind, then you need Screaming Frog SEO Spider to help you get this laundry list finished as quick as possible.

PS: I hope this shows… when I recommend something – GET IT! I always show you the tools that’ll get you ahead of the pack.