I was working in Google Webmaster Tools, looking at a (sadly) large list of 404 errors. I found that there was an issue with the URL to a category, and thus the subsequent pages in that category ended up with bad URLs as well. So I corrected the category level URL, tested a few URLs and all seemed good. But Google was telling me that I had 320 URLs with issues… Before I checked the “Marked as Fixed” button on all of the URLs, I wanted to verify that all 320 issues were fixed.
That’s a lot of clicking and page checking…
There has to be a better way, right? So I did a bit of Googleing for a bulk link checking tool. I came up with nothing that would do the trick. Sad. And odd.
Never the less, I did find a solution to my problem – just build it. It’s a pretty simple task really, right? I just need a bulk list of URLs, reach out to the server and check the header response, and that’s it, right? Well the answer is yes – the hiccup in that though is I learned that PHP errors when you attempt multiple
get_headers requests (despite them having multiple functions to do this). So rather than using a PHP loop, I had my PHP loop call AJAX requests that will each do the PHP call individually.
And to not be greedy and keep the solution to my self, here it is! Feel free to use as you need. It’s not pretty, and I’m aware I could do some polishing up around the edges, but it does work. So good enough, right?
- URLs need to be complete
– starting with http://
- Separate URLs with a semi-colon (
;) and no spaces. This results in one really long string of URLs.
- Like I said, I used this for checking my Google Webmaster Tools URL Errors report. So I downloaded the data to Excel, then used Excel and/or a text editor to create the long string of URLs separated by the semi-colon.
A few enhancements have been made since this post was written:
- You can separate URLs using multiple delimiter methods and select which it is via a radio button. Options include a New Line (carriage return), comma, semi-colon, or tab.
- The tool now adds
http://to the beginning of a URL if it is not present
- Another counter was added to show how many need to be checked. If you are processing a large number of URLs this will allow you to know when the tool has completed.