In early August Google announced that they have released an API to the Google Search Analytics data from Google Search Console (formerly Google Webmaster Tools).
I started building upon this API and was able to import data into a MySQL database. All was going well, and then I ran into my first snag in that my computer died. It took me a few weeks to be able to at least recover the code from the localhost dev site I was building this at.
So fast forward a few weeks and I have the code back and a computer to develop it on. All is going fine, my import to MySQL from Google Search Console (Google Webmaster Tools) is working great for the one website I was testing this with. Then I start to explore adding more websites, and nothing but errors. I confirm that I’m making a successful OAuth2.0 connection to Google, etc. Insert too much time spent in frustration, analyzing code, debugging, etc.
Bottom line was this – though you have granted access to the API via OAuth2.0, you still need to grant access to your user in Google Search Console for each website you are attempting to access.
Grant Access to Google Search Console from a Google Developers Console API User
Let me expand on that just a bit because this was all kind of confusing to me.
In the Google Developers Console, you first need to create a project if you haven’t done so already.
Next, you need to add access to the Google Search Console API. This is done within the Google Developers Console while your project is active.
- Choose APIs & auth -> APIs
- In the search field, search for Google Search Console API
- Choose this API from the list.
- Click the Enable API button.
Then also within the project, configure two sets of credentials – an OAuth2.0 client ID and a Service account.
- OAuth2.0 client ID
- As I understand it, this is the basic authentication level that I believe is required to get your API calls access. Typically, this level of access should prompt an authorization screen that the user would click a button to grant access. In my case, I want the authentication built into a configuration script and thus I need to add a Service account as well.
- Service accounts
- These allow you to authenticate the user in the code base and thus skip the authentication screen when the application needs to grant access. This requires programming in your authentication information.
Now that you have your authentication credentials setup, you need to grant access to this API “user” within Google Search Console (Google Webmaster Tools). To do this, you’ll need to copy the email address displayed for the Service account. Then head over to Google Search Console.
- Log into Google search console
- From the right hand side of the screen for a website, choose Manage property -> Add or remove users
- Choose ADD A NEW USER
- Enter the email address you copied above from Google Developers Console in the User email field.
- Set the Permissions drop-down to FULL
- Click the ADD button
This now grants the Google Developers Console API user access to the Google Search Console website property, and thus it’s data as well.
This should be everything you need to have your program talk to Google Search Console (Google Webmaster Tools) via API authenticated via OAuth2.0
Google Search Console API – PHP
I’m a PHP developer for the back end, so that’s my language of choice. But there are other languages that can access the API – but that’s out of scope of this article.
I’m not going to go into detail on how to program access to the API in this post. But Google does offer a PHP library for accessing their APIs. Feel free to explore an use that as you wish.
Import Google Search Analytics Data from Google Search Console to MySQL
Again, there are many ways to accomplish the task of importing your search analytics data from Google to a locally hosted location, but I’m not going to go into full detail on how to accomplish that.
What I will do however, is share with you a github repository that offers a set of instructions and a tool to copy your data from Google Search Console to your own MySQL database.
I should note – I’m the developer of that github repository and would love input and/or feedback on the project. At the time of writing this it is in it’s infancy, but I intend to develop this into a more full suite of tools in the future.
Help / Troubleshooting
If you install and run the Organic Search Analytics tool to manage your own data and run into issues or something that just isn’t quite right, please look to the official github page for this project. Feel free to reach out for help as well if needed, posting any error messages or strange situations you may find on the github repository by creating an issues. This is where the codebase lives, gets updated, and I as the developer as well as other users can help you find a resolution to any issues you may come across.