Keyword research is the foundation of producing content that drives traffic and, as a byproduct, also drives revenue.

In this extensive guide, we’ll be sharing our complete playbook on keyword research for SEO.

Note: This is a longer guide. I suggest working through it section by section.

Operationally, How Do We Do Keyword Research?

The most common questions we get about keyword research relate to the operational side of running through the process.

Things like: “how often should we be doing keyword research?”.

We’ve talked at great lengths about how SEO is ultimately not very complex. It can be distilled down to a handful of initiatives you derive the most value from. Yet, on the operational level, what is complex is being able to execute everything to a high standard, doing so consistently, and scaling that process.

Keyword research is a relatively straightforward process – at least to begin with. It becomes more nuanced when you consider aligning it with content strategy – the business value associated with covering certain topics and prioritizing.

We do keyword research in cycles where each round has a specific aim or focus. This could be something such as when a software company is launching a new feature or when we want to start focusing on more content surrounding a specific feature. What we do in these rounds varies and is something that we prioritize based on what is going on.

Alright, so – without further ado – let’s dive in. First, we’ll take a look at some of the tools that we actively use and can recommend:

The Leading Keyword Research Tools

Ahrefs

Although it goes without saying that Ahrefs has lost a lot of popularity after their recent pricing stunt, they’re still our favorite SEO tool at the moment. Everything from the data they provide to the various ways you can access it across their backlinks reports, keyword research tool through to their content explorer tool.

Their keyword database is arguably not as “complete” as the one Semrush provides, though it’s accepted that no SEO data provider can truly provide 100% accurate and complete datasets anyway (which is a constraint you have to learn to work with). More and more search behavior is moving to the longtail, and these tools aren’t that great at that, meaning for a lot of keywords that do get volume, they just look like keywords that get no volume whatsoever. This can be a bit misleading, but working with the best data available is still better than trying to fly blind.

Sidenote: Anyone who has, out of principle, stopped using Ahrefs because of their pricing increase is, in our view, not using it to its full advantage. If you use Ahrefs properly, it is still very good value even at the increased price. And I am a big believer that we cannot expect service, performance, and functionality to improve over time if they do not account for that with a price increase. Their software has come a long way, and the price increase was justified. If you continue hitting the usage limits, and find them restrictive, then I suggest you make sure you are taking full advantage of the data you are already getting from the usage. If you keep arbitrarily using it to get more and more data that you ultimately get no results with, this is simply not the right approach.

Semrush

Many find Semrush’s keyword database to be more accurate than that of Ahrefs. In our testing, this is true in certain verticals – but not always true in other cases. As a general rule of thumb, you’ll be fine with either of these, and I would use whichever you feel most comfortable working with and getting useful data out of. Not only do they largely do the same things, but you can always alternate between them to get the best of both worlds.

Some Free Alternatives

I’ll be very blunt. We don’t really recommend these per se, but for those of you working on smaller sites and not really doing this for a “business”, here are some free tools. It is worth noting that we cannot vouch for the quality of the data that they provide:

Google Keyword Planner: This is Google’s own keyword research tool. It’s completely free to use, but you will need to have a Google Ads account in which you’ve created at least one ad in order to be able to use it.
SurferSEO’s Keyword Surfer Chrome Extension:
Keywords Everywhere

Our Keyword Research Playbook

0 – The Different Starting Points

Strategy is all about prioritizing. Even when you work with million-dollar budgets, you still have a limited capacity to decide what you actually do first.

This is particularly true for keyword research.

Each keyword research cycle will start with a different general objective that dictates which starting point is the most suitable.

Broadly speaking, there are three starting points:

Start by picking 1 or more direct competitors (depending on their site’s current standing), running it through Ahrefs, and performing a clustering analysis of the keywords that they rank for.

Pathway #1 is perfect for software companies operating in an industry where they have a clear product-market fit. They’re not looking to generate new demand, they’re looking to capture existing demand. They have a product that is the solution to a set of problems that other products try to solve for – but you’ve objectively done a better job of solving.

So, simply put, using a competitor that may have been in the space for 10+ years already and the treasure-trove of data that you’ll be able to get from their site is a good starting point. In fact, not doing so is going to set you back in the grand scheme of things. But competitor research is also just one of many good starting points.

Note: Even if this is the basis of your keyword research cycle, there should always be some level of nuance that begins to play a role when digging into each individual topic. Certainly, this is always the case for us. This means that depending on what it is that your software product does differently or better (in your view) – you are going to prioritize certain searches (keywords), alter the approach you use to target them (how you write), and more.

Find organic competitors in search (i.e., sites that you compete with in terms of content but not necessarily in terms of the product/service that you sell).

Pathway #2 is somewhat similar to the first, but it involves finding an organic competitor. Organic competitors are what we call sites that compete in search but not directly on product/service sold. This means that within the informational content they produce, the keywords they target may result in featuring our product as a different part of the solution. But this pathway is also more suitable for strategies such as top-of-the-funnel searches designed to drive awareness, i.e., think of the number of companies that can write about project management.

Consider:

a project management software company
a project management services firm
just a general resource on project management
or a tool that is used in addition to other, more traditional project management solutions.

The list really does go on.

Start from a set of seed keywords when we want to go broader.

The last (and currently least common) way we kick off keyword research cycles is by starting with seed keywords. This is generally reserved for one of two things:

When the way a company competes isn’t 100% clear and edges toward demand generation. In those cases, a telling way to start is by performing a cycle of keyword research with the basis of seed keywords that are related to what the product helps with. This can also help identify sites that are direct or organic competitors.
When we’re launching a new feature or update to something that falls into a specific category. Say, for example, you develop software for web hosting providers and are announcing a feature that allows them to run containerized deployments on Docker. You run a cycle dedicated to topics around Docker that are worth prioritizing in the 1-2 months leading up to your release, as well as continuing to work on these afterwards, since they will have the potential to highlight this feature.

Now that we’ve had a look at examples of some of the starting points that we use let’s dive into some of the specific ways we get data out of “tools”, to turn it into something that’s actually useful and actionable.

1 – Find direct competitors with Ahrefs Content Explorer

Starting point: A seed keyword or specific keyword that you wish to target.

In this case, we turn to the Ahrefs Content Explorer.

Think of this as a search engine that allows you to filter by metrics that are important to your prioritization and research process, such as:

Website Authority
Traffic
Language

As well as various other metrics.

The process here is relatively simple: you plug in your target keyword into Ahrefs’ Content Explorer and start digging.

The objective when using this tool in Ahrefs is varied. It can be to find a specific content idea, to find the ranking post for a specific keyword that you wish to target (so you can confirm what the primary target keyword is), etc.

You’ll generally find that you get the most useful results when setting the search type to “In Title”.

After which, play around with the filters to adjust to your scenario. For example, if you’re working with a brand new site it can help to filter between 1 to 20. We often set this to between 5 to 10 lower than our actual domain authority until we know we can start ranking for more competitive searches.

2 – Extract Organic Keywords from Competitors

Starting point: You already know that your starting point is a specific competitor that is very aligned in terms of product.

In this scenario, the process of getting the raw data will be relatively straightforward (and most of the work will come from the prioritization step before you get things assigned to the writer(s) on your team).

So, simply head to Ahrefs > Site Explorer, as shown below:

Drop in the URL identified / being used as the starting point:

And then head to the Organic Keywords tab:

Sidenote: Depending on whether your intention is to run this data through a keyword clustering process (a good publicly available solution for this would be Keyword Insights). But if you choose not to do this, you may benefit from skipping/using the Top pages tab report in Ahrefs directly instead.

However, for us to get the most complete dataset, we use the organic keywords report, run a full export, and cluster this data.

If you wish to reduce the amount of raw data that you have to process, you can apply some general filters in Ahrefs, for example:

Limiting the position by up to 8 helps ensure that you’re limiting it to content that this site has covered, which they rank well for (in some cases increasing and reducing based on data set size)
Filtering by keywords with a volume from 50 ensures that we start by tackling keywords that have proven demand. Tackling lower and zero search volume keywords will come from other sources anyway, but when it comes to generating the core keyword clusters, we want to limit considerations to the keywords that have considerable potential to drive traffic.
As above, the traffic from the 5 filters similarly allows us to filter out keywords that generate little to no traffic initially.
Keyword filters are to be used to exclude branded search, i.e., in the above screenshot, Kinsta has been excluded. In some cases, companies like Atlassian have many products, so exclusions will need to be added for all of their products (i.e. bitbucket, atlassian, jira, confluence, statuspage, etc.). Not that these are pages that will never be tackled, but initially, they conflate the dataset with topics that are almost always likely to be of no interest to us covering.
And lastly, the URL filter (unused above) can be a great additional way to filter out non-suitable subsites that the organic search competitor has, e.g., support.atlassian.com (which is ranking for help/product-related searches).

3 – Clustering Your Keywords Together

The next step in our process is to cluster this data to make it more manageable to work with.

A cluster is essentially an indication that all these keywords can be targeted on the same page.

Think of phrases such as “best WordPress form plugins” and “form builders for WordPress”. These clusters are generated by determining that most results that rank for a keyword also rank for the other keywords in that cluster.

On the other hand, the best way to think of a hub is a cluster of your clusters, i.e., “are there any clusters that are similar to each other?”. The logic behind how we determine this is by using the intent/meaning of the actual words (the fundamental meaning behind searches). The hub tab is not to be relied on definitively. It is a general guide to finding similar content pieces so that you can easily group them together as we plan content production (as well as easily find internal linking opportunities).

Sidenote: You can use the pivot table editor to sort through ideas that show a dominant intent of longer-form articles simply by opening the editor, scrolling down, and filtering by specific options. That way, you can view a list of keywords that are clearly indicating the need to be approached with longer-form articles. (selecting the status numbers 7-10 will be ones where the top 10 results are dominated by “articles”).

The name given to the hub in our datasheet is always simply inherited from the name of the cluster that has the most volume in that hub. This specific name can be ignored as it’s simply an identifier.

4 – Content Selection & Production

Now that you have data from your first keyword research cycle, it’s time to work through this list to make sure that you’re prioritizing content based on a whole range of things that you wish to prioritize based on.

We generally prioritize based on:

The business relevance of the topic – is there any value in ranking for this keyword that can drive revenue for us?
Resources to produce content (to a high standard) – if you’re intending to cover advanced topics to a very high level, such as a topic on “penetration testing”, the question then becomes – do you have an in-house resource to rely on to answer specific questions to make sure that your approach to the article is…
1. technically sound
2. better than the competition
3. aligns with the business values surrounding the topic?
Resources to achieve content velocity – choosing content ideas, each of which will take a very good amount of research at the beginning, is a great way to burn people out and lose momentum. You want to make sure that you’re choosing content ideas that are suitable for the people you have onboard to be able to see things through to completion.

Otherwise, you’ll quickly build up “task debt”, which is a lot of ongoing work that just needs finalization sitting in the final stages.
Current business priorities – for example, is ranking for this topic something that is going to remain useful for the business, or is the particular product/service that it relates to not a focus?

On a keyword level, when it comes to narrowing down:

Keyword variants that return the same search results should be identified in the clustering process, but a final check before dedicating resources to content production is recommended.
If you have to choose between multiple keyword variations, choose the one that is most natural and sounds better (not just the one with the highest volume in every single case).
If you have multiple keywords with the same intent (i.e., where the same posts generally rank), then choose one based on a mix of…
1. the one that is most natural
2. the one with the highest volume and
3. the one that makes the most sense for your business to cover.
  
  Unless it is possible to rank for both intents (i.e., there are sites that have 2 posts ranking in a SERP, such as a listicle of “penetration testing tools” or a guide on “penetration testing”).

Writing To Satisfy Search Intent

Producing content that has the potential to actually satisfy search intent is fundamentally a simple process.

That being said, it’s hard to do well and get results from doing so when the bar to get results is to do it better than other brands.

We’ve got a dedicated guide on the content strategy we use to produce empirically better content than our competitors could even if they tried (even with 10x the budget we have). This process involves constantly setting the bar higher than your competition. Even if it makes your life harder until you actually earn a competitive advantage over the players that have been producing content (and building up site authority) in your vertical for the last few years, if not decades.

After Action Report – Keyword Research That Works

If you want to earn search traffic, you need to produce content on topics that people are searching for. And if you want that search traffic to be of any value to your business, you need to run target keyword research cycles.

We hope this guide helps you run through your very first keyword research cycle the ScaleMath way. And if you have absolutely any questions – or want us to run this process for you – let us know, we’re here to help.

Keyword Research for SEO – Our Complete Playbook