Keyword research is the work of finding out which words people use to search for what you sell, how often they use them and in what order it makes sense to tackle them. Filling a spreadsheet with search volumes is the easiest part of the job, and it is also where most mistakes are made. Once the keywords are chosen, you still have to decide which changes to make first.
On three websites we measured something that the guides on the first page do not mention: between 54% and 82% of organic clicks come from queries that Google Search Console does not show, not even to the site owner. If your research starts and ends inside a keyword tool, you are planning on a fraction of real demand.
Search volume is a rounded estimate, not a measurement
The number next to a keyword is less precise than we treat it. Google Ads documentation says so in full: "Your search volume statistics are rounded", and historical statistics are shown only for exact match, whatever match type you selected. Every other tool starts from there or from clickstream samples, so it inherits the problem and adds its own estimate on top.
Then there is the size of the tail. Ahrefs counted 2.3 billion keywords with fewer than 10 searches a month, almost 93% of its US database. It is a US figure and should be read as one, but the shape of demand does not change when you cross the Atlantic: most searches are strings nobody has ever recorded. In a smaller market such as the UK, that tail is shorter in absolute terms and therefore harder to read in the tools.
How much demand you cannot see: a test on three sites
Search Console does not show all the queries that bring in traffic. Google filters out those too rare to be anonymous and documents it in the post A deep dive into Search Console performance data filtering and limits (Google Search Central, October 2022). The total in the Performance report stays correct; the sum of the rows does not.
We measured the gap. For three Search Console properties of Italian sites, over 1 May - 31 July 2026, traffic from Italy, we compared total clicks with the sum of clicks from the queries actually listed, splitting them by word count with a regex filter.
| Site | Total clicks (Italy) | Clicks from listed queries | Clicks from hidden queries | Hidden share |
|---|---|---|---|---|
| Outdoor equipment e-commerce | 3,055 | 1,269 | 1,786 | 58.5% |
| B2B site in the dental sector | 71 | 13 | 58 | 81.7% |
| visilay.com | 63 | 29 | 34 | 54.0% |
| Total | 3,189 | 1,311 | 1,878 | 58.9% |
The B2B site has small numbers and should be taken for what it is, a hint. The e-commerce site does not: 3,055 clicks is a solid base, and almost six clicks in ten come from searches that appear in no list at all, neither in your Search Console nor in a keyword tool.
The second part of the test is about query length, and it contradicts something you often read: that the long tail brings the best clicks.
| Query length | Clicks | Impressions | CTR |
|---|---|---|---|
| 1 word | 25 | 18,817 | 0.13% |
| 2 words | 677 | 30,089 | 2.25% |
| 3 words | 377 | 22,531 | 1.67% |
| 4 words or more | 190 | 16,026 | 1.19% |
On this site the highest CTR is on two-word queries, not the longest ones, and the single-word keyword collects 18,817 impressions for 25 clicks. That says two things: the difference between long tail and short tail does not automatically turn into a click advantage, and a good average position on a generic term can be worth almost nothing in traffic.
Step 1. Start from the data you already have, not from the tools
You do not buy your first list of keywords, you extract it. Four sources, in the order it pays to open them.
- The Performance report in Google Search Console, filtered on queries with many impressions and few clicks. These are searches for which Google already considers you relevant and where you are losing.
- Google Ads search terms, which, unlike Search Console, also show the commercial queries someone paid to be found for.
- Recordings of sales calls and support tickets. The words customers use on the phone are the same ones they type, and no tool contains them.
- The site's internal search, if it is switched on and tracked. It is the only place where you can read what people were looking for and did not find.
Semrush put this source at the top of its 2026 guide, calling it first-party data and explicitly mentioning sales calls and onboarding tickets.
Step 2. Expand the list without trusting a single tool
From 10-15 seed terms you get to a few thousand variants. Databases differ a lot from one market to another: Ahrefs and Semrush have deep coverage of English-language queries and make international comparison easy, while Google Ads Keyword Planner remains the only source with real cost-per-click data. We have put the current prices of the main suites, along with the other digital marketing tools a business needs, in a separate table. The comparison between the various keyword research tools matters less than the rule: cross-check at least two, and use a third as referee when they disagree.
Then there are the free sources that return real questions instead of estimates: Google's autocomplete suggestions, the People Also Ask box, the related searches at the bottom of the page, AlsoAsked, Reddit threads and industry forums. The PAA box is particularly useful because it is fed by the same system that decides featured snippets, so it tells you in advance which phrasings Google treats as answers.
Step 3. Read the intent on the UK SERP, not the US one
A keyword does not have an intent in the abstract: it has the format Google gives it in that country. The same query in the US and in the UK often returns two different pages, because the available competitors change and so does searcher behaviour. Anyone who takes an American guide and publishes it for a British audience ends up with the wrong format for the SERP they are up against.
The check takes five minutes: search the keyword from an incognito browser with UK language and location settings, open the first eight results and note what kind of page each one is. If they are eight guides, a service page will not rank, however well it is written. If they are eight product pages, a blog post will not either. Reading the search intent is worth more than any difficulty metric, and you do it by looking at the real SERP, not at the label a tool has given the query.
Step 4. Group by intent, not by word
If "how to do keyword research" and "seo keyword research" return the same eight results out of ten, they are the same question and belong on a single page. Two pages answering the same question take strength from each other, and the second one almost always ends up ranking neither for its own keyword nor for the other one.
Keyword clustering is done by comparing the URLs on the first page: a high overlap means the same cluster. Then for each cluster you choose a main keyword and use the others as subheadings and variants in the text, without forcing them. Repeating them mechanically only produces keyword stuffing, which Google has recognised for twenty years. On how to turn a cluster into a page that holds up, we wrote a separate guide on how to create content for SEO.
Step 5. Rank by value, not by volume
Volume tells you how many times a string is typed. It does not tell you how many of those searches will end in a click on a website, and in 2026 the difference is substantial. The Datos and SparkToro study by Rand Fishkin, based on a clickstream panel between September 2022 and May 2024, found that 58.5% of Google searches in the US and 59.7% in the EU end without any click, and that out of every 1,000 searches fewer than 400 clicks reach sites not owned by Google, in both markets.
On the weight of AI Overviews, the best sample is still from the US. The Pew Research Center followed 900 American adults in March 2025 and found that a click on a traditional result happened in 8% of visits when there was an AI summary, against 15% when there was not. About one search in five, in that sample, produced an AI summary. These are numbers from a single market and should be declared as such, but the direction is the one we also see on our own projects.
The chance of finding an AI Overview above the results also depends on the shape of the query. SEO.com analysed 2.37 million US keywords and found an AI Overview in 25.8% of cases, with a very uneven distribution: 17.8% on one- or two-word queries, 57.3% on queries of eight words or more. In other words, if your priority list is made only of long, conversational questions, half of the forecast traffic may never arrive. The factors that decide citations in AI Overviews differ from those of classic ranking and have to be assessed keyword by keyword.
The priority grid we use when we set up an SEO strategy has four columns: commercial value (does this query bring in enquiries?), click potential (how much of the SERP is still clickable?), reachability (who is on the first page and how strong are they?), volume. In that order. Volume breaks ties; it does not set the ranking.
Low-volume technical keywords are worth more than the big ones
The clearest case among ours is Macropix, a Milan manufacturer of custom LED screens. Of the nine keywords that bring it the most qualified enquiries today, none contains the company name: they are "ledwall", 4,400 searches a month in Italy, and then "schermo pubblicitario" (advertising screen), "monitor pubblicitario" (advertising monitor), "totem led" (LED totem), "ledwall outdoor". People looking for an industrial supplier type the process, the material, the size.
That is why in SEO for manufacturing a query with 90 searches a month can be worth more than one with 9,000: the denominator is small but the share of people who are buying is very high. The same goes for most B2B SEO projects, where the number of potential client companies in a single country runs to hundreds, not millions.
When the search happens inside an assistant
Part of demand no longer goes through a search box. People using ChatGPT, Perplexity, Gemini or Copilot do not type "best crm for small business": they write three lines describing their company and ask for advice. That prompt appears in no keyword tool, and it never will.
Google, inside AI Mode, breaks the request into several parallel searches before answering, a technique known as query fan-out. This means the page that gets cited is not the one optimised for the user's question, but the one that answers one of the sub-questions the system generated well. The practical consequence is concrete: it pays to cover the whole semantic field of a topic rather than hold a single string. It is the work that, in generative engine optimisation, comes before any on-page optimisation.
Meanwhile, the AI visibility report in Bing Webmaster Tools is the only free source that shows the queries used to build Copilot's answers. It does not cover Google, but it is the first data of this kind available without a subscription. We have set out how to build a strategy that covers both channels on the page for our AI SEO services.
The mistakes we keep finding in other people's keyword research
- Volumes taken from a single tool and treated as exact measurements.
- One page for each variant of the same question, with the result that they cancel each other out.
- Keywords chosen on a US SERP and applied to a UK site.
- No column estimating how much of the SERP is still clickable.
- Lists that ignore the queries already bringing in impressions, which is the ground closest to a win.
- The file delivered once and never opened again: in many sectors volumes move from season to season.
Keyword research is not a document, it is a process to be reread every quarter along with the rest of your on-page optimisation. If you want to see how it translates into real projects, our numbers are on the projects page, and the full method on the SEO services page.
One last way of doing keyword research that almost nobody uses: sort your Search Console queries by impressions, descending, and filter for those with zero clicks. It is not a list of missed opportunities, it is the list of promises your title made and your page did not keep. Fixing those costs far less than winning a new keyword, and you have had the data to do it in-house for months.
Keyword research FAQs
It is the work of identifying the words people use in search engines to look for a product, a service or an answer, and deciding in what order to tackle them. It has four stages: collecting the terms, checking the intent on the SERP, grouping by intent and prioritising by commercial value.
They are rounded estimates. Google Ads documentation states explicitly that volume statistics are rounded and that historical data applies only to exact match. Other tools start from that source or from clickstream panels, so they need cross-checking: use at least two databases and treat volume as an order of magnitude, not a measurement.
A quarterly review of the file is enough for most sites, with a monthly check of the Search Console Performance report. Seasonal sectors and those affected by regulatory changes need more frequent reviews.
Not to get started. Google Search Console, Google Ads Keyword Planner, Google Trends, autocomplete suggestions and the People Also Ask box cover the most useful part of the work at no cost. A paid database is needed when you have to estimate competitive difficulty or benchmark against competitors.
One main keyword and all the variants that return the same SERP. If two queries show very different results you need two pages; if they show almost identical ones, two pages will cannibalise each other.
