Long-tail ecommerce search statistics for exact matches and substitutes By Checkout Reply, Checkout Reply. Sources checked 2026-11-11. A query-length study of 22,458 Amazon ESCI test searches shows where exact products and alternatives appear in candidate lists. ## Quick answer In Amazon ESCI’s US test slice, 72.42% of candidate pairs on one- or two-term searches were exact, versus 58.09% on searches with five or more terms. Substitutes appeared at least once on 72.15% of the longer queries, a query-level measure. Amazon Science, Shopping Queries Dataset (ESCI) (https://github.com/amazon-science/esci-data/tree/main/shopping_queries_dataset) ## Key findings and sources ### Long-tail candidate lists had fewer exact pairs In Checkout Reply’s Amazon ESCI US test reanalysis, 72.42% of one- or two-term candidate pairs were exact versus 58.09% for five or more terms. Amazon Science, Shopping Queries Dataset (ESCI) (https://github.com/amazon-science/esci-data/tree/main/shopping_queries_dataset) 22,458 US query IDs in the Amazon ESCI large-version test split Group by whitespace term count and divide exact pairs by all pairs in each length band. ### Over seven in ten five-plus-term queries had a substitute In the Amazon ESCI US test slice, 3,810 of 5,281 queries with at least five terms had a judged substitute (72.15%), compared with 3,122 of 5,878 shorter one- or two-term queries (53.11%). Amazon Science, Shopping Queries Dataset (ESCI) (https://github.com/amazon-science/esci-data/tree/main/shopping_queries_dataset) 22,458 US query IDs in the Amazon ESCI large-version test split Collapse judged pairs to query ID and count any substitute per query. ### Exact pair share changed more than ten points In the Amazon ESCI US benchmark, exact share was 14.32 percentage points higher among one- or two-term candidate pairs than among pairs on queries with at least five terms. Amazon Science, Shopping Queries Dataset (ESCI) (https://github.com/amazon-science/esci-data/tree/main/shopping_queries_dataset) 22,458 US query IDs in the Amazon ESCI large-version test split Subtract the unrounded five-plus-term exact share from the one- or two-term exact share. ### Almost every query in the longest band still had an exact candidate In the Amazon ESCI US test slice, only one of 5,281 queries with at least five terms lacked a judged exact candidate. This curated benchmark does not establish a live no-results rate. Amazon Science, Shopping Queries Dataset (ESCI) (https://github.com/amazon-science/esci-data/tree/main/shopping_queries_dataset) 22,458 US query IDs in the Amazon ESCI large-version test split Collapse candidate pairs to query ID and count queries with zero E labels. ## Compare the results Figure description: Exact candidate shares decline and substitute shares increase across three query-length groups. US Amazon ESCI test pairs. Exact and substitute shares are computed within each query-length group. ## Longer searches carry more requirements A one-word search may leave the category or pack size up to the shopper. A longer request can combine an item, color, use case and compatibility detail. The extra words can be useful retrieval signals while also making each candidate answer more conditions. Of 109,760 judged pairs on one- or two-term searches, 79,484 were exact. Of 105,714 pairs on searches with five or more terms, 61,413 were exact. Those are shares of the benchmark candidate pairs. They do not say that a shopper had fewer good choices, because the candidate set is curated and a query can still contain an exact match. ## Count the queries where a fallback exists Pair share answers how many of the judged entries belong to a label. Query presence answers whether a shopper request has at least one judged alternative. An individual query with many substitute candidates can raise pair share without expanding the set of requests where a fallback is available. In the longest group, 3,810 of 5,281 queries had at least one substitute. The shortest group had 3,122 of 5,878. Only one query in the longest group lacked any exact candidate, so this benchmark cannot establish that long requests commonly return no useful item. ## Test what the substitute keeps and changes Build a small set of realistic multi-constraint queries and log which constraints a proposed result preserves. For each test, retain the original request, item type, requested variant and the explicit difference the fallback would introduce. Judge the recommendation at the query level and record whether a verified alternative actually exists. For customer replies, preserve a hard constraint such as compatibility or pack size and make the changed preference explicit. A report that looks only at total matches can hide failure on rare but expensive constraints. Compare your own result set and final cart variant before applying this benchmark to a production ranking decision. ## Study data Table: Amazon Science, Shopping Queries Dataset (ESCI) Measure | Value | Context 1–2 terms unique query IDs | 5,878 queries | Original grouped count 1–2 terms candidate pairs | 109,760 query-product pairs | Original grouped count 1–2 terms exact candidate pairs | 79,484 query-product pairs | Original grouped count 1–2 terms substitute candidate pairs | 17,330 query-product pairs | Original grouped count 1–2 terms irrelevant candidate pairs | 9,916 query-product pairs | Original grouped count 1–2 terms queries without an exact candidate | 0 queries | Original grouped count 1–2 terms queries with a substitute | 3,122 queries | Original grouped count 1–2 terms queries with an irrelevant candidate | 2,007 queries | Original grouped count 1–2 terms exact pair share | 72.42% | Original recomputed share or rate 1–2 terms substitute pair share | 15.79% | Original recomputed share or rate 1–2 terms query with substitute share | 53.11% | Original recomputed share or rate 3–4 terms unique query IDs | 11,299 queries | Original grouped count 3–4 terms candidate pairs | 210,288 query-product pairs | Original grouped count 3–4 terms exact candidate pairs | 136,451 query-product pairs | Original grouped count 3–4 terms substitute candidate pairs | 48,869 query-product pairs | Original grouped count 3–4 terms irrelevant candidate pairs | 19,424 query-product pairs | Original grouped count 3–4 terms queries without an exact candidate | 0 queries | Original grouped count 3–4 terms queries with a substitute | 7,326 queries | Original grouped count 3–4 terms queries with an irrelevant candidate | 3,986 queries | Original grouped count 3–4 terms exact pair share | 64.89% | Original recomputed share or rate 3–4 terms substitute pair share | 23.24% | Original recomputed share or rate 3–4 terms query with substitute share | 64.84% | Original recomputed share or rate 5+ terms unique query IDs | 5,281 queries | Original grouped count 5+ terms candidate pairs | 105,714 query-product pairs | Original grouped count 5+ terms exact candidate pairs | 61,413 query-product pairs | Original grouped count 5+ terms substitute candidate pairs | 28,855 query-product pairs | Original grouped count 5+ terms irrelevant candidate pairs | 13,098 query-product pairs | Original grouped count 5+ terms queries without an exact candidate | 1 queries | Original grouped count 5+ terms queries with a substitute | 3,810 queries | Original grouped count 5+ terms queries with an irrelevant candidate | 2,284 queries | Original grouped count 5+ terms exact pair share | 58.09% | Original recomputed share or rate 5+ terms substitute pair share | 27.3% | Original recomputed share or rate 5+ terms query with substitute share | 72.15% | Original recomputed share or rate ## What changes when you compare the figures Table: Calculations from the published values above. Comparison | Result | Calculation 1–2 terms exact pair share | 72.42% | = 79484 / 109760 * 100 1–2 terms substitute pair share | 15.79% | = 17330 / 109760 * 100 1–2 terms query with substitute share | 53.11% | = 3122 / 5878 * 100 3–4 terms exact pair share | 64.89% | = 136451 / 210288 * 100 3–4 terms substitute pair share | 23.24% | = 48869 / 210288 * 100 3–4 terms query with substitute share | 64.84% | = 7326 / 11299 * 100 5+ terms exact pair share | 58.09% | = 61413 / 105714 * 100 5+ terms substitute pair share | 27.3% | = 28855 / 105714 * 100 5+ terms query with substitute share | 72.15% | = 3810 / 5281 * 100 Long minus short query substitute coverage | 19.03 percentage points | = (3810 / 5281 - 3122 / 5878) * 100 Short minus long exact pair share | 14.32 percentage points | = (79484 / 109760 - 61413 / 105714) * 100 Long minus short queries with irrelevant candidate | 9.11 percentage points | = (2284 / 5281 - 2007 / 5878) * 100 ## Methodology and sources Filter public Amazon ESCI examples to US large-version test rows. Count whitespace-separated terms in the query after trimming outer whitespace. Group one or two, three or four, and at least five terms. First count candidate-pair labels. Independently collapse by query ID to count presence of at least one substitute or irrelevant candidate. Compare unrounded rates and round displayed numbers to two decimals. The benchmark provides judged candidates without observed impressions or rank. [Download the source data (CSV)](decision-table.csv) [Download the calculations (CSV)](calculations.csv) [Download the citation notes (CSV)](citation-notes.csv) [Download the evidence behind the new findings (CSV)](finding-evidence.csv) [Download the reproducible analysis script](replay_esci_link_assets.py) [Download the source-labeled chart (SVG)](evidence-chart.svg) ## Related reading Ecommerce model number search statistics across exact and substitute products (https://checkoutreply.com/guides/ecommerce-model-number-search-statistics/) AI shopping recommendation accuracy statistics (https://checkoutreply.com/guides/ai-shopping-recommendation-accuracy-statistics/) Ecommerce product review statistics and what shoppers use to decide (https://checkoutreply.com/guides/ecommerce-product-review-statistics/) AI shopping price and availability error statistics (https://checkoutreply.com/guides/ai-shopping-price-availability-statistics/) What to text when a shopper leaves checkout (https://checkoutreply.com/guides/first-imessage-abandoned-checkout/) How to answer “Will it get here by Friday?” before they buy (https://checkoutreply.com/guides/delivery-question-before-checkout/) Browse checkout recovery research (https://checkoutreply.com/research/) ## Sources and definitions Amazon Science, Shopping Queries Dataset (ESCI) (https://github.com/amazon-science/esci-data/tree/main/shopping_queries_dataset) The source data includes the population, period, denominator and limitations for every figure. Checkout Reply · checkoutreply.com