Query optimizers / personal lessonsLesson 1 of 5
SQLite · Lesson 1

Reading SQLite query plans

See which tables SQLite visits, which indexes narrow a search, and where it needs extra work to sort the result.

You need to be able to read SQL with WHERE and JOIN. SQLite runs inside an application or command-line tool and stores a database in a local file. There is no database server to configure.

The series uses ordinary SQLite 3 syntax, with maintenance advice for version 3.46 and later. Plan wording and choices can change between versions. Unless identified as recorded output, plans and counts are teaching examples.

Run a small experiment

Install the SQLite command-line shell if you do not already have it. Download experiment.sql, then run these shell commands from the download directory. Choose a new filename so you do not modify an existing database.

sqlite3 --version
sqlite3 basics.db < experiment.sql
sqlite3 basics.db

The script creates 10,000 orders, shows plans before and after indexes, and leaves the test data available. In the SQLite prompt, try:

EXPLAIN QUERY PLAN
SELECT total FROM orders WHERE customer_id = 42;

EXPLAIN QUERY PLAN describes the strategy without running the SELECT to completion. Running the SELECT without that prefix retrieves the rows. Plain EXPLAIN lists the instructions SQLite executes in its virtual machine. This series uses EXPLAIN QUERY PLAN instead.

Type .quit to leave the shell. Deleting your disposable database file afterward removes the experiment. Commands beginning with a dot are shell commands, not SQL for your application.

Read the access path

SEARCH orders USING INDEX orders_customer_idx (customer_id=?)
SEARCH orders
SQLite uses a lookup or range to visit a subset of entries. That subset may still be large.
USING INDEX
The named index supplies the lookup. SQLite may still fetch table rows for values missing from that index.
(customer_id=?)
The customer_id equality narrows the search. The question mark stands for the lookup value. It can appear even when the SQL contains a literal value such as 42.
SCAN orders
A scan walks through a table or index rather than looking up a bounded subset. A LIMIT can still stop it early.
COVERING INDEX
The index supplies all the values this access needs, so separate table-row lookups are unnecessary.

Separate the required answer from the route to it

Start with the question the SQL asks. A query that selects total for customer 42 must return the qualifying totals, including repeated equal totals if several orders have the same amount. Unless it requests an order, it does not promise which qualifying row appears first. The query planner chooses a way to produce that answer. Creating an ordinary nonunique index changes the available routes, not the meaning of the SELECT.

This distinction gives you a correctness test before a performance test. Suppose a rewrite returns only the distinct totals. It might return fewer rows and finish sooner, but it no longer answers the same question. Likewise, adding LIMIT 20 is not an optimization of a query whose caller needs every order. It changes the requirement. First establish which rows, duplicates, columns, and ordering the application actually needs.

Think of the table and an index as different organizations of information. In the ordinary rowid tables used here, the table stores complete records addressed by rowid. A secondary index organizes its keys and carries row identifiers. An index on customer_id places the entries for one customer together. SQLite can locate that group without first examining every other customer's complete record.

If the query also needs total and total is absent from the index, finding the group is only part of the work. SQLite uses the row identifiers to retrieve the missing values from the table. An index that contains every required value can avoid those separate retrievals. Coverage therefore depends on a particular query and access, not on an index name or a permanent property called "fast."

Consider two queries with the same customer filter. One selects only id; the other selects id, total, and a large notes field. The first may be covered by the customer index under the schema assumptions here. The second may need table records. Their qualifying row identifiers are the same, but the values they require differ. Do not transfer a timing result between them merely because their WHERE clauses match.

Before running the small experiment, predict what an added customer index changes. It should make a bounded customer lookup available. It does not by itself put that customer's orders in date order, make every requested value available inside the index, or reduce the number of orders that belong to the customer. Afterward, compare that prediction with the plan. If SQLite chooses another route, record that observation rather than treating the index's existence as proof that it was used.

Keep the plan and the measurements separate

EXPLAIN QUERY PLAN

Which table or index?Which lookup bounds?Which loop order?Separate sorting needed?

Separate measurement

How many rows actually match?How long does the real SELECT take?What changes across repeated runs?Optional counters if the build supports them
A SEARCH line describes an access path, not a measured row count. Count matching rows and time the statement separately.

EXPLAIN QUERY PLAN shows the chosen strategy, not measured performance. It does not print actual or estimated row counts, per-node elapsed times, or loop counts. Measure those separately with counts, timings, or optional SQLite counters.

A lookup can still inspect rejected candidates

SELECT total FROM orders
WHERE customer_id = 42 AND created_at >= '2025-01-01';

-- With an index on customer_id alone:
-- SEARCH orders USING INDEX orders_customer_idx (customer_id=?)

The displayed search bound mentions customer_id, not the date. SQLite must still check the date condition. The animation uses invented counters to show which candidates pass the date filter. SQLite does not print these counters in the plan.

Which candidates pass the date filter?

This smaller example has six orders for customer 42. The date filter keeps orders from 2025. Only rows that pass the filter enter the result.

0 rows checked, 0 returned, 0 removed by the date filter.Customer 426 candidate rowsDate filteryear ≥ 2025Returned0 rowsRemoved0 rows
Row 1 · 2024
Waiting
Row 2 · 2025
Waiting
Row 3 · 2024
Waiting
Row 4 · 2025
Waiting
Row 5 · 2025
Waiting
Row 6 · 2024
Waiting

Each step checks one candidate against the date filter.

Teaching counters (not SQLite plan output)
Candidates checked: 0
Returned: 0
Rejected: 0
Step 0 / 6

These counters show how six candidates become three results. Ordinary SQLite EXPLAIN QUERY PLAN does not print returned or rejected counts. SQLite checks the date during this access. The plan need not show a separate filter node.

Account for candidates, rejected rows, and returned rows

A result row count is not a work count. For a separate invented example, suppose customer 42 has twelve orders, of which four meet a date condition. An index on customer_id identifies twelve candidate entries. SQLite still has to decide which of those twelve satisfy the date requirement. Eight fail that test and four can contribute to the answer. The twelve candidates, eight rejections, and four outputs describe different quantities.

If the index does not contain the date, SQLite may need each candidate's table record to check it. That can include records that never reach the final result. Adding date after customer in a composite index can let the search itself exclude the older group. The SQL's result should remain the same four orders even though the route considers fewer candidates. The index-design lesson develops the required key order.

The question mark in a search detail describes a lookup constraint, not an unknown measured row count. In a join, its value can come from the current outer row. In a simple query, it can represent a literal or bound parameter. Reading customer_id=? therefore tells you which relationship narrows the access. It does not tell you whether that value is rare, common, or absent from this particular database.

Use the filtering animation to predict the next distinction before advancing it. Identify which condition selected a candidate and which condition still needs checking. Then compare a candidate that passes with one that fails. If a counter rises for both, it measures attempted work rather than successful output. This is why a query that returns very little can still inspect many records.

Now imagine that the date condition rejects all twelve candidates. The answer is empty, but the route still had to establish that emptiness. Conversely, an equality lookup for a customer with no index entries can establish that absence without visiting twelve candidate records. Both queries return zero rows. Their result sizes alone cannot distinguish their access work.

A separate COUNT query helps establish how many rows satisfy a condition, but it does not reconstruct this route. SQLite might answer the count using a smaller covering index. The original query might fetch total from table records, sort them, or repeat the access inside a join. Use a count as evidence about the data and use the plan as evidence about the chosen access. Neither substitutes for timing the actual statement when elapsed performance is the question.

Why reading the table can be reasonable

If few rows match, an index can avoid reading unrelated table rows. If almost all rows match and the index is not covering, following many index entries and then fetching their table rows may cost more than reading the table directly.

Animated scan comparison

Compare a table scan with an index scan

Both methods search the same 24-row table. Amber dots mark matching rows. Check marks show rows already returned.

Changing the number of matches resets both scans.

Sequential scanRead pages in order

Sequential scan: 0 table-page visits, 0 distinct pages, 0 matching rows returned.Read the next table pageAll 24 rows must be checked6 PAGES · 4 ROWS PER PAGEP1P2•P3P4•P5P6•

Each step follows an index entry or reads a table page.

Pages reached
0 / 6
Matches returned
0 / 3
More counters
Table-page visits (including repeats)
0
Rows checked
0

Index scanFollow entries to rows

Index scan: 0 table-page visits, 0 distinct pages, 0 matching rows returned.Find matching index entries3 matching entries in key order6 PAGES · 4 ROWS PER PAGEP1P2•P3P4•P5P6•

Each step follows an index entry or reads a table page.

Pages reached
0 / 6
Matches returned
0 / 3
More counters
Table-page visits (including repeats)
0
Rows checked
0
Index entries followed
0
Step 0 / 6
What this animation simplifies

Each step shows one action, such as following an index entry or reading a table page. These actions take different amounts of time in a database. Finishing this animation first does not mean a query is faster.

The table-page visit counter includes repeated visits to the same page. It does not count disk reads. A database may already have the page in memory. This example leaves out index-page reads, transaction visibility checks, and prefetching.

Index entries follow key order, which can differ from the order of rows in the table. Try "One row", then "Every row". When every row matches, both scans reach every table page, but the index scan still performs a lookup for each row. The cost example below assigns a cost to each kind of work.

Compare invented costs for these two approaches
Invented costs, not SQLite estimates

When does each plan cost less?

This separate example has 100,000 rows in 1,000 pages. It uses these invented costs:

  • A sequential scan costs 1 unit per page plus 0.01 per row checked.
  • An index scan costs 20 units to start, then 4 units per matching row fetched. The checkbox lowers the fetch cost to 1 unit.

Change the number of matches to compare the totals.

Sequential scan
2,000
Index scan
4,020

The sequential scan costs less in this example.

Show the arithmetic and the limits of this model

Sequential cost = (1,000 pages × 1 unit) + (100,000 rows × 0.01 units) = 2,000.

Index cost = 20 + (matching rows × fetch cost).

At 4 units per fetch, the two plans have equal cost at 495 matching rows, or 0.495% of the table. At 1 unit per fetch, they have equal cost at 1,980 rows, or 1.98%.

The slider starts at 1,000 matches. At that count, changing the fetch cost from 4 to 1 makes the index scan cheaper than the sequential scan.

These percentages apply only to this example. SQLite uses its own cost model and statistics. The actual cost depends on table and index sizes, cached data, whether the index covers the query, and which rows the query needs. Do not use these thresholds to judge a real plan.

Judge the work rather than the SCAN or SEARCH label. How many rows does the query need? Does the chosen path supply the requested order and values?

Read the loop order

SELECT c.id, o.id
FROM customers c JOIN orders o ON o.customer_id = c.id
WHERE c.id <= 5;

-- One possible plan:
|--SEARCH c USING INTEGER PRIMARY KEY (rowid<?)
`--SEARCH o USING COVERING INDEX orders_customer_idx (customer_id=?)

SQLite executes joins using nested loops. Here, the outer loop finds customers. For each customer, the inner loop finds matching orders. An INTEGER PRIMARY KEY in an ordinary rowid table is an alias for the rowid, which explains the rowid wording.

Five customers with two matching orders each produce ten pairs. These counts describe the example data, not plan output. SQLite can choose a different loop order from the table order in the SQL. Check the plan.

Each customer triggers another lookup

For this example, assume the planner expects two customers. Change how many actually match. Each customer in this example has two orders.

Changing the customer count resets playback.

Assumed input

2 customers
2 lookups
4 joined rows

Model work so far

0 / 2 customers
0 lookups
0 joined rows

0 order lookups completed for 2 customers, returning 0 joined rows.Next customerOuter scanLook up ordersInner index scanJoined result0 rows

Select a customer, then look up their orders. Repeat for each customer.

Teaching counters (not SQLite plan output)
Rows per lookup: —
Lookups completed: 0
Total rows so far: 0
Step 0 / 4

Each lookup in this model returns two rows. The total grows with the number of completed lookups. The dash means no lookup has run yet. SQLite EQP does not print these counters; the display explains row counts, not query speed.

Sorting can prevent an early stop

EXPLAIN QUERY PLAN
SELECT id, created_at FROM orders
ORDER BY created_at DESC LIMIT 20;

-- Without a suitable ordering index:
|--SCAN orders
`--USE TEMP B-TREE FOR ORDER BY

The scan supplies candidate rows and a temporary sorting structure supplies the requested order. Without ordered access, SQLite must consider all qualifying candidates to know which 20 are newest. Returning only 20 rows does not imply reading only 20.

A date index can provide the order directly. In the downloadable experiment, the later plan is SCAN orders USING COVERING INDEX orders_date_idx, with no separate sorting entry. Scanning backward can produce descending order even when the index was created with its default ascending order.

Connect a plan to a prediction

Test the scan comparison without turning it into a timing claim

Begin the side-by-side comparison with a small matching set. Follow the table scan through its pages and the indexed route through its selected entries. Ask which unrelated records the indexed route avoids. Then increase the matching set while keeping the same two access methods. As more records qualify, the opportunity to avoid unrelated records decreases. This is a change in the amount of work that can be saved, not a change in the required answer.

The picture uses small pages so you can follow individual records. It deliberately omits details of a real database's B-tree traversal, caching, operating system, and storage device. A displayed page visit is an event in this teaching model. It is not automatically one physical disk read. Two accesses to a page already in memory can behave differently from two reads that require storage, even when the drawing counts both visits.

Coverage changes the comparison again. If all selected values are available in an index, SQLite need not follow each matching entry back to the table for those values. A broad covering index scan can therefore be useful. Conversely, a narrow-looking SEARCH can still fetch a large group of records when the searched value is common. The plan vocabulary describes the route; it does not rank routes independently of their data and requirements.

Consider the newest-orders example. With no suitable order available, an early candidate is not necessarily one of the newest twenty. A later candidate can displace it. SQLite has to consider the qualifying input before it knows the answer. It may use a bounded sorting strategy for a limit, so do not infer that every candidate must be stored simultaneously. The essential distinction is that the limit does not itself permit ignoring unseen candidates.

With an index that supplies the requested order, SQLite can encounter the newest qualifying records first. When enough records have passed all remaining conditions, it can stop. If another filter rejects many entries along that ordered route, reaching twenty accepted rows can still require many candidates. Ordering and filtering cooperate, but satisfying one does not prove that the other is cheap.

Finish by describing a plan in complete claims rather than labeling it good or bad. For example, say that it traverses a date index in the requested order, obtains the selected values without table lookups, and shows no separate sort. Then name what is missing: the number of candidates actually visited and the elapsed time under the application's conditions. This produces a testable explanation instead of a conclusion based on one capitalized word.

Exercises

Is SCAN always worse than SEARCH?

SCAN orders USING COVERING INDEX orders_date_idx

A query asks for the newest 20 orders. Is this line alone evidence of a bad plan?

Hint

Consider ordering, coverage, and how many rows are needed.

Worked answer

Walk the newest entries and stop at 20

● Input▧ Work✓ Match× Rejected◆ Result

Newest first · numbers below are recency ranks

◆ 1◆ 2◆ 3◆ 4◆ 5◆ 6◆ 7◆ 8◆ 9◆ 10◆ 11◆ 12◆ 13◆ 14◆ 15◆ 16◆ 17◆ 18◆ 19◆ 20· 21· 22· 23· 24

STOP after 20 qualifying entries

No table-row fetches. No separate sorting pass.

This example has a 24-row covering index and no further filter. The scan returns the first 20 entries and leaves four unread. A SCAN line does not report how many entries SQLite visited.

No. A covering index scan can supply a required order without table lookups or a separate sort. A broad SEARCH can still visit many entries. Compare the work and timings for queries that return the same result.

What does this plan leave out?

SEARCH orders USING INDEX orders_customer_idx (customer_id=?)

Can you tell whether the query returned five rows or 50,000? Can you tell how many milliseconds it took?

Hint

Which parts of this line describe how SQLite finds rows? Which parts, if any, report measurements?

Worked answer

The same SEARCH line can describe different result sizes

● Input▧ Work✓ Match× Rejected◆ Result
SEARCH orders … (customer_id=?)
Possible dataset: 5 matches

◷ Time: still unknown

Possible dataset: 20 matches

◷ Time: still unknown

Each dot is one row in these two invented datasets. Both can produce the same customer-bounded SEARCH line. Count the matches and time the queries to compare their result sizes and performance.

It shows an index lookup using customer_id. It gives neither the number of matching rows nor the elapsed time. Count rows matching the condition and time the full query separately. This line also does not tell you how many candidates SQLite rejected.

Why can 20 result rows require many lookups?

A dashboard wants the newest orders from trial customers. Read the query and example plan. Use the supplied row counts to explain how much work the query needs.

SELECT o.id, o.created_at
FROM customers c JOIN orders o ON o.customer_id=c.id
WHERE c.segment='trial'
ORDER BY o.created_at DESC LIMIT 20;

-- Example EXPLAIN QUERY PLAN detail:
|--SCAN c
|--SEARCH o USING INDEX orders_customer_idx (customer_id=?)
`--USE TEMP B-TREE FOR ORDER BY

Separate counts show 400 trial customers, each with exactly 30 orders. These are facts about this example's data, not counters printed by the plan.

  1. Which access is the outer loop? Which lookup repeats?
  2. How many joined rows are candidates for sorting?
  3. Why does returning 20 rows not imply only 20 lookups?
  4. Does the plan prove that sorting spills to disk? Does it report elapsed time?
  5. Choose one next test and state what evidence it would provide.
Hint

Trace customers into order lookups. Multiply the supplied data counts. Without suitable ordered access, SQLite must consider the candidates before knowing which 20 are newest.

Worked answer

Choose 20 results from 12,000 candidates

● Input▧ Work✓ Match× Rejected◆ Result
Candidates considered · 1 dot = 100 pairs
Select the newest 20not the first 20 encountered

Result · 1 diamond = 1 output row

0 candidates so farSelection not final yet

0 / 400 customer lookups

Step 0 / 4
400 lookups × 30 orders = 12,000 candidate pairs. Each animation step represents 100 lookups. Candidate dots represent 100 rows each; result diamonds represent one row each. The funnel shows selection, not memory usage or elapsed time.

The customer scan is outer. Each qualifying customer triggers an order lookup. In this example, 400 × 30 = 12,000 customer-order pairs become candidates for the newest 20.

The displayed access path does not supply global date order across those customers. SQLite must consider all the candidates to choose the newest 20. The LIMIT caps the result size, but does not let this plan stop after finding its first 20 matches.

The temporary B-tree line does not prove a disk spill, and EQP does not report elapsed time. Time the actual SELECT separately. An index on the customer segment could reduce the customer scan. It may leave the larger cost of order lookups and sorting unchanged. Test it on a copy and compare the plan and repeated timings instead of assuming it will help.

You could also test a date index that lets SQLite read the newest orders first. It could stop after 20 trial-customer matches, but might inspect many non-trial orders along the way. Compare the work and timings rather than assuming this is faster. You cannot calculate an estimate error from this plan because it prints no numerical estimate.

Reading checklist

  1. State what the query must return, including ordering and limits.
  2. Identify each SCAN or SEARCH and the constraints shown beside it.
  3. For joins, follow the nesting order of the table accesses.
  4. Look for covering access and temporary sorting structures.
  5. Separate what the plan shows from counts or timings that still need measuring.

Sources and next step

Lesson 2 explains the statistics behind these choices, including what you can inspect when the plan does not print estimates.