Menu:
cost-model:: The Optimizer Cost Model
MySQL provides optimizer control through system variables that affect how query plans are evaluated, switchable optimizations, optimizer and index hints, and the optimizer cost model.
File: manual.info.tmp, Node: controlling-query-plan-evaluation, Next: switchable-optimizations, Prev: controlling-optimizer, Up: controlling-optimizer
The task of the query optimizer is to find an optimal plan for executing an SQL query. Because the difference in performance between 'good' and 'bad' plans can be orders of magnitude (that is, seconds versus hours or even days), most query optimizers, including that of MySQL, perform a more or less exhaustive search for an optimal plan among all possible query evaluation plans. For join queries, the number of possible plans investigated by the MySQL optimizer grows exponentially with the number of tables referenced in a query. For small numbers of tables (typically less than 7 to 10) this is not a problem. However, when larger queries are submitted, the time spent in query optimization may easily become the major bottleneck in the server's performance.
A more flexible method for query optimization enables the user to control how exhaustive the optimizer is in its search for an optimal query evaluation plan. The general idea is that the fewer plans that are investigated by the optimizer, the less time it spends in compiling a query. On the other hand, because the optimizer skips some plans, it may miss finding an optimal plan.
The behavior of the optimizer with respect to the number of plans it evaluates can be controlled using two system variables:
The 'optimizer_prune_level' variable tells the optimizer to skip certain plans based on estimates of the number of rows accessed for each table. Our experience shows that this kind of 'educated guess' rarely misses optimal plans, and may dramatically reduce query compilation times. That is why this option is on ('optimizer_prune_level=1') by default. However, if you believe that the optimizer missed a better query plan, this option can be switched off ('optimizer_prune_level=0') with the risk that query compilation may take much longer. Note that, even with the use of this heuristic, the optimizer still explores a roughly exponential number of plans.
The 'optimizer_search_depth' variable tells how far into the 'future' of each incomplete plan the optimizer should look to evaluate whether it should be expanded further. Smaller values of 'optimizer_search_depth' may result in orders of magnitude smaller query compilation times. For example, queries with 12, 13, or more tables may easily require hours and even days to compile if 'optimizer_search_depth' is close to the number of tables in the query. At the same time, if compiled with 'optimizer_search_depth' equal to 3 or 4, the optimizer may compile in less than a minute for the same query. If you are unsure of what a reasonable value is for 'optimizer_search_depth', this variable can be set to 0 to tell the optimizer to determine the value automatically.
File: manual.info.tmp, Node: switchable-optimizations, Next: optimizer-hints, Prev: controlling-query-plan-evaluation, Up: controlling-optimizer
The 'optimizer_switch' system variable enables control over optimizer behavior. Its value is a set of flags, each of which has a value of 'on' or 'off' to indicate whether the corresponding optimizer behavior is enabled or disabled. This variable has global and session values and can be changed at runtime. The global default can be set at server startup.
To see the current set of optimizer flags, select the variable value:
mysql> SELECT @@optimizer_switch\G
*************************** 1. row ***************************
@@optimizer_switch: index_merge=on,index_merge_union=on,
index_merge_sort_union=on,
index_merge_intersection=on,
engine_condition_pushdown=on,
index_condition_pushdown=on,
mrr=on,mrr_cost_based=on,
block_nested_loop=on,batched_key_access=off,
materialization=on,semijoin=on,loosescan=on,
firstmatch=on,duplicateweedout=on,
subquery_materialization_cost_based=on,
use_index_extensions=on,
condition_fanout_filter=on,derived_merge=on,
prefer_ordering_index=on
To change the value of 'optimizer_switch', assign a value consisting of a comma-separated list of one or more commands:
SET [GLOBAL|SESSION] optimizer_switch='COMMAND[,COMMAND]...';
Each COMMAND value should have one of the forms shown in the following table.
Command Syntax Meaning
'default' Reset every optimization to its default value
'OPT_NAME=default' Set the named optimization to its default value
'OPT_NAME=off' Disable the named optimization
'OPT_NAME=on' Enable the named optimization
The order of the commands in the value does not matter, although the 'default' command is executed first if present. Setting an OPT_NAME flag to 'default' sets it to whichever of 'on' or 'off' is its default value. Specifying any given OPT_NAME more than once in the value is not permitted and causes an error. Any errors in the value cause the assignment to fail with an error, leaving the value of 'optimizer_switch' unchanged.
The following list describes the permissible OPT_NAME flag names, grouped by optimization strategy:
Batched Key Access Flags
* 'batched_key_access' (default 'off')
Controls use of BKA join algorithm.
For 'batched_key_access' to have any effect when set to 'on', the 'mrr' flag must also be 'on'. Currently, the cost estimation for MRR is too pessimistic. Hence, it is also necessary for 'mrr_cost_based' to be 'off' for BKA to be used.
For more information, see *note bnl-bka-optimization::.
Block Nested-Loop Flags
* 'block_nested_loop' (default 'on')
Controls use of BNL join algorithm.
For more information, see *note bnl-bka-optimization::.
Condition Filtering Flags
* 'condition_fanout_filter' (default 'on')
Controls use of condition filtering.
For more information, see *note condition-filtering::.
Derived Table Merging Flags
* 'derived_merge' (default 'on')
Controls merging of derived tables and views into outer query
block.
The 'derived_merge' flag controls whether the optimizer attempts to merge derived tables and view references into the outer query block, assuming that no other rule prevents merging; for example, an 'ALGORITHM' directive for a view takes precedence over the 'derived_merge' setting. By default, the flag is 'on' to enable merging.
For more information, see *note derived-table-optimization::.
Engine Condition Pushdown Flags
* 'engine_condition_pushdown' (default 'on')
Controls engine condition pushdown.
For more information, see *note engine-condition-pushdown-optimization::.
Index Condition Pushdown Flags
* 'index_condition_pushdown' (default 'on')
Controls index condition pushdown.
For more information, see *note index-condition-pushdown-optimization::.
Index Extensions Flags
* 'use_index_extensions' (default 'on')
Controls use of index extensions.
For more information, see *note index-extensions::.
Index Merge Flags
* 'index_merge' (default 'on')
Controls all Index Merge optimizations.
* 'index_merge_intersection' (default 'on')
Controls the Index Merge Intersection Access optimization.
* 'index_merge_sort_union' (default 'on')
Controls the Index Merge Sort-Union Access optimization.
* 'index_merge_union' (default 'on')
Controls the Index Merge Union Access optimization.
For more information, see *note index-merge-optimization::.
Limit Optimization Flags
* 'prefer_ordering_index' (default 'on')
Controls whether, in the case of a query having an 'ORDER BY'
or 'GROUP BY' with a 'LIMIT' clause, the optimizer tries to
use an ordered index instead of an unordered index, a
filesort, or some other optimization. This optimzation is
performed by default whenever the optimizer determines that
using it would allow for faster execution of the query.
Because the algorithm that makes this determination cannot
handle every conceivable case (due in part to the assumption
that the distribution of data is always more or less uniform),
there are cases in which this optimization may not be
desirable. Prior to MySQL 5.7.33, it ws not possible to
disable this optimization, but in MySQL 5.7.33 and later,
while it remains the default behavior, it can be disabled by
setting the 'prefer_ordering_index' flag to 'off'.
For more information and examples, see *note limit-optimization::.
Multi-Range Read Flags
* 'mrr' (default 'on')
Controls the Multi-Range Read strategy.
* 'mrr_cost_based' (default 'on')
Controls use of cost-based MRR if 'mrr=on'.
For more information, see *note mrr-optimization::.
Semijoin Flags
* 'duplicateweedout' (default 'on')
Controls the semijoin Duplicate Weedout strategy.
* 'firstmatch' (default 'on')
Controls the semijoin FirstMatch strategy.
* 'loosescan' (default 'on')
Controls the semijoin LooseScan strategy (not to be confused
with Loose Index Scan for 'GROUP BY').
* 'semijoin' (default 'on')
Controls all semijoin strategies.
The 'semijoin', 'firstmatch', 'loosescan', and 'duplicateweedout' flags enable control over semijoin strategies. The 'semijoin' flag controls whether semijoins are used. If it is set to 'on', the 'firstmatch' and 'loosescan' flags enable finer control over the permitted semijoin strategies.
If the 'duplicateweedout' semijoin strategy is disabled, it is not used unless all other applicable strategies are also disabled.
If 'semijoin' and 'materialization' are both 'on', semijoins also use materialization where applicable. These flags are 'on' by default.
For more information, see *note semijoins::.
Subquery Materialization Flags
* 'materialization' (default 'on')
Controls materialization (including semijoin materialization).
* 'subquery_materialization_cost_based' (default 'on')
Use cost-based materialization choice.
The 'materialization' flag controls whether subquery materialization is used. If 'semijoin' and 'materialization' are both 'on', semijoins also use materialization where applicable. These flags are 'on' by default.
The 'subquery_materialization_cost_based' flag enables control over the choice between subquery materialization and 'IN'-to-'EXISTS' subquery transformation. If the flag is 'on' (the default), the optimizer performs a cost-based choice between subquery materialization and 'IN'-to-'EXISTS' subquery transformation if either method could be used. If the flag is 'off', the optimizer chooses subquery materialization over 'IN'-to-'EXISTS' subquery transformation.
For more information, see *note subquery-optimization::.
When you assign a value to 'optimizer_switch', flags that are not mentioned keep their current values. This makes it possible to enable or disable specific optimizer behaviors in a single statement without affecting other behaviors. The statement does not depend on what other optimizer flags exist and what their values are. Suppose that all Index Merge optimizations are enabled:
mysql> SELECT @@optimizer_switch\G
*************************** 1. row ***************************
@@optimizer_switch: index_merge=on,index_merge_union=on,
index_merge_sort_union=on,
index_merge_intersection=on,
engine_condition_pushdown=on,
index_condition_pushdown=on,
mrr=on,mrr_cost_based=on,
block_nested_loop=on,batched_key_access=off,
materialization=on,semijoin=on,loosescan=on,
firstmatch=on,duplicateweedout=on,
subquery_materialization_cost_based=on,
use_index_extensions=on,
condition_fanout_filter=on,derived_merge=on,
prefer_ordering_index=on
If the server is using the Index Merge Union or Index Merge Sort-Union access methods for certain queries and you want to check whether the optimizer performs better without them, set the variable value like this:
mysql> SET optimizer_switch='index_merge_union=off,index_merge_sort_union=off';
mysql> SELECT @@optimizer_switch\G
*************************** 1. row ***************************
@@optimizer_switch: index_merge=on,index_merge_union=off,
index_merge_sort_union=off,
index_merge_intersection=on,
engine_condition_pushdown=on,
index_condition_pushdown=on,
mrr=on,mrr_cost_based=on,
block_nested_loop=on,batched_key_access=off,
materialization=on,semijoin=on,loosescan=on,
firstmatch=on,duplicateweedout=on,
subquery_materialization_cost_based=on,
use_index_extensions=on,
condition_fanout_filter=on,derived_merge=on,
prefer_ordering_index=on
File: manual.info.tmp, Node: optimizer-hints, Next: index-hints, Prev: switchable-optimizations, Up: controlling-optimizer
One means of control over optimizer strategies is to set the 'optimizer_switch' system variable (see *note switchable-optimizations::). Changes to this variable affect execution of all subsequent queries; to affect one query differently from another, it is necessary to change 'optimizer_switch' before each one.
another way to control the optimizer is by using optimizer hints, which can be specified within individual statements. Because optimizer hints apply on a per-statement basis, they provide finer control over statement execution plans than can be achieved using 'optimizer_switch'. For example, you can enable an optimization for one table in a statement and disable the optimization for a different table. Hints within a statement take precedence over 'optimizer_switch' flags.
Examples:
SELECT /*+ NO_RANGE_OPTIMIZATION(t3 PRIMARY, f2_idx) */ f1
FROM t3 WHERE f1 > 30 AND f1 < 33;
SELECT /*+ BKA(t1) NO_BKA(t2) */ * FROM t1 INNER JOIN t2 WHERE ...;
SELECT /*+ NO_ICP(t1, t2) */ * FROM t1 INNER JOIN t2 WHERE ...;
SELECT /*+ SEMIJOIN(FIRSTMATCH, LOOSESCAN) */ * FROM t1 ...;
EXPLAIN SELECT /*+ NO_ICP(t1) */ * FROM t1 WHERE ...;
Note:
The note 'mysql': mysql. client by default strips comments from SQL statements sent to the server (including optimizer hints) until MySQL 5.7.7, when it was changed to pass optimizer hints to the server. To ensure that optimizer hints are not stripped if you are using an older version of the note 'mysql': mysql. client with a version of the server that understands optimizer hints, invoke *note 'mysql': mysql. with the '--comments' option.
Optimizer hints, described here, differ from index hints, described in *note index-hints::. Optimizer and index hints may be used separately or together.
*note optimizer-hints-overview::
*note optimizer-hints-syntax::
*note optimizer-hints-table-level::
*note optimizer-hints-index-level::
*note optimizer-hints-subquery::
*note optimizer-hints-execution-time::
*note optimizer-hints-query-block-naming::
Optimizer Hint Overview
Optimizer hints apply at different scope levels:
Global: The hint affects the entire statement
Query block: The hint affects a particular query block within a statement
Table-level: The hint affects a particular table within a query block
Index-level: The hint affects a particular index within a table
The following table summarizes the available optimizer hints, the optimizer strategies they affect, and the scope or scopes at which they apply. More details are given later.
Optimizer Hints Available
Hint Name Description Applicable Scopes
'BKA', 'NO_BKA'
Affects Batched Key Query block, table Access join
processing
'BNL', 'NO_BNL'
Affects Block Query block, table Nested-Loop join
processing
'MAX_EXECUTION_TIME'
Limits statement Global execution time
'MRR', 'NO_MRR'
Affects Multi-Range Table, index Read optimization
'NO_ICP'
Affects Index Table, index Condition Pushdown
optimization
'NO_RANGE_OPTIMIZATION'
Affects range Table, index optimization
'QB_NAME'
Assigns name to Query block query block
'SEMIJOIN', 'NO_SEMIJOIN'
semijoin strategies Query block
'SUBQUERY'
Affects Query block materialization, 'IN'-to-'EXISTS' subquery stratgies
Disabling an optimization prevents the optimizer from using it. Enabling an optimization means the optimizer is free to use the strategy if it applies to statement execution, not that the optimizer necessarily uses it.
Optimizer Hint Syntax
MySQL supports comments in SQL statements as described in note comments::. Optimizer hints must be specified within '/+ ... /' comments. That is, optimizer hints use a variant of '/ ... /' C-style comment syntax, with a '+' character following the '/' comment opening sequence. Examples:
/*+ BKA(t1) */
/*+ BNL(t1, t2) */
/*+ NO_RANGE_OPTIMIZATION(t4 PRIMARY) */
/*+ QB_NAME(qb2) */
Whitespace is permitted after the '+' character.
The parser recognizes optimizer hint comments after the initial keyword of note 'SELECT': select, note 'UPDATE': update, note 'INSERT': insert, note 'REPLACE': replace, and *note 'DELETE': delete. statements. Hints are permitted in these contexts:
At the beginning of query and data change statements:
SELECT /*+ ... */ ...
INSERT /*+ ... */ ...
REPLACE /*+ ... */ ...
UPDATE /*+ ... */ ...
DELETE /*+ ... */ ...
At the beginning of query blocks:
(SELECT /*+ ... */ ... )
(SELECT ... ) UNION (SELECT /*+ ... */ ... )
(SELECT /*+ ... */ ... ) UNION (SELECT /*+ ... */ ... )
UPDATE ... WHERE x IN (SELECT /*+ ... */ ...)
INSERT ... SELECT /*+ ... */ ...
In hintable statements prefaced by *note 'EXPLAIN': explain. For example:
EXPLAIN SELECT /*+ ... */ ...
EXPLAIN UPDATE ... WHERE x IN (SELECT /*+ ... */ ...)
The implication is that you can use note 'EXPLAIN': explain. to see how optimizer hints affect execution plans. Use note 'SHOW WARNINGS': show-warnings. immediately after note 'EXPLAIN': explain. to see how hints are used. The extended 'EXPLAIN' output displayed by a following note 'SHOW WARNINGS': show-warnings. indicates which hints were used. Ignored hints are not displayed.
A hint comment may contain multiple hints, but a query block cannot contain multiple hint comments. This is valid:
SELECT /*+ BNL(t1) BKA(t2) */ ...
But this is invalid:
SELECT /*+ BNL(t1) */ /* BKA(t2) */ ...
When a hint comment contains multiple hints, the possibility of duplicates and conflicts exists. The following general guidelines apply. For specific hint types, additional rules may apply, as indicated in the hint descriptions.
Duplicate hints: For a hint such as '/+ MRR(idx1) MRR(idx1) /', MySQL uses the first hint and issues a warning about the duplicate hint.
Conflicting hints: For a hint such as '/+ MRR(idx1) NO_MRR(idx1) /', MySQL uses the first hint and issues a warning about the second conflicting hint.
Query block names are identifiers and follow the usual rules about what names are valid and how to quote them (see *note identifiers::).
Hint names, query block names, and strategy names are not case-sensitive. References to table and index names follow the usual identifier case sensitivity rules (see *note identifier-case-sensitivity::).
Table-Level Optimizer Hints
Table-level hints affect use of the Block Nested-Loop (BNL) and Batched Key Access (BKA) join-processing algorithms (see *note bnl-bka-optimization::). These hint types apply to specific tables, or all tables in a query block.
Syntax of table-level hints:
HINT_NAME([@QUERY_BLOCK_NAME] [TBL_NAME [, TBL_NAME] ...])
HINT_NAME([TBL_NAME@QUERY_BLOCK_NAME [, TBL_NAME@QUERY_BLOCK_NAME] ...])
The syntax refers to these terms:
HINT_NAME: These hint names are permitted:
* 'BKA', 'NO_BKA': Enable or disable BKA for the specified
tables.
* 'BNL', 'NO_BNL': Enable or disable BNL for the specified
tables.
Note:
To use a BNL or BKA hint to enable join buffering for any inner table of an outer join, join buffering must be enabled for all inner tables of the outer join.
TBL_NAME: The name of a table used in the statement. The hint applies to all tables that it names. If the hint names no tables, it applies to all tables of the query block in which it occurs.
If a table has an alias, hints must refer to the alias, not the table name.
Table names in hints cannot be qualified with schema names.
QUERY_BLOCK_NAME: The query block to which the hint applies. If the hint includes no leading '@QUERY_BLOCK_NAME', the hint applies to the query block in which it occurs. For 'TBL_NAME@QUERY_BLOCK_NAME' syntax, the hint applies to the named table in the named query block. To assign a name to a query block, see *note optimizer-hints-query-block-naming::.
Examples:
SELECT /*+ NO_BKA(t1, t2) */ t1.* FROM t1 INNER JOIN t2 INNER JOIN t3;
SELECT /*+ NO_BNL() BKA(t1) */ t1.* FROM t1 INNER JOIN t2 INNER JOIN t3;
A table-level hint applies to tables that receive records from previous tables, not sender tables. Consider this statement:
SELECT /*+ BNL(t2) */ FROM t1, t2;
If the optimizer chooses to process 't1' first, it applies a Block Nested-Loop join to 't2' by buffering the rows from 't1' before starting to read from 't2'. If the optimizer instead chooses to process 't2' first, the hint has no effect because 't2' is a sender table.
Index-Level Optimizer Hints
Index-level hints affect which index-processing strategies the optimizer uses for particular tables or indexes. These hint types affect use of Index Condition Pushdown (ICP), Multi-Range Read (MRR), and range optimizations (see *note select-optimization::).
Syntax of index-level hints:
HINT_NAME([@QUERY_BLOCK_NAME] TBL_NAME [INDEX_NAME [, INDEX_NAME] ...])
HINT_NAME(TBL_NAME@QUERY_BLOCK_NAME [INDEX_NAME [, INDEX_NAME] ...])
The syntax refers to these terms:
HINT_NAME: These hint names are permitted:
* 'MRR', 'NO_MRR': Enable or disable MRR for the specified table
or indexes. MRR hints apply only to 'InnoDB' and 'MyISAM'
tables.
* 'NO_ICP': Disable ICP for the specified table or indexes. By
default, ICP is a candidate optimization strategy, so there is
no hint for enabling it.
* 'NO_RANGE_OPTIMIZATION': Disable index range access for the
specified table or indexes. This hint also disables Index
Merge and Loose Index Scan for the table or indexes. By
default, range access is a candidate optimization strategy, so
there is no hint for enabling it.
This hint may be useful when the number of ranges may be high
and range optimization would require many resources.
TBL_NAME: The table to which the hint applies.
INDEX_NAME: The name of an index in the named table. The hint applies to all indexes that it names. If the hint names no indexes, it applies to all indexes in the table.
To refer to a primary key, use the name 'PRIMARY'. To see the index names for a table, use *note 'SHOW INDEX': show-index.
QUERY_BLOCK_NAME: The query block to which the hint applies. If the hint includes no leading '@QUERY_BLOCK_NAME', the hint applies to the query block in which it occurs. For 'TBL_NAME@QUERY_BLOCK_NAME' syntax, the hint applies to the named table in the named query block. To assign a name to a query block, see *note optimizer-hints-query-block-naming::.
Examples:
SELECT /*+ MRR(t1) */ * FROM t1 WHERE f2 <= 3 AND 3 <= f3;
SELECT /*+ NO_RANGE_OPTIMIZATION(t3 PRIMARY, f2_idx) */ f1
FROM t3 WHERE f1 > 30 AND f1 < 33;
INSERT INTO t3(f1, f2, f3)
(SELECT /*+ NO_ICP(t2) */ t2.f1, t2.f2, t2.f3 FROM t1,t2
WHERE t1.f1=t2.f1 AND t2.f2 BETWEEN t1.f1
AND t1.f2 AND t2.f2 + 1 >= t1.f1 + 1);
Subquery Optimizer Hints
Subquery hints affect whether to use semijoin transformations and which semijoin strategies to permit, and, when semijoins are not used, whether to use subquery materialization or 'IN'-to-'EXISTS' transformations. For more information about these optimizations, see *note subquery-optimization::.
Syntax of hints that affect semijoin strategies:
HINT_NAME([@QUERY_BLOCK_NAME] [STRATEGY [, STRATEGY] ...])
The syntax refers to these terms:
HINT_NAME: These hint names are permitted:
* 'SEMIJOIN', 'NO_SEMIJOIN': Enable or disable the named
semijoin strategies.
STRATEGY: A semijoin strategy to be enabled or disabled. These strategy names are permitted: 'DUPSWEEDOUT', 'FIRSTMATCH', 'LOOSESCAN', 'MATERIALIZATION'.
For 'SEMIJOIN' hints, if no strategies are named, semijoin is used if possible based on the strategies enabled according to the 'optimizer_switch' system variable. If strategies are named but inapplicable for the statement, 'DUPSWEEDOUT' is used.
For 'NO_SEMIJOIN' hints, if no strategies are named, semijoin is not used. If strategies are named that rule out all applicable strategies for the statement, 'DUPSWEEDOUT' is used.
If one subquery is nested within another and both are merged into a semijoin of an outer query, any specification of semijoin strategies for the innermost query are ignored. 'SEMIJOIN' and 'NO_SEMIJOIN' hints can still be used to enable or disable semijoin transformations for such nested subqueries.
If 'DUPSWEEDOUT' is disabled, on occasion the optimizer may generate a query plan that is far from optimal. This occurs due to heuristic pruning during greedy search, which can be avoided by setting 'optimizer_prune_level=0'.
Examples:
SELECT /*+ NO_SEMIJOIN(@subq1 FIRSTMATCH, LOOSESCAN) */ * FROM t2
WHERE t2.a IN (SELECT /*+ QB_NAME(subq1) */ a FROM t3);
SELECT /*+ SEMIJOIN(@subq1 MATERIALIZATION, DUPSWEEDOUT) */ * FROM t2
WHERE t2.a IN (SELECT /*+ QB_NAME(subq1) */ a FROM t3);
Syntax of hints that affect whether to use subquery materialization or 'IN'-to-'EXISTS' transformations:
SUBQUERY([@QUERY_BLOCK_NAME] STRATEGY)
The hint name is always 'SUBQUERY'.
For 'SUBQUERY' hints, these STRATEGY values are permitted: 'INTOEXISTS', 'MATERIALIZATION'.
Examples:
SELECT id, a IN (SELECT /*+ SUBQUERY(MATERIALIZATION) */ a FROM t1) FROM t2;
SELECT * FROM t2 WHERE t2.a IN (SELECT /*+ SUBQUERY(INTOEXISTS) */ a FROM t1);
For semijoin and 'SUBQUERY' hints, a leading '@QUERY_BLOCK_NAME' specifies the query block to which the hint applies. If the hint includes no leading '@QUERY_BLOCK_NAME', the hint applies to the query block in which it occurs. To assign a name to a query block, see *note optimizer-hints-query-block-naming::.
If a hint comment contains multiple subquery hints, the first is used. If there are other following hints of that type, they produce a warning. Following hints of other types are silently ignored.
Statement Execution Time Optimizer Hints
The 'MAX_EXECUTION_TIME' hint is permitted only for *note 'SELECT': select. statements. It places a limit N (a timeout value in milliseconds) on how long a statement is permitted to execute before the server terminates it:
MAX_EXECUTION_TIME(N)
Example with a timeout of 1 second (1000 milliseconds):
SELECT /*+ MAX_EXECUTION_TIME(1000) */ * FROM t1 INNER JOIN t2 WHERE ...
The 'MAX_EXECUTION_TIME(N)' hint sets a statement execution timeout of N milliseconds. If this option is absent or N is 0, the statement timeout established by the 'max_execution_time' system variable applies.
The 'MAX_EXECUTION_TIME' hint is applicable as follows:
For statements with multiple 'SELECT' keywords, such as unions or statements with subqueries, 'MAX_EXECUTION_TIME' applies to the entire statement and must appear after the first *note 'SELECT': select.
It applies to read-only *note 'SELECT': select. statements. Statements that are not read only are those that invoke a stored function that modifies data as a side effect.
It does not apply to *note 'SELECT': select. statements in stored programs and is ignored.
Optimizer Hints for Naming Query Blocks
Table-level, index-level, and subquery optimizer hints permit specific query blocks to be named as part of their argument syntax. To create these names, use the 'QB_NAME' hint, which assigns a name to the query block in which it occurs:
QB_NAME(NAME)
'QB_NAME' hints can be used to make explicit in a clear way which query blocks other hints apply to. They also permit all non-query block name hints to be specified within a single hint comment for easier understanding of complex statements. Consider the following statement:
SELECT ...
FROM (SELECT ...
FROM (SELECT ... FROM ...)) ...
'QB_NAME' hints assign names to query blocks in the statement:
SELECT /*+ QB_NAME(qb1) */ ...
FROM (SELECT /*+ QB_NAME(qb2) */ ...
FROM (SELECT /*+ QB_NAME(qb3) */ ... FROM ...)) ...
Then other hints can use those names to refer to the appropriate query blocks:
SELECT /*+ QB_NAME(qb1) MRR(@qb1 t1) BKA(@qb2) NO_MRR(@qb3t1 idx1, id2) */ ...
FROM (SELECT /*+ QB_NAME(qb2) */ ...
FROM (SELECT /*+ QB_NAME(qb3) */ ... FROM ...)) ...
The resulting effect is as follows:
'MRR(@qb1 t1)' applies to table 't1' in query block 'qb1'.
'BKA(@qb2)' applies to query block 'qb2'.
'NO_MRR(@qb3 t1 idx1, id2)' applies to indexes 'idx1' and 'idx2' in table 't1' in query block 'qb3'.
Query block names are identifiers and follow the usual rules about what names are valid and how to quote them (see *note identifiers::). For example, a query block name that contains spaces must be quoted, which can be done using backticks:
SELECT /*+ BKA(@`my hint name`) */ ...
FROM (SELECT /*+ QB_NAME(`my hint name`) */ ...) ...
If the 'ANSI_QUOTES' SQL mode is enabled, it is also possible to quote query block names within double quotation marks:
SELECT /*+ BKA(@"my hint name") */ ...
FROM (SELECT /*+ QB_NAME("my hint name") */ ...) ...
File: manual.info.tmp, Node: index-hints, Next: cost-model, Prev: optimizer-hints, Up: controlling-optimizer
Index hints give the optimizer information about how to choose indexes during query processing. Index hints, described here, differ from optimizer hints, described in *note optimizer-hints::. Index and optimizer hints may be used separately or together.
Index hints apply to note 'SELECT': select. and note 'UPDATE': update. statements. They also work with multi-table *note 'DELETE': delete. statements, but not with single-table 'DELETE', as shown later in this section.
Index hints are specified following a table name. (For the general syntax for specifying tables in a note 'SELECT': select. statement, see note join::.) The syntax for referring to an individual table, including index hints, looks like this:
TBL_NAME [[AS] ALIAS] [INDEX_HINT_LIST]
INDEX_HINT_LIST:
INDEX_HINT [INDEX_HINT] ...
INDEX_HINT:
USE {INDEX|KEY}
[FOR {JOIN|ORDER BY|GROUP BY}] ([INDEX_LIST])
| {IGNORE|FORCE} {INDEX|KEY}
[FOR {JOIN|ORDER BY|GROUP BY}] (INDEX_LIST)
INDEX_LIST:
INDEX_NAME [, INDEX_NAME] ...
The 'USE INDEX (INDEX_LIST)' hint tells MySQL to use only one of the named indexes to find rows in the table. The alternative syntax 'IGNORE INDEX (INDEX_LIST)' tells MySQL to not use some particular index or indexes. These hints are useful if *note 'EXPLAIN': explain. shows that MySQL is using the wrong index from the list of possible indexes.
The 'FORCE INDEX' hint acts like 'USE INDEX (INDEX_LIST)', with the addition that a table scan is assumed to be very expensive. In other words, a table scan is used only if there is no way to use one of the named indexes to find rows in the table.
Each hint requires index names, not column names. To refer to a primary key, use the name 'PRIMARY'. To see the index names for a table, use the note 'SHOW INDEX': show-index. statement or the Information Schema note 'STATISTICS': information-schema-statistics-table. table.
An INDEX_NAME value need not be a full index name. It can be an unambiguous prefix of an index name. If a prefix is ambiguous, an error occurs.
Examples:
SELECT * FROM table1 USE INDEX (col1_index,col2_index)
WHERE col1=1 AND col2=2 AND col3=3;
SELECT * FROM table1 IGNORE INDEX (col3_index)
WHERE col1=1 AND col2=2 AND col3=3;
The syntax for index hints has the following characteristics:
It is syntactically valid to omit INDEX_LIST for 'USE INDEX', which means 'use no indexes.' Omitting INDEX_LIST for 'FORCE INDEX' or 'IGNORE INDEX' is a syntax error.
You can specify the scope of an index hint by adding a 'FOR' clause to the hint. This provides more fine-grained control over optimizer selection of an execution plan for various phases of query processing. To affect only the indexes used when MySQL decides how to find rows in the table and how to process joins, use 'FOR JOIN'. To influence index usage for sorting or grouping rows, use 'FOR ORDER BY' or 'FOR GROUP BY'.
You can specify multiple index hints:
SELECT * FROM t1 USE INDEX (i1) IGNORE INDEX FOR ORDER BY (i2) ORDER BY a;
It is not an error to name the same index in several hints (even within the same hint):
SELECT * FROM t1 USE INDEX (i1) USE INDEX (i1,i1);
However, it is an error to mix 'USE INDEX' and 'FORCE INDEX' for the same table:
SELECT * FROM t1 USE INDEX FOR JOIN (i1) FORCE INDEX FOR JOIN (i2);
If an index hint includes no 'FOR' clause, the scope of the hint is to apply to all parts of the statement. For example, this hint:
IGNORE INDEX (i1)
is equivalent to this combination of hints:
IGNORE INDEX FOR JOIN (i1)
IGNORE INDEX FOR ORDER BY (i1)
IGNORE INDEX FOR GROUP BY (i1)
In MySQL 5.0, hint scope with no 'FOR' clause was to apply only to row retrieval. To cause the server to use this older behavior when no 'FOR' clause is present, enable the 'old' system variable at server startup. Take care about enabling this variable in a replication setup. With statement-based binary logging, having different modes for the source and replicas might lead to replication errors.
When index hints are processed, they are collected in a single list by type ('USE', 'FORCE', 'IGNORE') and by scope ('FOR JOIN', 'FOR ORDER BY', 'FOR GROUP BY'). For example:
SELECT * FROM t1
USE INDEX () IGNORE INDEX (i2) USE INDEX (i1) USE INDEX (i2);
is equivalent to:
SELECT * FROM t1
USE INDEX (i1,i2) IGNORE INDEX (i2);
The index hints then are applied for each scope in the following order:
'{USE|FORCE} INDEX' is applied if present. (If not, the optimizer-determined set of indexes is used.)
'IGNORE INDEX' is applied over the result of the previous step. For example, the following two queries are equivalent:
SELECT * FROM t1 USE INDEX (i1) IGNORE INDEX (i2) USE INDEX (i2);
SELECT * FROM t1 USE INDEX (i1);
For 'FULLTEXT' searches, index hints work as follows:
For natural language mode searches, index hints are silently ignored. For example, 'IGNORE INDEX(i1)' is ignored with no warning and the index is still used.
For boolean mode searches, index hints with 'FOR ORDER BY' or 'FOR GROUP BY' are silently ignored. Index hints with 'FOR JOIN' or no 'FOR' modifier are honored. In contrast to how hints apply for non-'FULLTEXT' searches, the hint is used for all phases of query execution (finding rows and retrieval, grouping, and ordering). This is true even if the hint is given for a non-'FULLTEXT' index.
For example, the following two queries are equivalent:
SELECT * FROM t
USE INDEX (index1)
IGNORE INDEX FOR ORDER BY (index1)
IGNORE INDEX FOR GROUP BY (index1)
WHERE ... IN BOOLEAN MODE ... ;
SELECT * FROM t
USE INDEX (index1)
WHERE ... IN BOOLEAN MODE ... ;
Index hints work with *note 'DELETE': delete. statements, but only if you use multi-table 'DELETE' syntax, as shown here:
mysql> EXPLAIN DELETE FROM t1 USE INDEX(col2)
-> WHERE col1 BETWEEN 1 AND 100 AND COL2 BETWEEN 1 AND 100\G
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that
corresponds to your MySQL server version for the right syntax to use near 'use
index(col2) where col1 between 1 and 100 and col2 between 1 and 100' at line 1
mysql> EXPLAIN DELETE t1.* FROM t1 USE INDEX(col2)
-> WHERE col1 BETWEEN 1 AND 100 AND COL2 BETWEEN 1 AND 100\G
*************************** 1. row ***************************
id: 1
select_type: DELETE
table: t1
partitions: NULL
type: range
possible_keys: col2
key: col2
key_len: 5
ref: NULL
rows: 72
filtered: 11.11
Extra: Using where
1 row in set, 1 warning (0.00 sec)
File: manual.info.tmp, Node: cost-model, Prev: index-hints, Up: controlling-optimizer
To generate execution plans, the optimizer uses a cost model that is based on estimates of the cost of various operations that occur during query execution. The optimizer has a set of compiled-in default 'cost constants' available to it to make decisions regarding execution plans.
The optimizer also has a database of cost estimates to use during execution plan construction. These estimates are stored in the 'server_cost' and 'engine_cost' tables in the 'mysql' system database and are configurable at any time. The intent of these tables is to make it possible to easily adjust the cost estimates that the optimizer uses when it attempts to arrive at query execution plans.
*note cost-model-operation::
*note cost-model-database::
*note cost-model-database-modifications::
Cost Model General Operation
The configurable optimizer cost model works like this:
The server reads the cost model tables into memory at startup and uses the in-memory values at runtime. Any non-'NULL' cost estimate specified in the tables takes precedence over the corresponding compiled-in default cost constant. Any 'NULL' estimate indicates to the optimizer to use the compiled-in default.
At runtime, the server may re-read the cost tables. This occurs when a storage engine is dynamically loaded or when a 'FLUSH OPTIMIZER_COSTS' statement is executed.
Cost tables enable server administrators to easily adjust cost estimates by changing entries in the tables. It is also easy to revert to a default by setting an entry's cost to 'NULL'. The optimizer uses the in-memory cost values, so changes to the tables should be followed by 'FLUSH OPTIMIZER_COSTS' to take effect.
The in-memory cost estimates that are current when a client session begins apply throughout that session until it ends. In particular, if the server re-reads the cost tables, any changed estimates apply only to subsequently started sessions. Existing sessions are unaffected.
Cost tables are specific to a given server instance. The server does not replicate cost table changes to replicas.
The Cost Model Database
The optimizer cost model database consists of two tables in the 'mysql' system database that contain cost estimate information for operations that occur during query execution:
'server_cost': Optimizer cost estimates for general server operations
'engine_cost': Optimizer cost estimates for operations specific to particular storage engines
The 'server_cost' table contains these columns:
'cost_name'
The name of a cost estimate used in the cost model. The name is not case-sensitive. If the server does not recognize the cost name when it reads this table, it writes a warning to the error log.
'cost_value'
The cost estimate value. If the value is non-'NULL', the server uses it as the cost. Otherwise, it uses the default estimate (the compiled-in value). DBAs can change a cost estimate by updating this column. If the server finds that the cost value is invalid (nonpositive) when it reads this table, it writes a warning to the error log.
To override a default cost estimate (for an entry that specifies 'NULL'), set the cost to a non-'NULL' value. To revert to the default, set the value to 'NULL'. Then execute 'FLUSH OPTIMIZER_COSTS' to tell the server to re-read the cost tables.
'last_update'
The time of the last row update.
'comment'
A descriptive comment associated with the cost estimate. DBAs can use this column to provide information about why a cost estimate row stores a particular value.
The primary key for the 'server_cost' table is the 'cost_name' column, so it is not possible to create multiple entries for any cost estimate.
The server recognizes these 'cost_name' values for the 'server_cost' table:
'disk_temptable_create_cost' (default 40.0), 'disk_temptable_row_cost' (default 1.0)
The cost estimates for internally created temporary tables stored in a disk-based storage engine (either 'InnoDB' or 'MyISAM'). Increasing these values increases the cost estimate of using internal temporary tables and makes the optimizer prefer query plans with less use of them. For information about such tables, see *note internal-temporary-tables::.
The larger default values for these disk parameters compared to the default values for the corresponding memory parameters ('memory_temptable_create_cost', 'memory_temptable_row_cost') reflects the greater cost of processing disk-based tables.
'key_compare_cost' (default 0.1)
The cost of comparing record keys. Increasing this value causes a query plan that compares many keys to become more expensive. For example, a query plan that performs a 'filesort' becomes relatively more expensive compared to a query plan that avoids sorting by using an index.
'memory_temptable_create_cost' (default 2.0), 'memory_temptable_row_cost' (default 0.2)
The cost estimates for internally created temporary tables stored in the 'MEMORY' storage engine. Increasing these values increases the cost estimate of using internal temporary tables and makes the optimizer prefer query plans with less use of them. For information about such tables, see *note internal-temporary-tables::.
The smaller default values for these memory parameters compared to the default values for the corresponding disk parameters ('disk_temptable_create_cost', 'disk_temptable_row_cost') reflects the lesser cost of processing memory-based tables.
'row_evaluate_cost' (default 0.2)
The cost of evaluating record conditions. Increasing this value causes a query plan that examines many rows to become more expensive compared to a query plan that examines fewer rows. For example, a table scan becomes relatively more expensive compared to a range scan that reads fewer rows.
The 'engine_cost' table contains these columns:
'engine_name'
The name of the storage engine to which this cost estimate applies. The name is not case-sensitive. If the value is 'default', it applies to all storage engines that have no named entry of their own. If the server does not recognize the engine name when it reads this table, it writes a warning to the error log.
'device_type'
The device type to which this cost estimate applies. The column is intended for specifying different cost estimates for different storage device types, such as hard disk drives versus solid state drives. Currently, this information is not used and 0 is the only permitted value.
'cost_name'
Same as in the 'server_cost' table.
'cost_value'
Same as in the 'server_cost' table.
'last_update'
Same as in the 'server_cost' table.
'comment'
Same as in the 'server_cost' table.
The primary key for the 'engine_cost' table is a tuple comprising the ('cost_name', 'engine_name', 'device_type') columns, so it is not possible to create multiple entries for any combination of values in those columns.
The server recognizes these 'cost_name' values for the 'engine_cost' table:
'io_block_read_cost' (default 1.0)
The cost of reading an index or data block from disk. Increasing this value causes a query plan that reads many disk blocks to become more expensive compared to a query plan that reads fewer disk blocks. For example, a table scan becomes relatively more expensive compared to a range scan that reads fewer blocks.
'memory_block_read_cost' (default 1.0)
Similar to 'io_block_read_cost', but represents the cost of reading an index or data block from an in-memory database buffer.
If the 'io_block_read_cost' and 'memory_block_read_cost' values differ, the execution plan may change between two runs of the same query. Suppose that the cost for memory access is less than the cost for disk access. In that case, at server startup before data has been read into the buffer pool, you may get a different plan than after the query has been run because then the data is in memory.
Making Changes to the Cost Model Database
For DBAs who wish to change the cost model parameters from their defaults, try doubling or halving the value and measuring the effect.
Changes to the 'io_block_read_cost' and 'memory_block_read_cost' parameters are most likely to yield worthwhile results. These parameter values enable cost models for data access methods to take into account the costs of reading information from different sources; that is, the cost of reading information from disk versus reading information already in a memory buffer. For example, all other things being equal, setting 'io_block_read_cost' to a value larger than 'memory_block_read_cost' causes the optimizer to prefer query plans that read information already held in memory to plans that must read from disk.
This example shows how to change the default value for 'io_block_read_cost':
UPDATE mysql.engine_cost
SET cost_value = 2.0
WHERE cost_name = 'io_block_read_cost';
FLUSH OPTIMIZER_COSTS;
This example shows how to change the value of 'io_block_read_cost' only for the 'InnoDB' storage engine:
INSERT INTO mysql.engine_cost
VALUES ('InnoDB', 0, 'io_block_read_cost', 3.0,
CURRENT_TIMESTAMP, 'Using a slower disk for InnoDB');
FLUSH OPTIMIZER_COSTS;
File: manual.info.tmp, Node: buffering-caching, Next: locking-issues, Prev: controlling-optimizer, Up: optimization