動態

詳情 返回 返回

MySQL性能分析(三)之optimizer_trace詳解 - 動態 詳情

一、概述

optimizer_traceMySQL 5.6引入的一項跟蹤功能,它可以跟蹤優化器做出的各種決策(比如訪問表的方法、各種開銷計算、各種轉換等),並將跟蹤結果記錄到information_schema.optimizer_trace表中。此功能默認關閉,開啓後,可分析如下語句:

  • select
  • insert
  • replace
  • update
  • delete
  • explain
  • set
  • declare
  • case
  • if
  • return
  • call

二、開啓及關閉

2.1 開啓optimizer_trace

mysql命令行中,使用如下命令開啓optimizer_trace

set optimizer_trace="enabled=on",end_markers_in_json=on;

也可用set global全局開啓。但即使全局開啓optimizer_trace,每個Session也只能跟蹤它自己執行的語句:

set global optimizer_trace="enabled=on",end_markers_in_json=on;

2.2 關閉optimizer_trace

mysql命令行中,使用如下命令關閉optimizer_trace

SET optimizer_trace="enabled=off";

三、使用optimizer_trace

3.1 相關參數

optimizer_trace

  • optimizer_trace總開關,默認值:enabled=off,one_line=off
  • enabled:是否開啓optimizer_trace;on表示開啓,off表示關閉。
  • one_line:是否開啓單行存儲。on表示開啓;off表示關閉,將會用標準的JSON格式化存儲。設置成on將會有良好的格式,設置成off可節省一些空間。

optimizer_trace_features

  • 控制optimizer_trace跟蹤的內容,默認值:greedy_search=on,range_optimizer=on,dynamic_range=on,repeated_subselect=on,表示開啓所有跟蹤項。

greedy_search:是否跟蹤貪心搜索

  • range_optimizer:是否跟蹤範圍優化器

dynamic_range:是否跟蹤動態範圍優化

  • repeated_subselect:是否跟蹤子查詢,如果設置成off,只跟蹤第一條Item_subselect的執行

optimizer_trace_limit

  • 控制optimizer_trace展示多少條結果,默認1

optimizer_trace_max_mem_size

  • optimizer_trace堆棧信息允許的最大內存,默認1048576

optimizer_trace_offset

  • 第一個要展示的optimizer trace的偏移量,默認-1。

end_markers_in_json

  • 如果JSON結構很大,則很難將右括號和左括號配對。為了幫助讀者閲讀,可將其設置成on,這樣會在右括號附近加上註釋,默認off。

optimizer_trace_limit和optimizer_trace_offset這兩個參數經常配合使用,例如:SET optimizer_trace_offset= , optimizer_trace_limit=
這兩個參數配合使用,有點類似MySQL裏面的 limit語句。
默認情況下,由於optimizer_trace_offset=-1,optimizer_trace_limit=1,記錄最近的一條SQL語句,展示時,每次展示1條數據;
如果改成SET optimizer_trace_offset=-2, optimizer_trace_limit=1,則會記錄倒數第二條SQL語句;

三、使用

3.1 展示條目

開啓optimizer_trace功能,並設置要展示的數據條目數:

set optimizer_trace="enabled=on", end_markers_in_json=on;
set optimizer_trace_offset=-30, optimizer_trace_limit=30;

3.2 分析SQL語句

發送你想要分析的SQL語句,例如:

select *
from salaries
where from_date = '1986-06-26'
and to_date = '1987-06-26';

使用如下語句分析,即可獲得類似如下的結果:

  mysql> select * from information_schema.optimizer_trace limit 30 \G;
  *************************** 1. row ***************************
                            QUERY: select *
  from salaries
where from_date = '1986-06-26'
    and to_date = '1987-06-26'
                              TRACE: {
    "steps": [
      {
        "join_preparation": {
          "select#": 1,
          "steps": [
            {
              "expanded_query": "/* select#1 */ select `salaries`.`emp_no` AS `emp_no`,`salaries`.`salary` AS `salary`,`salaries`.`from_date` AS `from_date`,`salaries`.`to_date` AS `to_date` from `salaries` where ((`salaries`.`from_date` = '1986-06-26') and (`salaries`.`to_date` = '1987-06-26'))"
            }
          ] /* steps */
        } /* join_preparation */
      },
      {
        "join_optimization": {
          "select#": 1,
          "steps": [
            {
              "condition_processing": {
                "condition": "WHERE",
                "original_condition": "((`salaries`.`from_date` = '1986-06-26') and (`salaries`.`to_date` = '1987-06-26'))",
                "steps": [
                  {
                    "transformation": "equality_propagation",
                    "resulting_condition": "(multiple equal('1986-06-26', `salaries`.`from_date`) and multiple equal('1987-06-26', `salaries`.`to_date`))"
                  },
                  {
                    "transformation": "constant_propagation",
                    "resulting_condition": "(multiple equal('1986-06-26', `salaries`.`from_date`) and multiple equal('1987-06-26', `salaries`.`to_date`))"
                  },
                  {
                    "transformation": "trivial_condition_removal",
                    "resulting_condition": "(multiple equal(DATE'1986-06-26', `salaries`.`from_date`) and multiple equal(DATE'1987-06-26', `salaries`.`to_date`))"
                  }
                ] /* steps */
              } /* condition_processing */
            },
            {
              "substitute_generated_columns": {
              } /* substitute_generated_columns */
            },
            {
              "table_dependencies": [
                {
                  "table": "`salaries`",
                  "row_may_be_null": false,
                  "map_bit": 0,
                  "depends_on_map_bits": [
                  ] /* depends_on_map_bits */
                }
              ] /* table_dependencies */
            },
            {
              "ref_optimizer_key_uses": [
                {
                  "table": "`salaries`",
                  "field": "from_date",
                  "equals": "DATE'1986-06-26'",
                  "null_rejecting": false
                },
                {
                  "table": "`salaries`",
                  "field": "to_date",
                  "equals": "DATE'1987-06-26'",
                  "null_rejecting": false
                }
              ] /* ref_optimizer_key_uses */
            },
            {
              "rows_estimation": [
                {
                  "table": "`salaries`",
                  "range_analysis": {
                    "table_scan": {
                      "rows": 2838216,
                      "cost": 286799
                    } /* table_scan */,
                    "potential_range_indexes": [
                      {
                        "index": "PRIMARY",
                        "usable": false,
                        "cause": "not_applicable"
                      },
                      {
                        "index": "salaries_from_date_to_date_index",
                        "usable": true,
                        "key_parts": [
                          "from_date",
                          "to_date",
                          "emp_no"
                        ] /* key_parts */
                      }
                    ] /* potential_range_indexes */,
                    "setup_range_conditions": [
                    ] /* setup_range_conditions */,
                    "group_index_range": {
                      "chosen": false,
                      "cause": "not_group_by_or_distinct"
                    } /* group_index_range */,
                    "skip_scan_range": {
                      "potential_skip_scan_indexes": [
                        {
                          "index": "salaries_from_date_to_date_index",
                          "usable": false,
                          "cause": "query_references_nonkey_column"
                        }
                      ] /* potential_skip_scan_indexes */
                    } /* skip_scan_range */,
                    "analyzing_range_alternatives": {
                      "range_scan_alternatives": [
                        {
                          "index": "salaries_from_date_to_date_index",
                          "ranges": [
                            "0xda840f <= from_date <= 0xda840f AND 0xda860f <= to_date <= 0xda860f"
                          ] /* ranges */,
                          "index_dives_for_eq_ranges": true,
                          "rowid_ordered": true,
                          "using_mrr": false,
                          "index_only": false,
                          "rows": 86,
                          "cost": 50.909,
                          "chosen": true
                        }
                      ] /* range_scan_alternatives */,
                      "analyzing_roworder_intersect": {
                        "usable": false,
                        "cause": "too_few_roworder_scans"
                      } /* analyzing_roworder_intersect */
                    } /* analyzing_range_alternatives */,
                    "chosen_range_access_summary": {
                      "range_access_plan": {
                        "type": "range_scan",
                        "index": "salaries_from_date_to_date_index",
                        "rows": 86,
                        "ranges": [
                          "0xda840f <= from_date <= 0xda840f AND 0xda860f <= to_date <= 0xda860f"
                        ] /* ranges */
                      } /* range_access_plan */,
                      "rows_for_plan": 86,
                      "cost_for_plan": 50.909,
                      "chosen": true
                    } /* chosen_range_access_summary */
                  } /* range_analysis */
                }
              ] /* rows_estimation */
            },
            {
              "considered_execution_plans": [
                {
                  "plan_prefix": [
                  ] /* plan_prefix */,
                  "table": "`salaries`",
                  "best_access_path": {
                    "considered_access_paths": [
                      {
                        "access_type": "ref",
                        "index": "salaries_from_date_to_date_index",
                        "rows": 86,
                        "cost": 50.412,
                        "chosen": true
                      },
                      {
                        "access_type": "range",
                        "range_details": {
                          "used_index": "salaries_from_date_to_date_index"
                        } /* range_details */,
                        "chosen": false,
                        "cause": "heuristic_index_cheaper"
                      }
                    ] /* considered_access_paths */
                  } /* best_access_path */,
                  "condition_filtering_pct": 100,
                  "rows_for_plan": 86,
                  "cost_for_plan": 50.412,
                  "chosen": true
                }
              ] /* considered_execution_plans */
            },
            {
              "attaching_conditions_to_tables": {
                "original_condition": "((`salaries`.`to_date` = DATE'1987-06-26') and (`salaries`.`from_date` = DATE'1986-06-26'))",
                "attached_conditions_computation": [
                ] /* attached_conditions_computation */,
                "attached_conditions_summary": [
                  {
                    "table": "`salaries`",
                    "attached": "((`salaries`.`to_date` = DATE'1987-06-26') and (`salaries`.`from_date` = DATE'1986-06-26'))"
                  }
                ] /* attached_conditions_summary */
              } /* attaching_conditions_to_tables */
            },
            {
              "finalizing_table_conditions": [
                {
                  "table": "`salaries`",
                  "original_table_condition": "((`salaries`.`to_date` = DATE'1987-06-26') and (`salaries`.`from_date` = DATE'1986-06-26'))",
                  "final_table_condition   ": null
                }
              ] /* finalizing_table_conditions */
            },
            {
              "refine_plan": [
                {
                  "table": "`salaries`"
                }
              ] /* refine_plan */
            }
          ] /* steps */
        } /* join_optimization */
      },
      {
        "join_execution": {
          "select#": 1,
          "steps": [
          ] /* steps */
        } /* join_execution */
      }
    ] /* steps */
  }
  MISSING_BYTES_BEYOND_MAX_MEM_SIZE: 0
            INSUFFICIENT_PRIVILEGES: 0
  1 row in set (0.00 sec)

由上面的結果可知,OPTIMIZER_TRACE有四個字段:

  • query:查詢語句
  • trace:query字段對應語句的跟蹤信息
  • missing_bytes_beyond_max_mem_size:跟蹤信息過長時,被截斷的跟蹤信息的字節數。
  • insufficient_privileges:執行跟蹤語句的用户是否有查看對象的權限。當不具有權限時,該列信息為1且trace字段為空,一般在調用帶有sql security definer的視圖或者是存儲過程的情況下,會出現此問題。

四、結果分析

最核心的是trace字段的內容。我們逐段分析:

4.1 join_preparation

join_preparation段落展示了準備階段的執行過程。

{
  "join_preparation": {
    "select#": 1,
    "steps": [
      {
        -- 對比下原始語句,可以知道,這一步做了個格式化。
        "expanded_query": "/* select#1 */ select `salaries`.`emp_no` AS `emp_no`,`salaries`.`salary` AS `salary`,`salaries`.`from_date` AS `from_date`,`salaries`.`to_date` AS `to_date` from `salaries` where ((`salaries`.`from_date` = '1986-06-26') and (`salaries`.`to_date` = '1987-06-26'))"
      }
    ]
    /* steps */
  }
  /* join_preparation */
}

4.2 join_optimization

join_optimization展示了優化階段的執行過程,是分析optimizer trace的重點。這段內容超級長,而且分了好多步驟,不妨按照步驟逐段分析:

4.2.1 condition_processing

該段用來做條件處理,主要對where條件進行優化處理。

"condition_processing": {
  "condition": "WHERE",
  "original_condition": "((`salaries`.`from_date` = '1986-06-26') and (`salaries`.`to_date` = '1987-06-26'))",
  "steps": [
    {
      "transformation": "equality_propagation",
      "resulting_condition": "(multiple equal('1986-06-26', `salaries`.`from_date`) and multiple equal('1987-06-26', `salaries`.`to_date`))"
    },
    {
      "transformation": "constant_propagation",
      "resulting_condition": "(multiple equal('1986-06-26', `salaries`.`from_date`) and multiple equal('1987-06-26', `salaries`.`to_date`))"
    },
    {
      "transformation": "trivial_condition_removal",
      "resulting_condition": "(multiple equal(DATE'1986-06-26', `salaries`.`from_date`) and multiple equal(DATE'1987-06-26', `salaries`.`to_date`))"
    }
  ] /* steps */
} /* condition_processing */

其中:

  • condition:優化對象類型。where條件句或者是having條件句
  • original_condition:優化前的原始語句
  • steps:主要包括三步,分別是quality_propagation(等值條件句轉換),constant_propagation(常量條件句轉換),trivial_condition_removal(無效條件移除的轉換)
    • transformation:轉換類型句
    • resulting_condition:轉換之後的結果輸出

4.2.2 substitute_generated_columns

substitute_generated_columns用於替換虛擬生成列

"substitute_generated_columns": {
} /* substitute_generated_columns */

4.2.3 table_dependencies

分析表之間的依賴關係

{
  "table_dependencies": [
    {
      "table": "`salaries`",
      "row_may_be_null": false,
      "map_bit": 0,
      "depends_on_map_bits": [
      ] /* depends_on_map_bits */
    }
  ] /* table_dependencies */
}

其中:

  • table:涉及的表名,如果有別名,也會展示出來
  • row_may_be_null:行是否可能為null,這裏是指join操作之後,這張表裏的數據是不是可能為null。如果語句中使用了left join,則後一張表的row_may_be_null會顯示為true
  • map_bit:表的映射編號,從0開始遞增
  • depends_on_map_bits:依賴的映射表。主要是當使用straight_join強行控制連接順序或者left join/right join有順序差別時,會在depends_on_map_bits中展示前置表的map_bit值。

4.2.4 ref_optimizer_key_uses

列出所有可用的ref類型的索引。如果使用了組合索引的多個部分(例如本例,用到了index(from_date, to_date) 的多列索引),則會在ref_optimizer_key_uses下列出多個元素,每個元素中會列出ref使用的索引及對應值。

{
  "ref_optimizer_key_uses": [
    {
      "table": "`salaries`",
      "field": "from_date",
      "equals": "DATE'1986-06-26'",
      "null_rejecting": false
    },
    {
      "table": "`salaries`",
      "field": "to_date",
      "equals": "DATE'1987-06-26'",
      "null_rejecting": false
    }
  ] /* ref_optimizer_key_uses */
}

4.2.5 rows_estimation

顧名思義,用於估算需要掃描的記錄數。

{
  "rows_estimation": [
    {
      "table": "`salaries`",
      "range_analysis": {
        "table_scan": {
          "rows": 2838216,
          "cost": 286799
        } /* table_scan */,
        "potential_range_indexes": [
          {
            "index": "PRIMARY",
            "usable": false,
            "cause": "not_applicable"
          },
          {
            "index": "salaries_from_date_to_date_index",
            "usable": true,
            "key_parts": [
              "from_date",
              "to_date",
              "emp_no"
            ] /* key_parts */
          }
        ] /* potential_range_indexes */,
        "setup_range_conditions": [
        ] /* setup_range_conditions */,
        "group_index_range": {
          "chosen": false,
          "cause": "not_group_by_or_distinct"
        } /* group_index_range */,
        "skip_scan_range": {
          "potential_skip_scan_indexes": [
            {
              "index": "salaries_from_date_to_date_index",
              "usable": false,
              "cause": "query_references_nonkey_column"
            }
          ] /* potential_skip_scan_indexes */
        } /* skip_scan_range */,
        "analyzing_range_alternatives": {
          "range_scan_alternatives": [
            {
              "index": "salaries_from_date_to_date_index",
              "ranges": [
                "0xda840f <= from_date <= 0xda840f AND 0xda860f <= to_date <= 0xda860f"
              ] /* ranges */,
              "index_dives_for_eq_ranges": true,
              "rowid_ordered": true,
              "using_mrr": false,
              "index_only": false,
              "rows": 86,
              "cost": 50.909,
              "chosen": true
            }
          ] /* range_scan_alternatives */,
          "analyzing_roworder_intersect": {
            "usable": false,
            "cause": "too_few_roworder_scans"
          } /* analyzing_roworder_intersect */
        } /* analyzing_range_alternatives */,
        "chosen_range_access_summary": {
          "range_access_plan": {
            "type": "range_scan",
            "index": "salaries_from_date_to_date_index",
            "rows": 86,
            "ranges": [
              "0xda840f <= from_date <= 0xda840f AND 0xda860f <= to_date <= 0xda860f"
            ] /* ranges */
          } /* range_access_plan */,
          "rows_for_plan": 86,
          "cost_for_plan": 50.909,
          "chosen": true
        } /* chosen_range_access_summary */
      } /* range_analysis */
    }
  ] /* rows_estimation */
}

其中:

  • table:表名
  • range_analysis:
    • table_scan:如果全表掃描的話,需要掃描多少行(row,2838216),以及需要的代價(cost,286799)
    • potential_range_indexes:列出表中所有的索引並分析其是否可用。如果不可用的話,會列出不可用的原因是什麼;如果可用會列出索引中可用的字段;
    • setup_range_conditions:如果有可下推的條件,則帶條件考慮範圍查詢
    • group_index_range:當使用了group by或distinct時,是否有合適的索引可用。當未使用group by或distinct時,會顯示chosen=false, cause=not_group_by_or_distinct;如使用了group by或distinct,但是多表查詢時,會顯示chosen=false,cause =not_single_table。其他情況下會嘗試分析可用的索引(potential_group_range_indexes)並計算對應的掃描行數及其所需代價
    • skip_scan_range:是否使用了skip scan
    • analyzing_range_alternatives:分析各個索引的使用成本
      • range_scan_alternatives:range掃描分析
        • index:索引名
        • ranges:range掃描的條件範圍
        • index_dives_for_eq_ranges:是否使用了index dive,該值會被參數eq_range_index_dive_limit變量值影響。
        • rowid_ordered:該range掃描的結果集是否根據pk值進行排序
        • using_mrr:是否使用了mrr
        • index_only:表示是否使用了覆蓋索引
        • rows:掃描的行數
        • cost:索引的使用成本
        • chosen:表示是否使用了該索引
      • analyzing_roworder_intersect:分析是否使用了索引合併(index merge),如果未使用,會在cause中展示原因;如果使用了索引合併,會在該部分展示索引合併的代價。
    • chosen_range_access_summary:在前一個步驟中分析了各類索引使用的方法及代價,得出了一定的中間結果之後,在summary階段彙總前一階段的中間結果確認最後的方案
      • range_access_plan:range掃描最終選擇的執行計劃。
        • type:展示執行計劃的type,如果使用了索引合併,則會顯示index_roworder_intersect
        • index:索引名
        • rows:掃描的行數
        • ranges:range掃描的條件範圍
      • rows_for_plan:該執行計劃的掃描行數
      • cost_for_plan:該執行計劃的執行代價
      • chosen:是否選擇該執行計劃

4.2.6 considered_execution_plans(重要)

負責對比各可行計劃的開銷,並選擇相對最優的執行計劃。

{
  "considered_execution_plans": [
    {
      "plan_prefix": [
      ] /* plan_prefix */,
      "table": "`salaries`",
      "best_access_path": {
        "considered_access_paths": [
          {
            "access_type": "ref",
            "index": "salaries_from_date_to_date_index",
            "rows": 86,
            "cost": 50.412,
            "chosen": true
          },
          {
            "access_type": "range",
            "range_details": {
              "used_index": "salaries_from_date_to_date_index"
            } /* range_details */,
            "chosen": false,
            "cause": "heuristic_index_cheaper"
          }
        ] /* considered_access_paths */
      } /* best_access_path */,
      "condition_filtering_pct": 100,
      "rows_for_plan": 86,
      "cost_for_plan": 50.412,
      "chosen": true
    }
  ] /* considered_execution_plans */
}

其中:

  • plan_prefix:當前計劃的前置執行計劃。
  • table:涉及的表名,如果有別名,也會展示出來
  • best_access_path:通過對比considered_access_paths,選擇一個最優的訪問路徑
    • considered_access_paths:當前考慮的訪問路徑
      • access_type:使用索引的方式,可參考explain中的type字段
      • index:索引
      • rows:行數
      • cost:開銷
  • chosen:是否選用這種執行路徑
  • condition_filtering_pct:類似於explain的filtered列,是一個估算值
  • rows_for_plan:執行計劃最終的掃描行數,由considered_access_paths.rows X condition_filtering_pct計算獲得。
  • cost_for_plan:執行計劃的代價,由considered_access_paths.cost相加獲得
  • chosen:是否選擇了該執行計劃

4.2.7 attaching_conditions_to_tables

基於considered_execution_plans中選擇的執行計劃,改造原有where條件,並針對表增加適當的附加條件,以便於單表數據的篩選。

TIPS

這部分條件的增加主要是為了便於ICP(索引條件下推),但ICP是否開啓並不影響這部分內容的構造。
ICP參考文檔:https://www.cnblogs.com/Terry-Wu/p/9273177.html

{
  "attaching_conditions_to_tables": {
    "original_condition": "((`salaries`.`to_date` = DATE'1987-06-26') and (`salaries`.`from_date` = DATE'1986-06-26'))",
    "attached_conditions_computation": [
    ] /* attached_conditions_computation */,
    "attached_conditions_summary": [
      {
        "table": "`salaries`",
        "attached": "((`salaries`.`to_date` = DATE'1987-06-26') and (`salaries`.`from_date` = DATE'1986-06-26'))"
      }
    ] /* attached_conditions_summary */
  } /* attaching_conditions_to_tables */
}

其中:

  • original_condition:原始的條件語句
  • attached_conditions_computation:使用啓發式算法計算已使用的索引,如果已使用的索引的訪問類型是ref,則計算用range能否使用組合索引中更多的列,如果可以,則用range的方式替換ref。
  • attached_conditions_summary:附加之後的情況彙總
    • table:表名
    • attached:附加的條件或原語句中能直接下推給單表篩選的條件。

4.2.8 finalizing_table_conditions

最終的、經過優化後的表條件。

{
  "finalizing_table_conditions": [
    {
      "table": "`salaries`",
      "original_table_condition": "((`salaries`.`to_date` = DATE'1987-06-26') and (`salaries`.`from_date` = DATE'1986-06-26'))",
      "final_table_condition   ": null
    }
  ] /* finalizing_table_conditions */
}

4.2.9 refine_plan

改善執行計劃:

{
    "refine_plan": [
    {
        "table": "`salaries`"
    }
    ] /* refine_plan */
}

其中:

  • table:表名及別名

4.3 join_execution

join_execution段落展示了執行階段的執行過程。

"join_execution": {
  "select#": 1,
  "steps": [
  ] /* steps */
}

五、總結

在整個optimizer_trace中我們重點其實就是在跟蹤記錄tracejson樹,我們通過這棵樹中的內容可以具體去分析優化器究竟做了什麼事情,進行了哪些選擇,是基於什麼原因做的選擇,選擇的結果及依據。這一系列都可以輔助驗證我們的一些觀點及優化,更好的幫助我們對我們的數據庫的實例進行調整。

Add a new 評論

Some HTML is okay.