tidb: update tidb_index_serial_scan_concurrency docs (#21938)#23264
Open
ti-chi-bot wants to merge 1 commit into
Open
tidb: update tidb_index_serial_scan_concurrency docs (#21938)#23264ti-chi-bot wants to merge 1 commit into
ti-chi-bot wants to merge 1 commit into
Conversation
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
12 tasks
Member
Author
|
@0xPoe This PR has conflicts, I have hold it. |
|
@ti-chi-bot: ## If you want to know how to resolve it, please read the guide in TiDB Dev Guide. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
@ti-chi-bot: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
Contributor
There was a problem hiding this comment.
Code Review
This pull request updates TiDB documentation, including best practices, statistics, and system variables, and introduces new TPC-C performance test reports for TiDB Cloud v8.1.0 and v8.5.0. The primary feedback is critical: multiple files contain unresolved git merge conflict markers that must be resolved before merging. Additionally, several minor style guide violations—such as document personification, unnecessary wordiness, and incorrect verb tenses—were identified and should be corrected in the newly added performance test reports and best practices documentation.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Comment on lines
+1247
to
+1262
| <<<<<<< HEAD | ||
| ======= | ||
| ### tidb_build_sampling_stats_concurrency <span class="version-mark">New in v7.5.0</span> | ||
|
|
||
| - Scope: SESSION | GLOBAL | ||
| - Persists to cluster: Yes | ||
| - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No | ||
| - Type: Integer | ||
| - Unit: Threads | ||
| - Default value: `2` | ||
| - Range: `[1, 256]` | ||
| - This variable controls the following aspects of `ANALYZE` concurrency: | ||
| - The concurrency for merging samples collected from different Regions. | ||
| - The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. | ||
|
|
||
| >>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) |
Contributor
There was a problem hiding this comment.
Unresolved git merge conflict markers found in this section. Please resolve the conflict by keeping the newly added system variable tidb_build_sampling_stats_concurrency.
Suggested change
| <<<<<<< HEAD | |
| ======= | |
| ### tidb_build_sampling_stats_concurrency <span class="version-mark">New in v7.5.0</span> | |
| - Scope: SESSION | GLOBAL | |
| - Persists to cluster: Yes | |
| - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No | |
| - Type: Integer | |
| - Unit: Threads | |
| - Default value: `2` | |
| - Range: `[1, 256]` | |
| - This variable controls the following aspects of `ANALYZE` concurrency: | |
| - The concurrency for merging samples collected from different Regions. | |
| - The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. | |
| >>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) | |
| ### tidb_build_sampling_stats_concurrency <span class="version-mark">New in v7.5.0</span> | |
| - Scope: SESSION | GLOBAL | |
| - Persists to cluster: Yes | |
| - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No | |
| - Type: Integer | |
| - Unit: Threads | |
| - Default value: `2` | |
| - Range: `[1, 256]` | |
| - This variable controls the following aspects of `ANALYZE` concurrency: | |
| - The concurrency for merging samples collected from different Regions. | |
| - The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. |
Comment on lines
+433
to
+450
| <<<<<<< HEAD | ||
| Currently, when you run the `ANALYZE` statement, the task is divided into multiple small tasks. Each task only works on one column or index. You can use the `tidb_build_stats_concurrency` parameter to control the number of simultaneous tasks. The default value is `4`. | ||
| ======= | ||
| This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions. | ||
|
|
||
| #### `tidb_distsql_scan_concurrency` | ||
| #### `tidb_build_sampling_stats_concurrency` | ||
|
|
||
| This variable controls the following aspects of `ANALYZE` concurrency: | ||
|
|
||
| - The concurrency for merging samples collected from different Regions. | ||
| - The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. | ||
|
|
||
| The default value is `2`. | ||
|
|
||
| #### `tidb_analyze_partition_concurrency` | ||
|
|
||
| This variable controls the concurrency for saving `ANALYZE` results (writing TopN and histograms to system tables). The default value is `2`. The default value is `1` for v7.4.0 and earlier versions. | ||
| >>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) |
Contributor
There was a problem hiding this comment.
Unresolved git merge conflict markers found in this section. Please resolve the conflict by keeping the updated documentation for ANALYZE concurrency variables and removing the conflict markers.
Suggested change
| <<<<<<< HEAD | |
| Currently, when you run the `ANALYZE` statement, the task is divided into multiple small tasks. Each task only works on one column or index. You can use the `tidb_build_stats_concurrency` parameter to control the number of simultaneous tasks. The default value is `4`. | |
| ======= | |
| This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions. | |
| #### `tidb_distsql_scan_concurrency` | |
| #### `tidb_build_sampling_stats_concurrency` | |
| This variable controls the following aspects of `ANALYZE` concurrency: | |
| - The concurrency for merging samples collected from different Regions. | |
| - The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. | |
| The default value is `2`. | |
| #### `tidb_analyze_partition_concurrency` | |
| This variable controls the concurrency for saving `ANALYZE` results (writing TopN and histograms to system tables). The default value is `2`. The default value is `1` for v7.4.0 and earlier versions. | |
| >>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) | |
| This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions. | |
| #### `tidb_build_sampling_stats_concurrency` | |
| This variable controls the following aspects of `ANALYZE` concurrency: | |
| - The concurrency for merging samples collected from different Regions. | |
| - The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. | |
| The default value is `2`. | |
| #### `tidb_analyze_partition_concurrency` | |
| This variable controls the concurrency for saving `ANALYZE` results (writing TopN and histograms to system tables). The default value is `2`. The default value is `1` for v7.4.0 and earlier versions. |
Comment on lines
+454
to
465
| <<<<<<< HEAD | ||
| When you analyze regular columns, you can use the `tidb_distsql_scan_concurrency` parameter to control the number of Region to be read at one time. The default value is `15`. | ||
| ======= | ||
| This variable controls the following aspects of `ANALYZE` concurrency: | ||
| >>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) | ||
|
|
||
| #### `tidb_index_serial_scan_concurrency` | ||
| - The concurrency of scanning TiKV Regions. | ||
| - The concurrency of scanning Regions for special indexes (indexes generated from virtual columns). | ||
|
|
||
| <<<<<<< HEAD | ||
| When you analyze index columns, you can use the `tidb_index_serial_scan_concurrency` parameter to control the number of Region to be read at one time. The default value is `1`. | ||
|
|
Contributor
There was a problem hiding this comment.
Unresolved git merge conflict markers found in this section. Please resolve the conflict by keeping the updated tidb_analyze_distsql_scan_concurrency variable description and removing the deprecated tidb_index_serial_scan_concurrency section.
Suggested change
| <<<<<<< HEAD | |
| When you analyze regular columns, you can use the `tidb_distsql_scan_concurrency` parameter to control the number of Region to be read at one time. The default value is `15`. | |
| ======= | |
| This variable controls the following aspects of `ANALYZE` concurrency: | |
| >>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) | |
| #### `tidb_index_serial_scan_concurrency` | |
| - The concurrency of scanning TiKV Regions. | |
| - The concurrency of scanning Regions for special indexes (indexes generated from virtual columns). | |
| <<<<<<< HEAD | |
| When you analyze index columns, you can use the `tidb_index_serial_scan_concurrency` parameter to control the number of Region to be read at one time. The default value is `1`. | |
| This variable controls the following aspects of `ANALYZE` concurrency: | |
| - The concurrency of scanning TiKV Regions. | |
| - The concurrency of scanning Regions for special indexes (indexes generated from virtual columns). | |
Comment on lines
+923
to
+925
| ======= | ||
| The default value is `4`. | ||
| >>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) |
Contributor
There was a problem hiding this comment.
Unresolved git merge conflict markers found at the end of the file. Please remove these conflict markers and the misplaced default value description.
Comment on lines
+961
to
+996
| <<<<<<< HEAD | ||
| ======= | ||
| ### tidb_analyze_column_options <span class="version-mark">New in v8.3.0</span> | ||
|
|
||
| > **Note:** | ||
| > | ||
| > - This variable only works when [`tidb_analyze_version`](#tidb_analyze_version-new-in-v510) is set to `2`. | ||
| > - If you upgrade your TiDB cluster from a version earlier than v8.3.0 to v8.3.0 or later, this variable is set to `ALL` by default to keep the original behavior. | ||
| > - For a newly deployed TiDB cluster from v8.3.0 to v8.5.4, this variable is set to `PREDICATE` by default. | ||
| > - For a newly deployed TiDB cluster from v8.5.5 and v9.0.0 onwards, this variable is set to `ALL` by default. | ||
|
|
||
| - Scope: GLOBAL | ||
| - Persists to cluster: Yes | ||
| - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No | ||
| - Type: Enumeration | ||
| - Default value: `ALL` | ||
| - Value options:`ALL`, `PREDICATE` | ||
| - This variable controls the behavior of the `ANALYZE TABLE` statement. Setting it to `PREDICATE` means only collecting statistics for [predicate columns](/statistics.md#collect-statistics-on-some-columns); setting it to `ALL` means collecting statistics for all columns. In scenarios where OLAP queries are used, it is recommended to set it to `ALL`, otherwise collecting statistics can result in a significant drop in query performance. | ||
|
|
||
| ### tidb_analyze_distsql_scan_concurrency <span class="version-mark">New in v7.6.0</span> | ||
|
|
||
| > **Note:** | ||
| > | ||
| > In versions earlier than v7.6.0, regular `ANALYZE` region scans are controlled by `tidb_distsql_scan_concurrency`, and index statistics scans are controlled by `tidb_index_serial_scan_concurrency`. Therefore, to adjust the concurrency of scanning TiKV Regions for these versions, consider changing the value of `tidb_distsql_scan_concurrency`. | ||
|
|
||
| - Scope: SESSION | GLOBAL | ||
| - Persists to cluster: Yes | ||
| - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No | ||
| - Type: Integer | ||
| - Default value: `4` | ||
| - Range: `[0, 4294967295]`. In versions earlier than v8.2.0, the minimum value is `1`. When you set it to `0`, it adaptively adjusts the concurrency based on the cluster size. | ||
| - This variable controls the following aspects of `ANALYZE` concurrency: | ||
| - The concurrency of scanning TiKV Regions. | ||
| - The concurrency of scanning Regions for special indexes, such as indexes on generated virtual columns. | ||
|
|
||
| >>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) |
Contributor
There was a problem hiding this comment.
Unresolved git merge conflict markers found in this section. Please resolve the conflict by keeping the newly added system variables tidb_analyze_column_options and tidb_analyze_distsql_scan_concurrency.
### tidb_analyze_column_options <span class="version-mark">New in v8.3.0</span>
> **Note:**
>
> - This variable only works when [`tidb_analyze_version`](#tidb_analyze_version-new-in-v510) is set to `2`.
> - If you upgrade your TiDB cluster from a version earlier than v8.3.0 to v8.3.0 or later, this variable is set to `ALL` by default to keep the original behavior.
> - For a newly deployed TiDB cluster from v8.3.0 to v8.5.4, this variable is set to `PREDICATE` by default.
> - For a newly deployed TiDB cluster from v8.5.5 and v9.0.0 onwards, this variable is set to `ALL` by default.
- Scope: GLOBAL
- Persists to cluster: Yes
- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No
- Type: Enumeration
- Default value: `ALL`
- Value options:`ALL`, `PREDICATE`
- This variable controls the behavior of the `ANALYZE TABLE` statement. Setting it to `PREDICATE` means only collecting statistics for [predicate columns](/statistics.md#collect-statistics-on-some-columns); setting it to `ALL` means collecting statistics for all columns. In scenarios where OLAP queries are used, it is recommended to set it to `ALL`, otherwise collecting statistics can result in a significant drop in query performance.
### tidb_analyze_distsql_scan_concurrency <span class="version-mark">New in v7.6.0</span>
> **Note:**
>
> In versions earlier than v7.6.0, regular `ANALYZE` region scans are controlled by `tidb_distsql_scan_concurrency`, and index statistics scans are controlled by `tidb_index_serial_scan_concurrency`. Therefore, to adjust the concurrency of scanning TiKV Regions for these versions, consider changing the value of `tidb_distsql_scan_concurrency`.
- Scope: SESSION | GLOBAL
- Persists to cluster: Yes
- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No
- Type: Integer
- Default value: `4`
- Range: `[0, 4294967295]`. In versions earlier than v8.2.0, the minimum value is `1`. When you set it to `0`, it adaptively adjusts the concurrency based on the cluster size.
- This variable controls the following aspects of `ANALYZE` concurrency:
- The concurrency of scanning TiKV Regions.
- The concurrency of scanning Regions for special indexes, such as indexes on generated virtual columns.| > | ||
| > For TiDB Cloud, to modify the TiKV parameters of your cluster, you can contact [PingCAP Support](/tidb-cloud/tidb-cloud-support.md) for help. | ||
|
|
||
| The TiKV parameter [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) can make log recycling effective immediately after initialization. This document conducts tests with `prefill-for-recycle` enabled: |
Contributor
There was a problem hiding this comment.
Avoid personification of documents ("This document conducts tests"). Rephrase to use active voice or passive voice appropriately (e.g., "The tests in this document are conducted...").
Suggested change
| The TiKV parameter [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) can make log recycling effective immediately after initialization. This document conducts tests with `prefill-for-recycle` enabled: | |
| The TiKV parameter [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) can make log recycling effective immediately after initialization. The tests in this document are conducted with `prefill-for-recycle` enabled: |
|
|
||
| ## Test steps | ||
|
|
||
| This section introduces how to perform the TPC-C performance test step by step. |
Contributor
There was a problem hiding this comment.
Use "describes" instead of "introduces" for a more professional and precise technical writing tone.
Suggested change
| This section introduces how to perform the TPC-C performance test step by step. | |
| This section describes how to perform the TPC-C performance test step by step. |
| curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/pingcap/go-tpc/master/install.sh | sh | ||
| ``` | ||
|
|
||
| 2. Run the following `go-tpc tpcc` command to import 1,000 warehouses to the `tpcc` database. Replace `${HOST}`, `${THREAD}`, and `${PASSWORD}` with your actual values. This document conducts three tests with the `${THREAD}` value of `50`, `100`, and `200`. |
Contributor
There was a problem hiding this comment.
Avoid personification of documents ("This document conducts three tests"). Rephrase to state that the tests are conducted with three different thread counts.
Suggested change
| 2. Run the following `go-tpc tpcc` command to import 1,000 warehouses to the `tpcc` database. Replace `${HOST}`, `${THREAD}`, and `${PASSWORD}` with your actual values. This document conducts three tests with the `${THREAD}` value of `50`, `100`, and `200`. | |
| 2. Run the following `go-tpc tpcc` command to import 1,000 warehouses to the `tpcc` database. Replace `${HOST}`, `${THREAD}`, and `${PASSWORD}` with your actual values. The tests are conducted with three different thread counts: `50`, `100`, and `200`. |
| ANALYZE TABLE warehouse; | ||
| ``` | ||
|
|
||
| To accelerate the collection of statistics, execute the following SQL statements before collecting: |
Contributor
There was a problem hiding this comment.
Simplify the sentence and use active voice. "before collecting" can be simplified to "before running ANALYZE:".
Suggested change
| To accelerate the collection of statistics, execute the following SQL statements before collecting: | |
| To accelerate statistics collection, run the following SQL statements before running `ANALYZE`: |
References
- Avoid unnecessary words and repetition. (link)
|
|
||
| 6. Extract the tpmC data of `NEW_ORDER` from the result. | ||
|
|
||
| TPC-C uses tpmC (transactions per minute) to measure the maximum qualified throughput (MQTh, Max Qualified Throughput). The transactions are the NewOrder transactions and the final unit of measure is the number of new orders processed per minute. |
Contributor
There was a problem hiding this comment.
Improve the flow and grammar of the sentence. "The transactions are the NewOrder transactions" can be rephrased to "These transactions are NewOrder transactions".
Suggested change
| TPC-C uses tpmC (transactions per minute) to measure the maximum qualified throughput (MQTh, Max Qualified Throughput). The transactions are the NewOrder transactions and the final unit of measure is the number of new orders processed per minute. | |
| TPC-C uses tpmC (transactions per minute) to measure the maximum qualified throughput (MQTh, Max Qualified Throughput). These transactions are NewOrder transactions, and the final unit of measure is the number of new orders processed per minute. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is an automated cherry-pick of #21938
First-time contributors' checklist
What is changed, added or deleted? (Required)
See: pingcap/tidb#64023
Which TiDB version(s) do your changes apply to? (Required)
Tips for choosing the affected version(s):
By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.
For details, see tips for choosing the affected versions.
What is the related PR or file link(s)?
Do your changes match any of the following descriptions?