From 874dd8dbfc290a809de5dd8fde581275a4681adc Mon Sep 17 00:00:00 2001 From: Dongpo Liu Date: Tue, 14 Jul 2026 12:21:14 +0200 Subject: [PATCH 1/7] This is an automated cherry-pick of #21938 Signed-off-by: ti-chi-bot --- best-practices/tidb-best-practices.md | 2 +- statistics.md | 30 +++- system-variables.md | 146 +++++++++++++++++- ...v8.1-performance-benchmarking-with-tpcc.md | 123 +++++++++++++++ ...v8.5-performance-benchmarking-with-tpcc.md | 123 +++++++++++++++ troubleshoot-tidb-oom.md | 3 +- 6 files changed, 417 insertions(+), 10 deletions(-) create mode 100644 tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md create mode 100644 tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md diff --git a/best-practices/tidb-best-practices.md b/best-practices/tidb-best-practices.md index 88be54844279e..f783df2c72200 100644 --- a/best-practices/tidb-best-practices.md +++ b/best-practices/tidb-best-practices.md @@ -129,7 +129,7 @@ Lots of MySQL experience is also applicable to TiDB. It is noted that TiDB has i * Ensure the order of results through indexes - You can use indexes to filter or sort data. Firstly, get row IDs according to the index order. Then, return the row content according to the return order of row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but it can be modified through the [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) variable. + You can use indexes to filter or sort data. TiDB first gets row IDs according to the index order, and then returns the row content according to the order of those row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but you can increase it by adjusting [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50). * Reverse index scan diff --git a/statistics.md b/statistics.md index a7aa8f570e02d..bfa9b08c061fb 100644 --- a/statistics.md +++ b/statistics.md @@ -430,14 +430,37 @@ When you run the `ANALYZE` statement, you can adjust the concurrency using the f #### `tidb_build_stats_concurrency` +<<<<<<< HEAD Currently, when you run the `ANALYZE` statement, the task is divided into multiple small tasks. Each task only works on one column or index. You can use the `tidb_build_stats_concurrency` parameter to control the number of simultaneous tasks. The default value is `4`. +======= +This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions. -#### `tidb_distsql_scan_concurrency` +#### `tidb_build_sampling_stats_concurrency` +This variable controls the following aspects of `ANALYZE` concurrency: + +- The concurrency for merging samples collected from different Regions. +- The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. + +The default value is `2`. + +#### `tidb_analyze_partition_concurrency` + +This variable controls the concurrency for saving `ANALYZE` results (writing TopN and histograms to system tables). The default value is `2`. The default value is `1` for v7.4.0 and earlier versions. +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) + +#### `tidb_analyze_distsql_scan_concurrency` + +<<<<<<< HEAD When you analyze regular columns, you can use the `tidb_distsql_scan_concurrency` parameter to control the number of Region to be read at one time. The default value is `15`. +======= +This variable controls the following aspects of `ANALYZE` concurrency: +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) -#### `tidb_index_serial_scan_concurrency` +- The concurrency of scanning TiKV Regions. +- The concurrency of scanning Regions for special indexes (indexes generated from virtual columns). +<<<<<<< HEAD When you analyze index columns, you can use the `tidb_index_serial_scan_concurrency` parameter to control the number of Region to be read at one time. The default value is `1`. ### Persist ANALYZE configurations @@ -897,6 +920,9 @@ mysql> show warnings; +-------+------+-----------------------------------------------------------------+ 1 row in set (0.00 sec) ``` +======= +The default value is `4`. +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ## See also diff --git a/system-variables.md b/system-variables.md index d884ce66d3623..c45192d29bd85 100644 --- a/system-variables.md +++ b/system-variables.md @@ -958,6 +958,42 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Default value: `OFF` - This variable is used to set whether the `AUTO_INCREMENT` property of a column is allowed to be removed by executing `ALTER TABLE MODIFY` or `ALTER TABLE CHANGE` statements. It is not allowed by default. +<<<<<<< HEAD +======= +### tidb_analyze_column_options New in v8.3.0 + +> **Note:** +> +> - This variable only works when [`tidb_analyze_version`](#tidb_analyze_version-new-in-v510) is set to `2`. +> - If you upgrade your TiDB cluster from a version earlier than v8.3.0 to v8.3.0 or later, this variable is set to `ALL` by default to keep the original behavior. +> - For a newly deployed TiDB cluster from v8.3.0 to v8.5.4, this variable is set to `PREDICATE` by default. +> - For a newly deployed TiDB cluster from v8.5.5 and v9.0.0 onwards, this variable is set to `ALL` by default. + +- Scope: GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Type: Enumeration +- Default value: `ALL` +- Value options:`ALL`, `PREDICATE` +- This variable controls the behavior of the `ANALYZE TABLE` statement. Setting it to `PREDICATE` means only collecting statistics for [predicate columns](/statistics.md#collect-statistics-on-some-columns); setting it to `ALL` means collecting statistics for all columns. In scenarios where OLAP queries are used, it is recommended to set it to `ALL`, otherwise collecting statistics can result in a significant drop in query performance. + +### tidb_analyze_distsql_scan_concurrency New in v7.6.0 + +> **Note:** +> +> In versions earlier than v7.6.0, regular `ANALYZE` region scans are controlled by `tidb_distsql_scan_concurrency`, and index statistics scans are controlled by `tidb_index_serial_scan_concurrency`. Therefore, to adjust the concurrency of scanning TiKV Regions for these versions, consider changing the value of `tidb_distsql_scan_concurrency`. + +- Scope: SESSION | GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Type: Integer +- Default value: `4` +- Range: `[0, 4294967295]`. In versions earlier than v8.2.0, the minimum value is `1`. When you set it to `0`, it adaptively adjusts the concurrency based on the cluster size. +- This variable controls the following aspects of `ANALYZE` concurrency: + - The concurrency of scanning TiKV Regions. + - The concurrency of scanning Regions for special indexes, such as indexes on generated virtual columns. + +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_analyze_partition_concurrency > **Warning:** @@ -966,8 +1002,15 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Scope: SESSION | GLOBAL - Persists to cluster: Yes +<<<<<<< HEAD - Default value: `1` - This variable specifies the concurrency of reading and writing statistics for a partitioned table when TiDB analyzes the partitioned table. +======= +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Default value: `2`. The default value is `1` for v7.4.0 and earlier versions. +- Range: `[1, 128]`. Before v8.4.0, the value range is `[1, 18446744073709551615]`. +- For manual and auto `ANALYZE`, this variable controls the concurrency for saving `ANALYZE` results, including writing TopN and histograms to system tables. +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_analyze_version New in v5.1.0 @@ -982,6 +1025,76 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - If your cluster is upgraded from an earlier version, the default value of `tidb_analyze_version` does not change after the upgrade. - For detailed introduction about this variable, see [Introduction to Statistics](/statistics.md). +<<<<<<< HEAD +======= +### tidb_analyze_skip_column_types New in v7.2.0 + +- Scope: SESSION | GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Default value: "json,blob,mediumblob,longblob,mediumtext,longtext". Before v8.2.0, the default value is "json,blob,mediumblob,longblob". +- Possible values: "json,blob,mediumblob,longblob,text,mediumtext,longtext" +- This variable controls which types of columns are skipped for statistics collection when executing the `ANALYZE` command to collect statistics. The variable is only applicable for `tidb_analyze_version = 2`. Even if you specify a column using `ANALYZE TABLE t COLUMNS c1, ... , cn`, no statistics will be collected for the specified column if its type is in `tidb_analyze_skip_column_types`. + +``` +mysql> SHOW CREATE TABLE t; ++-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Table | Create Table | ++-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| t | CREATE TABLE `t` ( + `a` int DEFAULT NULL, + `b` varchar(10) DEFAULT NULL, + `c` json DEFAULT NULL, + `d` blob DEFAULT NULL, + `e` longblob DEFAULT NULL +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin | ++-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +1 row in set (0.00 sec) + +mysql> SELECT @@tidb_analyze_skip_column_types; ++----------------------------------+ +| @@tidb_analyze_skip_column_types | ++----------------------------------+ +| json,blob,mediumblob,longblob,mediumtext,longtext | ++----------------------------------+ +1 row in set (0.00 sec) + +mysql> ANALYZE TABLE t; +Query OK, 0 rows affected, 1 warning (0.05 sec) + +mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; ++---------------------------------------------------------------------+ +| job_info | ++---------------------------------------------------------------------+ +| analyze table columns a, b with 256 buckets, 500 topn, 1 samplerate | ++---------------------------------------------------------------------+ +1 row in set (0.00 sec) + +mysql> ANALYZE TABLE t COLUMNS a, c; +Query OK, 0 rows affected, 1 warning (0.04 sec) + +mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; ++------------------------------------------------------------------+ +| job_info | ++------------------------------------------------------------------+ +| analyze table columns a with 256 buckets, 500 topn, 1 samplerate | ++------------------------------------------------------------------+ +1 row in set (0.00 sec) +``` + +### tidb_auto_analyze_concurrency New in v8.4.0 + +- Scope: GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Type: Integer +- Default value: `3` +- Range: `[1, 2147483647]` +- This variable controls the number of concurrent auto-analyze operations that can run in a TiDB cluster. To accelerate statistics collection tasks, you can increase this concurrency based on the available resources in your cluster. +- Before v8.4.0, this concurrency is fixed at `1`. +- Starting from v8.5.7 and v9.0.0, the default value changes from `1` to `3`. If your cluster is upgraded from an earlier version, the value of this variable remains unchanged after the upgrade. + +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_auto_analyze_end_time - Scope: GLOBAL @@ -1028,7 +1141,12 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Type: Integer - Default value: `1` - Range: `[1, 256]` +<<<<<<< HEAD - This variable is used to set the concurrency of executing the automatic update of statistics. +======= +- This variable controls the concurrency for building statistics during auto `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously +- Starting from v8.5.7 and v9.0.0, the default value of this variable changes from `1` to `2`. If your cluster is upgraded from an earlier version, the value of this variable remains unchanged after the upgrade. +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_backoff_lock_fast @@ -1124,9 +1242,24 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Default value: `4` - Range: `[1, 256]` - Unit: Threads -- This variable is used to set the concurrency of executing the `ANALYZE` statement. -- When the variable is set to a larger value, the execution performance of other queries is affected. +- This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. + +<<<<<<< HEAD +======= +### tidb_build_sampling_stats_concurrency New in v7.5.0 +- Scope: SESSION | GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Type: Integer +- Unit: Threads +- Default value: `2` +- Range: `[1, 256]` +- This variable controls the following aspects of `ANALYZE` concurrency: + - The concurrency for merging samples collected from different Regions. + - The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. + +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_capture_plan_baselines New in v4.0 - Scope: GLOBAL @@ -2791,14 +2924,17 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified ### tidb_index_serial_scan_concurrency +> **Warning:** +> +> This variable is deprecated and no longer controls execution behavior. The concurrency of sequential index scans is now controlled by [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50), and [`ANALYZE TABLE`](/sql-statements/sql-statement-analyze-table.md) uses [`tidb_analyze_distsql_scan_concurrency`](#tidb_analyze_distsql_scan_concurrency-new-in-v760) to control the concurrency of index statistics scans. + - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Type: Integer - Default value: `1` - Range: `[1, 256]` - Unit: Threads -- This variable is used to set the concurrency of the `serial scan` operation. -- Use a bigger value in OLAP scenarios, and a smaller value in OLTP scenarios. +- This variable remains only for backward compatibility. Use [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50) to control the concurrency of sequential index scans, or [`tidb_analyze_distsql_scan_concurrency`](#tidb_analyze_distsql_scan_concurrency-new-in-v760) to control the concurrency of index statistics scans. ### tidb_init_chunk_size @@ -3225,7 +3361,7 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Default value: `1` -- This variable specifies the concurrency of merging statistics for a partitioned table when TiDB analyzes the partitioned table. +- This variable controls the concurrency for merging TopN results of partitioned tables. ### tidb_metric_query_range_duration New in v4.0 diff --git a/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md b/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md new file mode 100644 index 0000000000000..79e74aec086dd --- /dev/null +++ b/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md @@ -0,0 +1,123 @@ +--- +title: TiDB Cloud TPC-C Performance Test Report for TiDB v8.1.0 +summary: Introduce the TPC-C performance test results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.1.0. +--- + +# TiDB Cloud TPC-C Performance Test Report for TiDB v8.1.0 + +This document provides the TPC-C performance test steps and results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.1.0. This report can also be used as a reference for the performance of TiDB Self-Managed v8.1.0 clusters. + +## Test overview + +This test aims at showing the TPC-C performance of TiDB v8.1.0 in the Online Transactional Processing (OLTP) scenario. + +## Test environment + +### TiDB cluster + +The test is conducted on a TiDB cluster with the following settings: + +- Cluster type: [TiDB Cloud Dedicated](/tidb-cloud/select-cluster-tier.md#tidb-cloud-dedicated) +- Cluster version: v8.1.0 +- Cloud provider: AWS (us-west-2) +- Cluster configuration: + + | Node type | Node size | Node quantity | Node storage | + |:----------|:----------|:----------|:----------| + | TiDB | 16 vCPU, 32 GiB | 2 | N/A | + | TiKV | 16 vCPU, 64 GiB | 3 | 1000 GiB | + +### Parameter configuration + +> **Note:** +> +> For TiDB Cloud, to modify the TiKV parameters of your cluster, you can contact [PingCAP Support](/tidb-cloud/tidb-cloud-support.md) for help. + +The TiKV parameter [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) can make log recycling effective immediately after initialization. This document conducts tests with `prefill-for-recycle` enabled: + +```yaml +raft-engine.prefill-for-recycle = true +``` + +### Benchmark executor + +The benchmark executor sends SQL queries to the TiDB cluster. In this test, its hardware configuration is as follows: + +- Machine type: Amazon EC2 (us-west-2) +- Instance type: c6a.2xlarge + +## Test steps + +This section introduces how to perform the TPC-C performance test step by step. + +1. In the [TiDB Cloud console](https://tidbcloud.com/), create a TiDB Cloud Dedicated cluster that meets the [test environment](#tidb-cluster) requirements. + + For more information, see [Create a TiDB Cloud Dedicated cluster](/tidb-cloud/create-tidb-cluster.md). + +2. On the benchmark executor, connect to the newly created cluster and create a database named `tpcc`. + + To connect to the cluster, see [Connect to TiDB Cloud Dedicated via Private Endpoint](/tidb-cloud/set-up-private-endpoint-connections.md). + + To create the `tpcc` database, execute the following SQL statement: + + ```sql + CREATE DATABASE tpcc; + ``` + +3. Load TPC-C data to the `tpcc` database. + + 1. The test in this document is implemented based on [go-tpc](https://github.com/pingcap/go-tpc). You can download the test program using the following command: + + ```shell + curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/pingcap/go-tpc/master/install.sh | sh + ``` + + 2. Run the following `go-tpc tpcc` command to import 1,000 warehouses to the `tpcc` database. Replace `${HOST}`, `${THREAD}`, and `${PASSWORD}` with your actual values. This document conducts three tests with the `${THREAD}` value of `50`, `100`, and `200`. + + ```shell + go-tpc tpcc --host ${HOST} --warehouses 1000 prepare -P 4000 -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error + ``` + +4. To ensure that the TiDB optimizer can generate the optimal execution plan, execute the following SQL statements to collect statistics before conducting the TPC-C test: + + ```sql + ANALYZE TABLE customer; + ANALYZE TABLE district; + ANALYZE TABLE history; + ANALYZE TABLE item; + ANALYZE TABLE new_order; + ANALYZE TABLE order_line; + ANALYZE TABLE orders; + ANALYZE TABLE stock; + ANALYZE TABLE warehouse; + ``` + + To accelerate the collection of statistics, execute the following SQL statements before collecting: + + ```sql + SET tidb_build_stats_concurrency=16; + SET tidb_build_sampling_stats_concurrency = 16; + SET tidb_analyze_distsql_scan_concurrency=16; + ``` + +5. Run the following `go-tpc tpcc` command to conduct stress tests on the TiDB Cloud Dedicated cluster. For each concurrency, the test takes two hours. + + ```shell + go-tpc tpcc --host ${HOST} -P 4000 --warehouses 1000 run -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error + ``` + +6. Extract the tpmC data of `NEW_ORDER` from the result. + + TPC-C uses tpmC (transactions per minute) to measure the maximum qualified throughput (MQTh, Max Qualified Throughput). The transactions are the NewOrder transactions and the final unit of measure is the number of new orders processed per minute. + +## Test results + +The TPC-C performance of v8.1.0 in the [test environment](#test-environment) is as follows: + +| Threads | v8.1.0 tpmC | +|:--------|:----------| +| 50 | 43,660 | +| 100 | 75,495 | +| 200 | 102,013 | + +![TPC-C](/media/tidb-cloud/v8.1.0_tpcc.png) diff --git a/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md b/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md new file mode 100644 index 0000000000000..3bd5bbd2a576a --- /dev/null +++ b/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md @@ -0,0 +1,123 @@ +--- +title: TiDB Cloud TPC-C Performance Test Report for TiDB v8.5.0 +summary: Introduce the TPC-C performance test results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.5.0. +--- + +# TiDB Cloud TPC-C Performance Test Report for TiDB v8.5.0 + +This document provides the TPC-C performance test steps and results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.5.0. This report can also be used as a reference for the performance of TiDB Self-Managed v8.5.0 clusters. + +## Test overview + +This test aims at showing the TPC-C performance of TiDB v8.5.0 in the Online Transactional Processing (OLTP) scenario. + +## Test environment + +### TiDB cluster + +The test is conducted on a TiDB cluster with the following settings: + +- Cluster type: [TiDB Cloud Dedicated](/tidb-cloud/select-cluster-tier.md#tidb-cloud-dedicated) +- Cluster version: v8.5.0 +- Cloud provider: AWS (us-west-2) +- Cluster configuration: + + | Node type | Node size | Node quantity | Node storage | + |:----------|:----------|:----------|:----------| + | TiDB | 16 vCPU, 32 GiB | 2 | N/A | + | TiKV | 16 vCPU, 64 GiB | 3 | 1000 GiB | + +### Parameter configuration + +> **Note:** +> +> For TiDB Cloud, to modify the TiKV parameters of your cluster, you can contact [PingCAP Support](/tidb-cloud/tidb-cloud-support.md) for help. + +The TiKV parameter [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) can make log recycling effective immediately after initialization. This document conducts tests with `prefill-for-recycle` enabled: + +```yaml +raft-engine.prefill-for-recycle = true +``` + +### Benchmark executor + +The benchmark executor sends SQL queries to the TiDB cluster. In this test, its hardware configuration is as follows: + +- Machine type: Amazon EC2 (us-west-2) +- Instance type: c6a.2xlarge + +## Test steps + +This section introduces how to perform the TPC-C performance test step by step. + +1. In the [TiDB Cloud console](https://tidbcloud.com/), create a TiDB Cloud Dedicated cluster that meets the [test environment](#tidb-cluster) requirements. + + For more information, see [Create a TiDB Cloud Dedicated cluster](/tidb-cloud/create-tidb-cluster.md). + +2. On the benchmark executor, connect to the newly created cluster and create a database named `tpcc`. + + To connect to the cluster, see [Connect to TiDB Cloud Dedicated via Private Endpoint](/tidb-cloud/set-up-private-endpoint-connections.md). + + To create the `tpcc` database, execute the following SQL statement: + + ```sql + CREATE DATABASE tpcc; + ``` + +3. Load TPC-C data to the `tpcc` database. + + 1. The test in this document is implemented based on [go-tpc](https://github.com/pingcap/go-tpc). You can download the test program using the following command: + + ```shell + curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/pingcap/go-tpc/master/install.sh | sh + ``` + + 2. Run the following `go-tpc tpcc` command to import 1,000 warehouses to the `tpcc` database. Replace `${HOST}`, `${THREAD}`, and `${PASSWORD}` with your actual values. This document conducts three tests with the `${THREAD}` value of `50`, `100`, and `200`. + + ```shell + go-tpc tpcc --host ${HOST} --warehouses 1000 prepare -P 4000 -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error + ``` + +4. To ensure that the TiDB optimizer can generate the optimal execution plan, execute the following SQL statements to collect statistics before conducting the TPC-C test: + + ```sql + ANALYZE TABLE customer; + ANALYZE TABLE district; + ANALYZE TABLE history; + ANALYZE TABLE item; + ANALYZE TABLE new_order; + ANALYZE TABLE order_line; + ANALYZE TABLE orders; + ANALYZE TABLE stock; + ANALYZE TABLE warehouse; + ``` + + To accelerate the collection of statistics, execute the following SQL statements before collecting: + + ```sql + SET tidb_build_stats_concurrency=16; + SET tidb_build_sampling_stats_concurrency = 16; + SET tidb_analyze_distsql_scan_concurrency=16; + ``` + +5. Run the following `go-tpc tpcc` command to conduct stress tests on the TiDB Cloud Dedicated cluster. For each concurrency, the test takes two hours. + + ```shell + go-tpc tpcc --host ${HOST} -P 4000 --warehouses 1000 run -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error + ``` + +6. Extract the tpmC data of `NEW_ORDER` from the result. + + TPC-C uses tpmC (transactions per minute) to measure the maximum qualified throughput (MQTh, Max Qualified Throughput). The transactions are the NewOrder transactions and the final unit of measure is the number of new orders processed per minute. + +## Test results + +The TPC-C performance of v8.5.0 in the [test environment](#test-environment) is as follows: + +| Threads | v8.5.0 tpmC | +|:--------|:----------| +| 50 | 43,146 | +| 100 | 73,875 | +| 200 | 103,395 | + +![TPC-C](/media/tidb-cloud/v8.5.0_tpcc.png) diff --git a/troubleshoot-tidb-oom.md b/troubleshoot-tidb-oom.md index 1c2f102cd0681..c888f278f5e63 100644 --- a/troubleshoot-tidb-oom.md +++ b/troubleshoot-tidb-oom.md @@ -99,7 +99,6 @@ You can take the following measures to reduce the memory usage of SQL statements - Reduce the number of Regions to be read simultaneously or reduce the concurrency of operators to avoid memory problems caused by high concurrency. The corresponding system variables include: - [`tidb_distsql_scan_concurrency`](/system-variables.md#tidb_distsql_scan_concurrency) - - [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) - [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50) - The concurrency of sessions is too high near the time point when the problem occurs. In this case, consider scaling out the TiDB cluster by adding more TiDB nodes. @@ -195,4 +194,4 @@ To locate the root cause of an OOM issue, you need to collect the following info ## See also - [TiDB Memory Control](/configure-memory-usage.md) -- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) \ No newline at end of file +- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) From f641277c3be0b85c711cce1443f1aa5eb35ea0c3 Mon Sep 17 00:00:00 2001 From: qiancai Date: Wed, 15 Jul 2026 11:23:25 +0800 Subject: [PATCH 2/7] docs: skip inapplicable release-7.1 cherry-pick --- best-practices/tidb-best-practices.md | 2 +- statistics.md | 30 +--- system-variables.md | 146 +----------------- ...v8.1-performance-benchmarking-with-tpcc.md | 123 --------------- ...v8.5-performance-benchmarking-with-tpcc.md | 123 --------------- troubleshoot-tidb-oom.md | 3 +- 6 files changed, 10 insertions(+), 417 deletions(-) delete mode 100644 tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md delete mode 100644 tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md diff --git a/best-practices/tidb-best-practices.md b/best-practices/tidb-best-practices.md index f783df2c72200..88be54844279e 100644 --- a/best-practices/tidb-best-practices.md +++ b/best-practices/tidb-best-practices.md @@ -129,7 +129,7 @@ Lots of MySQL experience is also applicable to TiDB. It is noted that TiDB has i * Ensure the order of results through indexes - You can use indexes to filter or sort data. TiDB first gets row IDs according to the index order, and then returns the row content according to the order of those row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but you can increase it by adjusting [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50). + You can use indexes to filter or sort data. Firstly, get row IDs according to the index order. Then, return the row content according to the return order of row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but it can be modified through the [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) variable. * Reverse index scan diff --git a/statistics.md b/statistics.md index bfa9b08c061fb..a7aa8f570e02d 100644 --- a/statistics.md +++ b/statistics.md @@ -430,37 +430,14 @@ When you run the `ANALYZE` statement, you can adjust the concurrency using the f #### `tidb_build_stats_concurrency` -<<<<<<< HEAD Currently, when you run the `ANALYZE` statement, the task is divided into multiple small tasks. Each task only works on one column or index. You can use the `tidb_build_stats_concurrency` parameter to control the number of simultaneous tasks. The default value is `4`. -======= -This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions. -#### `tidb_build_sampling_stats_concurrency` +#### `tidb_distsql_scan_concurrency` -This variable controls the following aspects of `ANALYZE` concurrency: - -- The concurrency for merging samples collected from different Regions. -- The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. - -The default value is `2`. - -#### `tidb_analyze_partition_concurrency` - -This variable controls the concurrency for saving `ANALYZE` results (writing TopN and histograms to system tables). The default value is `2`. The default value is `1` for v7.4.0 and earlier versions. ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) - -#### `tidb_analyze_distsql_scan_concurrency` - -<<<<<<< HEAD When you analyze regular columns, you can use the `tidb_distsql_scan_concurrency` parameter to control the number of Region to be read at one time. The default value is `15`. -======= -This variable controls the following aspects of `ANALYZE` concurrency: ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) -- The concurrency of scanning TiKV Regions. -- The concurrency of scanning Regions for special indexes (indexes generated from virtual columns). +#### `tidb_index_serial_scan_concurrency` -<<<<<<< HEAD When you analyze index columns, you can use the `tidb_index_serial_scan_concurrency` parameter to control the number of Region to be read at one time. The default value is `1`. ### Persist ANALYZE configurations @@ -920,9 +897,6 @@ mysql> show warnings; +-------+------+-----------------------------------------------------------------+ 1 row in set (0.00 sec) ``` -======= -The default value is `4`. ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ## See also diff --git a/system-variables.md b/system-variables.md index c45192d29bd85..d884ce66d3623 100644 --- a/system-variables.md +++ b/system-variables.md @@ -958,42 +958,6 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Default value: `OFF` - This variable is used to set whether the `AUTO_INCREMENT` property of a column is allowed to be removed by executing `ALTER TABLE MODIFY` or `ALTER TABLE CHANGE` statements. It is not allowed by default. -<<<<<<< HEAD -======= -### tidb_analyze_column_options New in v8.3.0 - -> **Note:** -> -> - This variable only works when [`tidb_analyze_version`](#tidb_analyze_version-new-in-v510) is set to `2`. -> - If you upgrade your TiDB cluster from a version earlier than v8.3.0 to v8.3.0 or later, this variable is set to `ALL` by default to keep the original behavior. -> - For a newly deployed TiDB cluster from v8.3.0 to v8.5.4, this variable is set to `PREDICATE` by default. -> - For a newly deployed TiDB cluster from v8.5.5 and v9.0.0 onwards, this variable is set to `ALL` by default. - -- Scope: GLOBAL -- Persists to cluster: Yes -- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No -- Type: Enumeration -- Default value: `ALL` -- Value options:`ALL`, `PREDICATE` -- This variable controls the behavior of the `ANALYZE TABLE` statement. Setting it to `PREDICATE` means only collecting statistics for [predicate columns](/statistics.md#collect-statistics-on-some-columns); setting it to `ALL` means collecting statistics for all columns. In scenarios where OLAP queries are used, it is recommended to set it to `ALL`, otherwise collecting statistics can result in a significant drop in query performance. - -### tidb_analyze_distsql_scan_concurrency New in v7.6.0 - -> **Note:** -> -> In versions earlier than v7.6.0, regular `ANALYZE` region scans are controlled by `tidb_distsql_scan_concurrency`, and index statistics scans are controlled by `tidb_index_serial_scan_concurrency`. Therefore, to adjust the concurrency of scanning TiKV Regions for these versions, consider changing the value of `tidb_distsql_scan_concurrency`. - -- Scope: SESSION | GLOBAL -- Persists to cluster: Yes -- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No -- Type: Integer -- Default value: `4` -- Range: `[0, 4294967295]`. In versions earlier than v8.2.0, the minimum value is `1`. When you set it to `0`, it adaptively adjusts the concurrency based on the cluster size. -- This variable controls the following aspects of `ANALYZE` concurrency: - - The concurrency of scanning TiKV Regions. - - The concurrency of scanning Regions for special indexes, such as indexes on generated virtual columns. - ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_analyze_partition_concurrency > **Warning:** @@ -1002,15 +966,8 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Scope: SESSION | GLOBAL - Persists to cluster: Yes -<<<<<<< HEAD - Default value: `1` - This variable specifies the concurrency of reading and writing statistics for a partitioned table when TiDB analyzes the partitioned table. -======= -- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No -- Default value: `2`. The default value is `1` for v7.4.0 and earlier versions. -- Range: `[1, 128]`. Before v8.4.0, the value range is `[1, 18446744073709551615]`. -- For manual and auto `ANALYZE`, this variable controls the concurrency for saving `ANALYZE` results, including writing TopN and histograms to system tables. ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_analyze_version New in v5.1.0 @@ -1025,76 +982,6 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - If your cluster is upgraded from an earlier version, the default value of `tidb_analyze_version` does not change after the upgrade. - For detailed introduction about this variable, see [Introduction to Statistics](/statistics.md). -<<<<<<< HEAD -======= -### tidb_analyze_skip_column_types New in v7.2.0 - -- Scope: SESSION | GLOBAL -- Persists to cluster: Yes -- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No -- Default value: "json,blob,mediumblob,longblob,mediumtext,longtext". Before v8.2.0, the default value is "json,blob,mediumblob,longblob". -- Possible values: "json,blob,mediumblob,longblob,text,mediumtext,longtext" -- This variable controls which types of columns are skipped for statistics collection when executing the `ANALYZE` command to collect statistics. The variable is only applicable for `tidb_analyze_version = 2`. Even if you specify a column using `ANALYZE TABLE t COLUMNS c1, ... , cn`, no statistics will be collected for the specified column if its type is in `tidb_analyze_skip_column_types`. - -``` -mysql> SHOW CREATE TABLE t; -+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Table | Create Table | -+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| t | CREATE TABLE `t` ( - `a` int DEFAULT NULL, - `b` varchar(10) DEFAULT NULL, - `c` json DEFAULT NULL, - `d` blob DEFAULT NULL, - `e` longblob DEFAULT NULL -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin | -+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -1 row in set (0.00 sec) - -mysql> SELECT @@tidb_analyze_skip_column_types; -+----------------------------------+ -| @@tidb_analyze_skip_column_types | -+----------------------------------+ -| json,blob,mediumblob,longblob,mediumtext,longtext | -+----------------------------------+ -1 row in set (0.00 sec) - -mysql> ANALYZE TABLE t; -Query OK, 0 rows affected, 1 warning (0.05 sec) - -mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; -+---------------------------------------------------------------------+ -| job_info | -+---------------------------------------------------------------------+ -| analyze table columns a, b with 256 buckets, 500 topn, 1 samplerate | -+---------------------------------------------------------------------+ -1 row in set (0.00 sec) - -mysql> ANALYZE TABLE t COLUMNS a, c; -Query OK, 0 rows affected, 1 warning (0.04 sec) - -mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; -+------------------------------------------------------------------+ -| job_info | -+------------------------------------------------------------------+ -| analyze table columns a with 256 buckets, 500 topn, 1 samplerate | -+------------------------------------------------------------------+ -1 row in set (0.00 sec) -``` - -### tidb_auto_analyze_concurrency New in v8.4.0 - -- Scope: GLOBAL -- Persists to cluster: Yes -- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No -- Type: Integer -- Default value: `3` -- Range: `[1, 2147483647]` -- This variable controls the number of concurrent auto-analyze operations that can run in a TiDB cluster. To accelerate statistics collection tasks, you can increase this concurrency based on the available resources in your cluster. -- Before v8.4.0, this concurrency is fixed at `1`. -- Starting from v8.5.7 and v9.0.0, the default value changes from `1` to `3`. If your cluster is upgraded from an earlier version, the value of this variable remains unchanged after the upgrade. - ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_auto_analyze_end_time - Scope: GLOBAL @@ -1141,12 +1028,7 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; - Type: Integer - Default value: `1` - Range: `[1, 256]` -<<<<<<< HEAD - This variable is used to set the concurrency of executing the automatic update of statistics. -======= -- This variable controls the concurrency for building statistics during auto `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously -- Starting from v8.5.7 and v9.0.0, the default value of this variable changes from `1` to `2`. If your cluster is upgraded from an earlier version, the value of this variable remains unchanged after the upgrade. ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_backoff_lock_fast @@ -1242,24 +1124,9 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; - Default value: `4` - Range: `[1, 256]` - Unit: Threads -- This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. - -<<<<<<< HEAD -======= -### tidb_build_sampling_stats_concurrency New in v7.5.0 - -- Scope: SESSION | GLOBAL -- Persists to cluster: Yes -- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No -- Type: Integer -- Unit: Threads -- Default value: `2` -- Range: `[1, 256]` -- This variable controls the following aspects of `ANALYZE` concurrency: - - The concurrency for merging samples collected from different Regions. - - The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. +- This variable is used to set the concurrency of executing the `ANALYZE` statement. +- When the variable is set to a larger value, the execution performance of other queries is affected. ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_capture_plan_baselines New in v4.0 - Scope: GLOBAL @@ -2924,17 +2791,14 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified ### tidb_index_serial_scan_concurrency -> **Warning:** -> -> This variable is deprecated and no longer controls execution behavior. The concurrency of sequential index scans is now controlled by [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50), and [`ANALYZE TABLE`](/sql-statements/sql-statement-analyze-table.md) uses [`tidb_analyze_distsql_scan_concurrency`](#tidb_analyze_distsql_scan_concurrency-new-in-v760) to control the concurrency of index statistics scans. - - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Type: Integer - Default value: `1` - Range: `[1, 256]` - Unit: Threads -- This variable remains only for backward compatibility. Use [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50) to control the concurrency of sequential index scans, or [`tidb_analyze_distsql_scan_concurrency`](#tidb_analyze_distsql_scan_concurrency-new-in-v760) to control the concurrency of index statistics scans. +- This variable is used to set the concurrency of the `serial scan` operation. +- Use a bigger value in OLAP scenarios, and a smaller value in OLTP scenarios. ### tidb_init_chunk_size @@ -3361,7 +3225,7 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Default value: `1` -- This variable controls the concurrency for merging TopN results of partitioned tables. +- This variable specifies the concurrency of merging statistics for a partitioned table when TiDB analyzes the partitioned table. ### tidb_metric_query_range_duration New in v4.0 diff --git a/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md b/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md deleted file mode 100644 index 79e74aec086dd..0000000000000 --- a/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md +++ /dev/null @@ -1,123 +0,0 @@ ---- -title: TiDB Cloud TPC-C Performance Test Report for TiDB v8.1.0 -summary: Introduce the TPC-C performance test results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.1.0. ---- - -# TiDB Cloud TPC-C Performance Test Report for TiDB v8.1.0 - -This document provides the TPC-C performance test steps and results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.1.0. This report can also be used as a reference for the performance of TiDB Self-Managed v8.1.0 clusters. - -## Test overview - -This test aims at showing the TPC-C performance of TiDB v8.1.0 in the Online Transactional Processing (OLTP) scenario. - -## Test environment - -### TiDB cluster - -The test is conducted on a TiDB cluster with the following settings: - -- Cluster type: [TiDB Cloud Dedicated](/tidb-cloud/select-cluster-tier.md#tidb-cloud-dedicated) -- Cluster version: v8.1.0 -- Cloud provider: AWS (us-west-2) -- Cluster configuration: - - | Node type | Node size | Node quantity | Node storage | - |:----------|:----------|:----------|:----------| - | TiDB | 16 vCPU, 32 GiB | 2 | N/A | - | TiKV | 16 vCPU, 64 GiB | 3 | 1000 GiB | - -### Parameter configuration - -> **Note:** -> -> For TiDB Cloud, to modify the TiKV parameters of your cluster, you can contact [PingCAP Support](/tidb-cloud/tidb-cloud-support.md) for help. - -The TiKV parameter [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) can make log recycling effective immediately after initialization. This document conducts tests with `prefill-for-recycle` enabled: - -```yaml -raft-engine.prefill-for-recycle = true -``` - -### Benchmark executor - -The benchmark executor sends SQL queries to the TiDB cluster. In this test, its hardware configuration is as follows: - -- Machine type: Amazon EC2 (us-west-2) -- Instance type: c6a.2xlarge - -## Test steps - -This section introduces how to perform the TPC-C performance test step by step. - -1. In the [TiDB Cloud console](https://tidbcloud.com/), create a TiDB Cloud Dedicated cluster that meets the [test environment](#tidb-cluster) requirements. - - For more information, see [Create a TiDB Cloud Dedicated cluster](/tidb-cloud/create-tidb-cluster.md). - -2. On the benchmark executor, connect to the newly created cluster and create a database named `tpcc`. - - To connect to the cluster, see [Connect to TiDB Cloud Dedicated via Private Endpoint](/tidb-cloud/set-up-private-endpoint-connections.md). - - To create the `tpcc` database, execute the following SQL statement: - - ```sql - CREATE DATABASE tpcc; - ``` - -3. Load TPC-C data to the `tpcc` database. - - 1. The test in this document is implemented based on [go-tpc](https://github.com/pingcap/go-tpc). You can download the test program using the following command: - - ```shell - curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/pingcap/go-tpc/master/install.sh | sh - ``` - - 2. Run the following `go-tpc tpcc` command to import 1,000 warehouses to the `tpcc` database. Replace `${HOST}`, `${THREAD}`, and `${PASSWORD}` with your actual values. This document conducts three tests with the `${THREAD}` value of `50`, `100`, and `200`. - - ```shell - go-tpc tpcc --host ${HOST} --warehouses 1000 prepare -P 4000 -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error - ``` - -4. To ensure that the TiDB optimizer can generate the optimal execution plan, execute the following SQL statements to collect statistics before conducting the TPC-C test: - - ```sql - ANALYZE TABLE customer; - ANALYZE TABLE district; - ANALYZE TABLE history; - ANALYZE TABLE item; - ANALYZE TABLE new_order; - ANALYZE TABLE order_line; - ANALYZE TABLE orders; - ANALYZE TABLE stock; - ANALYZE TABLE warehouse; - ``` - - To accelerate the collection of statistics, execute the following SQL statements before collecting: - - ```sql - SET tidb_build_stats_concurrency=16; - SET tidb_build_sampling_stats_concurrency = 16; - SET tidb_analyze_distsql_scan_concurrency=16; - ``` - -5. Run the following `go-tpc tpcc` command to conduct stress tests on the TiDB Cloud Dedicated cluster. For each concurrency, the test takes two hours. - - ```shell - go-tpc tpcc --host ${HOST} -P 4000 --warehouses 1000 run -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error - ``` - -6. Extract the tpmC data of `NEW_ORDER` from the result. - - TPC-C uses tpmC (transactions per minute) to measure the maximum qualified throughput (MQTh, Max Qualified Throughput). The transactions are the NewOrder transactions and the final unit of measure is the number of new orders processed per minute. - -## Test results - -The TPC-C performance of v8.1.0 in the [test environment](#test-environment) is as follows: - -| Threads | v8.1.0 tpmC | -|:--------|:----------| -| 50 | 43,660 | -| 100 | 75,495 | -| 200 | 102,013 | - -![TPC-C](/media/tidb-cloud/v8.1.0_tpcc.png) diff --git a/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md b/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md deleted file mode 100644 index 3bd5bbd2a576a..0000000000000 --- a/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md +++ /dev/null @@ -1,123 +0,0 @@ ---- -title: TiDB Cloud TPC-C Performance Test Report for TiDB v8.5.0 -summary: Introduce the TPC-C performance test results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.5.0. ---- - -# TiDB Cloud TPC-C Performance Test Report for TiDB v8.5.0 - -This document provides the TPC-C performance test steps and results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.5.0. This report can also be used as a reference for the performance of TiDB Self-Managed v8.5.0 clusters. - -## Test overview - -This test aims at showing the TPC-C performance of TiDB v8.5.0 in the Online Transactional Processing (OLTP) scenario. - -## Test environment - -### TiDB cluster - -The test is conducted on a TiDB cluster with the following settings: - -- Cluster type: [TiDB Cloud Dedicated](/tidb-cloud/select-cluster-tier.md#tidb-cloud-dedicated) -- Cluster version: v8.5.0 -- Cloud provider: AWS (us-west-2) -- Cluster configuration: - - | Node type | Node size | Node quantity | Node storage | - |:----------|:----------|:----------|:----------| - | TiDB | 16 vCPU, 32 GiB | 2 | N/A | - | TiKV | 16 vCPU, 64 GiB | 3 | 1000 GiB | - -### Parameter configuration - -> **Note:** -> -> For TiDB Cloud, to modify the TiKV parameters of your cluster, you can contact [PingCAP Support](/tidb-cloud/tidb-cloud-support.md) for help. - -The TiKV parameter [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) can make log recycling effective immediately after initialization. This document conducts tests with `prefill-for-recycle` enabled: - -```yaml -raft-engine.prefill-for-recycle = true -``` - -### Benchmark executor - -The benchmark executor sends SQL queries to the TiDB cluster. In this test, its hardware configuration is as follows: - -- Machine type: Amazon EC2 (us-west-2) -- Instance type: c6a.2xlarge - -## Test steps - -This section introduces how to perform the TPC-C performance test step by step. - -1. In the [TiDB Cloud console](https://tidbcloud.com/), create a TiDB Cloud Dedicated cluster that meets the [test environment](#tidb-cluster) requirements. - - For more information, see [Create a TiDB Cloud Dedicated cluster](/tidb-cloud/create-tidb-cluster.md). - -2. On the benchmark executor, connect to the newly created cluster and create a database named `tpcc`. - - To connect to the cluster, see [Connect to TiDB Cloud Dedicated via Private Endpoint](/tidb-cloud/set-up-private-endpoint-connections.md). - - To create the `tpcc` database, execute the following SQL statement: - - ```sql - CREATE DATABASE tpcc; - ``` - -3. Load TPC-C data to the `tpcc` database. - - 1. The test in this document is implemented based on [go-tpc](https://github.com/pingcap/go-tpc). You can download the test program using the following command: - - ```shell - curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/pingcap/go-tpc/master/install.sh | sh - ``` - - 2. Run the following `go-tpc tpcc` command to import 1,000 warehouses to the `tpcc` database. Replace `${HOST}`, `${THREAD}`, and `${PASSWORD}` with your actual values. This document conducts three tests with the `${THREAD}` value of `50`, `100`, and `200`. - - ```shell - go-tpc tpcc --host ${HOST} --warehouses 1000 prepare -P 4000 -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error - ``` - -4. To ensure that the TiDB optimizer can generate the optimal execution plan, execute the following SQL statements to collect statistics before conducting the TPC-C test: - - ```sql - ANALYZE TABLE customer; - ANALYZE TABLE district; - ANALYZE TABLE history; - ANALYZE TABLE item; - ANALYZE TABLE new_order; - ANALYZE TABLE order_line; - ANALYZE TABLE orders; - ANALYZE TABLE stock; - ANALYZE TABLE warehouse; - ``` - - To accelerate the collection of statistics, execute the following SQL statements before collecting: - - ```sql - SET tidb_build_stats_concurrency=16; - SET tidb_build_sampling_stats_concurrency = 16; - SET tidb_analyze_distsql_scan_concurrency=16; - ``` - -5. Run the following `go-tpc tpcc` command to conduct stress tests on the TiDB Cloud Dedicated cluster. For each concurrency, the test takes two hours. - - ```shell - go-tpc tpcc --host ${HOST} -P 4000 --warehouses 1000 run -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error - ``` - -6. Extract the tpmC data of `NEW_ORDER` from the result. - - TPC-C uses tpmC (transactions per minute) to measure the maximum qualified throughput (MQTh, Max Qualified Throughput). The transactions are the NewOrder transactions and the final unit of measure is the number of new orders processed per minute. - -## Test results - -The TPC-C performance of v8.5.0 in the [test environment](#test-environment) is as follows: - -| Threads | v8.5.0 tpmC | -|:--------|:----------| -| 50 | 43,146 | -| 100 | 73,875 | -| 200 | 103,395 | - -![TPC-C](/media/tidb-cloud/v8.5.0_tpcc.png) diff --git a/troubleshoot-tidb-oom.md b/troubleshoot-tidb-oom.md index c888f278f5e63..1c2f102cd0681 100644 --- a/troubleshoot-tidb-oom.md +++ b/troubleshoot-tidb-oom.md @@ -99,6 +99,7 @@ You can take the following measures to reduce the memory usage of SQL statements - Reduce the number of Regions to be read simultaneously or reduce the concurrency of operators to avoid memory problems caused by high concurrency. The corresponding system variables include: - [`tidb_distsql_scan_concurrency`](/system-variables.md#tidb_distsql_scan_concurrency) + - [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) - [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50) - The concurrency of sessions is too high near the time point when the problem occurs. In this case, consider scaling out the TiDB cluster by adding more TiDB nodes. @@ -194,4 +195,4 @@ To locate the root cause of an OOM issue, you need to collect the following info ## See also - [TiDB Memory Control](/configure-memory-usage.md) -- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) +- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) \ No newline at end of file From 4a971d75ffea38dca39e752a400447c9a21b1c47 Mon Sep 17 00:00:00 2001 From: qiancai Date: Wed, 15 Jul 2026 12:17:04 +0800 Subject: [PATCH 3/7] docs: adapt cherry-pick for release-7.1 --- best-practices/tidb-best-practices.md | 2 +- statistics.md | 2 +- system-variables.md | 17 ++++++++++------- troubleshoot-tidb-oom.md | 3 +-- 4 files changed, 13 insertions(+), 11 deletions(-) diff --git a/best-practices/tidb-best-practices.md b/best-practices/tidb-best-practices.md index 88be54844279e..f783df2c72200 100644 --- a/best-practices/tidb-best-practices.md +++ b/best-practices/tidb-best-practices.md @@ -129,7 +129,7 @@ Lots of MySQL experience is also applicable to TiDB. It is noted that TiDB has i * Ensure the order of results through indexes - You can use indexes to filter or sort data. Firstly, get row IDs according to the index order. Then, return the row content according to the return order of row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but it can be modified through the [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) variable. + You can use indexes to filter or sort data. TiDB first gets row IDs according to the index order, and then returns the row content according to the order of those row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but you can increase it by adjusting [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50). * Reverse index scan diff --git a/statistics.md b/statistics.md index a7aa8f570e02d..42705219db467 100644 --- a/statistics.md +++ b/statistics.md @@ -430,7 +430,7 @@ When you run the `ANALYZE` statement, you can adjust the concurrency using the f #### `tidb_build_stats_concurrency` -Currently, when you run the `ANALYZE` statement, the task is divided into multiple small tasks. Each task only works on one column or index. You can use the `tidb_build_stats_concurrency` parameter to control the number of simultaneous tasks. The default value is `4`. +This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. The default value is `4`. #### `tidb_distsql_scan_concurrency` diff --git a/system-variables.md b/system-variables.md index d884ce66d3623..dabab2342a77c 100644 --- a/system-variables.md +++ b/system-variables.md @@ -967,7 +967,8 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Default value: `1` -- This variable specifies the concurrency of reading and writing statistics for a partitioned table when TiDB analyzes the partitioned table. +- Range: `[1, 18446744073709551615]` +- For manual and auto `ANALYZE`, this variable controls the concurrency for saving `ANALYZE` results, including writing TopN and histograms to system tables. ### tidb_analyze_version New in v5.1.0 @@ -1028,7 +1029,7 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Type: Integer - Default value: `1` - Range: `[1, 256]` -- This variable is used to set the concurrency of executing the automatic update of statistics. +- This variable controls the concurrency for building statistics during auto `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. ### tidb_backoff_lock_fast @@ -1124,8 +1125,7 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Default value: `4` - Range: `[1, 256]` - Unit: Threads -- This variable is used to set the concurrency of executing the `ANALYZE` statement. -- When the variable is set to a larger value, the execution performance of other queries is affected. +- This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. ### tidb_capture_plan_baselines New in v4.0 @@ -2791,14 +2791,17 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified ### tidb_index_serial_scan_concurrency +> **Warning:** +> +> This variable is deprecated and no longer controls execution behavior. The concurrency of sequential index scans is now controlled by [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50). + - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Type: Integer - Default value: `1` - Range: `[1, 256]` - Unit: Threads -- This variable is used to set the concurrency of the `serial scan` operation. -- Use a bigger value in OLAP scenarios, and a smaller value in OLTP scenarios. +- This variable remains only for backward compatibility. Use [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50) to control the concurrency of sequential index scans. ### tidb_init_chunk_size @@ -3225,7 +3228,7 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Default value: `1` -- This variable specifies the concurrency of merging statistics for a partitioned table when TiDB analyzes the partitioned table. +- This variable controls the concurrency for merging TopN results of partitioned tables. ### tidb_metric_query_range_duration New in v4.0 diff --git a/troubleshoot-tidb-oom.md b/troubleshoot-tidb-oom.md index 1c2f102cd0681..c888f278f5e63 100644 --- a/troubleshoot-tidb-oom.md +++ b/troubleshoot-tidb-oom.md @@ -99,7 +99,6 @@ You can take the following measures to reduce the memory usage of SQL statements - Reduce the number of Regions to be read simultaneously or reduce the concurrency of operators to avoid memory problems caused by high concurrency. The corresponding system variables include: - [`tidb_distsql_scan_concurrency`](/system-variables.md#tidb_distsql_scan_concurrency) - - [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) - [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50) - The concurrency of sessions is too high near the time point when the problem occurs. In this case, consider scaling out the TiDB cluster by adding more TiDB nodes. @@ -195,4 +194,4 @@ To locate the root cause of an OOM issue, you need to collect the following info ## See also - [TiDB Memory Control](/configure-memory-usage.md) -- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) \ No newline at end of file +- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) From 5d9e4cd137403a4d2baad72fec621c77151e5363 Mon Sep 17 00:00:00 2001 From: qiancai Date: Wed, 15 Jul 2026 12:19:17 +0800 Subject: [PATCH 4/7] docs: exclude best practices from release-7.1 --- best-practices/tidb-best-practices.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/best-practices/tidb-best-practices.md b/best-practices/tidb-best-practices.md index f783df2c72200..88be54844279e 100644 --- a/best-practices/tidb-best-practices.md +++ b/best-practices/tidb-best-practices.md @@ -129,7 +129,7 @@ Lots of MySQL experience is also applicable to TiDB. It is noted that TiDB has i * Ensure the order of results through indexes - You can use indexes to filter or sort data. TiDB first gets row IDs according to the index order, and then returns the row content according to the order of those row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but you can increase it by adjusting [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50). + You can use indexes to filter or sort data. Firstly, get row IDs according to the index order. Then, return the row content according to the return order of row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but it can be modified through the [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) variable. * Reverse index scan From 0f9f58e34e02bef455e617153ded6b363707269a Mon Sep 17 00:00:00 2001 From: qiancai Date: Wed, 15 Jul 2026 12:30:31 +0800 Subject: [PATCH 5/7] docs: retain release-7.1 index scan concurrency --- system-variables.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/system-variables.md b/system-variables.md index dabab2342a77c..0cb0cfd89d9e6 100644 --- a/system-variables.md +++ b/system-variables.md @@ -2791,17 +2791,14 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified ### tidb_index_serial_scan_concurrency -> **Warning:** -> -> This variable is deprecated and no longer controls execution behavior. The concurrency of sequential index scans is now controlled by [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50). - - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Type: Integer - Default value: `1` - Range: `[1, 256]` - Unit: Threads -- This variable remains only for backward compatibility. Use [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50) to control the concurrency of sequential index scans. +- This variable is used to set the concurrency of the `serial scan` operation. +- Use a bigger value in OLAP scenarios, and a smaller value in OLTP scenarios. ### tidb_init_chunk_size From fb10d5fc4d9f122adbcb9f685d4b126710bb083c Mon Sep 17 00:00:00 2001 From: qiancai Date: Wed, 15 Jul 2026 12:41:57 +0800 Subject: [PATCH 6/7] docs: retain release-7.1 OOM concurrency guidance --- troubleshoot-tidb-oom.md | 1 + 1 file changed, 1 insertion(+) diff --git a/troubleshoot-tidb-oom.md b/troubleshoot-tidb-oom.md index c888f278f5e63..545ec8c1d0c16 100644 --- a/troubleshoot-tidb-oom.md +++ b/troubleshoot-tidb-oom.md @@ -99,6 +99,7 @@ You can take the following measures to reduce the memory usage of SQL statements - Reduce the number of Regions to be read simultaneously or reduce the concurrency of operators to avoid memory problems caused by high concurrency. The corresponding system variables include: - [`tidb_distsql_scan_concurrency`](/system-variables.md#tidb_distsql_scan_concurrency) + - [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) - [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50) - The concurrency of sessions is too high near the time point when the problem occurs. In this case, consider scaling out the TiDB cluster by adding more TiDB nodes. From cd3a565e44b040957394f23119f83e3cbad5f85a Mon Sep 17 00:00:00 2001 From: qiancai Date: Wed, 15 Jul 2026 12:44:07 +0800 Subject: [PATCH 7/7] docs: exclude no-op OOM guidance from release-7.1 --- troubleshoot-tidb-oom.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/troubleshoot-tidb-oom.md b/troubleshoot-tidb-oom.md index 545ec8c1d0c16..1c2f102cd0681 100644 --- a/troubleshoot-tidb-oom.md +++ b/troubleshoot-tidb-oom.md @@ -195,4 +195,4 @@ To locate the root cause of an OOM issue, you need to collect the following info ## See also - [TiDB Memory Control](/configure-memory-usage.md) -- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) +- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) \ No newline at end of file