Skip to content

Prefer local DataNode when LOAD creates DataRegions - #18663

Open
luoluoyuyu wants to merge 3 commits into
apache:masterfrom
luoluoyuyu:load-prefer-local-node
Open

luoluoyuyu wants to merge 3 commits into
apache:masterfrom
luoluoyuyu:load-prefer-local-node

Conversation

@luoluoyuyu

Copy link
Copy Markdown
Member

Summary

Add a hidden, hot-reloadable DataNode configuration load_tsfile_prefer_local_node (default true). When enabled, LOAD passes the current DataNodeId to ConfigNode as preferredDataNodeId while creating a database or requesting get-or-create data partitions. ConfigNode then prefers placing newly created DataRegion replicas on the requesting node without modifying the existing region allocation algorithms themselves.

Design / How It Works

1. DataNode configuration

  • IoTDBConfig.loadTsFilePreferLocalNode is a volatile boolean, default true.
  • IoTDBDescriptor loads it both at startup and during hot reload.
  • It is intentionally not added to iotdb-system.properties.template, so it stays hidden from SHOW CONFIGURATION.

2. Database creation path

When LOAD auto-creates a database:

  • TreeSchemaAutoCreatorAndVerifier sets preferredDataNodeId on DatabaseSchemaStatement.
  • DatabaseSchemaTask.constructDatabaseSchema copies it to TDatabaseSchema.preferredDataNodeId.
  • ConfigNode persists the field and RegionBalancer can later read it.

3. Data-partition creation path

This covers the case where the database already exists and LOAD creates devices/tables or requests an unassigned series partition, which triggers DataRegion extension:

  • LoadTsFileScheduler.DataPartitionBatchFetcher reads load_tsfile_prefer_local_node.
  • It calls the new IPartitionFetcher.getOrCreateDataPartition(..., preferredDataNodeId) overload.
  • ClusterPartitionFetcher puts the value into TDataPartitionReq.preferredDataNodeId.
  • ConfigNode converts it into GetOrCreateDataPartitionPlan.
  • PartitionManager passes the hint down to DataRegion extension.
  • RegionBalancer prefers the request-level hint first, then falls back to the database-schema hint.

4. RegionBalancer preference handling

The original allocator is still called first:

regionGroupAllocator.generateOptimalRegionReplicasDistribution(...)

Only after that, preferDataNodeIfPossible adjusts the result:

  • If the preferred node is unavailable, do nothing.
  • If it is already in the replica set, move it to the first position.
  • If it is absent, replace the last replica with the preferred node.

Replication factor, candidate validity, and region identity are preserved.

5. New time partitions

New time partitions do not create a new DataRegion. A new time slot is mapped into an existing DataRegion for the same series-partition slot, so this path needs no preference change.

Validation

mvn -pl iotdb-core/datanode,iotdb-core/confignode -am -DskipTests -Drat.skip=true compile

Result: BUILD SUCCESS.

@luoluoyuyu luoluoyuyu changed the title TDB-318: Prefer local DataNode when LOAD creates DataRegions Prefer local DataNode when LOAD creates DataRegions Sep 17, 2026

@Caideyipi Caideyipi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

发现 3 个需要关注的问题,详见 inline comments。

Comment thread iotdb-protocol/thrift-confignode/src/main/thrift/confignode.thrift Outdated

@Caideyipi Caideyipi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

复查当前提交 1b5eb12,未发现新的阻塞问题。

之前的 3 条问题均已修复:preferredDataNodeId 已仅保留在分区请求中,数据库 schema 不再持久化或经 Pipe 传播该字段;Tablet 转换的 tree/table 路径均能在同步 INSERT 分析期间传递 hint,并在 finally 中清理 ThreadLocal。已检查普通请求/关闭开关时的默认行为,以及 hint 不会将候选集合之外的 DataNode 强行加入副本集,移除中的节点仍被排除。

验证依据:代码调用链复查、diff 检查及当前提交的 CI。相关 LOAD 单测、Linux 单测、集群 IT 和中英文编译检查通过;Windows DataNode 单测唯一失败为 WALNodeWaitForRollFileTest.testWaitForNextReadyNotWokenByFlushWithoutRoll 的 10 秒等待超时,其调用链不涉及本次改动。本次未另行在本地运行测试。

非阻塞说明:PR 描述第 2/3 节仍写着数据库 schema 持久化和 fallback,建议同步为当前的请求级 hint 设计。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants