Skip to content

[To dev/1.3] [Pipe] Fix TsFile rate limit accounting (#18344) - #18347

Open
Caideyipi wants to merge 1 commit into
apache:dev/1.3from
Caideyipi:cp-f3f7d12-dev-1.3
Open

[To dev/1.3] [Pipe] Fix TsFile rate limit accounting (#18344)#18347
Caideyipi wants to merge 1 commit into
apache:dev/1.3from
Caideyipi:cp-f3f7d12-dev-1.3

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

Backport #18344 (f3f7d12) to dev/1.3.

  • Apply TsFile rate limiting and disk I/O accounting after each read, using the actual number of bytes read.
  • Avoid accounting for EOF reads and over-accounting the final partial chunk across air-gap, SSL sync, sync Thrift, and async Thrift transfer paths.
  • Add unit coverage for async and air-gap TsFile transfer accounting.

The dev/1.3 branch uses the older non-V2 transfer request types, so the AirGap test conflict was resolved by retaining the 1.3 request types while adding the new byte-accounting assertion.

Verification

  • mvn spotless:apply -pl iotdb-core/datanode,iotdb-core/node-commons
  • mvn -Ddevelocity.off=true test -pl iotdb-core/datanode -Dtest=IoTDBDataRegionAirGapSinkTest,PipeTransferTsFileHandlerRateLimitTest (3 tests passed)

This PR has:

  • been self-reviewed.
  • added or modified unit tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant