Skip to content

[Pipe] Improve timeout diagnostics and DROP PIPE observability on dev/1.3 - #18340

Open
Caideyipi wants to merge 1 commit into
apache:dev/1.3from
Caideyipi:cp-834418ed-dev-1.3
Open

[Pipe] Improve timeout diagnostics and DROP PIPE observability on dev/1.3#18340
Caideyipi wants to merge 1 commit into
apache:dev/1.3from
Caideyipi:cp-834418ed-dev-1.3

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

Backport #18297 (834418ed5edc62a10c45f5457a7f94a214eac67e) to dev/1.3.

Changes

  • Improve timeout-stage diagnostics and report pending DataNodes.
  • Introduce PipeStatus.PRE_DELETE and make DROP PIPE persist the pre-delete state before notifying DataNodes and removing ConfigNode metadata.
  • Keep drop-procedure serialization backward compatible.
  • Prevent heartbeat updates, failure recording, auto-restart, task-agent state changes, and ALTER/START/STOP operations from interfering with a Pipe being dropped.
  • Adapt the change to the Java 8 and tree-model-only codebase on dev/1.3; master-only table-model and new i18n infrastructure are intentionally excluded.

Verification

  • Spotless and Checkstyle for node-commons and confignode.
  • Reactor compile and test-compile for both modules.
  • Focused unit tests: PipeMetaDeSerTest, AbstractOperatePipeProcedureV2Test, DropPipeProcedureV2Test, PipeTaskInfoAutoRestartTest, PipeHeartbeatParserTest, PipeTableRespTest, and ConfigPhysicalPlanSerDeTest (97 ConfigNode tests plus PipeMetaDeSerTest).
  • git diff --cached --check before commit.

This PR has:

  • been self-reviewed.
  • added comments explaining the intent of non-obvious compatibility logic.
  • added or updated unit tests to cover new code paths.

Key changed/added classes (or packages if there are too many classes) in this PR
  • AbstractOperatePipeProcedureV2
  • DropPipeProcedureV2
  • PipeTaskInfo
  • PipeHeartbeatParser
  • PipeTaskAgent
  • PipeStatus

…e#18297)

* Improve Pipe procedure timeout diagnostics

* Add pre-delete state to drop pipe procedure

* Address pipe timeout diagnostic review comments

* Fix pipe timeout message i18n
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant