[AURON #2419] RowNullChecker may mis-detect null join keys for complex Arrow row types

[weimingdiit]

Which issue does this PR close?

Closes #2419

Rationale for this change
Auron's sort merge join uses encoded Arrow rows for join key comparison and uses RowNullChecker to skip rows with null join keys.

RowNullChecker previously inspected Arrow row bytes manually. This is fragile for complex row encodings such as struct, list, and dictionary, because their layout is controlled by Arrow RowConverter and should not be interpreted with fixed or estimated field lengths.

This can cause RowNullChecker to mis-detect null join keys for complex key types.

What changes are included in this PR?
Adds an Arrow RowConverter to RowNullChecker.
Updates RowNullChecker's has_nulls path to decode Rows through Arrow RowConverter before building the null-key mask.
Preserves conservative join key null semantics:
top-level null key values are treated as null join keys
nested null fields or elements inside non-null struct/list keys do not make the key null
Keeps the existing RowNullChecker public method signatures unchanged.
Keeps the existing byte-level helper path unchanged.
Adds unit coverage for complex struct, list, and dictionary key rows.

Are there any user-facing changes?
No user-facing API changes.

How was this patch tested?
UT.

Was this patch authored or co-authored using generative AI tooling?

• Yes
• No
  Generated-by: OpenAI Codex (GPT-5)

[Copilot]

Pull request overview

This PR fixes incorrect join-key null detection for complex Arrow row encodings (e.g., struct/list/dictionary) by switching RowNullChecker::has_nulls from manual byte inspection to decoding rows via Arrow’s RowConverter, then deriving a null-key mask from the decoded top-level key arrays.

Changes:

• Add and retain an Arrow RowConverter inside RowNullChecker for row decoding.
• Rework has_nulls to decode Rows into key columns and compute validity based on top-level nulls.
• Add unit tests covering complex key types (struct, list, dictionary) and ensuring nested nulls don’t invalidate a non-null top-level key.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[weiqingy]

nit: row_converter isn't in the # Parameters list, and it carries a precondition that only shows up at runtime: it has to be the converter that produced rows, or convert_rows trips assert!(Arc::ptr_eq(...), "rows were not produced by this RowConverter") (arrow-row/src/lib.rs:701-704). All three producers get it right today, but a fourth would find out the hard way. Worth spelling out in the doc?

Same block, also a nit: true bits indicate rows where all fields are non-null reads a little ambiguously against the semantics in the issue, since a non-null struct with nested nulls stays valid. Maybe "top-level key fields"?

[weimingdiit]

Good point. I updated the documentation to include row_converter and clarify that it must be the RowConverter that produced rows, as required by RowConverter::convert_rows.

I also changed the validity description to refer explicitly to top-level key fields, so it does not imply that nested nulls inside a valid struct make the key null.