Skip to content

fix(visualization): declare the numeric-only chart settings as numbers - #7233

Open
kz930 wants to merge 1 commit into
apache:mainfrom
kz930:fix/7213-numeric-viz-settings
Open

fix(visualization): declare the numeric-only chart settings as numbers#7233
kz930 wants to merge 1 commit into
apache:mainfrom
kz930:fix/7213-numeric-viz-settings

Conversation

@kz930

@kz930 kz930 commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Eight settings across three chart operators accept only numbers — each is used solely as float(...) or int(...) — but were declared as plain strings, so the form let any text through and each operator then mishandled it differently:

  • Gauge Chart delta, threshold; Bullet Chart thresholdValue — discarded silently, leaving a finished-looking chart with the indicator missing
  • Bullet Chart deltaReference, step start/end — error page
  • Contour Plot gridSizeValueError, and the same when left blank

This PR declares them as Option[Double] / Option[Int]:

  • the form rejects a non-number before the run, and all three operators behave the same way
  • the generated code no longer parses: values are spliced as numbers, so the conversions, the try/except ValueError blocks and Gauge Chart's json.loads of its steps are gone, along with an import json that became dead
  • a step whose bounds are not both filled in is dropped while the list is built, which is what Bullet Chart's "Invalid step values" note existed for
  • @JsonDeserialize(contentAs = ...) names the boxed class: Scala erases Option's element type, so without it Jackson leaves the raw JSON value inside the Option and the first use throws ClassCastException, and the primitive class would read a blank as 0
  • Gauge Chart's step bounds are included beyond the seven the issue lists — same declaration, same consumer, and a non-numeric bound was swallowed by a bare except, plotting the gauge with no steps at all

Compatibility: a numeric string saved before this change still loads as a number and a blank one as unset; a workflow that stored a value that is not a number now fails to load rather than silently dropping it — those charts were already rendering without the setting.

Any related issues, documentation, discussions?

Fixes #7213.

How was this PR tested?

  • existing specs for the three operators updated to the new types
  • per field: deserialization tests for a JSON number, a numeric string, blank, null and absent — plus one that uses the value as a number, the case a round trip cannot catch, since a round trip writes a number back and a missing contentAs survives it
  • generated-code tests pin the assignments (delta_ref = 40.0, threshold_val = None, grid_size = 10 as the form's default) and the dropped half-filled step
  • ran the generated Python against a pandas DataFrame for all three operators, configured and left unset: each plots a figure, and the inverted-step case still reports start ≥ end
  • whole workflow-operator module: 2033 tests passing, scalafmtCheck clean

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (claude-opus-5[1m])

@Yicong-Huang Yicong-Huang added the release/v1.2 back porting to release/v1.2 label Aug 1, 2026
@github-actions
github-actions Bot requested a review from xuang7 August 1, 2026 07:22
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Backport auto-label report

This fix: PR was checked against each actively-supported release branch. release/* labels drive the post-merge backport, so add or remove one to change where this fix lands.

Release branch Analysis
release/v1.2 Already labeled — this fix is queued to backport here.

Auto-label run.

@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @aglinxinyuan, @eugenegujing, @carloea2
    You can notify them by mentioning @aglinxinyuan, @eugenegujing, @carloea2 in a comment.

@codecov-commenter

codecov-commenter commented Aug 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.09%. Comparing base (348fa88) to head (b309bfe).
⚠️ Report is 24 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #7233      +/-   ##
============================================
+ Coverage     80.58%   82.09%   +1.51%     
- Complexity     3890     4081     +191     
============================================
  Files          1162     1162              
  Lines         46264    46315      +51     
  Branches       5153     5160       +7     
============================================
+ Hits          37280    38024     +744     
+ Misses         7319     6584     -735     
- Partials       1665     1707      +42     
Flag Coverage Δ *Carryforward flag
access-control-service 70.00% <ø> (ø)
agent-service 77.42% <ø> (ø) Carriedforward from 348fa88
amber 79.32% <100.00%> (+3.72%) ⬆️
computing-unit-managing-service 43.60% <ø> (+23.11%) ⬆️
config-service 65.97% <ø> (ø)
file-service 66.80% <ø> (ø)
frontend 83.22% <ø> (ø) Carriedforward from 348fa88
notebook-migration-service 78.89% <ø> (ø)
pyamber 97.36% <ø> (ø) Carriedforward from 348fa88
workflow-compiling-service 26.31% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 2 better · 🔴 5 worse · ⚪ 8 noise (<±5%) · 0 without baseline

Compared against main 436b37e benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 377 0.23 27,814/34,055/34,055 us 🔴 +20.2% / 🔴 +123.1%
bs=100 sw=10 sl=64 807 0.492 124,161/137,316/137,316 us ⚪ within ±5% / 🔴 +27.9%
🔴 bs=1000 sw=10 sl=64 926 0.565 1,075,112/1,240,656/1,240,656 us 🔴 +10.1% / 🔴 +20.3%
Baseline details

Latest main 436b37e from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 377 tuples/sec 418 tuples/sec 782.98 tuples/sec -9.8% -51.9%
bs=10 sw=10 sl=64 MB/s 0.23 MB/s 0.255 MB/s 0.478 MB/s -9.8% -51.9%
bs=10 sw=10 sl=64 p50 27,814 us 23,138 us 12,467 us +20.2% +123.1%
bs=10 sw=10 sl=64 p95 34,055 us 37,123 us 15,695 us -8.3% +117.0%
bs=10 sw=10 sl=64 p99 34,055 us 37,123 us 18,950 us -8.3% +79.7%
bs=100 sw=10 sl=64 throughput 807 tuples/sec 835 tuples/sec 996.92 tuples/sec -3.4% -19.1%
bs=100 sw=10 sl=64 MB/s 0.492 MB/s 0.51 MB/s 0.608 MB/s -3.5% -19.1%
bs=100 sw=10 sl=64 p50 124,161 us 122,059 us 100,899 us +1.7% +23.1%
bs=100 sw=10 sl=64 p95 137,316 us 141,213 us 107,385 us -2.8% +27.9%
bs=100 sw=10 sl=64 p99 137,316 us 141,213 us 115,226 us -2.8% +19.2%
bs=1000 sw=10 sl=64 throughput 926 tuples/sec 928 tuples/sec 1,024 tuples/sec -0.2% -9.6%
bs=1000 sw=10 sl=64 MB/s 0.565 MB/s 0.566 MB/s 0.625 MB/s -0.2% -9.6%
bs=1000 sw=10 sl=64 p50 1,075,112 us 1,073,131 us 989,300 us +0.2% +8.7%
bs=1000 sw=10 sl=64 p95 1,240,656 us 1,126,807 us 1,031,689 us +10.1% +20.3%
bs=1000 sw=10 sl=64 p99 1,240,656 us 1,126,807 us 1,058,394 us +10.1% +17.2%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,530.09,200,128000,377,0.230,27814.47,34054.79,34054.79
1,100,10,64,20,2479.07,2000,1280000,807,0.492,124160.99,137315.72,137315.72
2,1000,10,64,20,21595.35,20000,12800000,926,0.565,1075111.81,1240656.01,1240656.01

Eight settings across three chart operators accept only numbers -- every one of
them is used solely as `float(...)` or `int(...)` -- but were declared as plain
strings, so the form let any text through and each operator then mishandled it
differently: Gauge Chart's Delta and Threshold and Bullet Chart's Threshold Value
discarded the value silently, leaving a finished-looking chart with the indicator
missing; Bullet Chart's Delta Reference and its step bounds rendered an error
page; Contour Plot's Grid Size raised a ValueError, as did leaving it blank.

Declaring them numeric moves the check into the schema, so the form rejects a
non-number before the run and every operator behaves the same way. The parsing
disappears from the generated code with it: the values are spliced as numbers, so
`float(...)`, `int(...)`, the `try/except ValueError` blocks and Gauge Chart's
`json.loads` of its steps are all gone, along with the `import json` that became
dead. Half-filled step bounds are dropped while building the list rather than
checked in Python, which is what the "Invalid step values" note existed for.

`@JsonDeserialize(contentAs = ...)` is required on each field and names the boxed
class on purpose. Scala erases `Option[Double]`'s element type, so without the
annotation Jackson leaves the raw JSON value inside the Option and the first use
throws ClassCastException; with the primitive class instead, a blank value coerces
to 0, which for a threshold is the difference between "at zero" and "not drawn".
Each field gets a test for the shapes a stored workflow can hold -- number,
numeric string, blank, null, absent -- including one that reads the value back as
a number, since a round-trip test cannot catch the missing annotation.

Workflows keep loading: a numeric string saved before this change still reads as a
number, and a blank one as unset. A workflow that stored a value that is not a
number at all now fails to load rather than silently dropping it -- those charts
were already rendering without the setting.

Gauge Chart's step bounds are fixed alongside the seven settings the issue lists:
they are the same declaration with the same consumer, and a non-numeric bound was
swallowed by a bare `except`, plotting the gauge with no steps at all.

Fixes apache#7213

Generated-by: Claude Code (claude-opus-5[1m])
@kz930
kz930 force-pushed the fix/7213-numeric-viz-settings branch from 3cc8582 to b309bfe Compare August 2, 2026 04:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

common fix release/v1.2 back porting to release/v1.2

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Numeric-only visualization settings are unconstrained strings, and each operator mishandles a non-numeric value differently

3 participants