Skip to content

fix: forward tolerance flags from get_jumpstart_configs - #6137

Open
evakravi wants to merge 1 commit into
aws:masterfrom
evakravi:fix/jumpstart-configs-tolerance-v3
Open

fix: forward tolerance flags from get_jumpstart_configs#6137
evakravi wants to merge 1 commit into
aws:masterfrom
evakravi:fix/jumpstart-configs-tolerance-v3

Conversation

@evakravi

Copy link
Copy Markdown
Member

Issue #, if available: #6130

Description of changes:

Problem

get_jumpstart_configs accepted no tolerate_vulnerable_model or tolerate_deprecated_model argument. It called verify_model_region_and_return_specs without them, so the callee fell back to its False defaults and re-ran the model gate. A caller that had asked to tolerate a flagged model still got VulnerableJumpStartModelError or DeprecatedJumpStartModelError, so both flags were unusable for exactly the models they exist to allow.

In v3 the reachable caller is ModelBuilder._ensure_metadata_configs, which resolves the same configs lazily and had no way to opt out of the gate. A ModelBuilder carrying tolerate_vulnerable_model=True still failed on a flagged model, even though every other lookup in that class already forwards the flag.

This blocks any release pipeline that deploys JumpStart models for validation. Such a pipeline has to deploy a model that is flagged, precisely because its job is to test that model.

The same gap exists in 2.x and is fixed by #6136. This PR is the v3 half.

Solution

Add tolerate_vulnerable_model and tolerate_deprecated_model to get_jumpstart_configs and forward both to verify_model_region_and_return_specs. Both default to False, so every existing caller keeps its current behavior and the gate still fires by default.

Pass both from _ensure_metadata_configs, using the same getattr idiom the surrounding call sites already use for these two flags, normalized to a bool so an unset attribute means "do not tolerate". A flagged model then resolves empty configs and the caller proceeds, which is the same outcome the flags already produce elsewhere.

Testing done:

Six tests in TestGetJumpstartConfigs. Two assert the flags reach the spec lookup that runs the gate, on the inference and training scopes. Two exercise the gate end to end through JumpStartModelsAccessor.get_model_specs, confirming a vulnerable model and a deprecated model resolve configs instead of raising. Two are regression guards: the gate still raises for a vulnerable model by default, and the default forwarded value is False.

Two more in TestEnsureMetadataConfigs cover the ModelBuilder caller: tolerance reaches the lookup when set, and defaults to False when unset.

Five of the six new sagemaker-core tests fail before the change, on the dropped argument:

E  TypeError: get_jumpstart_configs() got an unexpected keyword argument 'tolerate_vulnerable_model'
E  TypeError: get_jumpstart_configs() got an unexpected keyword argument 'tolerate_deprecated_model'
E  KeyError: 'tolerate_vulnerable_model'
5 failed, 3 passed

Both new sagemaker-serve tests fail before the change:

E  KeyError: 'tolerate_vulnerable_model'
2 failed, 2 passed

All pass after, with no regressions in either package:

$ python -m pytest sagemaker-core/tests/unit/test_jumpstart_utils.py -q
174 passed, 1 skipped

$ python -m pytest sagemaker-core/tests/unit -k jumpstart -q
693 passed, 4 skipped, 2841 deselected

$ python -m pytest sagemaker-serve/tests/unit/test_model_builder_utils_additional_gaps.py -q
44 passed, 3 subtests passed

Lint introduces nothing new. flake8 7.1.2 reports the same 135 findings on the touched files before and after this change, all pre-existing and on lines this diff does not touch. Under black 26.3.1 the added lines are already formatted. All four touched files are reformatted by that version on unmodified master, so this diff deliberately leaves that pre-existing formatting alone rather than mixing an unrelated reformat into a bug fix.

No integration test is included. Reproducing this requires a model whose published specs carry a vulnerability or deprecation flag, which is not something a test can provision, and the fix is a pure argument-forwarding change fully covered by unit tests.

Merge Checklist

General

  • I have read the CONTRIBUTING doc
  • I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
  • I used the commit message format described in CONTRIBUTING
  • I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
  • I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

  • I have added tests that prove my fix is effective or that my feature works (if appropriate)
  • I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
  • I have checked that my tests are not configured for a specific region or account (if appropriate)
  • I have used unique_name_from_base to create resource names in integ tests (if appropriate)
  • If adding any dependency in requirements.txt files, I have spell checked and ensured they exist in PyPi

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

get_jumpstart_configs accepted no tolerate_vulnerable_model or
tolerate_deprecated_model argument. It called
verify_model_region_and_return_specs without them, so the callee fell back to
its False defaults and re-ran the model gate. A caller that had asked to
tolerate a flagged model still got VulnerableJumpStartModelError or
DeprecatedJumpStartModelError, which made both flags unusable for that model.

Add both parameters, default them to False to keep current behavior for existing
callers, and forward them to verify_model_region_and_return_specs. Pass them
from ModelBuilder._ensure_metadata_configs, which resolves the same configs
lazily and had no way to opt out of the gate.

---
X-AI-Prompt: Can you fix the dropped JumpStart tolerance flags in v3 too?
X-AI-Tool: claude-code
@github-actions

github-actions Bot commented Jul 31, 2026

Copy link
Copy Markdown

Claude finished @evakravi's task in 44s —— View job


PR Review: forward tolerance flags from get_jumpstart_configs

  • Read the PR diff
  • Verify the callee accepts the forwarded arguments
  • Check all callers for backward compatibility
  • Assess test coverage

Summary

This is a clean, correct, and well-tested bug fix. get_jumpstart_configs was dropping the tolerance flags on the floor, so verify_model_region_and_return_specs fell back to its False defaults and re-ran the model gate. The fix adds both flags as optional parameters (defaulting to False) and forwards them.

Correctness ✅

  • verify_model_region_and_return_specs (sagemaker-core/src/sagemaker/core/jumpstart/utils.py:689) already accepts tolerate_vulnerable_model / tolerate_deprecated_model with matching False defaults, so the forwarding is exact.
  • The two new parameters are appended after the existing ones with False defaults — no positional argument is displaced, so every existing caller keeps identical behavior. Backward compatible.
  • The only production caller besides the new one is ModelBuilder._ensure_metadata_configs, which now forwards both flags.

_ensure_metadata_configs idiom ✅

tolerate_vulnerable_model=getattr(self, "tolerate_vulnerable_model", None) or False,
tolerate_deprecated_model=getattr(self, "tolerate_deprecated_model", None) or False,

The getattr(..., None) or False normalization matches the surrounding call sites in the same class (e.g. model_version on the line above) and correctly coerces an unset/None attribute to False. Consistent with existing conventions.

Tests ✅
Coverage is thorough for an argument-forwarding change:

  • Forwarding asserted on both inference and training scopes.
  • End-to-end gate behavior verified through get_model_specs (vulnerable + deprecated models resolve configs instead of raising).
  • Regression guards: default is False, and the gate still raises by default.
  • ModelBuilder caller covered for both set and unset tolerance.

Minor observations (non-blocking)

  • The new Args: docstring block documents only the two new parameters, not the pre-existing ones. That's acceptable here (the function previously had no Args: section at all), but a fuller docstring would be a nice follow-up.
  • No integration test — reasonable, as the PR notes: reproducing this requires a model with published vulnerability/deprecation flags, which a test can't provision, and the change is pure argument forwarding fully exercised by unit tests.

No correctness, security, or performance concerns. This looks ready to merge.
fix/jumpstart-configs-tolerance-v3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant