Skip to content

[Batch] Fix #33284: Fix Python 3.14 typing compatibility for batch module#33298

Open
YangAn-microsoft wants to merge 3 commits intoAzure:devfrom
YangAn-microsoft:fix/batch-python314-union-type
Open

[Batch] Fix #33284: Fix Python 3.14 typing compatibility for batch module#33298
YangAn-microsoft wants to merge 3 commits intoAzure:devfrom
YangAn-microsoft:fix/batch-python314-union-type

Conversation

@YangAn-microsoft
Copy link
Copy Markdown

@YangAn-microsoft YangAn-microsoft commented Apr 30, 2026

Related command
az batch

Description
The az batch commands fail to load arguments under Python 3.14 due to two breaking changes in the typing module:

  • The _name attribute was removed from Optional/Union types
  • str() representation changed to pipe syntax (e.g. X | None instead of typing.Optional[X])

This fix replaces version-specific introspection with stable public APIs (get_args, get_type_hints) that work identically in Python 3.13 and 3.14.

Testing Guide
Run the batch argument loading test:

python -m pytest src/azure-cli/azure/cli/command_modules/batch/tests/latest/test_batch_commands.py::TestBatchLoader::test_batch_load_arguments -v

Passes on both Python 3.13 and 3.14 with identical argument counts.

@azure-client-tools-bot-prd
Copy link
Copy Markdown

azure-client-tools-bot-prd Bot commented Apr 30, 2026

❌AzureCLI-FullTest
️✔️acr
️✔️latest
️✔️3.12
️✔️3.13
️✔️acs
️✔️latest
️✔️3.12
️✔️3.13
️✔️advisor
️✔️latest
️✔️3.12
️✔️3.13
️✔️ams
️✔️latest
️✔️3.12
️✔️3.13
️✔️apim
️✔️latest
️✔️3.12
️✔️3.13
️✔️appconfig
️✔️latest
️✔️3.12
️✔️3.13
❌appservice
❌latest
❌3.12
Type Test Case Error Message Line
Failed test_linux_webapp_quick_create_cd self = <azure.cli.testsdk.base.ExecutionResult object at 0x7f8d8a526ed0>
cli_ctx = <azure.cli.core.mock.DummyCli object at 0x7f8d8af23470>
command = 'webapp create -g clitest.rg000001 -n webapp-linux-cd000002 --plan plan-quick-linux-cd -u https://github.com/yugangw-msft/azure-site-test.git&nbsp;-r&nbsp;"NODE
20-lts"'
expect_failure = False

    def in_process_execute(self, cli_ctx, command, expect_failure=False):
        from io import StringIO
        from vcr.errors import CannotOverwriteExistingCassetteException
    
        if command.startswith('az '):
            command = command[3:]
    
        stdout_buf = StringIO()
        logging_buf = StringIO()
        try:
            # issue: stderr cannot be redirect in this form, as a result some failure information
            # is lost when command fails.
>           self.exit_code = cli_ctx.invoke(shlex.split(command), out_file=stdout_buf) or 0
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

src/azure-cli-testsdk/azure/cli/testsdk/base.py:303: 
                                        
env/lib/python3.12/site-packages/knack/cli.py:245: in invoke
    exit_code = self.exception_handler(ex)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
src/azure-cli-core/azure/cli/core/init.py:157: in exception_handler
    return handle_exception(ex)
           ^^^^^^^^^^^^^^^^^^^^
                                       _ 

ex = ValidationError("Linux Runtime 'NODE
Failed test_win_webapp_quick_create_cd The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:174
Failed test_win_webapp_quick_create_runtime The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:156
Failed test_download_win_web_log The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:480
Failed test_webapp_config The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:732
Failed test_linux_webapp The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:1261
Failed test_linux_webapp_remote_ssh The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:1366
Failed test_acr_integration The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:1427
Failed test_webapp_linux_acr_use_identity The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:2592
Failed test_webapp_deployment_source_track_runtimestatus_buildfailed The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:3368
Failed test_webapp_deployment_source_track_runtimestatus_runtimefailed The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:3381
Failed test_webapp_track_runtimestatus_buildfailed The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:3307
Failed test_webapp_track_runtimestatus_runtimefailed The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:3320
❌3.13
Type Test Case Error Message Line
Failed test_linux_webapp_quick_create_cd self = <azure.cli.testsdk.base.ExecutionResult object at 0x7fe97f082c10>
cli_ctx = <azure.cli.core.mock.DummyCli object at 0x7fe97feae990>
command = 'webapp create -g clitest.rg000001 -n webapp-linux-cd000002 --plan plan-quick-linux-cd -u https://github.com/yugangw-msft/azure-site-test.git&nbsp;-r&nbsp;"NODE
20-lts"'
expect_failure = False

    def in_process_execute(self, cli_ctx, command, expect_failure=False):
        from io import StringIO
        from vcr.errors import CannotOverwriteExistingCassetteException
    
        if command.startswith('az '):
            command = command[3:]
    
        stdout_buf = StringIO()
        logging_buf = StringIO()
        try:
            # issue: stderr cannot be redirect in this form, as a result some failure information
            # is lost when command fails.
>           self.exit_code = cli_ctx.invoke(shlex.split(command), out_file=stdout_buf) or 0
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

src/azure-cli-testsdk/azure/cli/testsdk/base.py:303: 
                                        
env/lib/python3.13/site-packages/knack/cli.py:245: in invoke
    exit_code = self.exception_handler(ex)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
src/azure-cli-core/azure/cli/core/init.py:157: in exception_handler
    return handle_exception(ex)
           ^^^^^^^^^^^^^^^^^^^^
                                       _ 

ex = ValidationError("Linux Runtime 'NODE
Failed test_win_webapp_quick_create_cd The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:174
Failed test_win_webapp_quick_create_runtime The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:156
Failed test_download_win_web_log The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:480
Failed test_webapp_config The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:732
Failed test_linux_webapp The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:1261
Failed test_linux_webapp_remote_ssh The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:1366
Failed test_acr_integration The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:1427
Failed test_webapp_linux_acr_use_identity The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:2592
Failed test_webapp_deployment_source_track_runtimestatus_buildfailed The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:3368
Failed test_webapp_deployment_source_track_runtimestatus_runtimefailed The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:3381
Failed test_webapp_track_runtimestatus_buildfailed The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:3307
Failed test_webapp_track_runtimestatus_runtimefailed The error message is too long, please check the pipeline log for details. azure/cli/command_modules/appservice/tests/latest/test_webapp_commands.py:3320
️✔️aro
️✔️latest
️✔️3.12
️✔️3.13
️✔️backup
️✔️latest
️✔️3.12
️✔️3.13
️✔️batch
️✔️latest
️✔️3.12
️✔️3.13
️✔️batchai
️✔️latest
️✔️3.12
️✔️3.13
️✔️billing
️✔️latest
️✔️3.12
️✔️3.13
️✔️botservice
️✔️latest
️✔️3.12
️✔️3.13
️✔️cdn
️✔️latest
️✔️3.12
️✔️3.13
️✔️cloud
️✔️latest
️✔️3.12
️✔️3.13
️✔️cognitiveservices
️✔️latest
️✔️3.12
️✔️3.13
️✔️compute_recommender
️✔️latest
️✔️3.12
️✔️3.13
️✔️computefleet
️✔️latest
️✔️3.12
️✔️3.13
️✔️config
️✔️latest
️✔️3.12
️✔️3.13
️✔️configure
️✔️latest
️✔️3.12
️✔️3.13
️✔️consumption
️✔️latest
️✔️3.12
️✔️3.13
️✔️container
️✔️latest
️✔️3.12
️✔️3.13
️✔️containerapp
️✔️latest
️✔️3.12
️✔️3.13
️✔️core
️✔️latest
️✔️3.12
️✔️3.13
️✔️cosmosdb
️✔️latest
️✔️3.12
️✔️3.13
️✔️databoxedge
️✔️latest
️✔️3.12
️✔️3.13
️✔️dls
️✔️latest
️✔️3.12
️✔️3.13
️✔️dms
️✔️latest
️✔️3.12
️✔️3.13
️✔️eventgrid
️✔️latest
️✔️3.12
️✔️3.13
️✔️eventhubs
️✔️latest
️✔️3.12
️✔️3.13
️✔️feedback
️✔️latest
️✔️3.12
️✔️3.13
️✔️find
️✔️latest
️✔️3.12
️✔️3.13
️✔️hdinsight
️✔️latest
️✔️3.12
️✔️3.13
️✔️identity
️✔️latest
️✔️3.12
️✔️3.13
️✔️iot
️✔️latest
️✔️3.12
️✔️3.13
️✔️keyvault
️✔️latest
️✔️3.12
️✔️3.13
️✔️lab
️✔️latest
️✔️3.12
️✔️3.13
️✔️managedservices
️✔️latest
️✔️3.12
️✔️3.13
️✔️maps
️✔️latest
️✔️3.12
️✔️3.13
️✔️marketplaceordering
️✔️latest
️✔️3.12
️✔️3.13
️✔️monitor
️✔️latest
️✔️3.12
️✔️3.13
️✔️mysql
️✔️latest
️✔️3.12
️✔️3.13
️✔️netappfiles
️✔️latest
️✔️3.12
️✔️3.13
️✔️network
️✔️latest
️✔️3.12
️✔️3.13
️✔️policyinsights
️✔️latest
️✔️3.12
️✔️3.13
️✔️postgresql
️✔️latest
️✔️3.12
️✔️3.13
️✔️privatedns
️✔️latest
️✔️3.12
️✔️3.13
️✔️profile
️✔️latest
️✔️3.12
️✔️3.13
️✔️rdbms
️✔️latest
️✔️3.12
️✔️3.13
️✔️redis
️✔️latest
️✔️3.12
️✔️3.13
️✔️relay
️✔️latest
️✔️3.12
️✔️3.13
️✔️resource
️✔️latest
️✔️3.12
️✔️3.13
️✔️role
️✔️latest
️✔️3.12
️✔️3.13
️✔️search
️✔️latest
️✔️3.12
️✔️3.13
️✔️security
️✔️latest
️✔️3.12
️✔️3.13
️✔️servicebus
️✔️latest
️✔️3.12
️✔️3.13
️✔️serviceconnector
️✔️latest
️✔️3.12
️✔️3.13
️✔️servicefabric
️✔️latest
️✔️3.12
️✔️3.13
️✔️signalr
️✔️latest
️✔️3.12
️✔️3.13
️✔️sql
️✔️latest
️✔️3.12
️✔️3.13
️✔️sqlvm
️✔️latest
️✔️3.12
️✔️3.13
️✔️storage
️✔️latest
️✔️3.12
️✔️3.13
️✔️synapse
️✔️latest
️✔️3.12
️✔️3.13
️✔️telemetry
️✔️latest
️✔️3.12
️✔️3.13
️✔️util
️✔️latest
️✔️3.12
️✔️3.13
️✔️vm
️✔️latest
️✔️3.12
️✔️3.13

@azure-client-tools-bot-prd
Copy link
Copy Markdown

azure-client-tools-bot-prd Bot commented Apr 30, 2026

️✔️AzureCLI-BreakingChangeTest
️✔️Non Breaking Changes

@yonzhan
Copy link
Copy Markdown
Collaborator

yonzhan commented Apr 30, 2026

Thank you for your contribution! We will review the pull request and get back to you soon.

@github-actions
Copy link
Copy Markdown

The git hooks are available for azure-cli and azure-cli-extensions repos. They could help you run required checks before creating the PR.

Please sync the latest code with latest dev branch (for azure-cli) or main branch (for azure-cli-extensions).
After that please run the following commands to enable git hooks:

pip install azdev --upgrade
azdev setup -c <your azure-cli repo path> -r <your azure-cli-extensions repo path>

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Updates the Azure CLI batch command module to be compatible with Python 3.14 typing/introspection changes, and aligns build/test/release tooling and docs to Python 3.14.

Changes:

  • Reworked batch model type introspection to rely on get_type_hints() / get_args() instead of _name and regex on str() of typing objects.
  • Added additional normalization in type conversion to handle Python 3.14 pipe unions and fully-qualified names.
  • Bumped Python version references across packaging metadata, CI pipelines, release scripts, and docs to 3.14.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/azure-cli/setup.py Advertises Python 3.14 support in package classifiers.
src/azure-cli/azure/cli/command_modules/batch/_command_type.py Fixes typing introspection/type-string normalization for Python 3.14.
src/azure-cli-testsdk/setup.py Advertises Python 3.14 support in package classifiers.
src/azure-cli-telemetry/setup.py Advertises Python 3.14 support in package classifiers.
src/azure-cli-core/setup.py Advertises Python 3.14 support in package classifiers.
scripts/release/macos/cask_generate.py Updates help text example to Python 3.14.
scripts/release/macos/build_binary_tar_gz.py Updates default Python version and packaging layout doc to 3.14.
scripts/release/homebrew/docker/formula_generate.py Updates Homebrew formula generation to Python 3.14.
scripts/release/debian/build.sh Updates embedded Python version used for Debian builds to 3.14.x.
scripts/regression_test/regression_test.yml Runs regression tests using Python 3.14.
doc/install_linux_prerequisites.md Updates stated supported Python range to include 3.14.
doc/extensions/authoring.md Updates extension authoring guidance/examples to Python 3.14.
doc/command_guidelines.md Updates coding practices to require Python 3.14 compatibility.
build_scripts/windows/scripts/build.cmd Updates embedded Python version used for Windows builds to 3.14.x.
azure-pipelines.yml Updates CI matrix/defaults and tasks to use Python 3.14.
azure-pipelines-full-tests.yml Updates full tests pipeline to Python 3.14.
.azure-pipelines/templates/macos/macos-sign-notarize-jobs.yml Updates macOS signing/notarization template default Python to 3.14.
.azure-pipelines/templates/macos/macos-publish-jobs.yml Updates macOS publish template default Python to 3.14.
.azure-pipelines/templates/macos/macos-cask-generation-and-tests.yml Updates macOS cask generation/tests template default Python to 3.14.
.azure-pipelines/templates/macos/macos-build-jobs.yml Updates macOS build template docs/default Python to 3.14.
.azure-pipelines/macos-standalone-release.yml Updates standalone macOS release pipeline default Python to 3.14.
.azure-pipelines/breaking-change-tests.yml Updates breaking change tests to run on Python 3.14.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/azure-cli/azure/cli/command_modules/batch/_command_type.py Outdated
Comment thread src/azure-cli/azure/cli/command_modules/batch/_command_type.py Outdated
Comment thread src/azure-cli/azure/cli/command_modules/batch/_command_type.py Outdated
Comment thread src/azure-cli/azure/cli/command_modules/batch/_command_type.py Outdated
In Python 3.14, several breaking changes affect how type annotations are
represented:
- The private _name attribute was removed from Union/Optional types
- str(Optional[X]) now produces X | None (pipe syntax) instead of
  	yping.Optional[X]
- get_type_hints() resolves ForwardRef strings to actual classes, so
  str() on resolved types produces fully-qualified names like
  �zure.batch.models._models.Foo instead of ForwardRef('_models.Foo')

Fix by replacing all version-specific introspection with stable public APIs:

1. get_optional_state(): replace _name == 'Optional' check with
   	ype(None) in get_args(type_hint), which works identically in all
   Python versions.

2. get_track1_attribute_map(): replace raw cls.__annotations__
   iteration (which contains unresolved ForwardRef strings) with
   get_type_hints() to get resolved types, then use get_args() to
   detect optional/union patterns instead of regex on string
   representations.

3. convert_to_track1_type(): add handlers for:
   - Top-level pipe union syntax: A | None -> A
   - Inner pipe union syntax: List[str | SomeType] -> List[SomeType]
   - <enum 'Name'> string representations (in addition to <class>)
   - Fully-qualified �zure.batch.models. and _enums. prefixes
     produced when get_type_hints() resolves ForwardRefs to real classes

Verified: batch.json doc output is bit-for-bit identical between
Python 3.13 and Python 3.14 after this fix.
- prefer non-str member for top-level pipe unions
- make inner pipe cleanup regex robust to whitespace
- add _enums alias mapping for get_type_hints
- remove debug print from attribute map
- keep _models alias mapped to azure.batch.models exports
@YangAn-microsoft YangAn-microsoft force-pushed the fix/batch-python314-union-type branch from 93e65ca to 14dc288 Compare April 30, 2026 05:59
@YangAn-microsoft YangAn-microsoft changed the title Fix batch module Python 3.14 typing compatibility [Batch] Fix batch module Python 3.14 typing compatibility Apr 30, 2026
@YangAn-microsoft YangAn-microsoft changed the title [Batch] Fix batch module Python 3.14 typing compatibility [Batch] Fix #33284: az batch: Fix Python 3.14 typing compatibility Apr 30, 2026
@YangAn-microsoft YangAn-microsoft changed the title [Batch] Fix #33284: az batch: Fix Python 3.14 typing compatibility [Batch] Fix #33284: Fix Python 3.14 typing compatibility for batch module Apr 30, 2026
@YangAn-microsoft YangAn-microsoft changed the title [Batch] Fix #33284: Fix Python 3.14 typing compatibility for batch module [Batch] Fix Python 3.14 typing compatibility for batch module Apr 30, 2026
@YangAn-microsoft YangAn-microsoft changed the title [Batch] Fix Python 3.14 typing compatibility for batch module [Batch] Fix Python 3.14 typing compatibility for batch module Fix #33284 Apr 30, 2026
@YangAn-microsoft YangAn-microsoft changed the title [Batch] Fix Python 3.14 typing compatibility for batch module Fix #33284 [Batch] Fix #33284: Fix Python 3.14 typing compatibility for batch module Apr 30, 2026
@YangAn-microsoft YangAn-microsoft changed the title [Batch] Fix #33284: Fix Python 3.14 typing compatibility for batch module [Batch] Fix #33284: Fix Python 3.14 typing compatibility for batch module Apr 30, 2026
@YangAn-microsoft
Copy link
Copy Markdown
Author

YangAn-microsoft commented May 1, 2026

The remaining 2 failed checks are unrelated to this batch change.
They match the App Service Node 20 EOL test break and are addressed in #33302.
After #33302 merges, I will sync with latest dev and rerun the failed checks.

@YangAn-microsoft
Copy link
Copy Markdown
Author

YangAn-microsoft commented May 1, 2026

Hi @cRui861 @wanghoppe @dpwatrous @wiboris,
Could you please help review this PR for Python 3.14 upgrade work? Thank you.

Note: 2 remaining check failures are expected to be addressed by #33302

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants