Skip to content

VMware: wait for NSX DV port group readiness - #13753

Open
Dogface2k wants to merge 3 commits into
apache:4.22from
Dogface2k:agent/wait-for-nsx-dvportgroup
Open

Dogface2k wants to merge 3 commits into
apache:4.22from
Dogface2k:agent/wait-for-nsx-dvportgroup

Conversation

@Dogface2k

Copy link
Copy Markdown
Collaborator

Description

This PR prevents VMware NSX-backed guest network implementation from failing when vCenter has not yet exposed the backing distributed virtual port group immediately after NSX segment creation.

The NSX branch in HypervisorHostHelper.prepareNetwork now uses the existing bounded waitForDvPortGroupReady path. The change does not retry the CloudStack mutation or create a second network; it only waits for vCenter inventory visibility before returning the already-created port group.

A regression test simulates an empty first vCenter lookup followed by the port group becoming available. The existing immediate-availability behavior remains covered.

Fixes: #13752

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • Build/CI
  • Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

Not applicable.

How Has This Been Tested?

Targeted regression and baseline tests were run on the 4.22 branch with Java 17:

HypervisorHostHelperTest#testPrepareNetwork
HypervisorHostHelperTest#testPrepareNetworkWaitsForNsxDvPortGroup

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

The Maven reactor built all 18 required modules successfully, including cloud-vmware-base, with zero Checkstyle violations.

How did you try to break this feature and the system with this change?

The regression test makes the first vCenter DV port group query return no result and the second return the expected port group. The pre-existing immediate-availability test also runs to verify the normal fast path remains unchanged. The production path remains bounded by the existing readiness timeout and returns failure if the port group never appears.

@boring-cyborg

boring-cyborg Bot commented Jul 30, 2026

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache CloudStack community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md)
Here are some useful points:

@Dogface2k
Dogface2k marked this pull request as ready for review July 30, 2026 15:30
@DaanHoogland DaanHoogland added this to the 4.22.2 milestone Jul 31, 2026

@DaanHoogland DaanHoogland left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clgtm

@codecov

codecov Bot commented Jul 31, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 17.85%. Comparing base (74e7105) to head (a779e96).
⚠️ Report is 30 commits behind head on 4.22.

Additional details and impacted files
@@            Coverage Diff            @@
##               4.22   #13753   +/-   ##
=========================================
  Coverage     17.84%   17.85%           
- Complexity    16025    16031    +6     
=========================================
  Files          5928     5928           
  Lines        534457   534457           
  Branches      65404    65404           
=========================================
+ Hits          95388    95425   +37     
+ Misses       428239   428199   -40     
- Partials      10830    10833    +3     
Flag Coverage Δ
uitests 4.02% <ø> (ø)
unittests 18.91% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18740

@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan help

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland [SL] I understand these words: "help", "hello", "thanks", "package", "test"
Test command usage: test [mgmt os] [hypervisor] [keepEnv] [qemuEv] [basicZone|securityGroups]
Mgmt OS options: ['suse15', 'alma10', 'ol10', 'rocky10', 'alma9', 'centos7', 'centos6', 'rocky9', 'alma8', 'ubuntu18', 'ol9', 'ol8', 'ubuntu22', 'debian12', 'ubuntu20', 'ubuntu26', 'rocky8', 'ubuntu24']
Hypervisor options: ['kvm-centos6', 'kvm-centos7', 'kvm-rocky8', 'kvm-rocky9', 'kvm-rocky10', 'kvm-ol8', 'kvm-ol9', 'kvm-ol10', 'kvm-alma8', 'kvm-alma9', 'kvm-alma10', 'kvm-ubuntu18', 'kvm-ubuntu20', 'kvm-ubuntu22', 'kvm-ubuntu24', 'kvm-ubuntu26', 'kvm-debian12', 'kvm-suse15', 'vmware-55u3', 'vmware-60u2', 'vmware-65u2', 'vmware-67u3', 'vmware-70u1', 'vmware-70u2', 'vmware-70u3', 'vmware-80', 'vmware-80u1', 'vmware-80u2', 'vmware-80u3', 'vmware-80u3e', 'xenserver-65sp1', 'xenserver-71', 'xenserver-74', 'xenserver-84', 'xcpng74', 'xcpng76', 'xcpng80', 'xcpng81', 'xcpng82', 'xcpng83']
Note: when keepEnv is passed, you need to specify mgmt server os and hypervisor or use the matrix command.
when qemuEv is passed, it will deploy KVM hyperviosr hosts with qemu-kvm-ev, else it will default to stock qemu.
When basicZone and/or securityGroups are passed it will create a zone of the last type specified (default is Advanced)
Package command usage: package [all(default value),kvm,xen,vmware,hyperv,ovm] - a comma separated list can be passed with package command to bundle the required hypervisor's systemVM templates. Not passing any argument will bundle all - kvm,xen and vmware templates.

Blessed contributors for kicking Trillian test jobs: ['rohityadavcloud', 'shwstppr', 'Damans227', 'vishesh92', 'Pearl1594', 'harikrishna-patnala', 'nvazquez', 'DaanHoogland', 'weizhouapache', 'borisstoyanov', 'vladimirpetrov', 'kiranchavala', 'andrijapanicsb', 'NuxRo', 'rajujith', 'sureshanaparti', 'abh1sar', 'sudo87', 'RosiKyu']

@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan test ol10 vmware-80u3

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a [SL] Trillian-Jenkins test job (ol10 mgmt + vmware-80u3) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

[SF] Trillian Build Failed (tid-16671)

@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan test ol9 vmware-80u3

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a [SL] Trillian-Jenkins test job (ol9 mgmt + vmware-80u3) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

[SF] Trillian Build Failed (tid-16791)

@Pearl1594 Pearl1594 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code lgtm. I don't have a test env to test this fix. But it looks pretty straightforward. Thanks @Dogface2k

@Pearl1594

Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19155

@Pearl1594

Copy link
Copy Markdown
Contributor

@blueorangutan test ol9 vmware-80u3

@blueorangutan

Copy link
Copy Markdown

@Pearl1594 a [SL] Trillian-Jenkins test job (ol9 mgmt + vmware-80u3) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

[SF] Trillian test result (tid-16935)
Environment: vmware-80u3 (x2), zone: Advanced Networking with Mgmt server ol9
Total time taken: 303029 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr13753-t16935-vmware-80u3.zip
Smoke tests completed. 137 look OK, 12 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_02_enableHumanReadableLogs Error 0.17 test_human_readable_logs.py
test_list_vms_metrics_admin Error 3623.52 test_metrics_api.py
test_list_vms_metrics_history Error 5.99 test_metrics_api.py
test_list_volumes_metrics_history Error 3624.95 test_metrics_api.py
test_01_vpn_usage Error 1.12 test_usage.py
test_01_deploy_vm_on_specific_host Error 19.07 test_vm_deployment_planner.py
test_02_deploy_vm_on_specific_cluster Error 3605.68 test_vm_deployment_planner.py
test_03_deploy_vm_on_specific_pod Error 4.59 test_vm_deployment_planner.py
test_04_deploy_vm_on_host_override_pod_and_cluster Error 4.57 test_vm_deployment_planner.py
test_05_deploy_vm_on_cluster_override_pod Error 3604.33 test_vm_deployment_planner.py
test_02_offline_migrate_VM_with_two_data_disks Error 3658.75 test_vm_life_cycle.py
test_03_live_migrate_VM_with_two_data_disks Error 45.83 test_vm_life_cycle.py
test_08_migrate_vm Error 3629.38 test_vm_life_cycle.py
test_09_expunge_vm Failure 427.85 test_vm_life_cycle.py
test_10_attachAndDetach_iso Failure 48.74 test_vm_life_cycle.py
test_11_destroy_vm_and_volumes Error 1842.75 test_vm_life_cycle.py
test_13_destroy_and_expunge_vm Error 1973.62 test_vm_life_cycle.py
test_01_migrate_vm_strict_tags_success Error 8.91 test_vm_strict_host_tags.py
test_02_migrate_vm_strict_tags_failure Error 8.83 test_vm_strict_host_tags.py
test_01_restore_vm_strict_tags_success Error 8.91 test_vm_strict_host_tags.py
test_02_restore_vm_strict_tags_failure Error 8.95 test_vm_strict_host_tags.py
test_01_scale_vm_strict_tags_success Error 8.96 test_vm_strict_host_tags.py
test_02_scale_vm_strict_tags_failure Error 8.96 test_vm_strict_host_tags.py
test_01_deploy_vm_on_specific_host_without_strict_tags Error 8.77 test_vm_strict_host_tags.py
test_02_deploy_vm_on_any_host_without_strict_tags Error 11.05 test_vm_strict_host_tags.py
test_03_deploy_vm_on_specific_host_with_strict_tags_success Error 8.74 test_vm_strict_host_tags.py
test_04_deploy_vm_on_any_host_with_strict_tags_success Error 11.87 test_vm_strict_host_tags.py
test_04_deploy_vnf_appliance Error 2025.08 test_vnf_templates.py
test_05_delete_vnf_template Error 1.11 test_vnf_templates.py
ContextSuite context=TestVnfTemplates>:teardown Error 2.23 test_vnf_templates.py
ContextSuite context=TestCreateVolume>:teardown Error 2234.97 test_volumes.py
test_11_attach_volume_with_unstarted_vm Error 1930.82 test_volumes.py
test_13_migrate_volume_and_change_offering Error 2410.60 test_volumes.py
ContextSuite context=TestVolumes>:teardown Error 1824.06 test_volumes.py
test_01_verify_ipv6_vpc Error 2374.56 test_vpc_ipv6.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL Error 3611.54 test_vpc_redundant.py
test_02_redundant_VPC_default_routes Error 3611.65 test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers Error 3609.80 test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics Error 3610.49 test_vpc_redundant.py
test_05_rvpc_multi_tiers Error 3686.67 test_vpc_redundant.py
test_01_VPC_nics_after_destroy Failure 5771.22 test_vpc_router_nics.py
test_02_VPC_default_routes Error 3609.75 test_vpc_router_nics.py
test_01_redundant_vpc_site2site_vpn Failure 3687.56 test_vpc_vpn.py
test_01_redundant_vpc_site2site_vpn Error 3687.82 test_vpc_vpn.py
test_01_vpc_site2site_vpn_multiple_options Failure 3687.51 test_vpc_vpn.py
test_01_vpc_site2site_vpn_multiple_options Error 3687.76 test_vpc_vpn.py
test_01_vpc_remote_access_vpn Error 1957.89 test_vpc_vpn.py
test_01_vpc_site2site_vpn Failure 3614.47 test_vpc_vpn.py
test_01_vpc_site2site_vpn Error 3614.83 test_vpc_vpn.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Ready

Development

Successfully merging this pull request may close these issues.

VMware NSX network implementation can fail before backing DV port group is visible

5 participants