Skip to content

OCPBUGS-77412: reload router if route is changed on DCM#739

Open
jcmoraisjr wants to merge 1 commit intoopenshift:masterfrom
jcmoraisjr:OCPBUGS-77412-reload-on-route-changes
Open

OCPBUGS-77412: reload router if route is changed on DCM#739
jcmoraisjr wants to merge 1 commit intoopenshift:masterfrom
jcmoraisjr:OCPBUGS-77412-reload-on-route-changes

Conversation

@jcmoraisjr
Copy link
Member

@jcmoraisjr jcmoraisjr commented Feb 26, 2026

DCM (dynamic update) is removing+adding routes whenever they change. This causes disruption on the changed route, and can cause a longer outage in case the API call to add the route fails.

This update is skipping the dynamic update in case of any route change, either creation, change and deletion. This is the simplest approach for the current release, a proper fix on the API calls should be done post 4.22 in the scope of https://issues.redhat.com/browse/OCPBUGS-77344

The dynamic update skip is being added just after the route registration on the AddRoute method, this ensures that the free slots can be found in case of scale out. If skipping the registration, the free slots cannot be found and scale in/out would force a reload.

@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. labels Feb 26, 2026
@openshift-ci-robot
Copy link
Contributor

@jcmoraisjr: This pull request references Jira Issue OCPBUGS-77412, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.22.0) matches configured target version for branch (4.22.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @ShudiLi

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

DCM (dynamic update) is removing+adding routes whenever they change. This causes disruption on the changed route, and can cause a longer outage in case the API call to add the route fails.

This update is skipping the dynamic update in case of any route change, either creation, change and deletion. This is the simplest approach for the current release, a proper fix on the API calls should be done post 4.22 in the scope of https://issues.redhat.com/browse/OCPBUGS-77412

The dynamic update skip is being added just after the route registration on the AddRoute method, this ensures that the free slots can be found in case of scale out. If skipping the registration, the free slots cannot be found and scale in/out would force a reload.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested review from ShudiLi, bentito and gcs278 February 26, 2026 13:33
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Feb 26, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign jcmoraisjr for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot
Copy link
Contributor

@jcmoraisjr: This pull request references Jira Issue OCPBUGS-77412, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.22.0) matches configured target version for branch (4.22.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @ShudiLi

Details

In response to this:

DCM (dynamic update) is removing+adding routes whenever they change. This causes disruption on the changed route, and can cause a longer outage in case the API call to add the route fails.

This update is skipping the dynamic update in case of any route change, either creation, change and deletion. This is the simplest approach for the current release, a proper fix on the API calls should be done post 4.22 in the scope of https://issues.redhat.com/browse/OCPBUGS-77344

The dynamic update skip is being added just after the route registration on the AddRoute method, this ensures that the free slots can be found in case of scale out. If skipping the registration, the free slots cannot be found and scale in/out would force a reload.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

DCM (dynamic update) is removing+adding routes whenever they change.
This causes disruption on the changed route, and can cause a longer
outage in case the API call to add the route fails.

This update is skipping the dynamic update in case of any route change,
either creation, change and deletion. This is the simplest approach for
the current release, a proper fix on the API calls should be done post
4.22 in the scope of https://issues.redhat.com/browse/OCPBUGS-77344

The dynamic update skip is being added just after the route registration
on the AddRoute method, this ensures that the free slots can be found in
case of scale out. If skipping the registration, the free slots cannot
be found and scale in/out would force a reload.
@jcmoraisjr jcmoraisjr force-pushed the OCPBUGS-77412-reload-on-route-changes branch from 1dc432a to 32b14b1 Compare February 26, 2026 13:48
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Feb 26, 2026

@jcmoraisjr: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@ShudiLi
Copy link
Member

ShudiLi commented Feb 27, 2026

tested it with 4.22.0-0-2026-02-27-012336-test-ci-ln-y144372-latest

1.
% oc get clusterversion
NAME      VERSION                                                AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.22.0-0-2026-02-27-012336-test-ci-ln-y144372-latest   True        False         28m     Cluster version is 4.22.0-0-2026-02-27-012336-test-ci-ln-y144372-latest

2.
 % oc -n openshift-ingress get deploy/router-default -oyaml | grep -A1 "/usr/bin/openshift-router"
        - /usr/bin/openshift-router
        - --v=5

3.
% oc get route
NAME           HOST/PORT                                                              PATH   SERVICES                                               PORT          TERMINATION   WILDCARD
unsec-route2   unsec-route2-default.apps.ci-ln-y144372-76ef8.aws-4.ci.openshift.org          unsec-apach(42%),unsec-apach2(28%),unsec-apach3(28%)   unsec-apach                 None

4.
% oc get deployment 
NAME             READY   UP-TO-DATE   AVAILABLE   AGE
appach-server    2/2     2            2           32m
appach-server2   2/2     2            2           32m
appach-server3   2/2     2            2           32m

5.
 % oc -n openshift-ingress rsh router-default-b7864749c-646s9
sh-5.1$ echo show info | socat - /var/lib/haproxy/run/haproxy.sock |grep Pid
Pid: 89

6. scale up/down deployments
 % oc scale deployment appach-server  --replicas=3
deployment.apps/appach-server scaled
 % oc scale deployment appach-server  --replicas=1            
deployment.apps/appach-server scaled
 % oc scale deployment appach-server2  --replicas=3
deployment.apps/appach-server2 scaled
% oc scale deployment appach-server3  --replicas=3
deployment.apps/appach-server3 scaled
% oc get deployment 
NAME             READY   UP-TO-DATE   AVAILABLE   AGE
appach-server    1/1     1            1           34m
appach-server2   3/3     3            3           34m
appach-server3   3/3     3            3           34m

7. Check the router pod logs
% oc -n openshift-ingress logs router-default-b7864749c-646s9 | grep map | grep add 
% oc -n openshift-ingress logs router-default-b7864749c-646s9 | grep map | grep del
%

8. Check the Pid, which wasn't changed as expected
% oc -n openshift-ingress rsh router-default-b7864749c-646s9 
sh-5.1$ echo show info | socat - /var/lib/haproxy/run/haproxy.sock |grep Pid
Pid: 89

@ShudiLi
Copy link
Member

ShudiLi commented Feb 27, 2026

/label qe-approved
/verified by @ShudiLi

@openshift-ci openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label Feb 27, 2026
@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Feb 27, 2026
@openshift-ci-robot
Copy link
Contributor

@ShudiLi: This PR has been marked as verified by @ShudiLi.

Details

In response to this:

/label qe-approved
/verified by @ShudiLi

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. qe-approved Signifies that QE has signed off on this PR verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants