Increasing CPU Cycle Reservation for Orchestrator VMs
Cisco ACI Multi-Site Orchestrator VMs require a certain amount of dedicated CPU cycles. While new deployments apply CPU cycle reservation automatically, if you upgrade the Orchestrator from a release prior to Release 2.1(1), you will need to update each Orchestrator VM's settings manually.
Configuring appropriate CPU cycle reservation can resolve or help prevent a number of seemingly random issues, such as:
-
Orchestrator GUI items requiring one or more retries to load.
-
One or more nodes changing to
Unknown
status and then resolving back toReady
some time later on its own:# docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION t8wl1zoke0vpxdl9fysqu9otb node1 Ready Active Reachable 18.03.0-ce kyriihdfhy1k1tlggan6e1ahs * node2 Unknown Active Reachable 18.03.0-ce yburwactxd86dorindmx8b4y1 node3 Ready Active Leader 18.03.0-ce
-
Transient heartbeat misses in the Orchestrator logs (located in
/var/log/messages
) with the following example log entries:node2 dockerd: [...] level=error msg="agent: session failed" backoff=100ms error="rpc error: code = Canceled desc = context canceled" module=node/agent [...] node2 dockerd: [...] level=error msg="heartbeat to manager [...] failed" error="rpc error: code = Canceled desc = context canceled" [...]
To update the CPU cycle reservation setting, repeat the following steps for each Orchestrator VM:
Procedure
Step 1 |
Log in to the vSphere client. |
Step 2 |
Navigate to the ESX host where your Orchestrator VM is located. |
Step 3 |
Shut down the VM. |
Step 4 |
Right click the VM and choose Edit Settings |
Step 5 |
In the Virtual Hardware tab, expand the CPU category. |
Step 6 |
In the Reservation field, enter |
Step 7 |
Click OK to save the changes. |
Step 8 |
Power on the VM and wait for the Orchestrator cluster to stabilize with all nodes healthy. |