[HCI-VN] Error during upgrade: Virtual Network upgrade post-processing failed
Problem Description
690 upgrade to 691, upgrade process error: Virtual Network upgrade post-processing failed
Effective Troubleshooting Steps
1.690 upgrade to 691 error during upgrade: Virtual Network upgrade post-processing failed

2. Check the upgrade log in the background and find that the removal of the old version of the image failed, and the error is that the image is being referenced

3. Check the running status of the container in the background and find that the old version of the container has exited and the new version is running

4. After investigation, the container group colleagues found that the vstone task occupied the corresponding image layer. The reason was that a vstone-hci-6.9.0-x86_64:minimal-latest was manually deployed in the environment before the upgrade. This manually added container will not be processed during the upgrade process.
The container continues to run and occupies the same image layer as vn-controller-6.9.0, causing vn-controller to fail to clean up the old image.
Execute the following command to view container information:
ctr -n vstone c ls
ctr -n vstone i ls

5. Manually close the vstone container and retry the upgrade. The upgrade is successful
docker -n vstone rm -f vstone;
docker -n vstone rmi docker.sangfor.com/cloud_test_docker/vstone-hci-6.9.0-x86_64:minimal-latest;
docker -n vstone ps
Note: The image name in the second command above is the IMAGE of the ctr -n vstone c ls execution result:

Root Cause
Before the upgrade, a vstone-hci-6.9.0-x86_64:minimal-latest container was manually deployed in the environment. This manually added container will not be processed during the upgrade process.
The container continues to run and occupies the same image layer as vn-controller-6.9.0, causing vn-controller to fail to clean up the old image.
Solution
Manually shut down the container and delete the image
Scope of Operation Impact
None
Is this a temporary solution?
Disabled
Original Link
https://support.sangfor.com.cn/cases/list?product_id=33&type=1&category_id=26861&isOpen=true