Procedura di ridistribuzione per Ultra-M AutoVNF

Introduzione

In questo documento vengono descritti i passaggi necessari per ridistribuire una funzionalità AutoVNF in un modulo Ultra-M. La funzionalità AutoVNF è responsabile della configurazione di singole funzionalità VNFM (Virtual Network Function Manager) e VNF (Virtual Network Function). 

Controlli preliminari

1. Accedere a OpenStack Platform Director (OSPD) e verificare lo stato delle macchine virtuali (VM) nel rapporto di stato.

[stack@labucs300-ospd ~]$ cat /var/log/cisco/ultram-health/*.report | grep -i xxx

[stack@labucs300-ospd ~]$ cat /var/log/cisco/ultram-health/ultram_health_uas.report
---------------------------------------------------------------------------------------------------------
 VNF-ID/VNFD-ID           | UAS Node | Status| Error Info, if any         
---------------------------------------------------------------------------------------------------------
10.10.10.40/LABPGW300-UAS | autovnf  | :-)   | LABPGW300-UAS:(alive)
                          |          |       |   labucs300-UAS-LABPGW300-UAS-core-UAS2-2:(alive)
                          |          |       |   labucs300-UAS-LABPGW300-UAS-core-UAS2-1:(alive)
10.10.10.40/LABPCF300-UAS | autovnf  | :-)   | LABPCF300-UAS:(alive)
                          |          |       |   labucs300-UAS-LABPCF300-UAS-core-UAS1-2:(alive)
                          |          |       |   labucs300-UAS-LABPCF300-UAS-core-UAS1-1:(alive)
10.10.10.45/LABPCF300-UGP | vnf-em   | :-)   | LABPCF300-UGP:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-EM1-3:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-EM1-2:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-EM1-1:(alive)
10.10.10.45/LABPCF300-ESC | esc      | :-)   | LABPCF300-ESC:(alive)
                          |          |       |   LABPCF300-LABPCF300-ESC-core-ESC1-1:(alive)
                          |          |       |   LABPCF300-LABPCF300-ESC-core-ESC1-2:(alive)
10.10.10.45/LABPCF300-UGP | vnf      | :-)   | LABPCF300-UGP:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-LABPCF300-CF-VDU-1:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-LABPCF300-CF-VDU-0:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-LABPCF300-SF-VDU-3:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-LABPCF300-SF-VDU-2:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-LABPCF300-SF-VDU-1:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-LABPCF300-SF-VDU-0:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-LABPCF300-SF-VDU-6:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-LABPCF300-SF-VDU-5:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-LABPCF300-SF-VDU-4:(alive)
10.10.10.48/LABPGW300-UGP | vnf-em   | :-)   | LABPGW300-UGP:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-EM2-2:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-EM2-3:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-EM2-1:(alive)
10.10.10.48/LABPGW300-ESC | esc      | :-)   | LABPGW300-ESC:(alive)
                          |          |       |   LABPGW300-LABPGW300-ESC-core-ESC2-1:(alive)
                          |          |       |   LABPGW300-LABPGW300-ESC-core-ESC2-2:(alive)
10.10.10.48/LABPGW300-UGP | vnf      | :-)   | LABPGW300-UGP:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-LABPGW300-SF-VDU-4:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-LABPGW300-SF-VDU-5:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-LABPGW300-SF-VDU-6:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-LABPGW300-SF-VDU-0:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-LABPGW300-SF-VDU-1:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-LABPGW300-SF-VDU-2:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-LABPGW300-SF-VDU-3:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-LABPGW300-CF-VDU-0:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-LABPGW300-CF-VDU-1:(alive)
---------------------------------------------------------------------------------------------------------

2. Verificare lo stato di AutoVNF.

[stack@labucs300-ospd ~]$ source *core
[stack@labucs300-ospd ~]$ nova list | grep LABPGW300-UAS-core-UAS2
| 8608fda4-b763-4753-95ff-2e07852098e3 | labucs300-UAS-LABPGW300-UAS-core-UAS2-1 | ACTIVE | - | Running | labucs300-UAS-LABPGW300-ORCH-NW=192.0.2.15; labucs300-UAS-LABPGW300-MGMT-NW=192.0.2.7 |
| 19f4496c-3907-4ea5-84c9-e5a6ef222392 | labucs300-UAS-LABPGW300-UAS-core-UAS2-2 | ACTIVE | - | Running | labucs300-UAS-LABPGW300-ORCH-NW=192.0.2.17; labucs300-UAS-LABPGW300-MGMT-NW=192.0.2.16

labucs300-UAS-LABPGW300-UAS-core-UAS2-2 -> VNFM2-UAS-VIP Primary
labucs300-UAS-LABPGW300-UAS-core-UAS2-1 -> Secondary 

3. Accedere ad AutoIT e controllare i log di heartbeat.

ubuntu@labucs300-autoit-2:~$ grep "'restarting'" /var/log/cisco/uas/heartbeat.log
2021-02-22 01:41:42,808 - 192.0.2.15: Notify Event: {'action': 'restart', 'source': 'heartbeat', 'event': 'restarting', 'ip': '192.0.2.15'}
2021-02-22 01:45:42,251 - 192.0.2.15: Notify Event: {'action': 'restart', 'source': 'heartbeat', 'event': 'restarting', 'ip': '192.0.2.15'}
2021-02-23 01:43:36,013 - 192.0.2.15: Notify Event: {'action': 'restart', 'source': 'heartbeat', 'event': 'restarting', 'ip': '192.0.2.15'}
2021-02-23 01:45:55,785 - 192.0.2.15: Notify Event: {'action': 'restart', 'source': 'heartbeat', 'event': 'restarting', 'ip': '192.0.2.15'}
2021-02-24 01:45:19,680 - 192.0.2.15: Notify Event: {'action': 'restart', 'source': 'heartbeat', 'event': 'restarting', 'ip': '192.0.2.15'}

ubuntu@labucs300-autoit-2:~$ cd /var/log/cisco/uas
ubuntu@labucs300-autoit-2:/var/log/cisco/uas$ grep "Rebooting Instance" uas_USPCHBWorker.log
2019-06-26 18:26:13,088 - Rebooting Instance: 19f4496c-3907-4ea5-84c9-e5a6ef222392
2019-06-29 03:45:12,710 - Rebooting Instance: 19f4496c-3907-4ea5-84c9-e5a6ef222392
2020-07-17 00:46:25,800 - Rebooting Instance: 19f4496c-3907-4ea5-84c9-e5a6ef222392
2020-07-18 00:47:13,347 - Rebooting Instance: 19f4496c-3907-4ea5-84c9-e5a6ef222392
2020-07-18 05:11:11,133 - Rebooting Instance: 19f4496c-3907-4ea5-84c9-e5a6ef222392
2020-07-18 5:16:07,333 - Rebooting Instance: 19f4496c-3907-4ea5-84c9-e5a6ef222392 

Backup AutoVNF

AutoVNF è responsabile della visualizzazione di singole VNFM e VNF. AutoDeploy invia a AutoVNF la configurazione necessaria per creare un'istanza sia di VNFM che di VNF, mentre AutoVNF esegue questa operazione. Per far conoscere la VNFM,

AutoVNF comunica direttamente con VIM/openstack e, una volta attivato VNFM, utilizza VNFM per attivare VNF.

AutoVNF dispone di ridondanza 1:1 e in configurazione UltraM, 2 VM AutoVNF in esecuzione in modalità HA.

Dettagli backup AutoVNF:

• Configurazione in esecuzione
• DB CDB ConfD
• Registri AutoVNF (da ogni istanza di AutoVNF)
• Configurazione Syslog

Importante: I backup devono essere eseguiti prima di attivare/disattivare il POD/sito specificato e caricati nel server di backup. 

1. Impostare ha_debug su ON su AutoIT principale e secondario in /opt/uas_baseconfig.txt.

Nota: Il flag ha_debug è tutto in maiuscolo. È ON o OFF.

ubuntu@labucs300-autoit-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 172.16.181.7
ha-secret: f99d04acb84807c4c6c6c0eaad392b5c
orch-ips: 172.16.181.5,172.16.181.8
orch-intf: eth0
prov-ha-vip: 172.16.181.13
prov-intf: eth0
profile: AUTOIT
ha_debug: ON

ubuntu@labucs300-autoit-1:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 172.16.181.7
ha-secret: f99d04acb84807c4c6c6c0eaad392b5c
orch-ips: 172.16.181.5,172.16.181.8
orch-intf: eth0
prov-ha-vip: 172.16.181.13
prov-intf: eth0
profile: AUTOIT
ha_debug: ON

2. Arrestare il servizio AutoIT su AutoIT principale con questi comandi:

ssh   ubuntu@ < AutoIT Floating IP>
sudo -i
service autoit stop

In questo modo, AutoIT non attiva il ripristino automatico di Ultra Automation Services (UAS) quando la chiusura viene avviata in fasi successive.

ubuntu@labucs300-autoit-2:~$ sudo -i
root@labucs300-autoit-2:~#
root@labucs300-autoit-2:~# service autoit status
autoit start/running, process 25001
root@labucs300-autoit-2:~# service autoit stop

3. Impostare ha_debug su ON su AutoVNF (UAS) primario e secondario nel file /opt/uas_baseconfig.txt.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 192.0.2.8
ha-secret: d5a9fee60ddae4fe357677dcf1320e51
orch-ips: 192.0.2.15,192.0.2.17
orch-intf: eth0
profile: AUTOVNF
ha_debug: ON

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ ssh ubuntu@192.0.2.15
ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-1:~$ cat /opt/uas_baseconfig.txt
ha: true
ha-vip: 192.0.2.8
ha-secret: d5a9fee60ddae4fe357677dcf1320e51
orch-ips: 192.0.2.15,192.0.2.17
orch-intf: eth0
profile: AUTOVNF
ha_debug: ON

4. Arrestare gli UAS secondari da OSPD con i relativi commenti.

 . corerc ; openstack server stop <VMName>

labucs300-UAS-LABPGW300-UAS-core-UAS2-2 -> VNFM2-UAS-VIP Primary
labucs300-UAS-LABPGW300-UAS-core-UAS2-1 -> Secondary

[stack@labucs300-ospd ~]$ nova list | grep LABPGW300-UAS-core-UAS2
| 8608fda4-b763-4753-95ff-2e07852098e3 | labucs300-UAS-LABPGW300-UAS-core-UAS2-1 | ACTIVE | - | Running | labucs300-UAS-LABPGW300-ORCH-NW=192.0.2.15; labucs300-UAS-LABPGW300-MGMT-NW=192.0.2.7 |
| 19f4496c-3907-4ea5-84c9-e5a6ef222392 | labucs300-UAS-LABPGW300-UAS-core-UAS2-2 | ACTIVE | - | Running | labucs300-UAS-LABPGW300-ORCH-NW=192.0.2.17; labucs300-UAS-LABPGW300-MGMT-NW=192.0.2.16

[stack@labucs300-ospd ~]$. corerc ; openstack server stop labucs300-UAS-LABPGW300-UAS-core-UAS2-1

5. Arrestare i servizi uas-confd e autonf su UAS con questi comandi:

service uas-confd stop
service autovnf stop

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ sudo -i 
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service uas-confd status
uas-confd start/running, process 1305
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service autovnf status
autovnf start/running, process 24208
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service uas-confd stop
uas-confd stop/waiting
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service autovnf stop
autovnf stop/waiting

6. Eseguire un backup del database di configurazione UAS e copiarlo su un server di backup con i seguenti comandi:

cd /opt/cisco/usp/uas/confd-latest/var/confd/
tar -cvf <pod>_<VNF>_UAS_cdb_backup.tar cdb/

root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# cd /opt/cisco/usp/uas/confd-latest/var/confd/
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# tar -cvf Autovnf_cdb_backup.tar cdb/
cdb/
cdb/O.cdb
cdb/C.cdb
cdb/aaa_init.xml
cdb/A.cdb
root@labucs300-uas-LABPGW300-uas-core-uas2-2:/opt/cisco/usp/uas/confd-latest/var/confd# ll Autovnf_cdb_backup_cdb_backup.tar 
total 1612
drwxr-xr-x 3 root root 4096 Jan 24 2017 ..
drwxr-xr-x 2 root root 4096 Jan 24 2017 log
drwxr-xr-x 8 root root 4096 Oct 11 11:30 webui
drwxr-xr-x 2 root root 4096 Oct 19 19:18 candidate
drwxr-xr-x 2 root root 4096 Oct 23 13:00 rollback
drwxr-xr-x 2 root root 4096 Oct 28 17:00 cdb
drwxr-xr-x 3 root root 4096 Oct 28 17:00 state
drwxr-xr-x 8 root root 4096 Oct 31 18:00 .
-rw-r--r-- 1 root root 1617920 Oct 31 18:00 Autovnf_cdb_backup.tar

7. Riavviare i servizi uas-confd e autovnf su UAS con questi comandi:

service uas-confd start
service autovnf start

root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service uas-confd start
uas-confd start/running, process 13852
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service autovnf start
autovnf start/running, process 13853

8. Riavviare UAS secondario da OSPD con questi comandi:

. corerc ; openstack server start <VMName>

[stack@labucs300-ospd ~]$. corerc ; openstack server start labucs300-UAS-LABPGW300-UAS-core-UAS2-1

9. Verificare che gli attributi UAS primario e secondario siano attivi nel comando show uas.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ sudo -i
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# confd_cli -u admin -C
Welcome to the ConfD CLI
admin connected from 127.0.0.1 using console on labucs300-uas-LABPGW300-uas-core-uas2-2
labucs300-uas-LABPGW300-uas-core-uas2-2#show uas 
uas version 6.2.0
uas state active
uas external-connection-point 192.0.2.8
INSTANCE IP STATE ROLE 
-----------------------------------
0.0.0.0 error CONFD-Secondary
192.0.2.15 alive CONFD-Secondary
192.0.2.17 alive CONFD-Primary

10. Modificare ha_debug in OFF su AutoVNF (UAS) principale e secondario nel file /opt/uas_baseconfig.txt.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 192.0.2.8
ha-secret: d5a9fee60ddae4fe357677dcf1320e51
orch-ips: 192.0.2.15,192.0.2.17
orch-intf: eth0
profile: AUTOVNF
ha_debug: OFF
ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ ssh ubuntu@192.0.2.15
ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-1:~$ cat /opt/uas_baseconfig.txt
ha: true
ha-vip: 192.0.2.8
ha-secret: d5a9fee60ddae4fe357677dcf1320e51
orch-ips: 192.0.2.15,192.0.2.17
orch-intf: eth0
profile: AUTOVNF
ha_debug: OFF

11. Riavviare il servizio AutoIT sul servizio AutoIT primario con il comando service autoit start.

ubuntu@labucs300-autoit-2:~$ sudo -i
root@labucs300-autoit-2:~# service autoit start

12. Verificare che l'interfaccia AutoIT primaria e secondaria appaia attiva con il comando show uas.

ubuntu@labucs300-autoit-2:~$ sudo -i
root@labucs300-autoit-2:~# confd_cli -u admin -C 
Welcome to the ConfD CLI
admin connected from 127.0.0.1 using console on labucs300-autoit-2
labucs300-autoit-2#show uas
uas version 6.2.0
uas state active
uas external-connection-point 172.16.181.7
INSTANCE IP STATE ROLE 
-----------------------------------
172.16.181.5 alive CONFD-Secondary 
172.16.181.8 alive CONFD-Primary

13. Impostare ha_debug su OFF su AutoIT principale e secondario nel file /opt/uas_baseconfig.txt.

ubuntu@labucs300-autoit-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 172.16.181.7
ha-secret: f99d04acb84807c4c6c6c0eaad392b5c
orch-ips: 172.16.181.5,172.16.181.8
orch-intf: eth0
prov-ha-vip: 172.16.181.13
prov-intf: eth0
profile: AUTOIT
ha_debug: OFF

ubuntu@labucs300-autoit-1:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 172.16.181.7
ha-secret: f99d04acb84807c4c6c6c0eaad392b5c
orch-ips: 172.16.181.5,172.16.181.8
orch-intf: eth0
prov-ha-vip: 172.16.181.13
prov-intf: eth0
profile: AUTOIT
ha_debug: OFF

14. Raccogliere i registri da UAS e trasferirli su un server di backup.

root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# cd /opt/cisco/usp/uas/confd-latest/var/confd/
root@labucs300-uas-LABPGW300-uas-core-uas2-2:/opt/cisco/usp/uas/confd-latest/var/confd# cd /opt/cisco/usp/uas/scripts/
root@labucs300-uas-LABPGW300-uas-core-uas2-2:/opt/cisco/usp/uas/scripts# sudo ./collect-uas-logs.sh 
Dumping output for show transaction in file /tmp/uas-logs/transactions.txt
Dumping output for show log in file /tmp/uas-logs/transactions.txt
Dumping output for show running-config in file /tmp/uas-logs/confd_output.txt
Dumping output for show uas in file /tmp/uas-logs/confd_output.txt
Dumping output for show usp in file /tmp/uas-logs/confd_output.txt
..........

15. Accedere al file AutoVNF secondario e ripetere il passaggio precedente per raccogliere i log e trasferirli sul server di backup.

16. Eseguire il backup della configurazione syslog sulle VM AutoVNF primarie e secondarie e trasferirle sul server di backup. I file si trovano nelle seguenti directory:

root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# ls /etc/rsyslog.d/00-autovnf.conf 
/etc/rsyslog.d/00-autovnf.conf
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# ls /etc/rsyslog.conf
/etc/rsyslog.conf

17. Abilitare il servizio automatico con il comando service autoit start su AutoIT primario.

ubuntu@labucs300-autoit-2:~$ sudo -i
root@labucs300-autoit-2:~#
root@labucs300-autoit-2:~# service autoit start
autoit start/running, process 25001

18. Impostare la modalità flag ha_debug su OFF nella /opt/uas_baseconfig.txt su Primary AutoVNF e AutoIT.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 192.0.2.8
ha-secret: d5a9fee60ddae4fe357677dcf1320e51
orch-ips: 192.0.2.15,192.0.2.17
orch-intf: eth0
profile: AUTOVNF
ha_debug: OFF

ubuntu@labucs300-autoit-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 172.16.181.7
ha-secret: f99d04acb84807c4c6c6c0eaad392b5c
orch-ips: 172.16.181.5,172.16.181.8
orch-intf: eth0
prov-ha-vip: 172.16.181.13
prov-intf: eth0
profile: AUTOIT
ha_debug: OFF

19. Confermare i servizi autonf e uas-confd in esecuzione su UAS.

root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service uas-confd status
uas-confd start/running, process 1305
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service autovnf status
autovnf start/running, process 24208

Ridistribuzione automatica VFN

1. Accedere a Distribuzione automatica e prendere nota dell'istanza di UAS.

ubuntu@labucs300-autodeploy-2:~$ sudo su 
root@labucs300-autodeploy-2:/home/ubuntu# confd_cli -u admin -C
Welcome to the ConfD CLI
admin connected from 127.0.0.1 using console on labucs300-autodeploy-2
labucs300-autodeploy-2#show nsr
 nsr         LABSGW300-instance
 nsd         LABSGW300
 vnfr        [ LABPCF300-LABPCF300-ESC LABPCF300-LABPCF300-UGP ]
 vnf-package [ usp_6_2_b8 ]
 vim-artifact vim_art_rack
nsr LABPGW300-instance
 nsd         LABPGW300
 vnfr        [ LABPGW300-LABPGW300-ESC LABPGW300-LABPGW300-UGP ]
 vnf-package [ usp_6_2_b8 ]
 vim-artifact vim_art_rack
nsr labucs300-UAS-instance
 nsd         labucs300-UAS
 vnfr        [ labucs300-UAS-LABPCF300-UAS labucs300-UAS-LABPGW300-UAS ]
 vnf-package [ usp_6_2_b8 ]
 vim-artifact vim_art_rack

2. Disattivare AutoVNF da AutoDeploy con il comando deactivate nsd-id <nsd-id> vnfd <vnfd-id>.

ubuntu@labucs300-autodeploy-2:~$ /opt/cisco/usp/uas/confd-6.3.1/bin/confd_cli -u admin -C
Welcome to the ConfD CLI
admin connected from 10.10.10.10 using ssh on labucs300-autodeploy-2
labucs300-autodeploy-2#nsd:deactivate nsd-id labucs300-UAS vnfd [LABPGW300-UAS]
transaction-id 1560431372-357328 

3. Confermare lo stato della transazione.

labucs300-autodeploy-2#show transaction 
                                                           DEPLOYMENT                                                 STATUS 
TX ID                               TX TYPE                ID            TIMESTAMP                        STATUS      DETAIL 
------------------------------------------------------------------------------------------------------------------------------
1560431372-357328                   activate-ns-deployment labucs300-UAS 2019-06-13T13:09:32.357355-00:00 in-progress - 
1560431372-357328/1560431373-102024 activate-ns-deployment labucs300-UAS 2019-06-13T13:09:33.102041-00:00 in-progress - 

4. Controllare i registri della transazione. In questo caso, transazioni labucs300-UAS: 1560431372-357328 e 1560431372-357328/1560431373-102024.

labucs300-autodeploy-2#show log 1560431372-357328 | display xml
<config xmlns="http://tail-f.com/ns/config/1.0">
<log xmlns="http://www.cisco.com/usp/nfv/usp-transaction">
<tx-id>1560431372-357328</tx-id>
<log>
2019-06-13 13:09:33,367 - Send Deployment notification for: labucs300-UAS-instance
2019-06-13 13:09:33,375 - Deployment activate-ns-deployment: labucs300-UAS started
2019-06-13 13:09:33,378 - Adding NSR: labucs300-UAS-instance
2019-06-13 13:09:33,385 - Start pipeline of 1 tasks
2019-06-13 13:09:33,390 - Scheduling Task: labucs300-UAS
2019-06-13 13:09:33,400 - Waiting for all workers to finish the transactions
2019-06-13 13:15:00,006 - Deployment activate-ns-deployment: labucs300-UAS succeeded
2019-06-13 13:15:00,020 - Send Deployment notification for: labucs300-UAS-instance
2019-06-13 13:09:33,437 - Send Deployment notification for: labucs300-UAS-instance-deploy
2019-06-13 13:09:33,441 - Deployment activate-ns-deployment: labucs300-UAS started
……………

labucs300-autodeploy-2#show log 1560431372-357328/1560431373-102024 | display xml
<config xmlns="http://tail-f.com/ns/config/1.0">
<log xmlns="http://www.cisco.com/usp/nfv/usp-transaction">
<tx-id>1560431372-357328/1560431373-102024</tx-id>
<log>
2019-06-13 13:09:33,437 - Send Deployment notification for: labucs300-UAS-instance-deploy
2019-06-13 13:09:33,441 - Deployment activate-ns-deployment: labucs300-UAS started
2019-06-13 13:09:33,446 - Adding NSR: labucs300-UAS-instance, VNFR: labucs300-UAS-LABPCF300-UAS, vlrs: None
2019-06-13 13:09:33,453 - Adding NSR: labucs300-UAS-instance, VNFR: labucs300-UAS-LABPGW300-UAS, vlrs: None
2019-06-13 13:09:33,463 - VNF deployment pre-check success(all-not-present)
2019-06-13 13:09:33,472 - VNF-Package deployment pre-check success(all-not-present)
2019-06-13 13:09:33,481 - VIM-Artifact deployment pre-check success
2019-06-13 13:09:33,487 - Skipping VIM-Orch pre-deployment, since VIM-Orch is not defined
2019-06-13 13:09:33,496 - Skipping VIM pre-deployment, since VIM is not defined
2019-06-13 13:09:33,499 - NS pre-check success
2019-06-13 13:09:33,503 - Copying '/home/ubuntu/usp-6_2_b8.iso' to '/var/cisco/isos/labucs300-UAS_usp_6_2_b8'
2019-06-13 13:09:53,359 - Updated path to URL 'http://172.16.181.14:5000/isos/labucs300-UAS_usp_6_2_b8'

5. Attendere il completamento della transazione e confermare lo stato.

labucs300-autodeploy-2#show transaction 
                                                           DEPLOYMENT                                            STATUS 
TX ID                               TX TYPE                ID            TIMESTAMP                        STATUS DETAIL 
------------------------------------------------------------------------------------------------------------------------------
1560431372-357328                   activate-ns-deployment labucs300-UAS 2019-06-13T13:09:32.357355-00:00 success - 
1560431372-357328/1560431373-102024 activate-ns-deployment labucs300-UAS 2019-06-13T13:09:33.102041-00:00 success - 

6. Attivare AutoVNF VNFD da AutoDeploy con il comando activate nsd-id <nsd-id> vnfd <vnfd-id>.

ubuntu@labucs300-autodeploy-2:~$ /opt/cisco/usp/uas/confd-6.3.1/bin/confd_cli -u admin -C
Welcome to the ConfD CLI
admin connected from 10.253.110.47 using ssh on labucs300-autodeploy-2
labucs300-autodeploy-2#nsd:activate nsd-id labucs300-UAS vnfd [LABPGW300-UAS]
transaction-id 1560431371-357330 

7. Controllare lo stato della transazione e raccogliere i log dalla transazione con questi comandi:

show transaction
show log <transaction-id> | display xml
show log <transaction-id> | display xml

8. Attendere il completamento delle transazioni. Il comando show transaction permette di visualizzare lo stato delle transazioni.

Ripristino backup

1. Su AutoIT primario impostare la modalità flag ha_debug su ON in /opt/uas_baseconfig.txt.

ubuntu@labucs300-autoit-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 172.16.181.7
ha-secret: f99d04acb84807c4c6c6c0eaad392b5c
orch-ips: 172.16.181.5,172.16.181.8
orch-intf: eth0
prov-ha-vip: 172.16.181.13
prov-intf: eth0
profile: AUTOIT
ha_debug: ON

2. Su Autorizzazione servizio arresto principale AutoIT. In questo modo si impedisce ad AutoIT di ripristinare automaticamente UAS.

ubuntu@labucs300-autoit-2:~$ sudo -i
root@labucs300-autoit-2:~#
root@labucs300-autoit-2:~# service autoit status
autoit start/running, process 25001
root@labucs300-autoit-2:~# service autoit stop

3. In UAS primario impostare la modalità flag ha_debug su ON in /opt/uas_baseconfig.txt.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 192.0.2.8
ha-secret: d5a9fee60ddae4fe357677dcf1320e51
orch-ips: 192.0.2.15,192.0.2.17
orch-intf: eth0
profile: AUTOVNF
ha_debug: ON

4. Su UAS secondario impostare la modalità flag ha_debug su ON in /opt/uas_baseconfig.txt.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-1:~$ cat /opt/uas_baseconfig.txt
ha: true
ha-vip: 192.0.2.8
ha-secret: d5a9fee60ddae4fe357677dcf1320e51
orch-ips: 192.0.2.15,192.0.2.17
orch-intf: eth0
profile: AUTOVNF
ha_debug: ON

5. Su OSPD arrestare il server UAS secondario con il comando OpenStack.

labucs300-UAS-LABPGW300-UAS-core-UAS2-2 -> VNFM2-UAS-VIP Primary
labucs300-UAS-LABPGW300-UAS-core-UAS2-1 -> Secondary

[stack@labucs300-ospd ~]$. corerc ; openstack server stop labucs300-UAS-LABPGW300-UAS-core-UAS2-1

6. Sul server UAS primario interrompere il servizio uas-confd.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ sudo -i 
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service uas-confd status
uas-confd start/running, process 1305
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service uas-confd stop
uas-confd stop/waiting

7. Sul server UAS primario, copiare l'archivio di backup del database CDB nella directory /opt/cisco/usp/uas/confd-last/var/confd/.

root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# cp Autovnf_cdb_backup.tar to /opt/cisco/usp/uas/confd-latest/var/confd/

8. In UAS primario eliminare i file nella directory CBD.

root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# cd /opt/cisco/usp/uas/confd-latest/var/confd/ ; rm cdb/*

9. Sul file di estrazione UAS primario dal file di backup CDB.

root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# cd /opt/cisco/usp/uas/confd-latest/var/confd/ ; tar -xvf <archive_backup_tar_file>

10. Al riavvio OSPD, l'UAS primario con i comandi OpenStack.

[stack@labucs300-ospd ~]$ source *core
[stack@labucs300-ospd ~]$ nova list | grep LABPGW300-UAS-core-UAS2
| 8608fda4-b763-4753-95ff-2e07852098e3 | labucs300-UAS-LABPGW300-UAS-core-UAS2-1 | ACTIVE | - | Running | labucs300-UAS-LABPGW300-ORCH-NW=192.0.2.15; labucs300-UAS-LABPGW300-MGMT-NW=192.0.2.7 |
| 19f4496c-3907-4ea5-84c9-e5a6ef222392 | labucs300-UAS-LABPGW300-UAS-core-UAS2-2 | ACTIVE | - | Running | labucs300-UAS-LABPGW300-ORCH-NW=192.0.2.17; labucs300-UAS-LABPGW300-MGMT-NW=192.0.2.16

labucs300-UAS-LABPGW300-UAS-core-UAS2-2 -> VNFM2-UAS-VIP Primary
labucs300-UAS-LABPGW300-UAS-core-UAS2-1 -> Secondary

[stack@labucs300-ospd ~]$ nova reboot --hard 19f4496c-3907-4ea5-84c9-e5a6ef222392
Request to reboot server <Server: auto-testautovnf1-uas-2> has been accepted.

11. Attendere l'accensione dell'UAS primario. Controllare lo stato dell'UAS primario dopo il riavvio. Lo stato dell'UAS primario è attivo mentre quello dell'UAS secondario mostra uno stato sconosciuto.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ sudo -i
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# confd_cli -u admin -C
Welcome to the ConfD CLI
admin connected from 127.0.0.1 using console on labucs300-uas-LABPGW300-uas-core-uas2-2
labucs300-uas-LABPGW300-uas-core-uas2-2#show uas 
uas version 6.2.0
uas state active
uas external-connection-point 192.0.2.8
INSTANCE IP STATE ROLE 
-----------------------------------
192.0.2.15 unknown CONFD-Secondary 
192.0.2.17 alive CONFD-Primary

12. Su OSPD, avviare l'UAS secondario con il comando OpenStack.

[stack@labucs300-ospd ~]$. corerc ; openstack server start labucs300-UAS-LABPGW300-UAS-core-UAS2-1

13. In OSPD verificare che gli stati dell'UAS primario e secondario siano attivi.

[stack@labucs300-ospd ~]$ openstack server list | grep labucs300-UAS-LABPGW300
| 19f4496c-3907-4ea5-84c9-e5a6ef222392 | labucs300-UAS-LABPGW300-UAS-core-UAS2-2 | ACTIVE | labucs300-UAS-LABPGW300-ORCH-NW=192.0.2.17; labucs300-UAS-LABPGW300-MGMT-NW=192.0.2.16 | labucs300-UAS-usp_6_2_b8-core-uas |
| 8608fda4-b763-4753-95ff-2e07852098e3 | labucs300-UAS-LABPGW300-UAS-core-UAS2-1 | ACTIVE | labucs300-UAS-LABPGW300-ORCH-NW=192.0.2.15; labucs300-UAS-LABPGW300-MGMT-NW=192.0.2.7 | labucs300-UAS-usp_6_2_b8-core-uas |

14. In UAS primario verificare che gli stati dell'UAS primario e secondario siano attivi.

labucs300-uas-LABPGW300-uas-core-uas2-2#show uas 
uas version 6.2.0
uas state active
uas external-connection-point 192.0.2.8
INSTANCE IP STATE ROLE 
-----------------------------------
192.0.2.15 alive CONFD-Secondary
192.0.2.17 alive CONFD-Primary

15. Sul servizio di avvio automatico principale.

ubuntu@labucs300-autoit-2:~$ sudo -i
root@labucs300-autoit-2:~# service autoit start

16. Verificare che la sessione SSH (Secure Shell) rimanga attiva per un paio di minuti fino agli UAS primario e secondario.

17. Su UAS primario impostare la modalità flag ha_debug su OFF in /opt/uas_baseconfig.txt.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 192.0.2.8
ha-secret: d5a9fee60ddae4fe357677dcf1320e51
orch-ips: 192.0.2.15,192.0.2.17
orch-intf: eth0
profile: AUTOVNF
ha_debug: OFF

18.  Su UAS secondario impostare la modalità flag ha_debug su OFF in /opt/uas_baseconfig.txt.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-1:~$ cat /opt/uas_baseconfig.txt
ha: true
ha-vip: 192.0.2.8
ha-secret: d5a9fee60ddae4fe357677dcf1320e51
orch-ips: 192.0.2.15,192.0.2.17
orch-intf: eth0
profile: AUTOVNF
ha_debug: OFF

19. Su AutoIT primario impostare la modalità flag ha_debug su OFF in /opt/uas_baseconfig.txt.

 ubuntu@labucs300-autoit-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 172.16.181.7
ha-secret: f99d04acb84807c4c6c6c0eaad392b5c
orch-ips: 172.16.181.5,172.16.181.8
orch-intf: eth0
prov-ha-vip: 172.16.181.13
prov-intf: eth0
profile: AUTOIT
ha_debug: OFF

20. In AutoVNF controllare i file 00-autovnf.conf e rsyslog.conf e ripristinarli dal backup precedente.

ubuntu@autoit-tb1-autovnf1-core-avf-1:~#sudo su
root@autoit-tb1-autovnf1-core-avf-1:~#ls /etc/rsyslog.d/00-autovnf.conf
00-autovnf.conf

root@autoit-tb1-autovnf1-core-avf-1:~#/home/ubuntu#ls /etc/rsyslog.conf
rsyslog.conf

Controlli successivi

In OSPD verificare che entrambi gli AutoVNF siano attivi e controllare il rapporto di verifica dello stato di Ultra-M.

[stack@labucs300-ospd ~]$ cat /var/log/cisco/ultram-health/*.report | grep -i xxx

[stack@labucs300-ospd ~]$ cat /var/log/cisco/ultram-health/ultram_health_uas.report