By default, the proxy cache paths are stored in the file system. We recommend changing them to in-memory drives by creating a cache location in tmpfs as shown here.

1. Create directories for the different proxy cache paths under /home.

   As an example, these directories must be created for the primary Finesse server. The same steps should be followed for the secondary Finesse and Cisco Unified Intelligence Center servers.

   mkdir -p /home/primaryFinesse/rest
   mkdir -p /home/primaryFinesse/desktop
   mkdir -p /home/primaryFinesse/shindig
   mkdir -p /home/primaryFinesse/openfire
   mkdir -p /home/primaryCUIC/cuic
   mkdir -p /home/primaryCUIC/cuicdoc
   mkdir -p /home/client_temp
   mkdir -p /home/proxy_temp
   
   echo "tmpfs /home/primaryFinesse/rest tmpfs size=1510M,rw,auto,noexec,nodev,nosuid,gid=root,uid=root,mode=1700 0 0" >>
   /etc/fstab echo "tmpfs /home/primaryFinesse/desktop tmpfs size=20M,rw,auto,noexec,nodev,nosuid,gid=root,uid=root,mode=1700 0 0" >>
   /etc/fstab echo "tmpfs /home/primaryFinesse/shindig tmpfs size=500M,rw,auto,noexec,nodev,nosuid,gid=root,uid=root,mode=1700 0 0" >>
   /etc/fstab echo "tmpfs /home/primaryFinesse/openfire tmpfs size=10M,rw,auto,noexec,nodev,nosuid,gid=root,uid=root,mode=1700 0 0" >>
   /etc/fstab echo "tmpfs /home/primaryCUIC/cuic tmpfs size=100M,rw,auto,noexec,nodev,nosuid,gid=root,uid=root,mode=1700 0 0" >>
   /etc/fstab echo "tmpfs /home/primaryCUIC/cuicdoc tmpfs size=100M,rw,auto,noexec,nodev,nosuid,gid=root,uid=root,mode=1700 0 0" >>
   /etc/fstab echo "tmpfs /home/client_temp tmpfs size=2048M,rw,auto,noexec,nodev,nosuid,gid=root,uid=root,mode=1700 0 0" >>
   /etc/fstab echo "tmpfs /home/proxy_temp tmpfs size=2048M,rw,auto,noexec,nodev,nosuid,gid=root,uid=root,mode=1700 0 0" >> /etc/fstab

   Note	Increase the client and proxy_temp caches by 1 GB for each new Finesse cluster added to the configuration.

2. Mount the new mount points with the mount -av command.

3. Use the df -h command to validate if the file system has mounted the new mount points.

4. Change the proxy_cache_path locations in the Finesse and Cisco Unified Intelligence Center cache configuration files. For example, to change the paths for the Finesse primary, go to <nginx-install-directory>conf/conf.d/finesse/caches and change the existing cache location /usr/local/openresty/nginx/cache/finesse25/ to the newly created filesystem location /home/primaryFinesse.

   ##Must-change /usr/local/openresty/nginx/cache/finesse25 location would change depending on folder extraction
   proxy_cache_path /home/primaryFinesse/desktop levels=1:2 use_temp_path=on keys_zone=desktop_cache_fin25:10m max_size=15m inactive=3y use_temp_path=off;
   proxy_cache_path /home/primaryFinesse/shindig levels=1:2 use_temp_path=on keys_zone=shindig_cache_fin25:10m max_size=500m inactive=3y use_temp_path=off;
   proxy_cache_path /home/primaryFinesse/openfire levels=1:2 use_temp_path=on keys_zone=openfire_cache_fin25:10m max_size=10m inactive=3y use_temp_path=off;
   proxy_cache_path /home/primaryFinesse/rest levels=1:2 use_temp_path=on keys_zone=rest_cache_fin25:10m max_size=1500m inactive=40m use_temp_path=off; 

5. Follow the same steps for the Finesse secondary server and Cisco Unified Intelligence Center server.

   Note	Ensure that the sum of all the tmpfs drive sizings created in all the previous steps are added to the final memory sizing for the deployment. This is because these drives are memory blocks that are configured to look like disks to the application and they consume memory.

Nginx reverse-proxy produces many logs. Configure log rotation to ensure optimum use of disk space, else the logs fill up the disk. The steps to configure log rotation is as follows:

1. Copy the configuration file from /usr/local/openresty/nginx/logrotate/saproxy to /etc/logrotate.d/reverseproxy.

2. Ensure that there’s a file in the /etc/cron.daily/logrotate. This does the log rotation daily based on the configuration in the /etc/logrotate.d/reverseproxy.

   Note	If log rotation has to be done more frequently, such as every hour, copy the configuration file from /etc/cron.daily/logrotate to /etc/cron.hourly/logrotate.

3. Log files under /usr/local/openresty/nginx/logs/ are rotated based on the configuration in /etc/logrotate.d/reverseproxy. That is, rotate the log when the file size exceeds 100 MB and keep no more than 20 most recently rotated files.

The status of the log rotation is available in the /var/lib/logrotate/logrotate.status log file.

Use self-signed certificates until the reverse-proxy is ready to be rolled out into production. On a production deployment, use only a Certificate Authority-signed (CA-signed) certificate.

1. Generate OpenResty Nginx certificates for SSL folder content. Before you generate certificates, you must create a folder called ssl under /usr/local/openresty/nginx. Generate two certificates (one for <reverseproxy_primary_fqdn> and another for <reverseproxy_secondary_fqdn>) with the help of the following commands:

   1. sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /usr/local/openresty/nginx/ssl/nginx.key -out /usr/local/openresty/nginx/ssl/nginx.crt (pass hostname as: <reverseproxy_primary_fqdn>)

   2. sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /usr/local/openresty/nginx/ssl/nginxnode2.key -out /usr/local/openresty/nginx/ssl/nginxnode2.crt (pass hostname as: <reverseproxy_secondary_fqdn>)

   3. Ensure that the certificate path is /usr/local/openresty/nginx/ssl/nginx.crt and /usr/local/openresty/nginx/ssl/nginxnode2.crt, because these are already configured in Finesse Nginx configuration files.

2. Change the permission of the private key 400 (r--------).

3. Configure the firewall and iptables on the reverse-proxy to enable the firewall to communicate with the ports that are configured to listen to the OpenResty Nginx server.

4. Add the IP address and hostname of all the configured servers in the /etc/hosts file of the reverse-proxy server.

Note	The provided configuration is for a sample 2000 Agent deployment and must be expanded appropriately for larger deployments.

A CA-signed certificate can be installed on the reverse-proxy with these steps:

1. Generate the certificate signing request (CSR).

   To generate the CSR and private key, enter openssl req -new -newkey rsa:4096 -keyout nginx.key -out nginx.csr after you log in to the proxy. Follow the prompt, and provide the details. This generates the CSR (nginx.csr in the example) and the RSA private key (nginx.key in the example) of 4096 bits.

   For example:

   
   [root@reverseproxyhost.companyname.com ssl]# openssl req -new -newkey rsa:4096 -keyout nginx.key -out nginx.csr
   Generating a RSA private key
   .....+++++
   ...................................................................................................................................................+++++
   writing new private key to 'nginx.key'
   Enter PEM pass phrase:passphrase
   Verifying - Enter PEM pass phrase:passphrase
   -----
   You are about to be asked to enter information that will be incorporated
   into your certificate request.
   What you are about to enter is what is called a Distinguished Name or a DN.
   There are quite a few fields but you can leave some blank
   For some fields there will be a default value,
   If you enter '.', the field will be left blank.
   -----
   Country Name (2 letter code) [XX]:US
   State or Province Name (full name) []:CA
   Locality Name (eg, city) [Default City]:Orange County
   Organization Name (eg, company) [Default Company Ltd]:CompanyName
   Organizational Unit Name (eg, section) []:BusinessUnit
   Common Name (eg, your name or your server's hostname) []:reverseproxyhostname.companydomain.com
   Email Address []:john.doe@comapnydomain.com
   Please enter the following 'extra' attributes
   to be sent with your certificate request
   A challenge password []:challengePWD
   An optional company name []:CompanyName

   Write down the PEM passphrase. This is used to decrypt the private key during the deployment.

2. Obtain the signed certificate from the CA.

   Send the CSR to the certificate authority and obtain the signed certificate.

   Note	If the certificate received from the CA is not a certificate chain containing all the respective certificates, compose all the relevant certificates into a single certificate chain file.

3. Deploy the certificate and key.

   Decrypt the key generated in the first step with the openssl rsa -in nginx.key -out nginx_decrypted.key command. Place the CA-signed certificate and the decrypted key inside the folder /usr/local/openresty/nginx/ssl in the reverse-proxy machine. Update or add the following SSL configurations related to the certificate in the OpenResty Nginx configurations in the following file: /usr/local/openresty/nginx/conf/conf.d/ssl/ssl.conf:

   
   ssl_certificate /usr/local/openresty/nginx/ssl/ca_signed_cert.crt;
   ssl_certificate_key/usr/local/openresty/nginx/ssl/nginx_decrypted.key;

4. Configure permissions for the certificates.

   Enter chmod 400 /usr/local/openresty/nginx/ssl/ca_signed_cert.crt and chmod 400 /usr/local/openresty/nginx/ssl/nginx_decrypted.key, so that the certificate has read-only permission and is restricted to the owner.

5. Reload OpenResty Nginx.

1. Create a custom Diffie-Hellman parameter by using the following commands:

   openssl dhparam -out /usr/local/openresty/nginx/ssl/dhparam.pem 2048

   chmod 400 /usr/local/openresty/nginx/ssl/dhparam.pem

2. Modify the server configuration to use the new parameters in the file /usr/local/openresty/nginx/conf/conf.d/ssl/ssl.conf by using the following command:

   ssl_dhparam /usr/local/openresty/nginx/ssl/dhparam.pem;

Note	In order to enable the Online Certificate Status Protocol (OCSP) stapling, the server should be using a CA-signed certificate and the server should have access to the CA which signed the certificate.

Add or update this configuration in the file: /usr/local/openresty/nginx/conf/conf.d/ssl/ssl.conf:

• ssl_stapling on;

• ssl_stapling_verify on;

The default OpenResty Nginx configuration file (/usr/local/openresty/nginx/conf/nginx.conf) has to be modified to contain the following entries to enforce security and enhance performance. The following content should be used to modify the default configuration file (nginx.conf) which is created during the OpenResty Nginx installation:

# Increasing number of worker processes will not increase the processing the request. The number of worker process will be same as number of cores
# in system CPU. OpenResty Nginx provides "auto" option to automate this, which will spawn one worker for each CPU core.
worker_processes  auto;

# Process id file location
pid /usr/local/openresty/nginx/logs/nginx.pid;

# Binds each worker process to a separate CPU
worker_cpu_affinity auto;

#Defines the scheduling priority for worker processes. This should be calculated by "nice" command. In our proxy set up the value is 0
worker_priority 0;


error_log  /usr/local/openresty/nginx/logs/error.log info;

#user root root;

# current limit on the maximum number of open files by worker processes, keeping 10 times of worker_connections

worker_rlimit_nofile 102400;

events {
    multi_accept on;


    # Sets the maximum number of simultaneous connections that can be opened by a worker process.
    # This should not be more the current limit on the maximum number of open files i.e. hard limit of the maximum number of open files for the user (ulimit -Hn)
    # The appropriate setting depends on the size of the server and the nature of the traffic, and can be discovered through testing.
    worker_connections  10240;
    #debug_connection 10.78.95.21
}


http {

    include       mime.types;


    default_type  text/plain;


    ## Must-change Change with DNS resolver ip in deployment
    resolver 192.168.1.3;


    ## Must-change change lua package path to load lua libraries
    lua_package_path  "/usr/local/openresty/lualib/resty/?.lua;/usr/local/openresty/nginx/lua/?.lua;;"


    ## Must-change change proxy_temp folder as per cache directory configurations
    proxy_temp_path /usr/local/openresty/nginx/cache/proxy_temp 1 2 ;
    ## Must-change change client_temp folder as per cache directory configurations
    client_body_temp_path /usr/local/openresty/nginx/cache/client_temp 1 2 ;


    lua_shared_dict userlist 50m;
    lua_shared_dict credentialsstore 100m;
    lua_shared_dict userscount 100k;
    lua_shared_dict clientstorage 100m;
    lua_shared_dict blockingresources 100m;
    lua_shared_dict tokencache_saproxy 10M;
    lua_shared_dict tokencache_saproxy125 10M;
    lua_shared_dict ipstore 10m;
    lua_shared_dict desktopurllist 10m;
    lua_shared_dict desktopurlcount 100k;
    lua_shared_dict thirdpartygadgeturllist 10m;
    lua_shared_dict thirdpartygadgeturlcount 100k;
    lua_shared_dict corsheadersstore 100k;


    init_worker_by_lua_block {
        local UsersListManager = require('users_list_manager')
        local UnauthenticatedDesktopResourcesManager = require("unauthenticated_desktopresources_manager")
        local UnauthenticatedResourcesManager = require("unauthenticated_thirdpartyresources_manager")
        --  Must-change Replace saproxy.cisco.com with reverseproxy fqdn

        if  ngx.worker.id() == 0 then
            UsersListManager.getUserList("saproxy.cisco.com", "https://saproxy.cisco.com:8445/finesse/api/Users")
            UnauthenticatedDesktopResourcesManager.getDesktopResources("saproxy.cisco.com", "https://saproxy.cisco.com:8445/desktop/api/urls?type=desktop") 
            UnauthenticatedResourcesManager.getThirdPartyGadgetResources("saproxy.cisco.com", "https://saproxy.cisco.com:8445/desktop/api/urls?type=3rdParty")
        end 
    }

    include conf.d/*.conf;

    sendfile        on;

    tcp_nopush     on;

    server_names_hash_bucket_size 512;

By default, the Nginx configuration for Finesse requests on port 8445. At a time, only one port can be enabled from a reverse-proxy to support Finesse requests. If port 443 needs to be supported, edit the <nginx-install-directory>conf/conf.d/finesse.conf file to enable listening on 443 and disable listening on 8445.

Client SSL certificate authentication for connections from reverse-proxy hosts can be enabled on Cisco Unified Intelligence Center, Finesse, IdS, and LiveData by using the new CVOS CLI option utils system reverse-proxy client-auth enable/disable/status.

By default this is disabled and has to be enabled by the administrator by executing the CLI on each upstream server independently. After this option is enabled, Cisco Web proxy Service running on upstream host will start authenticating client certificates in TLS handshake for connections originating from trusted reverse-proxy hosts added by using the CLI utils system reverse-proxy allowed-hosts add <proxy-host>.

The following is the configuration block for the same in proxy config files named ssl.conf and ssl2.conf.

#Must-change /usr/local/openresty/nginx/ssl/nginx.crt change this location accordingly
proxy_ssl_certificate /usr/local/openresty/nginx/ssl/nginx.crt;
#Must-change /usr/local/openresty/nginx/ssl/nginx.key change this location accordingly
proxy_ssl_certificate_key /usr/local/openresty/nginx/ssl/nginx.key;

The SSL certificate used for outbound traffic (proxy to upstream) can be the same as the SSL certificate that is configured for inbound traffic (SSL connector for component server blocks). If self-signed certificate is used as proxy_ssl_certificate, it has to be uploaded to the tomcat trust store of the upstream components (Finesse/IdS/Cisco Unified Intelligence Center/Livedata) for it to be authenticated successfully.

Upstream server certificate validation by reverse-proxy is optional and disabled by default. If you wish to achieve full TLS mutual auth between reverse-proxy and upstream hosts, the following configuration has to be uncommented in the ssl.conf and ssl2.conf files.

#Enforce upstream server certificate validation at proxy ->
#this is not mandated as per CIS buit definitely adds to security.
#It requires the administrator to upload all upstream server certificates to the proxy certificate store
#Must-Change Uncomment below lines IF need to enforce upstream server certificate validation at proxy
#proxy_ssl_verify on;
#proxy_ssl_trusted_certificate /usr/local/openresty/nginx/ssl/finesse25.crt;
proxy_ssl_trusted_certificate: This file should contain the all upstream certificate enteries concatenated together

mutual TLS (mTLS) is a standard security requirement for connections established from DMZ into the data center. For more information, see Nginx CIS behcmarks-https://www.cisecurity.org/benchmark/nginx

mTLS requires that both the server and client be pre-configured with mutual information about each other, as well as that the mutual certificates be properly verified. Hence the term Mutual TLS. A properly configured proxy server will be able to circumvent TCP rate limits and provide the client IP to the server for logging purposes. As a result, it is critical that the proxy identity be verified before connecting as a reverse-proxy. For security reasons, it is therefore recommended that this feature be used and turned on.

This requires the upstream component certificates to be made available to the proxy and vice-versa. Reverse-proxy by default establishes verified TLS connections to the upstream server and it is the proxy verification at the client which is optional. Therefore this needs to be enabled at the upstream client server.

Enabling mutual TLS

The mutual TLS needs to be enabled at the upstream component servers using the provided CLI.

Use the utils system reverse-proxy client-auth enable CLI to enable proxy certificate verification at the upstream component server.

After running the CLI, upload the proxy SSL certificate corresponding to the reverse-proxy hostname used to connect to the same server. This can be used to verify TLS connections when the reverse-proxy attempts to establish an upstream connection.

The VPN-less configuration assumes that the proxy is accessed by clients/agents from the internet, who have separate individual IPs which can be used for enforcing security features. However, not all deployments dedicatedly use agents from the internet with their own unique IP addresses. Most deployments have agents accessing the reverse-proxy deployments both from the internet and from LAN using the same reverse-proxy access URI.

So, if you have a deployment which uses agents behind a proxy or a NAT that looks like what is shown above, certain configuration changes have to be made to ensure that the end-user IPs are correctly communicated to the reverse-proxy. The steps to configure are as follows:

1. The Forward proxy (device A in the diagram above) has to be well known in advance.

2. The Forward proxy device has to transmit the agent IPs in a predefined header. For example, X-REAL-IP as shown above.

3. If there are other intermediary devices such as a Load Balancer or WAF at the network where Finesse is deployed, before the requests reach the reverse-proxy, these devices must be able to allow the Forward proxy by its IP address and then transmit the HTTP header without any changes.

   Note	It’s crucial that the Forward proxy IP address is identified and only requests from this IP should be allowed to have the X-REAL-IP transmitted.

4. The nginx.conf http block and maps.conf geo block files should be updated with the list of Forward proxies' private and public IPs as mentioned in the ##Must-change comments.

Note

Deployments that don’t have the HTTP header support can skip the steps 2–4. However, this causes a suboptimal deployment which is functional but loses certain security features listed above. These are dependent on client IP knowledge and these deployments are therefore not suggested. Ensure that the deployment can’t support multiple HTTP header names to transmit the client IP corresponding to different Forward proxies that the network is interacting with.

Deployments such as these should transmit or detect the final client IP of the users who are connecting from behind the Forward proxy A and this would be an agreement between Load Balancer and the Forward Proxy.

The Load Balancer or the final intermediary devices that forward requests to the VPN-less reverse-proxy should transmit the required headers. It must be configured as described in the section above. The Forward proxy information isn’t required to be added to the VPN-less configuration, if the intermediary device is able to identify the correct client IPs and transmit them to the reverse-proxy using the steps mentioned above.

However, if the actual client IP resolution isn’t set up between the Forward proxy and the Load Balancer, the reverse-proxy considers the IP of the Forward proxy as the actual client IP.

In this case, to avoid rate-limiting to block the Forward proxy, its private, and public IP addresses must be configured in nginx.conf http block and maps.conf geo block files. Both the files must be updated with the list of Forward-proxies' IP as mentioned in the ##Must-change comments, so that the proxy isn’t blocked or rate limited. This would be a suboptimal deployment and transmitting the actual client IP is recommended for a more effective deployment.