Source: https://www.cisco.com/c/en/us/support/docs/unified-communications/unified-communications-manager-callmanager/214231-certificate-regeneration-process-for-cis.html#anc19 


Tomcat Certificate

Identify if third party certificates are in use.

  1. Navigate to each server in your cluster (in separate tabs of your web browser) begin with the publisher, followed by each subscriber.  Navigate to Cisco Unified OS Administration > Security > Certificate Management > Find
    • Observe from Description column if Tomcat states Self-signed certificate generated by system. If Tomcat is third party signed, follow the link provided and perform those steps after the Tomcat regeneration
    • Third Party Signed certificates - https://supportforums.Cisco.com/docs/DOC-6119
  2. Select Findin order to show all the certificates
    • Select the Tomcat pem Certificate
    • Once open select Regenerate and wait until you see the Success pop-up then close pop-up or go back and select Find/List
  3. Continue with each subsequent Subscriber, follow the same procedure in step 2 and complete on all Subscribers in your cluster
  4. After all Nodes have regenerated the Tomcat certificate, restart the tomcat service on all the nodes. Begin with the publisher then followed by the subscribers.
    • In order to restart Tomcat you need to open a CLI session for each node and execute the command utils service restart Cisco Tomcat

 IPSEC Certificate

Note: CUCM/Instant Messaging and Presence (IM&P) before version10.X the DRF Master Agent runs on both CUCM Publisher and IM&P Publisher.  DRF Local service runs on the subscribers respectively.  Versions 10.X and higher, DRF Master Agent runs on the CUCM Publisher only and DRF Local service will be on CUCM Subscribers and IM&P Publisher and Subscribers.

Note: The Disaster Recovery System uses an Secure Socket Layer (SSL) based communication between the Master Agent and the Local Agent for authentication and encryption of data between the CUCM cluster nodes. DRS makes use of the IPSec certificates for its Public/Private Key encryption. Be aware that if you delete the IPSEC truststore (hostname.pem) file from the Certificate Management page, then DRS will not work as expected. If you delete the IPSEC-trust file manually, then you must ensure that you upload the IPSEC certificate to the IPSEC truststore. For more details, refer to the certificate management help page in the Cisco Unified Communications Manager Security Guides.

  1. Navigate to each server in your cluster (in separate tabs of your web browser) begin with the publisher, followed by each subscriber.  Navigate to Cisco Unified OS Administration > Security > Certificate Management > Find
    • Select the IPSEC pem Certificate.
    • Once open select Regenerate and wait until you see the Success pop-up then close pop-up or go back and select Find/List
  2. Continue with subsequent Subscribers; follow the same procedure in step 1 and complete on all subscribers in your cluster
  3. After all Nodes have regenerated the IPSEC certificate then restart services.
    • Navigate to the Publisher's Cisco Unified Serviceability
      1. Cisco Unified Serviceability > Tools > Control Center - Network Services
      2. Select Restart on Cisco DRF Master service
      3. Once the service restart completes, select Restart on the Cisco DRF Local Service on the publisher then continue with the subscribers and select Restart on the Cisco DRF Local Service

The IPSEC.pem certificate in the publisher must be valid and must be present in all subscribers as IPSEC truststores. The subscribers IPSEC.pem certificate will not be present in the publisher as IPSEC truststore in a standard deployment. In order to verify the validity compare the serial numbers in the IPSEC.pem certificate  from the PUB with the IPSEC -trust in the SUBs. They must match.

CAPF Certificate

Warning: Ensure you have identified if your Cluster is in Mixed-Mode before you proceed.  Refer to section Identify if your cluster is in Mix-Mode or Non-secure Mode.

  1. Navigate to the Cisco Unified CM Administration > System > Enterprise Parameters.
    • Check the section Security Parameters and verify if the Cluster Security Mode is set to 0 or 1. If the value if 0 then the cluster is in Non-Secure Mode. If it is 1 then the cluster is in mixed-mode and you will need to update the CTL file prior to the restart of services.  See Token and Tokenless links below.
  2. Navigate to each server in your cluster (in seperate tabs of your web browser) begin with the publisher, then each subscriber.  Navigate to Cisco Unified OS Administration > Security > Certificate Management > Find
    • Select the CAPF pem Certificate.
    • Once open select Regenerate and wait until you see the Success pop-up then close pop-up or go back and select Find/List
  3. Continue with subsequent subscribers; follow the same procedure in step 2 and complete on all subscribers in your cluster
    • If cluster is in Mixed-Mode ONLY and the CAPF has been regenerated – Update the CTL before you proceed further Token - Tokenless
    • If cluster is in Mixed Mode then the Call Manager service will also need to be restarted prior to the restart of other services
  4. After all Nodes have regenerated the CAPF certificate, restart services
    • Navigate to publisher's Cisco Unified Serviceability
      1. Cisco Unified Serviceability > Tools > Control Center - Feature Services
      2. Begin with the publisher and select Restart  on the Cisco Certificate Authority Proxy Function Service only where running
  5. Navigate to Cisco Unified Serviceability > Tools > Control Center - Network Services
    • Begin with the publisher then continue with the subscribers, select Restart on Cisco Trust Verification Service
    • Navigate to Cisco Unified Serviceability > Tools > Control Center - Feature Services
    • Begin with the publisher then continue with the subscribers, restart Cisco TFTP Service only where running.
  6. Reboot all Phones
    • Cisco Unified CM Administration > System > Enterprise Parameters
    • Select Reset then you will see a pop-up with the statement You are about to reset all devices in the system. This action cannot be undone. Continue?,select OK and then select Reset

The phones will now reset. Monitor their actions via RTMT tool to ensure the reset was successful and that devices register back to CUCM.  Wait for the phone registration to complete before you proceed to next certificate. This process of phones registration can take some time. Be advised, devices that had bad ITLs prior to regeneration process might not register back to the cluster.

CallManager Certificate

Warning: Ensure you have identified if your Cluster is in Mixed-Mode before you proceed. Refer to section Identify if your cluster is in Mix-Mode or Non-secure Mode.

Warning: Do not regenerate CallManager.PEM and TVS.PEM certificates at the same time.  This will cause an unrecoverable mismatch to the installed ITL on endpoints which will require the removal the ITL from ALL endpoints in the cluster.

  1. Navigate to the Cisco Unified CM Administration > System > Enterprise Parameters.
    • Check the section Security Parameters and verify if the Cluster Security Mode is set to 0 or 1. If the value if 0 then the cluster is in Non-Secure Mode. If it is 1 then the cluster is in mixed-mode and you will need to update the CTL file prior to the restart of services.  See Token and Tokenless links below.
  2. Navigate to each server in your cluster (in seperate tabs of your web browser) begin with the publisher, then each subscriber.  Navigate to Cisco Unified OS Administration > Security > Certificate Management > Find
    • Select the CallManager pem Certificate.
    • Once open select Regenerate and wait until you see the Success pop-up then close pop-up or go back and select Find/List
  3. Continue with subsequent subscribers; follow the same procedure in step 2 and complete on all subscribers in your cluster.
    • If cluster is in Mixed-Mode ONLY and the CAPF has been regenerated – Update the CTL before you proceed further Token - Tokenless
  4. Log into Publisher’s Cisco Unified Serviceability
    • Navigate to Cisco Unified Serviceability > Tools > Control Center - Feature Services
    • Begin with the publisher then continue with the subscribers, restart Cisco CallManager Service where running.
  5. Navigate to Cisco Unified Serviceability > Tools > Control Center - Feature Services
    • Begin with the Publisher then continue with the subscribers, restart Cisco CTIManager Service only where running
  6. Navigate to Cisco Unified Serviceability > Tools > Control Center - Network Services
    • Begin with the Publisher then continue with the subscribers, restart Cisco Trust Verification Service
  7. Navigate to Cisco Unified Serviceability > Tools > Control Center - Feature Services
    • Begin with the Publisher then continue with the subscribers, restart Cisco TFTP Service only where running
  8. Reboot all Phones
    • Cisco Unified CM Administration > System > Enterprise Parameters
    • Select Reset then you will see a pop-up with the statement You are about to reset all devices in the system. This action cannot be undone. Continue?,select OK and then select Reset

The phones will now reset. Monitor their actions via RTMT tool to ensure the reset was successful and that devices register back to CUCM.  Wait for the phone registration to complete before you proceed to next certificate. This process of phones registration can take some time. Be advised, devices that had bad ITLs prior to regeneration process might not register back to the cluster.

TVS Certificate

Warning:  Do not regenerate CallManager.PEM and TVS.PEM certificates at the same time.  This will cause an unrecoverable mismatch to the installed ITL on endpoints which will require the removal the ITL from ALL endpoints in the cluster.

  Note: TVS authenticates certificates on behalf of Call Manager. Regenerate this certificate last.

  1. Navigate to each server in your cluster (in separate tabs of your web browser) begin with the publisher, then each subscriber.  Navigate to Cisco Unified OS Administration > Security > Certificate Management > Find
    • Select the TVS pem Certificate.
    • Once open select Regenerate and wait until you see the Success pop-up then close pop-up or go back and select Find/List
  2. Continue with subsequent subscribers; follow the same procedure in step 1 and complete on all subscribers in your cluster
    • After all Nodes have regenerated the TVS certificate, restart the services:
      • Log into Publisher's Cisco Unified Serviceability
        • Navigate to Cisco Unified Serviceability > Tools > Control Center - Network Services
        • On the publisher select Restart on Cisco Trust Verification Service.
      • Once the service restart completes, continue with the subscribers and restart the Cisco Trust Verification Service
  3. Begin with the Publisher then continue with the subscribers, restart Cisco TFTP Service only where running.
  4. Reboot all Phones
    • Cisco Unified CM Administration > System > Enterprise Parameters
    • Select Reset then you will see a pop-up with the statement You are about to reset all devices in the system. This action cannot be undone. Continue?,select OK and then select Reset

The phones will now reset. Monitor their actions via RTMT tool to ensure the reset was successful and that devices register back to CUCM.  Wait for the phone registration to complete before you proceed to next certificate. This process of phones registration can take some time. Be advised, devices that had bad ITLs prior to regeneration process might not register back to the cluster.

ITLRecovery Certificate

Note: The ITLRecovery Certificate is used when devices lose their trusted status. The certificate appears in both the ITL and CTL (when CTL provider is active).
If devices lose their trust status, you can use the command utils itl reset localkey for non-secure clusters and the command utils ctl reset localkey for mix-mode clusters. Read the Security guide for your Call Manager version to become familiar with how the ITLRecovery certificate is used and the process required to recover trusted status.
If the cluster has been upgraded to a version that supports a key length of 2048 and the clusters server certificates have been regenerated to 2048 and the ITLRecovery has not been regenerated and is currently 1024 key length, the ITL recovery command will fail and the ITLRecovery method will not be able to be used.

  1. Navigate to each server in your cluster (in seperate tabs of your web browser) begin with the publisher, then each subscriber.  Navigate to Cisco Unified OS Administration > Security > Certificate Management > Find
    • Select the ITLRecovery pem Certificate.
    • Once open select Regenerate and wait until you see the Success pop-up then close pop-up or go back and select Find/List
  2. Continue with subsequent Subscribers; follow the same procedure in step 2 and complete on all subscribers in your cluster
  3. After all Nodes have regenerated the ITLRecovery certificate, services will need to be restarted in the order as follows:
    • If you are in Mixed Mode – Update the CTL before you proceed Token - Tokenless
    • Log into Publisher's Cisco Unified Serviceability
      • Navigate to Cisco Unified Serviceability > Tools > Control Center - Network Services
      • On the publisher select Restart on Cisco Trust Verification Service.
    •  Once the service restart completes, continue with the subscribers and restart the Cisco Trust Verification Service
  4. Reboot all Phones
    • Cisco Unified CM Administration > System > Enterprise Parameters
    • Select Reset then you will see a pop-up with the statement You are about to reset all devices in the system. This action cannot be undone. Continue?,select OK and then select Reset
  5. Phones will now upload the new ITL/CTL while they reset.


Delete Expired Trust Certificates

Note: Identify the trust certificates that need to be deleted, no longer required, or have expired.  Do not delete the five base certificates which include the CallManager.pem, tomcat.pem, ipsec.pem, CAPF.pem and TVS.pem. Trust certificates can be deleted when appropriate.  The service restarts below are designed to clear any in memory information of legacy certificates within those services.

  1. Navigate to Cisco Unified Serviceability > Tools > Control Center - Network Services
    • From the drop down select the CUCM Publisher
    • Select Stop Certificate Change Notification
    • Repeat for every Call Manager node in your cluster
    • If you have an IMP Server
      • From the drop down menu select your IMP servers one at a time and Select Stop Platform Administration Web Services and Cisco Intercluster Sync Agent
  2. Navigate to Cisco Unified OS Administration > Security > Certificate Management > Find
    • Find the expired trust certificates. (For versions 10.X and higher you can filter by Expiration. Fr versions below 10.0 you will need to identify the specific certificates manually or via the RTMT alerts if received)
    • The same trust certificate can appear in multiple nodes. It must be deleted individually from each node.
    • Select the trust certificate to be deleted (dependent on your version you will either get a pop-up or you will be navigated to the certificate on same page)
      • Select Delete (you will get a pop-up that begins with you are about to permanently delete this certificate...)
      • Select OK
  3. Repeat the process for every trust certificate to be deleted
  4. Upon Completion, services will need to be restarted that are directly related to the certificates deleted. You do not need to reboot phones in this section.  Call Manager and CAPF will be endpoint impacting.
    • Tomcat-trust: restart Tomcat Service via command line (See Tomcat Section)
    • CAPF-trust: restart Cisco Certificate Authority Proxy Function (see CAPF Section) Do not reboot endpoints
    • CallManager-trust: CallManager Service/CTIManager (See CallManager Section) Do not reboot endpoints
      • Impacts endpoints and causes restarts
    • IPSEC-trust: DRF Master/DRF Local (See IPSEC Section)
    • TVS (Self-Signed) does not have trust certificates
  5. Restart Services Previously Stopped in step 1

Verify

There is currently no verification procedure available for this configuration.

Troubleshoot

There is currently no specific troubleshooting information available for this configuration.