Chapter 1: Introduction

This document provides basic problem determination procedures when failures occur with the AKM server.

Change log

Version Date Description
2.1.13.001 6/18/2012 Initial release
2.1.13.002 9/12/2013 Application of style guide, clarifications, additional screenshots, addition of index.
3.0.0.001 2/10/2015 Updates.
4.5.3.001 5/22/2017 Updates for new File Manager.

Chapter 2: Problem Determination

This document is a set of step-by-step procedures to identify and resolve problems with the Alliance Key Manager Enterprise Edition key management server.

IMPORTANT: If you are experiencing a business interruption and you have not yet activated your failover server, you should consider doing that now.

Alliance Key Manager key retrieval or encryption service problems

Use these steps when your applications cannot retrieve encryption keys or perform remote encryption or decryption tasks.

 

Step 1 - Verify a connection to the key server’s web service

Obtain the IP address of the key server from your network administrator. Use a web browser on your PC to open a secure HTTPS connection to port 3886 on the key server. If you are using an older Windows Internet Explorer on Windows XP, change to Firefox or another browser that supports AES128 or AES256 connections. You may receive a message about an untrusted connection or an unknown certificate. Accept these exceptions and continue. Your browser URL path should look something like this:

https://10.0.0.1:3886

If you are unable to connect to the server, skip to Step 2.

If you receive a login panel, continue with the following steps:

Log in to the web interface using the administrative password:

image alt text

NOTE: If you have permanently lost the password, you must initiate recovery operations. Stop now and open a problem ticket on the Townsend Security support site.

If you are able to log in, continue with the following steps:

Expand the options in the left frame of the web interface. Click the Running Processes link. You may receive a browser message about an untrusted certificate, or your browser may block pop-ups. Accept the security exception and accept pop-ups for this session.

Scroll down and locate an entry under the “Command” column for akmd which looks like this:

/usr/sbin/akmd

image alt text

If you are not able to find this entry, select the option under Custom Commands to Start AKM. Recheck the “Running Processes” list to be sure the process started. If the process started, you are finished with problem determination.

If the process is not running, or will not re-start, continue with the following steps:

Click the File Manager link in the left frame. Navigate to the /var/log/townsend directory. Select the file akmerror.log and double click to download the file.

If you are not able to view the file, skip to Step 2.

If you are able to view the file akmerror.log, continue with the following steps:

Navigate to the bottom of the akmerror.log file entries to view the most recent entries in the log file. Observe the error messages or 4-digit error codes. Examples of error messages are messages that refer to an invalid license, an OpenSSL session failure, etc. Please see the AKM Error Codes Reference for more information.

If you are able to resolve the problem, you are finished with problem determination.

If there are error messages and you cannot resolve the problem, open a problem ticket on the Townsend Security website.

If you reach this point and the problem is not resolved, and you observe audible alarms from the key manager server, please continue with hardware problem determination in Step 2.

Hardware problems

 

Step 2 - Hardware problem determination

Locate the key server in your data center. Observe the power indicator light.

If the power indicator light is not on, apply power to the server and wait a few moments for the server to start. If the server starts, verify proper operation of the key server in Step 1.

If the power indicator light is on, continue with the following steps:

Listen for an alarm from the server. A disk failure or power supply failure will produce an audible alarm.

If you can hear an alarm, open a problem ticket on the Townsend Security website and submit a request to initiate a disk or power supply replacement.

If you do not hear an alarm, continue with the following steps:

Check that network cables are plugged in and secured.

Observe the status lights on the front of the key server. Do you observe any disk lights that are red?

If you observe a disk light that is red, open a problem ticket on the Townsend Security website.

If you do not observe a disk light that is red, continue with the next step.

Observe the remaining status lights on the front of the server. Please see the AKM Hardware Installation Guide EEfor more information on the status lights.

If you observe any other red status lights, open a problem ticket on the Townsend Security website and ask to start hardware diagnostics.

If you do not observe any other red status lights, continue with problem determination.

At this point you should have verified that the key server web interface is active, the key management application (akmd) is active, there are no errors in the akmserver.log file, and that there are no obvious hardware errors. Please continue with the following steps.

Networking problems

Step 3 - Networking problems - system log collection

System logs are not being received by your log collection server or SIEM server. This is usually caused by a missing or invalid network route. Use a web browser to connect to port 3886 on the key server. Log on and expand the options in the left frame. Select Network Configuration. Click Routing and Gateways.

image alt text

If no gateway is configured, or an invalid gateway address is configured, add a TCP route, or change the TCP route to provide a path for sending system logs.

Continue with the following steps.

AKM Administrative Console problems

Step 4 - AKM Administrative Console problems

This section of the problem determination guide assumes that you have installed the AKM Administrative Console Windows application. Please use the following procedures for problem determination.

You try to add a new key server definition, but you can’t locate the Java Key Store file.

Your security administrator should provide you with the JKS file for the administrative client, and the JKS file for the certificate authority. After you receive the necessary JKS files you can now configure the AKM key server definition. You are finished with problem determination.

If the problem is not resolved, continue with the following steps.

You have configured a key server definition, but you cannot connect to the key server to create or manage keys.

The IP address or port number for the administration may be incorrect. The default port number for administration is 6001. Consult with your security administrator and correct the IP address and/or port number. If this resolves the problem you are finished.

There may not be a TCP route to the key server. Consult with your network administrator to determine the proper gateway for connections to the key server. Update your Windows network configuration accordingly. If this resolves the problem, you are finished.

If the problem is not resolved, continue with the following steps.

Enabling verbose logging for troubleshooting

If the administrative console is operational, use the Set Log Level command to turn logging level to 50. Try connecting from the client using the Admin NOOP command. Observe the response. If unsuccessful, log in to the web interface and view the akmtrace.log file via File Manager. See if you can resolve the problem, or contact Townsend Security for assistance.

When done with verbose logging, set the log level back to 00. Running with trace enabled will quickly use up hard disk space to store verbose log files.

You are able to connect the administrative console to the key server, but you can’t authenticate with Dual Control.

One security administrator must send the Authorize Administrator command with a lease time, and a second administrator can then perform key management for that period of time. Try sending the Authorize Administrator command again.

If you are still not able to perform work as the second administrator, ask your network administrator to log in to the web interface and view the akmerror.log file in File Manager.

Client connection problems

Step 5 - Client connection problems

If your client application is having trouble connecting with the AKM server, check your client configuration. Observe the key server configuration information, then log in to the web interface and view the akm.conf file via File Manager. Check to make sure you have the right IP addresses and port numbers in your client configuration.