Redp5707 - IBM Storage Scale - Encryption
Redp5707 - IBM Storage Scale - Encryption
Phillip Gerrard
Luis Bolinches
Mika Heino
Sukumar Vankadhara
Redpaper
IBM Redbooks
March 2024
REDP-5707-00
Note: Before using this information and the product it supports, read the information in “Notices” on page v.
This edition applies to IBM Storage Scale 5.1.8.0 and IBM Storage Scale System 6.1.8.3 on IBM Storage
Scale System 3500.
Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .v
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Now you can become a published author, too! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii
Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii
Stay connected to IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii
This information was developed for products and services offered in the US. This material might be available
from IBM in other languages. However, you may be required to own a copy of the product or product version in
that language in order to access it.
IBM may not offer the products, services, or features discussed in this document in other countries. Consult
your local IBM representative for information on the products and services currently available in your area. Any
reference to an IBM product, program, or service is not intended to state or imply that only that IBM product,
program, or service may be used. Any functionally equivalent product, program, or service that does not
infringe any IBM intellectual property right may be used instead. However, it is the user’s responsibility to
evaluate and verify the operation of any non-IBM product, program, or service.
IBM may have patents or pending patent applications covering subject matter described in this document. The
furnishing of this document does not grant you any license to these patents. You can send license inquiries, in
writing, to:
IBM Director of Licensing, IBM Corporation, North Castle Drive, MD-NC119, Armonk, NY 10504-1785, US
This information could include technical inaccuracies or typographical errors. Changes are periodically made
to the information herein; these changes will be incorporated in new editions of the publication. IBM may make
improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time
without notice.
Any references in this information to non-IBM websites are provided for convenience only and do not in any
manner serve as an endorsement of those websites. The materials at those websites are not part of the
materials for this IBM product and use of those websites is at your own risk.
IBM may use or distribute any of the information you provide in any way it believes appropriate without
incurring any obligation to you.
The performance data and client examples cited are presented for illustrative purposes only. Actual
performance results may vary depending on specific configurations and operating conditions.
Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products and cannot confirm the
accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the
capabilities of non-IBM products should be addressed to the suppliers of those products.
Statements regarding IBM’s future direction or intent are subject to change or withdrawal without notice, and
represent goals and objectives only.
This information contains examples of data and reports used in daily business operations. To illustrate them
as completely as possible, the examples include the names of individuals, companies, brands, and products.
All of these names are fictitious and any similarity to actual people or business enterprises is entirely
coincidental.
COPYRIGHT LICENSE:
This information contains sample application programs in source language, which illustrate programming
techniques on various operating platforms. You may copy, modify, and distribute these sample programs in
any form without payment to IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating platform for which the sample
programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore,
cannot guarantee or imply reliability, serviceability, or function of these programs. The sample programs are
provided “AS IS”, without warranty of any kind. IBM shall not be liable for any damages arising out of your use
of the sample programs.
The following terms are trademarks or registered trademarks of International Business Machines Corporation,
and might also be trademarks or registered trademarks in other countries.
AIX® IBM Security® Redbooks (logo) ®
Db2® Power® WebSphere®
Guardium® POWER®
IBM® Redbooks®
Intel, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of Intel
Corporation or its subsidiaries in the United States and other countries.
The registered trademark Linux® is used pursuant to a sublicense from the Linux Foundation, the exclusive
licensee of Linus Torvalds, owner of the mark on a worldwide basis.
Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other
countries, or both.
Red Hat is a trademark or registered trademark of Red Hat, Inc. or its subsidiaries in the United States and
other countries.
Other company, product, or service names may be trademarks or service marks of others.
This IBM® Redbooks® Redpaper discusses the configuration, implementation and use of
encryption with IBM Storage Scale systems. It is intended to be used by technical
professionals and anyone who wants to know more about encryption with IBM Storage Scale
systems or anyone who is hands on with the software and devices.
Authors
Phillip Gerrard is a Project Leader for the International Technical Support Organization
working out of Beaverton, Oregon. As part of IBM for over 15 years he has authored and
contributed to hundreds of technical documents published to IBM.com and worked directly
with IBM's largest customers to resolve critical situations. As a team lead and Subject Matter
Expert for the IBM Storage Protect support team, he is experienced in leading and growing
international teams of talented IBMers, developing and implementing team processes as well
as creating and delivering education. Phillip holds a degree in computer science and
business administration from Oregon State University.
Luis Bolinches is part of the IBM Storage Scale development team. He has been working
with Scale since version 3.4 and with ESS prior to GA. With a background of Networking,
Power® systems and Linux, he is regularly involved with large customer engagements,
development of ESS deployment solutions, and customer facing events.
Mika Heino is a senior IT Management Consultant with IBM Lab Services working in Finland
for local and international IBM accounts. He has a degree in Telecommunications and
Computer Science from Turku University of Applied Sciences. Mika has 25 years experience
with Linux, IBM AIX® and IBM i, and server virtualization for both Intel and IBM POWER®. He
is a Master Certified Technical Specialist for Storage Systems with more than 15 years of
experience with storage area networks (SANs), IBM Storage Systems, and storage
virtualization.
Find out more about the residency program, browse the residency index, and apply online at:
ibm.com/redbooks/residencies.html
Comments welcome
Your comments are important to us!
We want our papers to be as helpful as possible. Send us your comments about this paper or
other IBM Redbooks publications in one of the following ways:
Use the online Contact us review Redbooks form found at:
ibm.com/redbooks
Send your comments in an email to:
[email protected]
Mail your comments to:
IBM Corporation, IBM Redbooks
Dept. HYTD Mail Station P099
2455 South Road
Poughkeepsie, NY 12601-5400
Storage Scale clusters can span global environments and consist of various disk
technologies. There is a need to protect this data against unauthorized access regardless of
where it is stored. Beginning with General Parallel File System (GPFS) 4.1, the encryption
feature is available for configuration and use. Encryption is implemented in Storage Scale file
systems with the following features:
No special requirements for various storage hardware
Generates finer levels of security granularity by cascading different security levels by using
specific keys
Provides Federal Information Processing Standard (FIPS)-compliant encryption
Data at rest means that the data is encrypted on disk and is decrypted by the Storage Scale
layer before it is sent to the requester. Protection of data at rest uses encryption to make data
unreadable to any party who does not possess the necessary decryption keys.
Encryption is often paired together with secure data deletion for increased data security.
When you use modern storage hardware, it is difficult, and sometimes impossible for SSDs, to
verify that a piece of data is deleted, overwritten, or made unavailable or inaccessible to a
cyberattacker. Secure data deletion allows an extra layer of security by removing encryption
data from the remote key server, which effectively makes the data unreadable even if it is
accessed.
The use of encryption at the file system level does not actively protect against network traffic
snooping. If data confidentiality and possible traffic snooping is a concern, the use of encryption at the
scale server level might be a better choice. One beneficial side effect of encrypting data blocks
before the node sends them to a logical block device or an NSD server is that it prevents an
attacker from reading the unencrypted network traffic.
The proposed solution does not claim to defend against an active network cyberattacker that
modifies or crafts its own packets. The cyberattacker might, for example, compromise another
Storage Scale node or delete data that belongs to other tenants. In these cases, the use of a
cipherList can be used to help defend against an active network attacker.
The most common use case that is addressed by the encryption feature is that some
customers require a Federal Information Processing Standard (FIPS) 140-2 compliant
solution. The implementation procedure that is provided in this IBM Redbooks publication
describes the steps to build a FIPS-compliant solution. The GSKit that is included with
Storage Scale is FIPS certified, and it is used by Storage Scale for file system crypto-related
functions.
This chapter helps to provide details about the configuration procedure for an encryption
capable Storage Scale environment and provides common-use encryption cases. For more
Note: Encryption is only supported on IBM AIX, Linux x86_64, and Linux ppc64le.
Encryption on a Windows operating system is not supported as of the writing of this
IBM Redbooks publication.
Note: For more information and for updates about supported RKM versions, see
IBM Storage Scale Overview.
Every file is encrypted and decrypted using its file encryption key (FEK). The i-node metadata
for the file system is not encrypted.
The FEK is stored and encrypted in the extended attribute (EA) in the i-node.
The FEK is used for encrypting and decrypting the data on disk and is not accessible directly.
All FEKs are encrypted with a master encryption key (MEK) when stored in the i-node.
An FEK cannot be stored unencrypted and is encrypted with at least one MEK. Therefore, to
access the FEK, you must have access to all the MEKs that were used to encrypt the FEK.
It is possible to wrap an FEK up to eight times with any other specific combination of MEKs.
See Figure 1-1 on page 4.
Note: It is possible to wrap an FEK up to eight times with any other specific combination of
MEKs.
The Remote Key Manager (RKM) must be from a supported key server within the Storage
Scale environment. In this publication, an IBM Security® Key Lifecycle Manager (SKLM) is
configured and shows how Storage Scale uses it as an RKM. This includes showing how a
Storage Scale node gets the MEK from the RKM by authorizing itself with a certificate called
keystore.
Separating the key management from the Storage Scale cluster allows for a higher level of
security by managing the keys on a system outside of Storage Scale. By separating the
administrator rights, you can ensure that even an administrator with superuser rights from
inside the cluster cannot change the key configuration on the remote key management
system.
This approach is referred to as the “Four eyes principle” or “Two persons rule”.
Whichever supported RKM is chosen, always use a highly available (HA) configuration. If the
access to RKM is lost, it likely has a negative impact on data accessibility. It is also
tremendously important to back up the MEK. Losing the keys on the MEK can cause access
issues and cause data to no longer be viable. This IBM Redbooks publication provides an
example that shows how to set up an HA SKLM as the RKM.
Because MEKs are stored remotely, access to the MEKs is managed on a per node basis
with certificates. These certificates are held and accessed out of a keystore of a node. This
means that a node is required to have a valid certificate to get access to the MEK.
The policy rules that are defined in the Storage Scale environment determine whether a file
gets encrypted or not. The encryption state of a file is determined at file creation time by
encryption-specific Storage Scale file placement policy rules. A maximum of eight rules can
be applied per file. All encryption and decryption operations happen in the node that is
performing the write and read operations of the file.
The primary goal for using encryption in an IBM Storage Scale environment is to ensure that
the data at rest is protected from unauthorized access. An individual might gain physical
access to a disk removed from the environment or gain the ability to access data from nodes
that are not explicitly allowed. Because the at rest data is encrypted, the data is not in a
readable format, which renders the data useless to that individual.
IBM Storage Scale encryption can also be viewed as a way to control access to the data in a
file system with the use of keys. This combined with multi-clusters provides a method of
access control for data in a shared environment. This example is also covered within this
IBM Redbooks publication.
The SED provides hardware-based encryption, which protects the data at rest by locking the
drives while the power is off. Also, SED also supports hardware-based crypto erase. GNR
uses these SED capabilities to build and manage drives and provides encryption at rest
capabilities for all GNR data.
Note: Read the product documentation to confirm which IBM Storage Scale Server
configurations are supported for encryption within GNR.
The use of SED features with GNR includes the following benefits:
Use of Cryptographic Erase for disposal of disks
Instantaneous change of Authentication key (rekey) without loss of data
Encryption of all data with minimal performance degradation
Simplified key management
Only signed firmware can be loaded onto an SED
Note: Always check product documentation for details on which key servers are supported
for GNR encryption.
This MEK is sensitive and requires careful handling. A compromised MEK can potentially
compromise all data that is protected by this key, and the loss of an MEK is equivalent to
losing access to all drives encrypted using this MEK. It is an IBM policy that if IBM facilities
are used to store encrypted data belonging to a customer, the responsibility of encryption key
management lies with the customer. IBM does not store the encryption key.
When an SED is removed from its owner's system it loses power, and its recorded data is
locked against unauthorized access. When power is restored, the host system must prove
that it owns the drive. This is done by providing the drive with the appropriate ownership
credentials or password. These credentials are provided to the drive through the MEK for the
GPFS cluster.
The following Figure 1-2 shows an overview of the components that are involved in GNR
encryption.
When selecting which encryption method to use for securing data, these examples are just a
few scenarios to consider. This is not a comprehensive list of all the possible points to be
considered before making a final decision.
This chapter includes a detailed step-by-step procedure to set up an environment for running
an encrypted file system.
For a successful installation of the GKLM server on an RHEL 8.8 host with a minimal installation, some
additional software packages are required. The additional packages that are installed are listed in
Example 2-2 and can be found with the RHEL 8.8 OS installation media.
IBM GKLM software installation also requires some of the OS settings to be set to specified
values. The settings are listed in Example 2-4.
1. Installation packages that are used for GKLM installation are stored on a local server at
/root/GKLM420/. See Example 2-1.
Example 2-1
SGKLM_4.2.0_F_LINUX_SERVER_1OF2.tar
SGKLM_4.2.0_F_LINUX_SERVER_2OF2.tar
4.2.0-ISS-GKLM-FP0001-Linux.tar.gz
SGKLM_4.2.0_LICENSE_MP.zip
2. Additional RHEL8 packages must be installed with the minimal installation. These are
required by IBM GKLM for installation. See Example 2-2.
Example 2-2
root@gklm01 GKLM420]# yum install ksh
[root@gklm01 GKLM420]# yum install libstdc++-8.5.0-18.el8.i686
[root@gklm01 GKLM420]# yum install pam-1.3.1-25.el8.i686
[root@gklm01 GKLM420]# yum install unzip
[root@gklm01 GKLM420]# yum install binutils
[root@gklm01 GKLM420]# yum install net-tools
3. Before installing GLKLM, rename the license file and decompress the installation files.
See Example 2-3.
4. Change the U-mask and confirm the configured shell. Set the umask to 0022. The bash
shell must be installed. See Example 2-4.
Example 2-4
[root@gklm01 disk1]# umask
0022
[root@gklm01 disk1]# which bash
/usr/bin/bash
Example 2-5
[root@gklm01 disk1]# firewall-cmd --zone=public --permanent --add-port=9443/tcp
Success
[root@gklm01 data]# firewall-cmd --zone=public --permanent --add-port=1441/tcp
success
[root@gklm01 data]# firewall-cmd --zone=public --permanent --add-port=5696/tcp
Success
[root@gklm01 data]# firewall-cmd --zone=public --permanent --add-port=1111/tcp
success
[root@gklm01 disk1]# firewall-cmd --reload
success
[root@gklm01 disk1]# firewall-cmd --list-ports
1441/tcp 2222/tcp 5696/tcp 9443/tcp
[root@gklm01 disk1]#
Example 2-6
[root@gklm01 disk1]# ls -la /tmp/
total 602144
drwxrwxrwt. 19 root root 4096 Aug 4 17:00 .
7. Disable SELINUX. A reboot is required for changes to take effect. See Example 2-7.
Example 2-7
[root@gklm01 disk1]# vi /etc/selinux/config
8. Create an encryption string that is used to encrypt the passwords for the internal
IBM Db2® Admin and sklmAdmin users then define passwords for their use. See
Example 2-8.
Example 2-8
[root@gklm01 /] # ./root/GKLM420/disk1/im/tools/imcl encryptString OME1por2@per3
1y232N6S6QG3PNdv1NvFbg==
[root@gklm01 /] #
9. Modify the silent installation file used for GKLM server installation. See Example 2-9.
Example 2-9
[root@gklm01 disk1] # vi SKLM_Silent_Linux_Resp.xml
<server>
<repository location='/root/GKLM420/disk1/im'/>
<repository location='/root/GKLM420/disk1/'/>
</server>
<!--The DB2 Group name should not be more than 8 characters -->
<data key='user.DB2_ADMIN_PWD,com.ibm.gklm42.db2.lin.ofng
value='1y232N6S6QG3PNdv1NvFbg=='/>
<data key='user.CONFIRM_PASSWORD,com.ibm.gklm42.db2.lin.ofng'
value='1y232N6S6QG3PNdv1NvFbg=='/>
</profile>
<data key='user.SKLM_ADMIN_PASSWORD,com.ibm.gklm42.linux'
value='1y232N6S6QG3PNdv1NvFbg=='/>
<data key='user.SKLM_ADMIN_CONF_PWD,com.ibm.gklm42.linux'
value='1y232N6S6QG3PNdv1NvFbg=='/>
10.Perform GKLM server silent installation on host GKLM01. See Example 2-10.
Example 2-10
[root@gklm01 disk1]# ./silent_install.sh SKLM_Silent_Linux_Resp.xml -acceptLicense
No preinstalled IBM Installation Manager found on the system.
Installing IBM Security Guardium Key Lifecycle Manager v4.2.0.0
Fri Aug 4 12:22:51 EEST 2023 - SKLM Prerequisite check started.
Db2 pre-requisite check - PASSED.
Checking required shell - PASSED
Checking executable permissions - PASSED
Checking required umask - PASSED
Checking kernel parameters - WARNING
Checking SELinux - PASSED
Checking CPU speed - PASSED
Checking RAM - PASSED
Fri Aug 4 12:22:51 EEST 2023 - SKLM Prerequisite check - PASSED with WARNING.
The Prerequisite check passed with one or more warnings. Review the warning
details in /tmp/SKLMPrereqCheck.log. Press any key to continue.Installation of IBM
Security Guardium Key Lifecycle Manager is not supported on the current operating
system. Continuing the installation on an unsupported operating system.
Installed com.ibm.cic.agent_1.9.2003.20220917_1018 to the
/opt/IBM/InstallationManager/eclipse directory.
Installed com.ibm.gklm42.db2.lin.ofng_11.5.8.0 to the /opt/IBM/DB2GKLMV42
directory.
Installed com.ibm.websphere.liberty.BASE_22.0.12.20221107_1900 to the
/opt/IBM/WebSphere/Liberty directory.
Installed com.ibm.java.jdk.v8_8.0.7020.20221026_1851 to the
/opt/IBM/WebSphere/Liberty directory.
Installed com.ibm.gklm42.linux_4.2.0.0 to the /opt/IBM/GKLMV42 directory.
CRIMA1137W WARNING: The following packages do not support the 64-bit version of
Installation Manager that you are using: IBM Db2 version 11.5.8.0, IBM Security
Explanation: The 64-bit version of Installation Manager checks each package for
64-bit support. If a package does not support the 64-bit version, you receive a
warning.
User Action: Use a 32-bit version of Installation Manager to install the package.
WARNING: Support for using Java SE 7 and Java SE 7.1 with WebSphere Liberty ended.
The Liberty kernel can no longer run with Java SE 7 or Java SE 7.1.
The installation process is complete. Please look into Installation Manager logs
for details.
[root@gklm01 disk1]#
Note: Check the kernel parameter kernel_msgmni value with the command sysctl -a |
grep kernel.msgmni. If the value is set to current RAM * 1024 (in GB), ignore the warning
on kernel parameters pre-check.
Ignore the error message not supported operating system displayed during the
installation. This is an error that is returned from the install check utility for SGKLM 4.2.
RHEL8.8 is listed as supported. For more information, see IBM Security Guardium Key
Lifecycle Manager Support Matrix.
11.Modify Db2 permissions to allow GKLM access to the database on host GKLM01. See
Example 2-11.
Example 2-11
[root@gklm01 /]# su - klmdb42
12.Backup the current configuration and install the latest FixPack for GKLM on host GKLM01.
As of this writing the latest fix pack is version 4.2.0.1. See Example 2-12.
Example 2-12
[root@gklm01 /]# exec bash
[root@gklm01 /]# cd /opt/IBM/WebSphere/Liberty/bin/
Example 2-13
[root@gklm02 GKLM420]# cd /root/GKLM420/
[root@gklm01 GKLM420]# ./silent_updateSKLM.sh /opt/IBM/InstallationManager/
/opt/IBM/WebSphere/Liberty/
About to install FixPack...
Name: WebSphere Liberty V_220012
Launching InstallManager...
/opt/IBM/InstallationManager//eclipse/tools/imcl -input
/root/GKLM420/sklm/SKLM_Silent_Update_Linux_Resp.xml -silent -acceptLicense
Updated to com.ibm.java.jdk.v8_8.0.8000.20230314_0926 in the
/opt/IBM/WebSphere/Liberty directory.
Updated to com.ibm.websphere.liberty.BASE_23.0.3.20230319_1900 in the
/opt/IBM/WebSphere/Liberty directory.
Updated to com.ibm.gklm42.linux_4.2.0.1 in the /opt/IBM/GKLMV42 directory.
CRIMA1137W WARNING: The following packages do not support the 64-bit version of
Installation Manager that you are using: IBM Security Guardium Key Lifecycle
Manager version 4.2.0.1, IBM Security Guardium Key Lifecycle Manager installed
version 4.2.0.0. If you continue, you might have issues with installation and
deployment. For information about 64-bit mode support for a package, see the
package documentation.
Explanation: The 64-bit version of Installation Manager checks each package for
64-bit support. If a package does not support the 64-bit version, you receive a
warning.
User Action: Use a 32-bit version of Installation Manager to install the package.
WARNING: Support for using Java SE 7 and Java SE 7.1 with WebSphere Liberty ended.
The Liberty kernel can no longer run with Java SE 7 or Java SE 7.1.
Installation of fix pack is complete. See the Installation Manager log files for
more information.
[root@gklm01 GKLM420]#
The installation is performed using the operating system RHEL 8.8 from installation media.
The Red Hat installation type is a minimal installation without any additional packages selected.
Additional software packages are required for successful installation of GKLM server on RHEL 8.8
minimal installation. Additional packages installed are listed in Example 2-15 on page 16 and are on RHEL
8.8 OS installation media.
IBM GKLM software installation requires some of the OS settings to be set to specified
values. These settings are verified in Example 2-17 on page 16.
2. Install additional packages for RHEL8 minimal installation that are required by IBM GKLM
installation. See Example 2-15.
Example 2-15
[root@gklm02 GKLM420]# yum install ksh
[root@gklm02 GKLM420]# yum install libstdc++-8.5.0-18.el8.i686
[root@gklm02 GKLM420]# yum install pam-1.3.1-25.el8.i686
[root@gklm02 GKLM420]# yum install unzip
[root@gklm02 GKLM420]# yum install binutils
[root@gklm02 GKLM420]# yum install net-tools
3. Before installing GLKLM, rename the license file and decompress the installation files.
See Example 2-16.
4. Change U-mask and confirm the configured shell. The umask must be set to 0022 and a
bash shell must be installed. See Example 2-17.
Example 2-17
[root@gklm02 disk1]# umask
0022
[root@gklm02 disk1]# which bash
/usr/bin/bash
5. Add firewall rules to allow connection to the GKLM web-based GUI. The example uses the
default ports that are used by GKLM. See Example 2-18.
Example 2-18
[root@gklm02 disk1]# firewall-cmd --zone=public --permanent --add-port=9443/tcp
Success
[root@gklm02 data]# firewall-cmd --zone=public --permanent --add-port=1441/tcp
success
[root@gklm02 data]# firewall-cmd --zone=public --permanent --add-port=5696/tcp
Success
[root@gklm02 data]# firewall-cmd --zone=public --permanent --add-port=2222/tcp
success
[root@gklm02 disk1]# firewall-cmd --reload
success
[root@gklm02 disk1]# firewall-cmd --list-ports
1441/tcp 2222/tcp 5696/tcp 9443/tcp
6. Verify that /tmp permissions are set to 777. See Example 2-19.
Example 2-19
[root@gklm02 disk1]# ls -la /tmp/
total 602144
drwxrwxrwt. 19 root root 4096 Aug 4 17:00 .
7. Disable SELINUX. A reboot is required for changes to take effect. See Example 2-20.
Example 2-20
[root@gklm02 disk1]# vi /etc/selinux/config
8. Create an encryption string that is used to encrypt the passwords for the internal Db2
Admin and sklmAdmin users then define passwords for their use. See Example 2-21.
Example 2-21
[root@gklm02 /] # ./root/GKLM420/disk1/im/tools/imcl encryptString OME1por2@per3
1y232N6S6QG3PNdv1NvFbg==
[root@gklm02 /] #
9. Modify the silent installation file used for GKLM server installation. See Example 2-22.
Example 2-22
[root@gklm02 disk1] # vi SKLM_Silent_Linux_Resp.xml
<server>
<repository location='/root/GKLM420/disk1/im'/>
<repository location='/root/GKLM420/disk1/'/>
</server>
<!--The DB2 Group name should not be more than 8 characters -->
<data key='user.DB2_ADMIN_PWD,com.ibm.gklm42.db2.lin.ofng
value='1y232N6S6QG3PNdv1NvFbg=='/>
<data key='user.CONFIRM_PASSWORD,com.ibm.gklm42.db2.lin.ofng'
value='1y232N6S6QG3PNdv1NvFbg=='/>
</profile>
<data key='user.SKLM_ADMIN_PASSWORD,com.ibm.gklm42.linux'
value='1y232N6S6QG3PNdv1NvFbg=='/>
<data key='user.SKLM_ADMIN_CONF_PWD,com.ibm.gklm42.linux'
value='1y232N6S6QG3PNdv1NvFbg=='/>
Explanation: The 64-bit version of Installation Manager checks each package for
64-bit support. If a package does not support the 64-bit version, you receive a
warning.
User Action: Use a 32-bit version of Installation Manager to install the package.
WARNING: Support for using Java SE 7 and Java SE 7.1 with WebSphere Liberty ended.
The Liberty kernel can no longer run with Java SE 7 or Java SE 7.1.
Installation process is complete. Please look into Installation Manager logs for
details.
[root@gklm02 disk1]#
If the value is set to current RAM * 1024 (in GB), then ignore the warning from the kernel
parameters pre-check.
Ignore the error message not supported operating system displayed during the
installation. This is an error returned from the install check utility for SGKLM 4.2. RHEL 8.8
is listed as supported. For more information, see IBM Security Guardium Key Lifecycle
Manager Support Matrix.
11.Modify Db2 permissions to allow GKLM access to the database on host GKLM02. See
Example 2-24.
Example 2-24
root@gklm02 /]# su - klmdb42
12.Backup the current configuration and install the latest FixPack for GKLM on host GKLM02.
As of this writing the latest fix pack is version 4.2.0.1. See Example 2-25.
Example 2-25
[root@gklm02 /]# exec bash
Example 2-26
[root@gklm02 GKLM420]# cd /root/GKLM420/
[root@gklm02 GKLM420]# ./silent_updateSKLM.sh /opt/IBM/InstallationManager/
/opt/IBM/WebSphere/Liberty/
About to install FixPack...
Name: WebSphere Liberty V_220012
Launching InstallManager...
/opt/IBM/InstallationManager//eclipse/tools/imcl -input
/root/GKLM420/sklm/SKLM_Silent_Update_Linux_Resp.xml -silent -acceptLicense
Updated to com.ibm.java.jdk.v8_8.0.8000.20230314_0926 in the
/opt/IBM/WebSphere/Liberty directory.
Explanation: The 64-bit version of Installation Manager checks each package for
64-bit support. If a package does not support the 64-bit version, you receive a
warning.
User Action: Use a 32-bit version of Installation Manager to install the package.
WARNING: Support for using Java SE 7 and Java SE 7.1 with WebSphere Liberty ended.
The Liberty kernel can no longer run with Java SE 7 or Java SE 7.1.
Installation of fix pack is complete. See the Installation Manager log files for
more information.
[root@gklm02 GKLM420]#
The IBM GKLM server web-based GUI is accessible on every GKLM server installation at
https://<GKLM SERVER IP>:9443.
To install the GKLM server, the web-based GUI will be used for the following tasks:
GKLM Server certificate creation
GKLM Server replication setup between configured master and clone servers
Creation of GKLM Server web-based GUI users
During the initial installation of the IBM GKLM server instance, a default administrative user
account, SKLMAdmin, is automatically created.
The password for user SKLMAdmin is the password that was defined before the installation
when the encrypted password string was created. See Example 2-8 on page 11. For the
To login to IBM Security Guardium Key Lifecycle Manager, open a browser at the IP address
of the server.
Note: The downloaded and renamed license file gklm.license.zip must be available on
the workstation where the license installation is being performed.
Figure 2-2 Trial version warning message is shown until license is applied
2. After the server certificate is created, the server must be restarted for the changes to take
effect and to apply the certificate.
a. Select sklmadmin to open the drop-down menu. Select Restart Server.
After the server is restarted, the server certificate that is currently in use can be confirmed
from the Configuration section of the GUI.
Figure 2-8 Confirming the certificate being used by the GKLM Server
When the correct key name is being served, the configuration is complete.
After replication is configured, the following data on the master server is copied to the
replication server:
IBM Security Guardium Key Lifecycle Manager database tables
Truststore and keystore with the master key
IBM Security Guardium Key Lifecycle Manager configuration files
To restore configuration on any server, the password of the backup file must be known.
1. From the Administration drop-down menu, select Backup and Restore. See Figure 2-9.
3. Enter a password and, if wanted, a description into the Create Backup panel, then click
Create Backup. See Figure 2-11 on page 27.
Note: Ensure that the password is stored or recorded safely for any backup file created.
This password is required to be entered during the restore of this backup file.
After the backup operation is completed, the backup file name and location is displayed in
the list of currently available backups.
4. To prepare to create a clone of the master server, transfer the backup file from the master
server GKLM01 to the clone server GKLM02. See Example 2-27. The example uses scp
to transfer the backup file.
Example 2-28 Copying GKLM server backup to restore location on the clone server
[root@gklm02 / ]# cp
/opt/IBM/WebSphere/Liberty/products/sklm/data/sklm_v4.2.0.1_20230811121021+0300_backup.jar
/opt/IBM/WebSphere/Liberty/products/sklm/data/restore/
Note: If the owner of the backup file is not set to klmdb42: klmdb42 on the target
system, the restore is not performed and the backup file is not visible on the
web-based GUI.
3. Log in into the clone server GKLM02 web-based GUI and select Administration →
Backup&Restore.
4. After the window opens, click Browse, if necessary, to select the restore folder. See
Figure 2-13.
5. Click Display Backups to refresh the contents of that folder.
Figure 2-13 Displaying backup versions available for the GKLM server
6. Select a file from the list of available backup files and select Restore.
7. On the Restore from Backup window, provide the password that is associated with the
backup file. Click Restore Backup. See Figure 2-14 on page 29.
8. Click OK to start the restore operation. After the restore is completed, the server restarts.
See Figure 2-15.
After the restore is complete, the system the server displays a message that the restore
was successful. and then the IBM Security Guardium Key Lifecycle Manager application
restarts. See Figure 2-16 on page 30.
Note: Replication must be started on both the master and clone servers.
Test replication by clicking Replicate Now and confirm that replication was successful.
Note: More details related to replication events can be found by reviewing the logs on both
the master and clone servers. Replication Log files are located in
/opt/IBM/WebSphere/Liberty/products/sklm/logs/replication on each server.
In addition to the default Administrator user, other user IDs can be configured from the GKLM
server web-based GUI. Each user can be a member of a predefined or a user-created role
and group, depending on the role of the user. See Figure 2-22 on page 34.
Note: Users that are configured locally on a GKLM server instance are not replicated
between a Master and Clone server by GKLM replication. The administrator that defines
GKLM users must maintain users manually on each server.
Table 2-1 shows the configuration information for the current GKLM servers. See Table 2-2 on
page 35 for a list of GKLM clients to be added.
GKLM servers include the following components, which are installed and configured at initial
installation time:
IBM Security Key Lifecycle Manager
IBM WebSphere® Application Server
IBM Db2 relational database
IBM Security Key Lifecycle Manager stores its key materials in a Db2 relational database.
Manual backups of the GKLM server configuration and its stored keys can be performed from
the web-based GUI of the server at any time by selecting Administration →
Backup&Restore then clicking Create. See Figure 2-23 on page 35.
The GKLM server automatically backs up its configurations and stored keys every 24 hours.
Automatic backup of the configuration also occurs if new encryption keys are created or if the
configuration of the GKLM server is changed.
Manual and automatic backups are stored by default on a local file system of the GKLM
server. The backup files should be backed up periodically by an external backup procedure.
Clients
Each GKLM client is an IBM Storage Scale cluster. Table 2-2 lists all clusters currently
configured with GKLM-based encryption and their member nodes.
Installation and initial configuration of IBM Storage Scale client nodes and clusters is not
covered in this publication.
IBM Storage Scale cluster RB_Storage_Cluster has storage devices that are attached to it
and has a local IBM Storage Scale file system defined.
IBM Storage Scale cluster RB_Remote_Cluster does not have any local IBM Storage Scale
file systems and will be performing remote cluster mounts from RB_Storage_Cluster.
Both IBM Storage Scale clusters will be defined as clients of the GKLM servers.
Environment details for the IBM Storage Scale nodes and clusters are provided in Table 3-1.
Table 3-1 IBM Storage Scale nodes and cluster example settings
Name IP address Storage Scale Cluster Has local storage
version
Scale1 10.134.184.51 DME / 5.1.7.1 RB_Storage_Cluster.cloud.stg.for Yes
um.ibm.com
Scale2 10.134.184.52 DME / 5.1.7.1 RB_Storage_Cluster.cloud.stg.for Yes
um.ibm.com
Scale3 10.134.184.53 DME / 5.1.7.1 RB_Storage_Cluster.cloud.stg.for Yes
um.ibm.com
RemoteScale1 10.134.184.54 DME / 5.1.8.1 RB_Remote_Cluster.cloud.stg.fo No
rum.ibm.com
RemoteScale2 10.134.184.55 DME / 5.1.8.1 RB_Remote_Cluster.cloud.stg.fo No
rum.ibm.com
RemoteScale3 10.134.184.56 DME / 5.1.8.1 RB_Remote_Cluster.cloud.stg.fo No
rum.ibm.com
Moomi 10.134.184.47 DME / 5.1.8.1 RB_Muumilaakso.cloud.stg.foru No
m.ibm.com
In this example environment, the encryption is implemented with file system policy per fileset.
Access to data is granted for the clients at IBM Storage Scale cluster level per each encrypted
fileset.
We create 5 different independent filesets inside file system fs01 and enable encryption for
these filesets as shown in Table 3-2 on page 39.
fs01 No
fs01 Muumitalo No
Example 3-1 Checking the IBM Storage Scale cluster RB_Storage_Cluster configuration
[root@scale1 ~]# mmlsconfig FIPS1402mode
FIPS1402mode no
[root@scale1 ~]#
[root@RemoteScale1 ~]# mmlsconfig nistCompliance
nistCompliance SP800-131A
[root@RemoteScale1 ~]#
Example 3-2 Checking the IBM Storage Scale cluster RB_Remote_Cluster current configuration.
[root@RemoteScale1 ~]# mmlsconfig FIPS1402mode
FIPS1402mode no
[root@RemoteScale1 ~]#
[root@scale1 ~]# mmlsconfig nistCompliance
nistCompliance SP800-131A
[root@scale1 ~]#
GKLM server processes must be restarted for configuration changes to take effect.
Add the GKLM servers GKLM01 and GKLM02 to encryption configuration on cluster
RB_Storage_Cluster.
Perform the following commands from any of the nodes in the cluster.
1. Add keyservers to Storage Scale cluster RB_Storage_Cluster by using the mmkeyserv
command. See Example 3-5.
[root@scale1 ~]#
3. Create a Tenant (Device Group) RBStorageCluster to server GKLM01. See Example 3-7.
Perform the following commands from any of the nodes in the cluster.
1. Add keyservers to IBM Storage Scale cluster RB_Remote_Cluster. See Example 3-11.
[root@RemoteScale1 ~]#
3. Add a tenant (Device Group) RBRemoteCluster to server GKLM01. See Example 3-13.
Example 3-13 Adding a tenant (Device Group) RBRemoCluster to GKLM server GKLM01
[root@RemoteScale1 ~]# mmkeyserv tenant add RBRemoteCluster --server
gklm01.cloud.stg.forum.ibm.com
Enter password for the key server gklm01.cloud.stg.forum.ibm.com:
mmkeyserv: Propagating the cluster configuration data to all
affected nodes. This is an asynchronous process.
[root@RemoteScale1 ~]#
Perform the following commands from any of the nodes in the cluster.
1. Add keyservers to IBM Storage Scale cluster RB_Muumilaakso. See Example 3-17.
[root@moomi ~]#
3. Add a Tenant (Device Group) Muumilaakso to GKLM server GKLM01. See Example 3-19.
Example 3-19 Adding a Tenant (Device Group) Muumilaakso to GKLM server GKLM01
[[root@moomi ~]# mmkeyserv tenant add RBMuumilaakso --server
gklm01.cloud.stg.forum.ibm.com
Enter password for the key server gklm01.cloud.stg.forum.ibm.com:
mmkeyserv: mmsdrfs propagation completed.
[root@moomi ~]#
6. Create a key for cluster RB_Muumilaakso on tenant RBMuumilaakso. See Example 3-22.
3. Show the registered remote key manager (RKM) for RB_Storage_Cluster. See
Example 3-25.
Example 3-29 Showing the registered client settings for RB_Muumilaakso cluster
[root@moomi ~]# mmkeyserv client show
RBClientCluster3
Label: RBClientCluster3
Key Server: gklm01.cloud.stg.forum.ibm.com
Tenants: RBMuumilaakso
Certificate Expiration: 2026-08-23 15:39:50 (+0300)
Certificate Type: system-generated
[root@moomi ~]#
2. Show the registered tenants for RB_Muumilaakso cluster. See Example 3-30.
[root@moomi ~]#
Always review current IBM documentation for remote mounts before configuring a new
environment. For the latest version at the time of writing, see Mounting a remote GPFS file
system.
It is also possible to mount filesets instead of file systems on the remote clusters. For more
information, see mmauth command.
On the owningCluster system, the system administrator issues the mmauth genkey command
to generate a public and private key pair. The key pair is placed in /var/mmfs/ssl. The public
key file is id_rsa.pub.
1. Run the mmauth genkey command. See Example 3-32
5. On the accessingCluster system, the system administrator issues the mmauth genkey
command to generate a public/private key pair. The key pair is placed in /var/mmfs/ssl.
The public key file is id_rsa.pub. See Example 3-35.
The system administrator of the owningCluster can rename the key file and put it in any
available directory on the node on which they are working. In Example 3-37 on page 51, the
key files are renamed to id_rsa_RB_Remote_Cluster.pub and id_rsa_RB_Muumilaakso.pub
during the scp transfer to reflect the name of the cluster that owns this key.
On the owningCluster system run the mmauth add command to authorize
accessingCluster to mount file systems that are owned by owningCluster. This is done by
using the key file that was received from the administrator of the accessingCluster system.
See Example 3-38.
Note: When you use IBM Storage Scale, instead of mounting a complete file system, you
can mount filesets.
If the accessing cluster is mounting the remote file system in read-only mode, then only a
subset of the events is generated. For more information, see File audit logging events.
9. On accessingCluster, use the mmremotecluster add command to define the cluster name,
contact nodes and public key for owningCluster. See Example 3-40.
10.By using the mmremotefs command, the system administrator of accessingCluster can
locate the serving cluster and mount its file systems.
On accessingCluster, RemoteScale1 and moomi, run one or more mmremotefs commands
to identify the file systems in owningCluster that are to be accessed by nodes in
accessingCluster. See Example 3-41.
A file system can have only one active policy installed. Adding encryption rules into an
existing file system can be done by adding the encryption rules to the existing policy file and
installing the policy for the file system with the mmchpolicy command. The mmchpolicy
command is shown in Example 3-50 on page 58.
To review or confirm the file system current policy name, installation time and “installed by”
information use the mmlspolicy filesystem command. See Example 3-43.
Encryption on IBM Storage Scale can be implemented either on a file system or a fileset level.
For this environment, the encryption of data is defined to the fileset with an encryption policy
enabled in a parent file system.
When no encryption policy is applied to a file system or to a fileset within a file system, the file
system attribute encryption is set to its default value of no. This can be checked using the
mmlsfs command. See Example 3-44.
When encryption is enabled for the first time either on a file system or on a member file set,
the attribute for encryption for the entire file system is set to enabled. This setting cannot be
reverted after it is enabled.
Note: After the encryption attribute for the file system has been set to enabled, any client
node mounting the file system must be running a version of Storage Scale that supports
encryption.
If the version of IBM Storage Scale that is installed on the client nodes does not support the
encryption feature, the nodes cannot mount the entire file system whether the encryption
policy includes that file system.
GKLM infrastructure is critical to prevent the loss of access or loss of data. If you lose access
to the GKLM server completely, there is no way to recover your data from the IBM Storage
Scale file system. If the key server is lost, data can be recovered from only a restored data
backup.
By default, the GKLM server is contacted at the time that the file system is mounted and then
every time the encryptionKeyCacheExpiration seconds pass. The default is 900 seconds,
which is 15 minutes. If a node cannot reach the GKLM server, it loses access to the encrypted
file system. When setting encryptionKeyCacheExpiration to 0, the check with SKLM happens
only during the file system mount. Certain security implications apply if
encryptionKeyCacheExpiration is set to 0 and there is no rechecking of access to the key
server.
It is suggested that when using GKLM, follow the “four eyes” principle. This means that no
single person or team has access to both the GKLM system and the IBM Storage Scale
nodes.
Example 3-46 list the contents of the policy file FinlandEnc.policy. Installation of the policy is
shown in Example 3-50 on page 58.
Note: Use the correct RKM ID for the key at the end of the key string, separated with
“:”-sign. RKM ID is displayed during the key creation in Example 3-10 on page 43.
To check encryption status of files stored on filesets Turku, Järvenpää, New_York and Tokyo on
file system fs01 use the mmlsattr -L command:
Activate and confirm the policy on file system fs01 as shown in Example 3-50.
[root@scale1 ~]#
Example 3-51 Use mmlsfs and mmlsattr commands to check encryption status
[root@scale1 ~]# mmlsfs fs01 --encryption
flag value description
------------------- ------------------------ -----------------------------------
--encryption yes Encryption enabled?
[root@scale1 ~]#
Note: After an encryption policy is applied to the file system, the file system attribute
encryption changes to yes.
In Example 3-51 on page 58, the attribute encrypted for files, which is already present for the
file system, does not change when a change is made to the encryption policy of the file
system.
Any new file that is created or any existing file that is modified after the policy for the file
system is assigned or updated has its attribute encrypted set to yes.
Example 3-52 New and updated versus existing file encryption settings
[root@scale1 ~]# vi /ibm/fs01/Järvenpää/LB_address
[root@scale1 ~]# mmlsattr -L /ibm/fs01/Järvenpää/LB_address | grep -i Encrypted
Encrypted: yes
[root@scale1 ~]# vi /ibm/fs01/Turku/LB_address
[root@scale1 ~]# mmlsattr -L /ibm/fs01/Turku/LB_address | grep -i Encrypted
Encrypted: no
[root@scale1 ~]#
As shown in Example 3-52, after changing a file system policy to encrypt contents on filesets
Turku and Järvenpää the files created on these filesets have their attribute encrypted set to
yes.
Only IBM Storage Scale cluster nodes that have encryption keys defined for tenant
RBStorageCluster are allowed to read or write files on filesets Turku and Järvenpää as shown
in Example 3-53.
[root@moomi Tokyo]# cd
[root@moomi ~]# cat /ibm/fs01/Järvenpää/LB_address
cat: /ibm/fs01/Järvenpää/LB_address: Operation not permitted
After the commands that are described in this section are run, the configuration of the
clusters matches Figure 3-1. The green lines between each cluster and fileset indicate that
the cluster has access to this fileset. If a cluster is linked to a fileset with a red line, it cannot
access the contents of those filesets.
To add encryption to filesets Tokyo and New_York, modify existing encryption policy
FinlandEnc.policy to include encryption settings for filesets Tokyo and New_York with an
encryption key for the IBM Storage Scale cluster RB_Remote_Cluster assigned to tenant
RBRemoteCluster.
Optionally, you can create a new policy file, copy the content of the currently active policy into
that file and then add additional rules for Tokyo and New_York encryption. If this is done, then
the new policy file must be assigned to the file system so that the new policy is used instead
of the current active policy.
Add the lines into an existing policy file. The contents of the modified policy file is shown in
Example 3-54.
[root@scale1 ~]#
Note: Encryption key for filesets Turku and Järvenpää is using a different encryption key
than filesets Tokyo and New_York.
Before you apply the new policy to file system fs01, tenant RBRemoteCluster must be added
to the configuration of cluster RBStorageCluster, which is the owner of the file system. See
step 3 on page 44. Also, add the owning cluster RBStorageCluster as a client to the newly
introduced tenant.
The process of adding the tenant and adding a client to the tenant is shown in Example 3-55.
Use the mmchpolicy command to apply a new policy to a file system fs01.
After the new policy has been applied to file system fs01 all new files created on filesests
Turku, Järvenpää, Tokyo and New_York are encrypted. Files created on other filesets on file
system fs01 are not encrypted.
Access to the encrypted filesets is allowed from all nodes of clusters RBStorageCluster and
RBRemoteCluster.
A 3rd cluster that remotely mounts file system fs01 from RBStorageCluster does not have
access to any encrypted content.
Turku Yes No No
Järvenpää Yes No No
3.6 File system and file access and attributes of encrypted files
When encryption is enabled for the IBM Storage Scale environment by installing a policy for a
file system, the following changes occur:
Any node that mounts an encrypted file system or a file system that contains a fileset that
is enabled for encryption must run an version of IBM Storage Scale that supports
encryption. A node without the gpfs.crypto package installed as part of the IBM Storage
Scale software installation cannot mount the entire file system. This is true even if the
encryption is applied only for a fileset within the file system being mounted and not for the
whole file system.
After the encryption is enabled for a file system or a fileset within the file system, the file
system, attribute encryption will be changed to yes. This change is permanent and the
attribute cannot be changed to no again.
Any file that exists on a file system when the encryption is enabled for that file system or a
fileset will NOT automatically be encrypted.
Any file created or modified after the encryption has been enabled for a file system or for a
fileset that is encrypted by the policy rules becomes encrypted.
When a new file is created on a location that has encryption enabled, the file has its
attribute Encrypted set to yes. To access an encrypted file, the client accessing the file
must have access to the encryption key associated with that file.
Contents of an encrypted fileset and folder can be listed by a client that does not have the
required encryption keys available, but any other access to the files is denied.
To determine whether a file is encrypted or not, examine the attributes of the file.
3. When the output from step 2 confirms all drives in each recovery group are SED capable,
use the mmkeyserv command to create a key by using a GKLM server.
4. After a key is created with step 3, use the mmvdisk sed enroll command to enable
encryption on drives in each recovery group.
5. Confirm that the output of the mmvdisk sed list command shows that all drives in each
recovery group are enrolled and unlocked.
6. To assign a new encryption key to the drives, use the mmvdisk sed rekey command to
change an existing key, such as one used to enroll the drives in step 4 on page 66, to a
new key. The rekey command uses a new Master Encryption Key (MEK) from the GKLM
server and configures all the SED drives to use the new key as the MEK.
Note: Use the mmhealth command to monitor the missing or locked state of the pdisk
that is triggering events or alerts. The output advises the user to determine whether the
drive is locked by using the mmvdisk sed list command.
When the user needs to replace a physical disk in the disk enclosure, the
mmlspdisk --not-ok command can help find the pdisk that needs to be replaced. After the
pdisk is located, the user can run the mmchcarrier command to start the replacement
procedure. After a drive is replaced with a new drive with the mmchcarrier command in the
GNR system that is configured for encryption, the new drive is configured to support
encryption, too. After a successful configuration, the SED drives are unlocked so that data
can be accessed and modified.
When the pdisks are deleted, SED is disabled, and the drives are restored back to their
original configuration.
4.7 Migration
When working with existing recovery groups where a file system is already created, and an
active workload is present, SED can be enabled as a migration. The migration can be done
while a workload is present on a live system. This migration can be done using the mmvdisk
sed enroll command. See the steps from 4.1, “Enabling encryption on IBM Storage Scale
system” on page 66 sub-steps 2-5. The migration uses an MEK from a GKLM server and
configures all the drives of a recovery group to enable SED support.
If for any reason during the migration the enroll process is interrupted, then the same
command can be reissued to resume the migration. All the drives of the recovery group must
be in the OK state to migrate the drives of a recovery group to use SED support.
REDP-5707-00
ISBN 0738421189
Printed in U.S.A.
®
ibm.com/redbooks