API, Microservices

Insights

Introduction

Cloud Data Access (CDA) is a critical new z/OS component documented in this paper due to its pivotal role in modernizing mainframe environments. It enables secure, programmatic, and seamless interaction with cloud object storage. CDA's RESTful API abstraction and secure credential management within DFSMSdfp facilitate hybrid cloud deployments, unlocking the scalability and cost-efficiency of cloud resources for z/OS workloads. By bridging the gap between z/OS and cloud object stores, CDA empowers mainframe applications with advanced data archiving, disaster recovery, and analytics capabilities, directly addressing the evolving demands of enterprise modernization.

CDA’s architecture acts as a vital intermediary, facilitating communication between z/OS applications and cloud object storage via RESTful APIs. The integrated secure credential store is a cornerstone of CDA’s design, effectively mitigating the risk of exposing sensitive authentication information within application code.

As a new z/OS technology, this document provides crucial API specifications to expedite early-stage Proof-of-Concept (PoC) development. While empirical validation through extensive production deployments is currently limited, the inherent design of CDA, which simplifies cloud access and ensures robust security, presents a significant theoretical advantage for modernizing z/OS environments. This documentation serves as a foundational resource for developers to explore and evaluate CDA’s potential, ultimately accelerating its adoption in enterprise mainframe modernization strategies.

Strategic Value of CDA

z/OS storage environments are increasingly adopting a hybrid cloud architecture, integrating on-premises disk and tape with cloud object storage. This approach leverages the strengths of both: on-premises storage for performance-sensitive data and cloud storage for backup, archive, and unstructured data. DFSMS components, such as Object Access Method (OAM) and Transparent Cloud Tiering (TCT) already utilize cloud object storage, demonstrating existing integration. Many z/OS clients are implementing this hybrid model to optimize their storage infrastructure. CDA consumes IBM hybrid cloud data management functions, simplifying the integration and management of cloud storage for z/OS applications. This abstraction layer allows applications to leverage cloud capabilities without directly managing the complexities of cloud interaction.

Figure 1. Bird’s-eye View of Hybrid Storage Architecture on System Z

CDA enables z/OS applications to easily exchange data with applications in cloud and distributed environments, a key feature for legacy modernization. This cross-platform data sharing capability allows older systems to integrate with modern applications and services, supporting hybrid application architectures.

Cloud Providers and Authentication Methods Supported

The following S3 compatible cloud providers are supported by CDA:

The following authentication methods are supported by CDA:

Authentication Mechanism

CDA provides an ISPF panel-oriented application to help with cloud credential storage. It also allows z/OS programs to invoke CDA APIs, thus allowing cloud provider agnostic interaction with the cloud. CDA uses provider JSON files that describe the details on how to interact with a particular cloud object server. Sample provider files are provided by IBM for common providers, which are customizable. The cloud credentials provided in the panel are encrypted using random encryption keys and are securely stored in IBM Cryptographic Service Facility (ICSF) Cryptographic Key Data Set (CKDS).

Major Components of CDA (End-user Perspective)

Figure 2. Major components of CDA

As demonstrated in the above diagram and elaborated upon below, the provider, key, and configuration files represent the principal CDA components relevant to end-users.

1. Provider file

Every cloud provider that is accessed through CDA needs a corresponding provider file. A provider file is a JSON object made up of key names and values. IBM provides sample provider files along with CDA, which can be customized to provide values pertaining to the specific cloud provider. The system expects provider files to reside in one of two locations: the user's ~/gdk/providers/ directory or the default /usr/lpp/dfsms/gdk/providers/ directory.

CDA configuration requires system administrator customization of provider files. These files contain connection parameters, supported operations, request/response parameters, threshold values, and other configuration settings. Provider files also include CDA-specific variables which are known as ‘CDA variables’.

2. Key file

Cloud credentials are securely stored in file named gdkkeyf.json within the user's ~/gdk/ directory. User or System Administrators input these credentials via CDA ISPF panels, which are then encrypted using a random key managed by the IBM ICSF CKDS.

3. Config file

Config file contains some of the configuration details pertaining to CDA/GDK API execution, like “log-level” : “INFO”/”WARNING”/”ERROR”, information on whether web enablement toolkit related logging has to happen or not (“web-toolkit-logging”: true/false) and so on. This file is made available as gdkconfig.json in the ~/gdk/ directory. The options provided in this file can aid in debugging the application.

Pre-requisites

CDA requires z/OS v2.4 or later. Note that z/OS v2.4 and v2.5 support is limited to Amazon S3 cloud storage. Prerequisites include the z/OS Client Web Enablement Toolkit and ICSF. CDA configuration involves several system administrator tasks, including RACF (or equivalent security product) configuration, SSL services setup, provider file customization, and ISPF panel customization. Please refer to the relevant IBM manuals for detailed information.

Pricing/Licensing

Cloud Data Access (CDA) is integrated with z/OS and is not a separately licensed product. It's included as part of the base z/OS offering with DFSMSdfp, starting with z/OS version 3.1.

Processing Thresholds and Performance Benchmarks

Cloud Data Access (CDA) processing thresholds and performance benchmarks are not yet publicly available. Contact IBM support for this information. However, IBM documentation notes that a z/OS Client Web Enablement Toolkit-compliant streaming exit can be used for chunking and sequencing large data transfers. This enables applications to handle very large data volumes within single requests/responses.

z/OS Client Web Enablement Toolkit

CDA leverages the z/OS Client Web Enablement Toolkit's APIs for its underlying functionality. This toolkit allows traditional z/OS programs to interact with modern web services, acting as clients in RESTful web applications. z/OS applications can initiate requests to web servers on z/OS or other platforms. The toolkit provides the following components to simplify client/server interactions for these applications.

Modes of CDA Usage

CDA API Details

The list of CDA APIs available for public use are listed in Appendix-1

Programming Languages Supported by CDA APIs

The following programming languages supported by CDA APIs on Z –

Linkage Conventions and Considerations for CDA APIs

Based on the programming language, interface macro/header file/copybooks as per Appendix-2 must be included in the program as per below table.

The IBM recommended approach for linkage is to use the linkable stub routine GDKCSS from SYS1.CSSLIB while binding the object module.

Please note that there are many environmental considerations that an end-user has to be aware of, like the ones related to OMVS segment for the associated user ID, Impact of z/OS UNIX limit of processes with a POSIX(ON) environment on the number of concurrent connection, z/OS Language Environment Heap runtime option, z/OS Language Environment Runtime Environment REUSE (RTEREUS) option and so on. IBM manuals must be referenced to get detailed information on these.

CDA API Invocation Syntax Format

Please refer to Appendix-3 for sample invocation formats in Assembler, C/C++ and COBOL.

Sample Programs Invoking CDA APIs

CDA provides one or more sample programs in its supported languages. They can be found in SYS1.SAMPLIB with the 3-character ID of GDK. For example, GDKIC1 is a sample C program.

Sample Use Case Ideas Involving CDA APIs

CDA Batch Utility - GDKUTIL

GDKUTIL, a new DFSMS batch utility included with base z/OS (starting with version 2.5), facilitates data sharing between z/OS and the cloud. It requires no additional hardware or software and can be used to upload or download sequential data. Detailed documentation is available in the DFSMS-dfp utilities manual.

Pre-requisites for GDKUTIL

Excerpts from a Sample GDKUTIL Job:

//SAMPUTIL EXEC PGM=GDKUTIL,REGION=0M
//SYSIN DD *
UPLOAD PROVIDER(S3CLOUD)
/*
//OBJNAME DD *
/mybucket/myreport
/*
//LOCNAME DD *
MY.TEST.REPORT
/*
//SYSPRINT DD SYSOUT=*
//SYSOUT DD SYSOUT=*

In the above example, the sequential file named ‘MY.TEST.REPORT’ is getting uploaded to an S3 cloud bucket. The PS file would get placed in a bucket on S3CLOUD named ‘mybucket’, with an object name ‘myreport’.

Potential Use Cases of GDKUTIL Batch Utility

Conclusion

CDA's integration with DFSMSdfp provides several benefits:

IBM Cloud Data Access APIs and GDKUTIL can potentially be used in various modernization solutions as they provide simple means to integrate system Z environment with cloud and distributed environment. Since CDA is a relatively new addition to z/OS base, CDA based PoCs need to be conducted in the Z environments where the pre-requisites are met and the feasibility of integrating this with various modernization solutions must be explored further before arriving at final solution proposals.

References

Throughout the preparation of this whitepaper, information and insights were drawn from a range of reputable sources, including research papers, articles, and resources. Some of the key references that informed the content of this white paper include:

Appendix-1 – List of CDA APIS

Appendix-2 – Programming Interface Files

Appendix-3 – Invocation Syntax Format