Mainframe Connector uses an ANTLR4-based parser that supports standard COBOL copybooks.
This document lists the preprocessing tasks performed by the copybook parser,
the supported data types and usage restrictions, and how to configure field
modifiers using the transcoder configuration with the qsam decode and qsam
encode commands.
Preprocessing
Before parsing a copybook, the copybook parser preprocesses the data and performs the following steps:
- Removes comment lines (
*in column 7, or inline*>and*>CEcomments). - Resolves line continuation (
-in column 7 followed by a quoted literal). - Blanks out sequence number areas (columns 1–6) and column 73 and beyond. Code must start at column 8 or later.
- Preserves preprocessor-specific statements like
EJECT,SKIP1,SKIP2,SKIP3, andTITLE. These statements are parsed, but ignored. Copybooks containing preprocessor parameters that can be used byCOPY REPLACINGare not supported by the copybook parser. In these copybooks, identifiers are surrounded by colons (:).
Supported data types and restrictions
The following are the data types supported by the copybook parser and the restrictions for their use:
- Only a single top-level
01record is allowed in a copybook, and the01record cannot have anOCCURSclause. - Level
66(RENAMES/ alias) and level77(standalone) fields are not supported. Level88(condition names) is parsed and ignored. - All elementary (non-group) fields must have a
PICTUREclause, except for hexadecimal floating-point fields (COMP-1andCOMP-2). The followingPICTUREfields are supported:- Alphabetic (
PIC A), alphanumeric (PIC X), and alphanumeric-edited (A,X,9,B,0,/). PIC G(DBCS; requiresUSAGE DISPLAY-1).PIC N(national or DBCS) and national-edited (N,B,0,/).PIC U(UTF-8), includingPIC U BYTE-LENGTH n.- Numeric (
9,S,V) and numeric-edited ($,B,/,V,Z,0,9,,,.,*,+,-,CR,DB). Note thatPIC P(scaling position) is not supported. - Zoned decimal and packed decimal fields with precision up to 18 are
decoded into 64-bit types (
LONGorDECIMAL64), and higher-precision fields are decoded into arbitrary-precision types (BIG_INTEGERorBIG_DECIMAL). Whenschema_validation_modeis set toBIG_QUERY, the maximum supported precision is 76 (BIGNUMERIC) and the maximum scale is 38 (forPOSTGRES, max precision is 147,455 and max scale is 16,383; forMYSQL, max precision is 65 and max scale is 30).
- Alphabetic (
- IBM Hexadecimal Floating-Point (HFP) is supported using
COMP-1/COMPUTATIONAL-1(single precision, 4 bytes) andCOMP-2/COMPUTATIONAL-2(double precision, 8 bytes), or by applying thehfpfield modifier in the transcoder configuration. REDEFINESclauses are supported on statically sized fields when all redefined variants have the same byte size. Redefined fields cannot be referenced as the length field of anOCCURS DEPENDING ONclause or as thetarget_fieldof anull_ifmodifier.OCCURSclauses are supported for fixed-length arrays (OCCURS n TIMES), dynamic-length arrays (OCCURS min TO max TIMES DEPENDING ON field), and packed dynamic-length arrays (OCCURS UNBOUNDED DEPENDING ON field). TheASCENDING/DESCENDING KEYandINDEXED BYphrases in anOCCURSclause are not supported, andDEPENDING ONmust reference a field name in scope.- Only the following
USAGEclauses are supported:DISPLAY(default for alphanumeric and zoned decimal fields)DISPLAY-1(DBCS)NATIONAL(default forPIC Nfields; encoded asUTF-16BE)UTF-8(default forPIC Ufields)BINARY,COMP,COMPUTATIONAL,COMP-4,COMPUTATIONAL-4,COMP-5,COMPUTATIONAL-5COMP-3,COMPUTATIONAL-3,PACKED-DECIMALCOMP-1,COMPUTATIONAL-1,COMP-2,COMPUTATIONAL-2
DATEandTIMESTAMPare supported using the transcoder configuration.- Null indicators are supported using the transcoder configuration.
- Double-byte character set (DBCS) fields
PIC GandPIC Nare supported. By default,PIC Nfields without aUSAGEclause are assignedUSAGE NATIONAL(UTF-16BE). To use aPIC Nfield as DBCS, either specifyUSAGE DISPLAY-1in the copybook or override its character encoding in the transcoder configuration. - Variable-length character strings are supported using the transcoder configuration.
- The
SIGNclause (LEADING,TRAILING,SEPARATE CHARACTER) is supported. - Comments are supported.
Support for date and timestamp fields
Mainframe Connector supports decoding and encoding date and timestamp
fields using the
format_date
and
format_timestamp
field modifiers in the
transcoder configuration
(--transcode-configuration). You can apply these modifiers across matching
fields using field_suffixes or to specific fields using field_overrides:
{
"field_suffixes": [
{
"suffix": "SUFFIX_STRING",
"modifier": {
"format_date": {
"formats": [
{
"size": FIELD_SIZE,
"pattern": "FORMAT_PATTERN"
}
]
}
}
}
]
}
The following list describes the configuration properties in more detail:
suffix: Matches any copybook field whose name ends with-SUFFIX_STRINGor_SUFFIX_STRING(case-insensitive). Ensure thatsuffixdoesn't contain a leading hyphen or underscore. Alternatively, usefield_overrideswith"field": "FIELD_NAME"to target an individual field.format_date/format_timestamp: Defines the modifier used to parse or format the field:format_datecan be applied to sized string fields (PIC X) or integer numeric fields (such asPIC 9(n),COMP, orCOMP-3with scale0and precision up to 18, which are dynamically converted to their unsigned string representation before formatting).format_timestampcan be applied to sized string fields (PIC X). Timestamps are interpreted in UTC unless a timezone or offset is included in the pattern.
formats: A list ofDateTimeFormatobjects, each specifying:size: The size of the field that the pattern applies to (byte length for string fields, or precision for numeric fields).pattern: A date or timestamp pattern. During decoding (qsam decode), formats matching the field's size are evaluated in order until one matches. During encoding (qsam encode), only the first matching format for the field's size is used. For more information on valid formats, see Supported date and timestamp formats.
- Removing suffixes (Optional): To remove
-SUFFIX_STRINGor_SUFFIX_STRINGfrom the output field name, add arenametransformation to thetransformationsarray in your transcoder configuration.
Examples
To format 8-byte fields ending with
-DT8or_DT8asDATEwith patternyyyyMMdd, and 10-byte fields ending with-DTor_DTwith eitherMM-dd-yyyyorMM/dd/yyyy:{ "field_suffixes": [ { "suffix": "DT8", "modifier": { "format_date": { "formats": [ { "size": 8, "pattern": "yyyyMMdd" } ] } } }, { "suffix": "DT", "modifier": { "format_date": { "formats": [ { "size": 10, "pattern": "MM-dd-yyyy" }, { "size": 10, "pattern": "MM/dd/yyyy" } ] } } } ] }To format 26-byte fields ending with
-TIMESTAMP,_TIMESTAMP,-TS, or_TSasTIMESTAMPwith patternyyyy-MM-dd HH.mm.ss.SSSSSSand strip the suffix from the output column names:{ "field_suffixes": [ { "suffix": "TIMESTAMP", "modifier": { "format_timestamp": { "formats": [ { "size": 26, "pattern": "yyyy-MM-dd HH.mm.ss.SSSSSS" } ] } } }, { "suffix": "TS", "modifier": { "format_timestamp": { "formats": [ { "size": 26, "pattern": "yyyy-MM-dd HH.mm.ss.SSSSSS" } ] } } } ], "transformations": [ { "rename": { "find": "^(.*)[-_](TIMESTAMP|TS)$", "replace": "$1" } } ] }
Support for null indicators
Mainframe Connector supports null indicators through the
null_if
field modifier in the
transcoder configuration
(--transcode-configuration).
To configure a suffix-based null indicator (where a field such as COL1-NID
acts as the null indicator for COL1 in the same record), define an entry in
field_suffixes with "is_inverse": true and "target_field": "$self":
{
"field_suffixes": [
{
"suffix": "NULL_INDICATOR_NAME",
"is_inverse": true,
"modifier": {
"null_if": {
"target_field": "$self",
"null_value": "NULL_VALUE",
"non_null_value": "NOT_NULL_VALUE"
}
}
}
]
}
The following list describes the parameters you can use to configure null indicators:
suffix: Corresponds to the suffixes-NULL_INDICATORNAMEorNULL_INDICATOR_NAMEthat identify a null-indicator field in a copybook.is_inverse: When set totrueon aFieldSuffix, the modifier is applied to the sibling field whose name matches the indicator field without the suffix (for example,COL1when the indicator isCOL1-NID1), and"$self"refers to the indicator field itself. You can also applynull_ifdirectly on a specific field usingfield_overridesand settarget_fieldto the name of the indicator field.null_valueornull_values: Specifies the single value (null_value) or list of values (null_values) intarget_fieldthat signal that the referenced field isnull.non_null_valueornon_null_values: (Optional) Specifies the single value (non_null_value) or list of values (non_null_values) intarget_fieldto signal that the referenced field is notnull. If omitted, any value other thannull_valueornull_valuesis treated as not null. At least one ofnull_value(s)ornon_null_value(s)must be specified.- Excluding or keeping the null-indicator field: By default, the
null-indicator field is kept as a column in the output schema. To omit the
null-indicator field from the output table while still using it during
transcoding, add an
excludetransformation for the indicator field in thetransformationsarray (do not apply thefillermodifier to a null-indicator field, asfillerskips decoding the field altogether). - Overriding the indicator field type: (Optional) To force a
null-indicator field to be decoded as
bytesorbinary, add a non-inversefield_suffixes(orfield_overrides) rule for the indicator field with"modifier": { "bytes": {} }or"modifier": { "binary": {} }. Forbytes, express thenull_valueandnon_null_valueas hexadecimal strings (for example,"FF"or"00").
Examples
In the following examples, COL1-NID1 and FIELD-NID2 are excluded from the
output table after being evaluated as null indicators, while COL2-NULL is
retained in the output schema.
Copybook snippet:
01 REC.
10 COL1-NID1 PIC S9(4) USAGE COMP.
10 COL1 PIC S9(6) USAGE COMP.
10 FIELD PIC X(10).
10 FIELD-NID2 PIC X(1).
10 COL2 PIC X(10).
10 COL2-NULL PIC X(1).
Transcoder configuration (transcode-configuration.json):
{
"field_suffixes": [
{
"suffix": "NID1",
"is_inverse": true,
"modifier": {
"null_if": {
"target_field": "$self",
"null_value": "-1",
"non_null_value": "0"
}
}
},
{
"suffix": "NID2",
"is_inverse": true,
"modifier": {
"null_if": {
"target_field": "$self",
"null_value": "?"
}
}
},
{
"suffix": "NULL",
"is_inverse": true,
"modifier": {
"null_if": {
"target_field": "$self",
"null_value": "?"
}
}
}
],
"transformations": [
{
"exclude": {
"field": "COL1-NID1"
}
},
{
"exclude": {
"field": "FIELD-NID2"
}
}
]
}
Support for DBCS fields
When using DBCS fields (PIC G or PIC N with USAGE DISPLAY-1), ensure the
following:
PIC Gfields must explicitly specifyUSAGE DISPLAY-1.PIC Nfields without aUSAGEclause default toUSAGE NATIONAL(UTF-16BE). To treat aPIC Nfield as a DBCS field, specifyUSAGE DISPLAY-1in the copybook (or override its encoding in the transcoder configuration).- By default,
USAGE DISPLAY-1fields are decoded and encoded using thecp300(x-IBM300) DBCS character set, which expects raw DBCS bytes without shift-out (0x0E) and shift-in (0x0F) control characters. - To use a different DBCS character set (such as
Cp834) or a multi-byte character set (MBCS) that uses shift-out (0x0E) and shift-in (0x0F) bytes (such asx-IBM930,x-IBM933,x-IBM935,x-IBM937,x-IBM939,x-IBM942,x-IBM942C,x-IBM943,x-IBM943C,x-IBM949,x-IBM949C,x-IBM950,x-IBM964,x-IBM970, orx-IBM1364), configure thestringfield modifier in the transcoder configuration (--transcode-configuration).
You can set the DBCS encoding globally for all USAGE DISPLAY-1 fields using
defaults.dbcs,
or for specific fields using field_overrides or field_suffixes:
{
"defaults": {
"dbcs": {
"string": {
"encoding": "x-IBM930"
}
}
}
}
Support for variable-length character strings
The copybook parser supports variable-length character string group fields using
the varlen field
modifier in the
transcoder configuration:
10 VAR-GRP.
15 VAR-LEN PIC 9(4) USAGE COMP.
15 VAR-TEXT PIC X(50).
When using the varlen field modifier, ensure the following:
Apply the
varlenmodifier to the group item (for example,VAR-GRP) usingfield_overridesorfield_suffixesin the transcoder configuration:{ "field_overrides": [ { "field": "VAR-GRP", "modifier": { "varlen": {} } } ] }The group item must contain exactly two subfields:
- The first subfield must decode to a 64-bit integer (
LONG, such as a binaryCOMPor zoned decimal integer field) that specifies the byte length of the string data. - The second subfield must be an unmodified string field (such as
PIC X(n),PIC N(n), orPIC U(n)).
- The first subfield must decode to a 64-bit integer (
No specific subfield suffixes (such as
-LENor-TEXT) are required, and the resulting column in the output schema uses the name of the group item (VAR-GRP).
Unsupported fields and constructs
The following sections describe the fields and constructs that are not supported by the copybook parser.
Unsupported COBOL constructs
The copybook parser can parse the following COBOL constructs, even though these constructs are not supported. If you use these constructs in your copybook, Mainframe Connector shows an error:
ALIGNEDBLANK WHEN ZEROCOMMON,OWN,LOCALDYNAMIC LENGTH(such asPIC X DYNAMIC LENGTHorPIC U DYNAMIC LENGTH LIMIT n)INTEGER,STRINGJUSTIFIEDASCENDING/DESCENDING KEYwithinOCCURSINDEXED BYwithinOCCURSRECEIVED BYRECORD AREARENAMESSYNCHRONIZED/SYNCTHREAD-LOCALTYPE IS ...TYPEDEFUSING LANGUAGE/CONVENTION
Unsupported data types
The following USAGE types can be parsed by the grammar, but are not supported
by the copybook compiler:
BITCONTROL-POINTDATEDOUBLEEVENTFUNCTION-POINTERINDEXKANJILOCKPOINTERPROCEDURE-POINTERREALSQLTASK