SORT/DFSORT Utility

What is SORT/DFSORT?

SORT/DFSORT (Data Facility Sort) is a high-performance sorting, merging, and copying tool for z/OS environments. Beyond basic sorting operations, DFSORT provides extensive data manipulation capabilities, allowing you to filter, reformat, summarize, and analyze data during processing.

DFSORT is primarily used for:

• Sorting data in ascending or descending order
• Merging multiple pre-sorted datasets
• Copying data with optional transformations
• Filtering records based on complex conditions
• Reformatting data during processing
• Summarizing and reporting on data

JCL Requirements

To use DFSORT, you need to specify the following in your JCL:

• EXEC statement specifying PGM=SORT or PGM=ICEMAN
• SORTIN DD for the input dataset
• SORTOUT DD for the output dataset
• SYSIN DD containing the DFSORT control statements
• SYSOUT DD for messages
• Optional SORTWKnn DD statements for work files

Basic DFSORT JCL template:

1
2
3
4
5
6
7
8
//SORTSTEP EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DSN=INPUT.DATASET,DISP=SHR
//SORTOUT  DD DSN=OUTPUT.DATASET,DISP=(NEW,CATLG,DELETE),
//            SPACE=(CYL,(5,2)),DCB=(RECFM=FB,LRECL=80,BLKSIZE=27920)
//SYSIN    DD *
  SORT FIELDS=(1,10,CH,A)
/*

SORT Statement Syntax

The SORT statement defines the sorting operation:

1
  SORT FIELDS=(p1,m1,f1,s1,p2,m2,f2,s2,...)

Where:

• p - Starting position of the field
• m - Length of the field
• f - Format of the field (CH for character, BI for binary, PD for packed decimal, etc.)
• s - Sort direction (A for ascending, D for descending)

Examples of SORT Statements

Sort by character field in positions 1-10 ascending:

1
  SORT FIELDS=(1,10,CH,A)

Sort by multiple fields (positions 20-25 descending, then 1-5 ascending):

1
  SORT FIELDS=(20,6,CH,D,1,5,CH,A)

Sort by packed decimal field in positions 10-12:

1
  SORT FIELDS=(10,3,PD,A)

Filtering with INCLUDE/EXCLUDE

The INCLUDE and EXCLUDE statements allow you to filter records based on conditions:

1
2
  INCLUDE COND=(p1,m1,f1,operator,constant)
  EXCLUDE COND=(p1,m1,f1,operator,constant)

Where operators include:

• EQ - Equal to
• NE - Not equal to
• GT - Greater than
• GE - Greater than or equal to
• LT - Less than
• LE - Less than or equal to

Examples of Filtering

Include only records with 'DEPT10' in positions 1-6:

1
  INCLUDE COND=(1,6,CH,EQ,C'DEPT10')

Include records with numeric value greater than 100 in packed decimal format:

1
  INCLUDE COND=(25,4,PD,GT,+100)

Complex condition with AND/OR logic:

1
2
3
  INCLUDE COND=(1,3,CH,EQ,C'ABC',AND,
                        10,5,CH,EQ,C'12345',OR,
                        20,2,BI,GT,X'0064')

Reformatting with INREC and OUTREC

INREC reformats records before sorting, while OUTREC reformats after sorting. Both statements allow you to rearrange fields, remove unwanted data, insert constants, and perform calculations.

Basic Field Reformatting

Copy specific fields to create a new record layout:

1
2
3
4
  OUTREC FIELDS=(1,10,        /* Copy positions 1-10 */
                              15,5,        /* Copy positions 15-19 */
                              C'CONSTANT', /* Insert literal */
                              35,10)       /* Copy positions 35-44 */

Field Manipulation

Perform calculations and format numeric fields:

1
2
3
4
  OUTREC FIELDS=(1,20,                   /* Copy name field */
                              25,4,ZD,TO=PD,LENGTH=3,  /* Convert ZD to PD */
                              45,6,PD,TO=ZD,LENGTH=10, /* Convert PD to ZD with leading zeros */
                              52,5,ZD,TO=FS,LENGTH=7)  /* Format as signed numeric */

Date Formatting

Convert dates between formats:

1
2
3
  INREC FIELDS=(1,20,
                             21,8,Y4T,TOJUL=Y4T,   /* Convert YYYYMMDD to YYYYDDD */
                             45,7,Y4T,TOGREG=MDY)  /* Convert YYYYDDD to MM/DD/YYYY */

Summarization with SUM

The SUM statement allows you to combine records with matching keys by adding specified numeric fields. This is particularly useful for creating subtotals and totals.

1
2
3
4
  SORT FIELDS=(1,10,CH,A)    /* Sort by key field */
  SUM FIELDS=(25,4,ZD,        /* Add field in positions 25-28 */
              35,6,PD,        /* Add field in positions 35-40 */
              NONE)           /* No additional control fields */

When using SUM:

• Records with identical sort keys will be combined into a single record
• Fields specified in SUM will be added together
• Non-summarized fields will contain values from the first record of each group
• Use NONE when all control fields are included in the SORT statement

DFSORT ICETOOL

ICETOOL is an enhanced DFSORT utility that provides additional capabilities through a set of operators. It simplifies many common tasks and provides enhanced reporting features.

Key ICETOOL operators:

• COPY - Copy a dataset
• COUNT - Count records
• DISPLAY - Create formatted reports
• OCCUR - Show unique values and their occurrences
• SELECT - Select records based on occurrence counts
• SORT - Sort records
• STATS - Provide statistical information
• UNIQUE - Create a dataset with unique records

ICETOOL Example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
//TOOLSTEP EXEC PGM=ICETOOL
//TOOLMSG  DD SYSOUT=*
//DFSMSG   DD SYSOUT=*
//SORTIN   DD DSN=INPUT.DATASET,DISP=SHR
//SORTOUT  DD DSN=OUTPUT.DATASET,DISP=(NEW,CATLG),
//            SPACE=(CYL,(5,2)),DCB=(RECFM=FB,LRECL=80)
//REPORT   DD SYSOUT=*
//TOOLIN   DD *
    DISPLAY FROM(SORTIN) LIST(REPORT) -
      TITLE('Employee Department Report') -
      HEADER('EMPLOYEE') HEADER('DEPARTMENT') HEADER('SALARY') -
      FIELDS=(5,20,CH,1,4,CH,25,10,ZD,E'$$$,$$$,$$9.99')
    SORT FROM(SORTIN) TO(SORTOUT) USING(CTL1)
//CTL1CNTL DD *
    SORT FIELDS=(1,4,CH,A)
    INCLUDE COND=(25,10,ZD,GT,+50000)
/*

Practical Examples

Example 1: Basic Sorting and Filtering

1
2
3
4
5
6
7
8
9
//SORT01   EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DSN=EMPLOYEE.DATA,DISP=SHR
//SORTOUT  DD DSN=EMPLOYEE.SORTED,DISP=(NEW,CATLG),
//            SPACE=(CYL,(1,1)),DCB=(RECFM=FB,LRECL=80)
//SYSIN    DD *
    SORT FIELDS=(10,20,CH,A,1,9,CH,A)
    INCLUDE COND=(5,2,CH,EQ,C'10',OR,5,2,CH,EQ,C'20')
/*

This example sorts employee records by name (positions 10-29) and then by employee ID (positions 1-9). It includes only employees from departments 10 and 20.

Example 2: Report Generation with Reformatting

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//SORT02   EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DSN=SALES.DATA,DISP=SHR
//SORTOUT  DD SYSOUT=*
//SYSIN    DD *
    SORT FIELDS=(1,5,CH,A,10,8,PD,D)
    INCLUDE COND=(30,6,PD,GT,+10000)
    OUTREC FIELDS=(1,5,             /* Region Code */
                  X'40',            /* Space */
                  10,8,PD,TO=ZD,    /* Sales Amount as ZD */
                  X'40',            /* Space */
                  30,6,PD,TO=ZD,    /* Quantity as ZD */
                  X'40',            /* Space */
                  C'Report Generated on ',
                  DATE=(DMY,4-),    /* Current Date DD/MM/YYYY */
                  X'40',            /* Space */
                  TIME)             /* Current Time HH:MM:SS */
/*

This example creates a formatted report of sales data. It sorts by region and descending sales amount, includes only records with quantities greater than 10,000, and reformats the output with descriptive text and current date/time.

Example 3: Summarization and Totals

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//SORT03   EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DSN=MONTHLY.SALES,DISP=SHR
//SORTOUT  DD DSN=SALES.SUMMARY,DISP=(NEW,CATLG),
//            SPACE=(CYL,(1,1)),DCB=(RECFM=FB,LRECL=50)
//SYSIN    DD *
    SORT FIELDS=(1,5,CH,A,6,4,CH,A)  /* Region, Department */
    SUM FIELDS=(10,4,ZD,             /* Quantity */
                15,8,PD,             /* Amount */
                NONE)
    OUTREC FIELDS=(1,5,              /* Region */
                  X'40',             /* Space */
                  6,4,               /* Department */
                  X'40',             /* Space */
                  10,4,ZD,TO=ZD,     /* Sum of Quantity */
                  X'40',             /* Space */
                  15,8,PD,TO=ZD)     /* Sum of Amount */
/*

This example summarizes sales data by region and department. It sums up the quantity and amount fields, and creates a formatted summary record for each unique region/department combination.

Exercises

Best Practices

• Always specify SORTWKnn DD statements for large sorts to improve performance
• Use NOBLKSET only when absolutely necessary, as it can severely impact performance
• Process data before sorting with INREC to reduce the amount of data being sorted
• Use INCLUDE/EXCLUDE early in the process to filter out unwanted records
• Avoid unnecessary data conversions which can impact performance
• When possible, use ICETOOL for complex reporting and analysis tasks
• Consider using JOINKEYS for matching records between files instead of manual preprocessing
• Use OPTION statements to customize DFSORT behavior for specific requirements
• Always verify successful completion by checking job return codes

Performance Considerations

• Sort performance is affected by the amount of data, key length, and available resources
• SORTSIZE can be used to control the amount of main storage used for sorting
• Filtering data with INCLUDE/EXCLUDE before sorting can significantly improve performance
• Using efficient data formats (binary vs. character) for sort keys can speed up processing
• For large sorts, ensure adequate SORTWK space is available
• Consider enabling DFSORT installation options for optimal performance
• Use DYNSORT=YES in environments with dynamic allocation capabilities