Saturday, March 14, 2015

JCL: Empty Dataset Check

Introduction

There are times where you want to execute some steps in JCL, if a given input dataset is NOT empty and execute some other steps if the dataset is EMPTY. For example, we run a given JCL several times a day that appends comma separated records into a given dataset and we need to include the header only during the first time.

How do we do it ?

For example, Lets say a job that runs several time a day and append some records to a dataset called &HLQ..S00.ACCTINFO.D&YYMMDD..CSV where the job has to add header only when the job runs for the first time and subsequent runs just the records alone.

Option 1 : Using ICETOOL

We can use the ICETOOL utility that comes as a part of SORT to check whether a file is empty. It can be done using the COUNT statement. See example below.


  //*******************************************************        
  //*   Check whether the file is empty                           
  //*   HLQ=XYZ, 
  //*   LMMDDYY=150314                           
  //*******************************************************        
  //S0000002 EXEC PGM=ICETOOL                                      
  //IN       DD DSN=&HLQ..S00.ACCTINFO.D&LYYMMDD..CSV,             
  //         DISP=SHR                                              
  //TOOLMSG  DD SYSOUT=*                                           
  //DFSMSG   DD SYSOUT=*                                           
  //TOOLIN   DD DATA                                               
  COUNT FROM(IN) EMPTY                                             
  /*                                                               
Here ICETOOL returns a NON ZERO RETURN CODE, if the file is empty, so the step to include the header can be run using COND parameter or IF statement

Option 2 : Using IDCAMS Utility

We can also use the IDCAMS utility to check whether a file is empty. It can be done using the REPRO command with COUNT(1) parameter. See example below.

There is a subtle difference between using IDCAMS and ICETOOL utilities if the file is not empty. IDCAMS writes the first record into the OUTFILE(OUT) DD name, if the file is NOT empty. Suppose if the file contents are too sensitive that you don't want that dumped into a SYSOUT, the OUT DD can be coded as a temporary dataset with DISP=(NEW,DELETE,DELETE) so that the contents are not visible to anyone after the job has completed.


  //*******************************************************        
  //*   Check whether the file is empty                           
  //*   HLQ=XYZ, 
  //*   LMMDDYY=150314                           
  //*******************************************************        
  //S0000002 EXEC PGM=IDCAMS                               
  //IN       DD DSN=&HLQ..S00.ACCTINFO.D&LYYMMDD..CSV,     
  //         DISP=SHR                                      
  //OUT      DD SYSOUT=*                                   
  //SYSPRINT DD SYSOUT=*                                   
  //SYSIN    DD DATA                                       
    REPRO INFILE(IN) OUTFILE(OUT) COUNT(1)                  
  /*                                                       
Here IDCAMS returns a NON ZERO RETURN CODE, if the file is empty, so the step to include the header can be run using COND parameter or IF statement

Example JCL

Here is a complete example using the ICETOOL to accomplish what was discussed in the "Introduction" section. This JOB contains 4 steps.

  1. Allocate a Dataset if it doesn't exists using IEFBR14.
  2. Empty dataset check using ICETOOL, if empty returns a non-zero Return Code.
  3. Conditionally execute IEFBR14 to include header if previous step's RC is greater than zero.
  4. Step to append records to dataset. I have used IEBGENER, it can be a user written or another built-in program


  //*** YOUR JOB CARD HERE **
  //*
  //    SET HLQ=JUNI                                     
  //    SET LYYMMDD=150314                                 
  //*******************************************************
  //* Allocate Dataset if it doesnt exist                 *
  //*******************************************************
  //S0000001 EXEC PGM=IEFBR14                              
  //DD001    DD DSN=&HLQ..S00.ACCTINFO.D&LYYMMDD..CSV,     
  //         DISP=(MOD,CATLG,DELETE),                      
  //         DCB=(LRECL=80,RECFM=FB),                      
  //         SPACE=(TRK,(1),RLSE)                          
  //*                                                      
  //*******************************************************
  //*   Check whether the file is empty                   *
  //*******************************************************
  //S0000002 EXEC PGM=ICETOOL                              
  //IN       DD DSN=&HLQ..S00.ACCTINFO.D&LYYMMDD..CSV,     
  //         DISP=SHR                                      
  //TOOLMSG  DD SYSOUT=*                                   
  //DFSMSG   DD SYSOUT=*                                   
  //TOOLIN   DD DATA                                       
  COUNT FROM(IN) EMPTY
  /*                           
  //  IF (S0000002.RC > 0) THEN       
  //******************************************************* 
  //* Add Header to Empty File                            * 
  //*******************************************************  
  //S0000003 EXEC PGM=IEBGENER           
  //SYSUT1   DD   *                     
  AcctNbr,LastName,FirstName,MI,DOB,AcctType,Balance,End 
  /*              
  //SYSUT2   DD DSN=&HLQ..S00.ACCTINFO.D&LYYMMDD..CSV,   
  //         DISP=OLD                                     
  //SYSPRINT DD SYSOUT=*                                   
  //SYSIN    DD DUMMY                                    
  //*
  //  ENDIF                                                
  //*******************************************************
  //* Append detailed records                             *
  //*******************************************************
  //S0000004 EXEC PGM=IEBGENER                             
  //SYSUT1   DD   *                                        
  987262601,Doe,John,J,05/22/1997,Savings,12500.00,Z       
  987262602,Smith,John,K,04/14/1968,Current,-100.00,Z      
  /*                                                       
  //SYSUT2   DD DSN=&HLQ..S00.ACCTINFO.D&LYYMMDD..CSV,     
  //         DISP=(MOD,CATLG,DELETE),                      
  //         DCB=(LRECL=80,RECFM=FB),                      
  //         SPACE=(TRK,(1),RLSE)                          
  //SYSPRINT DD SYSOUT=*                                   
  //SYSIN    DD DUMMY                                      
  //*                                                      
  //*******************************************************
  //*               End of JOB                            *
  //*******************************************************

Wednesday, March 4, 2015

ISPF "find" command: Pattern Matching

Introduction

In the previous post called ISPF find command: The Basics written on Mar 3, 2015 , I have written about basic usage find command including search direction, recursive search, case sensitive and insenstive search etc. You may find the blog here at http://mktutes.blogspot.in/2015/03/ispf-find-command-basics.html

In this post, I will discuss about using patterns called picture strings as search criteria instead of literal strings.

Pattern Matching using the Picture string

Perhaps this is the most advanced feature and the coolest one in my opinion. So far we have used literal strings as search criteria and enhanced the search by restricting by rows and columns, prefix,suffix and full words. By using picture strings, we can classify characters broadly into 9 different categories and assign a symbol to each category and use these symbols to find any char that belongs to the character group instead of specific character. For example if we want to find a date of birth that has the following pattern 'dd/mm/ccyy', we can find all the dates using picture string irrespective of the actual numbers. First lets see the 9 categories.

  • = - any character
  • @ - alphabets; both upper and lower case versions
  • # - numeric characters
  • $ - special characters such as #, @ ! etc...
  • ¬ - non-blank characters
  • . - invalid or non-printable characters (dot)
  • - - non-numeric characters (dash)
  • < - lower case alphabets
  • > - upper case alphabets

Examples using picture string

Finding dates that has the following format "mm/dd/ccyy'. The dataset contains two dates 05/25/1980 and 05/25/2014

find p'##-##-####'
will match both dates.

find p'##-##-20##'
will match only the 2nd date alone.

Finding COBOL paragraphs that were commented out (Assume the para names start at pos 8 and they have a nnnn-some-para-name format.)

find p'¬####-@@@@'
find occurrence of a non blank char at pos 7 followed by 4 numbers and four alpha chars at-least

Using Picture String in find has a lot of potential. For example, we can find US phone numbers using pattern p'###-###-####' or Indian phone numbers using p'+91-#####-#####', commented lines in a COBOL program by using find p'¬' 7 .

I am planning to write another post are two on using find and exclude commands together and using change command with picture string in the future.



“I’ve noticed that sometimes when we aren’t actively searching for something, what we seek, finds us.” -- Darryl Webb.

Tuesday, March 3, 2015

ISPF "find" command: The Basics

Introduction

The find command is used to search the next occurrence of a string or pattern from current cursor position. This command is executed as find 'text' and the "find" can be abbreviated as f "text". This command works on various panels in ISPF; member list panel where we can search member name, 3.4 where we can search dataset names and on edit / view / browse datasets. This topic mainly covers the later part.

Command Usage

The basic usage is find 'text' where the cursor will move to the next occurrence of 'text'. This basic usage can be enhanced to do advanced search by additional options. Here are the additional options and examples

Restrict search to specific rows

Suppose you have marked line 10 with label .a and line 30 with label .b and you want to look for a string between these lines. Here is how to do the same.


    find 'text' between lines marked by .a and .b
    find 'text' .a .b   

    find 'text' between line 1 and line with label .a 
    find 'text' .zf .a  

    find 'text' between label .a is and the last line. 
    find 'text' .a .zl

    Note: labels .zf, .zl and .zcsr refers 
       to first, last and current line.
    

Restrict search to specific columns

In order to search for a string between column position 10 50, we can do the following


    find 'text' between column 10 and 50.
    find 'text' 10 50   

    find 'text' between col 10 and last col (2nd operand is omitted).
    find 'text' 10      

    if the 2nd operand larger than LRECL, its is used instead.
    find 'text' 10 1000 
    

Search Directions

We can use next, prev, first, last and all along with find to find the next, previous, last and first occurrence and all occurrences respectively.


    Cursor goes to last occurrence of 'text'.
    find 'text' last    

    Cursor goes to previous occurrence from current cursor position.
    find 'text' prev    
    

Restrict string match by occurrence as prefix, suffix or standalone word. Suppose we have the following words in the dataset: text, context and texting.


    Find all 3 words. 
    find 'text' chars or
    find 'text'                  

    Find only "texting" not the other two. 
    find 'text' prefix 

    Match to "context" alone. 
    find 'text' suffix     

    Match to "text" as a whole word. 
    find 'text' word
    
Note: prefix and suffix can be abbreviated to pre and suf

Recursive find

After first match, next occurrence can be found by using the PF5 key and when the last occurrence is reached, the command wraps and goes to the first occurrence of the search string.

Repeating last search

find *
will use the lastly used search string and find the next occurrence. If * needs to be found literally then wrap * between single or double quotes.

Case in-sensitive search

find t'text'
will find text TEXT teXT etc...

Case sensitive search

find c'text'
will find text alone, not TEXT or Text...


Note:In another post, I will write about using patterns instead of literal texts as search criteria in the find command.


Sunday, March 1, 2015

IEFBR14 - What does it do and why is it named so?

Overview

Whenever I ask people what does IEFBR14 do, the answer I get almost all the times is "Its an utility program that is used to create or delete datasets and then I ask them; Are you sure ?

Well, Though it appears that IEFBR14 create/delete datasets, Its not true exactly. It is actually done by the system itself, based on what is coded in the "DISP=" parameter. However, a DD name cannot be there in a JOB standalone and it should be always be associated with a step "EXEC" so IBM supplied a dummy program that can be coded along with the DD statements whose DISP= will be used by the system to create new dataset or delete existing one.

Why is it called IEFBR14?

The name contains two parts; IEF and BR14. Almost all the IBM utility programs have a 3 byte prefix such as IEB, IEF, IEH.. For example: utility programs start with IEF are for job management, prefix IEB is for dataset utilities (IEBGENER), prefix IEH for system utilities (IEHLIST) and IEW for program and linkage modules (IEWL - Linkage step in compile JCL).  The BR14 stands for assembler instruction BR 14 that means branch to register 14. This assembly instruction by convention means return to caller which is similar to a COBOL program that has STOP-RUN alone in the procedure division.

The next time you run your own COBOL program, add two extra DD statements; one for dataset creation and another one to delete a dataset and make sure the DD names are not used in the program. You will see that the creation and deletion happens after the step is completed just like those DD statements coded with IEFBR14.

For example: In order to delete a dataset called HLQ.A.B.C, using one of these two steps does exactly the same thing.

//STEP0001 EXEC PGM=MYSAMPLE
//DD001      DD      DSN=HLQ.A.B.C,
//                  DISP=(OLD,DELETE)
//*
Note: MYSAMPLE is one of an existing program

//STEP0001 EXEC PGM=MYSAMPLE
//STEP0001 EXEC PGM=IEFBR14
//DD001      DD      DSN=HLQ.A.B.C,
//                  DISP=(OLD,DELETE)
//*

Here is a wikipedia entry about IEFBR14 including how the assembly code looks like.  IEFBR14 @ Wikipedia

Welcome

This blog series is an attempt to share the knowledge I have gained during my 14 years long experience in the IT industry as a developer, tech lead and designer. Inspiration to write came from the training sessions that I have taken for the campus freshers batch on mainframes. These sessions were rigorous and the trainer can accommodate just the basics so I thought, probably a follow-up on commands, procedures, steps that covers slightly advanced topics that may help building knowledge repository using the training sessions as foundation.

Once the idea of sharing knowledge had taken root, I was mulling over about the medium of sharing; email series, blogs, a google site... Finally, I have decided to start with blogging. Based on the response, I might migrate to a google site. Lets see....

In this series, I will post brief write-ups on the following to start with.
  1. Mainframe 
    1. z/OS concepts.
    2. TSO, line and editing commands.
    3. Utility programs like SORT, IEBGENER, etc..
    4. JCL, REXX and SQL for DB2.
  2.  Linux 
    1. Concepts - File system, OS components etc...
    2. Commands and shell scripting.
  3. Regular Expressions
  4. Perl Scripting
  5. MS Excel - Using functions, pivot tables, conditional formatting etc...
  6. SAS Programming.
  7. MongoDB
  8. Interesting tidbits on technology
Well, that's a lot of topics and I hope it will be of use to my trainees and others who may read this by chance.

Appreciate your comments and suggestions on topics on the above categories.

Happy Learning !!
MK