Working with IRRDBU00 data¶
The mfpandas.IRRDBU00 is made to work with IRRDBU00 data.
It will give you methods to get:
standard dataframes: strict conversion of IRRDBU00 recordtypes to Pandas DataFrames
augmented dataframes: like standard dataframes but with extra columns added
specialised dataframes: preselected queries we all want to do on our RACF
handy add-ons: extra query features, data structures, xlsx-generation etc.
Below you will see one of the core pieces of code of the parsing stucture. The ‘key’ of the dictionary (0100, 0101, etc.) is the recordtype from the IRRDBU00 unload as described in https://www.ibm.com/docs/en/zos/3.1.0?topic=records-irrdbu00-record-types.
If, for instance, we’re looking at the “Group Basic Data” in recordtype 0100 you’ll see an ‘interal name’ of GPBD and a ‘dataframe name’ of _groups.
This means the resulting Pandas DataFrame is available, after parsing as .groups:
_recordtype_info = {
'0100': {'name':'GPBD', 'df':'_groups'},
'0101': {'name':'GPSGRP', 'df':'_subgroups'},
'0102': {'name':'GPMEM', 'df':'_connects'},
The parsing of records is done via irrdbu00-offsets.json (which is autogenerated from the docs). Therefore the Standard DataFrames look ‘just like the documentation’. For example, the ‘0100’-records are parsed then available via the .groups method of the IRRDBU00 class.
It will have all the fields as documented at https://www.ibm.com/docs/en/zos/3.1.0?topic=records-record-formats-produced-by-database-unload-utility#idg63092__title__1
Column
Description
GPBD_RECORD_TYPE
Record type of the Group Basic Data record (0100).
GPBD_NAME
Group name as taken from the profile name.
GPBD_SUPGRP_ID
Name of the superior group to this group.
GPBD_CREATE_DATE
Date that the group was defined.
Here’s the output of the .groups.info() call on a fully parsed unload:
>>> from mfpandas import IRRDBU00
>>> r = IRRDBU00(irrdbu00='/path/to/unload')
>>> r.parse()
True
>>> r.groups.info()
<class 'pandas.core.frame.DataFrame'>
Index: 12912 entries, $AAAA to Z$FGRP01
Data columns (total 10 columns):
# Column Dtype
--- ------ -----
0 GPBD_RECORD_TYPE object
1 GPBD_NAME object
2 GPBD_SUPGRP_ID object
3 GPBD_CREATE_DATE object
4 GPBD_OWNER_ID object
5 GPBD_UACC object
6 GPBD_NOTERMUACC object
7 GPBD_INSTALL_DATA object
8 GPBD_MODEL object
9 GPBD_UNIVERSAL object
So now, to get all the group names from the unload all you need to do is a simple:
r.groups['GPBD_NAME'].values
And you’ll get a list of all group names.
IRRDBU00 Examples¶
Passphrase report¶
Suppose you need to get a report of all the users on the system that still don’t have a passphrase.
First you create an IRRDBU00-unload via the following JCL:
//UNLOAD EXEC PGM=IRRDBU00,PARM=NOLOCKINPUT
//SYSPRINT DD SYSOUT=*
//INDD1 DD DISP=SHR,DSN=SYS1.BACKUP
//OUTDD DD DSN=HLQ.TO.UNLOAD.FILE,
// DISP=(,CATLG,DELETE),
// SPACE=(CYL,(100,150)),
// DCB=(RECFM=VB,LRECL=4096,BLKSIZE=20480)
After submitting the above JCL, you transfer the HLQ.TO.UNLOAD.FILE (ASCII) to your Linux box (currently mfpandas does not run on z/OS, due to Pandas not working nicely on z/OS yet…)
For more information on the IRRDBU00-utility please refer to the IBM Documenation <https://www.ibm.com/docs/en/zos/3.1.0?topic=database-using-racf-unload-utility-irrdbu00>
Once the file is receieved you create a folder for your work and install the mfpandas library like below:
henri@linux-box:~/$ mkdir my_cool_project
henri@linux-box:~/$ cd my_cool_project
henri@linux-box:~/my_cool_project$ python -m venv virtualenv
(virtualenv) henri@linux-box:~/my_cool_project$ pip install mfpandas
(virtualenv) henri@linux-box:~/my_cool_project$ cp ~/Downloads/HLQ.TO.UNLOAD.FILE irrdbu00
Now, just for a quick and dirty result you enter an interactive python terminal and:
>>> from mfpandas import IRRDBU00
>>> racf = IRRDBU00(irrdbu00='/home/henri/irrdbu00')
>>> racf.parse_fancycli(recordtypes=['0200'])
24-07-01 20:35:15 - parsing /home/henri/irrdbu00
24-07-01 20:35:23 - progress: .......... (26.98%)
After parsing is finished can start interactively coding the solution:
24-07-01 20:35:51 - total parse time: 47.209619 seconds
>>> users_without_phrase = racf.users.loc[racf.users.USBD_PHR_ALG=='NOPHRASE']
>>> for user in users_without_phrase['USBD_NAME'].values:
... print(f'User {user} is still not using a passphrase')
...
User IBMUSER is still not using a passphrase
User TEST001 is still not using a passphrase
User TEST002 is still not using a passphrase
>>>
As you can see above, with some relatively easy to learn ‘Pandas Queries’ (https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.loc.html), using the standard IBM labelnames you can quickly het some results. It’s a small feat to then extend that code with some ‘RACF COMMAND GENERATION’ to give all these users a new ‘one time’ passphrase they must change after first logon with said passphrase:
>>> cmds = []
>>> for user in users_without_phrase['USBD_NAME'].values:
... print(f'User {user} is still not using a passphrase')
... commands.append(f'ALU {user} PRASE('mfpandas_gave_me_a_passphrase'))
...
>>> with open('/givethemprases.txt') as f:
... f.writelines(commands)
After which you can easily stick that on the end of an IKJEFT01 to execute the commands :)
Special users report¶
Assuming you still have the irrdbu00-unload file available:
from mfpandas import IRRDBU00
import time
r = IRRDBU00('/path/to/irrdbu00-unload')
r.parse()
while r.status['status'] != 'Ready':
time.sleep(1)
Let’s find all special users and their last logon date:
>>>r.specials[['USBD_NAME','USBD_LASTJOB_DATE']]
USBD_NAME USBD_LASTJOB_DATE
5491 IBMUSER 1984-12-15
5830 EMERG01 2024-01-05
Let’s see if we have a datasetprofile protecting HSM.SECRET.SAUCE:
>>> r.dataset_profile_for(datasetname='HSM.SECRET.SAUCE')[['DSBD_NAME','DSBD_UACC']]
DSBD_NAME DSBD_UACC
49324 HSM.*.** NONE