Formatting SMF records

One tool I created a few years ago deals with formatting SMF records from z/OS. I designed it to take MQ’s 115 and 116 records and generate output that users could import to spreadsheets and SQL databases. I made a video about the original version of the tool here, and I wrote about more enhancements to that tool in this post.

If you want to know more about how to use the data after formatting, then Lyn has a lot of posts at her blog site talking about analyses she has run and lessons she has learned.

Most recently I’ve added (with help) some formatting options to deal with a couple of additional record types. That work showed how non-standardised SMF records are, and how much manual work is needed to create formatters. This post will talk about how something that supposedly has a common format does not really.

The SMF “standard”

I once heard SMF records called self-describing. This is clearly not true. Once you get past the common structure that all SMF records start with, then each structure type is unique. And you need to read the macros or header files that describe what a record looks like. As an example, this comes from the MQ C header file csqdsmfc.h to describe one element that appears in some of the records:

typedef struct {
signed   short int qistid; 
signed   short int qistll;
char     qisteyec[4];       
unsigned long  int qistmget;  
unsigned long  int qistmput;  
unsigned long  int qistmblr;  
unsigned long  int qistdcre;  
,,,
unsigned long  int qistgetd;  
unsigned long  int qistgetb;   
unsigned long  int FILLER___001;      
} qist;

Anything taking that definition and wanting to print it out is effectively going to be acting as a language compiler – interpretting the lines and converting them into suitable print statements. That particular structure is simple; others involve arrays of various sizes, sub-structures and bitfields. But even a real compiler would find it hard to do something usable with that structure – the field names alone are not very helpful.

When I started writing mqsmfcsv, I found a statement in SMF documentation that records should all follow a similar pattern, based around “triplets”. The idea is that after a common header structure, all of the structures should be accessible through a set of small control blocks which contain three pieces of information about the real structures – how many there are, how long each one is, and where they start in the record. Though even that might not always be consistent – the example I read back then said that some products may do {offset,length,count} triplets while others may do {offset,count,length}.

The MQ structures

At least MQ stuck to just one of these formats for all of its triplets in both the 115 and 116 records. But you still need to know which triplet refers to which structure. There is no guarantee of being able to programatically decode which structure you are looking at.

As an example, the 115 subtype 1 records may have 11 triplets populated, but only triplets 9 and 11 refer to documented external structures (QSST and QJST if you really want to know). Many of the MQ structures do have eyecatchers that allow you to determine what structure you are looking at, but it’s still not all of them. The structures are not versioned in the way you might expect, and it would be impossible to add a version field now without breaking compatibility rules. There is no way to know this without reading the documentation and cross-referencing it with the header files.

Adding new record types to mqsmfcsv

Andrew sent a Pull Request to the github repository adding support for Advanced Message Security SMF records. Those normally appear as type 180, although for some reason that number can actually be configured in MQ.

And I also had a request to add support for another product that can often be used with MQ – z/OS Connect – which uses type 123 records.

Once I started reading more about these structures I got more annoyed at the variations and lack of consistency. It turns out that AMS does not use triplets at all, but DUPLETS (or DOUBLETS). As there can only be one instance of each structure in the definition of the overall record, the count field was considered redundant. And the zCEE records have two different versions (not distinguishable by subtypes, instead by a different mechanism) which do use triplets, but the two versions use different data sizes – either {32,16,16} or {32,32,32} bits.

As I merged the PR and worked on the zCEE types, I was able to write code that dealt with all these options, but I wish I didn’t have to. Though at least there is now a better structure to the code which should allow more record types to be added faster in future if needed. And if additional variations on the triplet theme are needed, I can see how to slot them in.

Best Practices

As I started to write this, and I looked up a few reference documents, I came across a best practices list for designers of SMF records. I don’t know when that document originally appeared, but I know that MQ designers have broken at least the first three guidelines at various times.

I hope that as older releases go out of service, noone will now be generating SMF data in what are essentially these incompatible formats. Or that they at least have appropriate tools to cope with that level. I think that more recent changes have been a bit more sensitive. And tools that alert me to planned SMF structure updates for future versions of MQ will hopefully give me time to complain in advance of them being publically released.

Conclusion

Decisions made in the 1960s about SMF design continue to be felt today. It’s a shame that freedoms given to individual products were not locked down more strongly. But the basic mechanism is still valuable, helping people make the best of their MVT – er, sorry, z/OS – systems.

Additional thought

One further aspect I was thinking about later – this is a prime example of how you really need to develop real tools in conjunction with data producers. In MQ, the csq4smfd sample program is a trivial dump formatter that is (usually) kept in step with the corresponding SMF structure definitions, but it is not expected to deal with historic formats. So layout changes that might affect compatibility don’t really matter to it and can be ignored.

Once the next version of the product has been released, it becomes near-impossible to fix any structure problems as that could introduce even worse compatibility issues.

So we really need user-level tools like SupportPac MP1B and mqsmfcsv at minimum to be updated early enough to call out problems. And ideally, the commercial products too should validate the new layouts before they are finalised.

This is not just an MQ statement – it ought to apply to all products. And not just for SMF data but any similar external interfaces.

That can be tricky to manage, especially with the shortened cycles that may not allow regular beta releases of code, or where resources are not available to actually test early versions and write the code to exploit new features. But we shouldn’t stop trying to encourage that behaviour.

This post was last updated on October 29th, 2020 at 10:19 am