I was working on something recently where I had to upgrade various components in the tooling. And I was getting more and more annoyed that the upgrades broke my existing programs and scripts. None of that was MQ’s fault and I’ll write more about the project once it’s available alongside the newly-announced MQ 9.3. [That article is now published here.] But it got me thinking about the efforts we’ve made to keep MQ application compatibility across its lifetime. I wanted to show how we’ve achieved that across a quarter century (and more). And how that has preserved the work that developers have put into their MQ programs. In particular, I want to see if old compiled programs can still work with a current queue manager.
Source compatibility
Most people probably understand how the MQ API makes it possible to add new function without fundamentally changing the interface. Using versioned structures means that new fields get added to a structure, and the queue manager code can use the version to decide whether or not to look at or fill in those fields. There are some aspects of the original API design which are hard or near-impossible to extend, and which I wish had been done differently. But the basic idea has held up very well.
It means that you can take application source code written many years ago, and recompile it on a new system, knowing it still builds. Not bumping the “default” version for a structure means that you get the same behaviour as before – you only have to move to an MQCNOv6 if you want to use the CcdtUrl attribute. And that requires that you use a particular version of MQ to recognise it. If you use an MQCNOv6 against an old queue manager, the application will fail. So you can continue to use an older structure version without changing the code as the defaults have not changed.
Application problems on migrating to new versions have typically happened when the app has not been written to expect new message formats or receiving newer error codes, or when MQ has tightened up its validation to catch applications that have not completely followed rules..
Binary compatibility
What I was more interested in for this exercise however was binary compatibility. Can I take an old compiled application program and still run it against today’s queue manager.
I visited the Hursley Museum to see if they could help. They could. Their catalogue listed a copy of MQSeries V2.2.1 for AIX. (Actually the first box they dug out from the shelves had an empty CD case, but they found another unopened package that did have the CD.) It’s not quite the oldest we ever did for AIX, which was Version 2.1, but I couldn’t locate that level on a medium that I could still read. Using AIX was ideal for this experiment because it’s a system I have access to and where we still develop MQ. But none of my machines have an 8mm or QIC tape drive attached these days.
The only other still-supported platforms from that time are z/OS and iSeries. All the other early platforms either died away or lost enough support that we don’t do current versions of MQ for them. Other operating systems such as Windows (starting with NT) and Linux came to the queue manager world later.
The box
I put the CD into the system and looked at the dates:
$ cd /cdrom $ ls -al total 46300 -r-xr-xr-x. 1 root root 5683200 Oct 17 1995 dtextbrw.obj -r-xr-xr-x. 1 root root 41728000 Oct 17 1995 mqm.obj -r-xr-xr-x. 1 root root 5438 Oct 17 1995 .toc
I considered doing a “proper” install of the product, but that would mess up too much of my development environment. Being able to isolate MQ installations was a feature from many years later. And I wasn’t bothered about trying to run the queue manager code anyway. Noone should be running that kind of unsupported level. Though I would not be surprised to find there are still unlabelled boxes running a business-critical process hidden in a cupboard somewhere. Something to try another day, if I can find a disposable AIX image to play with.
It was the applications that were more interesting. Those are the things that you – the users – have developed across many years. And for which you may have lost the source by now, but the programs still have value.
The sample programs
So I extracted the sample programs from the installation image and put them in a local directory.
$ ls -l total 672 -rwxrwxrwx 1 metaylor system 10891 13 Nov 1995 amqsbcg -rwxrwxrwx 1 metaylor system 14236 13 Nov 1995 amqscic0 -rwxrwxrwx 1 metaylor system 13654 13 Nov 1995 amqscic21 -rwxrwxrwx 1 metaylor system 135980 13 Nov 1995 amqsdlq -rwxrwxrwx 1 metaylor system 7341 13 Nov 1995 amqsech -rwxrwxrwx 1 metaylor system 7133 13 Nov 1995 amqsgbr -rwxrwxrwx 1 metaylor system 6591 13 Nov 1995 amqsget -rwxrwxrwx 1 metaylor system 6590 13 Nov 1995 amqsgetc -rwxrwxrwx 1 metaylor system 9135 13 Nov 1995 amqsinq -rwxrwxrwx 1 metaylor system 7741 13 Nov 1995 amqsput -rwxrwxrwx 1 metaylor system 7740 13 Nov 1995 amqsputc -rwxrwxrwx 1 metaylor system 9123 13 Nov 1995 amqsreq -rwxrwxrwx 1 metaylor system 8907 13 Nov 1995 amqsset -rwxrwxrwx 1 metaylor system 7785 13 Nov 1995 amqstrg -rwxrwxrwx 1 metaylor system 30653 13 Nov 1995 amqsxa4x -rwxrwxrwx 1 metaylor system 30971 13 Nov 1995 amqsxaex
This machine has MQ 9.3 installed as its primary version, so those will be the shared libraries (libmqm etc) that the sample programs try to load.
Trying the most basic experiment, of putting and getting messages:
$ echo "Hello from 1995" | ./amqsput Q V9300_A Sample AMQSPUT0 start target queue is Q Sample AMQSPUT0 end $ ./amqsget Q V9300_A Sample AMQSGET0 start message <Hello from 1995> no more messages Sample AMQSGET0 end
And doing the same again but using the client connectivity …
$ export MQSERVER="SYSTEM.DEF.SVRCONN/TCP/localhost(3820)" $ echo "Hello from 1995 Client" | ./amqsputc Q V9300_A Sample AMQSPUT0 start target queue is Q Sample AMQSPUT0 end $ ./amqsgetc Q V9300_A Sample AMQSGET0 start message <Hello from 1995 Client> no more messages Sample AMQSGET0 end
The next experiment was to put a message using a current application program which sets the GroupId. Doing that requires an MQMDv2. The v2 structure was not available in the V2.2.1 days, so what would happen?
$ ./amqsbcg Q V9300_A AMQSBCG0 - starts here MQCONN to V9300_A MQOPEN - 'Q' MQGET of message number 1 **** Message descriptor **** StrucId : 'MD ' Version : 1 Report : 0 MsgType : 8 Expiry : -1 Feedback : 0 Encoding : 273 CodedCharSetId : 819 Format : 'MQHMDE ' Priority : 0 Persistence : 0 MsgId : X'414D512056393330305F412020202020625930D040003A01' CorrelId : X'000000000000000000000000000000000000000000000000' BackoutCount : 0 ReplyToQ : ' ' ReplyToQMgr : 'V9300_A ' ** Identity Context UserIdentifier : 'metaylor ' AccountingToken : X'04363530350000000000000000000000000000000000000000000006' ApplIdentityData : ' ' ** Origin Context PutApplType : '6' PutApplName : 'amqsput ' PutDate : '20220422' PutTime : '06225952' ApplOriginData : ' ' **** Message **** length - 85 bytes 00: 4D44 4520 0000 0002 0000 0048 0000 0222 'MDE .......H...' 10: 0000 04B8 4D51 5354 5220 2020 0000 0000 '....MQSTR ....' 20: 2021 2223 2425 2627 2829 2A2B 2C2D 2E2F ' !"#$%&'()*+,-./' 30: 3031 3233 3435 3637 0000 0001 0000 0000 '01234567........' 40: 0000 0018 FFFF FFFF 4865 6C6C 6F20 6672 '........Hello fr' 50: 6F6D 2047 6F 'om Go ' No more messages MQCLOSE MQDISC $ ./amqsget Q V9300_A Sample AMQSGET0 start message <MDE > no more messages Sample AMQSGET0 end
It’s done its best – the MQMDE that is the alternative to the MQMDv2 extension has been read. But it doesn’t know to skip past that to find the real message data.
The V2 client libraries and PROtocol
The final test was to try to use the V2.2.1 client libraries properly, completely bypassing the installed V9.3 shared libraries. While I had been fairly confident the previous tests would work, I was more sceptical about this last one.
I had to set LIBPATH to point into my extracted tree instead of the default directories. I also had to set NLSPATH so that the message catalog containing error messages could be read, when things didn’t work.
$ export MQSERVER='SYSTEM.DEF.SVRCONN/TCP/localhost(3820)' $ export NLSPATH=/mq221/usr/lib/nls/msg/prime/%N $ export LIBPATH=/mq221/usr/lpp/mqm/lib $ echo "Hello again" | ./amqsputc Q V9300_A Sample AMQSPUT0 start target queue is Q Sample AMQSPUT0 end $ ./amqsgetc Q V9300_A Sample AMQSGET0 start message <Hello again> no more messages Sample AMQSGET0 end $
And it works – the client library located and dynamically loaded what it needed to for the communications, and the channel protocol correctly negotiated and connected to the queue manager. I also used system tools such as ldd and truss to verify that the right libraries were indeed being found. The DIS CHS(*) on the queue manager showed – as expected – empty RVERSION and RPRODUCT values, another indication that older libraries were in use.
Summary
It’s not always been possible to maintain that binary compatibility for old applications in all environments, even where we’ve continued to deliver queue manager code. Any breakage has tended to be driven by operating system changes rather than MQ itself. For example, AIX changed some of how to compile threaded programs round about that V2.2 timeframe.
MQ has continued to evolve significantly since that early release. All the new capabilities – functional, environments, APIs – that we talk about elsewhere and which people need for modern applications and deployments are there. If you want to write a NodeJS program in a “serverless” environment that has messages going through MQ, then you can do that. But we bring in those enhancements without ignoring where we’ve been.
I found the basic proof that we can also still run the old stuff to be quite impressive. And I hope you do too.
This post was last updated on June 24th, 2022 at 04:51 pm
One thought on “MQ application compatibility across a quarter century”