A new way to develop MQ programs running in Node
MQ has always supported a wide range of API styles, languages and environments to enable applications to be written in whichever way a developer feels at home. There is the full-function procedural API, often called from C or Cobol programs; object-oriented varieties of that interface such as for Java and C#; and there is the JMS model.
One of my previous projects involved creation of a Go binding for MQ. There are client APIs and protocols with simpler interfaces that can also connect to MQ, such as MQ Light and MQTT. And there are more than just these examples – the new REST messaging API is yet another. One environment where a lot of new applications are developed is Node.js, with JavaScript being the associated programming language.
And so, as part of making it easy for people to access MQ from Node, I’ve now published a JavaScript API implementation on github.
The MQ JavaScript API
The package exposes the IBM MQ programming interface via a wrapper layer implemented in JavaScript. A Node.js developer should be able to easily send and receive messages via MQ, and interact with other MQ-enabled applications in the organisation. It removes the need for a developer to worry about how to map the MQI elements to the underlying C libraries and instead enables a focus on the business application code. For example, JavaScript strings are used instead of the fixed-length, spaced-padded fields that you have to work with in some other languages.
The package is based on the full MQI. It uses essentially the same verbs and structures as the procedural C or COBOL interface, but with a more appropriate style for this environment. Someone using this package should still have a basic understanding of the procedural MQI, as you do need to decide which options and fields may need to be set for each verb.
The verbs follow the JavaScript style, invoking user-supplied callback functions on completion. In all cases, the callbacks are presented with an MQError object as the first parameter when an error or warning occurs (null otherwise), followed by other relevant objects and data. If the application does not specify a callback, then either an exception is thrown, or the verb returns.
Synchrony
Node is designed around an event-driven model, with asynchronous operations and callback functions at its heart. Any interface should support this where appropriate. The majority of MQ verbs do not take much time to run, so a synchronous approach for them ought to be acceptable. The initial connection to a queue manager may take a while, particularly when TLS client channels are involved, but that connection should not happen often in a well-written application. So the MQ verbs in this package are essentially synchronous, even though they invoke callbacks in the JavaScript style for returned errors and data.
The exception is when you need to get messages from a queue. Traditional MQ applications often issue a blocking MQGET with a wait-interval. The program can continue doing other work while that MQGET is waiting by using multiple application threads. But the Node programming model does not allow applications to directly control threads. Everything in your program runs on the same main execution thread.
To deal with that thread-blocking, this MQI implementation includes two mechanisms for retrieving messages from a queue:
- GetSync() is the call that does an MQGET(wait) synchronously. In a Node environment, it blocks the execution thread until it completes. That may be OK for an immediate retrieval where the wait time is set to zero, but it is not recommended for any times where you want to wait a while for a message to arrive.
- Get() is the call that works asynchronously. The callback that you supply to this function is invoked truly asynchronously. To stop the callback being called for further messages, there is a new GetDone() function.
Sample programs amqsget and amqsgeta demonstrate the two different techniques for retrieving message.
Unimplemented operations
The most important MQI verbs and parameters are implemented here, with enough function for many applications. Where there are missing details within the verbs, these are shown by TODO markers in the source files.
But the MQI verbs that I have not wrapped for now include:
MQSET and MQINQMQBEGINAll of the message property controlsMQSUBRQ, MQSTAT
There are also no structure definitions for elements in message contents such as the MQRFH2 or MQDLQ headers.
For this initial version, I have not provided JavaScript versions of the structures needed to programmatically build all the pieces needed to connect as a client – the MQCD and MQSCO control blocks.
You can also make remote client connections with mechanisms such as the CCDT or MQSERVER environment variable. I have included the ability to send a userid and password during the connection.
How to write an application
The package includes several sample programs to put and get messages, and to subscribe to topics. I’ll show some extracts here to give a flavour of the interface.
Connecting
This fragment shows how a JavaScript application connects to a queue manager and opens a queue:
mq = require('ibmmq'); // import package containing MQI library
MQC = mq.MQC; // refer to constant values exported by the package.
mq.Conn("QM1", function(err,hConn) {
if (err) {
console.log(err);
} else {
console.log("MQCONN successful ");
// Define what we want to open, and how we want to open it.
od = new mq.MQOD();
od.ObjectName = "SYSTEM.DEFAULT.LOCAL.QUEUE";
openOptions = MQC.MQOO_OUTPUT;
mq.Open(hConn,od,openOptions,function(err,hObj) {
if (err) {
console.log(err);
} else {
console.log("MQOPEN successful");
}
});
}
});
You can see how the callback functions are invoked at the end of the call instead of getting a directly returned value. You can also see the use of structures such as the MQOD describing an object to be accessed.
Putting a message
Putting a message is simple:
msg = "Hello from Node at " + new Date();
mqmd = new mq.MQMD(); // Defaults are fine.
pmo = new mq.MQPMO();
mq.Put(hObj,mqmd,pmo,msg,function(err) {
if (err) {
console.log(err);
} else {
console.log("MQPUT successful");
}
});
The message data can be a string for simplicity, or a Buffer (an unstructured array of bytes) for when another application may need a specific message layout.
Documentation
There is no separate documentation at the moment. You can see in the various source parts which fields are exposed for each structure like the MQMD, and the sample programs show how to call the API. The regular MQ reference documentation is language-agnostic and explains much more detail about the MQI options and fields.
How it works
This package is a wrapper over the C MQI. It builds on a commonly-used Node package called the Foreign Function Interface (FFI) to call functions in the MQ libraries such as libmqm. My wrapper takes the JavaScript objects managed in your application and maps them onto the C structures.
I did consider writing a Node “Addon” library but that brought more complexity to the implementation. While it might have allowed more of the MQI operations to be pushed to a separate asynchronous thread, that didn’t seem to bring enough benefit to be worth the cost. It’s something that I may look at again for the future, but not immediately.
Get it
You can download the package from GitHub. It includes the wrapper library source code along with some samples to show how to use it.
I have not yet pushed the library to the NPM repository that holds many Node packages. That’s obviously something that ought to be done at some point, but I wanted to get some early feedback on the interface, some more testing, and some stability before putting it out there.
The package was developed on a Linux system, with MQ V9. It only needs an MQ runtime (no header files or SDK), so in theory programs could be run in an environment where only the MQ Redistributable Client is installed but I’ve not tried it in such a stripped-down system yet. That might make it easy to create a Docker image containing only the things needed to run a program. Another idea for the future.
Other JavaScript routes into MQ
There are several other ways in which you can write JavaScript programs that access MQ:
- Take a look at the MQ Light client available from NPM. MQ supports connections from MQ Light clients via AMQP channels.
- The MQTT protocol has an implementation here. MQ supports connections from MQTT clients via the XR service and Telemetry channels.
- MQ V9.0.4 includes a simple REST API for messaging that is accessible from any environment. See here for more information.
These interfaces may be enough for many messaging applications, even though they do not give access to the full services available from MQ such as transactions.
Status and Feedback
The first public release of this API is intended to encourage feedback from potential users. Does it feel “right”. Does it have useful function? What is missing? What is broken?
The code I’ve released is not formally supported; it is not part of the MQ product. But I do intend to followup on any feedback left on the github repository Issues page. And then we may consider a more official delivery of it later.
Conclusion
A central value of MQ has always been to enable development and execution of applications in whatever environment someone wants to work. Wherever your programs run, they should be able to reliably exchange data with other applications. This package is aimed at keeping that value alive and current, making it simple to work in a commonly-used system.
Update – Jan 2018
Since this post was originally written, there have been a number of updates to the MQ Nodejs client wrapper. I’ve updated the text above to strike out things that are now possible. But in summary:
- Just about all the MQI verbs are now available including handling of message properties and access to MQINQ/MQSET.
- The code has been updated for JSDoc descriptions of the verbs and structures, pointing to the Knowledge Center where needed for more information.
- More sample programs demonstrate use of other MQI verbs
- I added support for the MQCD and MQSCO structures for programmatic access to client channels
- The package is also now available from the NPM repository so you do not need to explicitly clone it from github.
This post was last updated on November 23rd, 2019 at 12:07 am