Python MQI Interface - pymqi. Version 0.5d

Pymqi is a Python extension for IBM's Messaging & Queueing middleware, MQSeries (aka IBM WebSphere MQ family). This allows Python scripts to make calls directly to MQI to connect queues and get/put messages on them etc.

Pymqi combines the power of Python with the benefits of the messaging model. It can be used to develop test harnesses for MQ based systems, for rapid prototyping of MQ applications, for development of administrative GUI's (when combined with e.g., TkInter), and even for mainstream MQ application development!

Pymqi does not replace MQI, but is layered on top of it, so you must have MQ (either client or server) installed before you can use pymqi.

Pymqi consists of several modules that are used together:

Its easiest to use the pymqi package. Here is some minimal code to put a message on a queue:

 
    import pymqi

    qmgr = pymqi.QueueManager()
    putQ = pymqi.Queue(qmgr, 'TESTQ1')
    putQ.put('Hello from Python!')
And here's some more to get it again:
 
    import pymqi

    qmgr = pymqi.QueueManager()
    getQ = pymqi.Queue(qmgr, 'TESTQ1')
    print 'Heres the message:', getQ.get()

Easy, Eh?

For the MQI calls that pymqi supports, the full set of options are implemented. This allows you access to the full functionality of the MQI call from Python. Pymqi defines classes that are synonymous with the MQI structure parameters, such as MQGMO etc. This lets you program MQI 'the Python way'.

Download, Build & Install

Download the latest version from here. The download package is a source distribution with a distutils setup.py file. Download, unzip & untar the file, then cd into the pymqi directory. To build a MQI client, run python setup.py build client, to build a MQI server, python setup.py build server. Finally, run python setup.py install

If you are building pymqi on Windows and you don't have MSVC installed, you can use MinGW instead. Jaco Smuts has written notes here explaining how its done.

If you port pymqi to other platforms, please let me know.

Supported Functionality

The following MQI calls are supported.

In support of these, pymqi implements the following structures. Others will follow to support additional MQI calls as required.

For a client build, pymqi.__mqbuild__ is set to the string 'client', otherwise it is set to 'server'. The supported command levels (from 5.0 onwards) for the version of MQI linked with pymqi are available in the tuple pymqi.__mqlevels__. To determine if a particular level is supported, do something like:

 
    if '5.3' in pymqi.__mqlevels__:
        print 'New MQI things to try!'

Platforms

This version of pymqi has been built and tested on Linux Redhat FC4, MQ 5.2/5.3/6.0 & Python 2.2. Earlier versions have been known to work on Windows/NT, AIX & Solaris. It ought to build & work on any 32-bit platform that supports MQI & Python. 64-bit support is on the TODO list. If you port pymqi to other platforms and would like to contribute your changes, please mail lsmithson@open-networks.co.uk.

PCF Interface

Pymqi supports Programmable Command Format (PCF) if built with MQ version 5.3. This allows you to easily administer and configure MQ. The full set of PCF commands is available.

To use PCF with pymqi, instantiate a PFCExecute object as you would a QueueManager. Then call a PCF command, passing a dictionary of attributes and values, as appropriate to the command. Commands and attributes are as defined by IBM in cmqc.h, cmqcfc.h, and in their documentation.

You can also use PCF to query a Queue Manager. In this case, PCFExecute returns a list of dictionaries, with the attributes and values you have requested.

An example PCFExecute usage is given below.

 
    import pymqi, CMQC
    from CMQCFC import *
        
    # Connect to default Queue Manager
    pcf = pymqi.PCFExecute()

    # Ping the Queue Manager
    pcf.MQCMD_PING_Q_MGR()
        
    # Create a channel "FNURGLE"
    chanArgs = {MQCACH_CHANNEL_NAME : "FNURGLE",
                MQIACH_CHANNEL_TYPE : CMQC.MQCHT_RECEIVER}
    pcf.MQCMD_CREATE_CHANNEL(chanArgs)
        
    # Query all queues beginning with "QTEST"
    queues = pcf.MQCMD_INQUIRE_Q({CMQC.MQCA_Q_NAME : "QTEST*"})
    for q in queues:
        print pcf.stringifyKeys(q)

MQBEGIN, Distributed Transactions, XA and all that

The use of QueueManager.begin() allows a pymqi application to coordinate distributed transactions. This means that updates to queues & databases (or anything else supported by the XA interface) are linked. If a queue get/put fails, any database updates within the transaction are backed out. Similarly, if a database update fails, any queue get/put operations within the transaction are undone.

This depends on the MQ Transaction Manager, which in turn coordinates resources through the XA interface. To make this work from Python, you must use an XA enabled database API. The only one I'm aware of that works is DCOracle2. You will need to apply these (unofficial) patches.

You will also have to configure the Queue Manager with XA information for the resources you want it to coordinate. See the MQSeries System Administration book for more details.

You might also be interested in a Python extension for the XA Switch interface. This lets Python scripts participate directly in the Distributed Transaction two phase commit protocol. See here for more details. Pyxasw & pymqi are related but independent of each other.

This code fragment shows a transactional MQ put & DCO2 database insert, coordinated by MQ.

 
    import pymqi, DCOracle2

    ...

    qmgr = pymqi.QueueManager()
    q = pymqi.Queue(qmgr, 'TESTQ1')
    pmo = pymqi.pmo(Options = CMQC.MQPMO_SYNCPOINT)
    md = pymqi.md()

    # Begin a global transaction
    qm.begin()
    # Connect a XA managed database
    conn=DCOracle2.connectXA()
    curs = conn.cursor()
    # Do a transactional put & db insert
    q.put('TM comes to Linux!', md, pmo)
    curs.execute("INSERT INTO TESTTABLE VALUES(42, 'Lala')")
    # Now commit the transaction
    qmgr.commit()

Inquire & Set

Pymqi supports a simple interface to the MQI MQINQ & MQSET calls.

The QueueManager.inquire() and Queue.inquire() calls allow you to inquire on a single MQ attribute. An integer or string attribute value is returned, as appropriate.

The Queue.set() call lets you set a single Queue attribute. The value passed must be of the appropriate type.

When inquiring or setting on a queue, the queue must be specifically opened for inquire or set. The operations cannot be mixed on a queue, neither can a get/put queue be used for inquire or set. No such restriction applies to QueueManager objects.

An example inquire is shown below.

 
    import pymqi, CMQC

    ...

    qmgr = pymqi.QueueManager()
    q = pymqi.Queue(qmgr, 'TESTQ1')
    print 'Queue depth:', q.inquire(CMQC.MQIA_CURRENT_Q_DEPTH)
    print 'Queue Manager platform:', qmgr.inquire(MQIA_PLATFORM) 

A FAQ

1. Why can't I connect pymqi to the Queue Manager?

If you see this error:
 
pymqi.MQMIError: MQI Error. Comp: 2, Reason 2058: FAILED: MQRC_Q_MGR_NAME_ERROR
And you're certain the Queue Manager is running, then you're probably running in an MQ Client environment without telling pymqi where the server is. If you don't know the difference between an MQ Client and Server, please read the IBM documentation.

The answer is to either:

If it still doesn't work, then you've probably built pymqi against the MQ server library rather than the client.

Caveats/Bugs

If a MQI message is truncated, pymqi.get will raise an exception and you will not be able to access the partially received message. This is bad news if you're using MQGMO_ACCEPT_TRUNCATED_MSG, as the message will be lost for good. The work round is to let pymqi allocate the buffer for you, or specify a big enough buffer, or stop accepting truncated messages. The latter options don't work too well, as you can't find out how big the actual message is!

Pymqi unpacks the the MsgId and CorrelId fields of the MQMD structure as a '24s', regardless of what you put in it. This leads to some asymmetry when sending integers with these field. e.g., On the put side:

 
	md.MsgId = struct.pack('l', 42)
And on the get side:
 
	msgId = struct.upack('l', md.MsgId[0:4])[0]

Test Scripts

The scripts directory contains pymqi test scripts & utilities.

Documentation

Pymqi has plenty of __doc__ strings. They're reproduced here for convenience.

Sami Salkosuo of IBM has written a developerWorks article about pymqi here. It gives a good overview and includes several good example applications.

Acknowledgments

Thanks to the following (in no particular order) for their code, suggestions, ports, bug-fixes etc.

Pascal Gauthier
John Kittel
Yves Lepage
John OSullivan
Tim Couper
Maas-Maarten Zeeman
Rich LaMarche
Kevin Kalbfleisch
Mauricio Strello
Brian Vicente
Jaco Smuts
Dariusz Suchojad

Disclaimer

You are free to use this code in any way you like, subject to the Python & IBM disclaimers & copyrights. I make no representations about the suitability of this software for any purpose. It is provided "AS-IS" without warranty of any kind, either express or implied. So there.

Consulting

I'm available for Python, MQ, Linux/Unix & C/C++ consulting assignments. See http://www.open-networks.co.uk and my cv for more details.

Contact Me

Feedback, Questions, Comments, Suggestions & Bugs to lsmithson@open-networks.co.uk. Or see my Home Page.

Hosted By

SourceForge.net Logo

Donate

Support This Project Donate to Python MQI Interface


(c) Copyright L. Smithson 2007.
$Id: pymqi.html,v 1.31.2.4 2006/03/16 12:35:14 lsmithson Exp $