EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Architecture of MsgConnect

Introduction

When it comes to information exchange, Windows operating systems can offer a developer various APIs and technologies - from low-level pipes, memory-mapped files, sockets and windows messages to higher-level COM, COM+, DCOM, and Automation. The only problem is that all these tools have different approaches, interfaces and ideologies. Developer has not to only develop a protocol for information exchange, but also choose an appropriate method of exchange. Real problems come with design and requirements change and the nightmare begins when more than one platform joins a game.

The idea behind the MsgConnect library is to have a single interface for message exchanging in every possible mean on every possible platform. With MsgConnect, while having a single interface for C++, C, Delphi, Visual Basic, C# and Java, you can exchange information with Windows, Windows CE, Palm, .NET and Java, using sockets with TCP, UDP and HTTP transports or simply memory-mapped files. This means that you don't have to reinvent the wheel when your solution grows, requirements change and platforms add.

This article will cover architecture of MsgConnect library and will describe library classes and most important methods and properties of these classes.

MsgConnect components

Let's take a closer look at MsgConnect library.

MsgConnect acts as a proxy between your application or service and underlying platform APIs. The library hides platform differences from you and exposes all needed functionality via well-designed object model.

Library is divided into messaging and transport layers. Developer operates on messaging layer, while messaging layer uses transport layer for sending and receiving messages. Developer can select an appropriate transport for messaging level. Depending on transport class, messaging layer can send messages in various means and there's no need in rewriting of high-level code and logic in application or service. Developer can also control how messages are routed and dispatched on higher-level messaging layer. Messaging layer can also apply transformations over the data being sent. This makes easy to compress, encrypt and sign the messages being sent over the network.

It's important that you'll have exactly the same structure of your application in any supported language and framework - all job is already done and you don't have to write communication code yourself when using MsgConnect. From another point of view, MsgConnect offers very flexible framework and has no restriction on the entire solution architecture - you can use MsgConnect for synchronous peer-to-peer messaging and grow to asynchronous messaging still using MsgConnect framework.

MsgConnect Class Hierarchy

MsgConnect library has a pretty simple hierarchy of classes.

Messenger class

Messenger class is a heart of MsgConnect framework. Using an instance of Messenger class, an application sends and receives messages. Messenger class is a glue that joins queue and transport systems together. Members of Messenger class resemble well-known Windows messaging functions. Windows developers will find it familiar to operate methods like GetMessage(), PeekMessage(), SendMessage() and PostMessage(). Besides these Windows messaging alike methods, Messenger class has several advanced methods for sending messages along with set of specific methods for messages routing. These advanced functions reflect more complex nature of the MsgConnect library.

Important methods of the Messenger class are:

  • PostMessage() - a method for one-way messaging, when no result is required.
  • SendMessage() - a method that sends a message and returns a result from peer.
  • SendMessageCallback() - a method that sends a message asynchronously. The application is notified about result using the callback function mechanism. No notification is given if there is no result.
  • SendMessageTimeout() - a method that waits for result from peer for a specified period of time and fails if no result returned or returns a result otherwise.
  • SendMessageTimeoutCallback() - a method that sends a message asynchronously. The application is notified about result, timeout or error using the callback function mechanism.
  • PeekMessage() - retrieves incoming message from queue and leaves it in the queue until GetMessage() called.
  • ForwardMessage() - forwards incoming message to another recipient.
  • DispatchMessages() - dispatches incoming messages. During the call:
    • application is notified about completed, failed and expired messages, sent using SendMessageCallback and SendMessageTimeoutCallback;
    • incoming messages are dispatched by finding appropriate MessageHandler object and firing it's OnMessage event. If no handler is found, Queue's OnUnhandledMessage is called. If there's no queue with the name specified in incoming message, the message is rejected.
  • GetMessage() - retrieves and deletes incoming message from queue.
  • MsgWaitForMultipleObjects() - returns when one of the following occurs: an incoming message arrives in the queue, either any one or all objects are in signaled state, or timeout interval elapses. The calling thread enters the wait state when this function is called
  • WaitMessage() and WaitMessageEx() - return when an incoming message arrives in the queue. The calling thread enters the wait state when this function is called. WaitMessageEx() lets you specify the time limit to wait.

Messenger class gives developers an interface for sending, receiving and forwarding messages and controlling message queue, associated with the Messenger instance. Like in any modern framework, a developer doesn't have to write message loops to retrieve incoming messages - MsgConnect library offers message queue class which simplifies message handling mechanism.

Message structure

Message structure is yet another core part of MsgConnect library. Message structure represents a message that is sent by Messenger class and dispatched by Queue and MessageHandler classes. Message structure encapsulates a data being transferred between peers.

Important fields of the Message structure are:

  • MsgCode - application-defined message code. Message is dispatched using this code.
  • Result - result of the message processing.
  • Param1, Param2 -user-defined data associated with message.
  • Data - binary data associated with message.
  • DataSize - size of the associated binary data.
  • DataType - specifies whether the message data is constant or the recipient can change the data.
  • Priority - message priority. Developer can use this field to change the order of messages in the input queue.

In order to use Data field of the Message structure, developer has to use utility functions of MsgConnect - MCMemAlloc() and MCMemFree(). Alternatively, Message structure can be filled with one line of code using one of Messenger's static methods CreateMessageSimple, CreateMessageFromBinary and CreateMessageFromText. The latter functions automatically allocate memory and copy binary or text information to the message.

Queue class

Queue class provides an effective mean for dispatching incoming messages - developer only has to write OnMessage() handler for MessageHandler class and add its instance to Handlers list of Queue class instance. Alternatively MsgConnect allows developer to handle incoming messages without creating specialized handler classes, by handling OnUnhandledMessage event.

Important properties and events of the Queue class are:

  • Messenger property - a reference to Messenger class instance.
  • QueueName property - a name of the current queue object that is used for message addressing purposes.
  • Handlers property - contains a list of message handler objects.
  • OnUnhandledMessage event - an event fired by the queue when no appropriate handler was found for incoming message.

When all properties of Queue class instance are set properly, a Messenger instance will use a name of queue specified in request address string to find an appropriate queue to send message to. A queue instance will use handlers list to handle incoming message in an appropriate way. If no handler is found within the handlers list, OnUnhandledMessage event handler will be called.

MessageHandler class

MessageHandler class represents a handler for specified range of messages. Developer has to instantiate MessageHandler class, set a range of message codes to process and create OnMessage handler to receive and process a message that falls into specified range.

Important properties and methods of the MessageHandler class are:

  • Enabled - specifies whether a handler will take part in message handling process. Developer can use this property to temporary disable a message handler.
  • MsgCodeLow, MsgCodeHigh - a range of messages this handler will intercept.
  • OnMessage - an event fired when incoming message within the specified range is received and is about to be processed.

BaseTransport class

Messenger class uses instances of transport classes to send messages. All transport classes are descendants of BaseTransport class. BaseTransport class is an abstract class that provides a common interface for all transport layer classes. Developer can use this interface to control transport protocol of his or her choice.

Important properties of the BaseTransport class are:

  • Messenger - a reference to Messenger class instance.
  • Compressor - a reference to message compressor class instance. Developer can use this property if there's a need in messages compression. Both peers should use compressors of one type in order to function correctly.
  • Encryptor - a reference to message cipher class instance. Encryptor object will be used by MsgConnect library to encrypt, securely send message and decrypt it on the remote side. Both peers should use ciphers of one type and use the same credentials in order to successfully communicate.
  • Sealer - a reference to message sealer class instance. Sealers sign message data, so peers can be sure in data integrity.
  • MaxMsgSize - largest message size that could be handled by the transport. All messages larger than this size will be discarded without any notifications for the remote side. Developer can use this limit to prevent messenger flooding with junk messages of large size.
  • MaxTimeout - maximum amount of time elapsed between receiving of two data chunks. If time elapsed between receiving of two data chunks will exceed this value, the connection will be considered broken. Note, that some transports will ignore this value, since they have no connection idea.

MsgConnect library has the following transport classes out-of-the-box:

  • DirectTransport - a transport that sends messages directly to the messenger or to another messenger instance in the same process. Developer can easily create multi-threaded message exchange in the application or service.
  • MMFTransport - a transport that uses memory-mapped files to transfer data. Developer can use this class to implement inter-process communications between service and GUI or between multiple instances of the application running on the same computer.
  • InetTransport family (SocketTransport, HttpTransport, HttpTransportClient) - a transport family that uses TCP and HTTP protocols respectively to transfer messages over network.
  • UDPTransport - a transport that uses UDP protocol to deliver messages. With broadcast support, it can be used for discovery and notification of applications, running on other computers.

BaseCompression class

BaseCompression class provides MsgConnect library with common interface to message compressors that can reduce the size of messages being transferred between peers. MsgConnect has ZLibCompression class out-of-the-box that implements ZLib compression algorithm.

BaseEncryption class

BaseEncryption class is a common interface to message ciphers that protect messages from being read by third party. MsgConnect has AESEncryption class out-of-the-box that implements AES encryption algorithm.

BaseSealing class

BaseSealing class provides MsgConnect library with common interface to message sealers that sign messages to ensure data integrity especially when transferring over the network.

Out-of-the-box sealing classes in MsgConnect library are:

  • AdlerSealing - provides transparent calculation and validation using Adler hash algorithm.
  • CRC32Sealing - provides transparent calculation and validation using CRC32 algorithm.
  • MD5Sealing - provides transparent calculation and validation using MD5 hash algorithm.

Router class

Router class extends addressing abilities of Messenger class. An instance of the Router class contains a table of routing rules. Messenger class can easily transfer an incoming message to another recipient.

Important properties and methods of the Router class:

  • Messenger property - a reference to the instance of the Messenger class.
  • SetRoutingRule() method - a method that adds a new rule to the routing table.

Out-of-the-box Classes

Here is the shortlist of out-of-the-box classes that will allow you to start writing messaging applications with minimum effort.

Transport classes:

  • DirectTransport - transport for in-process messaging.
  • MMFTransport - transport for inter-process messaging using memory-mapped files.
  • SocketTransport - TCP-based transport for network messaging.
  • HttpTransport - HTTP-based transport for network messaging.
  • UDPTransport - UDP-based transport for network messaging and service discovery.

Compression classes:

  • ZLibCompression - implementation of ZLib compression algorithm.

Encryption classes:

  • AESEncryption - implementation of AES encryption algorithm.

Sealing classes:

  • AdlerSealing - Adler algorithm for message signing.
  • CRC32Sealing - CRC32 algorithm for message signing.
  • MD5Sealing - MD5 algorithm for message signing.

How It Works

So, how is all described stuff joined together and works as a single entity? The answer to this question is a diagram below.

To start processing messages with MsgConnect, developer has to create an instance of Messenger class, link it with transport and queue class instances and implement appropriate message handlers.

The magic begins with SendMessage() or PostMessage() methods of Messenger class calls. Application puts appropriate data to Message structure and calls either one of the SendMessage*() method family or PostMessage() method. An instance of Messenger class finds a suitable transport for transferring a message to recipient and initiates information exchange process. Transport object preprocesses a message with transformers (if set) - a message can be compressed, encrypted and signed at this stage - and transfers the result of transformation to recipient. Recipient's transport layer checks the signature, decrypts and decompresses a message and processes it with its handlers. If the information exchange was initiated with the method from SendMessage*() method family, the recipient will return a result and optionally message data back to the sender. The recipient can change the message data if needed. Whether the data can be changed, is determined by DataType field of the message. Returned message experiences the same transformations as the message that was sent to the recipient before. If the message was sent with PostMessage, no reply is sent.

All these processes are done with minimum effort from developer's side when using MsgConnect.

Addressing Messages

MsgConnect library uses specially formed address string to determine the address of the peer. An address string contains name of the transport to use, destination messenger name - could be IP address or aplha-numeric name - depends on transport being used, and a destination queue name. MsgConnect will easily determine exact peer coordinates and deliver a message there.

Here are some examples of address strings that are used by MsgConnect library:

  • ExternalSocketTransport:212.22.48.192:1458|ControlQueue
  • Local:SecondaryWindowQueue
  • HTTP:www.eldos.com/balance.dll:8080|Main
  • MMF:MyCoolMapping|Transfer

Prioritizing Messages

When the message is sent, it will be placed to the end of the recipient's queue, where it will wait for its turn to be processed. Sometimes sender needs to send a kind of urgent message (or out-of-band message) that has to be processed as soon as possible by the peer. Developer can use priorities to change the order in which the messages are placed to the outgoing queue. The bigger priority number, the faster a message will be processed by the peer. On the other hand, developer can decide to use some kind of keep-alive messages that have to be sent from time to time to the remote side. Obviously, this kind of messages needs to have the lowest priority in the queue and MsgConnect makes this possible with ease.

Message routing

The world changes - the rules of the solution reflect these changes. Only yesterday your server was doing all possible computations, but today the volume of the data being processed has grown significantly and you have to move the hard-working part of the server to another machine with more silicon power stuffed in there. But all clients are still not aware of the moves and sending messages to the old machine. Upgrade client side? No. Use message forwarding. The server on the old machine will still receive requests from the client, but after peeking into incoming messages, will forward them to the new server. So clients will not see any differences.

You can also decide to hide actual message recipients behind the outer proxy for security reasons. The outer server will accept all incoming messages and only forward them to the actual performers without exposing them to the Internet.

Imagine how you would implement this yourself. MsgConnect can do this hard job for you - see ForwardMessage() method of Messenger class and Router class for the reference.

Secure Messaging

Today the security is one of the most important things of the software solutions. The systems you're developing can be used for transferring very important and top secret information. MsgConnect library offers a great number of tools that will help a developer to build secure messaging systems with ease. With MsgConnect you'll be able to encrypt the messages passing to peers and verify peers' credentials to protect the system from false messages. Refer to AESEncryption class, MessageCredentials structure and OnValidateCredentials event of Messenger class descriptions for more details on how to secure your messages.

Conclusion

As shown above, information exchange can be done easily if using the right tool. MsgConnect significantly simplifies the life of the developer who faced this problem. Adding the capability to transparently exchange information between systems working on different computers and different software platforms will take just 20-30 minutes with this library. Now you can develop your solution without worrying of upcoming requirements changes and project growth.

Return to the list

|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!