EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Peer-to-peer Communications With MsgConnect

What do we call a distributed system? While there is no single definition of such a system, commonly it is assumed to comprise several (several dozens or more) autonomous and related with a common goal computers, each of which has its own local memory, and these computers communicate with each other by passing messages through a network.

Quite common practice while designing distributed systems is to use client-server architecture. This architecture took significant place in current networking because it's relatively easy to organize it and to maintain. The main logic of client-server system is implemented on a single central node that makes a problem of security and integrity of data much easier. The concept of conversation between a client and a server is not hard to design, create and document.

However from technical point of view creation of effective server software can be a long and painful process. The things can become much worse if the logic of your application requires the server to send a request to the client, i.e. to invert the roles of client and server. On this stage one has to redesign the complete protocol stack to add such a possibility. Sometimes it occurs more effective and logical to use the peer-to-peer architecture.

Peer-to-peer networking in distributed computing is an architecture that loads tasks or parts of work between peers, or nodes. Peers are equally privileged, equipotent participants in the distributed application in sense of a role they play, though some of them may differ in sense of available resources such as processing power, disk storage or network bandwidth, and may have not full scope of the system. In that case some peers make a portion of their resources directly available to other network participants on their request. Peers are both suppliers and consumers of resources, in contrast to the traditional client-server model where servers supply, and clients consume.

In peer-to-peer networks nodes provide resources to each other, which may include communication channels, storage space and computing power. The more nodes arrive in peer-to-peer system the more capacity the system provides. In contrast, in typical client-server architecture clients share only their demands but not their resources. In this case, the more clients join the system the less resources are available to serve each client. The distributed nature of peer-to-peer networks also increases robustness by enabling peers to find the data without relying on a centralized index server. In this case there is no single point of failure in the system.

Peer-to-peer networks are typically used for connecting nodes via ad hoc connections. Sharing content files with audio, video, data or anything in digital format is very common, and real time data, such as audio or telephony traffic, is also passed using peer-to-peer technology.

Why MsgConnect?

MsgConnect is based on the concept of exchanging messages — blocks of data that have a fixed part with predefined fields and optionally have a data part. Using messages you can send commands to other processes, receive replies, transfer data across multiple processes and do plenty of other useful things.

MsgConnect was born as emulation of Windows Messaging subsystem for sending messages across the network. The idea was successful and EldoS engineers extended it. Now MsgConnect is widely used by developers to implement data transfer between applications running on the same computer or on various mobile devices like Android, BlackBerry (through Java Mobile), iPhone (Ñ++ version for MacOS X and iPhone/iPad).

MsgConnect simplifies your application development significantly by taking care of all the low-level tasks for you. It was designed to be small, fast and effective and at the same time to provide complete service. With MsgConnect you don't have to write and painfully test multithreaded server code, you won't need to split the data stream into messages and dispatch these messages. All your efforts will be directed at business logic whilst MsgConnect takes care about data and messages transfer.

While utilizing the same concept of message queue that was used in Windows, MsgConnect provides much more reliability and functionality like identification, on-the-fly compression, encryption and integrity checking of the message being sent and received. With MsgConnect you don't care about transferring data, instead you just send messages (which can contain all necessary data).

With MsgConnect you use methods similar to Windows messaging subsystem — SendMessage to send a message with a confirmation of a second party, PostMessage to transmit data without confirmation, SendMessageCallBack to send a message and be notified through user-defined callback routine.

Where to Use MsgConnect

In our previous article, "MsgConnecting Desktops and Mobile Devices", we described an example of fictitious hotel, in which we intended to implement a distributed system for everything that a hotel needs. Now let's narrow our scope and imagine two nodes of that system, for example Accounting and Personal Department. Each of these nodes has it's own database, something like Finances for accounting and Persons for personal department. Occasionally one of these nodes will have a need to make a request to his partner's database. If you try to assign them roles of servers and clients, you will face serious problems.

But you really don’t have to! If you use the power of MsgConnect, each of your nodes can act as a client or a server or both, depending on a situation.

And if you imagine two more nodes, for example Cleaners and Guards, where Cleaners with his relatively low importance is physically connected to a single node Guards, the node Guards may act as a repeater to provide internet connection to Cleaners and in the same time make requests to Cleaners database for cleaning schedule.

What to Think About

When designing a message passing philosophy you have to make several important decisions, for example like followings.

Reliability

Whether your messages should be transferred reliably? A reliable protocol may ensure reliability on a per-recipient basis, as well as provide properties that relate the delivery of data to different recipients.

Reliable protocols typically incur more overhead than unreliable ones, and as a result, are slower and less scalable. This often isn't an issue for unicast protocols (one-to-one), but it may be a problem for multicast (many-to-one) protocols.

TCP, the main protocol used in the Internet today, is a reliable unicast protocol. UDP, often used in computer games or other situations where speed is important and the loss of a little part of data is not, is an unreliable protocol. Your hotel system definitely may include some voice communications using UDP.

Both reliable and unreliable transmission of data is easy to implement with the help of MsgConnect. You don't have to deal with TCP or UDP directly; instead you just use the right transport (see later).

Synchronous or Not

Should your messages be synchronous or asynchronous?

Synchronous message passing require the sender and receiver to wait for each other to transfer the message. That is, the sender will not continue until the receiver has received the message and confirmed it.

Synchronous communication has several advantages. One of them is that reasoning about the program can be simplified in that there is a synchronization point between sender and receiver on message transfer. The second advantage is that no buffering is required. The message can always be stored on the receiving side, because the sender will not continue until the receiver is ready.

Asynchronous message passing systems deliver a message from sender to receiver without waiting for the receiver to be ready. The advantage of asynchronous communication is that the sender and receiver can overlap their computation because they do not wait for each other.

Both of those techniques can be implemented with the help of MsgConnect.

Unicast or Multicast

Whether your messages should be passed one-to-one, one-to-many, or many-to-one (like in client-server architecture)?

Unicast transmission assumes sending messages to a single network destination identified by a unique address. Broadcasting means transmitting the same data to all possible destinations, while with multicasting you send data only to interested destinations by using special address assignments.

All of these possibilities are available for you when using MsgConnect, in case you choose the right transport.

The Right Transport

MsgConnect hides the specifics of certain protocols like TCP/UDP or HTTP as much as possible; instead you just create a message that is to be sent later with the help of convenient transport. Available transports are MCSocketTransport (through TCP), MCHTTPTransport (through HTTP and TCP) and MCUDPTransport. To transfer a message you have to select the transport and to supply the transport-specific address of delivery.

Transferring files

You don't have to bother with FTP when you need to send a file from one node of your distributed system to another. For that purpose you split your file by parts of appropriate size and send them like messages with a data sections; you know already that messages created with MsgConnect can contain data section. Of course, you have to choose a reliable transport for that case.

Conclusion

With MsgConnect you gain a whole power of distributed application running in a cross-platform environment including desktops and mobile devices, each of which may act as a client or a server in the same time, or you can make the things work without central server at all, designing a robust, scalable and easy maintained peer-to-peer system.

Return to the list

|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!