EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Really great news, and a little bad news

Also by EldoS: CallbackDisk
Create virtual disks backed by memory or custom location, expose disk images as disks and more.
#2066
Posted: 01/25/2007 17:34:18
by Christopher Zielinski (Basic support level)
Joined: 04/16/2006
Posts: 28

Wow, things can get tricky when there are various people doing different things in different areas of the world.

1. With a previous version of MsgConnect (v.1.5.7.46 – I think), all of our UDP PostMessage (MsgConnect) messages worked as expected.

2. With the current version of MsgConnect (v.1.5.7.52) some UDP PostMessage (MsgConnect) messages work, but most do not. The only changed that was introduced was an upgrade in MsgConnect.

3. Originally, I had thought that this issue was related to our use of Multi-cast UDP communications, but further testing and research found that direct (round-robin) UDP communications were also failing with this latest release of MsgConnect (for Delphi). To make the issue more frustrating, everything was working for me in my development environment/network, but not working for our software quality assurance department that is remote to my local.

Now for the great news!

Testing in my network differed from that reported by our software quality assurance department. I found this incredibly difficult to believe, but I persevered and pressed on for an explanation. An explanation has been found and with that explanation, a hope that MsgConnect can be further improved...

Our software quality assurance department was running my server application (that uses MsgConnect) in a VMware player! With previous releases of MsgConnect that we had used, everything worked as expected in this testing environment, but something in MsgConnect has changed and that change prevents it from working properly within VMware player now.

You may download the free VMware Player from here:
http://www.vmware.com/products/player/
http://www.vmware.com/download/player/

The following describes our software testing environment where MsgConnect UDP messaging previously worked but does not appear to work with the latest release:

- Microsoft Windows 2003 Server running within a VMware player is used to host our (MsgConnect-based) server-side software solution.

- Microsoft Windows XP is used to execute our client-side software solution.

- Both server and client computers are connected together via a hub.
#2081
Posted: 01/26/2007 10:10:53
by Eugene Mayevski (EldoS Corp.)

This is what I mentioned in the previous letter - the VMWare router plays it's role in making some UDP messages underliverable. But UDP is not a reliable protocol and nobody guarantees that the messages will be delivered. So I don't see any problems here. If you need guaranteed delivery, use TCP.


Sincerely yours
Eugene Mayevski
#2117
Posted: 01/29/2007 13:18:34
by Christopher Zielinski (Basic support level)
Joined: 04/16/2006
Posts: 28

Quote
Eugene Mayevski wrote:
This is what I mentioned in the previous letter - the VMWare router plays it's role in making some UDP messages underliverable. But UDP is not a reliable protocol and nobody guarantees that the messages will be delivered. So I don't see any problems here. If you need guaranteed delivery, use TCP.


Some more bad news... I am afraid.

It was my intention to use UDP, but in my haste, I discovered that I was using the "guaranteed delievery" with "TCP"... so everything I have written above remains true, EXCEPT it should read "TCP" rather than "UDP", because of the cutting and pasting of code. The actual code that was tested was this:

Code
procedure TdmRsiCommSvr.BroadcastMessage(rMsg: TMcMessage);
var
  iCnt, iEnd: Integer;
  sAddr: String;
begin
  try
    // MyMessenger.PostMessage(fGetUdpBroadcastDest(), rMsg, nil);
    iEnd := (dmRsiNwData.UserAccts.Count - 1);
    for iCnt := 0 to iEnd do begin
      if (dmRsiNwData.UserAccts.Items[iCnt].OnLine) then
        with dmRsiNwData.UserAccts.Items[iCnt].NetAcct.PeerAddr do begin
          sAddr := '';
          sAddr := (sAddr + TCP.Name);
          sAddr := (sAddr + ':');
          sAddr := (sAddr + inet_ntoa(in_addr(NetAddr)));
          sAddr := (sAddr + ':');
          sAddr := (sAddr + IntToStr(NetPort));
          sAddr := (sAddr + '|');
          sAddr := (sAddr + MyQueue.Name);
          MyMessenger.PostMessage(sAddr, rMsg, nil);
      end;
    end;
  except
    on e: Exception do
      dmRsiEvtLog.LogEvent(CAT_SYSTEM_EVENT, MSG_EXCEPTION, _MODULE_NAME, e);
  end;
end;


To use "UDP" rather than "TCP", I would have had to change this:

Code
sAddr := (sAddr + TCP.Name);


To this:

Code
sAddr := (sAddr + UDP.Name);


Such a simple mistake... oh my.

So now that we have removed UDP from the equation, what do you think would cause our "BroadcastMessage(rMsg: TMcMessage);" to fail on two client computers (one of which is running in VMware, the other being Win2K), while all of our other computers work fine?

#2118
Posted: 01/29/2007 13:24:28
by Christopher Zielinski (Basic support level)
Joined: 04/16/2006
Posts: 28

P.S.: We have 9 of our clients connecting to our server and 2 of those clients do not receive some of their MsgConnect messages from our server process.

Both clients that do not receive TCP MsgConnect messages are Microsoft Windows 2000 O/S, but some of our clients that do work are also Microsoft Windows 2000!
#2119
Posted: 01/29/2007 13:27:44
by Christopher Zielinski (Basic support level)
Joined: 04/16/2006
Posts: 28

P.P.S. Our problem computers are not multi-homed computers, that is that they have a single TCP/IP address on the network. When both our server and client are run on the same problem computer, it works fine.
#2121
Posted: 01/30/2007 03:15:44
by Eugene Mayevski (EldoS Corp.)

1) Try using SendMessageTimeoutCallback instead of PostMessage and see the result. I mean handle OnError and OnTimeout
2) maybe those client are behind NAT or firewall? Try connecting to the destination port of those computers using telnet and see if connection succeeds.


Sincerely yours
Eugene Mayevski
#2123
Posted: 01/30/2007 12:25:23
by Christopher Zielinski (Basic support level)
Joined: 04/16/2006
Posts: 28

Quote
Eugene Mayevski wrote:
1) Try using SendMessageTimeoutCallback instead of PostMessage and see the result. I mean handle OnError and OnTimeout
2) maybe those client are behind NAT or firewall? Try connecting to the destination port of those computers using telnet and see if connection succeeds.

Sincerely yours,
Eugene Mayevski


Hello Eugene,

Thank you for your kind reply and suggestful tips.

With regards to your point number 2 -- both client and server processes are running in an "open" private network where firewall (or similar) communication limitations do not exist.

With regards to your point number 1 -- well I can try this.

I have some more really good news! When we use a different computer system, everything works fine!!! With the somewhat bad news being, what do we do it this was in a production environment?

Today I am working in our software quality assurance department offices, so I can experience and debug this issue first-hand. I have narrowed down this issue to a single Microsoft Windows 2000 machine that is used by our primary software quality assurance tester. When our server-side software is executed on this specific machine, our client-side software will not function properly on it as well (n.b. this configuration will work on another machine), and another Microsoft Windows 2000 machine at this site will also not function properly (n.b. although there are other Microsoft Windows 2000 machines that do with this server configuration).

I suspect that there is an issue with this test computer, because the Delphi Remote Debugger keeps dieing on me (by throwing an exception), while debugging our server process on this machine, whereas I was able to successfully perform the same debugging tasks in my development environment. That said, these two issues very well could be independent of each other and it is just a coincidence that they both happen on this machine.

Unless I/we can recreate this issue elsewhere, I will consider it closed. If it occurs elsewhere, I will provide further details/findings related to it.

Oddly though -- a previous version of MsgConnect (n.b. I am not 100% which version it was and will never be so careless again -- recording what of MsgConnect we use in each build) does work on this problem machine.

While I can say "Oh, it only occurs on this single machine", I am still concerned and wonder what I should actually do if this issue actually happens in a production environment at a very remote location. The issue is very specific to MsgConnect, as it is specifically related to the messaging between the client and server process (nothing else is failing)... actually, all of the messages to the server (via SendMessage) always work (the client initiates the connection with the server via a login process), it is the PostMessage's back to the client that are failing, when a failure does occur (and this failure has now been observed with both UDP and TCP).

Perhaps handling the OnError and OnTimeout will fix and/or shed further light on the issue for us… it such testing providing anything tangible… I will certainly let you know.
#2125
Posted: 01/30/2007 17:19:39
by Christopher Zielinski (Basic support level)
Joined: 04/16/2006
Posts: 28

Great news!

After working on this issue/problem at our software quality assurance department's physical site, I now have a much better picture of what is going on.

THE LATEST VERSION OF MSGCONNECT IS THE MOST STABLE RELEASE I HAVE WORKED WITH ! ! !

I am really happy that the UDP port closing issue has been resolved because that issue appears to be related to our software occasionally hanging and MsgConnect utilizing 100% of the CPU.

As for my issue reported here... well our software quality assurance department sent me down the wrong trail by stating that the previous version of our software (that uses a prior release of MsgConnect) worked in the same configuration (when nothing else in our software changed). MY OWN TESTING verified that this statement could not possibly be true right now -- that is, if they were to install that previous build today, it is not possible for it to work (although I am confident that it really did work in the past).

MsgConnect communications appear to fail, simply because they were not being performed. Our application is extremely complex... but the actual reason the client was not getting the messages that we were expecting was due to missing identifiers in a realtime hardware event. What made this so difficult to trace down was the fact that it works in almost all installations and the fact that I could not use Delphi’s Remote Debugger to debug the server process on the computer system where the issue did manifest itself.

I regret any trouble that I may have caused you with this incident, am very happy that I have stuck with MsgConnect through its bugs and my own, and look forward to a smooth future with its use.

Code prior to the MsgConnect broadcast implementation…

Code
  if (fDevId = NIL_DEVICE_ID) then Abort;
  if (fCallId = NIL_CALL_ID) then Abort;
Also by EldoS: CallbackDisk
Create virtual disks backed by memory or custom location, expose disk images as disks and more.

Reply

Statistics

Topic viewed 5389 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!