EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Using SSH waiting for command to finish

Also by EldoS: CallbackFilter
A component to monitor and control disk activity, track file and directory operations (create, read, write, rename etc.), alter file data, encrypt files, create virtual files.
#26416
Posted: 09/12/2013 18:44:40
by Anthony Wilder (Standard support level)
Joined: 05/09/2013
Posts: 8

Hi,

we use eldos SSH SecureBlack box to run "mv" commands on large databases to backup storage however. The only way we can determine that the file has moved all the way to the backup storage. is by using "firebird" to attempt to open the destination file.

This could raise a possibility of corruption.

How can we know that an SSH command has been completed? Have not been able to identify this with the samples available.

Using SecureBlackbox 10.
#26417
Posted: 09/12/2013 23:36:48
by Eugene Mayevski (EldoS Corp.)

If you use command channel (i.e. just set Command property of the component), then the server will close connection (or better say the command channel, and this in turn will cause closing the connection) once the command is complete.

If you use shell tunnel, then you need to parse output, as there's no way to learn anything in automated way (shell is for human reading, not for automated processing).


Sincerely yours
Eugene Mayevski
#26435
Posted: 09/15/2013 20:01:18
by Anthony Wilder (Standard support level)
Joined: 05/09/2013
Posts: 8

Hi Eugene,

In order to do this we need to pass the following list of commands to the linux server.

chown -f firebird:firebird NEWDBFILE
mv -f NEWDBFILE FINALDBFILE
chown -f firebird:firebird FINALDBFILE

can we pass this set of commands to the SSH tunnel and know when they are finished?

If so, is there a demo for this?
#26436
Posted: 09/16/2013 00:21:56
by Eugene Mayevski (EldoS Corp.)

there exist two options:
1) create a script on the server OR
2) put the above commands to Commands property of the TElSimpleSSHClient component. This will make the component to use command tunnel, execute commands one by one and close connection.

There's no sample needed for option two - you need to set Commands property and connect, the component will do the rest.


Sincerely yours
Eugene Mayevski
#26438
Posted: 09/16/2013 01:50:47
by Anthony Wilder (Standard support level)
Joined: 05/09/2013
Posts: 8

Hi Eugene,

I already Use the Commands property.
I have appended my code.
It seems to me this may actually be a Bug, then If you are saying the Commands property should wait for the commands to finish.

Note: Client is a TElSimpleSSHClient object.


Code
function TRunSSH.RunSSHCommands(strHostName, strSSHPassword : String;
  slCommands: TStringList; CheckHasFinished : THasTerminated ): Boolean;
begin
  Result := FALSE;
  CreateSSH( strSSHPassword );
  Client.Commands.ASsign(slCommands);
  Client.Address := strHostName;
  try
    Client.Open;
    if( Client.Active )then
      begin
        SendToOutput('Connection started');
        SendToOutput('Server: ' + Client.ServerSoftwareName);
        SendToOutput('Version: SSHv2');
        SendToOutput('PublicKey algorithm: ' + IntToStr(Client.PublicKeyAlgorithm));
        SendToOutput('Kex algorithm: ' + IntToStr(Client.KexAlgorithm));
        SendToOutput('Block algorithm: ' + IntToStr(Client.EncryptionAlgorithmServerToClient));
        SendToOutput('Compression algorithm: ' + IntToStr(Client.CompressionAlgorithmServerToClient));
        SendToOutput('MAC algorithm: ' + IntToStr(Client.MacAlgorithmServerToClient));
        OutputAnyResponse;
        if( Assigned( CheckHasFinished ))then
          Result := CheckHasFinished
        else
          Result := TRUE;
        Client.Close;
      end;
  except
    on E:Exception do
      begin
        SendToOutput(E.Message);
      end;
  end;
  FreeSSH;
end;
#26440
Posted: 09/16/2013 02:08:39
by Eugene Mayevski (EldoS Corp.)

What particular problem are you having?

There's no room for a "bug" in the code and here's why. Commands are executed one by one and when the command channel is closed (i.e. the command is complete) is determined by the server. The component doesn't send all commands immediately but opens a new channel for the command after the existing command is done. So if the server closes the channel prematurely, there's not much we can do.

There's one more alternative in your case: combine everything into one command:

Quote

chown -f firebird:firebird NEWDBFILE; mv -f NEWDBFILE FINALDBFILE ; chown -f firebird:firebird FINALDBFILE


And put it to Command property.

If you set RequestTerminal to true, the shell will execute all commands in one line and within one CommandChannel.


Sincerely yours
Eugene Mayevski
#26471
Posted: 09/16/2013 19:32:25
by Anthony Wilder (Standard support level)
Joined: 05/09/2013
Posts: 8

Unfortunately when I changed the code to:

Code
function TRunSSH.RunSSHCommands(strHostName, strSSHPassword : String;
  slCommands: TStringList; CheckHasFinished : THasTerminated ): Boolean;
begin
  Result := FALSE;
  CreateSSH( strSSHPassword );
  slCommands.Delimiter := ';';
  Client.Command := slCommands.DelimitedText;
  Client.RequestTerminal := TRUE;
//  Client.Commands.Assign(slCommands);

  Client.Address := strHostName;
  try
    Client.Open;
    if( Client.Active )then
      begin
        SendToOutput('Connection started');
        SendToOutput('Server: ' + Client.ServerSoftwareName);
        SendToOutput('Version: SSHv2');
        SendToOutput('PublicKey algorithm: ' + IntToStr(Client.PublicKeyAlgorithm));
        SendToOutput('Kex algorithm: ' + IntToStr(Client.KexAlgorithm));
        SendToOutput('Block algorithm: ' + IntToStr(Client.EncryptionAlgorithmServerToClient));
        SendToOutput('Compression algorithm: ' + IntToStr(Client.CompressionAlgorithmServerToClient));
        SendToOutput('MAC algorithm: ' + IntToStr(Client.MacAlgorithmServerToClient));
        OutputAnyResponse;
        if( Assigned( CheckHasFinished ))then
          Result := CheckHasFinished
        else
          Result := TRUE;
        Client.Close;
      end;
  except
    on E:Exception do
      begin
        SendToOutput(E.Message);
      end;
  end;
  FreeSSH;
end;



The SSH completely failed when reading the failed message it is important to note that the "bash" failure is wrong "op.old" does exist. Below is the linux error:

Quote

bash: chown -f firebird:firebird /data/burgesV2/data/backup/op.old: No such file or directory bash: mv -f /data/burgesV2/data/backup/op.old /data/burgesV2/data/op.oldtest: No such file or directory bash: chown -f firebird:firebird /data/burgesV2/data/op.oldtest: No such file or directory


So close.
#26474
Posted: 09/17/2013 00:44:58
by Eugene Mayevski (EldoS Corp.)

That sounds very strange - bash has understood your command but haven't found the file in question. We are really close, as the client has performed its job correctly and it's the server-side glitch that we need to discover now.

1) Please try to set RequestTerminal to false. Sometimes this makes the server start different interpreter and ignore per-user shell configuration (and this in turn changes results).

2) the paths indeed sound strange. Are you sure that the path starting with "/data" is a correct absolute path for your system? This is not a path relative to user's home folder.


Sincerely yours
Eugene Mayevski
#26502
Posted: 09/18/2013 01:05:12
by Anthony Wilder (Standard support level)
Joined: 05/09/2013
Posts: 8

OK,

1. I took request terminal off, there was no change.
2. Below is a list of 2 separate processes. The secong uses the "Commands" property and it works. The first uses the "Command" property and it fails.
3. I added a "ls -l <path>" command to verify directory existence.

It seems it is to do with the 2 properties work very differently. I know no more that that.


Kind Regards,

Robert.

Quote

<14>1 2013-09-18T15:53:43.767+10:00 10.230.0.76 BRDBAdmin.exe 5960 - - ls -l /data/burgesV2/data
<14>1 2013-09-18T15:53:43.767+10:00 10.230.0.76 BRDBAdmin.exe 5960 - - chown -f firebird:firebird /data/burgesV2/data/op.oldtest
<14>1 2013-09-18T15:53:43.767+10:00 10.230.0.76 BRDBAdmin.exe 5960 - - mv -f /data/burgesV2/data/op.oldtest /data/burgesV2/data/backup/op.old
<14>1 2013-09-18T15:53:43.768+10:00 10.230.0.76 BRDBAdmin.exe 5960 - - chown -f firebird:firebird /data/burgesV2/data/backup/op.old
<14>1 2013-09-18T15:53:43.884+10:00 10.230.0.76 BRDBAdmin.exe 5960 - - Server key received (RSA). Fingerprint is 5c:2d:2c:0f:98:45:23:ec:29:8c:48:ed:28:69:ba:e8
<14>1 2013-09-18T15:53:43.922+10:00 10.230.0.76 BRDBAdmin.exe 5960 - - Authentication succeeded
<14>1 2013-09-18T15:53:43.994+10:00 10.230.0.76 BRDBAdmin.exe 5960 - - Connection started
<14>1 2013-09-18T15:53:43.995+10:00 10.230.0.76 BRDBAdmin.exe 5960 - - Server: OpenSSH_5.3
<14>1 2013-09-18T15:53:43.996+10:00 10.230.0.76 BRDBAdmin.exe 5960 - - Version: SSHv2
<14>1 2013-09-18T15:53:43.996+10:00 10.230.0.76 BRDBAdmin.exe 5960 - - PublicKey algorithm: 1
<14>1 2013-09-18T15:53:43.996+10:00 10.230.0.76 BRDBAdmin.exe 5960 - - Kex algorithm: 3
<14>1 2013-09-18T15:53:43.998+10:00 10.230.0.76 BRDBAdmin.exe 5960 - - Block algorithm: 16
<14>1 2013-09-18T15:53:43.999+10:00 10.230.0.76 BRDBAdmin.exe 5960 - - Compression algorithm: 0
<14>1 2013-09-18T15:53:44.000+10:00 10.230.0.76 BRDBAdmin.exe 5960 - - MAC algorithm: 11
<14>1 2013-09-18T15:53:44.508+10:00 10.230.0.76 BRDBAdmin.exe 5960 - - Connection closed.
<14>1 2013-09-18T15:53:44.509+10:00 10.230.0.76 BRDBAdmin.exe 5960 - - bash: ls -l /data/burgesV2/data: No such file or directory bash: chown -f firebird:firebird /data/burgesV2/data/op.oldtest: No such file or directory bash: mv -f /data/burgesV2/data/op.oldtest /data/burgesV2/data/backup/op.old: No such file or directory bash: chown -f firebird:firebird /data/burgesV2/data/backup/op.old: No such file or directory
<9>1 2013-09-18T15:53:44.562+10:00 10.230.0.76 BRDBAdmin.exe 5960 - - ISC ERROR CODE:335544344 ISC ERROR MESSAGE: I/O error during open operation for file /data/burgesV2/data/backup/op.old Error while trying to open file No such file or directory
<14>1 2013-09-18T15:54:39.635+10:00 10.230.0.76 BRDBAdmin.exe 6212 - - ls -l /data/burgesV2/data
<14>1 2013-09-18T15:54:39.636+10:00 10.230.0.76 BRDBAdmin.exe 6212 - - chown -f firebird:firebird /data/burgesV2/data/op.oldtest
<14>1 2013-09-18T15:54:39.636+10:00 10.230.0.76 BRDBAdmin.exe 6212 - - mv -f /data/burgesV2/data/op.oldtest /data/burgesV2/data/backup/op.old
<14>1 2013-09-18T15:54:39.637+10:00 10.230.0.76 BRDBAdmin.exe 6212 - - chown -f firebird:firebird /data/burgesV2/data/backup/op.old
<14>1 2013-09-18T15:54:39.728+10:00 10.230.0.76 BRDBAdmin.exe 6212 - - Server key received (RSA). Fingerprint is 5c:2d:2c:0f:98:45:23:ec:29:8c:48:ed:28:69:ba:e8
<14>1 2013-09-18T15:54:39.763+10:00 10.230.0.76 BRDBAdmin.exe 6212 - - Authentication succeeded
<14>1 2013-09-18T15:54:39.832+10:00 10.230.0.76 BRDBAdmin.exe 6212 - - Connection started
<14>1 2013-09-18T15:54:39.833+10:00 10.230.0.76 BRDBAdmin.exe 6212 - - Server: OpenSSH_5.3
<14>1 2013-09-18T15:54:39.834+10:00 10.230.0.76 BRDBAdmin.exe 6212 - - Version: SSHv2
<14>1 2013-09-18T15:54:39.834+10:00 10.230.0.76 BRDBAdmin.exe 6212 - - PublicKey algorithm: 1
<14>1 2013-09-18T15:54:39.835+10:00 10.230.0.76 BRDBAdmin.exe 6212 - - Kex algorithm: 3
<14>1 2013-09-18T15:54:39.838+10:00 10.230.0.76 BRDBAdmin.exe 6212 - - Block algorithm: 16
<14>1 2013-09-18T15:54:39.839+10:00 10.230.0.76 BRDBAdmin.exe 6212 - - Compression algorithm: 0
<14>1 2013-09-18T15:54:39.840+10:00 10.230.0.76 BRDBAdmin.exe 6212 - - MAC algorithm: 11
<14>1 2013-09-18T15:54:40.348+10:00 10.230.0.76 BRDBAdmin.exe 6212 - - total 65551960 lrwxrwxrwx. 1 root root 23 Jul 12 13:50 backup -> /data1/burgesV2/backup/ -rw-r--r--. 1 root root 12788475392 Sep 12 09:16 op.fbk -rw-rw----. 1 firebird firebird 27903131648 Sep 18 15:54 op.fdb -rw-rw----. 1 firebird firebird 56655872 Jul 13 06:54 opinit.fdb -rw-rw----. 1 firebird firebird 26376929280 Sep 18 15:37 op.oldtest
#26503
Posted: 09/18/2013 01:11:26
by Eugene Mayevski (EldoS Corp.)

Please try using "ls -l /" as the first command when using Command property - let's see if bash doesn't like "/data/..." path or it doesn't like the way commands are merged into one line.

Also from your previous message:
Quote
Anthony Wilder wrote:
The SSH completely failed when reading the failed message


What exactly "completely failed" means in this case? What are the symptoms?


Sincerely yours
Eugene Mayevski
Also by EldoS: BizCrypto
Components for BizTalk® and SQL Server® Integration Services that let you securely store and transfer information in your business automation solutions.

Reply

Statistics

Topic viewed 3568 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!