EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Encoding issues while encrypting and decrypting

Also by EldoS: BizCrypto
Components for BizTalk® and SQL Server® Integration Services that let you securely store and transfer information in your business automation solutions.
#25394
Posted: 06/22/2013 12:58:37
by Karan  (Basic support level)
Joined: 06/06/2013
Posts: 10

I've written out EncryptAndSign and DecryptAndVerify methods and they work fine. They both read a text file and write back to a text file.

I need to encrypt a xml file, send it over the network via streams (i'm using C# streams), and decrypt at the other end. To accomplish this, i call my EncryptAndSign method which saves the encrypted version of the xml file to a different file. Then i read this encrypted file, convert it to a byte array and send it over using tcp. On the other end, i first receive the byte array, convert it to string and write it to a file. Then i call DecryptAndVerify on the file i just wrote (received encrypted file) with the hope that it will write the plainText xml to a file.

The issue i'm facing is related to reading/writing the encrypted file correctly or between various conversions of strings to byte array or vice versa(as far as i know). I've tried different encoding schemes, like UTF8, Unicode and ASCII. I get different exceptions from DecryptAndVerify like "Invalid Header" when using ASCII and "Unexpected packet" when using Unicode.

The following is my code:

Code for Server:

Code
public void ExchangeKeysForServer(StreamReader sr, StreamWriter sw, TcpClient client)
        {

            StreamReader fileStreamReader;
            StreamWriter fileStreamWriter;

            fileStreamReader = new StreamReader("..\\..\\..\\..\\certificates\\rsa-publickey.txt");

            string selfRsaPublicKey = fileStreamReader.ReadToEnd();

            fileStreamReader.Close();

            fileStreamReader = new StreamReader("..\\..\\..\\..\\certificates\\rsa-secretkey.txt");

            string selfRsaSecretKey = fileStreamReader.ReadToEnd();

            fileStreamReader.Close();

            Console.WriteLine(selfRsaPublicKey + Environment.NewLine + selfRsaSecretKey);

            OpenPGP objPGP = new OpenPGP();

            bool encStatus = objPGP.EncryptAndSignText("..\\..\\..\\..\\certificates\\rsa-publickey.txt", "encrypted-self-rsa-publickey.txt", objOtherKeyRing, objSelfKeyRing, "password for encryption here");

            //read encrypted self rsa public key from file
            fileStreamReader = new StreamReader("encrypted-self-rsa-publickey.txt");

            //send encrypted self rsa public key
            SendMessage(sw, Encoding.ASCII.GetBytes(fileStreamReader.ReadToEnd()));

            fileStreamReader.Close();

            //receive encrypted other's rsa public key and write to file
            string encryptedOtherRsaPublicKey = Encoding.ASCII.GetString(ReadMessage(sr));


            fileStreamWriter = new StreamWriter("encrypted-other-rsa-publickey.txt");

            fileStreamWriter.WriteLine(encryptedOtherRsaPublicKey);

            fileStreamWriter.Close();

          
            bool decStatus = objPGP.DecryptAndVerifyText("encrypted-other-rsa-publickey.txt", "other-rsa-publickey.txt", objSelfKeyRing, objOtherKeyRing, "password for decryption", "other-rsa-publickey.txt");

            Console.ReadKey();

        }




Code for Client:

Code
public void ExchangeKeysForClient(StreamReader sr, StreamWriter sw, TcpClient client)
        {
            StreamReader fileStreamReader;
            StreamWriter fileStreamWriter;

            fileStreamReader = new StreamReader("..\\..\\..\\..\\certificates\\rsa-publickey.txt");

            string otherRsaPublicKey = fileStreamReader.ReadToEnd();

            fileStreamReader.Close();

            fileStreamReader = new StreamReader("..\\..\\..\\..\\certificates\\rsa-secretkey.txt");

            string otherRsaSecretKey = fileStreamReader.ReadToEnd();

            fileStreamReader.Close();

            Console.WriteLine(otherRsaPublicKey + Environment.NewLine + otherRsaSecretKey);

            OpenPGP objPGP = new OpenPGP();

  
            bool encStatus = objPGP.EncryptAndSignText("..\\..\\..\\..\\certificates\\roshan-rsa-publickey.txt", "encrypted-other-rsa-publickey.txt", objSelfKeyRing, objOtherKeyRing, "password2");

            //receive encrypted self rsa public key
            string encryptedSelfRsaPublicKey = Encoding.ASCII.GetString(ReadMessage(sr));

            fileStreamReader = new StreamReader("encrypted-other-rsa-publickey.txt");

            //send encrypted other's rsa public key
            SendMessage(sw, Encoding.ASCII.GetBytes(fileStreamReader.ReadToEnd()));

            fileStreamReader.Close();

            fileStreamWriter = new StreamWriter("encrypted-self-rsa-publickey.txt");

            fileStreamWriter.WriteLine(encryptedSelfRsaPublicKey);

            fileStreamWriter.Close();

            bool decStatus = objPGP.DecryptAndVerifyText("encrypted-self-rsa-publickey.txt", "self-rsa-publickey.txt", objOtherKeyRing, objSelfKeyRing, "password for decryption", "self-rsa-publickey.txt");

            Console.ReadKey();

          
        }


Code for Sending and Receiving Data over sockets:

Code

private static void SendMessage(StreamWriter sw, byte[] message)
        {
            string message_string = Convert.ToBase64String(message);
            Console.WriteLine("Sending: " + message_string);
            sw.WriteLine(message_string);
            sw.Flush();
        }
        private static byte[] ReadMessage(StreamReader sr)
        {
            string message = sr.ReadLine();
            Console.WriteLine("Receiving: " + message);
            return Convert.FromBase64String(message);
        }



Code for OpenPGP methods:

Code

public bool EncryptAndSignText(string sourceFile, string destFile, TElPGPKeyring encryptingKeyRing, TElPGPKeyring signingKeyRing, string signingSecretKeyPassword)
        {
            _encryptionStatus = false;
            
            FileStream inputStream, encryptedStream;
            
            FileInfo sourceInfo = new FileInfo(sourceFile);

            var pgpwriter = new TElPGPWriter();

            pgpwriter.Armor = false;

            pgpwriter.ArmorHeaders.Clear();

            pgpwriter.Compress = true;

            pgpwriter.CompressionAlgorithm = 2;

            pgpwriter.CompressionLevel = 9;

            pgpwriter.EncryptingKeys = encryptingKeyRing;

            pgpwriter.SigningKeys = signingKeyRing;

            //use only public key for encyprtion
            pgpwriter.EncryptionType = TSBPGPEncryptionType.etPublicKey;

            pgpwriter.Passphrases.Clear();

            //password for signing via private key
            
            _secretKeyPasswordToSignData = signingSecretKeyPassword;
            pgpwriter.Passphrases.Add(signingSecretKeyPassword);
            

            pgpwriter.OnKeyPassphrase += new SBPGPStreams.TSBPGPKeyPassphraseEvent(providePasswordForEncryption);

            pgpwriter.Filename = sourceInfo.Name;

            pgpwriter.InputIsText = true;

            //TD: experiment with normal and low values
            pgpwriter.Protection = SBPGPConstants.TSBPGPProtectionType.ptHigh;

            pgpwriter.SignBufferingMethod = TSBPGPSignBufferingMethod.sbmRewind;

            //TD: experiment with aes-128 and other encryption types
            pgpwriter.SymmetricKeyAlgorithm = SBPGPConstants.Unit.SB_PGP_ALGORITHM_SK_AES128;

            //TD: experiment with other hash functions
            pgpwriter.HashAlgorithm = SBPGPConstants.Unit.SB_PGP_ALGORITHM_MD_SHA256;

            pgpwriter.Timestamp = DateTime.Now;

            //to be compatible with pgp <= 2.6
            pgpwriter.UseNewFeatures = false;
            pgpwriter.UseOldPackets = false;

            //open a stream to the input file
            inputStream = new FileStream(sourceFile, FileMode.Open);

            encryptedStream = new FileStream(destFile, FileMode.Create);

            try
            {

                //encryt and sign entire file (count = 0)
                pgpwriter.EncryptAndSign(inputStream, encryptedStream, 0);

                _encryptionStatus = true;
                //pgpwriter.Encrypt(inputStream, encryptedStream, 0);
            }

            catch (Exception ex)
            {
                _encryptionStatus = false;
                Console.WriteLine(ex.Message);
            }
          

            finally
            {
                encryptedStream.Close();

                inputStream.Close();
            }

            return _encryptionStatus;
        }




public bool DecryptAndVerifyText(string sourceFile, string destFile, TElPGPKeyring decryptingKeyRing, TElPGPKeyring verificationKeyRing, string decryptingSecretKeyPassword, string decryptedFileName)
        {
            FileStream inputStream;

            _verificationStatus = false;

            var pgpreader = new TElPGPReader();

            
            _secretKeyPasswordToVerifyData = decryptingSecretKeyPassword;
            pgpreader.KeyPassphrase = _secretKeyPasswordToVerifyData;
            _decryptedtFileName = decryptedFileName;

            pgpreader.OnCreateOutputStream += new TSBPGPCreateOutputStreamEvent(saveDecryptedData);
            pgpreader.OnKeyPassphrase += new TSBPGPKeyPassphraseEvent(providePasswordForVerfication);
            pgpreader.OnSignatures += new SBPGPStreams.TSBPGPSignaturesEvent(VerifySignatureValidity);

            pgpreader.DecryptingKeys = decryptingKeyRing;
            pgpreader.VerifyingKeys = verificationKeyRing;

            inputStream = new FileStream(sourceFile, FileMode.Open);

            try
            {
                pgpreader.DecryptAndVerify(inputStream, 0);

                _verificationStatus = true;
            }

            catch (Exception ex)
            {
                _verificationStatus = false;
                Console.WriteLine(ex.Message);
            }

            finally
            {
                inputStream.Close();
            }

            return _verificationStatus;

        }



The error i get is from the lines:

Code

            bool decStatus = objPGP.DecryptAndVerifyText("encrypted-other-rsa-publickey.txt", "other-rsa-publickey.txt", objSelfKeyRing, objOtherKeyRing, "password for encryption", "other-rsa-publickey.txt");


bool decStatus = objPGP.DecryptAndVerifyText("encrypted-self-rsa-publickey.txt", "self-rsa-publickey.txt", objOtherKeyRing, objSelfKeyRing, "password for decryption", "self-rsa-publickey.txt");



1) Which encoding should i use by default for writing and reading encrypted data out of text files?

2) Can i avoid writing and reading to files altogether i.e. can my encrypt and decrypt methods return a string(or use some other stream like MemoryStream)?
#25395
Posted: 06/22/2013 13:47:08
by Eugene Mayevski (EldoS Corp.)

Thank you for detailed description of your problem.

First of all, please get read of StreamReader and StreamWriter class - they cause unnecessary complications in particular with encodings. Use Stream class and its descendants directly.

OpenPGP operates with *binary* data so your encoding-related problems and questions most likely come from StreamReader/StreamWriter.

Next you indeed don't need temporary files and you can use MemoryStream or any other stream descendant.


Sincerely yours
Eugene Mayevski
#25396
Posted: 06/22/2013 14:04:41
by Karan  (Basic support level)
Joined: 06/06/2013
Posts: 10

Thanks for the quick reply Eugene.

Since my codebase is quite large, changing streams will take some time. So i have to ask you, which descendant of the stream class should i use (i mostly have to transfer text and binary data so i need interoperability between them i.e. i should be able to convert from bye array to strings and vice versa easily)?

I gave memorystream a try and i was able to get a byte array as the encrypted output. But i faced the same problem i.e. when i tried converting that byte array into string, i was not sure which encoding to use. So my next question would be - how do i read/write to a memorystream without using StreamReader/Writer and having my strings correctly read?
#25397
Posted: 06/22/2013 14:34:51
by Eugene Mayevski (EldoS Corp.)

I don't understand why you need to deal with Text. Let me explain how PGP works.

1) you pass a stream with data to the encryptor. The stream is treated as having *binary* data.
2) The data is encrypted. Encrypted block is by default binary as well. If you need it to be textual, set Armor property to true - this tells the encryptor to base64-encode the resulting encrypted binary data.
3) the decryptor takes the encrypted block (it can take both binary and "armored" blocks) and decrypts it back to the binary data.

As I understand, you are having problems serializing text to stream on step 1 and deserializing it on step 3. In this case you need to use UTF8 for both steps.


Sincerely yours
Eugene Mayevski
#25398
Posted: 06/22/2013 14:54:28
by Karan  (Basic support level)
Joined: 06/06/2013
Posts: 10

Okay, i understand your point. I'm just hesitating to change streams as i'll have to make 100 changes (and testing on top of it) to get it working. So i was wondering whether a tweak in the encoding area would get it up and running. I think i'll have to take the long route now.

I changed by GetBytes() and GetString() calls to use UTF8. However, by doing this, my DecryptAndVerifyText throws an exception "Unexpected packet".
#25399
Posted: 06/22/2013 15:05:40
by Eugene Mayevski (EldoS Corp.)

If you can reduce your code to some test case which we can compile and run locally, you can post it to HelpDesk and we will try to find what is wrong. I can't help you by just looking into your code because it's a set of code pieces and, you know, it's hard for human to run the code in their heads :).


Sincerely yours
Eugene Mayevski
#25412
Posted: 06/23/2013 10:44:15
by Karan  (Basic support level)
Joined: 06/06/2013
Posts: 10

Thanks for the replies Eugene.

I figured out that by using Default encoding in C# and by using MemoryStream within both EncryptAndSign() and DecryptAndVerify(), i was able to return encrypted and decrypted strings (in the same default encoding) and was able to interoperate between them.

Here is the code i used to convert byte array to string:

Code
string encryptedString = Encoding.Default.GetString(encryptedStringBytes);
Also by EldoS: Callback File System
Create virtual file systems and disks, expose and manage remote data as if they were files on the local disk.

Reply

Statistics

Topic viewed 2433 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!