EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Blackbox (C++ Edition for Windows) confuses me !

Also by EldoS: CallbackDisk
Create virtual disks backed by memory or custom location, expose disk images as disks and more.
#33137
Posted: 04/27/2015 03:49:45
by Jens Olesen (Standard support level)
Joined: 11/20/2009
Posts: 7

Hello

I finally made my application build with SBB and the TElZipWriter class - and it crashes when I call the Compress function.

I am afraid I got it all wrong because the methods confuses me - am I missing something fundamental ?
- Why doesn't the sample app 'consolezip.cpp' include code that actually calls the 'SetLicenseKey' ?
- Why doesn't the 'CreateArchive' function take a fully qualified filename as parameter (path+name of the zip-file) ?

The few lines of code that interacts with SBB is listed here (This code will be called many many times to do some archiving stuff):

Code
   SetLicenseKey("---");

   TElZipWriter wrt(NULL);
   
    wrt.set_StrongEncryption(false);
   wrt.set_WinZipEncryption(false);
   wrt.set_CompressionAlgorithm (SB_ZIP_COMPRESSION_REDUCE_1);
   wrt.set_CompressionLevel (1); // 1 is lowest, 9 is highest
   wrt.set_CaseSensitiveFilenames (false);
   //wrt.set_Name (zip_file_name); // What's this ??
   wrt.CreateArchive ();
   //wrt.Open(zip_file_name, false);
   
   wrt.Add (src_fnam);
   wrt.Compress (zip_file_name); // Crash here.
   wrt.Close();


Thanks
#33139
Posted: 04/27/2015 04:35:22
by Ken Ivanov (EldoS Corp.)

Hi Jens,

Thank you for contacting us.

We can't put the license key in the samples' code because they expire with time. Besides, it's one of unobtrusive ways to let the users know they do need to set the license keys in their code (as the samples won't compile without the license key being set).

Quote
Why doesn't the 'CreateArchive' function take a fully qualified filename as parameter (path+name of the zip-file) ?

Our ZIP components let the user choose the name of the output file on the compression stage and not on the archive creation stage. That's just a component design specific.

Regarding the crash, did you try running the ConsoleZIP sample and checking whether it exposes the same problem?

If it does, what exactly crash is that? Is it an AV, some other exception or the application just shuts down for you?

Ken
#33140
Posted: 04/27/2015 04:38:02
by Dmytro Bogatskyy (EldoS Corp.)

Thank you for contacting us.
Quote
- Why doesn't the sample app 'consolezip.cpp' include code that actually calls the 'SetLicenseKey' ?

What SecureBlackbox version are you testing?
I have checked the 'ConsoleZIP' sample from the last version and it has a SetLicenseKey() method call with notes.
Quote
Why doesn't the 'CreateArchive' function take a fully qualified filename as parameter (path+name of the zip-file) ?

You can specify either archive name (e.g. 'archive.zip') or absolute path (e.g. 'c:\temp\archive.zip') on Compress() method call.
Quote
The few lines of code that interacts with SBB is listed here (This code will be called many many times to do some archiving stuff):

Please put your code in try..catch block, for example:
Code
         try
         {
// your code
         }
         catch (SBException E)
         {
            std::cout << "Compression error : " << E.what() << std::endl;
         }


Then, you should catch exception with a message 'Unsupported compression algorithm'.
At the moment SB_ZIP_COMPRESSION_REDUCE_1 compression algorithm is not supported. Only SB_ZIP_COMPRESSION_DEFLATE, SB_ZIP_COMPRESSION_DEFLATE64, SB_ZIP_COMPRESSION_STORED, SB_ZIP_COMPRESSION_BZIP2 compression algorithms are supported.
#33142
Posted: 04/27/2015 05:56:58
by Jens Olesen (Standard support level)
Joined: 11/20/2009
Posts: 7

Thanks both for helping !

I downloaded the toolkit a few days ago.
File changes.txt says:
April 9, 2015
Version 12.0.269. Maintenance update


Sorry - I found the SetLicenseKey() - By bad !

However I am still having problems. I succeeded in creating a new archive, but now it fails (File already exists) when I call Compress on an existing archive.
I call Open(), Add() and Compress().
The archive already exists, but the file to add was never added before.

Have a look at my code: This function will be called again and again, putting many files into a few archives.
Is the logic / sequence of calls correct ?

Code
try {
   SetLicenseKey("B05A...38CC");

   TElZipWriter wrt(NULL);
   
    wrt.set_StrongEncryption(false);
   wrt.set_WinZipEncryption(false);
   wrt.set_CompressionAlgorithm (SB_ZIP_COMPRESSION_DEFLATE);
   wrt.set_CompressionLevel (1); // 1 is lowest, 9 is highest
   wrt.set_CaseSensitiveFilenames (false);
   //wrt.set_Name (zip_file_name); // What's this ??
   if(true == ExistFile(zip_file_name))
      wrt.Open(zip_file_name, false);
   else {
     wrt.CreateArchive ();
   }
   wrt.Add (src_fnam);
   wrt.Compress (zip_file_name);
   wrt.Close();
   }
   catch (SBException E)
   {
      TLog(__FILE__, __LINE__, LOG_TYPE_ERROR, "SBB Exception: %s", E.getErrorMessage ().c_str());
   }


Thanks again !
#33143
Posted: 04/27/2015 06:27:35
by Vsevolod Ievgiienko (EldoS Corp.)

Please try to call Compress method without parameters. This should help.
#33146
Posted: 04/27/2015 08:45:34
by Jens Olesen (Standard support level)
Joined: 11/20/2009
Posts: 7

Thanks - that helped !

I think your documentation could be improved a lot - but I made it work using the code below.

About license:
I want to test my code at a customer-site before I decide to buy the toolkit. It seems to work with the trial-license.
For how long will this trial license work ?
How will I see if / when it expires ?

Code
int AddFileToZIPArchive(const char * src_fnam, const char * zip_file_name)
{
   try {
       SetLicenseKey("B05A...38CC");
     TElZipWriter wrt(NULL);
   
      wrt.set_StrongEncryption(false);
     wrt.set_WinZipEncryption(false);
     wrt.set_CompressionAlgorithm (SB_ZIP_COMPRESSION_DEFLATE);
     wrt.set_CompressionLevel (1); // 1 is lowest, 9 is highest
     wrt.set_CaseSensitiveFilenames (false);
     //wrt.set_Name (zip_file_name); // What's this ??
     if(true == ExistFile(zip_file_name)) {
       wrt.Open(zip_file_name, false);
       wrt.Add (src_fnam);
       wrt.Compress();
     }
     else {
       wrt.CreateArchive ();
       wrt.Add (src_fnam);
       wrt.Compress(zip_file_name);
     }
     wrt.Close();
     PLOG(__FILE__, __LINE__, LOG_TYPE_INFO, "Success adding file <%s> to archive <%s>", src_fnam, zip_file_name);  
     return(0); // Success
   }
   catch (SBException E)
   {
      char buf[10*1024];
      sprintf(buf, "SBB Exception: [%s]. Failure during add of file <%s> to archive <%s>",  
                  E.getErrorMessage().c_str(),
                src_fnam,
                zip_file_name);
      PLOG(__FILE__, __LINE__, LOG_TYPE_ERROR, buf);

      return(1); // FAIL
   }
   catch(...) {
      PLOG(__FILE__, __LINE__, LOG_TYPE_ERROR, "Unknown Exception. File:<%s>, Archive:<%s>", src_fnam, zip_file_name);
      return(1);
   }
}
#33147
Posted: 04/27/2015 08:48:43
by Vsevolod Ievgiienko (EldoS Corp.)

Quote
For how long will this trial license work ?

The key expiration date can be found in LicenseKey.txt file from SBB distributive.

Quote
How will I see if / when it expires ?

The application will stop working and will throw corresponding exception.
#33323
Posted: 05/19/2015 10:02:37
by Jens Olesen (Standard support level)
Joined: 11/20/2009
Posts: 7

Now we have tested my application for some weeks, and we experience some problems:
1) When adding many (!!) files to an archive the compression gets slower and slower.
The application is executing directly on a physical file server and it is supposed to add 1,5 million (1,5 * 10^6) files to a single archive on a local (local to server) drive. At the moment it takes about 3 seconds for every file. Every file contains text (.csv / .xml) and a typical file is about 1500 bytes in size.
This means that it will take many days - maybe months before all the files are added the archive.
2) We have seen that some archive files are corrupted and cannot be used at all.
Our intention was to add all files in a folder to an archive and delete the files afterwards - but this is risky if the archive is corrupted.

Do you have any comments on this ?

Thanks
#33325
Posted: 05/19/2015 10:11:55
by Eugene Mayevski (EldoS Corp.)

Quote
Jens Olesen wrote:
When adding many (!!) files to an archive the compression gets slower and slower.


This is sort of expected. Did you check ZIP utility under the same condition? Packing or unpacking millions of files will take significant time there as well.

However it would be nice to get a dozen of sample files which you put to the archive so that we could test the issue ourselves. You can pass the files via the HelpDesk ( https://www.eldos.com/helpdesk/ ).

Quote
Jens Olesen wrote:
2) We have seen that some archive files are corrupted and cannot be used at all.


This needs reproducing. Without a reproducible test case (a set of source files + the corrupted archive) it's not possible to say anything about the problem.


Sincerely yours
Eugene Mayevski
#33326
Posted: 05/19/2015 10:34:30
by Jens Olesen (Standard support level)
Joined: 11/20/2009
Posts: 7

No, we haven't tested std. zip utillity.

We will accept to add file to archive with no compression at all - just to reduce the filecount in the folder from millions of files to 1 file. Is this possible ?
At the moment we use the 'SB_ZIP_COMPRESSION_DEFLATE' compressionalgorithm.

If the application is executing while the server is restarted: Could this cause corruption of the archive (When executing for many days server restart is likely to happen) ?

Thanks
Also by EldoS: Callback File System
Create virtual file systems and disks, expose and manage remote data as if they were files on the local disk.

Reply

Statistics

Topic viewed 1412 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!