EldoS | Feel safer!

Software components for data protection, secure storage and transfer

Small file transmission efficiency is low

Also by EldoS: Solid File System
A virtual file system that offers a feature-rich storage for application documents and data with built-in compression and encryption.
#37000
Posted: 06/14/2016 08:33:53
by xin han (Standard support level)
Joined: 11/11/2014
Posts: 2

We have tested our server performance of small file, not found any performance bottlenecks. and we found that it's too low to copy a directory that contains many small files to the cbfs-vdisk. and the concurrency is very low. do you have any idea to solve this problem, Enhance the small file copy speed .thank you !
#37002
Posted: 06/14/2016 11:18:26
by Eugene Mayevski (EldoS Corp.)

Thank you for contacting us.

This question is asked frequently and we have an explanation on https://www.eldos.com/cbfs/articles/7979.php . What you can try is play with the file cache (enable or disable it) and its size - maybe the speed can be made better by these manipulations.


Sincerely yours
Eugene Mayevski
#37003
Posted: 06/14/2016 21:26:04
by xin han (Standard support level)
Joined: 11/11/2014
Posts: 2

thank you for your reply
We found that when we copy a new file has three opening and closing operations, which led to our operations related to opening and closing are executed a total of six times,how to deal with this situation? File cache maybe speed up single file transfer, but how to accelerate concurrent multiple file it?
This performance is urgent and important for us!
#37004
Posted: 06/15/2016 02:40:20
by William Levra-Juillet (Priority Standard support level)
Joined: 09/05/2013
Posts: 49

Hi,

For what it's worth Explorer opens the file(s)/directorie(s) many time with access mode SYNCHRONIZE only. In that case we do nothing in our callback and return a fake/dummy "handle/UserCOntext". And the "close file" with that "user context"is also ignored.
You can also implement your own "level2" caching. Like caching the file status for xxx ms, as GetFileInformation is called many times on the same file for whatever reasons.
When a file/directory is open with "READ_ATTRIBUTES" only, we search in our cache, and if found, return a fake/dummy handle as well.

Basically we try to access our internal data (over the network) only when it's absolutely necessary.

HTH

Reply

Statistics

Topic viewed 1017 times

Number of guests: 1, registered members: 0, in total hidden: 0




|

Back to top

As of July 15, 2016 EldoS Corporation will operate as a division of /n software inc. For more information, please read the announcement.

Got it!