When talking about PKI-based security in the file system, people usually mean something similar to EFS in Windows. However, the question is broader.
PKI stands for Public Key Infrastructure, the set of standards and protocols that let you use asymmetric cryptography (also called public key cryptography) to sign and encrypt the data in order to prove authenticity of the data and to secure the data from unauthorized access.
There are several scenarios when PKI can be used. First use is providing control over user access to the files or the whole disk by encrypting the files or the disk. Next use is is providing authenticity by signing the data which is written and validating the signature when reading the data.
File and Storage encryption
EFS provides only encryption of the files and this encryption is done in a very complicated way. File encryption using certificates in SolFS is quite simple, but requires certain amount of coding, as SolFS itself doesn't have PKI functions.
PKI is based on asymmetric cryptography. Asymmetric (public key based) cryptography uses two keys - public and private (in PGP private key is erroneously called secret). However this type of cryptography is not suitable for encrypting the large amounts of data.
To encrypt the data convenient symmetric algorithms, such as 3DES, AES, Blowfish etc. are used. So in PKI the following approach is utilized: the random key for a symmetric algorithm is created and this key is used to encrypt the data. This key is called session key.
The key for symmetric algorithm is quite short and can be easily encrypted and decrypted using the asymmetric keys. Encryption of the session key is done by the public key, and decryption is done using the corresponding private key. The session key can be encrypted using several public keys, making it possible to provide access to the session key to multiple owners of multiple private keys.
The encrypted session key can be kept together with the data, however keeping it separately reduces chances for the successful cryptographic attack. On the other hand, keeping the key separately complicates key management significantly.
By default SolFS uses AES256-based encryption with SHA256 hashing. The symmetric key is 32 bytes in length and it's derived from the password you set for the file (or the storage) and certain internal data.
With SolFS you can use two different approaches to PKI encryption. First approach is to use SolFS built-in symmetric encryption and create, use and secure the session key in your application. You use the session key as the password for the files or storage. Second approach is more complex: you can implement your own encyrption and hashing procedures and tell SolFS to use them. In this case you decide how strong the key should be and what algorithms must be used for all steps of encryption. This approach is discussed in more details in "Implementing DRM with SolFS" article.
Whatever approach you use, you will need place to store the encrypted session key. SolFS offers you such place. If you encrypt the whole storage, you can keep your encrypted session key and information related to public and private keys (certificates, OpenPGP keys or their IDs) in RootData of the storage. If you use per-file encryption, you can use alternate data stream of the file to keep this information.
Signing the files
Another application for PKI is to ensure inegrity and authenticity of the data using signing algorithms which are public key based. This means that the data came from the source that was supposed to originate this data, and the information has not been altered on the way.
The data is signed using the private key (in fact, the hash of the data is signed, not the data itself). When the data is read, verification takes place. First the hash is calculated and compared with the signed hash, then the signature is verified using the public key. The public key itself is checked for validity before or after hash verification.
The signature can be wrapping or detached. Wrapping signatures include the original data, while detached signatures are separated from the data. In SolFS it makes sense to use detached signatures to avoid signature verification step when it's not required.
Signing and verification of the data are usually atomic operations and the details are hidden in low-level procedures which you rarely call directly.
SolFS provides storage for detached signature: you can store this signature in the alternate data stream of the file. And it's the job of your application to sign the data when or after it has been written, and validate the signature when the data is read (if validation is needed).
The ideas discussed above are not the only possible ways to secure your information. You are welcome to improve the offered schemes and create your own schemes if needed. Remember that the most creative you are, the stronger protection is. Just note, that cryptography itself must be implemented by professionals.
To implement the whole scheme correctly you need certain cryptographic library with PKI support. EldoS Corporation offers SecureBlackbox®, the product that will let you use both symmetric and PKI security with SolFS.