The OS caches the data read from the file and decrypted by CallbackFilter. How do I prevent this?
File data cache is always used by the OS and it's file system manager. You can't disable it or prevent the data from being placed to the cache. However, this is not a problem at all, and here is why.
Before the application can read any data from the file (and the file is identified by the open handle), it should open this file to obtain a file handle. If the file is not opened by the application, it can not be read by this application, even when another application has opened the file. So if you prevent the application from opening the file, you will also prevent this application from reading the data of this file. The cache itself is protected from access by applications. Cache still can be read from kernel-mode drivers, but if you have a malware kernel-mode driver installed, your system and data integrity is in big trouble no matter whether cache is used or not.
Within Windows the only way to to have the data encrypted until they reach the destination application is have the application itself encrypt or decrypt the data and keep the decrypted data only in non-paged memory.