What you need to do is handle file creation and opening requests (add callback rules for OnOpenFile and OnCreateFile). In the corresponding callbacks / event handlers your code needs to call GetOriginatorToken or other GetOriginator* method of CallbackFilter to obtain information about the calling process. With this information you can perform any checks you like, be it process file integrity check or anything else.
If the process should not access the file, you need to decline the request by setting ProcessRequest parameter to false or throwing ECbFltError with error code 5 (Access Denied).
The same checks can be implemented in directory enumeration and file information retrieval callbacks (OnEnumerateDirectoryC and OnGetFileInfoC).
For information about integrity checking of the caller process see this question.