Direct File Access with CBFS Filter

This article explores the CBFilter component and driver technology named "direct file access". It can be used to get access to files from event handlers or to bypass some OS access checks and restrictions.

When Direct File Access Is Used

Direct file access is utilized in various scenarios, which can be categorized into the following two groups:

1. Regular access to files: This is primarily aimed at bypassing OS checks and filter drivers, such as for backup purposes.
2. "Injected" access to files: In this case, when the OS issues a filesystem request related to a specific file (let's call it file X), an application receives the corresponding event from CBFilter. The application then accesses file X using direct file access within the context of the event handler.

The following scenarios are included in the first group:

• Database Backups: In scenarios in which databases are in constant use, such as SQL Server or Oracle databases, Outlook’s .pst files, which is a backup tool, can leverage Direct File Access to read database files directly without interrupting the database services. This allows for consistent backup even when the database is actively being written to.
• Virtual Machine Snapshots: When taking snapshots of virtual machines (VMs), direct access can be used to read the virtual hard disk files (VHD/VHDX) directly, ensuring that data are captured accurately, even if the VM is running.
• Antivirus Scanning: Antivirus software may need to scan files that are currently in use or locked by other applications. Direct File Access allows the antivirus to read these files for scanning, ensuring that it can detect threats without being blocked by file locks.
• System Monitoring Tools: Tools designed for system performance monitoring may need to access logs and configuration files that are currently being used by other applications. Direct File Access can enable these tools to gather real-time data without interference.

The second group of scenarios involves on-the-fly file modifications (such as encryption/decryption, compression, and filling placeholder files with data) as well as copying file data when the file is modified, renamed, or deleted. Examples of such use cases include the following:

• File System Synchronization: When a file in a synchronized folder (like OneDrive or Dropbox) is modified, the synchronization client can utilize Direct File Access to capture changes in real time, even if the file is in use by another application, ensuring that the latest version is always uploaded.
• Data Encryption Services: Applications that encrypt files on-the-fly can leverage Direct File Access to modify the content of files as they are being accessed, ensuring that the encryption occurs seamlessly without disrupting the user experience.
• File Auditing: In scenarios in whic files are accessed or modified, auditing applications can hook into filesystem events to track changes. By using Direct File Access, these applications can access the file's state before and after modifications, providing a comprehensive audit trail.
• Content Indexing Services: Search applications may need to index content from files currently open in other applications. Direct File Access allows these services to read the content without disrupting the operations of the original application, enhancing search capabilities.
• Placeholder Management in Cloud Storage: Applications that use placeholder files (like those used by cloud storage solutions) can benefit from Direct File Access to fill in the placeholder with actual data as the user accesses the file, ensuring that the data are readily available without requiring a full download.

How Direct Access Works

When an application opens a file using Direct File Access (via the CreateFileDirect and CreateFileDirectAsStream methods of the CBFilter component), the component generates a special mangled name for the file, incorporating the original name along with additional information. It then uses the Win32 API's CreateFile() function to open a file with this mangled name. The CBFilter driver receives the open request and recognizes that it needs to perform a direct file access operation. It decodes the mangled name and opens the requested file using the corresponding kernel-mode function.

The application receives a handle that does not point directly to the file on the filesystem but instead to a "file" managed by the CBFilter driver. Every operation using this handle is directed to CBFilter, which executes the corresponding operation on the file it opened in the filesystem. This secondary operation, however, constitutes a new request sent by the driver down the filter stack to the filesystem, rather than the original request made for the direct handle.

This means that every operation the application may need to perform with the given handle must be explicitly supported by the CBFilter driver. The driver must be capable of creating the corresponding secondary request to the filesystem, sending it, and processing the result appropriately.

Supported Operations

CBFilter offers two ways to utilize the direct handle obtained from CreateFileDirect (or the stream returned by CreateFileDirectAsStream):

• Basic File Operations: Common tasks, such as reading and writing, are supported by handling regular OS requests sent through the ReadFile and WriteFile Windows API functions (or any wrapper that uses these functions internally). This also applies to other operations like GetFileSize and GetFileInformation.

- Special Sperations: Certain tasks require additional preparation or handling. For these, CBFilter provides dedicated methods (refer to the documentation for a list of *Direct methods) that should be used instead of standard WinAPI functions.

Specifics of Use

The use and limitations of the Direct File Access functionality vary depending on whether an application requires regular or "injected" file access.

For regular file access, the application makes a call like this:

filter.CreateFileDirect("filename", /*Synchronize:*/ false, …)

For "injected" file access, the application makes the following call:

filter.CreateFileDirect("filename", /*Synchronize:*/ true, …)

The parameter is named "synchronize" because, in the second case, the driver must synchronize its access to the file with the original OS request to prevent deadlocks. This synchronization occurs by executing the direct request within the thread that initiated the original request to the driver.

Regular file access (when the synchronize parameter is false) is similar to common file operations, but only a subset of file operations is supported, as described in the "Supported Operations" section.

Injected file access is designed for "injecting" the file open operation into a standard file opening initiated by another process. The appropriate context for using CreateFileDirect with Synchronize set to True is within event handlers for BeforeCreateFile, BeforeOpenFile, AfterCreateFile, and AfterOpenFile events. This call should be made for the file associated with the fired event.

Example: If the AfterOpenFile event is triggered for "file.txt," it is correct for the event handler to make the following call:

filter.CreateFileDirect("file.txt", true,...)

However, calling

filter.CreateFileDirect("anotherfile.txt", true,...)

or

filter.CreateFileDirect("file.txt", false,...)

would not be appropriate.

The latter may fail because of the lack of synchronization, which is necessary to avoid deadlocks.

Note that synchronization is unnecessary (i.e., synchronize should not be True) if file isolation is enabled, meaning the Isolate parameter is set to True.

The use of direct handles obtained through injected file access is subject to specific limitations described in detail in the relevant help topic. One significant aspect of this access is that the driver opens the file in nonbuffered mode. In this mode, reading and writing must be performed in blocks that are multiples of 4096 bytes (e.g., 4096, 8192, 12288), which corresponds to the system memory page size on supported architectures.

Closing a Direct Handle

The handle obtained from CreateFileDirect must be closed by calling the CloseHandle Windows API function. The stream returned by CreateFileDirectAsStream should be closed or disposed of according to the rules of the language and framework used in your application.

A direct handle does not need to be closed in the same event handler in which it was opened. The application can store it in the FileContext or HandleContext parameters of the event and retrieve the direct handle in other event handlers related to the same file. This allows the direct handle to remain open while the main handle is still active. The application may even keep the handle open until the BeforeCloseFile, AfterCloseFile, or even the CleanupContext event.

The key difference is that the BeforeCloseFile and AfterCloseFile events are synchronous. Closing a direct handle within these events corresponds to closing the file, making it immediately available to the process using it as well as to other processes that may be waiting for access. Although this can be beneficial, a time cost is associated with firing a user-mode event from the driver. This delay means that until the event handler completes, the original close request cannot finish, preventing the caller process from proceeding.

In contrast, the CleanupContext event is asynchronous. This means the CleanupContext event may occur either before or after the close request returns to the process that opened and closed the file. This indeterminate behavior may be acceptable in some scenarios but less desirable in others, leaving the choice of approach to the developer.