Writing Shell Namespace Extensions

CBFS Shell is a software toolkit which simplifies the process of writing Windows Shell Namespace Extensions, and enables the use of .NET managed code.

Windows Shell Namespace Extensions are the mechanism provided to developers to extend and customize the Windows Explorer Shell. Technically, a Windows Shell Namespace Extension is an in-process extension. This .dll file is also a COM (Component Object Model) Server that is configured in the Registry, which enables Explorer to find it and load it in its own address space. Other extensions to the Shell can be written as out-of-process extensions, like preview handlers or context menus, but a Shell Namespace Extension always runs in-process.

A primary reason CBFS Shell exists is to address Microsoft's recommendation against writing in-process extensions using .NET. Microsoft does not support such a scenario and clearly states this in the Guidance for Implementing In-Process Extensions article, which is partially quoted here:

In-process extensions are loaded into any processes that trigger them. For example, a Shell namespace extension can be loaded into any process which accesses the Shell namespace either directly or indirectly. The Shell namespace is used by many Shell operations, such as the display of a common file dialog, the launch of a document through its associated application, or the obtaining of the icon used to represent a file. Because in-process extensions can be loaded into arbitrary processes, care must be taken that they do not negatively impact the host application or other in-process extensions.

One runtime of particular note is the common language runtime (CLR), also known as managed code or the .NET Framework. Microsoft recommends against writing managed in-process extensions to Windows Explorer or Windows Internet Explorer and does not consider them a supported scenario.

Writing a Shell In-Process Extension Is Challenging

Writing this type of extension requires you to write in-process extensions with another language, which is often C or C++. Although C++ is one of the most widely used programming languages in the world, the language is still more difficult to grasp for a majority of programmers than C# or VB.NET. Productivity is a key issue. And this issue is not only the language itself but also all of the important technologies, some of which are rather low-level, that one must master:

• Windows programming: You'll have to be a good C++ programmer to know your way around the Windows SDK, headers, and macros, which are mandatory to use.
• COM programming: On top of Windows programming, you'll also need to understand how to write low-level COM objects (again, without .NET).
• Shell programming: The Shell is a whole other world. Lots of COM interfaces need to be implemented, many of which are not fully documented. It may take a long time to understand the relationships among the various components of the system.

Writing a Shell In-Process Extension Is Tedious

Writing an in-process extension to the Shell presents some specific problems:

• Supporting 32- and 64-bit operating systems: Because Windows ships in 32- or 64-bit versions, Explorer may need one version or another. There is a good chance that you will need to provide two binaries for your extension (or three binaries if you also deploy your solution to ARM64 systems). CBFS Shell eliminates that issue. Instead, you only have to write one .NET executable file that can be compiled as "Any CPU".
• Restarting Explorer: You will have to restart the Explorer process (or processes) and all processes that load your extension in-process (all processes that use the Common Dialogs, Open, Save, and others) all the time, especially when you want to compile a new version (otherwise the .dll binary will remain locked by the system). With CBFS Shell and its unique out-of-process architecture, you can start and stop your server processes. As soon as the server becomes available, Explorer and other processes don't have to be restarted, because RPC cross-process communication will resume automatically without any timeout.
• Restarting all Common Dialog client apps: You will have to restart Explorer processes, but for the same reasons, you also will have to restart all processes that have used the Common Dialog (e.g., Notepad, Office apps) and that have loaded your extension in their process. This can be very painful during development times.
• Deployment is difficult: In-process Shell Namespace Extensions are loaded in explorer.exe processes but also in all Common Dialog client apps processes, which means nearly all applications using files on a machine. When you want to update your binaries, you probably will have to restart the machine. With the CBFS Shell architecture, you only have to stop and restart your updated .NET application.

Writing a Shell In-Process Extension Is Risky

As the name implies, an in-process extension is loaded in-process with explorer.exe processes and all Common Dialog client apps processes. It is not rare that a bug, in such a crucial binary, crashes its host process. The CBFS Shell out-of-process architecture prevents bugs in your code from crashing end user's vital applications.

You Need an Extension but Don't Want to Invest Much in That Technology

Writing a Shell Namespace Extension is a complex and involved procedure. Investing the time and resources required to create one may not make sense from a business perspective.