Tamed FileSystemWatcher

[Für Details siehe meinen kommenden Artikel “Gezähmte Beobachter” im Windows Developer Magazin.]

[July 11, 2015: Fixed bugs in code. See changelog in readme.txt]

This post shares robust wrappers around the standard FileSystemWatcher (FSW)  fixing problems commonly encountered when using it to monitor the file system in real-world applications.


Buffering and Recovering FSW

Simply replace the standard FSW with my BufferingFileSystemWatcher and you no longer need to worry about InternalBufferOverflowExceptions. Use my RecoveringFileSystemWatcher to automatically recover from typical transient watch path accessibility problems. Download complete code. For a file system watcher using polling instead of file system events see my FilePoller. To process files detected in either way I recommend using TPL DataFlow ActionBlocks. They allow you to easily process files without having to spawn a Thread or create a Task yourself and allow to configure the degree of parallelism desired. For tips about handling lots of files and using contig.exe to defragment NTFS indexes see NTFS performance and large volumes of files and directories.

Typical FileSystemWatcher Problems

If used properly the standard FileSystemWatcher (FSW) is way better than its reputation. However, there are typical problems one may encounter when first using the FSW:

  • Unexpected events.
  • Lost events.
  • InternalBufferOverflowExceptions.
  • No option to report files existing before the FSW started.

The standard FileSystemWatcher:

  • Reports exceptions via its Error event. Not via raising exceptions!
  • Does not report files that existed before .EnableRaisingEvents =True.
  • Does detect network disruptions, but does not automatically recover from them.
  • Does automatically handle renames of its watch path.

Continue reading

Posted in Computers and Internet | Tagged , , | 2 Comments

How to completely clean install Windows 10 on a UEFI PC

With Windows 10 I generally do recommend to simply upgrade which preserves your apps (modern and desktop), data and settings. In contrast to previous Windows versions the Win 10 upgrade is robust and has worked fine for me on several machines, so I consider a clean install not being worth the time needed.

With my Surface Pro 3 (SP3) however I wanted to do a clean install, including removing Windows 8.1 partitions no longer needed. A Win 10 upgrade from Win 8.1 left my Surface with the partitions shown in the picture below. I could not find any reliable information on which partitions are safe to delete.


Partitions after upgrading from Win 8.1 to Win 10

A clean install as described below removed 2 unneeded partitions and freed 5+ GB SSD space:


Partitions after Win 10 clean install

Continue reading

Posted in Computers and Internet | Tagged , | Leave a comment

Windows Service Worker Options

[Für Details siehe meinen kommenden Artikel “Windows-Services leicht gemacht” im dotnetpro Magazin.]

Implementation patterns used for Windows Services can seriously influence performance, reactivity, computing resource consumption, stability and energy consumption of our systems. This post shares alternative patterns for creating Windows Services. Download complete code.

Design Aspects

When designing Windows Services consider the following aspects:

  • How and when should the service be started?
  • If OnStart() takes > 30 sec the SCM will abort the service.
  • Choose a suitable pattern to implement processing. Ex: Event-driven, polling loops, poll-timers.
  • Maintain thread hygiene.
  • If OnStop() takes >~90 sec the SCM will abort the service. Exact time limit is undocumented and Windows version specific.
  • Implement graceful shutdown.
  • Windows Services must not have a direct UI. You can however create a UI and control the service via its OnCustomCommand method or implement a custom WCF-Interface, see Simple WCF-Services.
  • Robust error handling.
  • Sufficient tracing and logging.
  • Windows Services must be installed.
  • Windows Services cannot be started directly in the IDE. See test console app below.

Continue reading

Posted in Computers and Internet | Tagged | Leave a comment

Simple WCF

[Für Details siehe meinen kommenden Artikel “WCF-Services leicht gemacht” im dotnetpro Magazin.]

Choosing WCF options and configuring them can be quite cumbersome and demanding. This post shares a very simple approach for using WCF services. Download complete code. My simple approach uses:

  • Shared Interface-DLL
  • Self-hosting in a Windows Service.
    See Windows Service Worker Options.
  • Channel proxies instead of ClientBase proxies.
  • Simplified WCF configuration.
  • Custom method CreateChannel().
    Caching ChannelFactories.
  • Custom method CloseChannel().
    Handling Channel failures.

I am using a new Channel instance for each WCF call. This improves robustness against communication problems and against “The socket connection was aborted” exceptions after a client idle time > 10 min (= default ReceiveTimeout) with netTcpBinding even when using InstanceContextMode=PerCall. Continue reading

Posted in Computers and Internet | Tagged , , , | Leave a comment

My Windows 8.1 upgrade and usage experiences

This post describes my experiences with upgrading from Win7 to Win81 and using it for a few days. I hope this will help others to avoid some problems and make them aware of (silently) missing functionality in Win81. I posted this information for discussion in the Windows 8.1 IT Pro forum too.

Trying to do a backup in Win7 before the upgrade resulted in error “not enough disk space to create shadow copy.” This was caused be a test install of Win81 Preview as dual-boot which made a change to the common hidden System Reserved partition which did break Win7 Backup.
I fixed this using fsutil usn deletejournal /N /D F: as suggested here.

To keep my Win7 settings and desktop applications I did first upgrade to Win8 because MS removed keeping these with the Win81 upgrade.

The first upgrade rolled back with errors. Digging through the hidden error log C:\$Windows.~BT\Sources\Rollback\setupact.err revealed some drivers belonging to VMWare and DAEMON Tools causing problems. After uninstalling VMWare and DAEMON the upgrades worked fine. I remember some sort of compatibility check with previous test installs of Win8 and do not know if they were not executed during this upgrade or do not detect VMware and DAEMON generally.

Win 8.1 does (in contrast to Win7 and Win8) not(!) include a solution for a complete file based backup.

  • Image backup is still available, but difficult to find: A Link on the bottom left of “Control Panel\All Control Panel Items\File History”, or you can use wbAdmin on a command line.
  • Windows 7 Files Backup was removed in Win 8.1.
  • Win81 File history does (in contrast to Win8) not backup SkyDrive, it does still backup SkyDrive Pro though.
    Remember: syncing to the cloud is no replacement for a backup.

File History silently(!) skips EFS encrypted files. The event log shows the following “funny” error “…If you want it to be protected, remove encryption …”. Silently skipping the most important files is totally unacceptable, there should at least be a warning in action center.

With these limitations of File History sadly we are forced to use 3rd party backup solutions.
As feared my first try, using Acronis True Image 2014, was ugly. TI does not officially support Win81 yet (one of the problems why I prefer using built-in tools for basic tasks like backup). While TI worked fine with EFS and SkyDrive a reboot did hang for a night with the message “…operations pending…”. This is a known Acronis problem maybe not even related to Win81. Because the machine at this stage of shutdown did not allow me to remote login to kill the Acronis processes I was forced to do a hard reset – yuck.

SkyDrive and SkyDrive Pro do not support EFS encrypted files. The explanations for this are questionable at least. Why can’t SkyDrive just do a binary copy when syncing (like robocopy \efsraw does)? And yes, if one would not have the EFS certificate installed on each machine or even loose it the files would not be readable – security does not come for free. Remember: Private files should only be stored in the cloud when encrypted with a key controlled by the user only (transport encryption and encryption in the cloud is not sufficient). Not supporting EFS or offering an alternative solution forces us to trust yet another party like BoxCryptor. With SkyDrive being tightly integrated into Win81 and MS urging us to use it user controlled encryption is mandatory and no longer just nice to have.

There are reports of people having their SkyDrive account locked by Microsoft because the content or even the file names only did violate Microsoft’s very wide code of conduct. This happened with files that were private only, not shared with others.

The search charm is too slow, see post.
With Win7 / Start Menu or Win81 / Classic Shell searches for applications are instantaneous.

The reminder popups of the Mail app obscure the search box when searching right after logging in.

Unlocking BitLocker drives with a right-click form File Explorer does not work. Error “Application not found”.
The Fix it did not solve this for me. Editing the registry manually worked.

The All Apps view of the Start Menu is unusable for me.
I my case it shows a desert with hundreds of icons. Such a view is good for a folder with photo thumbnails but not for a list of apps. The icons are only distracting without helping to find something. The grouping titles are too pale. We need a filterable text-only view here. I just discovered a helpful but difficult to discover view of the all Apps view. In Apps view click on the minus in the bottom right corner an the following group view shows:


My Officejet Pro 8500 no longer supports scanning. No fix from HP available.

NUMLOCK is always disabled after startup. Switching off “fast startup” as suggested in the forums is not a viable solution. The registry change suggestions I found do not solve this problem.

The desktop task bar was annoyingly transparent. The only way I was able to change this was via Classic Shell settings. I hope this change persists if I decide to uninstall Classic Shell.

After playing with the new navigation setting “Show my desktop background on Start” the desktop icon labels showed with drop shadows which looks very ugly with solid color backgrounds. Had to edit the registry to get rid of this. Removing the drop shadows only works with”Show my desktop background on Start” disabled. The problem still surfaces from time to time and logging off and on again fixes it.

Some Windows updates (KB2887595, KB2837642) did hang causing countless reboots and forcing me to hard reset the machine. I never managed to install the cumulative update KB2887595.

Windows Update leaks disk space. After using Win 8.1 for some days my system SSD was full. 40+ GB free space had disappeared. This was solved by the Windows update rollbacks caused by the update hangs mentioned above.

Hyper-V seems to leak memory. Even after shutting down all VMs and applications Win 8.1 still uses nearly all of my 8GB RAM. Only a reboot frees it.

As a Windows Phone developer one is forced to migrate to Win8 because the phone emulator requires Hyper-V.
Hyper-V (needlessly?) requires the special hardware visualization feature SLAT, forcing me the abandon otherwise perfectly good notebooks.

VMWare does not install on a machine with Hyper-V enabled. This forces me to move from VMWare to Hyper-V on all machines sharing VM Images. Hyper-V is cumbersome to use: It is overly complicated to copy or move VMs. Without clicking on settings one cannot see where the files reside. The UI does not have end-user quality. Setting to share USB and other host devices difficult to discover, sometimes simply not reachable.

Copying passwords, credit card infos etc. from my password manager 1Password to Modern Apps is cumbersome.

Things I like in Win 8.1:

  • Works fine on my old Thinkpad X61s notebook
  • Improved file management
    • Quicker delete, especially when bypassing recycle bin using <Shift>.
    • Copying EFS files to remote folders is much quicker than in Win 7.
    • More robust external drive handling. Win 8.1 still does not allow to eject USB sticks sporadically.
    • Less problems with “file already exists” when copying large amounts of files.
Posted in Computers and Internet | Tagged , | 3 Comments

How to move from Hotmail and Gmail to Office 365 as a Server for Outlook desktop and Windows Phone 8

When moving to Office 365 I found it difficult to stop my Windows Phone syncing its calendar and contacts with Hotmail. Here is how I finally did it.

For the reasons I was using Hotmail and Gmail see Synchronize Windows Phone with Outlook Emails, Contacts, Calendar and Tasks.

Hotmail was the primary Windows account on my phone and Windows Phone does not allow changing the primary account without a hard reset (loosing some local information).
The phone’s settings under “Email+Account” for the primary account only allow to select if emails is synced. There are no checkboxes for calendar or contacts:


Configuring Windows Phone to use Office 365 instead of Hotmail

To deselect syncing the Hotmail calendar use the Calendar app settings:


Selecting “Show to-dos…” syncs Office 365 Tasks with the phone’s calendar.

To deselect syncing Hotmail contacts use the phone’s People app setting “filter my contact list”:


Copying contacts, emails and tasks via copy&paste

During the migration process I had to move calendar entries, tasks and many (5000+) emails from one Server to another. I did this by configuring all server accounts in a Outlook desktop app and simply using copy&paste to copy the items.

With the calendar items and tasks I first filtered for the ones I wanted (Ex. NOT category:holiday) an then used CTRL-A to select all, CTRL-C to copy them and CTRL-V to paste them. There is no right-click menu item for pasting in Outlook. With some email folders I simply did a CTRL-A to select all messages and copied them.

The Outlook desktop app automatically synced the copied items to the Office 365 server. Even after the status bar said all folders were up to date and the status message “Updating sent items” no longer showed, the number of messages was not correct for a while. After several hours all messages were synced and my desktop and notebook Outlook showed the same number of items in all folders.

Posted in Computers and Internet | Tagged , , | Leave a comment

Installing Windows 8 on an old X61s ThinkPad

[11 Nov 2012: added Bluetooth support]
[25 Feb 2013: Add warning against installing the newer ThinkPad Hotkey Features Integration driver for Win8]

Windows 8 installed fine on my old ThinkPad X61s and runs nicely and fast (the little old thing has an SSD though :-)).


First I did an upgrade install by selecting “Keep Windows settings, personal files and applications”.

  • I enabled Bluetooth before starting the upgrade because I read somewhere that it could not be  enabled in Win8 otherwise.
  • Win8 Setup asked to delete some incompatible applications (nothing I currently need), including MS Security Essentials.
  • Most of these applications could be uninstalled from within Win8 Setup by simply clicking a button.
  • A few I had to manually uninstall using “Programs and Features”.
  • Because I could not find a way to uninstall “Lenovo Auto Scroll Utility”, I uninstalled the ThinkPad TrackPoint driver. That stopped Win8 Setup complaining and it installed.
  • I had to manually install my EFS certificates.
  • I deleted the windows.old backup using the “Free up disk space…” to free disk space.

After these steps everything worked fine including the Fn keys. Only the battery manager reported to be not compatible


Experimenting with Win8 I successfully did a Refresh:

  • Refresh took quite long (1+ h, hanging at 5% for about 30 min).
  • It removed all desktop application like Office, VS, SQL Sever, …
    And placed a RemovedApps.html file on the desktop.
  • It removed all ThinkPad specific drivers.
  • After deleting the windows.old backup using the “Free up disk space…” App (selection System files) there was a lot of free disk space.
  • After Refresh the X61 worked fine,
    including the TrackPoint and its buttons, the speaker buttons and the wireless switch.
  • Device Manager showed the device “Video Controller” not working.
    And connecting external monitors or beamers  via Win+P was not supported.
  • I installed Intel Chipset Support for Windows 8.
  • After this Window Update installed the chipset drivers.
  • Device Manager “search automatically” now found a driver for the Video Controller.
    Win+P now offers to connect external displays.
  • Bluetooth did not work yet.
    Because I wanted to use a ThinkPad laser mouse without an extra Bluetooth dongle I decided to get Bluetooth working. Because Lenovo did not offer Win8 hotkey drivers I installed the X61s Hotkey Features Integration for Windows 7.
    I set the compatibility of the setup.exe to Win7 and deselected the Lenovo Auto Scroll Utility” because Win8 reported compatibility issues with this. Win8 automatically installed Bluetooth with enhanced Data Rate. With FN+F5 working now I can enable Bluetooth my laser mouse works fine.


  • Win+”.” to split the Start Screen does not work.
    This is by Win8 design because of the low 1024 screen resolution of the X61s.
  • Installing the newer ThinkPad Hotkey Features Integration driver for Win8  3.84.1500 which was offered by Lenovo System Update did disable the hardware WiFi switch. I to revert to the older Win7 driver mentioned above by restoring to an older restore point using system restore.
Posted in Computers and Internet | Tagged , | 24 Comments