Thoughts and Experiments on Cloud Encryption

[ Dies ist ein Auszug aus meinem Artikel “Wolkentresor” im dotnetpro Magazin 9.2015 mit einigen ergänzenden Abbildungen ]

Private files should only be stored in the cloud being encrypted via a key controlled and known by the user solely (= end-to-end encryption). Transport encryption and encryption in the cloud provided by storage providers are not sufficient here.

Attack Surface

The safety of encryption depends on good passwords. Passwords are good if they are not only strong but also easy to use. For an interesting discussion about better passwords see Toward Better Master Passwords.

Some cloud storage providers like Tresorit support automatic end-to-end encryption out of the box. For an overview and reviews of cloud storage providers and VPN services see Cloudwards. Many widely used services like DropBox and the deeply in Windows integrated OneDrive are sadly lacking this support. For these one can use additional encryption tools like:

boxcryptor: Good commercial tool.
CryptSync: Free open source tool based on 7-zip encryption.
Allows decrypting files even without CrypSync using 7-zip apps.

File Processor Experiment (TPL, async, parallel, MEF)

Many of my enterprise apps use files as interfaces between systems.

This large code example demonstrates important patterns I use and some new ideas. It uses TPL Dataflow blocks, async and parallelism for simplicity and to optimally utilize CPU and filesystem resources. The processing logic is encapsulated via MEF.

It contains a handy function RetryOnFileInUse() which one typically needs for file processing.
This can be refined using an exponential back-off as described in my blog post DB Concurrency Control with .NET – Details

Download code

See also:
– Tamed FileSystemWatcher
– A WebCrawler demonstrating the Beauty of TPL Dataflow

Global Error Handling for UWP-Apps

[ Für Details zur Fehlerbehandlung in .NET im Allgemeinen, unter Berücksichtigung von von multi-threaded Programmierung (insbesondere mit Task und Await)  siehe meinen Artikel „Lass da mal was sein“ im dotnetpro Magazin 3/2013.
Für Details zur Fehlerbehandlung in UWP-Apps siehe meinen Artikel „Was war da los?“ im dotnetpro Magazin 12/2016. ]

This post is the result of my experiments with handling unexpected errors in UWP-Apps.
For my approach about tracing see Tracing for UWP-Apps

It is debatable if showing information about unexpected exceptions to users is good idea. I have had Mail, Translator, Calculator, the Store app and other UWP-Apps abort without any feedback on crash or at restart. I personally find it annoying when apps simply disappear without any hint. The „little  Watson“ solution of displaying error info on the next app start feels weird to me. Both behaviors doe not conform with common user expectations.

If crash error details can be helpful for end users depends on the app type and the target audience. For enterprise apps I know from long experience that error notifications (even with deep technical infos) are definitely helpful. Users are often able to circumvent problems and keep working with defective apps without even contacting support (while support might have been made aware of the problem automatically in the background…).

Options to inform users about unexpected exceptions are:

• Never notify users about exceptions.
• Show dialog in App.UnhandledException
• Show dialog on next app start.
• Show exception details like Message and HResult
• Show „Sorry…“ only.

This following code sample implements the main options, configurable via _notifyUserAboutCrash.

Tracing for UWP-Apps

There are several tools available for tracing in UWP Apps:

• MetroLog
• PortiLog
• Event Tracing for Windows (ETW)
  via Diagnostics.FileLoggingSession

I wanted to use ETW via Diagnostics.FileLoggingSession but it proved to be unstable and there was no way to convert the etl files to human readable form to allow app users to examine the content before sending them as email attachments.

My main requirements for tracing tools are:

• Tool errors must never crash the app.
• Minimal impact on app performance.
• Ability to specify maximum file storage usage.
• Inject trace location (method name, line number).
• Unobtrusive trace calls that do not obscure the app code.

I make a distinction between tracing the program flow and variable values for developer usage and logging unexpected exceptions and important status messages for system management.

None of the tools available fulfilled all these requirement to my liking. Thus I wrote a little class Trace with the following features:

• Stores traces in alternating text files in ApplicationData.
• Short static trace methods.
• Trace levels Verbose, Debug, Info, Time, Status, Warning, Error, Fatal, Off
  The level „Status“ is intended to filter messages for logging errors and important status messages like started and stopped. This level is missing in many tracing frameworks and leads to either flooding logs intended for system management with info messages or not logging important status messages.
• Inject trace location via compiler generated parameter values like CallerMemberName and CallerLineNumber. These point to the correct source code location even in release. mode (when the app was compiled with the .NET Native toolchain.
• Buffer traces in a TPL BatchBlock to improve performance when writing to storage.
• CyclicBuffer to allow to trace the last n messages on error.
  Experimental feature only. Have not used it in production yet. Don’t know if it still works.

Download complete code.

Sample Trace Output

160628 18:57:47.837 Trace.Init@75|========== Travel Currencies V1.1.42.0
160628 18:57:47.837 Trace.Init@76|traceOutputLevel=Debug: traceTragets=Debug, BufferedFile traceSource=BackgroundTask
160628 18:57:47.837 Trace.Init@77|==========
160628 18:57:47.837 RefreshRatesTask.Run@24|->
160628 18:57:48.484 RefreshRatesTask.Run@39|Refreshing rates. RatesSource=Web
160628 18:57:48.488 License.GetLicenseInfo@37|
160628 18:57:48.495 License.GetLicenseInfo@42|LicenseInformation: isTrial=True TrialExpiration=12/31/9999 1:00:00 AM +01:00 isActive=True 
160628 18:57:48.498 Globals.CheckLicensingInformUser@35|
160628 18:57:48.503 PublisherCacheFile..ctor@13| folder.Path=C:\Users\pit\AppData\Local\Publishers\bmpfaqh7gxjd4\Currencies
160628 18:57:48.503 CurrentRatesSource.GetRatesFromFileAsync@40|
160628 18:57:48.511 PublisherCacheFile.ReadObjectAsync@38|->
160628 18:57:48.682 PublisherCacheFile.ReadObjectAsync@52|<- 160628 18:57:48.687 _RatesSourceBase.LoadRatesFromFileAsync@34|Found cached rates. RatesAsOfDate=6/28/2016 4:00:21 PM +00:00 160628 18:57:48.688 CurrentRatesSource.GetRatesFromWebAsync@51| 160628 18:57:48.688 PublisherCacheFile..ctor@13| folder.Path=C:\Users\pit\AppData\Local\Publishers\bmpfaqh7gxjd4\Currencies 160628 18:57:48.691 WebRatesSource.RefreshRatesFromWebAsync@10|->
160628 18:57:49.396 WebRatesSource.RefreshRatesFromWebAsync@27|RatesAsOfDate Utc=6/28/2016 4:00:21 PM +00:00 RatesAsOfDate Local=6/28/2016 6:00:21 PM +02:00
160628 18:57:49.396 WebRatesSource.RefreshRatesFromWebAsync@33|<-
160628 18:57:49.396 RatesRefresher.RefreshRatesToFile@64|Got new rates as of UTC 6/28/2016 4:00:21 PM +00:00

...

Quirks with date and number formats in UWP Apps and Windows 10

In Windows 10 users can configure several languages, a region, a Regional format and additionally several on-screen keyboards (OSK) in different languages. How to switch between OSK and their handling of region-specific decimal separators is not immediately obvious. UWP apps do by design ignore the Regional format configured by the user and there is a debate if developers should work around this.

Don’t Make the User Think

When displaying dates and numbers and designing data entry it is important to make intuitively clear what the data format is. We must avoid making users think to determine if a number is a day or a month or if a . (dot) is a decimal or a thousand separator.

Good Gear and Tools for Digital Nomads (with DE shopping links)

[ April 11, 2016: created ]
Für einige Überlegungen zum digitalen Nomadetum siehe meinen Artikel “.NET unter Palmen” im dotnetpro Magazin 3.2016.
[ Jan 22, 2018: Digital Nomad 35 backpack added ]

Digital nomads need robust and flexible gear to perform their work duties. Because I frequently get asked for good traveling stuff this post recommends hardware and software for the Windows ecosystem and other things helpful for working while traveling.

Good Gear and Tools for Digital Nomads (with US shopping links)

[ April 11, 2016: created ]
Für einige Überlegungen zum digitalen Nomadetum siehe meinen Artikel “.NET unter Palmen” im dotnetpro Magazin 3.2016.
[ Jan 22, 2018: Digital Nomad 35 backpack added ]

Digital nomads need robust and flexible gear to perform their work duties. Because I frequently get asked for good traveling stuff this post recommends hardware and software for the Windows ecosystem and other things helpful for working while traveling.

How to completely clean install Windows 10 on a UEFI PC

With Windows 10 I generally do recommend to simply upgrade which preserves your apps (modern and desktop), data and settings. In contrast to previous Windows versions the Win 10 upgrade is robust and has worked fine for me on several machines, so I consider a clean install not being worth the time needed.

With my Surface Pro 3 (SP3) however I wanted to do a clean install, including removing Windows 8.1 partitions no longer needed. A Win 10 upgrade from Win 8.1 left my Surface with the partitions shown in the picture below. I could not find any reliable information on which partitions are safe to delete.

Partitions after upgrading from Win 8.1 to Win 10

A clean install as described below removed 2 unneeded partitions and freed 5+ GB SSD space:

Partitions after Win 10 clean install

Windows Service Worker Options

[Für Details siehe meinen Artikel „Perfekter Service“ im dotnetpro Magazin 1/2016.]

Implementation patterns used for Windows Services can seriously influence performance, reactivity, computing resource consumption, stability and energy consumption of our systems. This post shares alternative patterns for creating Windows Services. Download complete code.

Design Aspects

When designing Windows Services consider the following aspects:

• How and when should the service be started?
• If OnStart() takes > 30 sec the SCM will abort the service.
• Choose a suitable pattern to implement processing. Ex: Event-driven, polling loops, poll-timers.
• Maintain thread hygiene.
• If OnStop() takes >~90 sec the SCM will abort the service. Exact time limit is undocumented and Windows version specific.
• Implement graceful shutdown.
• Windows Services must not have a direct UI. You can however create a UI and control the service via its OnCustomCommand method or implement a custom WCF-Interface, see Simple WCF-Services.
• Robust error handling.
• Sufficient tracing and logging.
• Windows Services must be installed.
• Windows Services cannot be started directly in the IDE. See test console app below.

Simple WCF

[Für Details siehe meinen kommenden Artikel “WCF Leicht und robust” im dotnetpro Magazin 1/2016.]

Choosing WCF options and configuring them can be quite cumbersome and demanding. This post shares a very simple approach for using WCF services. Download complete code. My simple approach uses:

• Shared Interface-DLL
• Self-hosting in a Windows Service.
  See Windows Service Worker Options.
• Channel proxies instead of ClientBase proxies.
• Simplified WCF configuration.
• Custom method CreateChannel().
  Caching ChannelFactories.
• Custom method CloseChannel().
  Handling Channel failures.

I am using a new Channel instance for each WCF call. This improves robustness against communication problems and against „The socket connection was aborted“ exceptions after a client idle time > 10 min (= default ReceiveTimeout) with netTcpBinding even when using InstanceContextMode=PerCall.