My Advice for .NET Application Architecture and Design

This post summarizes my general architecture and design guidelines for developing applications and points to some interesting NET technologies.

Reading this post will not make anyone an architect, but working through it should give you a good foundation for understanding and discussing architecture and design decisions.

BTW: Architects must write code to really understand what they are talking about and experience some of the pain their design decisions may cause for project managers, programmers, DBAs, operators and users (see MS WeSYP).

Table of contents:

  • Non-Functional Requirements Drive Design!
  • What is Architecture?
  • Logical Layers
  • Organizing Business Logic
    • Simple Domain Logic
      • Table Module pattern
    • Complex Scenarios
      • Domain Model pattern
      • Mobile Objects
    • Fat Database
  • Distributing Layers across Tiers
  • Distribution Patterns
    • Passing Data through Tiers
    • Collaboration
      • Instance-Based Collaboration
      • Service-Based Collaboration
  • Performance and Reliability Patterns
    • Load Balancing
      • Network load balancing
      • Component load balancing
    • Failover Clustering
    • Fail-fast Technique
    • Asynchronous Communication
  • Integration Patterns
    • File Transfer
    • Shared Database
    • Remote Procedure Invocation (RPC)
    • Messaging
  • Miscellaneous Aspects
  • Critical Design Aspects
    • Concurrency control
    • Handling Session state
    • Parallelism
    • Asynchronous Communication
  • Miscellaneous Aspects
  • Centralize Interoperability, Decentralize Implementation

Related content:

Non-Functional Requirements Drive Design!

Do not try a one-size-fits-all approach to application architecture. Resist the golden hammer syndrome: “For a small boy with a new hammer in his hand the whole world looks like a nail”. Ex: Do not use a 4-tier Java web app for everything.

Discover Smile, define, make measurable and communicate the non-functional requirements of your specific application with great care:

  • Quality requirements
    usability, reliability, performance, scalability, supportability, securability, …
  • IT Constraints
    systems management, installation, integration with other systems,
    company standards (ex: we use Java only)
  • Other
    legal, packaging, licensing, cost, time to market

Architecture is: “Decisions that are hard to change later”. Non-functional requirements are the main forces driving architecture and design decisions – not(!) the functional requirements (like manage products and inventory, print invoice).
Ex: I you  decide to use a stateful solution and discover later that your availability and scalability requirements need network load balancing, it will be very difficult to move to a stateless solution (required for networking load balancing without sticky sessions).

For how to make quality requirements measurable see Principles of Software Engineering Management, Tom Gilb.

Select the architecture archetype(s) appropriate for your application:

  • Mobile application
  • Rich client application
  • Rich internet application
  • Service application
  • Web application
  • Embedded system

Your application may comprise different application archetypes playing together.

Create a design that meets all the specific non-functional and functional requirements of your applications. With functional requirements you should avoid unquestioned anticipation of “future needs”. With non-functional requirements is a good idea to anticipate them.

Because non-functional requirements mainly drive our design, we generally can begin developing our  architecture while functional requirements are still incomplete and unstable.

What is Architecture?

Some definitions for Architecture:

  • Highest-level breakdown of a system into its parts
  • A shared understanding of a system’s design by the expert developers on a project
  • Architecture: decisions that are hard to change
    • Identify critical points!
      The difficult or impossible to refactor areas
      Ex: load balancing and component state, using an O/R mapper, (potential) process or machine boundaries
    • And get them right
    • The others don’t matter that much here
      Premature optimization is the root of all evil

Logical Layers

I use the following layer model to discuss and share my design decisions:

image
See  Microsoft® Application Architecture Guide, 2nd Edition (Patterns & Practices).

Each layer contains discrete component types grouped into sublayers:

image

Organizing Business Logic

Depending on business logic complexity and application type consider using different ways to organize business logic:

Simple Domain Logic

For applications with little or no domain logic use the Table Module pattern – regardless of the complexity or size of the application!
See Patterns of Enterprise Application Architecture, Martin Fowler.

The Table Module pattern blends well with stateless approaches.

.NET offers excellent tooling for this approach. Consider using the following technologies in your design:

  • Forms over Data with MS LightSwitch
  • OData
  • ADO.NET generic(!) DataSet’s.
    Typed DataSets are a half-baked solution.

The following example uses a single business object instance (ex. Orders) working on multiple data instances (ex: Orders, OrderItems):

image

Complex Scenarios

Use the Domain Model pattern for more complex scenarios:

  • very complex domain logic
  • when creating products
    (commonly not for simple custom solutions)
  • if external teams will code against you app

The Domain Model Pattern creates an object model of the domain that incorporates both behavior and data:

image

To implement a domain model approach, consider using Mobile Objects. Mobile Objects are the central feature of Rockford Lhotka’s Component Scalable Logical Architecture (CSLA).
See Expert C# 2008 Business Objects, Rockford Lhotka.
Mobile objects are rich business domain objects (behavior + data) that can move across tiers, are bindable in Silverlight, WPF and Windows forms and can save themselves to a data store:

image

Mobile object:

  • Business object encapsulating in one class:
    • Data
    • Validation
    • Business logic
    • Data access
  • Moves across tiers
    • Logic and data move from tier to tier
    • Object is passed by value
      Deep serialization (including private properties!)
    • Runs on UI tier for data validation
    • Runs on app server for data access
  • Fits nicely with .NET IDE UI features and ADO.NET
  • :-) All validation logic in one class.
    UI input validation. BL validation.
  • :-( Code must be deployed on clients and servers.

Fat Database

For purely data centric applications consider using a Fat Server approach doing everything in the database.

Distributing Layers across Tiers

Common options for distributing layers across tiers:

image

Tiered distribution pros and cons:

:-) Scalability
:shock: Performance

  • Must size hardware and network properly
  • Use chunky (vs. chatty) calls!

:-( Complexity

Distribution Patterns

When distributing components across .NET application domains, processes, machines or in the interned (Cloud) there are many aspects to consider.

Distribution considerations:

  • The first law of distributed object design:
    • Don’t distribute your objects
      Naïve OO approaches and tool vendors promise object location transparency.
      But performance is not location transparent!
  • You can distribute horizontally & vertically
    • Both adds complexity
  • Load balancing
    • Do you need the availability or scalability it offers?
    • Weigh the state management problem.
  • Do you need multi-tier distribution?
    • Adding tiers hurts performance and availability.
      Must size hardware and network to compensate.
      Consider using async communications.
    • Out-of-process vs. in-process calls
      1000 times slower (still just millisecs!)
    • Don‘t forget the wire!
      If an object could ever be in a separate process
      consider using coarse-grained interfaces.

Generally you have to distribute

  • Between client and server
    • Typically x-machine
  • Between application server and database server
    • Typically x-machine
      App and DB usage need different OS configurations.
      System management standards may enforce separation.
      Good: SQL is chunky.
    • Can run app in DB
      Ex: Stored procedures, .NET code in SQL Server
  • Because of vendor differences
    • SW packages typically run in their own process
    • SLA requires dedicated hardware.
  • Not(!) between web server and app server.
    • Typically x-process
    • Availability no longer is a problem.
      App errors no longer crash Apache or IIS 6
    • Security considerations might still be a reason to distribute.
  • Fight for minimizing unfounded x-process communication!

Passing Data through Tiers

Choose from common ways to represent data and pass it through tiers:

Collaboration

Choose between instance- and service-based collaboration.

WCF supports both collaboration types.
In homogenous .NET environments consider using IPC or TCP bindings for best performance.
WCF provides support to enable services to be discoverable at runtime in an interoperable way using WCF Discovery.

Some interesting .NET technologies in this realm:

Instance-Based Collaboration

  • Extends the OO model across process and network boundaries
  • Consumer can instantiate remote object instances, pass references around, invoke methods, deallocate instance
  • Ex: .NET Remoting CAO, WCF Per-Session Service, Java RMI

:-)

  • Uses the OO model for distributed computing
  • Simplifies development
  • Consumer has fine-grained lifetime control

:-(

  • Complex interaction model
  • Tight coupling
  • Lifetime complexities
    Ex. Protocol must handle dying clients
  • Interoperability across heterogeneous platforms is problematic

Use for near links. In a controlled environment.

Consider using brokers to decouple clients from servers:

image
See Integration Patterns, Microsoft.

Service-Based Collaboration:

  • Exposes only a “manger-like” interface to consumers
  • Consumer can invoke methods on interface
  • Ex: WebServices (SOAP)

:-)

  • Simplifies interaction
  • Enables interoperability across platforms using standard protocols
  • Less coupling by using messages

:-(

  • Uses non-OO model for remote objects
  • Must track state explicitly
  • Consumer has no lifetime control

Use for far links. Connected system may not be within single control-span.

image

I case you want to implement web services (SOAP): prefer WCF over ASMX web services.

Performance and Reliability Patterns

Common approaches to improve scalability and availability:

image

Load Balancing

Network load balancing:

image
See WCF Load Balancing

Component load balancing:

image

Load balancing pros and cons:

:-) Scalability

:-) Availability

:-( State management problem

Failover Clustering

image

Sample usage of server farms and clustering for scalability and availability:

image

Fail-fast Technique

Use the Fail-Fast technique to create robust apps:

  • Helps against Heisen-bugs (vs. Bor-bugs)
  • To make app software robust:
    • App kills itself when unexpected error occurs
      • Does not try to resolve problem
      • Does not try to cleanup
      • Relies on transactions protection by resource dispenser
        Ex: database, message queue
      • Client (user or process) has to check
        if last transaction was successful
    • App gets restarted by a monitor
      • Cluster Monitor
      • Windows service control manager (SCM)
    • If error happens again do machine reboot
  • Similar approach: App recycling
    • Preventively restart app
      • Periodically or
      • Based on requests served, memory usage,
        requests queued, quiet time
    • Is used by Apache and IIS

Critical Design Aspects

Some critical aspects one always should consider:

  • Concurrency control
  • Handling Session state
  • Parallelism
  • Asynchronous communication

Concurrency control

See DB Concurrency Control with .NET.

Handling Session state

Decide how to handle session state.

State definitions:

  • State: A property is part of the state of a component instance when it must retain its value between two invocations of any method.
  • State management: Mechanism used to logically store the state of a component instance, for the duration of an interaction (session) with it.
  • Options to handle state:
    • Stateful component:
      • Component retains state between requests.
        Ex: Standard OO classes.
      • For horizontal scalability you may use the component load balancing feature of a container.
    • Stateless component:
      • Component does not retain state between requests.
      • Use self-contained, service-based interfaces passing all data to fulfill request.
      • Or use client, server, database state (see next slides).
      • Blends nicely with network load balancing.

Stateful vs. Stateless Components:

  • Stateful business components
    • Fit nicely with OO thinking.
    • Might provide good performance.
      If horizontal scalability is not needed.
    • Can use component load balancing
      :-(  x-container synchronization
  • Stateless business components
    • Options
    • Store state in client
      and pass it with each component call
    • Store state in server memory or database
      and restore component state on each call
    • Scales nicely across server farms
    • Is simple (compared to component load-balancing)
    • Closer to procedural than OO thinking
    • Ex: use WCF per call services
    • Fits nicely with remote facades
      or message-based web services

Options for handling session state

  • Client Session State.
    • Hold state in Client.
    • Natural with rich clients.
    • Costs bandwidth with thin clients because state must be passed with every request.
    • Ex: ASP.NET ViewState
  • Server Session State.
    • Hold state in server memory.
    • Use dedicates state server against server affinity.
    • Memory is fast but does not scale well.
  • Database Session State.
    • Hold state in database.
    • Remember: a database is a large in-memory cache the accesses hard disks if it has nothing better to do

Parallelism

Consider using parallel programming to increase the responsiveness of an application and to make use of multi-core and many-core machines.

Some interesting .NET technologies in this realm:

Asynchronous Communication

Consider using asynchronous communication to increase the responsiveness, availability and scalability of your application:

Synchronous: Client thread is blocked during request

  • :-( Bad for scalability, load balancing
  • :-( Bad for high availability
  • :-) Easy to use

Asynchronous: Client thread is not blocked during request

  • Messaging
  • Async calls
  • :-) Good for scalability, load balancing
  • :-) Good for high availability
  • :-( More complex to use
    (Messaging is easy)

Some interesting .NET technologies in this realm:

Sample asynchronous architecture using a message queue:

image

Integration Patterns

The following overview shows common integration patterns:

  1. Integration Strategies
  2. Connection Applications
  3. Integration Topologies
    image

    See Microsoft Integration Patterns

Common options for integration multiple applications so that they can work together or exchange information are:

  • Focus on sharing data
    • File Transfer
      App produces files of shared data for other apps to consume.
    • Shared Database
      Apps store data they wish to share in a common database.
  • Focus on using functionality
    • Remote Procedure Invocation (RPC)
      App exposes some of its procedures so that they can be invoked remotely.
      Other apps invoke those to run behavior and exchange data.
    • Messaging
      Apps connect to a common messaging system, exchanging data and invoking behavior using messages.

See Enterprise Integration Patterns, Gregor Hohpe.

Have a look at Windows Azure AppFabric Service Bus relay service.
See Programming WCF Services: Mastering WCF and the Azure AppFabric Service Bus

Miscellaneous Aspects

Keep it simple and lean!
Good Solutions strike a balance between the size of the problem they solve and the effort to implement them.

Iterate!
Plan to throw away the first version,
you will throw one away anyway.
See The Secrets of Consulting, Gerald Weinberg.

Use an executable architecture!
Create it early, test it, develop with it.

Prototype and performance test early!
Using a (simulated) production environment
(client load, database size, topology).

Never forget the (potential) wire!
Cross-machine or low-bandwidth, high-latency connections I mean.

Only distribute vertically (x-Process, x-Machine) or horizontally (for load-balancing or availability) if you really have too.

Consider using asynchronous calls or messaging.

Prototype and performance test early using a (simulated) production environment(client load, database size, topology).

Consciously decide if using plain ADO.NET or an O/RM.
See .NET Database Access Options.

Beware of using or developing custom frameworks!
Stick with frameworks the are readily available out-of-the-box or as open source and invest your time and energy in providing a benefit to end users. Do not create your own frameworks like MVVM, O/R mappers, etc. Your colleagues will thank you for not leaving smart, half-baked frameworks behind for them to maintain and your customers will be glad to avoid a software house lock-in.

Centralize Interoperability, Decentralize Implementation

Some final words to our colleagues defining company standards:

Do define strict standards for application interoperability.
Ex: require all applications to offer their data or functionality via secured (web) services or OData feeds.

  • Reuse by delegation (Ex: using web service) works.
  • Reuse by sharing code or components generally does not work.
  • Mega (company) data models or repositories generally fail.

Choosing platforms like Java or .NET, development environments or database systems is an implementation decisions. While recommendations are fine, please do not try to enforce strict standards here. Leave these decisions to the project teams. Unrealistic standardization only gets bypassed and over the time you loose control over your application landscape.

Offer an easy route for controlled rapid application development by and for your business users (ex: actively support LightSwitch).

Books

[1] Principles of Software Engineering Management
Tom Gilb

[2] Patterns of Enterprise Application Architecture
Martin Fowler

[3] Expert C# 2008 Business Objects
Rockford Lhotka

[4] Expert Oracle Database Architecture: 9i and 10g Programming Techniques and Solutions
Thomas Kyte

[5] Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions
Gregor Hohpe

[6] Integration Patterns
Microsoft

[7] Microsoft Application Architecture Guide, 2nd Edition
Microsoft

[8] Programming WCF Services: Mastering WCF and the Azure AppFabric Service Bus
Juval Lowy

About these ads

About Peter Meinl

IT Consultant
This entry was posted in Computers and Internet and tagged , , , , , , , , . Bookmark the permalink.

3 Responses to My Advice for .NET Application Architecture and Design

  1. Pingback: DB Concurrency Control with .NET – Overview « DuongTienLam's Blog

  2. anupam singh says:

    well explained but I am quit confused yet .where to put what ?, I mean if I m using dependency resolver,repository,entity, abstract factory,interfaces,helpers, utility etc.
    however I can use BLL for business logic and DLL for database handling. please suggest if I use n tier web application ,how to manage it.

    • Peter Meinl says:

      Don’t try to make use of each and every pattern you can think of. Keep your solution simple!

      Dependency injection is a powerful and helpful pattern: If you don’t have solid experience only use it if the complexity of your solution really requires it.

      Repository, entities, …: Just use a standard O/R mapper of your dev platform, ex: the .NET Entity Framework.

      Abstract factory, interfaces: These are just coding patterns you can use when helpful.
      Try coding a part of your solution with and without them to get a feeling for the pros and cons.

      Helpers, utilities: Just create what you need. If your solution is not distributed across machines (like with a simple web application) you don’t even need to encapsulate them in a separate DLL.

      If the non-functional requirements of your application ask for a simple web application there might be no need to create separate business and data layers. Just write your business logic and access the database directly via the O/R Mapper.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s