My Advice for .NET Application Architecture and Design

This post summarizes my general architecture and design guidelines for developing applications and points to some interesting NET technologies.

Reading this post will not make anyone an architect, but working through it should give you a good foundation for understanding and discussing architecture and design decisions.

BTW: Architects must write code to really understand what they are talking about and experience some of the pain their design decisions may cause for project managers, programmers, DBAs, operators and users (see MS WeSYP).

Table of contents:

• Non-Functional Requirements Drive Design!
• What is Architecture?
• Logical Layers
• Organizing Business Logic
  • Simple Domain Logic
    • Table Module pattern
  • Complex Scenarios
    • Domain Model pattern
    • Mobile Objects
  • Fat Database
• Distributing Layers across Tiers
• Distribution Patterns
  • Passing Data through Tiers
  • Collaboration
    • Instance-Based Collaboration
    • Service-Based Collaboration
• Performance and Reliability Patterns
  • Load Balancing
    • Network load balancing
    • Component load balancing
  • Failover Clustering
  • Fail-fast Technique
  • Asynchronous Communication
• Integration Patterns
  • File Transfer
  • Shared Database
  • Remote Procedure Invocation (RPC)
  • Messaging
• Miscellaneous Aspects
• Critical Design Aspects
  • Concurrency control
  • Handling Session state
  • Parallelism
  • Asynchronous Communication
• Miscellaneous Aspects
• Centralize Interoperability, Decentralize Implementation

Related content:

• DB Concurrency Control with .NET
• Use the .NET TransactionScope
• Stored Procedures Pros & Cons
• Avoid unwanted Escalation to Distributed Transactions
• The SQLCommandBuilder is way Better than its Reputation
• .NET Database Access Options
  

Non-Functional Requirements Drive Design!

Do not try a one-size-fits-all approach to application architecture. Resist the golden hammer syndrome: “For a small boy with a new hammer in his hand the whole world looks like a nail”. Ex: Do not use a 4-tier Java web app for everything.

Discover , define, make measurable and communicate the non-functional requirements of your specific application with great care:

• Quality requirements
  usability, reliability, performance, scalability, supportability, securability, …
• IT Constraints
  systems management, installation, integration with other systems,
  company standards (ex: we use Java only)
• Other
  legal, packaging, licensing, cost, time to market

Architecture is: “Decisions that are hard to change later”. Non-functional requirements are the main forces driving architecture and design decisions – not(!) the functional requirements (like manage products and inventory, print invoice).
Ex: I you  decide to use a stateful solution and discover later that your availability and scalability requirements need network load balancing, it will be very difficult to move to a stateless solution (required for networking load balancing without sticky sessions).

For how to make quality requirements measurable see Principles of Software Engineering Management, Tom Gilb.

Select the architecture archetype(s) appropriate for your application:

• Mobile application
• Rich client application
• Rich internet application
• Service application
• Web application
• Embedded system

Your application may comprise different application archetypes playing together.

Create a design that meets all the specific non-functional and functional requirements of your applications. With functional requirements you should avoid unquestioned anticipation of “future needs”. With non-functional requirements is a good idea to anticipate them.

Because non-functional requirements mainly drive our design, we generally can begin developing our  architecture while functional requirements are still incomplete and unstable.

What is Architecture?

Some definitions for Architecture:

• Highest-level breakdown of a system into its parts
• A shared understanding of a system’s design by the expert developers on a project
• Architecture: decisions that are hard to change
  • Identify critical points!
    The difficult or impossible to refactor areas
    Ex: load balancing and component state,using an O/R mapper,(potential) process or machine boundaries
  • And get them right
  • The others don’t matter that much here
    Premature optimization is the root of all evil

Logical Layers

I use the following layer model to discuss and share my design decisions:

See  Microsoft® Application Architecture Guide, 2nd Edition (Patterns & Practices).

Each layer contains discrete component types grouped into sublayers:

Organizing Business Logic

Depending on business logic complexity and application type consider using different ways to organize business logic:

Simple Domain Logic

For applications with little or no domain logic use the Table Module pattern – regardless of the complexity or size of the application!
See Patterns of Enterprise Application Architecture, Martin Fowler.

The Table Module pattern blends well with stateless approaches.

.NET offers excellent tooling for this approach. Consider using the following technologies in your design:

• Forms over Data with MS LightSwitch
• OData
• ADO.NET generic(!) DataSet’s.
  Typed DataSets are a half-baked solution.

The following example uses a single business object instance (ex. Orders) working on multiple data instances (ex: Orders, OrderItems):

Complex Scenarios

Use the Domain Model pattern for more complex scenarios:

• very complex domain logic
• when creating products
  (commonly not for simple custom solutions)
• if external teams will code against you app

The Domain Model Pattern creates an object model of the domain that incorporates both behavior and data:

To implement a domain model approach, consider using Mobile Objects. Mobile Objects are the central feature of Rockford Lhotka’s Component Scalable Logical Architecture (CSLA).
See Expert C# 2008 Business Objects, Rockford Lhotka.
Mobile objects are rich business domain objects (behavior + data) that can move across tiers, are bindable in Silverlight, WPF and Windows forms and can save themselves to a data store:

Mobile object:

• Business object encapsulating in one class:
  • Data
  • Validation
  • Business logic
  • Data access
• Moves across tiers
  • Logic and data move from tier to tier
  • Object is passed by value
    Deep serialization (including private properties!)
  • Runs on UI tier for data validation
  • Runs on app server for data access
• Fits nicely with .NET IDE UI features and ADO.NET
• All validation logic in one class.
  UI input validation. BL validation.
• Code must be deployed on clients and servers.

Fat Database

For purely data centric applications consider using a Fat Server approach doing everything in the database.

• For the benefits of doing things in the database see “My Approach” in Expert Oracle Database Architecture: 9i and 10g Programming Techniques and Solutions, Thomas Kyte.
• For a more critical view see Stored Procedures Pros & Cons.

Distributing Layers across Tiers

Common options for distributing layers across tiers:

Tiered distribution pros and cons:

Scalability
Performance

• Must size hardware and network properly
• Use chunky (vs. chatty) calls!

Complexity

Distribution Patterns

When distributing components across .NET application domains, processes, machines or in the interned (Cloud) there are many aspects to consider.

Distribution considerations:

• The first law of distributed object design:
  • Don’t distribute your objects
    Naïve OO approaches and tool vendors promise object location transparency.
    But performance is not location transparent!
• You can distribute horizontally & vertically
  • Both adds complexity
• Load balancing
  • Do you need the availability or scalability it offers?
  • Weigh the state management problem.
• Do you need multi-tier distribution?
  • Adding tiers hurts performance and availability.
    Must size hardware and network to compensate.
    Consider using async communications.
  • Out-of-process vs. in-process calls
    1000 times slower (still just millisecs!)
  • Don‘t forget the wire!
    If an object could ever be in a separate process
    consider using coarse-grained interfaces.

Generally you have to distribute

• Between client and server
  • Typically x-machine
• Between application server and database server
  • Typically x-machine
    App and DB usage need different OS configurations.
    System management standards may enforce separation.
    Good: SQL is chunky.
  • Can run app in DB
    Ex: Stored procedures, .NET code in SQL Server
• Because of vendor differences
  • SW packages typically run in their own process
  • SLA requires dedicated hardware.
• Not(!) between web server and app server.
  • Typically x-process
  • Availability no longer is a problem.
    App errors no longer crash Apache or IIS 6
  • Security considerations might still be a reason to distribute.
• Fight for minimizing unfounded x-process communication!

Passing Data through Tiers

Choose from common ways to represent data and pass it through tiers:

• Simple params or array
• XML document
• ADO.NET generic DataSet
  Very easy and productive to use.
• ADO.NET typed DataSet
• Business Entity Components (see logical layers)
  Data Transfer Object (DTO)
• Entity Framework self-tracking Entities
• Entity Framework self-tracking POCOs
• CSLA Mobile Objects

Collaboration

Choose between instance- and service-based collaboration.

WCF supports both collaboration types.
In homogenous .NET environments consider using IPC or TCP bindings for best performance.
WCF provides support to enable services to be discoverable at runtime in an interoperable way using WCF Discovery.

Some interesting .NET technologies in this realm:

• Windows Azure AppFabric Service Bus relay service.
  See Programming WCF Services: Mastering WCF and the Azure AppFabric Service Bus
• Open Date Protocol OData
• WCF RIA Services

Instance-Based Collaboration

• Extends the OO model across process and network boundaries
• Consumer can instantiate remote object instances, pass references around, invoke methods, deallocate instance
• Ex: .NET Remoting CAO, WCF Per-Session Service, Java RMI

• Uses the OO model for distributed computing
• Simplifies development
• Consumer has fine-grained lifetime control

• Complex interaction model
• Tight coupling
• Lifetime complexities
  Ex. Protocol must handle dying clients
• Interoperability across heterogeneous platforms is problematic

Use for near links. In a controlled environment.

Consider using brokers to decouple clients from servers:

See Integration Patterns, Microsoft.

Service-Based Collaboration:

• Exposes only a “manger-like” interface to consumers
• Consumer can invoke methods on interface
• Ex: WebServices (SOAP)

• Simplifies interaction
• Enables interoperability across platforms using standard protocols
• Less coupling by using messages

• Uses non-OO model for remote objects
• Must track state explicitly
• Consumer has no lifetime control

Use for far links. Connected system may not be within single control-span.

I case you want to implement web services (SOAP): prefer WCF over ASMX web services.

Performance and Reliability Patterns

Common approaches to improve scalability and availability:

Load Balancing

Network load balancing:

See WCF Load Balancing

Component load balancing:

Load balancing pros and cons:

Scalability

Availability

State management problem

Failover Clustering

Sample usage of server farms and clustering for scalability and availability:

Fail-fast Technique

Use the Fail-Fast technique to create robust apps:

• Helps against Heisen-bugs (vs. Bor-bugs)
• To make app software robust:
  • App kills itself when unexpected error occurs
    • Does not try to resolve problem
    • Does not try to cleanup
    • Relies on transactions protection by resource dispenser
      Ex: database, message queue
    • Client (user or process) has to check
      if last transaction was successful
  • App gets restarted by a monitor
    • Cluster Monitor
    • Windows service control manager (SCM)
  • If error happens again do machine reboot
• Similar approach: App recycling
  • Preventively restart app
    • Periodically or
    • Based on requests served, memory usage,
      requests queued, quiet time
  • Is used by Apache and IIS

Critical Design Aspects

Some critical aspects one always should consider:

• Concurrency control
• Handling Session state
• Parallelism
• Asynchronous communication

Concurrency control

See DB Concurrency Control with .NET.

Handling Session state

Decide how to handle session state.

State definitions:

• State: A property is part of the state of a component instance when it must retain its value between two invocations of any method.
• State management: Mechanism used to logically store the state of a component instance, for the duration of an interaction (session) with it.
• Options to handle state:
  • Stateful component:
    • Component retains state between requests.
      Ex: Standard OO classes.
    • For horizontal scalability you may use the component load balancing feature of a container.
  • Stateless component:
    • Component does not retain state between requests.
    • Use self-contained, service-based interfaces passing all data to fulfill request.
    • Or use client, server, database state (see next slides).
    • Blends nicely with network load balancing.

Stateful vs. Stateless Components:

• Stateful business components
  • Fit nicely with OO thinking.
  • Might provide good performance.
    If horizontal scalability is not needed.
  • Can use component load balancing
      x-container synchronization
• Stateless business components
  • Options
  • Store state in client
    and pass it with each component call
  • Store state in server memory or database
    and restore component state on each call
  • Scales nicely across server farms
  • Is simple (compared to component load-balancing)
  • Closer to procedural than OO thinking
  • Ex: use WCF per call services
  • Fits nicely with remote facades
    or message-based web services

Options for handling session state

• Client Session State.
  • Hold state in Client.
  • Natural with rich clients.
  • Costs bandwidth with thin clients because state must be passed with every request.
  • Ex: ASP.NET ViewState
• Server Session State.
  • Hold state in server memory.
  • Use dedicates state server against server affinity.
  • Memory is fast but does not scale well.
• Database Session State.
  • Hold state in database.
  • Remember: a database is a large in-memory cache the accesses hard disks if it has nothing better to do

Parallelism

Consider using parallel programming to increase the responsiveness of an application and to make use of multi-core and many-core machines.

Some interesting .NET technologies in this realm:

• Task Parallel Library (TPL)
  See Parallel Programming with Microsoft .NET
• BackgroundWorker Class

Asynchronous Communication

Consider using asynchronous communication to increase the responsiveness, availability and scalability of your application:

Synchronous: Client thread is blocked during request

• Bad for scalability, load balancing
• Bad for high availability
• Easy to use

Asynchronous: Client thread is not blocked during request

• Messaging
• Async calls
• Good for scalability, load balancing
• Good for high availability
• More complex to use
  (Messaging is easy)

Some interesting .NET technologies in this realm:

• Asynchronous Programming Design Patterns
• Async language features
• Message Queuing (MSMQ)

Sample asynchronous architecture using a message queue:

Integration Patterns

The following overview shows common integration patterns:

1. Integration Strategies
2. Connection Applications
3. Integration Topologies
   
   See Microsoft Integration Patterns

Common options for integration multiple applications so that they can work together or exchange information are:

• Focus on sharing data
  • File Transfer
    App produces files of shared data for other apps to consume.
  • Shared Database
    Apps store data they wish to share in a common database.
• Focus on using functionality
  • Remote Procedure Invocation (RPC)
    App exposes some of its procedures so that they can be invoked remotely.
    Other apps invoke those to run behavior and exchange data.
  • Messaging
    Apps connect to a common messaging system, exchanging data and invoking behavior using messages.

See Enterprise Integration Patterns, Gregor Hohpe.

Have a look at Windows Azure AppFabric Service Bus relay service.
See Programming WCF Services: Mastering WCF and the Azure AppFabric Service Bus

Miscellaneous Aspects

Keep it simple and lean!
Good Solutions strike a balance between the size of the problem they solve and the effort to implement them.

Iterate!
Plan to throw away the first version,
you will throw one away anyway.
See The Secrets of Consulting, Gerald Weinberg.

Use an executable architecture!
Create it early, test it, develop with it.

Prototype and performance test early!
Using a (simulated) production environment
(client load, database size, topology).

Never forget the (potential) wire!
Cross-machine or low-bandwidth, high-latency connections I mean.

Only distribute vertically (x-Process, x-Machine) or horizontally (for load-balancing or availability) if you really have too.

Consider using asynchronous calls or messaging.

Prototype and performance test early using a (simulated) production environment(client load, database size, topology).

Consciously decide if using plain ADO.NET or an O/RM.
See .NET Database Access Options.

Beware of using or developing custom frameworks!
Stick with frameworks the are readily available out-of-the-box or as open source and invest your time and energy in providing a benefit to end users. Do not create your own frameworks like MVVM, O/R mappers, etc. Your colleagues will thank you for not leaving smart, half-baked frameworks behind for them to maintain and your customers will be glad to avoid a software house lock-in.

Centralize Interoperability, Decentralize Implementation

Some final words to our colleagues defining company standards:

Do define strict standards for application interoperability.
Ex: require all applications to offer their data or functionality via secured (web) services or OData feeds.

• Reuse by delegation (Ex: using web service) works.
• Reuse by sharing code or components generally does not work.
• Mega (company) data models or repositories generally fail.

Choosing platforms like Java or .NET, development environments or database systems is an implementation decisions. While recommendations are fine, please do not try to enforce strict standards here. Leave these decisions to the project teams. Unrealistic standardization only gets bypassed and over the time you loose control over your application landscape.

Offer an easy route for controlled rapid application development by and for your business users (ex: actively support LightSwitch).

Books

[1] Principles of Software Engineering Management
Tom Gilb

[2] Patterns of Enterprise Application Architecture
Martin Fowler

[3] Expert C# 2008 Business Objects
Rockford Lhotka

[4] Expert Oracle Database Architecture: 9i and 10g Programming Techniques and Solutions
Thomas Kyte

[5] Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions
Gregor Hohpe

[6] Integration Patterns
Microsoft

[7] Microsoft Application Architecture Guide, 2nd Edition
Microsoft

[8] Programming WCF Services: Mastering WCF and the Azure AppFabric Service Bus
Juval Lowy

About these ads