Chapter 18: Communication and Messaging

For more details of the topics covered in this guide, see Contents of the Guide.

Contents

• Overview
• General Design Guidelines
• Message-Based Communication Guidelines
• Contract-First Design
• Security Considerations
• Technology Options
• Additional Resources

Overview

One of the key factors that affect the design of an application—particularly a distributed application—is the way that you design the communication infrastructure for each part of the application. Components must communicate with each other; for example, to send user input to the business layer, and then to update the data store through the data layer. When components are located on the same physical tier, you can often rely on direct communication between these components. However, if you deploy components and layers on physically separate servers and client machines—as is likely in most scenarios—you must consider how the components in these layers will communicate with each other efficiently and reliably.

In general, you must choose between direct communication (such as method calls between components) and message-based communication. There are many advantages to using message-based communication, such as the ability to decouple your components from each other. Decoupling components not only improves maintainability but can also provide flexibility that makes it easier to change your deployment strategy in the future. However, message-based communication raises issues that you must consider, such as performance, reliability, and security.

This chapter presents design guidelines that will help you to choose the appropriate communication approach, understand how to get the best results from your chosen approach, and anticipate security and reliability issues that might arise. However, the bulk of this chapter focuses on designing a suitable message-based communication mechanism, together with guidelines for asynchronous and synchronous communication, data format, performance, security, interoperability, and choice of implementation technology.

General Design Guidelines

When designing a communication strategy for your application, consider the performance impact of communicating between layers, as well as between tiers. Because each communication across a logical or a physical boundary increases processing overhead, design for efficient communication by reducing round trips and minimizing the amount of data sent over the network. Consider the following guidelines when deciding on a communication strategy:

• Consider communication strategies when crossing boundaries. Understand each of your boundaries and how they affect communication performance. For example, the computer process, machine, and managed-to-unmanaged code transition all represent boundaries that that may be crossed when communicating with components of the application or external services and applications.
• Consider using message-based communication when crossing process boundaries. Use Windows Communication Foundation (WCF) with either the TCP or named pipes protocol for maximum performance.
• Consider using message-based communication when crossing physical boundaries. Consider using WCF to communicate with remote machines across physical boundaries. Consider using Microsoft Message Queuing for once only reliable delivery of messages.
• Maximize performance and responsiveness when accessing remote layers. When communicating with remote layers, reduce communication requirements by using coarse-grained message-based communication methods and use asynchronous communication if possible to avoid blocking or freezing the UI.
• Consider the serialization capabilities of the data formats passed across boundaries. If you require interoperability with other systems, consider XML serialization. Keep in mind that XML serialization imposes increased overhead. If performance is critical, consider binary serialization because it is faster and the resulting serialized data is smaller than the XML equivalent.
• Ensure that you protect messages and sensitive data during communication. Consider using encryption, digital certificates, and channel security features.
• Implement mechanisms to enforce idempotency and commutativity. Ensure that your application code can detect and manage messages that arrive more than once (idempotency) and multiple messages that arrive out of order (commutativity).

Message-Based Communication Guidelines

Message-based communication allows you to expose a service to your callers by defining a service interface that clients call by passing XML-based messages over a transport channel. Message-based calls are generally made from remote clients, but message-based service interfaces can support local callers as well. A message-based communication style is well suited to the following scenarios:

• You are implementing a business system that represents a medium to long term investment; for example, when building a service that will be exposed to and used by partners for a considerable time.
• You are implementing large scale systems that must offer high-availability, or must operate over unreliable networks. In this case, a message store and forward mechanism can provide improved reliability.
• You are building a service that you want to isolate from other services it uses, and from services that consume it. The use of message-based service interfaces that advertise interface details to clients makes it easy for any clients to use the service without requiring specific implementations for individual clients.
• You are dealing with real world business processes that use the asynchronous model.

Consider the following guidelines when using message-based communication:

• Be aware that a connection will not always be present, and that messages may need to be stored and then sent when a connection becomes available.
• Consider how to handle the case when a message response is not received. To manage the conversation state, your business logic can log the sent messages for later processing in case a response is not received.
• Consider using acknowledgements to force the correct sequencing of messages.
• Use standard protocols, such as HTTP for Internet communication and TCP for intranet communication. Do not implement a custom communication channel unless there is no default combination of endpoint, protocol, and format that suits your needs.
• If message response timing is critical for your communication, consider a synchronous programming model in which your client waits for each response message. Alternatively, where clients can continue to execute while waiting for a response, consider an asynchronous model.

When designing your message-based communication strategy, you should also consider specific topics that can affect the stability, reusability, performance, and overall success of your design. The following sections describe these issues in more detail:

• Asynchronous vs. Synchronous Communication
• Coupling and Cohesion
• Data Formats
• Interoperability
• Performance
• State Management

Asynchronous vs. Synchronous Communication

Consider the key tradeoffs when choosing between synchronous and asynchronous communication styles. Synchronous communication is best suited to scenarios in which you must guarantee the order in which calls are received, or when you must wait for the call to return. Asynchronous communication is best suited to scenarios in which responsiveness is important or you cannot guarantee that the target will be available. Consider the following guidelines when deciding whether to use synchronous or asynchronous communication:

• For maximum performance, loose coupling, and minimized system overhead, consider using an asynchronous communication model. If some clients can only make synchronous calls, consider wrapping existing asynchronous service methods in a component that performs synchronous communication with the client.
• In cases where you must guarantee the order in which operations take place, or where you use operations that depend on the outcome of previous operations, consider a synchronous model.
• For asynchronous in-process calls, use the platform features (such as Begin and End versions of methods and callbacks) to implement asynchronous method calls. For asynchronous out of process calls, such as calls across physical tiers and boundaries, consider using messaging or asynchronous service requests

If you choose asynchronous communication and cannot guarantee network connectivity or the availability of the target, consider using a store and forward message delivery mechanism to avoid losing messages. When choosing a store and forward design strategy, consider using local caches to store messages for later delivery in case of system or network interruption. Alternatively, consider using Message Queuing to queue messages for later delivery in case of system or network interruption or failure. Message Queuing can perform transacted message delivery and supports reliable once only delivery. If you need to interoperate with other systems and platforms at the enterprise level, or perform electronic data interchange, consider using BizTalk Server provide a robust delivery mechanism.

Coupling and Cohesion

Communication methods that impose interdependencies between the distributed parts of the application will result in a tightly coupled application. A loosely coupled application uses methods that impose a minimum set of requirements for communication to occur. Consider the following guidelines when designing for coupling and cohesion:

• For loose coupling, consider using a message-based technology such as ASP.NET Web services (ASMX) or WCF, and self describing data and ubiquitous protocols such as HTTP, REST, and SOAP.
• To maintain cohesion, ensure that interfaces contain only methods that are closely related in purpose and functional area.

Data Formats

The most common data formats for passing data across tiers are scalar values, XML, DataSets, and custom objects. Use the following table to understand the key considerations for choosing a data type.

Consider the following guidelines when selecting a data format for a communication channel:

• If your application works mainly with instance data, consider using simple values for better performance. Simple value types will reduce your initial development costs; however, they produce tight coupling that can increase maintenance costs if the types must change in the future.
• XML may require additional upfront schema definition but will result in loose coupling that can reduce future maintenance costs and increase interoperability (for example, if you want to expose your interface to additional XML-compliant callers).
• DataSets work well for complex data types, especially if they are populated directly from a database. However, it is important to understand that DataSets also contain schema and state information that increases the overall volume of data passed across the network, and their special format may restrict interoperability with other systems. Consider using DataSets if your application works mainly with sets of data and needs functionality such as sorting, searching, and data binding.
• Custom objects work best when none of the other options meet your requirements, or when you are communicating with components that expect a custom object. They tend to impose a lower overhead than DataSets and support both binary and XML serialization. Usually the custom objects you use to transmit data across the communication channel will be data transfer objects (DTOs) that contain data extracted from your business entities.
• Ensure that type information is not lost during the communication process. Binary serialization preserves type fidelity, which is useful when passing objects between client and server. However, this approach means that you must implement a more rigorous versioning system for interfaces. Default XML serialization serializes only public properties and fields and does not preserve type fidelity.

Interoperability

The main factors that influence interoperability of applications and components are the availability of suitable communication channels, and the formats and protocols that the participants can understand. Consider the following guidelines for maximizing interoperability:

• To enable communication with wide variety of platforms and devices, consider using standard protocols and data formats such as HTTP and XML. Keep in mind that your protocol decisions may affect the availability of clients you are targeting. For example, target systems might be protected by firewalls that block some protocols.
• Consider versioning issues for interfaces and contracts. Changes to a service may be required due to changing business needs, information technology requirements, or other issues. Where these changes result in an incompatible interface, message contract, or data contract, consider creating a new version that clients can use while allowing existing clients to use the previous version where they do not need to access the functionality exposed by the new interface. For more information, see "Service Versioning" at https://msdn.microsoft.com/en-us/library/ms731060.aspx.
• The data format you choose may affect interoperability. For example, target systems might not understand platform specific types, or might have different ways of handling and serializing types.
• Your security encryption and decryption decisions may affect interoperability. For example, some message encryption/decryption techniques might not be available on all systems.

Performance

The design of your communication interfaces and the data formats you use will also have a considerable impact on the performance of your application, especially when crossing process or machine boundaries. While other considerations, such as interoperability, may require specific interfaces and data formats, there are techniques you can use to improve performance related to communication between different layers or tiers of your application. Consider the following guidelines for performance:

• Do not pass unnecessary data to remote methods where possible, and minimize the volume of data sent across the network. This reduces serialization overhead and network latency. However, avoid fine-grained (chatty) interfaces for cross-process and cross-machine communication. These require the client to make multiple method calls to perform a single logical unit of work. Consider using the Façade pattern to provide a coarse-grained wrapper for existing chatty interfaces.
• Consider using DTOs to pass data as a single unit instead of passing individual data types one at a time.
• If serialization performance is critical for your application, consider using custom classes with binary serialization.
• If XML is required for interoperability, consider using attribute-based structures for large amounts of data instead of element-based structures.

State Management

It may be necessary for the communicating parties in an application to maintain state across multiple requests. Consider the following guidelines when deciding how to implement state management:

• Only maintain state between calls if it is absolutely necessary. Maintaining state consumes resources, can affect the performance of your application, and can limit your deployment options.
• If you are using a stateful programming model within a component or service, consider using a durable data store such as a database to store state information, and use a token to access the information.
• If you are designing an ASMX service, consider using the ApplicationContext class to preserve state because it provides access to the default state stores for application scope and session scope.
• If you are designing a WCF service, consider using the extensible objects provided by the platform for state management. These extensible objects allow state to be stored in various scopes such as service host, service instance context, and operation context. Note that all of these states are held in memory and are not durable. If you require durable state, you can use the durable storage (introduced in the .NET Framework 3.5) or implement your own custom solution.

Contract First Design

Traditionally, developers have built services using a code first approach where they design the service based on requirements, and expose an interface suited to the code and the requirements. However, the contract first approach is becoming more popular as it can reduce the incompatibilities that may occur between disparate systems and a wide range of clients.

Contract first design is the process of designing the service contract in terms of the data, messages, and interface it will expose, and then generating the service interface code from the contract. From there, you can implement the code behind the service interface that performs the processing required. This allows you to concentrate on the format of the messages and the data types they use at the beginning of the process to maximize interoperability and compatibility.

You can use modeling tools to help you design the interface, such as the patterns & practices Web Service Software Factory: Modeling Edition (see https://msdn.microsoft.com/servicefactory). Alternatively, you can design the interface using XML, XSD, and schemas; and then use tools such as WSDL.exe with the /server switch to generate the interface definition. Message bus technologies such as Microsoft BizTalk Server encourage the use of contract first design principles

Principles to bear in mind when applying contract first design are the following:

• Working with XML schemas and data types means that you do not and cannot think in terms of platform-specific data types. This can make it more difficult to define the interface, but ensures maximum interoperability and compatibility. Where you require complex data structures, compose them from simple and standard XML types that all clients can use.
• Consider the platforms, clients, and systems that may interact with the service. Plan for any limitations in data types or formats that this may impose.
• Consider using tools to help you design the service contracts. This can considerably simplify and speed up the process.
• Collaborate with interested parties during the contract design process if possible. Others may have specific requirements or requests that would make the contract easier to use, more widely acceptable, and maximize reusability.

For more information about contract first design, see "Contract-First Service Development" at https://msdn.microsoft.com/en-us/magazine/cc163800.aspx.

Security Considerations

A secure communication strategy will protect sensitive data from being read when passed over the network; protect sensitive data from being tampered with; and, if necessary, guarantee the identity of the caller. There are two fundamental areas of concern for securing communications: transport security and message security. For maximum protection, consider combining transport and message security techniques.

Transport Security

Transport security is used to provide point-to-point security between the two endpoints, and the transport layer passes user credentials and claims to the recipient. Protecting the channel prevents attackers from accessing all messages on the channel. Common approaches to transport security are Secure Sockets Layer (SSL) encryption and Internet Protocol Security (IPSec). Consider the following when deciding whether to use transport security:

• Transport security uses common industry standards that provide good interoperability, and is usually faster for encryption and signing since it is accomplished at lower layers—sometimes even in the network hardware. However, it supports a limited set of credentials and claims compared to message security.
• If interactions between the service and the consumer are not routed through intermediaries, you can use transport security. If the message passes through one or more intermediaries, use message security instead. With transport security, the message is decrypted and then reencrypted at each intermediary it passes through, which represents a security risk.
• Transport security is a good choice for securing communication between a client and a service located on a private network such as an intranet.

Message Security

Message security can be used with any transport protocol. You should protect the content of individual messages passing over the channel whenever they pass outside your own secure network, and even within your network for highly sensitive content. Common approaches to message security are encryption and digital signatures. Consider the following guidelines when deciding whether to use message security:

• Consider implementing message security for sensitive messages that pass out of your secure network, such as services exposed over the Internet. However, keep in mind that message security generally has a higher impact on communication performance than transport security. You can use partial or selective message encryption and signing to improve overall performance.
• If there are intermediaries between the client and the service, use message security for sensitive messages because it guarantees end-to-end security. Intermediate servers will terminate the SSL or IPSec connection when they receive the message, and then create a new SSL or IPSec connection to pass it to the next server. Therefore, there is a risk that messages that do not use message security will be accessible on the intermediate server.

Technology Options

On the Microsoft platform, you can choose between two messaging technologies: Windows Communication Foundation (WCF) and ASP.NET Web Services (ASMX). The following sections will help you to understand the capabilities of each, and choose the one most appropriate for your scenarios.

WCF Technology Options

WCF provides a comprehensive mechanism for implementing services in a range of situations, and allows you to exert fine control over the configuration and content of the services. The following guidelines will help you to understand how you can use WCF:

• Consider using WCF in the following situations:
  • Communicating with Web services where you require interoperability with other platforms that also support SOAP, such as the J2EE-based application servers.
  • Communicating with Web services using messages not based on SOAP, such as applications that use formats such as Really Simple Syndication (RSS).
  • Communicating using SOAP messages and binary encoding for data structures when both the server and the client use WCF.
  • Building REST Singleton and Collection Services, ATOM Feed and Publishing Protocol Services, and HTTP Plain XML Services.
• Consider using WS-MetadataExchange in SOAP requests to obtain descriptive information about a service, such as its Web Services Description Language (WSDL) definition and policies.
• Consider using WS-Security to implement authentication, data integrity, data privacy, and other security features.
• Consider using WS-Reliable Messaging to implement reliable end-to-end communication, even when one or more Web services intermediaries must be traversed.
• Consider using WS-Coordination to coordinate two-phase commit transactions in the context of Web services conversations.

WCF supports several different protocols for communication:

• For services accessed from the Internet, consider using the HTTP protocol.
• For services accessed from within a private network, consider using the TCP protocol.
• For services accessed from the same machine, consider using the named pipes protocol, which supports a shared buffer or streams for passing data.

ASMX Technology Options

ASMX provide a simpler solution for building Web services based on ASP.NET and exposed through an Internet Information Services (IIS) Web server. ASMX has the following characteristics:

• Can be accessed over the Internet using only the HTTP protocol. Uses port 80 by default, but this can be easily reconfigured.
• Has no support for Distributed Transaction Coordinator (DTC) transaction flow. You must program long-running transactions using custom implementations.
• Supports IIS authentication, Roles stored as Windows groups for authorization, IIS and ASP.NET impersonation, and SSL transport security.
• Supports the endpoint technology implemented in IIS.
• Provides cross-platform interoperability.

Additional Resources

For more information on the topics covered in this chapter, see the following articles:

• "Data Transfer and Serialization" at https://msdn.microsoft.com/en-us/library/ms730035.aspx.
• "Endpoints: Addresses, Bindings, and Contracts" at https://msdn.microsoft.com/en-us/library/ms733107.aspx.
• "Messaging Patterns in Service-Oriented Architecture" at https://msdn.microsoft.com/en-us/library/aa480027.aspx.
• "Principles of Service Design: Service Versioning" at https://msdn.microsoft.com/en-us/library/ms954726.aspx.
• "Web Service Messaging with Web Services Enhancements 2.0" at https://msdn.microsoft.com/en-us/library/ms996948.aspx.
• "Web Services Protocols Interoperability Guide" at https://msdn.microsoft.com/en-us/library/ms734776.aspx.
• "Windows Communication Foundation Security" at https://msdn.microsoft.com/en-us/library/ms732362.aspx.
• "XML Web Services Using ASP.NET" at https://msdn.microsoft.com/en-us/library/ba0z6a33.aspx.