Archive for November, 2012


This is the second in a  three-part series on multi-tenancy within Windows Azure applications. Here is a breakdown of the topics for these posts.

Post 1 –  Tenants and Instances

Post 2 – Combinations of Instances and Tenancy

Post 3 – Tenant Strategy and Business Application

Part 2 – Combinations of Instances and Tenancy

SIST (Single-Instance, Single-Tenant) Architecture

SIST is the simplest of designs since there is only one tenant with their own dedicated instance.  It is typically however the most expensive option since all costs are 100% the responsibility of the one tenant.  Cost savings via shared commuting infrastructure is thus a non-entity in the SIST case.

Applying the analogy from our previous commuting example means one rider has their own dedicated train getting to work.  Since the driver has their own train they can leave their own personal belongings in the car between days of driving and not worry about anyone having access to it. Any member of their family can use it, but only members of their family.

Applying it to your Windows Azure application would mean one customer of your application with their own dedicated Azure instance.  Application state within the process is dedicated to that one application and no other companies can have access to it.

Single-Instance Single-Tenant

Single-Instance Single-Tenant

SIMT (Single-Instance, Multi-Tenant) Architecture

SIMT adds complexity to the SIST model by allowing more than one tenant to concurrently execute within one instance.  This design is the first step in realizing cost savings from sharing resources found in that instance between customers.

The commuting example applied here is that of a single train where many riders from different companies share the cost of using that one train.  However unlike SIST, there are times of the day when the seats are hard to find and a rider may need to wait for one to come open.  There is overhead incurred to load and unload many riders from different companies on and off the train.  Since the train is a common shared entity if a rider leaves their personal belongings in the train between rides there is a very good chance it will not be there the next day. Or it will be accessed by someone from a rival company.

Applying this analogy to your SIMT Windows Azure application would mean at times a customer may have to share and wait for resources during busy times.  The overhead of context switching occurs since the CPU will switch between processing threads specific to that company. Memory and CPU can be taken up by other tenants in the VM process space. If state is used it must protected and managed so that each customer’s state makes it appear to them they are running in a dedicated SIST environment.

Single-Instance Multi-Tenant

Single-Instance Multi-Tenant

MIST (Multi-Instance, Single-Tenant) Architecture

Moving from single-instance to multi-instance the MIST configuration has multiple instances of the application with one tenant per dedicated instance.  This is really SIST but with more than one instance of the dedicated application per customer.

Applying our commuting example there is still only one customer per one dedicated train but there are now multiple trains for that customer.  Like SIST this does not translate into any shared costs savings for the rider.  But since there are now multiple trains, each with one customer, there is now the potential for more money to be made by the railroad company.  This is because it can now extend dedicated train service to many riders of that company.

Just like SIST, within an MIST Windows Azure application a customer should not have to share or wait for process resources.  Each customer of the application has a dedicated Azure instance space each with their own copy of the application. This could be an issue with respect to upgrades since each individual instance will need to have its application updated when an update occurs.  Application state within the process is dedicated to that one customer so no other customers can have access to it.

Since MIST is an upgrade over SIST there is the added complexity of having multiple applications possibly accessing the same shared resources outside of the dedicated instance space. That could mean accessing a shared Azure queue or table in SQL Azure.   So while you can make intra-process assumptions on state, data, and processing based upon one dedicated customer per process, you need to be aware of inter-process operations where customers need to process resources outside of their dedicated instance space.

Multi-Instance Single-Tenant

Multi-Instance Single-Tenant

MIMT (Multi-Instance, Multi-Tenant) Architecture

MIMT rounds out the multi-tenant configurations by allowing multiple tenants to run in multiple application instances.  It offers benefits with “potential” increased scalability and availability.  I use quotes around the word “potential” because contrary to the popular misconception bigger is not always better.   One cannot always assume that the MIMT architecture is the ultimate performance choice. While MIMT allows the best opportunity for optimized performance, too many cooks can spoil the broth.

The applied commuting analogy we are using in this case would mean multiple trains carrying multiple riders from multiple companies.  This has the same issues as a shared single train in the SIMT case.  Additionally there is the added complexity of different train tracks and thus different routes.  This can be a subtle point of confusion if not managed correctly and can negate the benefit of having multiple trains going to the same end point.

For example suppose a family wants to travel to a certain destination and coming from different locations after work – sister from college, brother from high school, mom from her work, and dad from his work location. If their goal is to meet for dinner at a certain stop along the route they need to coordinate the times they leave and arrive. This ensures they optimize their travel cost and time to arrive as close together as possible. It would not be ideal if dad arrived an hour before everyone and had to wait for them all to show up over staggered times. Or if brother took a train in the other direction he would go hungry.  Design and coordination of activities is important in making this a successful dinner trip.  If done right, it is more efficient if they all take different trains to the same endpoint (SIMT) and don’t have to all drive a car (SIST) to a single train station to ride together.  If not done optimally with different trains from different locations it can actually take longer than all driving the car to one train.

Applying this analogy to a MIMT Windows Azure application should be a fairly simple transition in your mind. Like an SIMT Windows Azure application may have to wait for resource, incur context switching overhead, and have to carefully manage individual state.  There is an additional complexity involved that can either improve on the SIMT situation or actually end up with worse performance than SIMT.  This is the ‘hard’ part of MIMT applications and is really application dependent in nature.

MIMT is an architecture that lends itself to use concurrency-based architecture patterns (Event Based Synchronous, Lock, Guarded Suspension, Concurrent Data Access, etc.). This allows you to coordinate and isolated work across multiple instances or across different threads in the same instance. Client 2 in the diagram below has two threads of execution in the third Azure Web instance and one thread in both the second and fourth Azure Web instances.  The code for Client 2 needs to manage that complex concurrency.

Mulit-Instance Multi-Tenant

Mulit-Instance Multi-Tenant

Sounds complex? Well that’s because it can be.  So here are some guidelines for coordinating multiple clients across your multi-tenant Azure application:

  • Synchronize access to all persistent and state data
  • Avoid long periods of locking or blocking while accessing resources
  • Avoid state as much as possible. Being stateless allows code to run properly on any thread on any server.

If you must have state, manage state correctly and in a central location that all threads can access regardless of what server they are running (Windows Server AppFabric Cache or a central database state server).

Minimize serialization and try to avoid deadlock situations

Carefully ensure your processing is optimized over not only multiple threads but over multiple servers.

Ensure none of your processing makes any assumptions about which Azure instance or thread any of your code running.

Don’t assume specific threads or instances will be accessing state or persistent data from across multiple Azure instances.

The above list is not all-inclusive yet it gives you some considerations of which to be aware. It may give you an appreciation of the potential complexity of ensuring your Azure MIMT app runs optimally across many VMs for many customers.  Your code can run on any thread on any VM at any given time.  Code must be written to be flexible and able to easily transition to any thread on any instance on any VM with the data still maintaining its integrity.  Remember my earlier statement that your MIMT app can be more efficient than an SIMT app if done correctly.  Or it can actually run slower if not designed correctly.

Various combinations of tenancy can exist throughout the spectrum of instances and tenants at the different application layers.  The requirements of your application will be the key factors in these tenancy decisions.  Application components and data should properly manage more than one tenant having access to them.

In the next and final post we will look in detail at tenacy strategy and business application.

This is the first of a  three-part series on multi-tenancy within Windows Azure applications. Here is a breakdown of the topics for this post as well as the final two posts.

Post 1 –  Tenants and Instances

Post 2 – Combinations of Instances and Tenancy

Post 3 – Tenant Strategy and Business Application

Part 1 – Tenants and Instances

One of the prime economic motivations to running your application in a Cloud environment is the ability to distribute the cost of shared resources among multiple customers.  At least that’s what all the evangelists and marketing folks like to tell us, right?  But the harsh reality of life in the Cloud is that an application’s ability to run safely and efficiently with multiple customers does not just magically ‘happen’ simply by deploying it to Azure.  An application’s architecture must be explicitly and carefully designed to support running with multiple customers across multiple (VM) virtual machines.  Its’ implementation must prevent customer-specific processing from negatively affecting processing for other tenants.

Here I will attempt to simplify the concept of tenancy in Windows Azure by first defining tenants and instances. There are deeper levels this discussion could be taken as entire books have been written on multi-tenancy in shared computing/storage environment (which is what the Cloud is after all). So we will only touch the tip of the iceberg when it comes to the science of instance allocations per tenant and multi-tenant data access.   We will look at various configurations of instances and tenancy and how to structure your application and data.  And finally we will wrap up with some strategies for multi-tenancy and how business models relate to tenancy.

Tenancy

Let’s first agree upon a definition for the term tenancy and how it applies to single- and multi-tenancy.  There are a few different definitions of this concept that can cloud (no pun intended) one’s understanding. Some define it as the relationship of clients to application instances.  Wikipedia defines it as “…a single instance of the software runs on a server, serving multiple client organizations (tenants)”. While I appreciate this definition for the sake of practicality I want to expand upon it if I may.   There are architectures where you can also have “more than one instance of the software running on a server”.   How you provision tenants to those multiple instances is your choice. You could have one tenant (customer or company) per instance and all the users from that company dedicated to that instance.  Or you could have more than one customer sharing that one instance but logically partitioned from each other’s data and processing. Each of those customers (or tenants) can have one or more users accessing the application.

So for the sake of this paper let’s agree that tenancy refers to a “customer/billing” relationship.   Suppose we have a SaaS Azure application that we sell to ten different companies (our “customers”).  Each company has 5,000 employees using our Cloud application.  Using our billing-relationship definition if we sell that service to 10 different companies we don’t have 5,000 tenants. Rather we have 10 tenants because of our 10 customer/billing relationships.  Later in this paper we will look at more about this type of relationship in the Business Model and Tenancy section. 

Tenant Types

There are two types of tenant environments we will need to consider. The simplest type is a single-tenant application where one customer has 100% dedicated access to an application’s process space.  A single tenant application is much more predictable and stable by its nature since there will never be more than one dedicated customer at any point in time in that VM.  That customer has all of its users accessing that dedicated instance of the application.

Contrast that with a multi-tenant environment where more than one customer shares the application’s process space and data.   Due to requirements of security and performance isolation, it’s more difficult to build a multi-tenant application.  You may have to plan for added complexity and development/test time to synchronize access to shared data and resources.   For a Windows Azure multi-tenant application similar complexity is required to ensure data and resources do not get corrupted with multiple tenants in the same or multiple process spaces.

Realize there are variations on this theme.  You could theoretically have only one user per instance of a VM.  So if a company had 2000 users it would need 2000 VMs. Not a very practical application of resources or costs.  More realistically, with our tenancy definition of customer/billing relationship, you would have 2000 users from that same company sharing the same VM instance. So if you have 20 companies you’d have 20 VMs.  Each VM would handle one or more users for that specific company only.  Getting more complex in tenancy we could have more than one company sharing an instance. So if you had 100 companies, you could provision 10 of them per instance and thus use 10 VMs. Each company would share a VM with 9 other companies, and you would have 10 VM instances of this. Confused? Hold that thought. We will break this down with picture and stick figures (okay… maybe no stick figures!) as we progress in this article.

Why Multi-Tenancy?

Why would one want to go through the trouble of making a Windows Azure application multi-tenant? Wouldn’t it just be so much easier to make an application single-tenant and not have to worry about complicated synchronization and a more complex billing model? From an infrastructure and development standpoint, it’s the difference in cost vs. simplicity.  Whether making the choice of the simplicity of driving your own private car to work or reducing your commuting cost with other riders using the public mass-transit system you have to decide what’s right for you.  For various reasons you may decide not to share your ride with others and pay more for your own custom commuting environment.  Maybe your schedule is unique, or you are scared of what could happen health-wise or crime-wise being sharing a commuting vehicle with strangers. Or you don’t want your commute time to be affected in any way by other riders and like to leave your briefcase in the car each night when you get home.  So you choose to pay more and have a dedicated way to work.

But for those commuters who can’t or won’t spend the increased cost of private commuting the choice of shared public transportation is their best option.  It offers a simple pay as you go model where the cost you pay is minimized since it is distributed across many customers. So there is no right or wrong answer – it depends upon the rider requirements.

Compare this to a software standpoint where there could be various requirements – government regulations or demanding SLAs – that you need to support for your customers.  It’s a tradeoff, right?  You want the benefits of operation isolation and simplified design found with a dedicated single-tenant application.  But it will cost you more since you aren’t sharing the cost of the resources over many customers.  So the choice when deciding to use a multi- or single-tenant environment depends upon your application requirements.

Instances and Tenancy

In the case of Windows Azure a new instance means running on a new VM instance isolated from the other instances.  These instances can communicate with each other typically through queues or WCF calls.  Let’s see how these terms combine to define the different types of applications running under Windows Azure.

There are four possible combinations of instances and tenancy.  The standard convention is “INSTANCE-TENANT” format when describing these combinations. These are listed in approximate complexity of architecture with SIST being the simplest.

  1. SIST (single-instance, single-tenant)
  2. SIMT (single-instance, multi-tenant)
  3. MIST (multi-instance, single-tenant)
  4. MIMT (multi-instance, multi-tenant)

A quirky way that helps me remember these in terms of Windows Azure is to equate the following:

“Instance” = “Azure virtual machine (VM) instance”

And

“Tenant” = “Customers sharing of the VM”

So in my personal Azure terms, I like to define the remaining combinations as follows. These are listed in order of increasingly complex development.

1. SIST (Single-Instance, Single-Tenant)

  • “Only one Azure VM dedicated to one company”

2.  SIMT (Single-Instance, Multi-Tenant)

  • “Only one Azure VM shared among multiple companies”

3.  MIST (Multi-Instance, Single-Tenant)

  • “Many Azure VMs each dedicated to one company”

4.  MIMT (Multi-Instance, Multi-Tenant)

  • “Many Azure VMs shared among multiple companies”

 In the next post we will look in detail at all four of these combinations of tenancy.