Azure MVP x 2!

Received my 2nd consecutive Azure MVP award today from Microsoft!   I love how Microsoft takes care of its partners… this is an awesome program!  Anyone considering becoming an MVP for sure it’s worth the time and effort.  For  some of my thoughts on becoming an MVP refer to “How to Become an MVP” https://michaelmckeownblog.wordpress.com/2014/04/22/how-to-become-an-mvp

And a special thanks to my manager Jeff Nuckolls and Aditi for supporting me in this program time and moneywise over the past year!

Azure Scale Units

When scaling Azure based upon load, you can scale horizontally (add more compute instances) or vertically (increasing the size of the VM). Azure Auto-scaling functionality works with horizontal scaling.  For more details about these options see my previous blog post on “Cloud Scaling Patterns“.

Azure Scale Units (SU) apply to vertical scaling, where you are wanting to scale ‘up’ the unit on which your VM is partitioned.  As new hardware becomes available Microsoft builds them into a new “scale unit”. If scaling UP, there are some sizes you can’t scale up (resize) to depending upon what SU your VM is currently partitioned.  VMs can only be resized to a size supported in the SU where the VM is deployed. Currently there are five scale unit types:

• Scale Unit 1: A0-A4 (original VM sizes) Basic VMs, no LB, no Autoscale, can only scale between A0-A4)
• Scale Unit 2: A0-A7 (like SU1 but adds A5-A7)
• Scale Unit 3: A8/A9 (“HPC” VMs, optimized networking with Infiniband)
• Scale Unit 4: A0-A7 and D1-D14 (D’s series with SSDs and better CPUs, and all A0-A7)
• Scale Unit 5: G1-G5 (monster powerful VMs up to 32 core Xeon CPUs/448 GB RAM/6596 GB SSD storage/64 data disks)

How to check the SU unit a VM is currently allocated

1. VM/Configure tab
2. Click on VM Size and drop down all sizes. Here, you see A0-A4 so you can tell what SU (#1) you are using. So you cannot scale up to anything above S4 in this case.

How to choose scale unit

• To ensure you get a SU that will meet your needs for scaling up, ensure the first VM deployed in that Cloud Service (or legacy Affinity Group) is in the upper range. So if you want SU2, deploy an S5 or above (S6 or S7) and you will be in SU2 at that point for all subsequent allocations
• When you deploy a smaller size, like A2, you can get put into many different scale units. You won’t necessarily be in SU1.

How to move to a bigger size VM that is not in your current SU:

• If you are in SU1 and need to move to a VM size that is not in SU1 (say A5-A7 in SU2) you can’t change it directly from the UI

1. Note the OS disk name in the usage overview
2. Delete the disk but “Keep the Attached Disks”
3. Go to VM/Disks and make sure that disk is not attached to any other VMs
4. Go to Gallery and choose that OS Disk and select the upgraded size you want to upgrade it to.

Quick Hits on Network Security Groups (NSG)

Network Security Groups (NSG) are an improvement upon VM endpoint Access Control List (ACL) rules. They provide network segmentation within an azure VNET at the infrastructure management level. This is an improvement over the earlier endpoint protection where you can define an ACL to define rules and permit or deny ingress traffic to the public IP port. ACLs only apply to incoming traffic, whereas NSGs apply to both ingress and egress traffic.

• You must first remove an ACL from any endpoints on a VM to use NSGs. The two (ACL and NSG) cannot co-exist on the same endpoint.
• Each VM endpoint has both a private and a public port. The private port is used by other VMs in the subnet to communicate with each other and does not use the Azure load balancer. The public port is exposed to the implicit Azure load balancer for ingress Internet traffic. ACLs apply to the public port. For a VM endpoint you’ll need to ensure that the VM firewall allows data to be transferred using the protocol and private port corresponding to the endpoint configuration
• Today there is no support in the portal for NSGs. It all needs to be done via the REST API or PowerShell.
• NSGs offer full control over incoming/outgoing traffic into a virtual machine in a VNET
  • NSGs do not work with VNETs that are linked to an affinity group.
  • Filters requests before they hit VM (ingress) and before they go out to Internet

• Provides a logical DMZ for database and app servers to be protected
  • Web proxies (mask actual incoming and outgoing IP addresses) and DNS servers are usually put in DMZ since they are exposed to the internet. NSGs can logically create this type of abstraction for your VNET.
• NSG provides control over traffic flowing in and out of your Azure services in a subnet or a VNET as a set of ACL rules
  • Are applied to all VMs in a subnet in a VNET in the same region. VMs must be in a regional VNET.
  • NSGs can be applied to an individual VM as well.
  • To give additional layers of protection you can associate a NSG to both a VM and another NSG to the VMs subnet
    • Ingress traffic the packet goes through the access rules first specified in the subnet, then secondly by the VM NSG rules
    • Egress traffic goes through the VM rules first, then the NSG rules in the subnet.
  • You can now control access coming in and out of the Internet to your VMs for an entire subnet via a since ACL rule
  • ACL rules on multiple VMs can be quickly changed in one operation vs. having to go into the firewall on each VM individually. You make the changes at the NSG level and don’t even need to go into the VM.
• Azure provides default NSG rules around the VNET, and access to the internet, that you can modify if needed. It also provides default ‘tags’ to identify the VIRTUAL_NETWORK, AZURE_LOADBALANCER and the INTERNET. These tags can be used in the Source/Destination IP address to simplify the process of defining rules.
• Defined rules are prioritized in their execution (as in the case in most business rules engines) so that the most importantly prioritized rules are executed first.
• You define the rules (in PowerShell or REST) using the following fields
  • Priority, Access Permissions, Source IP/ Source Port, Destination IP/Destination Port, Protocol

For more information on NSGs refer to “About Network Security Groups” at https://msdn.microsoft.com/library/azure/dn848316.aspx.

Affinity Groups – Kiss them Goodbye

Well it’s official!  Even though you probably have read the article “Kardashians to Kick Bruce and their Affinity Groups to the Curb” in People magazine, you were not really sure what to believe until you saw the official word from Microsoft.  Check out the content “About Regional VNets and Affinity Groups” at https://msdn.microsoft.com/library/azure/jj156085.aspx.   Improvements to the Azure virtual network infrastructure that have made communication between different parts of the Azure DC very quick.  No longer do you need to explicitly group resources in close proximity to each other to minimize latency between items like VNETs, VMs, and storage.

 

 

Aditi Dir of Cloud Services Jeff Nuckolls and Azure Internet of Things (IoT)

Take a few moments to check out this awesome installment of my favorite podcast, The Azure Podcast. This installment features our Director of Cloud Services, and my good buddy Jeff Nuckolls, sharing about the Internet of Things (IoT) and how he has positioned Aditi to handle this up and coming Cloud technology. IoT is all about the interconnection of intelligent machines transmitting massive amounts of data using various protocols into the Cloud. Once there, the data can be analyzed and converted into directives within a command-and-control relationship sending real-time commands back to the intelligent machines. In turn, these intelligent machines will modify their behavior and settings to compensate for factors identified from the data to optimize their efficiency or maximize safety. For example, a train traveling at high speeds with sensors up and down its frame can provide sensor data which in turn results in commands being sent back to modify its drag on the rails or its speed based upon incline, weather, and wind.

For a really interesting short (3-minute) very interesting video of what GE is doing in the world of intelligent machines and the IoT, check out GE Software: The Industrial Internet in the Real World..

Part 6 – Big Data in Azure with HDInsight

This is the sixth post in the Microsoft Azure Administration mini-series hosted primarily off the Aditi Technologies blog about Big Data with Microsoft HDInsight. It parallels my just-released Pluralsight course entitled Microsoft Azure New Administration Features (March 2014)

My company, Aditi Technologies, is working in parallel with me to discuss the admin “topic of the week” in this series on Microsoft Azure New Administration Features. So no need to rehash the work they are doing. For a highlight of the Azure HDInsight information that is in the actual Pluralsight course, go to http://blog.aditi.com/cloud/learn-from-expert-azure-hdinsight.

White Paper: Maximizing SQL Server On Windows Azure IaaS

Aditi, my company, has just published a whitepaper I wrote (originally for MSDN) entitled “Maximizing SQL Server On Windows Azure IaaS”. To access the entire paper, go to http://www.aditi.com/Resources/Whitepaper/Maximizing-SQL-Server-On-Windows-Azure-IaaS/index.html and enter a few brief pieces of info about yourself to download the entire paper.
I have included the Introduction section here to give you an idea of what it includes.

Introduction

I have divided the content in this paper into two distinct parts to cater to both the neophyte and the initiated IT Professional in the Windows Azure IaaS VM environment. The purpose of the first section is to get past the initial hurdle of understanding the basics of installing and configuring SQL Server in Azure IaaS. It is targeted at the IT person who has very limited experience, if any, with SQL Server on a Windows Azure VM (Azure IaaS). The end result of this section is to get a single database up and running in the Azure Cloud. Once this baseline of knowledge is established, in the second section I will provide some advanced topics in the manner of best practices and recommendations to help optimize the performance and increase the availability of SQL Server on Azure IaaS. So if you already have a basic knowledge of SQL Server on IaaS VMs you can review section one or just skip it entirely and go to the advanced topics of the second section as there is not a dependency between the different sections.

The first section opens discussing how to configure and manage Azure disks and images. The Azure disks map to supplemental disks in Azure IaaS VMs and are used for the SQL Server data and log files. There are four primary choice for installing and configuring SQL Server in Azure IaaS to ensure you have a basic working configuration. For instance you can use an existing SQL Server image from the Azure VM Gallery. You can create a base Windows Server image and do a manual installation of SQL Server. You can forklift an existing VHD image with SQL Server already installed up into the Cloud. Or you can capture an existing Azure VM image containing SQL Server and use it as a base for additional SQL Server VMs. I will cover some very basic SQL Server administration done remotely over the public Internet including using the Azure portal and the third party AzureWatch tool for monitoring your installation.

Within the second section we examine some key advanced topics for optimizing SQL Server 2012 performance and availability on Azure IaaS. Topics include properly managing your VMs and disks. We see how to optimize SQL Server disk performance within Azure storage where we discuss optimizing IOPS (Input Output Operations/Second). We look at how to optimize the use of azure storage and provisioning of disks to maximize the performance of SQL Server on Windows Azure IaaS VMs. We’ll discuss Azure geo-replication of data and how it relates to multi-disk layouts and storage IOPs. We look into host caching for data and OS disks, how to format disks optimally for azure, managing lifetime of disks, and how to handle disks larger than 1 TB. We will look at how to utilize some Azure IaaS concepts such as affinity groups, virtual networks and their associated subnet, and best practices around VM connectivity to maximize the performance and minimize latency of SQL Server on Azure IaaS.

Also within the second section we will look at how to utilize Azure IaaS best practices to increase availability, performance, and locality of your SQL Server VMs. Specifically I am referring to ways better manage your SQL Server VMs and to synergistically provide optimal configuration management for more than one VM hosting SQL Server. For example, availability sets provide specialized VM failover protection in case SQL Server goes down. You can utilize some Azure IaaS concepts such as affinity groups, virtual networks and their associated subnets, and best practices around VM connectivity to maximize the performance of SQL Server on Windows Azure IaaS VMs. We will expand upon the simple Internet SQL Server administration model in the first section to show how best to secure remote administration leveraging the benefits of a virtual network. More than one SQL Server can share and accept incoming requests via balancing of the client load using shared endpoints. Using a combination of availability sets, load-balancing endpoints, and affinity groups helps ensure that your application is always available and running as fast and efficiently as possible. We will look at ways to replicate data across more than SQL Server VM. Specifically we will focus on SQL Server 2012 AlwaysOn Availability Groups and Listeners as the primary data replication choice. I will present some design recommendations that are key to correctly implementing configuration of SQL Server AlwaysOn Availability Groups. As a part of AlwaysOn we discuss and how to properly configure SQL Server VMs using Azure Availability Sets and Windows Server Failover Clustering (WSFC), SQL Server Availability Groups, and Availability Group Listeners to manage failover between VMs transparently.

I would like to note that this paper will not painstakingly step you through the steps of how to complete a SQL Server setup of SQL Server AlwaysOn Availability Groups or their associated Availability Groups Listeners. This is documented fully on MDSN in most cases. Rather, I will tell you how to do this in Azure and show you the end result of the steps needed to set these entities up. At the end I will provide a concise summary of the high-level steps to configure SQL Server AlwaysOn Availability Groups and Availability Groups Listeners within the Azure IaaS environment.

Throughout this document I will endeavor to add value via best practices and share experiences I have been through. That means I will provide links to as much existing content whenever possible to save space and not clutter the paper. This gives you information on the topic which we are discussing as well as additional contextual information to enhance your understanding. My goal is to keep this document as concise as possible giving you the main points and key recommendations and not to document processes or content that is already developed.

SQL Server AlwaysOn Availability Groups, Clusters, and Availability Listeners in Azure IaaS

When reviewing your SQL Server HADR strategy for Azure IaaS here are main points of the different options: Windows Server Failover Cluster (WSFC), SQL Server AlwaysOn Availability Groups, Clusters, Availability Listeners, and Database Mirroring. WSFC provides infrastructure support and is required for both “AlwaysOn Availability groups” and “AlwaysOn Failover cluster”.

WSFC Clusters (sometimes called AlwaysOn Failover Clusters)

• WSFC Failover Cluster is used for HA
• WSFC provides redundancy at server level.
• Uses one shared (mirrored)  storage server as Single Point of Failure between to active (primary) and passive (secondary) nodes. Thus, no copy of Tx Log. Failover takes time to roll to next node.
• Primary is running (active) no SQL Server running in secondary. When primary fails secondary becomes active.
• Mirroring is an option on the cluster and does not do auto failover. You have to manually use the secondary at failover.   It will automatically start the secondary node if the active node fails (no heartbeat response in time and routes to secondary).
• Use the cluster name in the database connection string.  If failover then cluster will map it to secondary failover server

SQL Server AlwaysOn Availability Groups (SSAOGs)

• Can use more than two nodes
• Is a logical association across more than one SQL Server cluster nodes.
• Since TX log is always being updated the secondary servers are always up and running as active/active.
• No use for shared (mirrored) storage
• AAA Group constantly checks for failover –  if primary replica node is suddenly not responding AAL VIP points to a secondary and makes that the primary.
• Multiple secondary servers alternatively used for read-only queries, reporting, and backup.
• AlwaysOn Availability Group is used for HA and DR scenarios.
• Faster recovery and zero data loss using synchronous mode

SQL Server AlwaysOn Availability Group Listener (SSAOAGL)

• Use the AAAG Listener (VIP) in the database connection string.
• AAAGL abstracts out the actual database of the primary and secondary servers.
• Requires Windows Server 2012 to set up AAAGL in Azure

SQL Server Database Mirroring

• MS doesn’t recommend database mirroring, for any new development it shouldn’t be used. Also this feature will not be available in future versions of SQL Server(http://technet.microsoft.com/en-us/library/ms189852.aspx)
• Automatic failover in database mirroring is possible by specifying Failoverpartner attribute in connection string, I was not aware of this connection string parameter. So automatic failover is possible in database mirroring with some connection string parameters (http://technet.microsoft.com/en-us/library/ms175484.aspx).
• This link has some good details on HA and DR scenarios on SQL on Azure (IaaS) http://msdn.microsoft.com/en-us/library/jj870962.aspx