Category: Azure IT Ops


Azure Dev Test Labs

Practically speaking…wow! Take a look at the new (Public Preview) features called Azure Dev Test Labs. It rocks! This will be an incredibly popular feature for our customers.  With Azure Dev Test Labs your development and test team can dynamically use a self-service provisioning mechanism to quickly deploy Windows or Linux test environments in the Azure cloud. There is a built in process to template specific types of resource allocations. Once provisioned, there is a way to utilize quotas and policies to manage the expense and usage of those resources. That way you make sure your team does not ring up unnecessarily large bills for unused or over-provisioned resources.

With the advent of Azure Resource Manager and resource group templates a process exists to repeatedly allocate entities in Azure using a full reproducible and repeatable method.   The same concept of a template exists in Azure DevTest labs to allow you to create an on-demand Dev-Test environment in a few simple strokes on the mouse.  Leverage these templates via source control across groups in your company or across the enterprise. You can use Visual Studio online along with Git (version control system), and GitHub (web-page on which you can publish your Git repositories and collaborate with others) to add private artifacts. Artifacts can be items like tools that you want to install on a VM, actions that you want to run on the VM, or applications that you want to test. Azure DevTest Labs ships with a lot of key artifacts to get going.  There is a blade on the Azure Preview Portal that asks for the GIT Clone URI and the folder path. The Azure DevTest Lab will speak to the Git repository and sync the content.

DevTest Labs can be worked into your deployment process where you can set a limit to manage the costs and lifetime of you Azure Dev/Test resources. There are also existing pools of VMs from which you can quickly draw from to expedite the setup of your development or test environment.

To manage security of who has access by using role-based access control, such as the DevTest Lab user role. This allows to manage access at different levels (i.e. subscription, lab level, etc.) to use and provision resources.

For more info on Azure Dev/Test Labs go to https://azure.microsoft.com/en-us/services/devtest-lab/

 

I met with a customer today and the subject of Azure Scale Units (SU) and the limitations to scaling-up Azure VMs was discussed.  So I want to provide a simple definition of this issue and how to work around it.

Scale Units

Azure VMs that are part of the same Cloud service have free access to each other and share the load balancer and a virtual IP (i.e. http://myapp.cloudapp.net).  A Cloud Service is bound to a single Scale Unit, which Azure uses to scale out any of the VMs in a Cloud service. VMs can only be resized to a size supported in the Scale Unit (SU) where the VM is deployed. At times the word “stamp” is used to refer to the same concept as an Azure scale unit.  You can view a stamp or a scale sectioned off group of hardware in the DC that works together.

As new hardware becomes available MS builds it into a new SU. Currently there are five SU types. But keep in mind these will evolve and change as new hardware and new data centers are added to the Azure Cloud.  Scale Unit 1 is the oldest hardware in the DC, while SU 5 is the newest.

  • Scale Unit 1: A0-A4 (original VM sizes) Basic VMs (No LB, no Autoscale) and can only scale between A0-A4)
  • Scale Unit 2: A0-A7 (like SU1 but adds A5-A7)
  • Scale Unit 3: A8/A9 (“HPC” VMs, with Infiniband)
  • Scale Unit 4: A0-A7 and D1-D14 (D’s series and all A0-A7)
  • Scale Unit 5: G1-G5
  • How do you know which scale unit you are using?
  • Go into the VM/Configure ta
  • Click on VM Size and drop down all sizes.
  • If you see A0-A4 so you can tell what SU (#1) you are using. So you cannot scale up to anything above S4 in this case.

The Problem

Azure has generational hardware in its datacenters. Scale Units are groups of hardware resources in an Azure DC.  Due to the progressive rollout of constantly improving VM types (such as A8-A9), not all stamps support all of the new hardware. So if you are in an older stamp, and you try to scale up by increasing the VM type, you may or may not be able to do so. This depends up on the age and hardware functionality of the particular stamp to which your original VM is assigned. For instance, all the stamps will support A1, but not all support the new A8 and A9 VMs, or the D- and G-series VMs.

The Solution

There is not a portal setting or a  “scale-unit” PowerShell parameter to control the stamp to which your VM is assigned .  So what should you do?

>> If you want to allocate a new VM and make sure you can move up to bigger Scale Units in the future:

  • To ensure you get a SU that will meet your needs for scaling up, ensure the first VM deployed in that Cloud Service (or legacy Affinity Group) is in the upper range. So if you want SU2, deploy an A5 or above (A6 or A7) and you will be in SU2 at that point for all subsequent allocations.

>> If you want to move an existing VM to a new bigger size that is not in your current Scale Unit:

  • If you are in SU1 and need to move to a VM size that is not in SU1 (say A5-A7 in SU2) you can’t change it directly from the UI. So find the OS disk name in the Usage Overview
  • Delete the VM but be sure to choose “Keep the Attached Disks”
  • Go to VM/Disks and make sure that disk is not attached to any other VMs
  • Go to Gallery and create a new VM using that saved OS Disk and select the upgraded size to which you want to scale up

Note that once you allocate a G-series VM you can only change the VM size (scale up) to another G-series VM). If you want an A or D series VM you need to delete the VM, save the OS disk, etc. The D-series is similar in nature but also includes A0-A7 for SU #4.

Closing Thoughts

The key here is planning ahead of time what possible up-sizing could occur and in what tiers. It’s a part of procurement design.  “Sizing up” is not a very common process and typically it is done manually due to possibly undersized planning estimates.  So if you feel there is a good chance you are going to possibly eventually need a D-series, and are initially allocating planning on allocating a A-series, you should allocate a D series (recommend do the lowest of D series, D1, just to get the absolute lowest monthly usage cost). Once you allocate, drop down to the A series and run from there.  Later, if you need to scale up to a D series you have that capability to do so in that S4 scale unit.  You don’t have this option with the G-series Scale Units, which does not contains any D or A series options.

 

Azure Scale Units

When scaling Azure based upon load, you can scale horizontally (add more compute instances) or vertically (increasing the size of the VM). Azure Auto-scaling functionality works with horizontal scaling.  For more details about these options see my previous blog post on “Cloud Scaling Patterns“.

Azure Scale Units (SU) apply to vertical scaling, where you are wanting to scale ‘up’ the unit on which your VM is partitioned.  As new hardware becomes available Microsoft builds them into a new “scale unit”. If scaling UP, there are some sizes you can’t scale up (resize) to depending upon what SU your VM is currently partitioned.  VMs can only be resized to a size supported in the SU where the VM is deployed. Currently there are five scale unit types:

  • Scale Unit 1: A0-A4 (original VM sizes) Basic VMs, no LB, no Autoscale, can only scale between A0-A4)
  • Scale Unit 2: A0-A7 (like SU1 but adds A5-A7)
  • Scale Unit 3: A8/A9 (“HPC” VMs, optimized networking with Infiniband)
  • Scale Unit 4: A0-A7 and D1-D14 (D’s series with SSDs and better CPUs, and all A0-A7)
  • Scale Unit 5: G1-G5 (monster powerful VMs up to 32 core Xeon CPUs/448 GB RAM/6596 GB SSD storage/64 data disks)

How to check the SU unit a VM is currently allocated

  1. VM/Configure tab
  2. Click on VM Size and drop down all sizes. Here, you see A0-A4 so you can tell what SU (#1) you are using. So you cannot scale up to anything above S4 in this case.

How to choose scale unit

  • To ensure you get a SU that will meet your needs for scaling up, ensure the first VM deployed in that Cloud Service (or legacy Affinity Group) is in the upper range. So if you want SU2, deploy an S5 or above (S6 or S7) and you will be in SU2 at that point for all subsequent allocations
  • When you deploy a smaller size, like A2, you can get put into many different scale units. You won’t necessarily be in SU1.

How to move to a bigger size VM that is not in your current SU:

  • If you are in SU1 and need to move to a VM size that is not in SU1 (say A5-A7 in SU2) you can’t change it directly from the UI
  1. Note the OS disk name in the usage overview
  2. Delete the disk but “Keep the Attached Disks”
  3. Go to VM/Disks and make sure that disk is not attached to any other VMs
  4. Go to Gallery and choose that OS Disk and select the upgraded size you want to upgrade it to.

Network Security Groups (NSG) are an improvement upon VM endpoint Access Control List (ACL) rules. They provide network segmentation within an azure VNET at the infrastructure management level. This is an improvement over the earlier endpoint protection where you can define an ACL to define rules and permit or deny ingress traffic to the public IP port. ACLs only apply to incoming traffic, whereas NSGs apply to both ingress and egress traffic.

  • You must first remove an ACL from any endpoints on a VM to use NSGs. The two (ACL and NSG) cannot co-exist on the same endpoint.
  • Each VM endpoint has both a private and a public port. The private port is used by other VMs in the subnet to communicate with each other and does not use the Azure load balancer. The public port is exposed to the implicit Azure load balancer for ingress Internet traffic. ACLs apply to the public port. For a VM endpoint you’ll need to ensure that the VM firewall allows data to be transferred using the protocol and private port corresponding to the endpoint configuration
  • Today there is no support in the portal for NSGs. It all needs to be done via the REST API or PowerShell.
  • NSGs offer full control over incoming/outgoing traffic into a virtual machine in a VNET
    • NSGs do not work with VNETs that are linked to an affinity group.
    • Filters requests before they hit VM (ingress) and before they go out to Internet
  • Provides a logical DMZ for database and app servers to be protected
    • Web proxies (mask actual incoming and outgoing IP addresses) and DNS servers are usually put in DMZ since they are exposed to the internet. NSGs can logically create this type of abstraction for your VNET.
  • NSG provides control over traffic flowing in and out of your Azure services in a subnet or a VNET as a set of ACL rules
    • Are applied to all VMs in a subnet in a VNET in the same region. VMs must be in a regional VNET.
    • NSGs can be applied to an individual VM as well.
    • To give additional layers of protection you can associate a NSG to both a VM and another NSG to the VMs subnet
      • Ingress traffic the packet goes through the access rules first specified in the subnet, then secondly by the VM NSG rules
      • Egress traffic goes through the VM rules first, then the NSG rules in the subnet.
    • You can now control access coming in and out of the Internet to your VMs for an entire subnet via a since ACL rule
    • ACL rules on multiple VMs can be quickly changed in one operation vs. having to go into the firewall on each VM individually. You make the changes at the NSG level and don’t even need to go into the VM.
  • Azure provides default NSG rules around the VNET, and access to the internet, that you can modify if needed. It also provides default ‘tags’ to identify the VIRTUAL_NETWORK, AZURE_LOADBALANCER and the INTERNET. These tags can be used in the Source/Destination IP address to simplify the process of defining rules.
  • Defined rules are prioritized in their execution (as in the case in most business rules engines) so that the most importantly prioritized rules are executed first.
  • You define the rules (in PowerShell or REST) using the following fields
    • Priority, Access Permissions, Source IP/ Source Port, Destination IP/Destination Port, Protocol

For more information on NSGs refer to “About Network Security Groups” at https://msdn.microsoft.com/library/azure/dn848316.aspx.

Take a look at my new book on the topic of Azure Automation (part of the Azure Essentials series from Microsoft Press).  http://blogs.msdn.com/b/microsoft_press/archive/2015/03/06/free-ebook-microsoft-azure-essentials-azure-automation.aspx    This is the PDF format released ahead of Epub and Mobi, which will  be released later in the month.

Microsoft Azure Essentials: Azure Automation (ISBN 9780735698154), by Michael McKeown. This is the second ebook in Microsoft Press’s free Microsoft Azure Essentials series. Future ebooks will cover specific Azure topics, such as Azure Machine Learning, Azure Websites for Developers, and others.

New! I just released my fourth full-length Pluralsight Azure Course, this one on Azure Automation. This new technology allows you to create, test, execute, and schedule PowerShell Workflow runbooks in the Microsoft Azure portal. This course defines the process of runbook management and deployment, script management, and global assets to help centralize common and reusable automation resources. You can find the course at http://pluralsight.com/training/Courses/TableOfContents/microsoft-azure-automation.

A customer yesterday asked me about why the Affinity Group setting is suddenly no longer required as of a week or so ago to build a private VNET. It turns out that with the new capabilities of Azure VNETs an affinity group is no longer needed. The reason is the VNets can now span data centers within a region. A new VNet ‘type’ – Regional VNET – has been created to overcome some of the old limitations.

This makes perfect sense if you think about it the whole goal of an affinity group is to maximize locality to minimize latency between resources in the VNET. The physical implementation of the logical affinity group entity results in servers within one DC being put as close physically to each other as possible (taking into account fault and upgrade domains). But now that the VNets are no longer constrained to one DE the whole concept of allocating network resources close to optimize access time goes out the window.

Now instead of specifying an affinity group in the network config file you specify a Location (maps to ‘region’ – not sure why they added yet another term!) in the VirtualNetworkSiteName entity.

In an existing VNet you cannot convert an existing affinity group to a locality without recreating the network. However you can join an legacy VNet to a Regiona VNet and it will work fine.

However, an affinity group still is in use for storage and PaaS Cloud Services/IaaS VMs).

In this post I will discuss the resolution of IP and DNS addresses between IaaS VMs.

The key to optimizing and understanding the use and resolution of internal IP vs. external IP addresses is whether or not the client and server VMs are both in the same VNET. Either way, whether you use the internal IP / DNS Cloudapp.net to access a VM, if the client and server VMs are in the same VNET, the implicit (or explicit) DNS for the VNET realizes we are referring to a VM in the VNET and routes it to the internal IP. So this is not an issue as long as the VMs are contained in the same VNET (can be different or same subnets does not matter).

However if the VMs are not in the same VNET, then when using the DNS name the internal DNS router will send the request out to the public internet can back in through the external firewall.

As an example let’s look at a Web Role in one cloud service, and a SQL Server VM in another (remember IaaS VMs are contained within a cloud service ‘container’ too). The Web Role would communicate with the SQL Server by using the SQL Server’s publically accessible endpoint (i.e. mysqlserver.cloudapp.net:1433). You could make this more secure by adding an ACL to the SQL Server cloud service, but that is not depicted in the diagram.
ConnectingWithIPs

By contrast, if the Web Role cloud service and SQL Server VM cloud service are in the same Virtual Network, then the Web Role would communicate with the SQL Server instance using a direct (private) IP address on the network (e.g. 10.20.0.2:1433). Windows Azure’s built-in DNS does not span cloud services, even within a Virtual Network. So, to get DNS resolution, one would need to deploy their own DNS server or update a HOSTS file on the calling server (in this case the web server). Or, just go with IP address and forget about DNS resolution.

ConnectingWithVNET

One of the first tasks you may want to try after configure your Azure VNET and adding VMs to it is to try to ping them from either another VM on the VNET or a client PC.

By default the ICMP protocol used by ping is turned off.  So you can’t ping your VM unless you allow the ICMP Protocol. To enable ICMP remote desktop into your VM and using the Windows Firewall with Advanced Security create a custom inbound rule. Choose the ICMP protocol and enable all inbound connections.  Depending upon whether you are sourcing your ping request from VMs within the same VNET or not will determine if you can ping the external IP or not from outside the VNET.

Within Azure IaaS exists the often overlooked, yet important, subtleties of Virtual Machine (VM) Disks and Images.  Both of these allow you to generate Azure IaaS VMs using differing strategies.  In this post, I will clear up misunderstandings on the relationship between the two and how they map into the VM and VHD model for Azure.

Creating and Uploading VHDs

Azure VMs use Hyper-V fixed format VHDs.  So if you upload a dynamically formatted VHD it will be converted to fixed format. This may cause an unexpected bloat during the upload and yield a size greater than the 127 GB disk size limit imposed for Azure IaaS VM images. Thus it is recommended that you create a fixed format VHD before you upload it to Azure and don’t rely upon the conversion from dynamic to fixed format during the upload process.  VHDs can be uploaded using a PowerShell cmdlet Add-AzureVhd or using the CSUPLOAD SDK utility.

To generalize the VM Image, you must first run the Sysprep utility and check the Generalize checkbox.  The Sysprep utility sterilizes a VHD by removing system-specific data.  This permits you to use it generically as an image base for more than one VM instances.   So if you run Sysprep on a VHD you can load it as an Azure VM Image.

Note: Make sure that file C:\windows\system32\Sysprep\unattend.xml is removed from the location before you run Sysprep. If you don’t, the VHD may never load successfully into an Azure VM in later steps when you convert the VHD to a VM image.

There are situations when you don’t want to run Sysprep on a VHD since in some cases making that disk generic can render it useless for its desired application. For instance, you may have a Domain Controller on your VHD and executing Sysprep would make it ineffective as a DC. You will also most likely not be creating duplicate VMs from an image of a DC and therefore not need to run Sysprep.  If you don’t Sysprep the VHD can only load as an Azure VM Disk. This serves as a specific base type for a VM. It is not a generic image from which you will be creating multiple generic images.  In summary:

  • VM Disks are sterile images from Sysprep and create generic Azure VMs
  • VM Disks are not sterilized with Sysprep and create unique Azure VMs

Creating Azure VM Images

To create a generic Azure VM Image from a Sysprep’d VHD you will enter the blob URL of the location where you uploaded the VHD.  For a VM Image it is required that you check the box that says “I have run Sysprep on the virtual machine associated with this VHD”.

CreateVMImage

Using PowerShell you can also create a VM disk from a VHD using Add-AzureVMImage.

Creating Azure VM Disks

To create a unique Azure VM Disk from a VHD you will enter the blob URL of the location you uploaded the VHD. You are not given a Sysprep option since a VM Disk is not generic. It is based upon a specific VHD from which there will probably be only one instance.  Check “The VHD contains an operating system” if this is an OS disk.

CreateVMDIsk

Using PowerShell you can also create a VM disk from a VHD using Add-AzureDisk.

Once you have converted the VHDs into either Azure Disks or Images you can create Azure VMs from them.

Creating Azure VMs from Azure VM Image

You can create one or more Azure VMs from an Azure VM Image. Once you create an Azure VM Image (from a VHD) you will see your image(s) by clicking My Images in the Create a Virtual Machine wizard. You select your image just like you would one of the pre-defined Platform Images from the Gallery.

CreateVMfromMyImage

Creating an Azure VM from Azure VM Disk

You can create an Azure VM from an Azure VM Disk. Once you create an Azure VM Disk (from a VHD) you will see you disk(s) by clicking My Disk in the Create a Virtual Machine wizard. You select your disk just like you would one of the pre-defined Platform Images from the Gallery.

CreateVMfromMyDisk

 

Azure VM Best Practices

Here are a few tips when work with Azure VMs and VHDs:

  • Make sure when you attempt to create an image from a VHD that if you have previously done it before you delete the Azure Cloud Service that Azure creates for you when a VM is deleted.
  • Make sure you delete the link between the VHD and a VM by deleting the VM but retain the VHD. You can delete the VHD as well when doing this but you will have to upload the VHD again to blob storage which may take considerable time. Unless you are updating with a new VHD, deleting the VHD with the link is not desirable.
  • Azure will create its temporary drive on the D drive. So if you are uploading a VHD with a logical D drive then D: will be renamed to the E: drive. If you have any logging or application functionality that uses the D drive as a permanent drive you will need to change the logic of your application since there is no way to change the drive letter of the Azure temp drive to anything other than D.
  • If you take a VHD image that contains more than just the C: logical drive (say J: and K: drive as well) when converted to an Azure VM all logical drives will still exist in Azure. But they will be part of the one VHD file at the file system level. In other words, you won’t see one VHD for each logical disk on the volume.
  • Using PowerShell you can also create a VM Image from a VHD using Add-AzureVM.
  • Once a VM is created you can attach additional disks to your VM. It is a best practice for disks that will hold data, such as SQL Server and AD log files, to have them on a permanent drive that is not the main C: drive. The C: drive should be reserved for the OS and application installs. Utilize the other disks for log files and backups.
  • When creating a VM as a part of a VNET, open any firewall ports on the VM to allow traffic in or out.  For instance, you can open port 1433 for SQL Server to communicate with that VM
  • Create the virtual machine inside an affinity group to reduce latency with storage services and any networks of which it may be configured.

Do you agree with these best practices? Are there any that I have missed? Let me know in the comments below.