Snip/Shot by Toddy Mladenov

Accessing Windows Azure REST APIs with cURL

Tonight I was playing with cURL on my Mac wondering how easy would it be to develop few scripts to manage Windows Azure applications from non-Windows machine. As it turns out getting access to Windows Azure REST APIs was quite simple. Here are the steps I had to go though in order to be able to receive valid response from the APIs:

Set up Windows Azure management certificate from your Mac machine

The first thing I had to do is to create a self signed certificate that I can use to do the Service Management. Creating the cert with openssl (which is available on Mac) is quite simple - just type:

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout azure-cert.pem -out azure-cert.pem

During the creation openssl will ask you for all the necessary information like country name, organization name etc. and at the end will generate .pem file that contains the public and the private key.

In order to upload the certificate to your Windows Azure subscription using the Management Portal though you need to have the certificate in PKCS12 (or .pfx) format. Here is the openssl command that will do the work:

openssl pkcs12 -export -out azure-cert.pfx -in azure-cert.pem -name "My Self Signed Cert"

Now that you have the PKCS12 file you can go ahead and upload this to your Management Certificates using the portal.

Update: By writing this in the middle of the night I totally messed up what you need to do. PKCS12 you need if you want to enable SSL for your service. For management you only need the public key that you can export in .CER file. Here is the command that you use for this:

openssl x509 -outform der -in azure-cert.pem -out azure-cert.cer

Now you can upload the .CER to the Management Certificates section using the portal.

The initial set-up is done!

Using cURL to Access Windows Azure REST APIs

Now that you have the cert created and uploaded to Windows Azure you can easily play with the REST APIs. For example if you want to list all your existing hosted services you can use the List Hosted Services API as follows:

curl -E [cert-file] -H "x-ms-version: 2011-10-01" "https://management.core.windows.net/[subscr-id]/services/hostedservices"

where:

• cert-file is the path to the .pem file containing the certificate
• subscr-id is your Windows Azure subscription ID

Don't forget to specify the version header (the -H flag for cURL) else Windows Azure will return an error. As a result of the call above you will receive XML response with list of all the hosted services in your Windows Azure subscriptioin.

You can access any of the REST APIs by manually constructing the request and the URL as described in the Windows Azure Service Management REST API Reference.

I didn't get to any of my planned scripts but I can explore the APIs easily cURL.

Demystifying physicalDirectory or How to Configure the Site Entry in the Service Definition File for Windows Azure

Tweet

If you played a bit more with the sites configuration in Windows Azure you may have discovered some inconsistent behavior between what Visual Studio does and what the cspack.exe command line does when it relates to physicalDirecroty attribute. I certainly did! Here is the problem I encountered while trying to deploy PHP on Windows Azure.

 

Project Folder Structure

I was following the instructions on Installing PHP on Windows Azure leveraging Full IIS Support but decided to leverage the help of Visual Studio instead building the package by hand. Not a good idea for this particular scenario :( After creating my cloud solution in Visual Studio I ended up with the following folder structure:

+ PHPonAzureSol

     + PHPonAzure

          …

          - ServiceConfiguration.cscfg

          - ServiceDefinition.csdef

     + PHPRole

          …

          + bin

               - install-php.cmd

               - install-php-azure.cmd

          + PHP-Azure

               - php-azure.dll

          + Sites

               + PHP

                    - index.php

          + WebPI-cmd

               …

 

Where:

• PHPonAzureSol was my VS solution folder
• PHPonAzure was my VS project folder containing the CSDEF and CSCFG files
• and PHPRole was my VS project folder containing the code for my web role

The PHPRole folder contained the WebPI command line tool needed to install PHP in the cloud stored in the WebPI-cmd subfolder; the PHP extensions for Azure in the PHP-Azure subfolder; the installation scripts in the bin subfolder; and most importantly my PHP pages in Sites\PHP subfolder (in this case I had simple index.php page containing phpinfo()).

 

Configuring Site Entry in the CSCFG File

Of course my goal was to configure the site to point to the folder where my PHP files were stored. In this particular case this was the PHPonAzureSol\PHPRole\Sites\PHP folder if you follow the structure above. This is simply done by adding the physicalDirectory attribute to the Site tag in CSDEF. Here is how my Site tag looked like:

 

<Site name="Web" physicalDirectory="..\PHPRole\Sites\PHP">
     <Bindings>
          <Binding name="Endpoint1" endpointName="Endpoint1" />
     </Bindings>
</Site>

 

 

My expectation was that with this setting in CSDEF IIS will be configured to point to the content that comes from the physicalDirectory folder. Hence if I type the URL of my Windows Azure hosted service I should be able to see the index.php page (i.e. http://[my-hosted-service].cloudapp.net should point to your PHP code).

 

Visual Studio handling of physicalDirectory attribute

Of course when I used Visual Studio to pack and deploy my Web Role I was unpleasantly surprised. It seems Visual Studio ignores the physicalDirectory attribute from your CSDEF file, and points the site to your Web Role’s approot folder (or the content from PHPRole folder if you follow the structure above). Thus if I wanted to access my PHP page I had to type the following URL:

 

http://[my-hosted-service].cloudapp.net/Sites/PHP/index.php

 

Not exactly what I wanted :(

The reason for this is that Visual Studio calls cspack.exe with additional options (either /sitePhysicalDirectories or /sites) that overwrite the physicalDirectory attribute from CSDEF. As of now I am not aware of a way to change this behavior in VS.

Update (9-12-2011): As it seems VS ignores the physicalDirectory attribute ONLY if your web site is called Web (i.e. name="Web" as in the example above). If you rename the site to something else (name="PHPWeb" for example) you will end up with the expected behavior described below. Unfortunately name="Web" is the default setting, and this may result in unexpected behavior for your application.

 

cspack.exe handling of physicalDirectory attribute

Solution to the problem is to call cspack.exe from the command line (without the above mentioned options of course:)).

There are few gotchas about how you call cspack.exe using the folder structure that Visual Studio creates. After few trial-and-errors where I received several of those errors:

 

Error: Could not find a part of the path '[some-path-here]'.

 

I figured out that you should call cspack.exe from the solution folder (PHPonAzureSol in the above structure). Once I did this everything worked fine and I was able to access my index.php by just typing my hosted service’s URL.

 

How physicalDirectory attribute works?

For those of you interested how the physicalDirectory attribute works here is a simple explanation.

MSDN documentation for How to Configure a Web Role for Multiple Web Sites points out that physicalDirectory attribute value is relative to the location of the Service Configuration (CSCFG) file. This is true in the majority of the cases however I think the following two clarifications are necessary:

1. Because the attribute is present in the Service Definition (CSDEF) file the correct statement is that physicalDirectory attribute value is relative to the location of the Service Definition (CSDEF) file instead. Of course if you use Visual Studio to build your folder structure you can always assume that the Service Configuration (CSCFG) and the Service Definition (CSDEF) files are placed in the same folder. If you build your project manually you should be careful how you set the physicalDirectory attribute. This is of course important if you want to use relative paths in the attribute.
2. This one I think is much more important than the first one, and it states that you can use absolute paths in the physicalDirectory attribute. The physicalDirectory attribute can contain any valid absolute path on the machine where you build the package. This means that you can point cspack.exe to include any random folder from your local machine as your site’s root.

Here is how this works.

What cspack.exe does is to take the content of the folder configured in physicalDirectory attribute and copy it under [role]/sitesroot/[num] folder in the package. Here is how my package structure looked like (follow the path in the address line):

 

 

During deployment IIS on the cloud VM is configured to point the site to sitesroot\[num] folder, and serve the content from there. Here is how it is deployed in the cloud:

 

 

And here is the IIS configuration for this cloud VM:

 

Microsoft Windows Azure Development Cookbook Review

Recently I got my hands on the Microsoft Windows Azure Development Cookbook book written by Neil Mackenzie (@mknz), and honestly I was impress with the quality of the examples he gives into it. As a disclaimer I have to say that the book was sent to me by the publisher however there are no incentives for me to write this review. Once again I think the book provides very good hands on examples for the complete set of services that Windows Azure offers – starting with Windows Azure Storage, going through the Hosted Services and ending with SQL Azure and AppFabric.

The value I saw in the book is that 1.) it explains each of the services with real examples, and walks you step by step through what is happening, and 2.) is written from the customer point of view (Neil is a MVP, who is dealing with Windows Azure since it was announced at PDC08). I personally found the following three chapters very valuable even for me:

• Using the Shared Access  Signature for a container blob
• Using the retry policies with blob operations
• Autoscaling with the Windows Azure Service Management REST API

I am sure that I will refer to those three chapters very often for my own work on Windows Azure.

As a suggestion I would like to say to Neil that in the next edition I would be happy to see a section dedicated to the troubleshooting, debugging and profiling services on Windows Azure. From my own experience I can say that the majority of question I’ve seen have been in this area, and his experience with the platform as well as his independent view will allow him to provide really good troubleshooting tips and helpful workarounds.

I know Neil remotely, and we have interacted couple of time on MSDN forums or via Twitter. I can say that he is very knowledgeable about the topic, and you can learn a lot from the book and his personal blog.

 

I hope you will find the book a good resource, and kudos to Neil for writing it!

Configuring Tomcat Logging

If you looked at my recent posts I was playing with Java and Tomcat a lot, and trying to run those on Windows Azure. One of the things I wanted to achieve is to store Tomcat log files if folder different than the default Tomcat location. Surprisingly for me configuring Tomcat logging turned out to be not so intuitive. Let’s start with the basics…

 

Where are Tomcat Logs Stored?

By default Tomcat stores the log files under
$CATALINA_BASE\logs

Where CATALINA_BASE is the folder where Tomcat is installed. If you open that folder you will see something like this:

 

06/21/2011  02:49 PM  7,534 catalina.2011-06-21.log
06/21/2011  01:37 PM      0 host-manager.2011-06-21.log
06/21/2011  02:49 PM  1,872 localhost.2011-06-21.log
06/21/2011  02:49 PM      0 localhost_access_log.2011-06-21.txt
06/21/2011  01:37 PM      0 manager.2011-06-21.log

 

For more information what each file is about you can read the Tomcat Logging page.

My goal was to move those log files to a folder different from

$CATALINA_BASE\logs

 

How to Configure Tomcat Logging (Really How)?

If you search Google (or Apache’s web site) you will find out that in order to configure Tomcat logging you will need to either:

• edit the logging.properties file in $CATALINA_BASE\logs
• or create new logging.properties and set the java.util.logging.config.file System property to point to it

The easiest way to use the second approach is to set the Environment Variable

LOGGING_CONFIG=”-Djava.util.logging.config.file=[your_logging.properties_file_location]”

 

As you may expect the default logging.properties file is located in $CATALINA_BASE\conf.

Now, the hard part with this is that you CANNOT use Environment Variables in Java properties file. And of course this was what I really wanted to do. In general what I wanted to do is to use the %ROLEROOT% Environment Variable in the location path for all the log files (see What Environment Variables Can You Use in Windows Azure). The workaround to this problem is to set Java System property to use the Environment Variable (ie. the –D option for java.exe). Tomcat startup scripts use Environment Variable JAVA_OPTS for exactly this purpose:

 

set JAVA_OPTS=-DMY_SYSTEM_PROPERTY=%MY_ENVIRONMENT_VARIABLE

 

For Windows Azure specifically you can use the Variable tag in CSDEF:

 

<Variable name="JAVA_OPTS" value="-D[my_property_name]=%ROLEROOT%\[some_folder]" />

 

Next, in order to use the System property in the Java properties file you need to specify it in the following format:

 

${[my_property_name]}

 

Here is what I actually did. In CSDEF you set the Environment Variables as follows:

 

<Environment>
    <Variable name="TomcatLocalResourcePath"
              value="%ROLEROOT%\Approot\temp" />
    <Variable name="JAVA_OPTS" value="-DTomcatLocalResourcePath=%
              TomcatLocalResourcePath%" />
</Environment>

 

and in the logging.properties you use the Java System property as follows:

 

1catalina.org.apache.juli.FileHandler.directory = ${TomcatLocalResourcePath}


2localhost.org.apache.juli.FileHandler.directory = ${TomcatLocalResourcePath}

3manager.org.apache.juli.FileHandler.directory = ${TomcatLocalResourcePath}4host-manager.org.apache.juli.FileHandler.directory = ${TomcatLocalResourcePath}

 

This is all good, however it takes care only of the following log files:

 

catalina.2011-06-21.log
host-manager.2011-06-21.log
localhost.2011-06-21.log
manager.2011-06-21.log

 

What about localhost_access_log.2011-06-21.txt? The access log file in Tomcat is not configured via logging.properties file but in server.xml file. You can read more about the Access Log Valve (which controls the access log) on Apache’s web site. The simple thing that you need to do is to set the directory attribute on the Valve tag as follows:

 

<Valve className="org.apache.catalina.valves.AccessLogValve"
               directory="${TomcatLocalResourcePath}" 
               prefix="localhost_access_log." suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b"
               resolveHosts="false"/>

 

UNIX vs. Windows vs. Java Property Files

As a final note some clarification on when to use dollar sign $, percent % and dollar sign with curly braces ${} as I think it may be confusing for some people:

• As you know dollar sign $ is used to evaluate Environment Variables in UNIX. For example if you define the following Environment Variable in UNIX:
  
  setenv MYTEMPPATH /usr/temp
  
  you can use it later on as follows:
  
  setenv SOMEPATH $MYTEMPPATH/new
• In contrast Windows uses percent % to evaluate Environment Variables. Here is the same example for Windows:
  set MYTEMPPATH=C:\Temp
  
  and you can use it as follows:
  
  set SOMEPATH=%MYTEMPPATH%\new
  
  
• Property files in Java use the UNIX type of format but with curly braces to evaluate System properties. For example if you define the System property as follows:
  
  -DMyTempPath=C:\Temp
  
  you can use it in Java properties files as follows:
  
  some.property=${MyTempPath}

Running Java and Tomcat on Windows Azure VM Role

Just for an exercise I decided to see what is the experience for Java developers who would like to use Windows Azure as a cloud platform. Of course the easiest way to do this is to use the IaaS feature of Windows Azure – the VM Role.

Now, let’s go through the steps of preparing the VHD and deploying it to the cloud.

 

Preparing VM Role VHD with Java and Tomcat

In this particular case I used Hyper-V Manager on Windows Server 2008 R2 to prepare the VHD (for more details on VM Role you may refer to my previous post How to Deploy Windows Azure VM Role). As you may know at the time of this writing the only supported OS for VM Role is Windows Server 2008 R2. Here are the simple steps I went through:

1. Started Hyper-V Manager on the local machine (running Windows Server 2008 R2 of course) and created new Virtual Machine
   
   
2. Selected memory size
   
   
3. Next was the Networking interface
   
   
4. And finally the Virtual Disk (VHD)
   
   
   It is important to note that Windows Azure has limits on the Virtual Disk (VHD) size that you can mount in the cloud. Depending on how big your VM Role is (Extra Small, Small or Medium and above) you can mount different VHD sizes. For details look at the MSDN documentation for How to Create the Base VHD for a VM Role in Windows Azure. Oh, and yes, the screenshot is wrong
5. And I selected to install the operating system from the local DVD drive
   
   

Here the summary for my VM:

 

Configuring the Virtual Machine for Windows Azure

As mentioned in my previous post How to Deploy Windows Azure VM Role in order to run the VM in the cloud I have to install the Windows Azure VM Role Integration Components but before that I had to enable .Net 3.5 Framework.

 

As you can see from the screenshot above I selected the top node for .Net Framework 3.5.1 Features however this introduced some problems. It seems that WCF Activation feature occupies port 80 and I wasn’t able to run Tomcat on the default port. You don’t need WCF Activation in order to install and run the VM Role Integration Components so you don’t need to install it. Alternatively you can install the IIS Management Console and change the bindings for WCF to a different port (see How to change the TCP port for IIS service for details).

 

Installing Java and Tomcat on the Virtual Machine

Once I had the main operating system and Windows Azure Integration Components installed I had to install Java and Tomcat on the Virtual Machine. I decided to go with very simple configuration using Oracle’s JVM and Tomcat and downloaded the 32-bit/64-bit Windows Service Installer from Apache Tomcat’s web site. The reason I went with service installation was that I wanted Tomcat to start automatically when my instance is started.

Of course I changed the port for Tomcat from 8080 to 80 during the installation process.

 

 

Now that you have everything installed on your local Virtual Machine you can follow the instructions in How to Deploy Windows Azure VM Role to create a package and deploy the VHD to Windows Azure.

 

Some Tips

If I had more time to play with it I would have separated the base OS from the additional components (Java and Tomcat) in two different VHDs:

• Base Image will contain the OS only + the Windows Azure VM Role Integration Components
• Differencing Disk will contain the Java installation and the Tomcat installation

You can upload both the Base and Diff VHDs to Windows Azure and link them so that they are considered as one when your instance is started.

Using DebugView to Troubleshoot Windows Azure Deployments

Recently I fell into the same trap lot of other developers fall into – my Windows Azure role instances were cycling, and the logs were not helpful to find out what was happening. Fortunately I was able to connect via Remote Desktop to my instance, and use DebugView to troubleshoot the issue.

 

Here are the steps you need to go through to troubleshoot your deployment:

1. Configure your deployment for Remote Desktop access; for more details see my previous post Remote Desktop Connection to Windows Azure Instance
2. In Windows Azure Management Portal select your failing instance and click on the Connect button. Note that the button will not be enabled all the time; normally it becomes enabled when the Guest VM that is hosting your code is up and running (normally, just before your instance is about to become ready)
   
   
3. Download DebugView from Microsoft’s TechNet site. You may decide to download it directly on the VM if you want to deal with the Enhanced Security for IE. If not you can download it on your local machine and just copy it via the Remote Desktop session.
   
   
   
   I copied it under D:\Packages\GuestAgent but you can choose any other folder you have access to.
4. Start Dbgview.exe and select what you want to capture from the Capture menu. You can start and stop the capturing by clicking on the magnifying glass in the toolbar.
   
   

In my particular case DebugView was very helpful in troubleshooting command line script that I used as startup task.

What Environment Variables Can You Use in Windows Azure?

Yesterday I found the following forum post on Windows Azure Forums: How to get environment variable and information in startup task. At the same time I was prototyping some of our new features, and needed to achieve almost the same goal as in the forum post, hence I decided that it will be useful to publish some more details about what environment variables are available by default in Windows Azure, and how can you use them.

Environment Variables in Windows Azure

You may already know that you can define environment variables per role in Windows Azure Service Definition (CSDEF) file using the Environment tag:

 

<Environment>

     <Variable name="[my_name]" value="[my_value]" />

</Environment>

 

CSDEF schema allows you to put the Environment tag it either the Startup task tag or in the Runtime tag. It is important to know that environment variables that are defined for startup tasks are not available in the runtime and the reverse. Hence if you want to have environment variable that can be used in both the startup task and the runtime you need to define it twice.

Here an example for defining environment variables for startup tasks and for your role’s runtime:

 

<WorkerRole name="MyWorker">
     <Startup>
          <Task taskType="background" commandLine="my.cmd"

                               executionContext="limited"> 
               <Environment>
                    <Variable name="MY_ENV" value="my_value" />
               </Environment>
          </Task>
     </Startup>
     <Runtime>
          <Environment>
               <Variable name="MY_SECOND_ENV" value="my_value2" />
          </Environment> 
     </Runtime>

</WorkerRole>

 

Pre-Defined Environment Variables in Windows Azure

Of course the more interesting question is what are the pre-defined environment variables in Windows Azure, and whether you can leverage those. I wrote a simple Web Role that uses System.Environment.GetEnvironmentVariables(), iterates through all of them and prints them in a web page. Majority of the variables are standard Windows environment variables but here are some Windows Azure specific ones (Note: Those variables are specific to the process in which your role runs):

• @AccountUsername ** – this is the username you can use for Remote Desktop connection to the role instance
• @AccountEncryptedPassword ** – this is the encrypted password that you can use for Remote Desktop connection to the role instance. The password is encrypted using the SSL certificate for your Hosted Service
• @AccountExpiration ** – this is the timestamp when the Remote Desktop account expires
• @ConnectionString – is the diagnostics connection string for access to Windows Azure Storage. This is the configuration that you defined in the Service Configuration (CSCFG) file for your deployment. You can change this by modifying your CSCFG file.
• DiagnosticStore – this is the location where diagnostics information is stored locally before transferred to the Windows Azure storage account
• RdRoleId – this is the really unique identifier of the role instance. This information is not available in the Management Portal and it is build by concatenating the RoleDeploymentID and the RoleInstanceID.
  Example: eaaa6a386255466dada9dd158c5d4008.WebTest_IN_0
• RdRoleRoot – is is the same as the RoleRoot above
• RoleDeploymentID * – is the deployment identifier for your deployment and it is the same one you see in the Windows Azure Management Portal. This is the same for each role and role instance.
  Example: eaaa6a386255466dada9dd158c5d4008
• RoleInstanceID * – is the unique identifier of the role instance. The instance ID uses the RoleName as a prefix. 
  Example: WebTest_IN_0
• RoleName * – this is the role name.
  Example: WebTest
• RoleRoot – this is maybe the most important environment variable in Windows Azure, or at least the one that you need the most. It points to the root where your Windows Azure role code is placed. Very often you will want to access some resource files for your role using %RoleRoot%\AppRoot.
• __WaRuntimeAgent__ – this is the identifier for the runtime agent that is used by the Fabric controller to monitor your role. Again, it is very unlikely that you will need this one
• WA_CONTAINER_SID – this is the system identifier of the Windows Azure container. It is very unlikely you will need this one

 

All environment variables marked with asterisk (*) above are available as properties in Windows Azure Management Portal.

You can change the environment variables marked with double asterisks (**) above by clicking on the role in Windows Azure Management Portal and then on Configure Remote Access.

Also, most of the information stored in the environment variables is available through the Windows Azure Runtime APIs. However those environment variables are accessible from both your role’s startup tasks as well as from the runtime.

 

Windows Azure Full IIS Mode and Environment Variables

One thing to remember is that if you use Full IIS Mode in Windows Azure you will NOT be able to access the environment variables mentioned above. The reason for that is that you should use the standard IIS programing techniques to access your environment. You will know that you are running in Full IIS Mode if you have the Site element in your Service Definition file.

How to Deploy Windows Azure VM Role

For a project that I started in Windows Azure I needed access to the VM Role functionality and hence I decided to test the experience from customer point of view. Here are the steps I followed:

1. Request access to the VM Role functionality in Windows Azure
2. Preparing the VHD for VM Role 
3. Upload the VHD in the VM Image Repository on Windows Azure
4. Create a hosted service to host the VM Role
5. Upload Windows Azure Hosted Service Certificate
6. Create package and deploy the VM Role to the hosted service
7. Remote Desktop to the VM Role and install additional software if needed

Request Access to VM Role Functionality on Windows Azure

As you may already know Windows Azure VM Role feature is still in beta, and in order to use it you need to request access to the feature. You can do this using the following steps:

1. Go to Windows Azure Management Portal at http://windows.azure.com
2. Login using your Live ID, and click on Beta Programs in the left-side navigation
3. In the main pane you will see the list of Beta Programs that are available as well as checkboxes for each one
4. Click on the checkbox next to VM Role and click on the button Join Selected

 

After you complete this workflow you need to wait until your request gets approved. I had to wait approximately two weeks before I received notification that my request is approved. Keep in mind that each Beta Program has its own wait time because they have different quotas and are approved by different teams within Windows Azure. Those times can also change based on the number of requests received.

Important: Please read the notification email carefully! In the email you will find information how to enable VM Role features in the Visual Studio development environment. In essence you need to run one of the scripts below to add new registry key or just change the following [dword] registry key:

HKEY_CURRENT_USER\SOFTWARE\Wow6432Node\Microsoft\Windows Azure Tools for Microsoft Visual Studio 2010\1.0\VirtualMachineRoleEnabled​=1

 

• VM Role Features in VS2010 32-bit Registry Script
• VM Role Features in VS2010 64-bit Registry Script

The scripts above will enable the Add New Virtual Machine Role in the context menu in Visual Studio.

 

Preparing the VHD for Windows Azure VM Role

One important point I would like to emphasize when you prepare the VHD is the need to install the Windows Azure Integration Components. MSDN Library documentation is very explicit how you can do that using HyperV Manager.

Of course I don’t have the HyperV Manager on my Windows 7 machine and preparing the 64-bit image turned out to be a bit problematic (Note: At the moment Windows Azure supports only 64-bit VM Roles based on Windows Server 2008 R2).

The workaround I used was to install the Open Source VirtualBox software, and prepare the image with it. However you will need hardware virtualizations in order to create the 64-bit VHD.

 

Upload VHD Image to Windows Azure

The next thing you need to do is to upload the VHD file into Windows Azure image repository. I have created very simple VHD file with Windows 7 Ultimate using the Virtual PC in Windows 7.

In order to upload the VHD into the image repository you need to use the csupload.exe tool delivered as part of Windows Azure .Net SDK 1.3 or later. Here is how to use the tool:

1. Open Windows Azure SDK Command Prompt as Administrator
   
   
2. Execute the following command
   
   csupload Add-VMImage -Connection "SubscriptionId=[subscription_id]; CertificateThumbprint=[certificate_thumbprint]" -Description "[description]" -LiteralPath "[vhd_location]" –Name [vhd_filename] -Location "[azure_subregion]" -TempLocation %TEMP% -SkipVerify
   
   
   
   Where:
   subscription_id is the ID of the Windows Azure subscription where you want your VHD to be placed
   certificate_thumbprint is the thumbprint for the management certificate uploaded in the above mentioned subscription. Note: This is the Management Certificate used to manage Windows Azure services for this subscription and NOT the Hosted Service certificate you will need later on when you deploy the VM Role. 
   description is user friendly description you want to use
   vhd_location is the location of the VHD on your local machine or network
   vhd_filename is the name of the VHD file
   azure_subregion is the Windows Azure sub-region (you can find the names of the sub-regions in every offer description on Windows Azure web site – look for Data Transfer Details and expand the section)
   
   Few notes related to the csupload.exe tool:
   1.) Make sure that you have write permissions to the vhd_location folder else the command will fail during verification. If you do not have write permissions to the location folder you can use the –SkipVerify option or specify –TempLocation as I did above
   2.) At the time of this writing there was an issue with verification on 32-bit Windows 7, and if you are trying to upload the VHD from 32-bit Windows 7 machine you may encounter some errors. In this case use the –SkipVerify option.

Depending of the size of your VHD and the upstream speed of your Internet connection you may need to wait a while until the upload is complete.

Once the VHD image is uploaded to Windows Azure you will see it in the image repository in the Management Portal.

 

 

Create a hosted service to host the VM Role

You create the Hosted Service for VM Role the same way you create Hosted Service for Web or Worker role. One important thing you need to remember is to create the hosted service in the same location where you have uploaded the VHD. If you forget to do so you will receive the following (very explanatory) error message at deploy time:

 

HTTP Status Code: 400
Error Message: A parameter was incorrect. Details: One of the specified images cannot be used for creating the deployment. When you want to create a deployment that is based on a machine image, one of the following constraints must be met: (1) the hosted service and the image belong to the same affinity group, or (2) neither the hosted service nor the image belong to an affinity group, but their location constraints are the same, or (3) the hosted service belongs to an affinity group and the image has a location constraint equal to the location constraint of the affinity group. Here are the details about the current deployment: Image [image_name] does not belong to an affinity group. Its location constraint is [vhd_location].Hosted service [hosted_service_name] is not in an affinity group. Its location constraint is [hosted_service_location].
Operation Id: [some_operation_id]

 

I am not going to describe the Hosted Service creation process in details – you can do this quite easy from Windows Azure Management Portal.

 

Upload the Hosted Service Certificate

 

Before deploying VM Role you will need to upload Hosted Service Certificate (also used to enable SSL communication to your endpoint). Here is how you do that:

1. In Windows Azure Management Portal select the Hosted Services, Storage Accounts and CDN tab and then Hosted Services in the top left navigation
2. In the main pane expand the Hosted Service, into which you want to upload the certificate and select the Certificates
   
   
3. Click on the Add Certificate button in the ribbon
   
   
4. Browse for the certificate locally and type the password for it

Create Package and Deploy the VM Role to Windows Azure

In order to deploy the VM Role to Windows Azure you need to create a CSPKG package and CSCFG for your application and deploy those to the hosted service you created in the previous step. Here is how to do that:

1. In Visual Studio 2010 create a new Windows Azure Project
   
   
2. In the next step in the wizard it is important that you DO NOT create any Windows Azure Role – thus the project will only have CSPKG and CSCFG files created and no separate projects for the roles
   
   
3. Now that you have the project and have changed the Visual Studio configuration to show the Add->New Virtual Machine Role option you can click on the Roles node (or the Project node) in the project and add the VM Role
   
   
4. If you don’t have your Windows Azure account configured in Visual Studio you will need to do that first. You will need to use the same subscription you used to upload the VHD file for this configuration.
   
   
5. If you have your Windows Azure account already configured you can select the VHD from the drop down menu
   
   
6. In the configuration tab you can change the number of instances you want to have for the VM Role
7. You can also add endpoint for the VM Role and make it accessible from outside. I personally enabled port 80 and port 8080 for my test role
   
   
8. Once you are done with the configuration right-click on the project and select Publish
   
   
   
   You will see the publish dialog: 
   
   
9. Configure Remote Desktop for all the instances by clicking on the link right above the buttons
   
   
   
   Note: The certificate you choose in this window MUST be the same you uploaded to your Hosted Service in the previous section.
10. Click on the OK button and wait until the deployment is complete 
    
    
11. Once the deployment is done you can see your VM Role running in Windows Azure Management Portal
    
    

Remote Desktop to VM Role

You can Remote Desktop to the Windows Azure VM Role from Windows Azure Management Portal by selecting the instance you want to remote to and clicking on the Connect button in the ribbon:

 

 

You will be asked for the username and password – you should use the ones that you set up in step 9. in the section Create Package and Deploy the VM Role above. You may need to specify the machine name before the user name in order to login into the system – if you don’t have the actual machine name you can use the Hosted Service name and it will work:

 

 

I already had all the software installed on my VM and didn’t need to do any other installations.

 

Hope this step-by-step guide will help you bootstrap your Windows Azure VM Role deployment.

Remote Desktop Connection to Windows Azure Instance

Tweet

It has been long time since my last post related to Windows Azure. We have been concentrated on delivering all the new and exciting features for PDC2010, and of course we wanted to keep them secret.

In this morning’s PDC2010 keynote Bob Muglia revealed the secret, and I would like to start with my favorite feature – establish Remote Desktop connection to any Windows Azure instance. Here is the step-by-step guide how to enable the feature, and how to use it.

Configuring Windows Azure Deployment for RDP

The first thing I did is to create new basic Cloud Project in Visual Studio. I called it Hello RDP. Nothing new here. I modified the Default.aspx page just to make it say: “Hello, RDP!”.

Here are the steps (quite simple) that you need to go through to enable Remote Desktop in your deployment:

1. In Visual Studio right-click on the Cloud Project (Hello RDP in my case) and select Publish. You will be presented with the Deploy Windows Azure project dialog below:
   
   
   
   I have selected the Create Service Package Only option but you have the choice to deploy from Visual Studio to the cloud directly if you want.
2. Next click on the link Configure Remote Desktop connection above the OK button. You will see the Remote Desktop Configuration dialog where you can select the certificate you want to use to encrypt the credentials as well as the actual credentials you want to use to login to the instance and the expiration date for the password.
   
   
   In the Certificates drop down you can select a certificate from your local cert store or create a new one that will be sored locally. After you fill in all the information click on the OK button in the current dialog as well as in the deployment dialog.
   
   Your package will be created on the local hard drive.
   
   Note: If you do not have Hosted Service created and/or if you don’t have the certificate already uploaded through the Windows Azure Portal you should create the package locally and deploy it through the Portal. See next section for more details.

Configuring the Windows Azure Hosted Service

In the current implementation (as of PDC10) you need to create hosted service and upload the certificate before you do the deployment. There may be some changes for the official release of the new UI in order to make it more streamlined. Here is the current workflow:

1. Load the Windows Azure Portal and click on Compute, Storage & CDN in the navigation area
   
   
2. Click on the Compute Services node in the tree. In the grid you will see your subscription (or subscriptions if you have more than one)
   
   
3. Click on the New Hosted Service button, and you will get the dialog to fill in Hosted Service and Deployment information
   
   
   
   In the new Windows Azure Portal UI we combined the creation of Hosted Service and the Deployment in one step. However for RDP you should choose the “Do not deploy” option because you will receive error if you don’t have the certificate uploaded.
4. When you fill in all the information click the OK button and you will see your Hosted Service appear below your subscription.
   
   
5. Expand the Compute Services node in the tree and select the Service Certificates node
   
   
6. Click on Add Certificate to open the Upload Certificate dialog. Select the same certificate that you selected in the Visual Studio dialog when configuring your package, and click the Create button.
   
   Note: Make sure you select the right Hosted Service in the Target Hosted Service dialog
   
   
7. After it is uploaded you will see your certificate in the grid
   
   
8. Go back to the Compute Services view and click on New Production Deployment (or New Staging Deployment if you want to deploy to staging). Create a new Deployment dialog will appear where you can select the package and the configuration file you created in visual studio above.
   
   
9. After clicking OK you will see new line item in the grid that represents your deployment. In addition you will receive frequent updates what is happening with the deployment, and it will expand once the roles and instances are spun up. Really cute – isn’t it? We refresh the status every 10 seconds.
   
   
   Now, you just need to wait until your deployment is complete.

Connecting to the Windows Azure Instance

Once the deployment is complete the instances will be in Ready state, and when you select any one of them the Remote Desktop Connect button in the ribbon will light up.

 

When you click the Connect button you will be asked to download RDP file from the Windows Azure Portal (Note: Silverlight will also present you with security warning the first time you click on the Remote Access Connect button; you can select the checkbox in the warning if you don’t want to be warned in the future). You can select Open in the prompt for downloading the RDP file and the connection to the Windows Azure Instance will be established. Optionally you can save the RDP file locally for future use.

 

 

Voila! You are in!

 

 

Finally, it is highly recommended that you turn off Remote Desktop access to your Windows Azure instances when you don’t need it in order to avoid security risks. You can do this very easily from the Portal – just select the Role row, and uncheck the Remote Access Enabled checkbox:

 

Final Notes and Disclaimers About the Remote Desktop Feature

Couple of things I would like to mention at the end:

• First, the Remote Desktop feature is scheduled to be released at the end of the year, which means we are still working on it and there may be some changes in the workflow
• And second, the new Windows Azure Portal UI is also scheduled to release at the end of November and there may be some changes in the UI also. I have used the environment we use for the PDC demos to make the screenshots above, and if you are attending the conference you will see the same UI in the hands-on labs

I will try to update this blog post with any changes we make between PDC and the final release.

Hope you enjoy the new features in Windows Azure, and as always your feedback is highly appreciated.

 

Update (March 30th 2011) – Troubleshooting Remote Desktop Connectivity Issues

If you are experiencing issues establishing remote desktop connection to your Windows Azure instance you may want to check the RDP file that is downloaded from Windows Azure Management Portal. Here is what you need to do:

1. Click on Connect button in Windows Azure Management Portal
2. Save the RDP file on your local machine
3. Open the RDP file with Notepad and verify the following
full address:s:[your_dns_name].cloudapp.net
username:s:[your_remote_desktop_username]
LoadBalanceInfo:s:Cookie: mstshash=[role_name]#[instance_name]#Microsoft.WindowsAzure.Plugins.RemoteAccess.Rdp

In some occasions
#[instance_name]#Microsoft.WindowsAzure.Plugins.RemoteAccess.Rdp
part will be missing and this will prevent you from establishing remote connection to your Windows Azure instance. Add the necessary information to the RDP file and save it.

You can get the role_name and the instance_name from the Management Portal

 

  Advertisement

 

 

   

 

Did You Forget Your Windows Azure Demo Deployment Running? Graybox can help you!

Tweet

Windows Azure as any other cloud platform is very convenient for doing demos or quick tests without the hassle of procuring hardware and installing bunch of software. However what happens quite often is that people forget their Windows Azure demo deployments running, and this can cost them money (a lot sometimes).

Recently I bumped into a handy tool called Graybox. What it does it to regularly check your Windows Azure subscription, and notify you about your deployments.

 

Installing Graybox

You can download Graybox from here. Just unzip it in a local folder and that is it. The tool has no installer and application icon.

 

Configuring Graybox

To use Graybox you need two things:

• Windows Azure Subscription ID
• and Windows Azure Management API Certificate Thumbprint – keep in mind that you need to have the certificate imported on the machine where Graybox will be running as well as uploaded it into Windows Azure through the Windows Azure Developer Portal

The configuration is quite simple – you need to edit the Graybox.exe.config file, which is simple XML file, and add your subscription ID and management API thumbprint as follows:

 

<add key="SubscriptionId" value="[SUBSCRIPTION_ID]"/>

<add key="CertificateThumbprint" value="[API_THUMBPRINT]"/>

 

Optionally you can change the refresh rate using RefreshTimerIntervalInMinutes configuration setting:

 

<add key="RefreshTimerIntervalInMinutes" value="10"/>

 

 

 

Using Graybox to Receive Notifications About Forgotten Windows Azure Deployments

Once you have Graybox configured you can start it by double-clicking on Graybox.exe. I will suggest you add the Graybox executable to All Programs –> Startup menu in order to have it started every time you start Windows. Once started it reminds resident in memory, and you can access its functionality through the tray icon:

 

Right-clicking on Graybox’ tray icon shows you the actions menu:

As you can see on the picture above deployments are shown in the following format:

 

[hosted_service_name], [deployment_label] ([status] on [environment])

 

 

For example: toddysm-playground, test (Suspended on Production)

 

Thus it is easy to differentiate between the different deployments. You have the option to “kill” particular deployment (this means to suspend it if it is running AND delete it or just delete it if the deployment is suspended) or to “kill” all deployments.

If you have forgotten that you have deployments Graybox will pop a message above the system tray every couple of minutes to remind you that you have something running:

 

The refresh rate is the one you configured in Graybox.exe.config configuration file.

 

Graybox Limitations

There are two things you should keep in mind when planning to use Graybox:

• Graybox can track deployments in only one subscription; if you own more than one Windows Azure subscription you will not be able to see all your deployments
• In addition Graybox requires .NET 4.0, and will not work with earlier versions of .NET framework

 

More Information About Graybox

Using Graybox is free because you don’t pay for accessing the Windows Azure Management APIs.

You can get additional information on their Codeplex page.