« Windows Azure Diagnostics–Where Are My Logs? | Main | Update, Upgrade and VIP-Swap for Windows Azure Service–What are the Differences? »

May 02, 2010

Collecting Event Logs in Windows Azure

While playing with Windows Azure Diagnostics for my last post Windows Azure Diagnostics – Where Are My Logs? I noticed few things related to collecting Event Logs.

 

Configuring Windows Event Logs Data Source

First let’s configure the Event Logs data source. Here are the four simple lines that do that:

 

DiagnosticMonitorConfiguration dmc = DiagnosticMonitor.GetDefaultInitialConfiguration();
dmc.WindowsEventLog.DataSources.Add("Application!*");
dmc.WindowsEventLog.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);
DiagnosticMonitor.Start("DiagnosticsConnectionString", dmc);

 

I would highly recommend you to define the “Application!*” string globally, and make sure it is not misspelled. I spent quite some time wondering why my Event Logs are not showing up in my storage table, and the reason was my bad spelling skills. BIG THANKS to Steve Marx for opening my eyes! Smile

If you want to see other event logs like System and Security you should add those as data sources before you call DiagnosticMonitor.Start() method.

 

Filtering Events

The first thing I noticed is that the configuration above will transfer everything from the Application Event Log to WADWindowsEventLogsTable at one minute intervals. With “everything” I literally mean everything – doesn’t matter whether the event is generated by your application or something else running in parallel. There are two reasons why you don’t want to dump everything into your storage table: 1. it is too much noise, and 2. (but more important) the more data you dump the higher bill you will get (you will be charged for transactions and for data stored).

My suggestion is to always capture filtered events, and transfer only those to the storage table. Steve has a very short but useful post explaining how to capture filtered Windows Events with Windows Azure Diagnostics using XPath expressions. In my case I filtered to only events generated by my Web Role.

 

How are Windows Event Logs Transferred to Storage?

Once I had events transferred to WADWindowsEventLogsTable in my storage account, I used Cerebrata’s Azure Diagnostics Manager to browse through the data. I was running my tests on Development Fabric and storing the data in Development Storage, and surprisingly for me I saw the events duplicated in WADWindowsEventLogsTable. My first thought was: “This is a bug!”, and then the second: “Will I be double-charged?” Smile Nothing to worry about! (And again thanks to Steve for opening my eyes).

The explanation is simple. Because every Role Instance runs in separate VM, it has its own Windows Azure Monitoring Agent running. The Monitoring Agent for particular Role Instance reads the events from the Event Log for this particular VM, and transfers it to WADWindowsEventLogsTable. Thus you are able to see the events for every Role Instance. If you look at the WADWindowsEventLogsTable schema you will see that there is a column RoleInstance that identifies the Role Instance from which this event came. What happens in Development Fabric is that there is only one Event Log to read from, and because I had two instances running I was seeing the events duplicated.

 

Guidelines for Capturing Windows Events Using Windows Azure Diagnostics

As a conclusion here are some guidelines for capturing Windows Event Logs using Windows Azure Diagnostics.

  • As you can read from my previous post Windows Azure Diagnostics – Where Are My Logs? by default no logs are transferred to Windows Azure Storage tables. You need to explicitly set DiagnosticsMonitorconfiguration.WindowsEventLog.DataSources if you want to receive events into WADWindowsEventLogsTable
  • Always capture filtered Windows Events using XPath expressions in WindowsEventLog.DataSources in order to avoid noise and unnecessary charges
  • Keep in mind that Event Logs are collected in distributed environment, and you will need to mind the data you receive in your table. Using tools like Cerebrata’s Azure Diagnostics Managercan help here

 

Using Windows Azure Diagnostics is a good way to debug your application in the cloud, however you need to be careful not to grow your bill unnecessary.

 

Update: Two updates based on feedback that I received in the last few days:

  • Although I mention above that you can configure Security as event source you should be aware that this works only in Windows Azure Development Fabric. You will not be able to collect Security events in Windows Azure cloud environment because your role has no admin privileges.
  • I received the suggestion (guess from whomSmile) to post the Xpath expression I use to filter the events to only my role. Here it is:
    Application!*[System[Provider[@Name='SmileyButton']]]

Advertisement


   

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Travis

So this only works in the development fabric? That would explain why I couldn't get it working in actual azure.

ToddySM

@Travis: The note I have is only for the Security event logs. Application event logging as well as System event logging works in in the cloud.

Jon

How are you writing to the Event Log? I get a security exception when I try to do this. Thanks.

ToddySM

@Jon: Can you give more details what the exception is? Code snippet and more details will be helpful.

Travis Chase

How do you write to the Event Log? Creating a custom event source is not allowed so do you use and existing source in the Application log?

Piyush Chandra

@Travis Chase,
Check out this link it might be resolve your problem....
http://mindstick.com/Articles/d6a4b9da-1b7b-4483-9a18-bf53a0c21a3b/?Create%20Log%20Event%20in%20Windows%20Azure

Piyush Chandra

Hi,

I was reading your article and I would like to appreciate you for making it very simple and understandable. This article gives me a basic idea of Create Log Event in Windows Azure. I've found another nice post which also explained very well, for more information of these post check out this link...

http://www.mindstick.com/Articles/d6a4b9da-1b7b-4483-9a18-bf53a0c21a3b/?Create%20Log%20Event%20in%20Windows%20Azure

http://www.codeproject.com/Articles/326949/windows-azure-event-logs-notifier


Thank you very much!

Post a comment

Comments are moderated, and will not appear on this weblog until the author has approved them.

If you have a TypeKey or TypePad account, please Sign In