The below solution is served to you as an overall idea. In the real-life solution I added more conditions and logic to the Runbooks, I hope you will find this idea useful for you.

Let’s imagine a scenario;

It’s Friday night, you dining in a nice restaurant… taking your first bite and then – you getting an email from your monitoring system (Operations Manager for an example).

Arrrr, it’s again this LUN that got full, now your production DB is down, you know what the procedure is for this incident; Login the Server, navigate to the mapped drive and delete the buffer file which acts as a placeholder for scenarios like this.

So… you pay for your meal and drive back home as fast as you can to deal with the situation – not a happy ending for a supposed to be nice evening!

So, we have a well-known situation, a monitoring system which tell us we have a problem and an email system that available for us from anywhere, what if we could just combine these components together to one fully automated solution ? We have well-known problem – a LUN that got full and SQL services that went down as a result and we have an alerting mail that was sent to our smartphone (Windows Phone, Android, IPhone, etc.).

By insert Orchestrator into this equation we could automate our recovery tasks for incidents like we talked about, automation is not enough, we need control too. I don’t know any admin that would want to delete files or expand LUNs automatically without any control or supervising – that’s where our smartphones comes in.

My solution includes the above systems:

• System Center Operations Manager (Could be any other monitoring system that integrated with Orchestrator).
• System Center Orchestrator 2012 SP1 with the following not default Integration packs:
  • System Center Operations Manager –
  • Exchange User –
• Exchange Server (for sending & receiving the mails) – some dedicated mailbox should be created.

With the following integration packs I have to build two Runbooks, first Runbook which monitors the Operations Manager Alerts (let’s call it Auto_Trigger_By_SCOM) and second which monitors our dedicated mailbox for new mails (let’s call it Auto_Trigger_By_Mail).

The scenario is simple: when an alert is generated about our full LUN, the Auto_Trigger_By_SCOM Runbook triggered and sends mail to you – the IT guy. The mail is sent by the user of our dedicated mailbox. All the needed information should be in that mail for you to understand what is all about.  After you took a moment to think about that, replying that mail with some “Magic word” should trigger the Runbook which solving the problem by deleting the buffer file, how? By the Auto_Trigger_By_Mail Runbook that monitors our dedicated mailbox for new mails, it analyze the replied mail and execute (or not) the rest of the Runbook, this way, you can control your executions and recovery tasks without stop eating your great meal J