Run Book Automation - an Overview

What is Run Book Automation?

It makes sense to start at the beginning: a “Run Book” in its original form is a printed or electronic document containing the operational procedures which need to be followed to manage or supervise some system. In other words, how to run the system. So, in the event, say, of some operational software going wrong, system administrator personnel would consult a Run Book to follow a detailed step-by-step list of diagnostic and corrective actions to be taken.

The problem is that all of this management requires substantial intervention by knowledgeable and trained staff, and has been overwhelmingly manual in nature: even now, administrators can be seen tapping away at their keyboards, and in the best of cases, they might run some home-grown scripts while fiddling with multiple disparate systems and GUIs.

With IT and software systems growing ever more complicated and interconnected, even basic operations and troubleshooting require rapid responses and detailed expert knowledge. For effective management, these systems require intelligent automation and integration.

Run Book Automation does away with the manual effort. It allows operations processes, procedures and workflow to be automated, with huge benefits to companies and their personnel. And it extends far beyond troubleshooting – including process maintenance, service orchestration and systems integration to name just a few examples. But be careful not to confuse Run Book Automation with script scheduling: Run Book Automation (RBA) is light years ahead in terms of its capability, especially when coupled with real time intelligent reasoning that differentiates Pivetal’s Cortex RBA solution from all the others.

The business benefit

Run Book Automation brings highly quantifiable efficiency gains to your business. It allows you both to save money and to earn money:

• It allows you to join unconnected processes together and to benefit from faster operations
• It allows you to rapidly develop, deploy and exploit new operational processes or services
• It enforces compliance of mission-critical or operationally sensitive activities
• Because people forget or get distracted, implementing RBA reduces operational errors
• Because people hate repetitive tasks, RBA can automate these and let people focus on meaningful work

Run Book Automation Components

RBA tools vary, and many offerings give the appearance of RBA without providing the true flexibility and ease-of-use that RBA implies.

In very general terms, RBA tools will include some or all of the following within an integrated product:

• A graphical environment in which to develop RBA automation sequences: these provide a flowchart-like technique to specify the desired automation, reducing or removing the need for programming.
• A workflow toolset to provide orchestration between multiple external software or hardware systems, and which allows multi-step automation processes to be joined together.
• Interfaces and APIs which permit the RBA environment to interact with other electronic systems or to access databases or files.
• A task scheduler to trigger specific automation at desired times of day.

Pivetal's Run Book Automation Solution

Pivetal have been providing Run Book Automation since our inception, although at the time we called it extreme automation, which is much more evocative of what it really is. Our RBA automation solutions range from single computer solutions performing a few automation tasks per hour, up to mission-critical resilient configurations of ten or more servers performing hundreds of thousands of automations per day!

Our Cortex Automator RBA solution give you all of the above basic functions, and a lot more:

• GUI tools for human interaction: RBA is not always about “black box” automation; often enough RBA is used as an enabling tool to help people work through complex actions step-by-step, shielding them from the back-end complexity. Powerful and flexible GUI tools are required for this, and Cortex has a combination of web and thick client tools which are rapidly adaptable to specific needs.
• Advanced event handling: many automation activities are triggered by external systems, passing messages or events to the RBA solution. Cortex handles events across all common types of technologies, but more importantly includes throttling and prioritisation – vital when an avalanche of events arrives. Cortex can also generate events in many forms and send these on to other operational systems..
• Real-time multitasking capability: RBA must be able to handle multiple types of automation and multiple instances of the same type of automation simultaneously and in real time. This is especially important when human interaction is required. Cortex can easily handle thousands of simultaneous activities.
• Temporal reasoning: RBA is often used to detect and correct operational problems, but if the same problem happens several times within a stipulated time interval you’d probably want to handle it differently. With Cortex your automation can easily include this type of time-based analysis.
• Correlation analysis: while some RBA automation just needs to process tasks blindly and in isolation from each other, what happens if you’re correcting faults and suddenly many different devices report something similar? Cortex gives you powerful correlation analysis to identify groups and patterns, and to change the type of automated actions based on correlation results.
• Testing and simulation tools: you’re not likely to release untested automation into your operational environment, and Cortex therefore provides a comprehensive automation building and testing system called Cortex Studio. Build your automation graphically, and test it graphically, using path traces, breakpoints and watch windows. And in order to generate confidence, several graduated levels of simulation may be applied until the automation is pronounced safe for release. We’ve also included release signoff tools and a release manager application to update your operational platforms with new RBA automation in real time.
• Full activity logging: we automatically log everything – incoming, outgoing and internal, and in a variety of different ways. This allows you to provide compliance auditing data, perform focused troubleshooting, or quantify operational savings and efficiency gains.
• Scalability and redundancy: as our customers grow their automation systems, they need scalability. Cortex provides a transparent management infrastructure so that multiple instances of Cortex may operate collaboratively but under master control. And for mission-critical systems, Cortex provides redundancy through hot-standby or cold-standby platforms which can take over without any loss of service.

Some examples of what we've done

See our case studies for more detail, but here are some areas where we’ve worked:

• Intelligent auto-correction of software faults on switches, routers, servers and a multitude of complex IT/telecom hardware and software.
• Trouble ticket enrichment, including opening tickets, diagnosing the trouble, fixing the trouble, updating the tickets, closing the tickets and, where required, alerting field engineers with detailed information.
• Intelligent data manipulation by gathering, correlating and massaging information from multiple systems, before sending it on as processed data.
• Process orchestration where multiple existing systems are linked together and sequenced to work efficiently in concert under master control of RBA
• Service impact analysis where we identify all contributing causes for loss, or possible loss, of service, providing a summary of the impact to the customer
• Automation of configuration tasks where configuration is slow, complex and spans multiple software systems. Tasks can be automatically triggered or can be triggered and guided by human users.
• Routine maintenance activities to be performed around-the-clock on hardware and software systems to ensure peak operations and provide early identification of possible problems.
• Mission-critical systems management such as automated public telephony network protection during national emergencies.
• Self-service portals for adding, removing and changing user configurations on unified communications environments, PBXs and distributed systems.

^ Top