JP6087673B2 - Workaround execution management system and workaround execution management method - Google Patents

Description

  The present invention supports work-around confirmation work in an automatic work execution system such as runbook automation that automatically executes an operation procedure when a service desk performs work-around (primary response) on an incident detected by a failure monitoring device. The present invention relates to a work-around execution management system and a work-around execution management method that can reduce the initial risk of introducing an automatic work execution system.

  In recent computer systems, when a workaround (primary response) is performed quickly and accurately on a message containing problems and issues to be solved from monitored devices (customer computers, virtual machines) that occur daily An incident management system that manages a message as an incident has been adopted. In this incident management system, in addition to messages from monitored devices, inquiries and work requests from customers are also regarded as incidents, which are registered in the incident management system. In order to minimize the impact on business operations for incidents that occur every day from the beginning, we contact the relevant parties and provide primary responses.

  These workarounds conducted by the service desk lead to an increase in the number of processing and complexity of procedures as customers' computer systems expand and become complex, and as a countermeasure against this, we have introduced an automated work execution system such as runbook automation. Measures are taken to automate workarounds.

The following Patent Document 1 is cited as a document describing a technique for automating the workaround . The patent document 1 describes a problem based on an incident data extraction unit that extracts incident information including its own identification information and time data when a defect occurs, and determination condition data indicating a predetermined defect determination condition. A checklist processing unit that determines whether or not a problem has occurred, and when a problem has occurred, a command associated with determination condition data in advance is executed in order to eliminate the problem that has occurred a restoration command execution unit, technique and a transmitter Ru provided to each client that transmits information indicating the execution result of the incident data and commands to the management server is described. It's in this, to automate the minor problems of repair.

JP 2009-181441 A

  The technique described in Patent Document 1 described above is already established by pre-defining a command for recovery according to the content of a failure and automatically executing this command, thereby reducing the work performed by the service desk. However, it is difficult to apply complicated procedures to execution using techniques such as runbook automation. In other words, in the incident management system, the hardware and software configuration of the computer varies depending on the customer, and in particular, multiple operations may be branched and executed according to the command result, and the primary response corresponding to various incidents However, it is necessary to manually check whether these primary correspondences can be executed normally, even if they are defined in advance, this confirmation work is complicated, and it is difficult to apply to the execution using techniques such as runbook automation. There was a problem of being.

  In other words, the automated procedure is used in the initial stage of application, when a person monitors normality / abnormality, and if a recovery procedure has been established that will promptly recover in the event of an abnormality, the quality is temporarily There is a risk that the service desk deteriorates. For this reason, the service desk needs to regularly capture an automated procedure every certain period, and there is a problem that the operation is complicated and complicated.

An object of the present invention, a work-around (primary response) workaround execution management system and the work-around execution management method of the confirmation work can reduce the risk of working automatic execution system initially introduced to support for multiple incidents Is to provide.

The present invention relates to a first stage in which a service desk monitors a workflow operation in a production environment of a workflow that has been tested, and a workflow in which a normal operation of a first success rate or higher is confirmed in the first stage. A second stage for notifying the service desk of normality and abnormalities, and lowering the work flow in which normal operation less than the first success rate is confirmed to the first stage, and normal above the second success rate in the second stage A third stage is set for notifying the service desk of an abnormality in the workflow whose operation has been confirmed and lowering the work flow whose normal operation less than the second success rate has been confirmed to the second stage.
The present invention includes a first step of storing performance information including a success rate for each of the first to third stages in the work performance management database when the workflow having the flow name is executed, and a first step in the first stage. A second step of notifying the service desk of the normality and abnormality of the work flow confirmed to have a normal operation equal to or higher than the success rate, and raising the work flow confirmed to be normal operation equal to or higher than the first success rate to the second stage ; In the second stage, the service desk is notified of an abnormality in the work flow in which the normal operation of the second success rate or higher is confirmed, and the work flow in which the normal operation of the first success rate or higher is confirmed is raised to the third stage. And a third step for lowering the work flow in which the normal operation less than the second success rate is confirmed to the first stage, and a work flow in which the normal operation less than the second success rate is confirmed in the third step in the second stage. 4th work to lower To run the door.

The workaround execution management system and the workaround execution management method according to the present invention can support the workaround (primary response) confirmation work for a plurality of incidents, and can reduce the risk at the initial stage of the automatic work execution system introduction.

It is a figure which shows the computer system containing the workaround execution management system by embodiment of this invention. It is a figure for demonstrating the incident database by this embodiment. It is a figure for demonstrating the work performance management database by this embodiment. It is a figure which shows the application image for demonstrating the principle of this invention. It is a figure which shows an example of the work performance management screen by this embodiment. It is a figure which shows an example of the incident list screen by this embodiment. It is a flowchart which shows the workaround execution management operation | movement by this embodiment.

Hereinafter, an embodiment of a workaround execution management system to which a workaround execution management method according to the present invention is applied will be described in detail with reference to the drawings. First, the principle of the workaround execution management system according to the present embodiment will be described. Will be explained.
[principle]
In the workaround execution management system according to the present embodiment, in the prior art, after the service desk 14 completes the test of the workflow that is workaround (primary correspondence) and releases it to the production environment, all the workflows also in the production environment. In order to monitor whether the incident is valid (whether or not an error occurs), the service desk 14 has to constantly monitor whether the work flow operates normally for a certain period of time . On the other hand, in the present invention, first, in order to perform step-by-step verification of workaround (primary response) which is a response example given to an incident, this step (Step) is performed as a verification step of Step 1 to Step 3. Set 3 levels . Next, as shown in the application image of FIG. 4, the verification level at the time of performing this verification is set to Step 1 which is the stage where the service desk 14 has completed the workflow test and released to the production environment, and after Step 1 Step 2, which is a stage in which the device has a record of a predetermined number of executions or more and a certain success rate (first success rate) or more, and a step after the Step 2 and a predetermined success rate (second success rate) or more Is set as three stages with Step 3, which is the stage where the results are left. The verification stage is not limited to three stages, and may be further multi-staged, or the success rate and / or the predetermined number of executions may be changed from the outside such as a service desk or an administrator's computer. .

  In Step 1, since there is no work flow execution result, the service desk is a stage in which the service desk manually executes the work flow using the work automatic execution system 11, and even if the work flow execution fails, the service desk The desk 14 can respond immediately.

  In Step 2, the workaround execution management system 8 activates the automatic work execution system 11, automatically executes the work flow, and notifies the service desk 14 by e-mail or the like even if the execution result is normal.

  In Step 3, the workaround execution management system 8 starts the automatic work execution system 11, automatically executes the work flow, and notifies the service desk 14 by e-mail or the like only when the execution result is abnormal. It is a stage that does not require. If the workflow does not succeed at a certain success rate or higher in Step 2 or Step 3, the current Step level is lowered (Step 2 → Step 1, Step 3 → Step 2).

As described above, the workaround execution management system according to the present invention uses the following Step as a verification level for a workflow operation that is a workaround (primary correspondence) as a verification level for a large number of incidents in the workflow in the production environment of the workflow. Set.
- released to the production environment, the service desk 14 is a stage for monitoring the operation Step1.
- constant success rate in real device to the next of the Step1 leaving (first success rate) or more results, a step of notifying the normal and abnormal workflow execution service desk 14 Step2.
· The following constant success rate of the Step2 leaving proven (second success rate) than a step of notifying only the service desk 14 when workflow execution is abnormal Step3.
The S Tep2 or if workflow over a certain success rate in Step3 is not successful the system by Rukoto lowering the current Step level, verification of the operation of the workflow as a work-around (primary response) when automating work Can be efficiently performed without requiring constant monitoring of the service desk 14. In the present embodiment, each Step is also called a verification stage.

[Constitution]
As shown in FIG. 1, a computer system including a workaround execution management system according to the present embodiment includes a data center 1 in which a failure monitoring device 3 connected to a plurality of monitored devices 2 that are customer computers is installed, and the data center. 1 are connected via a network 4 manages the fault message from the fault monitoring apparatus 3 as an incident, Ru and a monitoring center 5 for managing the progress of the primary response to incidents. The monitoring center 5 includes the following constituent.
(1) Incident database 7 for storing the incident occurrence date / time, host name / occurrence system / customer name / message / corresponding case ID / corresponding case flow name for each incident ID uniquely assigned to the incident .
(2) An incident management system 6 that automatically registers an incident received from the failure monitoring apparatus 3 via the network 4 in the incident database 7 .
(3) A work performance management database storing performance information such as success cases, failure rates, execution times, failure times for each verification case (Step 1 to Step 3) for each response case ID uniquely assigned to a response case 9 .
(4) An automatic work execution system 11 that executes processing for each processing step, and triggered by the fact that the incident has been registered in the incident database 7, based on the registered contents of the incident database 7 and the work performance management database 9. A work-around execution management system 8 for determining whether to sound a patrol lamp (signal lamp) 13 or to activate the automatic work execution system 11 .
(5) Work result management screen 10 for confirming the execution result (result information) stored in the work result management database 9 stored by the work around execution management system 8 .
(6) A computer of the service desk 14 that refers to a registered incident and performs a primary response based on a response procedure manual according to a response case added to the incident .
(7) An execution log 12 which is a database for storing the result of the work flow 15 which is a processing step executed by the work automatic execution system 11 .

  These components are formed by computer and server including CPU, memory, input / output device, magnetic disk device, display unit, hardware such as database, and software, and the patrol lamp 13 is turned on as in a general computer system. The present invention is not limited to this, and other means that can issue a warning to the service desk 14 such as e-mail may be used.

The computer system including the workaround execution management system configured as described above transmits a failure message (incident message) generated by the failure monitoring device 3 of the data center 1 to the monitored device 2 to the incident management system 6 of the monitoring center 5. that sends. Incident management system 6 that has received this message, and automatically registered incidents received in the incident database 7, a work-around execution management system 8 this registration as a trigger is registered in the incidents database 7 and the work record management database 9 Whether the patrol lamp 13 is sounded or the automatic work execution system 11 is activated is determined based on the contents by a process described later . If you ringing the path cards 13, the service desk 14 using the work automatic execution system 11 executes the workflow 15 manually, when the execution result is NG in (failure), the line service desk 14 appropriate measures Uyo When the work automatic execution system 11 is activated, the work flow 15 is automatically executed and the execution result (result information) is recorded in the execution log 12.

The incident database 7 stores IncidentTable, and includes the following item information as shown in FIG .
(1) Corresponding case ID automatically assigned by the incident management system for each message received from the monitored device 2 .
(2) Date and time when the incident occurred .
(3) Host name of the monitored device where the incident occurred .
(4) A generation system representing an application system operating on the monitored device .
(5) Name of the customer who uses the monitored device .
(6) An actual message received from the monitored device .
(7) Response example (flow name) for solving a failure for the incident .
(8) The flow name of the work flow 15 executed by the work automatic execution system 11 .
(9) Execution status .

  The work performance management database 9 includes the current step of each case, the number of executions in each step, the number of successes in each step, the number of failures in each step, for each corresponding case given for each incident. , Operation result information such as success rate at each step and failure rate at each step is stored.

The work record management screen 10 is for displaying the verification status consisting running "current situation" column and "past week execution result" corresponding to the level section of the verification step (Step) of the incident, As shown in FIG. 5, the “current situation” column includes, for each case ID, the corresponding case, the current step, the number of executions in each step, the number of successes in each step, and the number of failures in each step. , Each item information such as success rate at each step and failure rate at each step is displayed as operation result information, and the “execution result for the past one week” column shows the corresponding case and execution category for each incident ID. And each item information of the execution result is displayed.

  As shown in FIG. 6, the incident list screen shows, for each incident ID, the date and time when the incident occurred, the host name of the monitored device where the incident occurred, and the application system running on the monitored device. The generation system, the name of the customer who uses the monitored device, the actual message received from the monitored device, the response case (flow name) for solving the fault for the incident, and the item information of the execution status Consists of

[Operation]
Now, the workaround execution management system configured as described above executes the following steps as shown in FIG .
(1) When the workaround execution management system 8 automatically executed from the incident management system 6 at the time of incident registration is activated, the incident case ID and the flow name defined in the corresponding case are extracted from the incident database 7 Step 701 .
(2) Step 702 of acquiring the current Step (verification level) from the work record management database 9 based on the corresponding case ID acquired in Step 701 .
(3) Step 703 for determining whether the verification level of the current Step acquired in Step 702 is Step 1, Step 2, or Step 3 .

(4) When the verification level of Step 1 is determined in Step 703, the patrol lamp 13 is sounded to prompt the service desk 14 to manually execute the work flow, and the workaround execution management system 8 displays the incident list screen (FIG. 6). Step 704 is updated to “Manual execution (Step 1)” .
(5) Step 705 for waiting until the end of a series of work execution in step 704 .
(6) Step 706 of monitoring the execution log 12 for manual execution at the service desk in step 704, waiting until the end of work, and acquiring the execution result .
(7) Step 707 that instructs the workaround execution management system 8 to execute the work flow when it is determined as Step2 in Step 703 .
(8) Step 708 which waits until the execution in step 707 is completed .

(9) Step 709 that obtains the result after completion of execution and notifies the result to the service desk regardless of whether the execution result of the workflow is normal or abnormal .
(10) Step 710 that instructs the workaround execution management system 8 to execute the work flow when it is determined as Step3 in Step 703 .
(11) Step 711 for waiting until execution is completed .
(12) Step 712 which acquires a result after completion of the execution by the step 710 and determines whether the execution result is normal or abnormal .
(13) Step 713 of notifying the service desk that an abnormality has occurred when it is determined in step 712 that there is an abnormality .
(14) Step 714 which records the execution result of each Step in the work performance management database 9 when it is determined normal by Step 706, Step 709, Step 713 and Step 712 .

(15) Step 715 of acquiring the execution result of the current Step of the work performance management database 9 and determining whether the number of executions is 5 or more and the success rate is 80% or more .
(16) When it is determined in step 715 that the number of executions is 5 or more and the success rate is 80% or more, step 716 ranks up “current step” of the case being processed .
(17) When it is determined in step 715 that the number of executions is 5 or more and the success rate is not 80% or more, the execution result of the current Step is acquired, and it is determined whether or not the failure rate is 21% or more. Step 717 terminates the process when it is determined that the rate is not 21% or more .
(18) When it is determined in step 718 that the failure rate is 21% or more, the current step (verification level) is lowered and the process is terminated 718 .
By executing these steps , the range of automation can be expanded step by step by changing the work requiring the involvement of the service desk 14 according to the operation results of each work flow.

This word over click around execution management system, in a production environment workflow, and the verify level for a number of incidents of workflow, released into production, service desk 14 is the step of monitoring the operating Step1, following the Step1 In Step 2, which is a stage in which the actual device has a record of a certain success rate (first success rate) or more and notifies the service desk 14 of normal or abnormal execution of the work flow, and a constant success rate (first (Success rate of 2) or more is set as three stages with Step 3, which is a stage that leaves a track record with the above and notifies the service desk 14 only when the work flow execution is abnormal . Then, by lowering the stage workflow less than the first or second success rate before Symbol Step2 or Step3, efficiently performed by the verification of the workflow as a work-around (primary response) when automate tasks Can do.

1 data center, 2 monitored equipment, 3 fault monitoring device,
4 network, 5 monitoring center, 6 incident management system,
7 incident database, 8 workaround execution management system,
9 Work results management database, 10 Work results management screen,
11 work automatic execution system, 12 execution log, 13 patrol,
14 Service Desk, 15 Work Flow

Claims (8)

An incident database that stores a message corresponding to an incident ID uniquely assigned to an incident and a flow name of a response case, an incident management system that registers a received incident in the incident database, and a work flow with the flow name are executed Confirming the work automatic execution system, the work result management database storing the result information including the success rate for each verification stage when the work flow of the flow name is executed, and the result information stored in the work result management database A work-around execution management system that displays a work result management screen and is connected to a service desk computer that controls the execution of the work flow, and manages the execution of the incident response work flow,
The first stage in which the service desk monitors the operation of the work flow in the production environment of the work flow that has been tested, and the normality and abnormality of the work flow in which the normal operation of the first success rate or higher is confirmed in the first stage. A second step of notifying the service desk and lowering the work flow in which the normal operation less than the first success rate is confirmed to the first step, and a normal operation exceeding the second success rate being confirmed in the second step. A third step of notifying the service desk of the abnormal work flow and lowering the work flow in which normal operation less than the second success rate is confirmed to the second step;
A first step of storing performance information including a success rate for each of the first to third stages when the work flow having the flow name is executed in a work performance management database;
The service desk is notified of the normality and abnormality of the work flow in which the normal operation equal to or higher than the first success rate is confirmed in the first stage, and the work flow in which the normal operation equal to or higher than the first success rate is confirmed to the second A second step to step up,
An abnormality in the work flow in which the normal operation equal to or higher than the second success rate is confirmed in the second stage is notified to the service desk, and the work flow in which the normal operation equal to or higher than the first success rate is confirmed is set to the third stage. And a third step of lowering the work flow in which normal operation less than the second success rate is confirmed to the first stage,
The third step second fourth Ruwa over click around execution management system to perform the steps of lowering the work flow that normal operation has been confirmed less than the success rate in the second stage of the. When the performance information in the work performance management database includes the number of executions when the workflow for each verification stage is executed, and the first and second success rates are determined to be equal to or greater than the predetermined number of executions, the second from that perform fourth step 請 Motomeko 1 wherein workarounds execution management system. It said first and second success rate and or 請 Motomeko 1 or 2, wherein the work-around execution management system that have a function of editing a value of a predetermined number of executions. Comprising a revolving lamp to alert the service desk computer according abnormalities of Oite workflow in the third step to the service desk computer 3 or from 請 Motomeko 1 that warning by using the revolving lamp workpiece Around execution management system. An incident database that stores a message corresponding to an incident ID uniquely assigned to an incident and a flow name of a response case, an incident management system that registers a received incident in the incident database, and a work flow with the flow name are executed Confirming the work automatic execution system, the work result management database storing the result information including the success rate for each verification stage when the work flow of the flow name is executed, and the result information stored in the work result management database is connected to the service desk computer to control the execution of the workflow displays an operation result management screen for, a work Arau down de execution management method of a computer system for managing the execution of the workflow incident response,
In the computer system,
The first stage in which the service desk monitors the operation of the work flow in the production environment of the work flow that has been tested, and the normality and abnormality of the work flow in which the normal operation of the first success rate or higher is confirmed in the first stage. A second step of notifying the service desk and lowering the work flow in which the normal operation less than the first success rate is confirmed to the first step, and a normal operation exceeding the second success rate being confirmed in the second step. A third step of notifying the service desk of the abnormal work flow and lowering the work flow in which normal operation less than the second success rate is confirmed to the second step;
A first step of storing performance information including a success rate for each of the first to third stages when the work flow having the flow name is executed in a work performance management database;
The service desk is notified of the normality and abnormality of the work flow in which the normal operation equal to or higher than the first success rate is confirmed in the first stage, and the work flow in which the normal operation equal to or higher than the first success rate is confirmed to the second A second step to step up,
An abnormality in the work flow in which the normal operation equal to or higher than the second success rate is confirmed in the second stage is notified to the service desk, and the work flow in which the normal operation equal to or higher than the first success rate is confirmed is set to the third stage. And a third step of lowering the work flow in which normal operation less than the second success rate is confirmed to the first stage,
The third step second fourth step and Ruwa over click-around execution management method is run to lower the work flow that normal operation has been confirmed less than the success rate in the second step in. When the performance information of the work performance management database includes the number of executions when the workflow for each verification stage is executed, and when the first and second success rates are determined to be greater than or equal to a predetermined number of executions, 請 Motomeko 5 workaround execution management method according to the 2 Ru to execute the fourth step. Said first and second success rate value and or 請 Motomeko 5 or 6 Symbol mounting workarounds execution management method Ru to execute the function of editing the predetermined number of executions. Provided revolving lamp to alert the service desk computer according abnormalities of Oite workflow in the third step to the service desk computer 7 any of 請 Motomeko 5 Ru is warned using the revolving lamp workpiece Around execution management method.

Images