JP7552895B2 - Process information analysis device, process information analysis method, and program - Google Patents

Description

The present invention relates to a process information analysis device, a process information analysis method, and a program.

ICT (Information and Communication Technology) systems have become a critical infrastructure that supports our lives and the social and economic foundations of our society. However, because various functions operate in a connected manner, system failures tend to be difficult to detect in advance and the causes are hard to understand, so in some cases it can take anywhere from half a day to a day to restore the system. This means that the impact on users is prolonged, resulting in significant business losses. Therefore, technology is needed that can detect abnormalities before a failure becomes apparent and identify the causes, thereby minimizing the impact on users.

Currently, anomaly detection and cause estimation technologies are being considered from the perspective of automating responses to ICT system failures. Anomaly detection technologies that detect the occurrence of failures include a method of analyzing the correlation between multiple numerical data using an autoencoder (Non-Patent Document 1), and a method of expressing temporal changes in numerical data using a mathematical model and analyzing the deviation of observed values from the model (Non-Patent Document 2). Cause estimation technologies that estimate the main cause of a failure include a method of creating a causal graph that shows the causal relationship between anomalies in each device and observed data and estimating the location of the anomaly (Non-Patent Document 3), a method of creating a causal graph that shows the causal relationship between multiple alarms generated at the time of a failure and estimating the alarm that is the main cause (Non-Patent Document 4), and a method of extracting events from log data such as syslog and creating and analyzing a causal graph that shows the causal relationship between events (Non-Patent Document 3).

There is also a method of representing the internal state of a system, i.e., process generation and file references, using a graph to detect anomalies and estimate their causes. One existing method of detecting anomalies using a graph is to analyze changes in the relationships between fixed vertices (Non-Patent Document 5).

Yasuhiro Ikeda, Yusuke Nakano, Keishiro Watanabe, Keisuke Ishibashi, Ryoichi Kawahara. Deep Learning-Based Anomaly Detection Technology - DeAnoS: Deep Anomaly Surveillance, NTT Technical Journal, May 2019. J. Takeuchi and K. Yamanishi. A unifying framework for detecting outliers and change points from time series. IEEE Transactions on Knowledge and Data Engineering, 18(4):482-492, 2006. Y. Matsuo, Y. Nakano, A. Watanabe, K. Watanabe, K. Ishibashi, and K. Kawahara. Root-cause diagnosis for rare failures using Bayesian network with dynamic modification, Proc. IEEE, ICC, 2018. B. Zong, J. S. Y. Wu, A. K. Singh, H. Cam, J. Han, and X. Yan. Towards scalable critical alert mining. KDD '14, pages 1057- 1066, 2014. Y. Park, C. E. Priebe, and A. Youssef. Anomaly detection in time series of graphs using fusion of graph invariants. IEEE Journal of Selected Topics in Signal Processing, 7(1):67-75, 2013.

In a system where various functions operate in a linked manner, it is difficult to detect problems in advance and the causes are hard to understand because of the influence of the system's internal state. A system is realized by the operation of a group of processes (commands executed by the system). Process operations include the creation of a new process, access to another process, and reference to a file, so the "internal state" of a system corresponds to the parent-child relationships between processes and the reference relationships between processes and files.

It is therefore hoped that analyzing the relationships between processes and files will enable anomaly detection and cause inference for complex failures. However, the technologies currently under consideration use explicit data such as traffic volume, CPU usage, and logs. Although this data may be generated by the operation of a process, it does not represent the detailed operation of the process itself, and is therefore insufficient for analyzing internal states. For this reason, existing technologies are insufficient for anomaly detection and cause inference for complex failures involving internal states.

It is necessary to focus on changes in parent-child relationships, such as vertices that are not fixed, where the number of processes generated or the number of files referenced change over time, and where a process disappears due to a certain process. However, with the conventional technology described above, it is difficult to handle graphs with unfixed vertices, and it is not possible to generate graphs that focus on changes in parent-child relationships.

In addition, rather than focusing on minute changes in the shape of the graph, it is necessary to focus on changes in parent-child relationships, such as the disappearance of a process caused by a certain process, in order to determine whether an abnormality exists. This is because abnormal behavior of a process affects its child processes, causing changes in parent-child relationships. Considering these factors, existing anomaly detection methods for graphs are unable to detect anomalies or infer the causes of complex failures.

The disclosed technology aims to calculate index values for detecting anomalies or estimating the causes of complex faults.

The disclosed technology is a process information analysis device that includes a process information acquisition unit that acquires process information indicating information about a target process group, an inter-process relationship estimation unit that estimates the relationship between the target process group and child processes of a process included in the process group and generates a bipartite graph, and an index value calculation unit that calculates an index value based on the bipartite graph.

It is possible to calculate index values to detect anomalies or estimate the causes of complex failures.

FIG. 2 is a functional configuration diagram of a process information analysis device. FIG. 1 is a diagram illustrating an example of a bipartite graph. 13 is a flowchart showing an example of the flow of an index value calculation process. FIG. 11 is a first diagram showing experimental results. FIG. 2 is a second diagram showing the experimental results. FIG. 2 illustrates an example of a hardware configuration of a computer.

Hereinafter, an embodiment of the present invention (the present embodiment) will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applicable is not limited to the following embodiment.

(Outline of the present embodiment)
The process information analysis device according to this embodiment is a device that analyzes process information to generate a bipartite graph showing similarity relationships between processes, and calculates index values for a group of processes to be monitored based on the generated bipartite graph.

A process is a program running on the OS (Operating System) of an information processing device. The process group to be monitored includes multiple processes.

(Functional configuration of the process information analysis device)
The process information analysis device 10 includes a process information acquisition unit 11, an inter-process relationship estimation unit 12, an index value calculation unit 13, and an output unit 14.

The process information acquisition unit 11 acquires process information indicating information about the target process group. The process information includes information about child processes whose parent processes are the target process group. A child process is a process generated by a parent process. Since abnormal behavior of a parent process affects its child processes, the process information analysis device 10 uses the information about the child processes to monitor the parent process.

Specifically, the process information acquisition unit 11 periodically acquires information on the CPU (Central Processing Unit) usage rate and memory usage rate of a process using the ps command in a UNIX (registered trademark) OS. The process information acquisition unit 11 also executes the strace command to acquire information on the file references of the process, the references to other processes, and the generation of child processes.

The inter-process relationship estimation unit 12 estimates the relationships between processes and generates a bipartite graph. Specifically, the inter-process relationship estimation unit 12 generates a graph at time t by using, with a time span of Δt, information from the process information by the strace command from time t to t+Δt and information by the ps command at time t.

The generated bipartite graph is a bipartite graph for a vertex set _V1 , which is a set of processes to be monitored (parent processes), and a vertex set _V2, which is a set of other processes (child processes of the processes to be monitored). The weights of the edges of the bipartite graph are the similarity of the file names referenced by each process (vertex) and the sum of the number of accesses between the processes.

The similarity of referenced files is expressed by the dot product of the numerical vectors of how many times each character (0-9, a-z) appears in the paths of files referenced by each process, counting from the root directory to the nth directory.

The method of calculating the similarity may be other than the above, for example, the number of times the same word appears or the number of times file extensions match. The natural number n is set arbitrarily in advance.

The index value calculation unit 13 calculates an index value based on the generated bipartite graph. Specifically, the index value calculation unit 13 calculates an eigenvalue _wt of an adjacency matrix of the bipartite graph.

The index value calculation unit 13 also calculates the number of file references for each process of _V1 and _V2 . The index value calculation unit 13 also determines that a process p of _V2 , whose edge weight is β or more, has a strong relationship with each parent process P, and adds the number of file references of p to the number of file references of P to calculate the number of file references of each parent process. This is vectorized to be u _(t,FILE) . Note that β is a preset threshold value.

Furthermore, the index value calculation unit 13 obtains and vectorizes the CPU utilization rate and memory utilization rate of each parent process, and sets these as u _(t,CPU) and u _(t,MEM) , respectively.

Then, the index value calculation unit 13 calculates the weighted sum of the calculated values as the index value at each time t.

を算出する。ここで、

where:

Figure 2 shows an example of a bipartite graph. The bipartite graph generated by the inter-process relationship estimation unit 12 has edge weights that are the sum of the similarity of the file names referenced by each process (vertex) and the number of accesses between the processes.

(Operation of the process information analyzer)
Next, a description will be given with reference to the drawings of the operation of the process information analysis device 10. The process information analysis device 10 starts the index value calculation process in response to a user operation or the like.

3 is a flowchart showing an example of the flow of the index value calculation process. The process information acquisition unit 11 acquires process information (step S11). Next, the inter-process relationship estimation unit 12 calculates the similarity of the reference file for each combination of a parent process and a child process (step S12). Next, the inter-process relationship estimation unit 12 generates a bipartite graph with the entire parent process _V1 and the entire child process _V2 as vertices (step S13).

Next, the index value calculation unit 13 calculates the eigenvalue _wt of the adjacency matrix of the bipartite graph (step S14). Then, the index value calculation unit 13 calculates the number of file references of each process (step S15). Here, for each parent process P, if there is a process p in _V2 whose edge weight is β or more, the index value calculation unit 13 adds the number of file references of p to the number of file references of P to calculate the number of file references of each parent process.

Then, the index value calculation unit 13 vectorizes the CPU usage rate and memory usage rate of each parent process (step S16). Then, the index value calculation unit 13 calculates an index value that is a weighted sum of the values calculated from step S14 to step S16 (step S17). The output unit 14 outputs information indicating the calculated index value.

(Results of demonstration experiment)
On a computing server used for research, we generated the following three pseudo-failures in 10 processes related to Kibana, elasticsearch, nginx, and cron, and their child processes, and monitored them for about a day.

Experiment 1: Do nothing → Run a program that puts a load on the CPU for 2 hours to reproduce the problem → Stop the program (return to normal).

Experiment 2: Do nothing → Run a program that generates files at regular intervals (every minute) for 2 hours and reproduce the problem → Stop the program (return it to normal).

Experiment 3: Do nothing → Run the program that repeatedly accesses the child process for 2 hours to reproduce the error → Stop the program (return to normal).

Figure 4 is the first diagram showing the experimental results. Figure 4 (1) shows the results of experiment 1 quantified by the process information analysis method according to the present embodiment. Figure 4 (2) shows the results quantified simply by the sum of the number of reference files, CPU usage, and memory usage without using a bipartite graph. In addition to pseudo-faults, cron operation and Kibana's CPU load increase occurred during the experiment. The experimental results show that the index value of this embodiment increases with respect to the occurrence of pseudo-faults, cron operation, and Kibana's CPU load increase. This means that the index value of this embodiment captures the change in the process better.

Figure 5 is a second diagram showing the experimental results. Figure 5 (1) shows the results of experiment 2 quantified using the process information analysis method of this embodiment. Figure 5 (2) shows the results of experiment 3 quantified using the process information analysis method of this embodiment. In both cases, cron operation occurred during the experiment. In both cases, the index value of this embodiment increased in value for pseudo-faults and cron operation. Therefore, it can be said that the index value of this embodiment is an index value that captures changes in the process.

(Example of computer hardware configuration)
Each of the functional units of the process information analysis device 10 described above can be realized by causing a computer to execute a program describing the processing contents described in this embodiment. Note that this "computer" may be a physical machine or a virtual machine on the cloud. When a virtual machine is used, the "hardware" described here is virtual hardware.

The above program can be recorded on a computer-readable recording medium (such as a portable memory) and stored or distributed. The above program can also be provided via a network such as the Internet or e-mail.

Figure 6 is a diagram showing an example of the hardware configuration of the computer. The computer in Figure 6 has a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, etc., which are all connected to each other via a bus B.

The program that realizes the processing on the computer is provided by a recording medium 1001, such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 via the drive device 1000 into the auxiliary storage device 1002. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.

When a program start instruction is received, the memory device 1003 reads out and stores the program from the auxiliary storage device 1002. The CPU 1004 realizes the functions related to the device according to the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network. The display device 1006 displays a GUI (Graphical User Interface) or the like according to a program. The input device 1007 is composed of a keyboard and a mouse, buttons, a touch panel, or the like, and is used to input various operation instructions. The output device 1008 outputs the calculation results. Note that the above computer may be equipped with a GPU (Graphics Processing Unit) or a TPU (Tensor processing unit) instead of the CPU 1004, or may be equipped with a GPU or TPU in addition to the CPU 1004. In that case, for example, the GPU or TPU may execute processing that requires special calculations, and the CPU 1004 may execute other processing, and the processing may be divided and executed.

(How to use this embodiment)
The accuracy of the process abnormality monitoring device can be improved by monitoring the index value output by the process information analysis device 10 according to the present embodiment. Also, the accuracy of factor estimation can be improved by inputting the index value and the bipartite graph to a device that estimates factors.

(Effects of this embodiment)
According to the process information analysis device 10 of the present embodiment, a bipartite graph showing the similarity of processes is generated based on the process information, and an index value is calculated based on the generated bipartite graph. This makes it possible to calculate an index value for performing anomaly detection or cause estimation for a complex fault in which the processes are interrelated with each other.

(Summary of the embodiment)
This specification describes at least the process information analysis apparatus, the process information analysis method, and the program described in the following sections.
(Section 1)
a process information acquisition unit for acquiring process information indicating information on a target process group;
an inter-process relationship estimation unit that estimates a relationship between the target process group and a child process of a process included in the process group, and generates a bipartite graph;
An index value calculation unit that calculates an index value based on the bipartite graph.
Process information analysis device.
(Section 2)
The process information includes file references, other process references, and child process creation information for the process.
The bipartite graph has edges weighted by the similarity of file names referenced by each process and the sum of the number of accesses between the processes.
2. The process information analysis apparatus according to claim 1.
(Section 3)
the index value calculation unit calculates an eigenvalue of an adjacency matrix of the bipartite graph, and calculates the index value based on the calculated eigenvalue;
3. The process information analysis device according to claim 2.
(Section 4)
the index value calculation unit calculates the number of file references for each of the processes, and calculates the index value based on the number of file references.
4. The process information analysis device according to claim 3.
(Section 5)
the index value calculation unit determines, as the number of file references of each parent process, the number of file references obtained by adding the number of file references of child processes whose edge weights are equal to or greater than a threshold value.
5. The process information analysis device according to claim 4.
(Section 6)
1. A computer-implemented method for analyzing process information, comprising:
acquiring process information indicating information on a target process group;
estimating a relationship between the target process group and a child process of a process included in the process group, and generating a bipartite graph;
and calculating an index value based on the bipartite graph.
Methods for analyzing process information.
(Section 7)
A program for causing a computer to function as each unit in the process information analysis device according to any one of claims 1 to 5.

Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and variations are possible within the scope of the gist of the present invention as described in the claims.

10 Process information analysis device 11 Process information acquisition unit 12 Inter-process relationship estimation unit 13 Index value calculation unit 14 Output unit 1000 Drive device 1001 Recording medium 1002 Auxiliary storage device 1003 Memory device 1004 CPU
1005 Interface device 1006 Display device 1007 Input device 1008 Output device

Claims (7)

a process information acquisition unit for acquiring process information indicating information on a target process group;
an inter-process relationship estimation unit that estimates a relationship between the target process group and a child process of a process included in the process group, and generates a bipartite graph;
An index value calculation unit that calculates an index value based on the bipartite graph.
Process information analysis device. The process information includes file references, other process references, and child process creation information for the process.
The bipartite graph has edges weighted by the similarity of file names referenced by each process and the sum of the number of accesses between the processes.
The process information analysis device according to claim 1 . the index value calculation unit calculates an eigenvalue of an adjacency matrix of the bipartite graph, and calculates the index value based on the calculated eigenvalue;
The process information analysis device according to claim 2 . the index value calculation unit calculates the number of file references for each of the processes, and calculates the index value based on the number of file references.
The process information analysis device according to claim 3 . the index value calculation unit determines, as the number of file references of each parent process, the number of file references obtained by adding the number of file references of child processes whose edge weights are equal to or greater than a threshold value.
The process information analysis device according to claim 4. 1. A computer-implemented method for analyzing process information, comprising:
acquiring process information indicating information on a target process group;
estimating a relationship between the target process group and a child process of a process included in the process group, and generating a bipartite graph;
and calculating an index value based on the bipartite graph.
Methods for analyzing process information. A program for causing a computer to function as each part of a process information analysis device described in any one of claims 1 to 5.

Images