JP7548422B2 - Feature quantity calculation device, feature quantity calculation method, and feature quantity calculation program - Google Patents

Description

The present invention relates to a feature calculation device, a feature calculation method, and a feature calculation program.

The structures of networks called botnets, which are made up of malicious servers hijacked by malicious programs, are diverse, and the shortest distance between malicious servers also varies depending on the structure. In recent years, there has been hope for technology that can detect the structure of such botnets. The features of a graph, in which IP hosts are nodes and end-to-end communications between IP hosts are edges, provide useful information for detecting the structure of a botnet.

Therefore, a technology called graph embedding is known that learns the features of nodes from a communication graph consisting of network flow information. For example, it is possible to generate a path of nodes from a graph and learn the similarity between nodes within a certain number of hops on the path (see Non-Patent Document 1).

Bryan Perozzi et al., “DeepWalk: Online Learning of Social Representations”, KDD '14, Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery, 2014

However, with conventional technology, it is difficult to learn high-quality features. For example, when focusing on a certain node, the contexts of neighboring nodes within a certain distance differ depending on the distance, so there is a problem that the quality of the obtained features decreases when nodes with different contexts are simultaneously learned.

The present invention has been made in consideration of the above, and aims to learn high-quality features from a graph representing a communication network.

In order to solve the above-mentioned problems and achieve the object, the feature learning device of the present invention is characterized by having a generation unit that uses communication information between nodes in a network to generate a graph representing communication between nodes; a selection unit that selects related nodes connected by paths of a predetermined length from the nodes of the generated graph; a classification unit that classifies nodes within a predetermined distance on the path for the selected nodes into groups according to the distance between the nodes; a learning unit that learns a model that represents the feature in the graph of each node in the group for each classified group; and a calculation unit that calculates a feature for each of the selected nodes by combining features estimated using the models learned for each group.

According to the present invention, it is possible to learn high-quality features from a graph representing a communication network.

FIG. 1 is a schematic diagram illustrating a schematic configuration of a feature calculation device. FIG. 2 is a diagram for explaining the processing of the feature calculation device. FIG. 3 is a diagram for explaining the process of the feature calculation device. FIG. 4 is a diagram for explaining the process of the feature calculation device. FIG. 5 is a flowchart showing the procedure of the feature amount calculation process. FIG. 6 is a diagram illustrating a computer that executes a feature amount calculation program.

Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are indicated by the same reference numerals.

[Configuration of the feature amount calculation device]
Fig. 1 is a schematic diagram illustrating an example of a schematic configuration of a feature calculation device. Fig. 2 to Fig. 4 are diagrams for explaining the processing of the feature calculation device. First, as illustrated in Fig. 1, a feature calculation device 10 is realized by a general-purpose computer such as a personal computer, and includes an input unit 11, an output unit 12, a communication control unit 13, a storage unit 14, and a control unit 15.

The input unit 11 is realized using input devices such as a keyboard and a mouse, and inputs various instruction information such as a command to start processing to the control unit 15 in response to an input operation by an operator. The output unit 12 is realized by a display device such as a liquid crystal display, a printing device such as a printer, etc.

The communication control unit 13 is realized by a NIC (Network Interface Card) or the like, and controls communication between the control unit 15 and an external device such as a server via the network. For example, the communication control unit 13 controls communication between the control unit 15 and a management device or the like that collects and manages communication information of the network.

The storage unit 14 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 14 stores in advance the processing program that operates the feature calculation device 10 and data used during execution of the processing program, or temporarily stores the data each time processing is performed. For example, the storage unit 14 stores a model 14a of the processing results of the learning unit described below. The storage unit 14 may be configured to communicate with the control unit 15 via the communication control unit 13.

The control unit 15 is realized using a CPU (Central Processing Unit) or the like, and executes a processing program stored in memory. As a result, the control unit 15 functions as an acquisition unit 15a, a generation unit 15b, a selection unit 15c, a classification unit 15d, a learning unit 15e, a calculation unit 15f, and an extraction unit 15g, as exemplified in FIG. 1. Note that each or some of these functional units may be implemented in different hardware. For example, the learning unit 15e and the calculation unit 15f may be implemented in different hardware. The control unit 15 may also include other functional units.

The acquisition unit 15a acquires the collected communication information of the network nodes. For example, the acquisition unit 15a acquires flow information of the IP host that is the target of the feature calculation process described below, via the input unit 11 or the communication control unit 13, from a management device that collects and manages communication information of the network. The acquisition unit 15a may store the acquired data in the storage unit 14. Alternatively, the acquisition unit 15a may transfer the information to the generation unit 15b described below without storing it in the storage unit 14.

The generation unit 15b uses communication information between network nodes to generate a graph representing communication between nodes. For example, as shown in Fig. 2, the generation unit 15b uses the acquired flow information of IP hosts to create a graph in which IP hosts are nodes and communications between IP hosts are edges. Fig. 2 illustrates a communication graph between a malicious (Bot) server and a C&C (Command and Control) server.

The selection unit 15c selects related nodes connected by paths of a predetermined length from among the nodes of the generated graph. For example, the selection unit 15c executes a random walk a predetermined number of times starting from each node, and generates a path of a predetermined length that includes the node for each starting node.

The classification unit 15d classifies nodes on the path within a certain distance of the selected node into groups according to the distance between the nodes. For example, for the graph shown in Figure 2, the classification unit 15d classifies node pair A-C into a group with distance 1, node pair A-B into a group with distance 2, and node pair A-E into a group with distance 3, as shown in Figure 3.

As shown in FIG. 4(a), the learning unit 15e learns a model 14a that represents the features in the graph of each node in each classified group for each classified group. In this embodiment, the learning unit 15e learns a different model 14a for each classified group.

The learning unit 15e may further learn a common model 14a for multiple groups within a predetermined distance range among the classified groups. For example, among the groups illustrated in FIG. 4(a), a common model 14a may be learned for multiple groups with a distance of 2 or less, i.e., a group with a distance of 1 and a group with a distance of 2. In this case, the learning unit 15e may be configured to learn a different model 14a for each group in principle, and may be able to select multiple groups that learn a common model 14a.

The calculation unit 15f calculates the feature amount for each selected node by combining the feature amounts estimated using the models 14a trained in each group. For example, as shown in FIG. 4(b), the calculation unit 15f combines all the feature amounts output by the models 14a trained in each group for each node to obtain the feature amount for the node.

The extraction unit 15g extracts the value of a dimension whose importance is equal to or greater than a predetermined threshold value from among the calculated features. Specifically, the extraction unit 15g uses the teacher data and the learned model 14a to calculate the importance of each dimension of the feature vector representing the feature of each node. For example, the extraction unit 15g calculates the importance of each dimension by Random Forest. Then, as shown in FIG. 4(c), the extraction unit 15g selects only important dimensions whose importance is equal to or greater than a predetermined threshold value, and sets them as the feature of the node.

Moreover, the extraction unit 15g outputs the calculated feature amount of each node via the output unit 12. Note that instead of or in addition to the extraction unit 15g, the calculation unit 15f may output the feature amount of each node calculated.

[Feature Calculation Processing]
Next, a feature calculation process performed by the feature calculation device 10 according to the present embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing a procedure of the feature calculation process. The flowchart in Fig. 5 is started, for example, when an operation input is made to instruct the start of the feature calculation process.

First, the generation unit 15b generates a graph representing communication between nodes using the communication information of the network nodes acquired by the acquisition unit 15a (step S1).

The selection unit 15c selects related nodes connected by paths of a predetermined length from among the nodes of the generated graph (step S2). The classification unit 15d classifies nodes within a predetermined distance on the path to the selected nodes into groups according to the distance between the nodes (step S3).

Next, the learning unit 15e learns a model 14a representing the features in the graph of each node in the group for each classified group (step S4).

At that time, the learning unit 15e learns a different model 14a for each classified group. Alternatively, the learning unit 15e may learn a common model 14a for multiple groups within a predetermined distance range among the classified groups.

Then, the calculation unit 15f calculates features for each selected node by combining the features estimated using the model 14a learned in each group (step S5).

In addition, the extraction unit 15g uses the teacher data and the trained model 14a to calculate the importance of each dimension of the feature vector representing the feature of each node. Then, the extraction unit 15g extracts only important dimensions whose importance is equal to or greater than a predetermined threshold value, and sets them as the feature of the node (step S6).

The extraction unit 15g also outputs the features of each node via the output unit 12 (step S7). This completes the series of feature calculation processes.

As described above, in the feature calculation device 10, the generation unit 15b uses communication information between nodes in the network to generate a graph representing communication between the nodes. The selection unit 15c selects related nodes connected by paths of a predetermined length from among the nodes in the generated graph. The classification unit 15d classifies nodes within a predetermined distance on the path from the selected nodes into groups according to the distance between the nodes. The learning unit 15e learns a model 14a representing the feature in the graph of each node in the group for each classified group. The calculation unit 15f then calculates the feature for each selected node by combining the feature estimated using the model 14a learned in each group.

In this way, the feature calculation device 10 divides the training data according to the distance between nodes, learns the similarity between nodes at each distance, and calculates the feature of each node by combining the features of each node learned at each different distance. This makes it possible to calculate the feature of a node taking into account the difference in the context of adjacent nodes due to distance. Therefore, the feature calculation device 10 is able to learn high-quality features from a graph representing a communication network.

In addition, the learning unit 15e learns a different model 14a for each classified group. This enables the feature calculation device 10 to learn the model 14a with higher accuracy.

In addition, the learning unit 15e learns a common model 14a for multiple groups within a predetermined distance range among the classified groups. This enables the feature calculation device 10 to learn the model 14a with high efficiency.

In addition, the extraction unit 15g uses the teacher data and the learned model 14a to calculate the importance of each dimension of the feature vector representing the feature of each node. Then, the extraction unit 15g extracts the value of the dimension with an importance equal to or greater than a predetermined threshold value from the calculated feature. This enables the feature calculation device 10 to efficiently calculate high-quality feature of each node.

[program]
A program in which the process executed by the feature amount calculation device 10 according to the above embodiment is written in a language executable by a computer can also be created. As an embodiment, the feature amount calculation device 10 can be implemented by installing a feature amount calculation program that executes the feature amount calculation process as package software or online software on a desired computer. For example, the feature amount calculation program can be executed by an information processing device, so that the information processing device can function as the feature amount calculation device 10. In addition, the information processing device also includes mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone System), and even slate terminals such as PDA (Personal Digital Assistant). The functions of the feature amount calculation device 10 may be implemented on a cloud server.

6 is a diagram showing an example of a computer that executes a feature calculation program. The computer 1000 has, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1031. The disk drive interface 1040 is connected to a disk drive 1041. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1041. The serial port interface 1050 is connected to a mouse 1051 and a keyboard 1052, for example. The video adapter 1060 is connected to a display 1061, for example.

Here, the hard disk drive 1031 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. Each piece of information described in the above embodiment is stored, for example, in the hard disk drive 1031 or memory 1010.

In addition, the feature calculation program is stored in the hard disk drive 1031, for example, as a program module 1093 in which instructions to be executed by the computer 1000 are described. Specifically, the program module 1093 in which each process executed by the feature calculation device 10 described in the above embodiment is described is stored in the hard disk drive 1031.

In addition, data used for information processing by the feature calculation program is stored as program data 1094, for example, in the hard disk drive 1031. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the hard disk drive 1031 into the RAM 1012 as necessary, and executes each of the above-mentioned procedures.

In addition, the program module 1093 and program data 1094 related to the feature calculation program are not limited to being stored in the hard disk drive 1031, and may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1041 or the like. Alternatively, the program module 1093 and program data 1094 related to the feature calculation program may be stored in another computer connected via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network), and read by the CPU 1020 via the network interface 1070.

The above describes an embodiment of the invention made by the inventor, but the present invention is not limited to the description and drawings that form part of the disclosure of the present invention according to this embodiment. In other words, other embodiments, examples, operational techniques, etc. made by those skilled in the art based on this embodiment are all included in the scope of the present invention.

REFERENCE SIGNS LIST 10 Feature quantity calculation device 11 Input unit 12 Output unit 13 Communication control unit 14 Storage unit 14a Model 15 Control unit 15a Acquisition unit 15b Generation unit 15c Selection unit 15d Classification unit 15e Learning unit 15f Calculation unit 15g Extraction unit

Claims (7)

A generation unit that generates a graph representing communications between nodes using communication information between nodes of the network;
a selection unit that selects, from among the nodes of the generated graph, related nodes that are connected by paths of a predetermined length;
a classification unit that classifies nodes on a path within a predetermined distance of the selected nodes into groups according to the distance between the nodes;
a learning unit that learns a model representing a feature amount in the graph of each node in the group for each of the classified groups;
a calculation unit that calculates a feature amount by combining feature amounts estimated using models trained in each group for each of the selected nodes;
A feature calculation device comprising: The feature calculation device described in claim 1, characterized in that the learning unit learns a different model for each of the classified groups. The feature calculation device described in claim 2, characterized in that the learning unit learns a common model for multiple groups within a predetermined distance range among the classified groups. The feature calculation device according to claim 1, further comprising an extraction unit that extracts, from the calculated features, values of dimensions having an importance equal to or greater than a predetermined threshold value. The feature calculation device described in claim 4, characterized in that the extraction unit uses training data and the learned model to calculate the importance of each dimension of the feature vector representing the feature of each node. A feature calculation method executed by a feature calculation device, comprising:
A generation step of generating a graph representing communications between nodes using communication information between nodes of the network;
a selection step of selecting, from among the nodes of the generated graph, related nodes connected by paths of a predetermined length;
a classification step of classifying nodes on a path within a predetermined distance of the selected nodes into groups according to the distance between the nodes;
a learning step of learning a model representing a feature amount in the graph of each node in each group, for each of the classified groups;
A step of calculating a feature amount by synthesizing the feature amounts estimated using the models trained in each group for each of the selected nodes;
A feature calculation method comprising: A generating step of generating a graph representing communications between the nodes using communication information between the nodes of the network in a computer;
a selection step of selecting, from among the nodes of the generated graph, related nodes connected by paths of a predetermined length;
a classification step of classifying nodes on a path within a predetermined distance of the selected nodes into groups according to the distance between the nodes;
a learning step of learning a model representing a feature amount in the graph of each node in each group, for each of the classified groups;
calculating a feature amount by combining the feature amounts estimated using the models trained in each group for each of the selected nodes;
A feature calculation program for executing the above.

Images