JP6088197B2 - Management device, bottleneck determination / path control program, and parallel distributed processing system - Google Patents

Description

  The present invention relates to a management apparatus, a bottleneck determination / path control program, and a parallel distributed processing system that perform a path control by OpenFlow and execute a task so that two or more terminal apparatuses are parallelized.

  In order to speed up a large amount of data processing such as high-definition video processing, dedicated and expensive computing devices and network devices have been required. However, in recent years, these devices have been improved in performance and cost, and parallel distributed processing using a general IA (Intel Architecture) server (that is, PC: (Personal Computer) server) and Ethernet (registered trademark) equipment. As a result, it is possible to increase the speed of data processing.

  While the conventional parallel distributed processing system improves the processing speed by increasing the number of servers, the network may be congested by inter-server traffic by consolidating and processing many servers (for example, non-patent) Reference 1). For this reason, in the conventional parallel distributed processing system, even if the number of servers is increased or the server performance is improved, the network may become a bottleneck and the processing speed may not increase.

  Here, the conventional parallel distributed processing system can construct a network at a relatively low cost by using an Ethernet device. When this Ethernet device is used, a spanning tree protocol is often used in a layer 2 network in order to avoid a loop problem due to a broadcast storm. However, this spanning tree protocol makes effective use of bandwidth because it logically blocks redundant paths other than the main one of the multiple paths existing in the layer 2 network until a failure occurs. I can't.

  Therefore, TRILL and OpenFlow have been proposed as methods that can effectively use a plurality of routes in a layer 2 network (for example, Non-Patent Documents 2 and 3). This TRILL selects the shortest path when the lengths of the plurality of paths are different, and selects a path with a preset algorithm when the lengths of the plurality of paths are the same. As a result, TRILL can distribute the network load and eliminate paths that are blocked and not used.

In OpenFlow, of the path control and packet transfer control performed by the conventional network switch, only the packet transfer control is performed by the OpenFlow compatible switch, and the path control is performed by the OpenFlow controller. At this time, in OpenFlow, a route can be changed in units of flows by a preset algorithm.
A flow is an arbitrary combination of identifiers such as an input switch port (physical port), a source / destination MAC address, a source / destination IP address, and a source / destination port number.

  Furthermore, the OpenFlow controller can detect the network topology by exchanging Link Layer Discovery Protocol (LLDP) packets with the OpenFlow compatible switch (for example, Non-Patent Document 4). This LLDP is a protocol for discovering neighboring nodes, and notifies neighboring nodes on the network of information related to the own device and obtains information related to other devices.

Huang et al., “Examination of parallel and distributed processing methods that guarantee processing speed”, IEICE Society Conference, B-7-60, 2011 IETF RFC5556, “Transparent Interconnection of Lots of Links (TRILL): Problem and Applicability Statement” The OpenFlow Switch Consortium (http://www.Openflowswitch.org/) IEEE802.1AB, “Station and Media Access Control Connectivity Discovery”

  However, in a conventional parallel distributed processing system, even when a path control technique such as OpenFlow is applied, only information related to the network is considered, so that a bottleneck related to the task being executed cannot be detected accurately. In addition, since the conventional parallel distributed processing system only considers information about the network, path control to avoid bottlenecks affects other flows used in the task being executed, creating a new bottleneck. May be induced.

  Therefore, the present invention provides a management device, a bottleneck determination / path control program, and a parallel distributed processing system that accurately detect a bottleneck related to a task being executed and do not induce a new bottleneck. Let it be an issue.

  In order to solve the above-described problem, a management device according to the first invention of the present application includes a transmission source terminal device that is a terminal device that accumulates data, and a terminal device that performs predetermined information processing on data accumulated in the transmission source terminal device A parallel processing unit comprising: an information processing terminal device, a transmission destination terminal device that is a terminal device that stores data processed by the information processing terminal device, and a switch that transfers the data between the terminal devices by OpenFlow. At least an information processing terminal that executes a task that is used in a distributed processing system, performs path control for a switch by OpenFlow, performs information processing on the data of the transmission source terminal device by the information processing terminal device, and transmits the data to the transmission destination terminal device A management device that instructs a terminal device to be parallelized by two or more devices, the network topology detecting means; To a road calculating means, a routing means, and the task input unit, and the terminal device determining unit, a terminal apparatus control unit, and bottlenecks determining means, comprising: a.

  According to such a configuration, the management device receives, from the terminal device, node information indicating the switch connected to the terminal device by the network topology detection unit, and from the switch, the terminal device or other switch connected to the switch Is detected, and based on the node information received from the terminal device and the switch, a network topology which is a connection form of the terminal device and the switch is detected.

  In addition, the management device, for each flow that is specified by the port number of the process activated by the terminal device that executes the task by the route calculation unit and that is a unit of the OpenFlow route control, is based on the network topology. A shortest path or a minimum cost path connecting the two is calculated as a main path, and path information indicating the calculated main path is generated.

  In addition, the management device transmits the route information generated by the route calculation unit to the switch by the route control unit. And a management apparatus inputs the task information containing the identification information of the terminal device which performs a task by a task input means. Furthermore, the management device determines the terminal device that executes the task based on the task information by the terminal device determination means.

  In addition, the management device generates transfer connection information in which the terminal device control unit associates the terminal device identification information determined by the terminal device determination unit with the port number of the process unique to each flow, and Command the terminal device to start up and create a transfer connection.

In addition, the management apparatus determines whether or not a bottleneck has occurred in the task being executed by using the bottleneck determination unit in consideration of not only information regarding the network but also information regarding the process.
Information related to the process is transfer connection information and task completion information.
The information regarding the network is path information, node information, and data transfer amount information.

  In addition, when it is determined by the route calculation means that a bottleneck has occurred in the task being executed, the management device considers not only information related to the network but also information related to the process, and the process executing the task The detour route of the flow used in is calculated, and detour route information indicating the calculated detour route is generated.

Further, in the management device according to the second invention of the present application, the network topology detection means determines whether or not the network topology has been changed based on the difference in network topology, and when the network topology is changed, based on the path information, It is determined whether or not the main route has been deleted, and when the main route is deleted, the route calculation means calculates a main route that substitutes for the deleted main route based on the changed network topology. Features.
According to this configuration, the management apparatus can calculate the optimum main route even when the main route is deleted.

The management device according to the third invention of the present application is characterized in that the route control means transmits detour route information to the switch for a preset number of flows.
According to this configuration, the management device bypasses all the flow paths that are bottlenecks at once, and a large amount of communication is avaliated into the flow paths in which the bottleneck has not occurred. The bottleneck can be prevented from being induced.

Further, in the management device according to the fourth invention of the present application, the route calculation means determines whether or not a bypass route exists based on the calculation result of the bypass route. It is characterized by presenting a message proposing the addition of resources.
According to such a configuration, the management device can propose addition of network resources to the user of the parallel distributed processing system when there is no detour path.

  In order to solve the above-described problem, a parallel distributed processing system according to the fifth invention of the present application is a transmission source terminal device that accumulates data, and an information processing terminal that performs predetermined information processing on the data accumulated in the transmission source terminal device The data is transferred between the terminal device that plays one or more roles among the device or the destination terminal device that stores the data processed by the information processing terminal device, and the terminal device using OpenFlow. A switch and a management device according to the first aspect of the present invention are provided.

  According to such a configuration, the parallel distributed processing system determines whether or not a bottleneck has occurred in the task being executed by the management apparatus in consideration of not only information related to the network but also information related to the process.

  In addition, when the parallel distributed processing system determines that a bottleneck has occurred in the task being executed by the management device, the task is being executed in consideration of not only information related to the network but also information related to the process. A detour route of the flow used in the process is calculated, and detour route information indicating the calculated detour route is generated.

  The management device according to the first invention of the present application uses hardware resources such as a CPU, a memory, and a hard disk provided in a computer as network topology detection means, route calculation means, route control means, task input means, terminal device determination means, and terminal. It can also be realized by a bottleneck determination / route control program for functioning as device control means and bottleneck determination means (the sixth invention of the present application). This program may be distributed through a communication line, or may be distributed by writing in a recording medium such as a CD-ROM or a flash memory.

According to the present invention, the following excellent effects can be obtained.
According to the first, fifth, and sixth inventions of the present application, both the information on the process and the information on the network are used, so that the bottleneck can be accurately detected and the path control is performed without generating a new bottleneck. Can do.
According to the second invention of the present application, even when the main route is deleted, the optimum main route can be calculated, so that the route control can be performed without generating a new bottleneck.

According to the third invention of the present application, it is possible to prevent a large amount of communication traffic from flowing into the path of a flow in which no bottleneck has occurred, and thus it is possible to suppress the occurrence of a new bottleneck.
According to the fourth aspect of the present invention, when there is no detour path, it is possible to propose addition of network resources to the user of the parallel distributed processing system.

1 is a schematic diagram of a parallel distributed processing system according to an embodiment of the present invention. It is explanatory drawing explaining the conventional OpenFlow. It is a block diagram which shows the structure of the parallel distributed processing system of FIG. (A)-(c) is explanatory drawing explaining the change of a network topology in the parallel distributed processing system of FIG. It is explanatory drawing explaining the main path | route in the parallel distributed processing system of FIG. It is a sequence diagram which shows operation | movement of the parallel distributed processing system of FIG. 7 is a flowchart of a network topology change determination / route control process of FIG. 6. It is a flowchart of the parallel process control process of FIG. 7 is a flowchart of bottleneck detection / detour route control processing in FIG. 6.

[Outline of parallel distributed processing system]
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate.
With reference to FIG. 1, the outline of the parallel distributed processing system 1 which concerns on embodiment of this invention is demonstrated.

The parallel distributed processing system 1 performs parallel distributed processing of data using the processing device 2, and includes the processing device 2 and a management device 10.
The processing device 2 performs parallel distributed processing according to control from the management device 10 and includes a server (terminal device) 20, a switch 30, and a communication line 40.

  The management device 10 collects and manages information necessary for parallel distributed processing from the processing device 2 and controls the processing device 2. Specifically, the management device 10 performs processing such as network topology change determination, path control, parallel process control, bottleneck detection, and detour path control, which will be described later (FIG. 6).

The server 20 plays one or more roles of the transmission source terminal device 20 ₁ , the information processing terminal device 20 ₂ , or the transmission destination terminal device 20 ₃ .
Source terminal apparatus 20 ₁ is adapted to accumulate the data to be parallel distributed processing, divided by the accumulated pre-specified size data (granularity) and packetized, and transmitted to the information processing terminal device 20 ₂ Server 20.
The information processing terminal device 20 ₂ is subjected to processing in the packet data received from the transmission source terminal device 20 _1, a server 20 transmits the processing result to the destination terminal device 20 _3.
Destination terminal apparatus 20 ₃ restores the data by arranging the packets of data received from the information processing terminal device 20 ₂ in a predetermined order, a server 20 for storing.

The switch 30 performs packet transfer control between the servers 20. In this embodiment, since OpenFlow is used as the path control technique, the switch 30 is an OpenFlow compatible switch.
The communication line 40 is, for example, a LAN (Local Area Network) cable that connects the server 20 and the switch 30.

<Example of information processing>
Hereinafter, an example of information processing performed in the parallel distributed processing system 1 will be described.
For example, the parallel distributed processing system 1 can perform video processing such as transcoding on video data as information processing. That is, the parallel distributed processing system 1 packetizes by dividing the image data stored in the transmission source terminal device 20 ₁ is transmitted to the information processing terminal device 20 _2. Further, the parallel distributed processing system 1, the information processing terminal device 20 _2, subjected to image processing in parallel with the packet of the video data, the transmission destination terminal device a packet of video data image processing has been performed 20 ₃ Send to. The parallel distributed processing system 1, the transmission destination terminal apparatus 20 _3, and restore the image processing processed data by arranging packets of video data image processing is performed in a predetermined order, it accumulates.
Needless to say, the information processing in the present invention is not limited to video processing.

<Description of OpenFlow>
The OpenFlow used in the parallel distributed processing system 1 will be specifically described with reference to FIG.
In FIG. 2, the OpenFlow controller is abbreviated as “controller”, and the OpenFlow compatible switch 30 is abbreviated as “switch”.
Further, in FIG. 2, input / output of signals and information associated with path control between the OpenFlow controller 10 and the OpenFlow compatible switch 30 is illustrated by a one-dot chain line.
In the present embodiment, the management apparatus 10 in FIG. 1 corresponds to the OpenFlow controller 10. Further, the switch 30 in FIG. 1 corresponds to the OpenFlow compatible switch 30.

  As shown in FIG. 2, the OpenFlow compatible switch 30 performs only packet transfer control among path control and packet transfer control in a conventional network switch. On the other hand, the OpenFlow controller 10 performs centralized path control on the OpenFlow compatible switch 30.

In OpenFlow, a flow is a path control unit, and the concept of action, statistical information, and matching rule is applied to each flow.
In this embodiment, one data transfer process identified by the destination port number is a flow. In other words, each flow can be identified by the port number (destination port number) of the process activated by the server 20 that executes the task. Therefore, the output switch port is switched for each flow according to the value of the destination port number.
An action is a process definition set in advance for each matching rule. For example, in the action, a direct route can be set between the server 20 and the switch 30 by designating the output switch port or rewriting the destination MAC address.
The statistical information is, for example, a data transfer amount (number of bytes) for each flow. Here, the controller 10 can determine the occurrence of a packet loss using the statistical information of the two adjacent switches 30.

  The matching rule is a rule for identifying a flow based on identification information such as an input switch port, a source / destination MAC (Media Access Control) address, a source / destination IP (Internet Protocol) address, and a source / destination port number. is there. Here, there are three packet A (input switch port number 1, destination port number 10000), packet B (input switch port number 2, destination port number 10000), and packet C (input switch port number 1, destination port number 10001). Consider the case where there are packets. In this case, if “input switch port = 1” is a matching rule, packets A and C have the same flow. If “destination port number = 10000” is a matching rule, packets A and B have the same flow. As described above, when receiving the packet, the OpenFlow compatible switch executes an action determined in accordance with the matching rule for communication of the packet that matches the matching rule of the stored route information. As a result, in OpenFlow, the path 50 can be controlled for each flow. Further, in OpenFlow, when a packet matches a plurality of matching rules, an action determined corresponding to a matching rule having a high priority is executed.

Here, it supplements about the relationship between a flow and a process.
As described above, since the flow and the process correspond to each other, the data transfer process between the processes activated by the server 20 becomes one flow. For example, the transmission data transfer processing between the processes started in the source terminal device 20 processes started by ₁ and the information processing terminal device 20 ₂ becomes 1 flow, process and the destination terminal which has started in the information processing terminal device 20 ₂ data transfer processing between the processes started in the device 20 ₃ is 1 flow.

[Configuration of management device]
The configuration of the management apparatus 10 will be described with reference to FIG.
In FIG. 3, in order to make the input / output of information necessary for parallel distributed processing easy to understand, arrows indicating input / output of each information are illustrated directly between the management apparatus 10 and the server 20. Each information is input / output via the switch 30.

  As shown in FIG. 3, the management device 10 includes a task input unit 110, an information management unit 120, an information display unit 130, a parallel process calculation unit (terminal device determination unit) 140, and a parallel process control unit (terminal device). Control means) 150, route calculation means 160, and route control means 170.

The task input means 110 is for the user to input task information. For example, the task input unit 110 may allow the user to input task information via an interface such as a web browser, or may describe the contents of the task information in a text file. Then, the task input unit 110 outputs the input task information to the parallel process calculation unit 140.
A user is, for example, an administrator or a user of the parallel distributed processing system 1.

  The task information is information necessary for parallel distributed processing. For example, the identification information of the server 20 that executes the task, the number of parallel distributed processing (number of parallelization), the data division size (granularity), parallel A path to data (file) used for distributed processing is included.

For example, the identification information of the server 20 includes the IP address of the server 20 to perform a task as a transmission source terminal device 20 _1, and the IP address of the server 20 to perform the task as a destination terminal device 20 _3.

Further, the number of parallelization, the number of performing the parallel distributed processing, i.e., contains the number of processes started by the information processing terminal device 20 _2. For example, parallel number, when the image processing for changing the color of certain image, the number of processes that are started by the information processing terminal device 20 ₂ for the video processing.

The process is a program that executes information processing according to the role of the server 20, and is started by parallel processing execution means 230 (not shown). In the present embodiment, it is assumed that each server 20 starts only one process regardless of the roles of the transmission source terminal device 20 ₁ , the information processing terminal device 20 _2, and the transmission destination terminal device 20 ₃ .

  The information management unit 120 manages (accumulates) information necessary for parallel and distributed processing, and includes a network topology detection unit 121, a bottleneck determination unit 122, a task completion determination unit 123, and a storage unit 124. .

  The network topology detection unit 121 detects a network topology based on node information stored in the storage unit 124 described later. Further, the network topology detection unit 121 determines whether or not the network topology has been changed based on the difference between the detected network topologies. Then, when the network topology is changed, the network topology detection unit 121 generates network topology information indicating the changed network topology and stores the network topology information in the storage unit 124.

<Change of network topology>
With reference to FIG. 4, the change of the network topology will be specifically described (see FIG. 3 as appropriate).
In FIG. 4, the server 20 connected to the switches 30a and 30b is not shown.

The network topology is a connection form of nodes (server 20 and switch 30) included in the processing device 2.
Here, the node information indicates the switch 30 connected to the server 20, the server 20 connected to the switch 30, or another switch 30. For this reason, if the node information of all nodes is used, the network topology detecting unit 121 can know the other nodes connected to each node, so that the connection form between the nodes can be grasped.

  The path 50 is a communication path formed by one or more switches 30 and the communication line 40 and connecting the two servers 20. In the present embodiment, a path 50 is set for each flow.

As shown in FIG. 4A, it is assumed that three switches 30a, 30b, and 30c exist in the network. The switches 30a and 30b are connected by a communication line 40, and the switches 30b and 30c are connected by another communication line 40. In this case, the network topology detection unit 121 detects the network topology shown in FIG. 4A, generates network topology information indicating the detected network topology, and stores it in the storage unit 124. Furthermore, the network topology detection unit 121 determines that the network topology has been changed because the network topology in FIG. 4A is detected in a state where the network topology does not exist.
In FIG. 4 (a), the results in the path 50 ₁ for connecting switches 30a, 30b are directly present one.

As shown in FIG. 4B, it is assumed that another communication line 40 is added between the switches 30a and 30c to the network topology of FIG. In this case, the network topology detecting unit 121 detects the network topology shown in FIG. 4B and generates network topology information indicating the detected network topology. Then, the network topology detection unit 121 updates the network topology information indicating the network topology of FIG. 4A stored in the storage unit 124 with the network topology information indicating the network topology of FIG. 4B. Furthermore, the network topology detection unit 121 determines that the network topology has been changed because there is a difference between the network topologies of FIGS. 4A and 4B.
Incidentally, in FIG. 4 (b), the path 50 ₁ for connecting switches 30a, 30b are directly, so that the switch 30a, 30c, a total of two existence of a path 50 ₂ passing through in the order of 30b to.

As shown in FIG. 4C, it is assumed that the communication line 40 is deleted between the switches 30a and 30b from the network topology of FIG. 4B. In this case, the network topology detection unit 121 detects the network topology in FIG. 4C and generates network topology information indicating the detected network topology. Then, the network topology detection unit 121 updates the network topology information indicating the network topology of FIG. 4B stored in the storage unit 124 with the network topology information indicating the network topology of FIG. 4C. Further, the network topology detection unit 121 determines that the network topology has been changed because there is a difference between the network topologies of FIGS. 4B and 4C.
In FIG. 4 (c), the results in the switch 30a, 30c, passage 50 ₂ through which in this order and 30b are present one. As described above, the communicable path 50 varies according to the change in the network topology.

Further, the network topology detection unit 121 performs the following processing when the network topology is changed as shown in FIGS. 4 (a) to 4 (c).
Based on the route information stored in the storage unit 124, the network topology detection unit 121 determines whether the communication line 40 of the main route has been deleted due to a change in the network topology. For example, a path 50 ₁ major pathway of FIG. 4 (b), then, consider the case where the communication line 40 is deleted between the switch 30a, 30b as shown in FIG. 4 (c). In this case, the network topology detection means 121, since the communication line 40 included in the path 50 ₁ is the main route is deleted, the identification information of the deleted communication line 40, as the communication circuit deletion information, the route calculation unit 160 Output.

<Description of main routes>
The main route 60 will be specifically described with reference to FIG.
The main route 60 is a route 50 that is most suitable for data transfer among one or more routes 50 (FIG. 4) connecting two servers 20. For example, when two servers 20 are connected by four routes 50, the optimum one route 50 among the four routes 50 becomes the main route 60 (not shown).
The calculation method of the main route 60 will be described in the route calculation means 160.

Furthermore, as shown in FIG. 5, two servers 20 plays the role of a source terminal device 20 _1, three server 20 plays the role of the information processing terminal device 20 _2, one server 20 is transmitted consider the case that plays the role of the former terminal device 20 _3.
In FIG. 5, illustration of the switch 30 arranged between the servers 20 is omitted.

In this case, the two source terminals 20 _1, between the three information processing terminal device 20 _2, the main path 60 becomes the six obtained by multiplying the number of both apparatuses. Furthermore, the information processing terminal device 20 ₂ for three, one server 20 is in between the transmission destination terminal device 20 _3, the main path 60 becomes the three obtained by multiplying the number of both apparatuses. That is, in the example of FIG. 5, there are nine main routes 60 in total.

Returning to FIG. 3, the description of the configuration of the management apparatus 10 will be continued.
The bottleneck determination means 122 determines whether or not a bottleneck has occurred in the task being executed based on transfer connection information, route information, task completion information, node information, and data transfer amount information described later. Judgment. In other words, the bottleneck determination unit 122 determines whether or not a bottleneck has occurred in the task being executed in consideration of not only information regarding the network but also information regarding the process.

<Determination of bottleneck>
The bottleneck determination will be specifically described.
Specifically, the bottleneck determination unit 122 detects a packet loss that has occurred between the two switches 30 based on the data transfer amount information transmitted from the switch 30. When a packet loss is detected, the bottleneck determination unit 122 determines that a bottleneck has occurred in the communication line 40 connected between the switches 30.

  When a bottleneck has occurred in the communication line 40, the bottleneck determination unit 122 identifies in which flow the bottleneck has occurred based on the path information and the node information. That is, the bottleneck determination unit 122 uses the communication line 40 in which the bottleneck has occurred (that is, a bottleneck) because the route information indicates the main route for each flow and the node information indicates the network topology. Flow) can be specified.

  When the bottleneck flow can be identified, the bottleneck determination unit 122 determines whether a bottleneck has occurred in the task being executed based on the transfer connection information and the task completion information. In other words, the bottleneck determination unit 122 uses the flow that becomes the bottleneck because the transfer connection information indicates the correspondence between the task, the process, and the flow, and the task completion information indicates completion or execution of the task. You can identify the tasks inside.

  Here, when a bottleneck occurs in the task being executed, the bottleneck determination unit 122 obtains the identification information of the communication line 40 in which the bottleneck has occurred and the identification information of the flow using the communication line 40. The bottleneck information is output to the route calculation unit 160 and is also stored in the storage unit 124.

On the other hand, the bottleneck determination unit 122 does not perform any processing when the bottleneck does not occur in the communication line 40.
Furthermore, there is a case where a flow that becomes a bottleneck cannot be identified, or a bottleneck does not occur in a task being executed. Also in this case, since the communication line 40 in which the bottleneck has occurred is not used in the flow, the bottleneck determination unit 122 does not affect the task execution even if the bottleneck is left, and does not perform any processing.

  The task completion determination unit 123 determines whether the task has been completed based on the task completion information input from the server 20. That is, when the task completion information is input, the task completion determination unit 123 determines that the task has been completed. When the task is completed, the task completion determination unit 123 deletes the transfer connection information of the completed task from the storage unit 124.

  The task completion determination unit 123 determines whether there is a task being executed based on the transfer connection information stored in the storage unit 124. That is, when the transfer connection information is stored in the storage unit 124, the task completion determination unit 123 determines that there is a task being executed. On the other hand, when the transfer connection information is not stored in the storage unit 124, the task completion determination unit 123 determines that there is no task being executed. If there is no task being executed, the task completion determination unit 123 outputs a route calculation instruction to the route calculation unit 160.

  The storage unit 124 includes transfer connection information generated by the parallel process control unit 150, route information and detour route information generated by the route calculation unit 160, network topology information generated by the network topology detection unit 121, and bottleneck determination unit. Reference numeral 122 denotes a storage device such as a memory or a hard disk for storing the bottleneck information generated.

  The information display means 130 presents the network topology information, path information, and bottleneck information stored in the storage means 124 to the user. The information display means 130 presents the message input from the route calculation means 160 to the user. For example, the information display unit 130 displays network topology information, route information, bottleneck information, and a message on a display (not shown).

The parallel process calculation unit 140 determines the server 20 that executes the task based on the task information input from the task input unit 110.
For example, the parallel process calculation unit 140 determines the server 20 that executes the task as follows.
In this case, parallel process calculating section 140 refers to the identification information of the task information, determines the transmission source terminal device 20 ₁ and the transmission destination terminal device 20 _3. Also, the parallel process calculating means 140 of the transmission source terminal device 20 ₁ and the transmission destination terminal device 20 ₃ remaining server 20 that has not been determined as the server 20 of the same number as the parallel number of task information, a round-robin It is determined as the information processing terminal device 20 _2.

  Then, the parallel process calculation unit 140 outputs server identification information indicating the determined identification information of the server 20 and the number of processes activated on each server 20 to the parallel process control unit 150.

The parallel process control unit 150 instructs the server 20 to start a process and create a transfer connection according to the server identification information input from the parallel process calculation unit 140, and notifies parameters necessary for parallel distributed processing. is there. At this time, since the parallel process control means 150 assigns a different destination port number for each process and can start a plurality of processes, data transfer processing and video processing processing can be parallelized.
The parameters are, for example, the presence / absence of transcoding and the bit rate in video processing.

Further, the parallel process control unit 150 generates transfer connection information by starting a task in accordance with the server specifying information, and stores the generated transfer connection information in the storage unit 124.
The transfer connection information is, for example, information in which task identification information, server 20 identification information included in the server identification information, flow identification information, and a process port number unique to each flow are associated with each other. .
For example, task identification information and task ID (IDentification) uniquely assigned in the parallel distributed processing system 1.
For example, the flow identification information is a flow ID uniquely assigned in the parallel distributed processing system 1.

The route calculation means 160 calculates a route.
In the present embodiment, the route calculation means 160 calculates all the main routes (step S17 in FIG. 7), calculates a new main route that replaces the deleted main route (step S14 in FIG. 7), Three types of route calculation are performed: calculation of a detour route (step S31 in FIG. 9). Hereinafter, three types of route calculation will be described in order.

<First type: Calculation of all major routes>
Based on the network topology information stored in the storage unit 124, the route calculation unit 160 calculates the shortest route connecting the servers 20 as the main route for all servers 20 in which a flow occurs. For example, the route calculation means 160 calculates the shortest route in the Dijkstra method as the main route. The route calculation unit 160 generates route information indicating the calculated main route, and outputs the generated route information to the route control unit 170 and stores it in the storage unit 124.

  This Dijkstra method is described, for example, in the reference document “E.W.

<Second type: Calculation of new main route>
When the communication line deletion information is input from the network topology detection unit 121, the route calculation unit 160 calculates a new main route based on the network topology information stored in the storage unit 124. This network topology information reflects changes in the network topology. For example, the route calculation means 160 calculates the shortest route in the Dijkstra method as a new main route instead of the main route that used the communication line 40 indicated by the communication line deletion information. The route calculation unit 160 generates route information indicating the calculated new main route, and outputs the generated route information to the route control unit 170 and stores it in the storage unit 124.

<Third type: Calculation of detour route>
When the bottleneck information is input from the bottleneck determination unit 122, the route calculation unit 160 is used in the process that is executing the task based on the transfer connection information, the route information, the task completion information, and the node information. The detour route that can detour the current flow is calculated. That is, the route calculation unit 160 calculates a detour route in consideration of not only information related to the network but also information related to the process.

  Specifically, the route calculation unit 160 identifies the process and flow used by the task being executed based on the transfer connection information and the task completion information. That is, the route calculation means 160 indicates that the transfer connection information indicates the correspondence between the task, the process, and the flow, and the task completion information indicates whether the task is completed or being executed. Can be specified.

  Next, the route calculation means 160 makes a detour based on the route information and the node information so as not to affect the flow used by the task being executed or a task with high importance (for example, a stream processing task). Calculate the route. For example, the route calculation unit 160 calculates the shortest route in the Dijkstra method as a bypass route using the virtual network topology obtained by removing the communication line 40 indicated by the bottleneck information from the network topology information stored in the storage unit 124. To do.

  Then, the route calculation unit 160 determines that the detour is possible when the detour route exists, and determines that the detour is not possible when the detour route does not exist. For example, when only one route exists between the two servers 20 and a bottleneck is detected on the one route, the route calculation unit 160 determines that the detour is impossible.

When the detour is possible, the route calculation unit 160 generates detour route information indicating the calculated detour route, and outputs the generated detour route information to the route control unit 170 and stores it in the storage unit 124.
On the other hand, when the detour is not possible, the route calculation unit 160 presents a preset message to the user via the information display unit 130. This message is a message for proposing addition of a network resource to the user of the parallel distributed processing system 1.

  The route control unit 170 transmits the route information input from the route calculation unit 160 to the switch 30 on the main route indicated by the route information and stores it in the storage unit 124. That is, in a normal state, the route calculation unit 160 performs route control so that the flow passes through the main route.

  Further, the route control unit 170 transmits the detour route information input from the route calculation unit 160 to the switch 30 on the route indicated by the detour route information and accumulates it in the accumulation unit 124. Here, when the flow is diverted to the detour path, the path control unit 170 can control the path so as to pass the detour path by increasing the priority in the OpenFlow.

  If all of the flow paths that are bottlenecks are detoured at once, a large amount of communication may be drowned into the flow paths in which the bottleneck has not occurred, and a new bottleneck may be induced. In order to prevent this, the route control unit 170 transmits detour route information to the switch for a preset number of flows (for example, one flow).

  Further, the route control unit 170 may add the survival time to the detour route information and invalidate the detour route information when the lifetime has elapsed. In this case, the route control unit 170 may use the route information before the detour again and delete the detoured route information that has become invalid from the storage unit 124.

[Server configuration]
Hereinafter, the configuration of the server 20 will be described.
As shown in FIG. 3, the server 20 includes an instruction execution unit 210, an information collection unit 220, a parallel processing execution unit 230, and a storage unit 240.
The instruction execution unit 210 receives an instruction from the management apparatus 10 and outputs the received instruction to the parallel processing execution unit 230 so that the parallel processing execution unit 230 executes the instruction.

  The information collecting means 220 finds adjacent nodes by exchanging ARP (Address Resolution Protocol) packets and LLDP packets. Then, the information collecting unit 220 transmits the discovered node information together with the identification information of the server 20 to the management apparatus 10.

  The parallel processing execution unit 230 executes parallel processing in accordance with the instruction input from the instruction execution unit 210. Specifically, the parallel processing execution unit 230 starts a process necessary for executing the task and creates a transfer connection in accordance with this instruction.

Specifically, the parallel processing execution unit 230 activates a process according to the roles of the transmission source terminal device 20 ₁ , the information processing terminal device 20 _2, and the transmission destination terminal device 20 ₃ and executes the command.
For example, if the server 20 operates as a transmission source terminal device 20 _1, the parallel process execution means 230 starts the process of sending a packet of video data. Then, the parallel process execution unit 230, the process started, packetizes by dividing the data stored in the storage unit 240, to the information processing terminal device 20 _2.
Further, for example, if the server 20 operates as the information processing terminal device 20 _2, the parallel process execution means 230 starts the process for performing image processing. Then, the parallel process execution unit 230, the process started, is subjected to processing in the packet data received from the transmission source terminal device 20 _1, it transmits the processing result to the destination terminal device 20 _3.
Further, for example, if the server 20 is operated as the transmission destination terminal device 20 _3, the parallel process execution means 230 receives the packet of the video data to the video processing has been performed, to restore arranged in a predetermined order Start the process. Then, the parallel process execution unit 230, the process started, the packet of data received from the information processing terminal device 20 _2, restores the information processing data already by arranging in a predetermined order, stored in the storage unit 240 To do.

In addition, the parallel processing execution unit 230 transmits task completion information indicating the completion of the task to the management apparatus 10 when a preset task completion condition is satisfied.
The task completion condition is a condition indicating that the task is completed. For example, task completion conditions, if the server 20 is operated as the transmission destination terminal device 20 _3, from the transmission destination terminal device 20 ₃ is the information processing terminal device 20 ₂ is the condition that it has received all the data after processing .

  The storage unit 240 is a storage device such as a memory or a hard disk that stores data to be subjected to parallel distributed processing.

[Switch configuration]
As shown in FIG. 3, the switch 30 includes an instruction execution unit 310, an information collection unit 320, and a packet transfer unit (data transfer unit) 330.
The instruction execution unit 310 receives route information from the management apparatus 10 and causes the packet transfer unit 330 to execute packet transfer processing according to the received route information.

The information collection unit 320 collects the data transfer amount of the switch 30 as OpenFlow statistical information, and transmits the data transfer amount information indicating the collected data transfer amount to the management apparatus 10.
Further, the information collecting unit 320 finds adjacent nodes by exchanging ARP packets and LLDP packets. Then, the information collecting unit 320 transmits the discovered node information to the management device 10.

  The packet transfer unit 330 transfers the received data packet in accordance with the path information input from the instruction execution unit 310. Further, the packet transfer unit 330 transfers information necessary for parallel and distributed processing between the management apparatus 10 and the server 20.

[Operation of parallel distributed processing system]
Hereinafter, the operation of the parallel distributed processing system 1 will be described with reference to FIG. 6 (see FIG. 3 as appropriate).
The management device 10 uses the node information transmitted from the processing device 2 (the server 20 and the switch 30) to perform a network topology change determination / route control process, and transmits the route information to the processing device 2 (the switch 30). (Step S1).

The management device 10 performs parallel process control processing according to the input task information, and transmits an instruction to the processing device 2 (server 20) (step S2).
The management device 10 uses the data transfer amount information transmitted from the processing device 2 (switch 30), performs bottleneck detection / detour route control processing, and transmits the detour route information to the processing device 2 (switch 30). (Step S3).

[Network topology change judgment / route control processing]
With reference to FIG. 7, the network topology change determination / path control processing will be specifically described (see FIG. 3 as appropriate).

The management apparatus 10 collects (receives) node information from the server 20 and the switch 30 by the information management unit 120.
The management apparatus 10 uses the network topology detection unit 121 to detect the network topology based on the node information.
The management apparatus 10 determines whether or not the network topology has been changed by the difference in the detected network topology by the network topology detection unit 121 (step S10).

When the network topology is changed (Yes in step S10), the management device 10 proceeds to the process of step S11.
The management apparatus 10 updates the network topology information stored in the storage unit 124 with the network topology information indicating the changed network topology by the network topology detection unit 121 (step S11).
The management apparatus 10 presents network topology information, route information, and bottleneck information to the user by the information display unit 130 (step S12).

The management apparatus 10 determines whether or not the communication line 40 on the main route has been deleted by the network topology detection unit 121 based on the route information (step S13).
When the communication line 40 on the main route is deleted (Yes in step S13), the management apparatus 10 proceeds to the process of step S14.

The management apparatus 10 uses the network topology detection unit 121 to generate communication line deletion information and deletes the route information using the deleted communication line 40 from the storage unit 124.
Based on the changed network topology information, the management device 10 calculates a new main route instead of the main route using the communication line 40 indicated by the communication line deletion information, based on the changed network topology information. Route information indicating the new main route is generated (step S14).

  The management apparatus 10 causes the route control unit 170 to transmit the route information generated in step S14 to the switch 30 on the main route indicated by the route information and store the route information in the storage unit 124 (step S15).

  The management apparatus 10 determines whether or not there is a task being executed by the task completion determination unit 123 based on the transfer connection information input from the parallel process control unit 150 (step S16).

When there is a task being executed (Yes in step S16), the management apparatus 10 returns to the process of step S16 and waits for the task being executed to end. As a result, the path control significantly changes the path of the flow being transferred, and can prevent a new bottleneck from being induced.
On the other hand, if there is no task being executed (No in step S16), the management apparatus 10 proceeds to the process of step S17.

  Based on the network topology information, the management device 10 calculates the main route for all servers 20 that can generate a flow, and generates route information indicating the calculated main route (step S17).

  The management apparatus 10 causes the route control unit 170 to transmit the route information generated in step S17 to the switch 30 on the main route indicated by the route information and store the route information in the storage unit 124 (step S18).

  If the network topology has not been changed (No in step S10), or after step S18, the management apparatus 10 ends the network topology change determination / path control process.

[Parallel process control processing]
The parallel process control processing will be specifically described with reference to FIG. 8 (see FIG. 3 as appropriate).
In FIG. 8, it is assumed that the calculation of the main route and the route control are completed between the servers 20 before the task information is input.

The management apparatus 10 receives the task information from the task input unit 110 and outputs the input task information to the parallel process calculation unit 140 (step S20).
Based on the task information, the management apparatus 10 determines the server 20 that executes the task, and generates server specifying information (step S21: parallel process calculation).

The management apparatus 10 generates the transfer connection information by starting the task according to the server identification information by the parallel process control means 150 and instructs the server 20 to start the process and create the transfer connection (step S22: parallel process). control).
The management apparatus 10 stores the transfer connection information in the storage unit 124 by the parallel process control unit 150 (step S23).

The management apparatus 10 determines whether the task is completed by the task completion determination unit 123 (step S24).
If the task has not been completed (No in step S24), the management apparatus 10 returns to the process in step S24.

When the task is completed (Yes in step S24), the management apparatus 10 proceeds to the process of step S25.
The management apparatus 10 deletes the transfer connection information of the completed task from the storage unit 124 by the task completion determination unit 123 (step S25).

[Bottleneck detection and detour route control processing]
With reference to FIG. 9, the bottleneck detection / detour route control will be specifically described (see FIG. 3 as appropriate).
The management apparatus 10 determines whether a bottleneck has occurred in the task being executed by the bottleneck determination unit 122 (step S30).

If a bottleneck has occurred in the task being executed (Yes in step S30), the management apparatus 10 proceeds to the process in step S31.
On the other hand, if a bottleneck has not occurred in the task being executed (No in step S30), the management apparatus 10 ends the bottleneck detection / detour path control process.

The management apparatus 10 outputs the bottleneck information to the route calculation unit 160 by the bottleneck determination unit 122.
The management device 10 calculates a bypass route based on the bottleneck information by the route calculation unit 160, and determines whether or not a bypass is possible based on the calculation result of the bypass route (step S31).

When detouring is possible (Yes in step S31), the management apparatus 10 proceeds to the process of step S32.
The management apparatus 10 outputs the detour route information to the route control unit 170 by the route calculation unit 160.
The management device 10 transmits the route information to the switch 30 on the main route indicated by the route information by the route control unit 170 and stores it in the storage unit 124 (step S32), and returns to the process of step S30. .

On the other hand, when detouring is impossible (No in step S31), the management apparatus 10 proceeds to the process of step S33.
The management device 10 outputs a message for suggesting addition of network resources by the route calculation means 160 (step S33), and returns to the processing of step S30.
That is, the management apparatus 10 repeats the bottleneck detection / detour path control process until it is determined that no bottleneck has occurred in the task being executed.

As described above, since the management apparatus 10 according to the embodiment of the present invention uses both information relating to the process and information relating to the network, the bottleneck can be accurately detected and a new bottleneck is not induced. Route control can be performed.
In addition, since the management apparatus 10 can calculate the optimum main path even when the main path is deleted, the management apparatus 10 can perform path control without inducing a new bottleneck.
Further, the management apparatus 10 can propose addition of network resources to the user of the parallel distributed processing system by a message.

  The present invention is not limited to the embodiments, and various modifications can be made without departing from the spirit of the present invention. Hereinafter, the modification of this invention is demonstrated concretely.

(Modification 1)
In the embodiment described above, a new main route is calculated when the communication line 40 on the main route is deleted as in step S14 (FIG. 7), but the present invention is not limited to this.
For example, the route calculation unit 160 may calculate a plurality of main routes in advance by the route calculation method described in the following reference, and store the route information indicating the calculated main routes in the storage unit 124.
Reference: JYYen., “Finding the K shortest loopless paths in a network.”, Management Science, 17: 712-716, 1971

(Modification 2)
In the above-described embodiment, the bypass route is calculated after the bottleneck is detected as in step S31 (FIG. 9), but the present invention is not limited to this.
For example, the route calculation unit 160 may calculate a detour route in advance in step S <b> 17 and store the detour route information indicating the calculated detour route in the storage unit 124.

(Modification 3)
In the above-described embodiment, the network topology is displayed, but the present invention is not limited to this.
For example, the flow for each process may be displayed in association with the network topology using information stored in the storage unit 124. Accordingly, the user can easily determine whether there is a bias in the arrangement of network resources or the presence of network resources that can be reduced.

(Modification 4)
In the above-described embodiment, the destination port number is used as the matching rule. However, this matching rule is not limited to the destination port number as long as it can be used in OpenFlow.

(Modification 5)
In the above-described embodiment, the server 20 has been described as playing any role of the transmission source terminal device 20 ₁ , the information processing terminal device 20 ₂ , or the transmission destination terminal device 20 _3. It is not limited to.
That is, the server 20 does not need to be given a predetermined role in advance. The server 20 is, like also serves as a transmission source terminal device 20 ₁ and the transmission destination terminal device 20 _3, may play multiple roles simultaneously.

(Modification 6)
In the above-described embodiment, the server 20 or the switch 30 is described as periodically transmitting node information, but the present invention is not limited to this.
For example, the server 20 or the switch 30 may transmit node information when an adjacent node is changed, and the network topology detection unit 121 may determine the change of the network topology based on this node information.

(Modification 7)
In the above-described embodiment, the route calculation unit 160 is described as performing route calculation by the Dijkstra method, but the present invention is not limited to this.
For example, the route calculation unit 160 may calculate a route having the minimum cost as the main route based on the cost value for each communication line 40 preset by the user.

(Modification 8)
The parallel process calculation unit 140 may increase or decrease the parallelization number based on the process progress information stored in the storage unit 124 during the execution of the task. For example, the parallel process calculation unit 140 increases the parallelization number when the process progress status is delayed from a preset target, and decreases the parallelization number when the process progress status has achieved the target. In this case, the parallel process control unit 150 generates transfer connection information of the increased / decreased flow according to the increase / decrease in the number of parallelizations, and accumulates it in the accumulation unit 124.
In this case, the information collection unit 220 collects the progress status of the process in the server 20 and transmits process progress information indicating the progress status of the collected process to the management apparatus 10. Then, the management apparatus 10 stores the process progress information transmitted from the server 20 in the storage unit 124.

DESCRIPTION OF SYMBOLS 1 Parallel distributed processing system 2 Processing apparatus 10 Management apparatus 110 Task input means 120 Information management means 121 Network topology detection means 122 Bottleneck determination means 123 Task completion determination means 124 Storage means 130 Information display means 140 Parallel process calculation means (Terminal device determination) means)
150 Parallel process control means (terminal device control means)
160 Route calculation means 170 Route control means 20 Server (terminal device)
210 Command execution means 220 Information collection means 230 Parallel processing execution means 240 Storage means 20 ₁ Source terminal device 20 ₂ Information processing terminal device 20 ₃ Destination terminal device 30 Switch 310 Command execution means 320 Information collection means 330 Packet transfer means (data Transfer means)
40 communication line 50 route 60 main route

Claims (6)

A transmission source terminal device that is a terminal device that stores data, an information processing terminal device that is a terminal device that performs predetermined information processing on data stored in the transmission source terminal device, and information processing by the information processing terminal device Is used in a parallel distributed processing system including a destination terminal device that is the terminal device that stores the data subjected to processing, and a switch that transfers the data between the terminal devices by OpenFlow. In addition to performing path control, at least two information processing terminal devices perform parallel execution of a task that performs data processing on the data of the transmission source terminal device by the information processing terminal device and transmits the data to the transmission destination terminal device. A management device that commands the terminal device to be
From the terminal device and the switch, it receives the node information indicating the connected the switch or the terminal device with each other, based on the received node information, wherein the terminal apparatus and detects the network topology is a topology of the switch Network topology detection means for
Based on the network topology, the shortest path or the minimum path connecting the terminal devices is specified for each flow that is specified by the port number of the process started by the terminal device that executes the task and is a path control unit of the OpenFlow. A route calculation means for calculating a cost route as a main route and generating route information indicating the calculated main route;
Route control means for transmitting the route information generated by the route calculation means to the switch;
Task input means for inputting task information including identification information of a terminal device that executes the task;
Terminal device determining means for determining a terminal device to execute the task based on the task information;
And identification information of the terminal apparatus determined by the terminal device determining unit, and generates a transfer connection information that associates a port number of the specific process for each of the flow, and activation of the process, the creation of transfer connection Terminal device control means for instructing the terminal device
Based on the transfer connection information, the route information, the task completion information transmitted from the terminal device , the node information, and the data transfer amount information transmitted from the switch , the bottle in the task being executed Bottleneck determination means for determining whether or not a neck has occurred,
The route calculation means includes
When a bottleneck occurs in the task being executed, the task is used in the process executing the task based on the transfer connection information, the route information, the task completion information, and the node information. A management device that calculates a detour route of a flow and generates detour route information indicating the calculated detour route. The network topology detection means includes
It is determined whether the network topology has been changed based on the difference in the network topology.If the network topology has been changed, it is determined whether the main route has been deleted based on the route information.
The route calculation means includes
The management apparatus according to claim 1, wherein when the main route is deleted, a main route that substitutes for the deleted main route is calculated based on the changed network topology.   The management apparatus according to claim 1, wherein the route control unit transmits the detour route information generated by the route calculation unit to the switch for a preset number of the flows.   The route calculation means determines whether or not the detour route exists based on the calculation result of the detour route. If the detour route does not exist, a message for proposing addition of a preset network resource is displayed. The management apparatus according to claim 1, wherein the management apparatus presents the management apparatus. A transmission source terminal device that is a terminal device that stores data, an information processing terminal device that is a terminal device that performs predetermined information processing on data stored in the transmission source terminal device, and information processing by the information processing terminal device Is used in a parallel distributed processing system including a destination terminal device that is the terminal device that stores the data subjected to processing, and a switch that transfers the data between the terminal devices by OpenFlow. In addition to performing path control, at least two information processing terminal devices perform parallel execution of a task that performs data processing on the data of the transmission source terminal device by the information processing terminal device and transmits the data to the transmission destination terminal device. In order to instruct the terminal device to be
From the terminal device and the switch, it receives the node information indicating the connected the switch or the terminal device with each other, based on the received node information, wherein the terminal apparatus and detects the network topology is a topology of the switch Network topology detection means,
Based on the network topology, the shortest path or the minimum path connecting the terminal devices is specified for each flow that is specified by the port number of the process started by the terminal device that executes the task and is a path control unit of the OpenFlow. A route calculation means for calculating a cost route as a main route and generating route information indicating the calculated main route;
Route control means for transmitting the route information generated by the route calculation means to the switch;
Task input means for inputting task information including identification information of a terminal device that executes the task;
Terminal device determining means for determining a terminal device for executing the task based on the task information;
And identification information of the terminal apparatus determined by the terminal device determining unit, and generates a transfer connection information that associates a port number of the specific process for each of the flow, and activation of the process, the creation of transfer connection Terminal device control means for instructing the terminal device,
Based on the transfer connection information, the route information, the task completion information transmitted from the terminal device , the node information, and the data transfer amount information transmitted from the switch , the bottle in the task being executed Function as a bottleneck determination means for determining whether or not a neck has occurred,
The route calculation means includes
When a bottleneck occurs in the task being executed, the task is used in the process executing the task based on the transfer connection information, the route information, the task completion information, and the node information. A bottleneck determination / route control program characterized by calculating a detour route of a flow and generating detour route information indicating the calculated detour route. A transmission source terminal device that stores data, an information processing terminal device that performs predetermined information processing on the data stored in the transmission source terminal device, or a transmission that stores data processed by the information processing terminal device A terminal device that plays one or more roles among the destination terminal devices,
A switch that transfers the data between the terminal devices by OpenFlow, and the management device according to claim 1,
A parallel distributed processing system comprising:

Images