JP7372221B2 - AI processing distribution method and system - Google Patents

Description

The present invention relates to an AI processing distribution method and system for distributing AI processing among multiple nodes, and in particular, it is possible to level out the processing load on each distributed node regardless of where AI processing traffic occurs. Regarding AI processing distribution method and system .

In recent years, deep learning that uses multilayer neural networks has been attracting attention as an AI processing method. In deep learning, neural networks are becoming more multi-layered, and are demonstrating high recognition accuracy in fields such as image recognition and speech recognition.

A neural network has an "input layer," a "middle layer," and an "output layer," and a network in which the "middle layer" has multiple deep hidden layers is called a DNN (Deep Neural Network). Among DNNs, those having a multilayer stacked structure in which convolution layers and pooling layers are alternately repeated in the "middle layer" are called CNNs (Convolutional Neural Networks).

Zhuoran Zhao, Kamyar Mirzazad Barijough, Andreas Gerstlauer, "DeepThings: Distributed Adaptive Deep Learning Inference on Resource-Constrained IoT Edge Clusters," IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems ( Volume: 37 , Issue: 11 , Nov. 2018)

A neural network is virtually constructed on a computer, and software executes calculation processing layer by layer, but since the amount of calculation is enormous, it is necessary to use not only a general-purpose CPU but also a GPU (Graphics Processing Unit). It is also used to perform arithmetic processing. However, for example, CNN has 20 to 30 convolutional layers, so further load reduction is required.

Load reduction can generally be achieved by load distribution, and Non-Patent Document 1 discloses a technique for distributing processing of a large number of tasks across multiple nodes.

However, AI processing requires multiple layers or steps to proceed in a predetermined order, and the processing results of the previous layer to be used in the next layer. When performing distributed processing on such a series of processes, if the unit of distribution is inappropriate, the consistency of the processing may be impaired.

Furthermore, if the distribution destination is not selected appropriately, problems such as load concentration on some nodes and traffic concentration on some networks may occur.

The purpose of the present invention is to solve the above-mentioned technical problems, and to provide an AI that can perform distributed processing without compromising the consistency of AI processing and preventing load concentration on some nodes or traffic concentration on some networks. An object of the present invention is to provide a processing distribution method and system .

In order to achieve the above object, the present invention is characterized in that an AI processing node that performs distributed AI processing on AI information has the following configuration.

(1) AI information acquisition means that acquires AI information, AI processing execution means that performs AI processing on a portion of the acquired AI information, outsource node determination means that determines an outsourced node for AI processing, and said outsourced node. The apparatus includes processing amount determining means for determining the processing amount of AI processing to be entrusted to a node, and distributed processing entrusting means for entrusting the determined processing amount of AI processing to the outsourcing node.

(2) The system further includes distributed processing entrusting means for entrusting the distributed processing of AI information together with the specification of the amount of distributed processing, and the AI processing execution means executes the AI processing of the specified amount of distributed processing.

(3) Means for monitoring the free capacity of other AI processing nodes is provided, and the outsource node determining means determines the outsource node based on the free capacity.

(4) Distance management means for managing distances to other AI processing nodes is provided, and the entrusted node determining means determines the entrusted node based on free capacity and distance.

(1) When AI processing traffic occurs in one AI processing node, a part of the AI processing is delegated to another AI processing node, and this is repeated from the other AI processing node to another AI processing node, resulting in multiple AI processing Since the AI processing is distributed to the AI processing nodes, it is possible to prevent the AI processing load from concentrating on the AI processing nodes near the point where the AI processing occurs.

(2) Each AI processing node is entrusted with an amount of distributed processing commensurate with its free capacity, and any further AI processing is entrusted to other AI processing nodes, and this process is repeated, resulting in an excessive load on each AI processing node. This can be prevented.

(3) Since the outsourcing company for AI processing is determined based on free capacity and distance from the outsourcing source, it is possible to minimize the generation of new traffic associated with distributed processing, and also prevent traffic concentration. It becomes like this.

(4) Since AI processing is distributed to each AI processing node in processing units such as layers and steps, the consistency of processing is not compromised due to distribution, and efficient distributed processing is possible.

1 is a diagram showing the configuration of an AI processing distribution system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of AI processing using a neural network that is subject to distributed processing according to the present invention. FIG. 2 is a functional block diagram showing the configuration of the main parts of the outsourcing server ES(m). FIG. 2 is a functional block diagram showing the configuration of the main parts of the entrusted server ES(s). FIG. 2 is a diagram showing an example of a DNN targeted for distributed processing by the present invention.

Embodiments of the present invention will be described in detail below with reference to the drawings. However, the present invention is not to be construed as being limited to the contents described in the embodiments described below, and those skilled in the art will be able to change the specific configuration without departing from the idea or gist of the present invention. It would be easily understood.

FIG. 1 is a diagram showing the configuration of an AI processing distribution system according to an embodiment of the present invention, and illustration of configurations unnecessary for explanation of the present invention is omitted here. The AI processing distributed system of the present invention is configured by interconnecting a plurality of edge servers ES via a wireless base station BS and a network NW.

Each edge server ES receives a large amount of information (AI information) that is subject to AI processing from a mobile terminal 1 such as a user's smartphone or tablet, a surveillance camera 2, or a moving or parked vehicle 3 from a wireless base station BS. and obtained via network NW.

In this embodiment, we focus on AI processing using a neural network including an "input layer", "middle layer", and "output layer" as shown in Figure 2, and each edge server ES has the necessary information for AI processing. Applications and models are pre-implemented.

In such a configuration, when a large amount of traffic occurs to one edge server ES for acquiring AI information from mobile terminal 1, etc., that one edge server ES functions as the outsourcing server for AI processing, and the said AI Distributed processing of information is entrusted to another edge server ES to be executed.

The other edge server ES executes a part of the entrusted AI processing, and entrusts the remaining distributed processing of the AI processing to another edge server ES for execution. The other edge server ES executes a part of the entrusted AI processing and requests the remaining AI processing to be distributed, thereby realizing distributed processing of AI processing by multiple edge servers ES. be done.

In this way, in this embodiment, the edge server ES that directly acquires AI information from the mobile terminal 1, etc. functions as the outsourcing server, processes all or part of the AI information itself, and handles all the remaining distributed processing. Delegating to another edge server ES.

3 and 4 are functional block diagrams showing the configuration of the main parts of each edge server ES, and illustrations of configurations unnecessary for explaining the invention are omitted. Although each edge server ES can be realized with the same configuration, the functions specific to the outsourcing server ES(m) are explained in Figure 3, and the functions specific to the outsourcing server ES(s) are explained in Figure 4. .

In the entrusted server ES(m) in FIG. 3, the AI information acquisition unit 101 receives AI information such as camera images, location information, vehicle information, and environmental information from the mobile terminal 1, surveillance camera 2, and/or vehicle 3 via the network. Get it with.

The free capacity monitoring unit 102 monitors the free capacity C of other edge servers ES that are registered in advance as candidates for AI processing. The free capacity C may be the free processing capacity of the CPU or GPU in the edge server ES. The distance management unit 103 manages the distance D to each edge server ES of the entrustee candidate. The distance D may be a distance on the network (hop count, round trip time, etc.) other than the physical distance between the edge servers ES.

The entrustee determining unit 104 applies the free capacity C and the distance D to a predetermined function or mapping table, and selects the entrustee server ES ( s).

The entrustment amount determination unit 105 determines the amount of distributed processing to be entrusted to the determined outsource server ES(s) based on the free capacity C of the outsource server ES(s). In this embodiment, the amount of entrusted processing is determined for each layer, such as processing only the input layer, processing only the input and intermediate layer, or processing only a specific hidden layer within the intermediate layer. The distributed processing entrustment unit 106 entrusts distributed processing of the determined entrusted processing amount to the determined entrusted server ES(s).

In the entrusted edge server ES(s) in FIG. 4, the distributed processing entrustment unit 107 accepts entrustment of the distributed processing of the AI processing. The AI processing unit 108 executes the AI processing of the entrusted amount of processing. The outsourced edge server ES(s) further includes the same or similar functions as the free capacity monitoring unit 102, distance management unit 103, outsource determination unit 104, outsource amount determination unit 105, and distributed processing outsourcing unit 106, By also functioning as the outsourcing server ES(m), the remaining AI processing is outsourced to other edge servers ES.

Next, the operation of this embodiment will be explained in detail. Here, as an example shown in Figure 5, we will take as an example the application to a DNN, which is composed of an input layer, a hidden layer, and an output layer, where the hidden layer includes multiple hidden layers, and the AI processing proceeds layer by layer. I will explain. In a DNN, hierarchical arithmetic processing is generally performed on input data (for example, image data) in which the processing result of a certain layer is used as input for the processing of the subsequent layer.

Further, in this embodiment, it is assumed that the edge server ES1 operates as the entrusted server ES(m), and the other edge servers ES2, ES3, ES4, and ES5 operate as the entrusted servers ES(s).

When the edge server ES1 as the outsource server ES(m) acquires a large number of camera images to be subjected to DNN image recognition from the user's mobile terminal 1, surveillance camera 2, or vehicle 3, the outsource determination unit 104 , based on the monitoring results of the free capacity C of other edge servers ES2, ES3, ES4, ES5 and the distance D to other edge servers ES2, ES3, ES4, ES5, the free capacity C is larger and the distance D is smaller. A nearby outsourcing candidate is determined as the outsourcing server ES(s). In this embodiment, the edge server ES2 is determined as the first entrusted server ES(s).

The entrustment amount determining unit 105 determines the amount of processing to be entrusted to the edge server ES2 based on the result of monitoring the free capacity C by the free capacity monitoring unit 102 regarding the edge server ES2. The present embodiment will be described assuming that the processing for one layer is determined as the entrusted processing amount. The distributed processing entrustment unit 106 of the edge server ES1 entrusts the AI processing of the determined entrusted processing amount (input layer only) to the edge server ES2.

In the edge server ES2, when the entrustment receiving unit 107 receives distributed processing for the input layer, the AI processing unit 108 executes the entrusted AI processing for the input layer. The entrustee determining unit 104 determines the next entrustee server ES(s) based on the free capacity C and distance D of the other edge servers ES3, ES4, and ES5.

Note that the edge server ES1 that has completed the distributed processing of the input layer has an increased free capacity C, so the entrustee determination unit 104 of the edge server ES2 selects the next entrustee server ES from among the edge servers ES1, ES3, ES4, and ES5. (s) may be determined. In this embodiment, the explanation will be continued assuming that the edge server ES3 has been determined to be the next entrustee.

The entrusted amount determination unit 105 of the edge server ES2 determines the entrusted processing amount to the outsourced edge server ES3 based on the monitoring result of the free capacity C by the free capacity monitoring unit 102. In this embodiment, since the processing of the input layer has been completed, depending on the free capacity C of the edge server ES3, for example, distributed processing of a part of the hidden layer of the intermediate layer is determined as the entrusted processing amount. The distributed processing entrustment unit 106 entrusts the AI processing of the determined entrusted processing amount to the edge server ES3, together with the processing results of the input layer.

Similarly, each outsourced edge server ES(s) executes AI processing of the outsourced processing amount, and selects the next outsourced edge server based on the available capacity C of the outsourced candidate and the distance D to the other outsourced candidate. and determining the outsourced processing amount, providing the results of distributed processing to the determined outsourced server ES(s), and further entrusting distributed processing of the determined outsourced processing amount until the AI completes.

Note that the edge server ES5 entrusted with the distributed processing of the output layer outputs the processing results in, for example, a histogram format, without determining the entrusted party or the amount of entrusted processing.

According to this embodiment, when AI processing traffic occurs on one edge server, a part of the AI processing is entrusted to another edge server, and this is repeated to the other edge servers so that AI processing is performed on multiple edge servers. Since the processing is distributed, it is possible to prevent the AI processing load from concentrating on edge servers near the point where the AI processing occurs.

In addition, according to this embodiment, the outsourcing company for AI processing is determined based on the free capacity C and the distance D from the outsourcing source, so it is possible to minimize the generation of new traffic associated with distributed processing. This makes it possible to prevent traffic concentration.

Furthermore, according to this embodiment, AI processing is distributed on a layer-by-layer basis, which not only makes it possible to avoid multiple AI processing, but also enables an edge server that finishes AI processing on a certain layer to start AI processing on the next layer. Since requests are made to other edge servers, it becomes possible to perform AI processing for a certain layer of other AI, and distributed processing prevents the consistency of AI processing from being compromised, enabling efficient distributed processing. .

In addition, in the above embodiment, DNN was explained as an example of AI processing, but the present invention is not limited to this. It can be similarly applied to CNNs consisting of repeated embedding layers and pooling layers. In this case, each convolutional layer and each pooling layer can be a unit of distributed processing.

Further, in the above embodiment, all distributed processing is entrusted to the edge server, but the present invention is not limited to this, and when the vehicle 3 is parked or running, the CPU for AI processing If the computer is equipped with a system equipped with a computer or GPU, distributed processing may be entrusted to that system.

However, the distance of the moving vehicle 3 from the outsourcing source or outsourcing company changes during the AI processing. Therefore, if it is not possible to estimate the moving speed and direction of the vehicle, it is desirable to make the distance conditions stricter than when evaluating the edge server ES as a consignee. For example, if the upper limit of the distance at which a fixed edge server ES can be used as a consignee for distributed processing is Dmax, the upper limit of distance is corrected to Dmax - Dm, taking into account the margin Dm equivalent to vehicle movement, and the distance at which it can be adopted as a consignee is Dmax. It is desirable to keep the upper limit shorter than in the case of edge server ES.

Furthermore, in the above embodiment, AI processing using neural networks such as CNN and DNN was explained as an example, but the present invention is not limited to this, and a series of processing is divided into layers and steps in chronological order. Any AI processing that can be divided can be similarly applied to AI processing that does not use neural networks.

1...Mobile terminal, 2...Surveillance camera, 3...Vehicle, 101...AI information acquisition unit, 102...Free space monitoring unit, 103...Distance management unit, 104...Consignee determination unit, 105...Consignment amount determination unit, 106... Distributed processing entrustment department, 107...Distributed processing entrustment department, 108...AI processing department

Claims (8)

In a system that distributes AI processing using multiple servers , in which data input into a multilayer neural network is processed layer by layer using software as AI information.
The plurality of servers are
A source server that entrusts distributed processing of input AI information to other servers,
including a plurality of outsourced servers that perform AI processing on at least part of the outsourced distributed processing and outsource the remaining distributed processing to other servers;
Each outsourced server is
an AI information acquisition means for acquiring AI information;
an AI processing execution means for performing AI processing on part of the acquired AI information;
A consignee determining means for determining a consignee server for AI processing;
processing amount determining means for determining the processing amount of AI processing to be entrusted to the outsourced server ;
and distributed processing entrustment means for entrusting the AI processing of the determined processing amount to the determined outsourcing server ,
An AI processing distributed system characterized in that the determined entrusted server repeats the process of further determining another entrusted server and entrusting the AI processing to the other entrusted server. Each of the outsourced servers further comprises distributed processing entrustment means for entrusting distributed processing of AI information together with specification of the amount of distributed processing,
2. The AI processing distributed system according to claim 1 , wherein the AI processing execution means executes AI processing with a specified amount of distributed processing. Each of the outsourced servers is equipped with free capacity monitoring means for monitoring at least one of free memory capacity and free processing capacity of other outsourced servers ,
3. The AI processing distribution system according to claim 1, wherein the entrustee determining means determines the entrustee server based on at least one of the free memory capacity and free processing capacity . Each of the servers includes distance management means for managing distances from other servers ,
4. The AI processing distribution system according to claim 3 , wherein the entrustee determining means determines the entrustee server based on at least one of the free memory capacity and free processing capacity and distance. 5. The processing amount determining means determines the processing amount of the AI processing to be entrusted to the entrusted server based on at least one of the free memory capacity and free processing capacity of the entrusted server. AI processing distributed system. The AI processing distributed system according to any one of claims 1 to 5 , wherein the AI processing is an AI processing using a neural network, and distributed processing is performed in each layer unit of an input layer, an intermediate layer, and an output layer. . 7. The AI processing distributed system according to claim 6 , wherein the AI processing is DNN (Deep Neural Network), and the intermediate layer is distributedly processed in units of hidden layers. In a method of distributing AI processing in which data input into a multi-layered neural network is processed layer by layer by software as AI information, using multiple servers including the outsourcing source server and the outsourcing server ,
The outsource server entrusts the distributed processing of the input AI information to the first outsource server,
Each outsourced server is
Obtain AI information,
AI processes some of the acquired AI information,
Determine the outsourced server for AI processing,
Determine the amount of AI processing to be entrusted to the outsourced server ,
Outsourcing the AI processing of the determined processing amount to the determined outsourcing server ,
An AI processing distribution method characterized in that the determined entrusted server repeats the process of further determining another entrusted server and entrusting the AI processing to the other entrusted server.

Images