JPH0769893B2 - Neural network simulator - Google Patents

Description

The present invention relates to a neural network simulation technique.

(Prior art) Conventionally, a simulator using software on a computer,
There have been simulators in which a plurality of microprocessors are connected by a bus and simulators in which a neural network is realized on one analog IC.

(Problems to be Solved by the Invention) However, the simulator using software cannot utilize the parallelism of the neural network and cannot perform high-speed simulation.

In a system in which multiple microprocessors are connected by a bus, the parallelism of the simulation algorithm cannot be used, and when the number of processors increases, communication between processors cannot be performed efficiently, resulting in a large scale. Therefore, high-speed simulation was impossible.

In a system that realized a neural network on an analog IC, once made, it was impossible to change the circuit and lack flexibility, and it was also impossible to integrate a large-scale network.

An object of the present invention is to provide a high-speed simulation of a large-scale neural network and a simulator capable of flexibly changing the simulated neural network.

(Means for Solving Problems) The present invention is an apparatus for simulating a neural network in which nodes exchange messages with each other via weighted links, and the internal states and outputs are changed according to the contents of the messages. Therefore, it is composed of a plurality of processors for processing the partial networks obtained by dividing the neural network in parallel, and a multi-stage connection network for connecting the plurality of processors and transferring a message between the processors. (A). A multiplexer that multiplexes messages from other processors with messages from itself, (b). An input buffer for buffering the output of the multiplexer, (c). An arithmetic unit for input processing of nodes in the sub-network allocated according to the message in the input buffer, (d). A storage device for storing the result of the input processing, (e). An internal bus for transferring data between an arithmetic unit that performs the input process and a storage device that stores the result of the input process, (f). Updating the internal state of the node from the content of the storage device that stores the result of the input processing and the content of the storage device that stores the internal state of the node, and linking with the content of the storage device that stores the internal state of the node An arithmetic unit that performs node output processing from the contents of the storage device that stores the connection information and link weights of (g). A storage device that is separated from the internal bus and that stores the internal state of the node and the connection information of the link and the weight of the link, (h). An internal bus for transferring data between arithmetic units that perform internal processing and output processing of the node, (i). An output buffer for buffering a message resulting from the output processing of the arithmetic unit that performs internal processing and output processing of the node, (j). And a selector for selecting a message in an output buffer to the multiplexer and another processor, and the multistage connection network transfers the message output from the processor to a predetermined processor designated by a destination of the message. And (k) .N × N (N ≧ 2) converter, which is composed of multiple router cells connected in multiple stages to send a message from an input port to a predetermined output port according to its destination address. This is a neural network simulator characterized by the following.

(Operation) In the present invention, the neural network to be simulated is divided into partial networks, which are assigned to a plurality of processors and processed in parallel so that the parallelism of the neural network is derived, and each processor is divided into two separate networks. By providing a bus and two arithmetic units and performing processing in parallel, the parallelism of the simulation algorithm can be extracted and high-speed simulation can be performed. Further, it is possible to flexibly change the simulated neural network by rewriting the contents of the storage device that stores the link connection information and the link weight.

(Embodiment) FIG. 1 is a block diagram of one processor 1 constituting a system, which includes a multiplexer 2, an input buffer 3, and
Arithmetic unit 4, internal bus 5, storage unit 6, selector 7, output buffer 8, arithmetic unit 9, and internal bus 10.
And a storage device 11.

FIG. 2 is an overall configuration diagram including the processor of FIG. 1, and comprises a plurality of processors 1 and a multistage connection network 14 for transferring data between them. The multistage connection network 14 is composed of a plurality of router cells 15.

Roussel 15 is an N × N (N ≧ 2) converter and sends a message from an input port to a predetermined output port according to the address of its destination. A message from the processor 1 is converted by a plurality of stages of router cells and reaches a predetermined processor 1 designated by the destination of the message.

Figure 3 shows a partial network of a neural network.
It is a figure explaining division into 18. A neural network has many nodes 16 (neurons) and weighted links 17
Composed of. Each node 16 sends and receives a message to and from another node 16 via a weighted link, and changes its internal state and output according to the content of the message.
This neural network has a plurality of whole networks (4 in FIG. 3).
Sub network 18) and each sub network 18 is assigned to the processor 1 and processed in parallel. The processor 1 simulates the functions of the node 16 and the weighted link 17 included in the allocated sub-network 18. If there is a weighted link 17 between the nodes 16 included in the same partial network 18 (that is, processed by the same processor 1), communication between the processors 1 is not necessary. Included in different sub-networks 18 (ie processed by different processors 1)
If there are weighted links 17 between nodes 16, interprocessor communication is required. In this case, messages are exchanged between the nodes via the multistage connection network 14.

Next, the processing of the processor will be described in detail with reference to FIG.

The input message 12 received from another processor and the message received from its own selector 7 are arbitrated by the multiplexer 2 and enter the input buffer 3.

The storage device 6 stores the result of the input processing for each node of the partial network to which the processor 1 is assigned.

The arithmetic unit 4 performs input processing according to the contents of the message while taking out the message from the input buffer 3, and stores the result in the storage unit 6.

The storage device 11 stores the internal state of each node of the partial network, the link connection information, and the link weight.

The arithmetic unit 9 is the result of the input processing of the storage device 6 and the storage device.
The internal state of each node in 11 is referenced and the internal state of each node is updated. The arithmetic unit 9 also refers to the internal state of each node of the storage unit 11, the connection information of the link, and the weight of the link to perform output processing of each node and write a message in the output buffer 8.

The selector 7 takes a message from the output buffer 8 and transfers the message as an output message 13 to another processor or to its own multiplexer 2 according to its contents.

(Effects of the Invention) As described above, in the present invention, the neural network to be simulated is divided into sub-networks, which are assigned to a plurality of processors and processed in parallel to bring out the parallelism of the neural network. In addition, by providing two separated buses and two arithmetic units and performing processing in parallel, the parallelism of the simulation algorithm is brought out and high-speed simulation is enabled. Further, the neural network to be simulated can be flexibly changed by rewriting the contents of the storage device that stores the link connection information and the link weight.

[Brief description of drawings]

FIG. 1 is a block diagram of one processor constituting the system, FIG. 2 is a configuration diagram including the processor of FIG. 1,
FIG. 3 is a diagram for explaining division of the neural network into partial networks. 1 ... Processor, 2 ... Multiplexer, 3 ... Input buffer, 4 ... Arithmetic device, 5 ... Internal bus, 6 ... Storage device, 7 ... Selector, 8 ... Output buffer, 9 ...
Arithmetic unit, 10 ... Internal bus, 11 ... Storage device, 12 ... Input message, 13 ... Output message, 14 ... Multi-stage connection network, 15 ... Router cell, 16 ... Node, 17 ...
… Weighted links, 18 …… Partial networks

Claims (1)

[Claims] 1. A device for simulating a neural network in which each node exchanges messages with each other via weighted links, and changes an internal state and an output according to the contents of the message, which is a divided part of the neural network. It is composed of a plurality of processors for processing the network in parallel and a multi-stage connection network for connecting the plurality of processors and transferring a message between the processors. A multiplexer that multiplexes messages from other processors with messages from itself, (b). An input buffer for buffering the output of the multiplexer, (c). An arithmetic unit for performing input processing of a node in the sub-network allocated according to the message in the input buffer; (d). A storage device for storing the result of the input processing, (e). An internal bus for transferring data between an arithmetic unit that performs the input process and a storage device that stores the result of the input process, (f). Updating the internal state of the node from the content of the storage device that stores the result of the input processing and the content of the storage device that stores the internal state of the node, and the link with the content of the storage device that stores the internal state of the node An arithmetic unit that performs node output processing from the contents of the storage device that stores the connection information and link weights of (g). A storage device that is separated from the internal bus and that stores the internal state of the node and the connection information of the link and the weight of the link, (h). An internal bus for transferring data between arithmetic units that perform internal processing and output processing of the node, (i). An output buffer for buffering a message resulting from the output processing of the arithmetic unit that performs internal processing and output processing of the node, (j). And a selector for selecting a message in an output buffer to the multiplexer and another processor, and the multistage connection network transfers the message output from the processor to a predetermined processor designated by a destination of the message. And (k) .N × N (N ≧ 2) converter, which is composed of multiple router cells connected in multiple stages for sending a message from an input port to a predetermined output port according to the address of its destination. A neural network simulator characterized by the following.