JP7655384B2 - COMMUNICATION CONTROL DEVICE, COMMUNICATION CONTROL METHOD, AND PROGRAM - Google Patents

Description

The present invention relates to a communication device that performs time-division transmission, such as Time Sensitive Networking.

Time Sensitive Networking (TSN), which guarantees delay and jitter on industrial and automotive Ethernet (registered trademark) and can coexist with best-effort communication, has been attracting attention in recent years. In the future, it is expected that deterministic time communication services will be realized over a wide area, such as closed loop control between distant factories, as well as within closed Local Area Networks (LANs) in factories and automobiles.

In order to expand the scope of application of TSN, it is effective to realize TSN communication in virtual environments such as containers and virtual machines (VMs) in order to flexibly provide TSN services.

In addition, by sharing TSN (particularly IEEE 802.1Qbv, hereafter referred to as Qbv) time slots among multiple applications rather than assigning them fixedly to specific classes, it becomes possible to accommodate a greater number of services.

On the other hand, when multiple containers/VMs (hereafter referred to as guests) exist on the same physical host and share a TSN transmission class, queuing delays may occur due to traffic collisions between guests, or order reversals may occur within the same class, making it impossible to achieve jitter guarantees.

C. Xu, K. Rajamani and W. Felter, "NBWGuard: Realizing Network QoS for Kubernetes," Computer Science Proceeding of the 19th International Middleware Conference Industry, 2018.

In an environment where multiple guests exist on the same physical host, when TSN communication is performed from a guest, the conventional technology disclosed in non-patent document 1, etc., cannot prevent traffic collisions between guests and guarantee delays other than by performing the same traffic control on the guest side as the host.

However, if the guest side were to perform the same traffic control as the host side, it would be necessary to implement a complex time slot allocation process for each guest. Also, it would be possible to disrupt the TSN communications of other guests through misconfiguration or intentional attacks. In other words, there is an issue that there is no mechanism to guarantee jitter (delay fluctuation) in guest-host communications without performing complex time-division transmission control on the guest side.

The present invention has been made in consideration of the above points, and aims to provide a mechanism for guaranteeing jitter in TSN communication in an environment where multiple guests are installed on the same physical host, without the need for complex time-division transmission control on the guest side.

According to the disclosed technology, a plurality of reception control units are provided for receiving packets from a server on which an application runs, and each reception control unit:
A receive buffer;
a reception determination unit that transmits the packet to a transmission interface connected to an external network at a time when the packet can be received at the transmission interface, and stores the packet in the reception buffer at a time when the packet cannot be received at the transmission interface,
the transmission interface includes a classifier and a plurality of transmission queues;
the communication control device further includes a class assigning unit between the plurality of reception control units and the transmission interface;
The class assignment unit
A time slot and a transmission class of the packet are determined based on schedule information for time division transmission control in the transmission interface, and a transmission class identifier is assigned to the packet.
A communications control device is provided.

According to the disclosed technology, in TSN communication in an environment where multiple guests are installed on the same physical host, a mechanism can be realized that guarantees jitter without complex time-division transmission control on the guest side.

FIG. 1 is a diagram for explaining a problem. FIG. 1 is a diagram for explaining a problem. FIG. 1 is a diagram for explaining a problem. FIG. 1 is a diagram for explaining an overview of an embodiment. 1 is an overall configuration diagram of a system according to an embodiment of the present invention; FIG. 2 is a configuration diagram of a host according to the first embodiment. FIG. 4 is a configuration diagram of a reception control unit in the first embodiment. FIG. 2 is a configuration diagram of a transmission control unit in the first embodiment. 13 is a process flow when a packet is received by a reception determination unit in reception determination method 1. 13 shows a transfer flow from a reception buffer to a class assignment unit in reception determination method 1. 13 is a flow chart showing a flow of calculation of a reception permitted time in a reception determination method 2. 13 is a process flow when a packet is received by a reception determination unit in reception determination method 2. 13 is a flow chart showing a flow of calculation of a reception permission time in the reception determination method 3. 13 is a process flow when a packet is received by a reception determination unit in reception determination method 3. 13 is a flow chart showing a flow of calculation of a reception permission time in the reception determination method 3. FIG. 1 is a diagram illustrating the relationship between variables. FIG. 11 is a diagram illustrating an example of schedule information. FIG. 11 is a diagram illustrating an example of schedule information. FIG. 11 is a configuration diagram of a host according to a second embodiment. FIG. 11 is a configuration diagram of a reception control unit in the second embodiment. FIG. 11 is a configuration diagram of a class assignment unit in the second embodiment. FIG. 13 is a configuration diagram of a host according to a third embodiment. FIG. 11 is a diagram illustrating an example of schedule information. FIG. 13 is a configuration diagram of a reception control unit in the fourth embodiment. FIG. 13 is a diagram illustrating an example of allocation of transmission queues in the fifth embodiment. FIG. 13 is a configuration diagram of a reception control unit in the fifth embodiment. FIG. 13 is a configuration diagram of a transmission control unit in the fifth embodiment. FIG. 23 is a configuration diagram of a host according to a sixth embodiment. 2 is an example of a hardware configuration of the device.

Hereinafter, an embodiment of the present invention (the present embodiment) will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applicable is not limited to the following embodiment.

Below, we will explain an embodiment of a technology that prevents collisions with other traffic by performing time division reception control in the receiving interface of a relay device that connects to a server that does not support time division transmission in a network that performs time division transmission such as Time Sensitive Networking.

In the present embodiment described below, the TSN method assumes the Qbv method, in which multiple transmission queues are prepared for each port of the communication device, and the gate opening and closing that controls whether transmission is possible is controlled by time based on an internal clock, but the present invention is applicable in ways other than the Qbv method.

In addition, this embodiment is based on the premise that a time slot in Qbv's time-division transmission schedule is shared by multiple guests (TSN Talkers), but the present invention is not limited to such a premise.

(Regarding the issues)
First, the problems with the technology according to the embodiment described later will be described in more detail. FIG. 1 shows an example of a configuration in which three guests (containers/VMs) exist on the same physical host and share a TSN transmission class. In such a configuration, queuing delays may occur due to traffic collisions between guests, and transmission order reversals may occur within the same transmission class. Note that FIG. 1 is a diagram for explaining the problems with the conventional technology, and the configuration of FIG. 1 itself is not a publicly available existing technology. The same is true for FIG. 2.

To address the above issues, as shown in Figure 2, it is possible to set up time-division transmission control on the guest side that follows the same Qbv transmission schedule as the host. This can prevent traffic collisions. However, in order to do this, it is necessary to assign classes and perform transmission control to perform Qbv on the guest side. To assign classes, it is necessary to implement the L2 network layer independently for each application, or to use functions such as iptables and tc provided by Linux (registered trademark).

When implementing the former on your own, the application creator must implement up to the network layer, while in order to use the latter functions such as iptables and tc, the guest needs special privileges to use the functions, which poses security issues. If special privileges are granted to the guest, it becomes possible for the guest to make configuration errors or launch deliberate attacks that could disrupt time-division transmission control. Furthermore, when sharing a time slot among multiple guests, complex time slot allocation must be implemented for each guest.

On the other hand, if time-division transmission control is not performed on the guest side, the application's transmission timing is not restricted, which may result in increased queuing delays due to a reversal of the enqueuing order of packets to the host-side transmission interface (hereinafter referred to as I/F), or the transmission time slot may be shifted.

An example of the above situation will be described with reference to Figure 3. The upper part of Figure 3 shows a transmission schedule indicating the original packet transmission order of a certain transmission class, and the lower part of Figure 3 shows the transmission order when an interruption of inter-guest traffic occurs. A to E represent packets transmitted during the open1 and open2 periods.

As shown in Figure 3, in the original transmission schedule, packet D is transmitted in the open2 transmission timeslot. However, if packet D is enqueued without waiting for the transmission schedule, packet D will arrive before packet C, prior to the original open2 transmission timeslot, meaning that C will not be able to be transmitted within one timeslot. Furthermore, if D is a time-sensitive packet, D will also be processed for transmission earlier than expected, resulting in increased jitter.

Next, we will explain prior art 1 and prior art 2 as examples of publicly available prior art.

<Prior Art 1>
Prior art 1 is a technology disclosed in Non-Patent Document 1. Prior art 1 uses an Intermediate Functional Block device (IFB), which is an extension function of Linux (registered trademark), to transfer packets received at a virtual I/F of a host connecting to a guest to the IFB. By using a Hierarchical Token Bucket (HTB) on the IFB to specify the maximum data transmission amount between pods from a Kubernetes Manifest file, the transmission and reception bandwidth for each pod can be restricted.

By applying conventional technology 1, it is possible to manage the bandwidth for each guest, but because HTB is a method that guarantees bandwidth at the expense of jitter, it cannot eliminate the worsening queuing delays and time slot discrepancies caused by traffic contention between guests.

<Prior Art 2>
As disclosed in "https://man7.org/linux/man-pages/man8/tc-etf.8.html", Linux kernel 4.19 introduces queue control called Earliest TxTime First (hereinafter, ETF), which performs transmission processing according to the transmission request time.

In ETF, the transmission request time is added as metadata within the kernel of the communication packet, and the rbtree data structure is used to sort the packets in the packet buffer (Qdisc) within the kernel linked to the network interface in order of the transmission request time closest to the current time, and the packets are dequeued to the ring buffer within the driver so that they arrive in time for the transmission request time.

By adopting ETF, packets will not be sent at the wrong time for communications closed to the host. However, packet metadata added within the guest is usually lost when the packet is passed from the virtual interface to the host, so it cannot be used in communications between the guest and the host.

<Problems with the prior art>
As mentioned above, in an environment with multiple guests in a host, when a guest performs TSN communication, it is not possible to prevent traffic collisions between guests and guarantee delays. To address this issue, it is possible to perform the same traffic control on the guest side as on the host side, but this requires implementing a complex time slot allocation process for each guest, and it is also possible to disrupt the TSN communication of other guests due to configuration errors or intentional attacks.

In other words, there is a problem in that there is no mechanism to compensate for jitter in communication between a guest and a host without complex time-division transmission control on the guest side. Below, we will explain the technology related to this embodiment that solves this problem.

(Overview of the embodiment)
First, an overview of this embodiment will be described with reference to Fig. 4. Fig. 4 shows an example of a host having three guests. The operation of each functional unit shown in Fig. 4 will be described in the examples described later.

In this embodiment, a buffer is provided in the host's receiving I/F that pairs with the guest's transmitting I/F, and receiving control is performed by opening and closing a gate in synchronization with the host's Qbv transmitting control.

By providing a buffer in the receiving I/F in this way, traffic flowing to the host side other than at the host's sending timing can be filtered and buffered in the receiving I/F until the host's sending timing arrives. This prevents traffic collisions between guests and guarantees jitter.

In other words, as shown in Figure 4, each guest application (TSN Talker) does not need to be aware of the transmission timing or TSN settings, and if it is not the transmission timing, the reception I/F can be controlled to buffer packets. In this way, by performing reception control in synchronization with transmission control, only traffic at the transmission timing is transferred, so packet collisions do not occur and jitter compensation can be achieved.

The system configuration and operation in this embodiment are described in detail below.

(Example of overall system configuration)
An example of the overall configuration of the system according to this embodiment is shown in Fig. 5. As shown in Fig. 5, this system includes a host 100, a TSN domain 500, and a client terminal 200.

Host 100 is a physical server that accommodates multiple guests. TSN domain 500 is a network composed of devices that support TSN (including both IEEE 802.1AS and Qbv). Figure 5 shows relay devices 300 and 400 as examples of devices that make up TSN domain 500. Client terminal 200 communicates with guests in host 100.

The host 100 has guests 110-1 to 110-n and a virtual relay device 120. The multiple guests 1 to n are connected to the virtual relay device 120 in the host 100 and communicate with the client terminal 200 via the transmission/reception interface of the host 100.

In this embodiment, a guest and a virtual relay device are used, but a physically wired bare metal server may be used instead of the guest, and a physical relay device may be used instead of the virtual relay device. The "guest" may also be called a "server." Also, the term "relay device" has a broad meaning that includes both a "virtual relay device" and a "physical relay device."

In addition, a "relay device" or a device that includes a "relay device" (e.g., a host or a physical NIC in Example 5) may be referred to as a "communications control device."

Below, the detailed configuration and operation of the host 100 will be explained using Examples 1 to 6. Examples 1 to 6 can be implemented in any combination.

(Example 1: Basic method)
First, the basic method will be described as a first embodiment.

<Configuration of Host 100>
Fig. 6 shows the configuration of the host 100. In the example shown in Fig. 6, the host 100 includes guests 110-1 to 110-3, a virtual relay device 120, a schedule DB (database) 130, a transmission control unit 140, and a physical NIC 150. Each guest includes an application and a vNIC which is a virtual transmission/reception I/F.

The virtual relay device 120 comprises a vNIC that connects to each guest 110, a reception control unit 121, and a class assignment unit 122. The "vNIC + reception control unit 121" may be referred to as a reception interface.

Each guest 110 and virtual relay device 120 are connected by their respective vNICs, and applications in each guest 110 communicate with the external network via the vNIC, the reception control unit 121, the class assignment unit 122 in the virtual relay device 120, the transmission control unit 140, and the physical NIC 150. Here, the "transmission control unit 140 + physical NIC 150" may be called a transmission interface. Also, the transmission control unit 140 and the physical NIC 150 (i.e., the transmission interface) may be included in the virtual relay device 120.

Multiple receiving control units 121-1 to 121-3 and a transmitting control unit 140 refer to the schedule DB 130.

In standard TSN, the terminal (Talker) that sends the packet assigns a class identifier (VLAN (Virtual LAN) tag) to control transmission using IEEE 802.1Qbv. In Example 1, however, the guest 110 that sends the packet does not assign a VLAN tag. Instead, based on the information in the schedule DB 130, the reception control unit 121 assigns class information as metadata for the packet, and the class assignment unit 122 assigns a class identifier to the packet based on the class information.

<Reception control unit 121, class assignment unit 122>
Fig. 7 shows an example of the configuration of the reception control unit 121 in Example 1. As shown in Fig. 7, the reception control unit 121 includes a reception determination unit 11, a metadata addition unit 12, and a reception buffer 13. Note that the reception determination unit 11 may include the function of the metadata addition unit 12.

The reception determination unit 11 checks to which class the packet received from the vNIC should be assigned based on the information in the schedule DB 130, determines whether it is a time when the packet can be received, and then passes the packet to the metadata assignment unit 12. The detailed process of the reception determination method will be described later.

The metadata assignment unit 12 assigns class information to the packet as metadata for the packet. If the current time is a time when the packet can be received, the packet is sent directly to the class assignment unit 122. If the current time is a time when the packet cannot be received, the packet is stored in the reception buffer 13, and when the time when the packet can be received arrives, the packet is sent to the class assignment unit 122.

The class assignment unit 122 obtains metadata class information from the packet and assigns a class identifier to the packet. In the case of a standard TSN, a VLAN tag is assigned and the class is specified in the 3-bit Priority Code Point (PCP) field in the VLAN tag.

<Transmission Control Unit 140>
Fig. 8 shows a configuration example of the transmission control unit 140 of the embodiment 1. As shown in Fig. 8, the transmission control unit 140 includes a class classification unit 141, a plurality of transmission queues, and a transmission gate opening/closing unit 142, and performs transmission control according to IEEE802.1Qbv. Note that, although the example of Fig. 8 shows transmission queues for eight classes, this is just an example.

The classifier 141 distributes packets arriving at the transmission controller 140 to transmission queues for each class. The transmission gate open/close unit 142 transmits packets to the external network via the physical NIC 150 only during times when transmission is permitted. The time-division transmission schedule of the transmission gate open/close unit 142 is determined by referring to the schedule DB 130.

Below, we will explain reception determination methods 1 to 3 of the reception control unit 121 in Example 1. The relationship on the time axis of each variable used in the explanation is shown in Figure 16. Note that Figure 16 shows variables when t _base is used.

<Method 1 of reception determination by the reception control unit 121 in the first embodiment>
A reception determination method 1 of the reception control unit 121 in the embodiment 1 will be described. The reception determination unit 11 of the reception control unit 120 acquires from the schedule DB 130 in advance the time T _cycle for one cycle in which the entire transmission schedule is repeated, the reference time t ₀ at which the transmission schedule starts, the offset value T _open from the reference time at which reception is permitted, and the offset value T _close at which permission ends.

Referring to Figures 9 and 10, an example of operation of the reception determination unit 11 is explained.

9 shows a process flow when a packet is received by the reception determination unit 11. In S101, the reception determination unit 11 receives a packet i from the guest 110. In S102, the reception determination unit 11 acquires the current timestamp value t _now .

In S103, the reception determination unit 11 calculates an offset value T _now for the current transmission schedule by T _now = (t _now - t ₀ ) mod T _cycle . In S104, the reception determination unit 11 judges whether T _now ≧T _open , and if Yes, proceeds to S105, and if No, proceeds to S107. In S105, the reception determination unit 11 judges whether T _now ≦T _close , and if Yes, proceeds to S106, and if No, proceeds to S108.

In S106, the reception determination unit 11 transfers the packet i to the class assignment unit 122. In S107, the reception determination unit 11 stores the packet i in the reception buffer 13 of the rbtree data structure based on the next reception permission time t ⁱ _open = t _now + (T _open - T _now ) + T _open . In S108, the reception determination unit 11 stores the packet i in the reception buffer 13 of the rbtree data structure based on the next reception permission time t ⁱ _open = t _now + (T cycle - T _now ) + T _open . The reception permission time is the time at which the transmission control unit 140 can start reception processing, and may be called the reception start time or the reception processing start time.

10 shows a transfer flow from the receiving buffer 13 to the class assignment unit 122. In S201, the receiving determination unit 11 acquires the current timestamp value t _now . In S202, the receiving determination unit 11 determines whether or not there is a packet i in the packet group in the receiving buffer 13 whose smallest t ⁱ _open is equal to t _now . If Yes (packet present), the process proceeds to S203, and if No, the process proceeds to S204.

In S203, the reception determination unit 11 transfers the packet i to the class assignment unit 122 and deletes it from the reception buffer 13. In S204, there is standby, and in S205, the reception determination unit 11 determines whether or not "new data has been inserted into the rbtree and min{t ⁱ _open } has been updated," and if Yes, the process proceeds to S206, and if No, the process returns to S201. In S206, when the time min{t ⁱ _open }-t _now has elapsed, the process returns to S201.

<Method 2 of determining reception by the reception control unit 121 in the first embodiment>
A second reception determination method of the reception control unit 121 in the first embodiment will now be described. In order to reduce the amount of processing calculations required when receiving a packet, the reception determination unit 11 of the reception control unit 121 constantly calculates the absolute times t _open and t _close , which indicate the start and end of the reception permission period, in a separate thread, and when receiving a packet, it only obtains the current time and determines whether the packet is within the reception permission period.

An example of the operation of the reception judgment unit 11 in reception judgment method 2 will be described with reference to Figures 11 and 12. The transmission process from the reception buffer is the same as in reception judgment method 1.

11 shows a flow of calculating the permitted reception time. In S301, the reception determination unit 11 sets the cycle start time t _base =t _0. The reception determination unit 11 calculates the reception start time _topen =t _base +T _open , and calculates the reception end time _toclose =t _base +T _close .

In S303, when T _cycle has elapsed, t _base is calculated as t _base =t _base +T _cycle in S304, and the process returns to S302.

12 shows a process flow when a packet is received by the reception determination unit 11. In S401, the reception determination unit 11 receives a packet i from the guest 110. In S402, the reception determination unit 11 acquires the current timestamp value t _now .

In S403, the reception determination unit 11 judges whether t _now ≧t _open , and if Yes, proceeds to S404, and if No, proceeds to S406. In S404, the reception determination unit 11 judges whether t _now ≦t _close , and if Yes, proceeds to S405, and if No, proceeds to S407.

In S405, the reception determination unit 11 transfers the packet i to the class assignment unit 122. In S406, the reception determination unit 11 stores the packet i in the reception buffer 13 of the rbtree data structure based on the reception start time t ⁱ _open =t _open of the packet i.

In S407, the reception determination unit 11 stores the packet i in the reception buffer 13 of the rbtree data structure based on the reception start time t ⁱ _open =t _open +T _cycle of the packet i.

<Method 3 of reception determination by the reception control unit 121 in embodiment 1>
The reception determination method 3 of the reception control unit 121 in the embodiment 1 will be described. In the reception determination method 3, the amount of calculations in the flow when a packet is received is further reduced by constantly calculating canRcv, which indicates whether reception is possible, and the next transmission permission start time in the reception permission time calculation flow of the reception determination method 2. When a packet is received, the truth value canRcv is determined, and if canRcv=false, the packet is only stored in the reception buffer 13.

An example of the operation of the reception determination unit 11 is shown with reference to Figures 13 and 14. The transmission process from the reception buffer 13 is the same as in reception determination method 1.

13 shows a flow of calculating the permitted reception time. In S501, the reception determination unit 11 calculates t _open =t ₀ +T _open , t _close =t ₀ +T _close , and canRcv =false. In S502, the reception determination unit 11 acquires the current timestamp value t _now .

In S503, the reception determination unit 11 judges whether t _now ≧t _open , and if Yes, proceeds to S504, and if No, proceeds to S506. In S504, the reception determination unit 11 judges whether t _now >t _close , and if Yes, proceeds to S505, and if No, proceeds to S507.

In S505, the reception determination unit 11 calculates canRcv=false, t _open =t _open +T _cycle , and t _close =t _close +T _cycle .In S506, canRcv=false is set, and in S507, canRcv=true is set.

Figure 14 shows the processing flow when the reception determination unit 11 receives a packet. In S601, the reception determination unit 11 receives packet i from the guest 110. In S602, the reception determination unit 11 determines whether canRcv = true, and if Yes, proceeds to S603, and if No, proceeds to S604.

In S603, the reception determination unit 11 transfers packet i to the class assignment unit 122. In S604, the reception determination unit 11 stores packet i in the reception buffer 13 of the rbtree data structure based on the next reception permission time t ⁱ _open =t _open .

When there are multiple (n) permitted reception times in one period, the first time t _open,1 , t _close,1 to the nth time t _open,n , t _close,n are prepared, and when t _now > t _close,i , a determination is made as to the next (i+1)th permitted reception time.

15 shows the flow of calculation of the permitted reception time in this case. In S701, i=1, and in S702 to S704, the reception determination unit 11 calculates t _open,i =t ₀ +T _open,i , t _close,i =t ₀ +T _close,i , and canRcv=false n times. In S705, i=1, and in S706, the reception determination unit 11 acquires the current timestamp value t _now . The following process is performed n times between S707 and S712.

In S708, the reception determination unit 11 judges whether t _now ≧t _open,i , and if Yes, proceeds to S709, and if No, proceeds to S713. In S709, the reception determination unit 11 judges whether t _now >t _close,i , and if Yes, proceeds to S710, and if No, proceeds to S714.

In S710, the reception determination unit 11 calculates canRcv=false, t _open,i =t _open,i +T _cycle , and t _close,i =t _close,i +T _cycle . In S713, canRcv=false is set, and in S714, canRcv=true is set.

<Schedule information managed in schedule DB 130>
Next, an example of schedule information managed by the schedule DB 130 will be described. The schedule DB 130 manages schedule information for time-division transmission control in the reception control unit 121 and the transmission control unit 140. An example of schedule information represented in a three-layer data structure is shown below. (a) in Fig. 17 shows the first layer, (b) shows the second layer, and (c) shows the third layer.

The first layer stores the relative time when the transmission starts (gate open) and ends (gate close) for the time slot, and the second layer stores the transmission class for each time slot. The third layer manages the flow information (receiving interface on the host (relay device) side, flow identifier, transmission amount) transmitted within the transmission class.

The transmission start/end relative time indicates the elapsed time from the reference time t0 at _which the transmission cycle starts. The flow identifier may be one or more of packet header information such as a transmitting/receiving MAC address, a transmitting/receiving IP address, a transmitting/receiving port number, a protocol type, a VLAN number, a VLAN PCP, a DSCP value, etc., or may be a hash value calculated from them.

The transmission control unit 140 changes the transmission processing of the queue associated with the transmission class in a time-division manner based on the information from the first and second hierarchical layers.

The data structure in which schedule information is organized for each reception I/F is shown in Fig. 18. In Fig. 8, the second to fourth hierarchical layers correspond to the first to third hierarchical layers in Fig. 17. The reception control unit 121 associated with each vNIC uses the values of Gate open/close of the time slot of the second hierarchical layer as T _open and T _close .

In addition, the metadata assignment unit 12 within the receiving control unit 121 refers to the fourth layer table based on flow identifier information such as the sending and receiving IP addresses of the received packet, and assigns matching transmission class information to the received packet as metadata for the packet.

When the flow identifier of a packet sent from guest 110 is uniquely determined (when only one type of communication is performed from the guest), the transmission class and flow identifier sent in one time slot are uniquely determined.

(Example 2: Method of class determination by classifier)
Next, a second embodiment will be described. In the second embodiment, the assignment of metadata of class information, which was previously performed by each reception control unit 121, is performed collectively by the class assignment unit 122. The class assignment unit 122 needs to determine the class of a packet, so it refers to schedule information from the schedule DB 130. In Fig. 19 showing the configuration of the host 100 of the second embodiment, a line is shown indicating that the class assignment unit 122 refers to schedule information.

Figure 20 shows an example of the configuration of the reception control unit 121 in Example 2. As shown in Figure 20, the reception control unit 121 in Example 2 includes a reception determination unit 11 and a reception buffer 13. The reception control unit 121 only performs the determination of whether reception is possible using the reception determination unit 11, and buffer processing when reception is not possible. Note that "reception possible" refers to whether or not a packet can be received by the transmission control unit 140.

Figure 21 shows an example of the configuration of the class assignment unit 122 in Example 2. As shown in Figure 21, the class assignment unit 122 includes a class determination unit 21 and an identifier assignment unit 22. The class determination unit 21 determines the class of a packet received from the reception control unit 121, and the identifier assignment unit 22 assigns an identifier such as a VLAN tag corresponding to the determined class. The processing is the same as in Example 1, except that class determination is performed by the class assignment unit 122 instead of the reception control unit 121.

(Embodiment 3: A method in which an interface is provided for each communication type)
Next, a description will be given of a third embodiment. In the first and second embodiments, when the guest 110 performs a plurality of types of communication (communications with different destination IP addresses, sending and receiving port numbers, protocol types, etc.), the amount of searches in the schedule DB 130 increases, causing delays.

Therefore, in Example 3, vNICs are prepared in the relay device for the number of communications performed by guest 110, and one vNIC is used for one communication type, thereby speeding up the class determination process.

Figure 22 shows an example in which a guest 110 is connected to a virtual relay device 120 on the host 100 side via three vNICs 1 to 3.

The three vNICs 1 to 3 may belong to different networks, or they may belong to the same L2 network. If they belong to the same L2 network, guest 110 further has a routing table for each vNIC it belongs to, and changes the routing table it references depending on the source IP address. Figure 22 shows such an example.

Figure 23 shows schedule information organized by receiving I/F by allocating only one communication to one vNIC. By using the schedule information shown in Figure 23, the receiving control unit 121 corresponding to each vNIC can identify the transmission class simply by referring to the time slot information for determining whether reception is possible, eliminating the need to refer to the DB just to determine the transmission class. This reduces the amount of searches and delays.

(Embodiment 4: A method in which the reception control unit 121 has a reception buffer for each time slot)
Next, a fourth embodiment will be described. In the fourth embodiment, the reception control unit 121 prepares a reception buffer for one time slot in advance, and performs the process of reception determination and fetching from the reception buffer on a packet basis, but performs the process on a time slot basis, thereby reducing the amount of calculation in the reception determination unit 11.

Figure 24 shows an example of the configuration of the reception control unit 121 in Example 4. As shown in Figure 24, the reception control unit 121 in Example 4 includes a reception determination unit 11, a transmission determination unit 15, and a reception buffer for each time slot.

The reception determination unit 11 determines which time slot to transmit a packet received from the vNIC based on information from the schedule DB 130, and stores the packet in a receiving buffer corresponding to the time slot in which the packet will be transmitted.

Based on the information in the schedule DB 130, the transmission determination unit 15 reads packets from the receiving buffer for each time slot on the same schedule as the transmission control unit 140 and transmits them to the class assignment unit 122.

(Embodiment 5: Method of dividing transmission queues by time slots)
Next, a fifth embodiment will be described. In the fifth embodiment, the reception control unit 121 does not control the reception timing by using a reception buffer, but the transmission control unit 140 distributes communications of the same class within one period of the transmission schedule to different transmission queues, thereby preventing deviation of the transmission schedule.

In the fifth embodiment, if the same transmission class is assigned multiple times to a schedule in one period, a separate transmission queue is prepared for each. Figure 25 shows an example of transmission permission classes in one period. In Figure 25, for example, 1-3 indicates the third transmission queue corresponding to transmission permission class 1. For example, 1-1, 1-2, and 1-3 indicate that different transmission queues are assigned to the same transmission class 1.

For ease of explanation, one transmission class is assigned at the same time, but it is also possible to assign multiple transmission classes at the same time.

Figure 26 shows an example of the configuration of the reception control unit 121 in Example 5. As shown in Figure 26, the reception control unit 121 does not have a reception buffer 13, but has a class determination unit 21. Based on the information in the schedule DB 130, the class determination unit 21 determines the transmission class of the received packet and which time slot in one period it is assigned to, assigns the transmission class and time slot information as metadata for the packet, and sends it to the class assignment. Note that the class assignment unit 122 may be included in the reception control unit 121, and the reception control unit 121 may perform the class identification assignment.

As in the second embodiment, the class assignment unit 122 may directly perform the judgment and even the class assignment, instead of the reception control unit 121. In this case, the reception control unit 121 is unnecessary. In this case, the class assignment unit 122 may be called the reception control unit 121. In other words, the class assignment unit 122 may be used as the reception control unit 121.

Figure 27 shows an example of the configuration of the transmission control unit 140. As shown in Figure 27, the transmission control unit 140 includes a class classification unit 141, a transmission gate opening/closing unit 142, and a plurality of transmission queues for each transmission class, and distributes packets to the transmission queues based on the transmission class and the assigned time slot information.

The transmission gate opening/closing unit 142 opens and closes the gate for each transmission queue according to the schedule information. In the fifth embodiment, the transmission queue in the transmission control unit 140 plays the role of a receiving buffer, thereby realizing jitter compensation.

(Example 6: Example of applying the technology according to the present invention to one physical NIC)
Next, a description will be given of embodiment 6. In embodiment 6, any one or a combination of any two or more of embodiments 1 to 5 is implemented within one physical NIC 150.

Figure 28 shows an example configuration of the host 100 in Example 6. As shown in Figure 28, the physical NIC 150 has multiple virtual NICs, VFs, and a reception control unit 121 (for cases other than Example 5), for each VF, and further includes a class assignment unit 122, a transmission control unit 140, and a schedule DB 130. The schedule DB 130 may be provided on the host side. In addition, in the configuration shown in Figure 28, the part consisting of the VFs, reception control unit 121, and class assignment unit 122 may be called a "relay device."

Each guest 110 bypasses the host's internal processing (such as network processing within the Linux kernel) and directly uses the VF on the physical NIC 150. This can be achieved, for example, by Single Root I/O Virtualization.

(Common to Examples 1 to 6)
The same functions may also be responsible for other functions as the reception control unit 121, the class assignment unit 122, and the transmission control unit 140 described above. For example, the effect of the technology according to the present invention can be achieved by having the functions of the class assignment unit 122 and the transmission control unit 140 within the reception control unit 121.

In addition, the time at which a packet is transmitted from the receiving control unit 121 to the transmitting control unit 141 may be set earlier by the amount of the transfer delay from the receiving control unit 121 to the transmitting control unit 140, after calculating the transfer delay in advance.

(Hardware configuration example)
Any of the devices described in this embodiment (host, guest, server having the same function as the guest, relay device, virtual relay device, communication control device) can be realized by, for example, having a computer execute a program describing the processing contents described in this embodiment. The computer may be a physical machine or a virtual machine on the cloud.

The above program can be recorded on a computer-readable recording medium (such as a portable memory) and can be stored or distributed. The above program can also be provided via a network such as the Internet or e-mail.

Figure 29 is a diagram showing an example of the hardware configuration of the computer. The computer in Figure 29 has a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, etc., which are all connected to each other via a bus B.

The program that realizes the processing on the computer is provided by a recording medium 1001, such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 via the drive device 1000 into the auxiliary storage device 1002. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.

When an instruction to start a program is received, the memory device 1003 reads out and stores the program from the auxiliary storage device 1002. The CPU 1004 realizes the functions related to the device in accordance with the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network. The display device 1006 displays a GUI (Graphical User Interface) based on a program, etc. The input device 1007 is composed of a keyboard and mouse, buttons, a touch panel, etc., and is used to input various operational instructions. The output device 1008 outputs the results of calculations.

(Summary of the embodiment)
In the present embodiment described above, in a relay device connected to multiple servers on which communication applications are running, packet reception processing is performed at the receiving interface on the relay device side connected to each server in synchronization with a transmitting interface connected to an external network that performs time-division transmission control.

More specifically, at the receiving interface, the flow information and reception time of the received packet are used to determine the time slot and class in which the received packet will be transmitted at the transmitting interface, and a transmission class identifier is assigned to the packet.

Furthermore, as described in Example 1, by matching the determined received packet with the reception processing start time calculated from the transmission time of the received packet at the transmitting interface, the packet is temporarily buffered until the reception processing start time, and the received packet is transferred from the receiving interface to the physical interface immediately before transmission processing is performed at the transmitting interface.

As described in Example 4, the receiving interface may be provided with an independent receiving buffer for the number of time slots included in one period of the time division transmission schedule performed by the transmitting interface, and packets may be transferred from the receiving buffer in synchronization with the time division transmission schedule performed by the transmitting interface.

As described in Example 3, the guest and host may be connected via multiple virtual transmitting/receiving interfaces, and one type of communication may be assigned to each pair of transmitting/receiving interfaces, thereby speeding up the class determination process in the receiving interfaces.

As described in Example 5, the transmitting interface may be provided with a virtual transmitting queue for each time slot of one period, and the receiving interface may determine which time slot a packet is to be transmitted in and assign an identifier, so that even packets received at a time other than the original time slot can be assigned to a transmitting queue corresponding to the original transmission time slot.

Furthermore, as described in Example 6, a communication method that combines any one or more of the above configurations may be implemented entirely within a single physical NIC.

The above-mentioned communication method does not exist in conventional technology.

In other words, since the bottleneck in switches is usually the sending side where multiple traffic streams converge, existing Linux (registered trademark) and commercially available switches do not provide a mechanism for controlling traffic on the receiving side, and the Qbv specifications similarly only stipulate control on the sending side.

On the other hand, the prior art does not provide a method for performing traffic control at multiple receiving interfaces and for multiple receiving interfaces and a transmitting interface to perform traffic control in cooperation with each other, as in the technology of the present invention.

In addition, since Qbv is controlled by the CoS value of the VLAN tag, delay guarantees within networks without VLAN tags inside hosts in virtual environments, or delay guarantees in multi-tenant data centers used by multiple different users, cannot be achieved by existing Qbv or control on the sending side alone, but can be solved by the technology of the present invention.

(Effects of the embodiment)
According to the embodiment described above, it is possible to prevent an increase in queuing delay and a shift in the transmission schedule due to traffic collision with other guest machines and to guarantee jitter without performing complicated time-division transmission control on the guest machine side. In addition, it is possible to nullify the influence of setting errors on the guest machine side and DoS attacks on the TSN transmission schedule.

(Additional Note)
This specification discloses at least the following communication control device, communication control method, and program.
(Section 1)
The system includes a plurality of reception control units for receiving packets from a server on which an application is running, and each reception control unit
A receive buffer;
a reception determination unit that transmits the packet to a transmission interface connected to an external network at a time when the packet can be received at the transmission interface, and stores the packet in the reception buffer at a time when the packet cannot be received at the transmission interface.
(Section 2)
2. The communication control device according to claim 1, further comprising: a class assignment unit that determines a time slot and a transmission class of the packet based on schedule information for time division transmission control in the transmission interface, and assigns a transmission class identifier to the packet.
(Section 3)
The communication control device described in claim 1 or 2, wherein the reception determination unit constantly calculates the start and end of the reception time based on schedule information for time-division transmission control in the transmission interface, and determines whether the current time is a time when the packet can be received in the transmission interface by comparing the current time with the start and end of the reception time.
(Section 4)
4. The communication control device according to claim 1, further comprising the reception control unit for each communication type in communication with the server.
(Section 5)
5. The communication control device according to any one of claims 1 to 4, wherein the reception control unit includes the reception buffer for each time slot included in one period of a time division transmission schedule in the transmission interface.
(Section 6)
A plurality of reception control units for receiving packets from a server on which an application runs;
a transmission control unit connected to an external network and having a plurality of transmission queues for each transmission class;
the reception control unit determines a transmission class and a time slot of the packet based on schedule information for time division transmission control in the transmission control unit, adds transmission class and time slot information to the packet, and transmits the packet to the transmission control unit;
The transmission control unit distributes the packets to one transmission queue based on the transmission class and time slot information assigned to the packets.
(Section 7)
A communication control method in a communication control device having a plurality of reception control units for receiving packets from a server on which an application runs, comprising:
A communication control method in which each reception control unit has a reception buffer, transmits the packet to a transmission interface connected to an external network at a time when the packet can be received at the transmission interface, and stores the packet in the reception buffer at a time when the packet cannot be received at the transmission interface.
(Section 8)
A program for causing a computer to function as each unit in the communication control device according to any one of claims 1 to 6.

Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and variations are possible within the scope of the gist of the present invention as described in the claims.

11 Reception determination unit 12 Metadata assignment unit 13 Reception buffer 15 Transmission determination unit 21 Class determination unit 22 Identifier assignment unit 100 Host 110 Guest 120 Virtual relay device 121 Reception control unit 122 Class assignment unit 130 Schedule DB
140 Transmission control unit 141 Classification unit 142 Transmission gate opening/closing unit 150 Physical NIC
200 Client terminal 300, 400 Relay device 500 TSN domain 1000 Drive device 1001 Recording medium 1002 Auxiliary storage device 1003 Memory device 1004 CPU
1005 Interface device 1006 Display device 1007 Input device

Claims (7)

The system includes a plurality of reception control units for receiving packets from a server on which an application is running, and each reception control unit
A receive buffer;
a reception determination unit that transmits the packet to a transmission interface connected to an external network at a time when the packet can be received at the transmission interface, and stores the packet in the reception buffer at a time when the packet cannot be received at the transmission interface,
the transmission interface includes a classifier and a plurality of transmission queues;
the communication control device further includes a class assigning unit between the plurality of reception control units and the transmission interface;
The class assignment unit
A time slot and a transmission class of the packet are determined based on schedule information for time division transmission control in the transmission interface, and a transmission class identifier is assigned to the packet.
Communications control device. 2. The communication control device according to claim 1, wherein the reception determination unit constantly calculates the start and end of the reception time based on schedule information for time division transmission control in the transmission interface, and determines whether the current time is a time when the packet can be received in the transmission interface by comparing the current time with the start and end of the reception time. The communication control device according to claim 1 , further comprising a reception control unit for each communication type in communication with the server. The communication control device according to claim 1 , wherein the reception control unit includes the reception buffer for each time slot included in one period of a time division transmission schedule in the transmission interface. A plurality of reception control units for receiving packets from a server on which an application runs;
a transmission control unit connected to an external network and having a plurality of transmission queues for each transmission class;
the reception control unit determines a transmission class and a time slot of the packet based on schedule information for time division transmission control in the transmission control unit, adds transmission class and time slot information to the packet, and transmits the packet to the transmission control unit;
The transmission control unit distributes the packets to one transmission queue based on the transmission class and time slot information assigned to the packets. A communication control method in a communication control device having a plurality of reception control units for receiving packets from a server on which an application runs, comprising:
a communication control method in which each reception control unit has a reception buffer, transmits the packet to a transmission interface connected to an external network at a time when the packet can be received at the transmission interface, and stores the packet in the reception buffer at a time when the packet cannot be received at the transmission interface,
the transmission interface includes a classifier and a plurality of transmission queues;
the communication control device further includes a class assigning unit between the plurality of reception control units and the transmission interface;
The class assignment unit
A time slot and a transmission class of the packet are determined based on schedule information for time division transmission control in the transmission interface, and a transmission class identifier is assigned to the packet.
Communications control method. A program for causing a computer to function as each unit in the communication control device according to any one of claims 1 to 5 .

Images