JP7750404B2 - Communication system, control device, communication method, and program - Google Patents

Description

The present invention relates to technology for reducing worst-case delay time and jitter in wide-area networks that apply time-division multiplexing communication methods such as Time Sensitive Networking.

Society 5.0 aims to create a Cyber Physical System (CPS) that will be used to solve social issues and create a more prosperous society by highly integrating the real world and cyberspace.

In a CPS, sensing data acquired from various devices in the real world is analyzed on the cloud, and the analysis results are fed back to the real world. Here, in many mission-critical communications, such as autonomous driving, the uploading of sensing data and the feedback of analysis results must be transmitted with low latency and low jitter.

In addition, the connection between the device and the cloud server must go via a long-distance, wide-area network compared to a local area network (LAN), etc.

In recent years, Time-Sensitive Networking (TSN) has been attracting attention as a technology that enables jitter-guaranteed communications over the widely used Ethernet (registered trademark).

TSN is comprised of multiple technical standards, and one such mechanism that guarantees maximum jitter even when passing through multiple nodes is Cyclic Queueing and Forwarding (CQF), defined in IEEE 802.1Qch (Non-Patent Document 1). CQF combines Per-Stream Filtering and Policing (PSFP), defined in IEEE 802.1Qci, and Time-Aware Shaper (TAS), defined in IEEE 802.1Qbv, to guarantee maximum jitter by alternately allocating time slots for transmission and reception in time-division communications.

CQF guarantees that the delay in E2E (end-to-end) transmission and reception is between (n-1)T and (n+1)T, where T is the time slot time and n is the number of hops, and the maximum jitter is kept within 2T. However, since it is assumed that the time slot time T is equal to or greater than the propagation delay and processing delay between each node, there is a problem of increased E2E transmission and reception delay and jitter in long-distance transmissions where propagation delays on the order of tens of milliseconds occur.

IEEE Standards, IEEE Std 802.1Qch-2017 Bridges and Bridged Networks-Amendment 29: Cyclic Queuing and Forwarding Segment Routing (SR) Based Bounded Latency draft-chen-detnet-sr-based-bounded-latency-01 https://datatracker.ietf.org/doc/pdf/draft-chen-detnet-sr-based-bounded-latency-01 Large-Scale Deterministic IP Network draft-qiang-detnet-large-scale-detnet-05 https://datatracker.ietf.org/doc/pdf/draft-qiang-detnet-large-scale-detnet-05

Technologies known to solve the issues with CQF include Cycle Specified Queueing and Forwarding (CSQF) (Non-Patent Document 2) and Deterministic IP (DIP) (Non-Patent Document 3), which are improvements on CQF.

However, these conventional technologies all require support for protocols such as Segment Routing (SR), support for proprietary headers, and the addition of time slot or transmit queue information on a packet-by-packet basis. This necessitates modification of packet headers in upper layers, which in itself poses the issue of processing delays and jitter.

Note that the need to modify packet headers, resulting in increased processing delays and jitter, is an issue that can arise not only with the technologies disclosed in Non-Patent Documents 2 and 3. Furthermore, a "packet" is an example of "data."

The present invention has been made in consideration of the above points, and aims to provide a technology for suppressing delays and jitter in data communication without requiring modification of data headers.

According to the disclosed technology, there is provided a communication system including a plurality of communication devices, which performs data transfer between the communication devices based on a time-division transmission schedule,
In each communication device, a predetermined value in the data is associated with a time slot, and in at least one communication device, the transmission schedule is corrected based on a propagation delay between the communication devices;
The predetermined value is a PCP value in a VLAN tag, and the transmission schedule is set so that adjacent time slots are associated with different PCP values.
A communication system is provided.

The disclosed technology makes it possible to reduce delays and jitter in data communication without requiring modification of data headers.

FIG. 10 is a diagram illustrating the definition of delay. 10A and 10B are diagrams illustrating examples of packet delay and jitter. 10A and 10B are diagrams illustrating examples of packet delay and jitter. FIG. 10 is a diagram for explaining prior art 2. 10A and 10B are diagrams illustrating examples of packet delay and jitter. FIG. 1 is a diagram illustrating an example of the overall configuration of a communication system. FIG. 2 illustrates an example of the configuration of a relay device. FIG. 2 illustrates an example of the configuration of a relay device and a controller. 10A and 10B are diagrams illustrating examples 1 and 2 of a transmission schedule. FIG. 1 is a diagram for explaining a first embodiment. FIG. 10 is a diagram for explaining a second embodiment. FIG. 10 is a diagram for explaining Example 2'. FIG. 10 is a diagram for explaining Example 3. FIG. 10 is a diagram illustrating an example of setting a transmission schedule. FIG. 10 is a diagram illustrating an example of setting a transmission schedule. FIG. 10 is a diagram illustrating an example of setting a transmission schedule. FIG. 10 is a diagram illustrating an example of setting a transmission schedule. FIG. 2 illustrates an example of a hardware configuration of the apparatus.

The following describes an embodiment of the present invention (the present embodiment) with reference to the drawings. The embodiment described below is merely an example, and the embodiments to which the present invention can be applied are not limited to the following embodiment.

Below, we will explain in detail the technology for suppressing E2E transmission and reception delays and maximum jitter by using only IEEE802.1Q VLAN tags, which are required to be supported by the TSN standard. Note that in the following explanation, data may be referred to as packets or frames. Packets or frames may also be referred to as data. Packets may also be referred to as frames, and frames may also be referred to as packets.

First, the definition of delay used in the following explanation is shown in Figure 1. Store-and-forward transfer is assumed at each node the packet passes through, and the queuing delay on the receiving port side that occurs in IEEE 802.1Qci, etc., is included in the store processing delay here. In the case of cut-through transfer, the definition of delay is the same as that shown in Figure 1, except that the serialization delay is always 6 bits of the destination MAC address.

As shown in Figure 1, the hop delay p _i is the sum of the forward processing delay d _{fwd_i,j} , the serialization delay d _{Srl_i,i+1} , the propagation delay d _{Prp_i,i+1} , and the store processing delay d _{Str_i+1,j} . The total delay p over n hops is p = Σ ⁿ _i=0 p _i .

(Regarding conventional technology and issues)
First, examples of conventional technologies and their problems will be described in detail with respect to the technology according to the present invention, which will be described later. Note that while the contents described in Non-Patent Documents 1 to 3 are publicly known technologies, the explanations of these problems described below are not publicly known. Also, although examples of packet delay and jitter are shown using Figures 2, 3, and 5, Figures 2, 3, and 5 are not publicly known.

<Prior art 1 (Non-patent document 1): Cyclic Queueing and Forwarding (CQF), IEEE 802.1Qch 2017>
Prior art 1 is a time division transmission/reception method in which time and frequency are synchronized between nodes, time slots are set at equal intervals, and receive time slots and transmit time slots are set alternately. This method is realized by alternately setting receive time slots in PSFP and transmit time slots in TAS.

With this method, the transmission slot is shifted by one slot for each hop, which absorbs jitter from propagation delay and processing delay and keeps maximum jitter to less than two time slot times. Figure 2 shows examples of transmission with the minimum delay (dotted line) and the maximum delay (solid line) for a 3-hop count. As shown in Figure 2, the maximum jitter (maximum delay - minimum delay) is (n+1)T - (n-1)T = 2T.

<CQF Issues>
CQF has a problem that if the time slot time T is set to be less than the hop delay, the received slot cannot be predicted, resulting in increased jitter.

In addition, for other time slots where an unexpected frame interruption occurs due to the above, it will no longer be possible to guarantee the transmission of frames that were originally scheduled to be transmitted.

Figure 3 shows an example where transmission and reception are not possible within one time slot in two of three hops. In the example shown in Figure 3, due to differences in queuing delay at each hop, the difference between reception slots increases with each hop, eventually expanding to 4T.

In reality, since the receive slot is not determined, it is difficult to guarantee transmission in the next transmit slot, which could further increase jitter. It is possible to guarantee jitter by setting the time slot time T to be greater than or equal to the maximum hop delay, but if long-distance transmission of, say, several hundred km is involved, T must be set to the order of several milliseconds. In that case, the maximum jitter will be several milliseconds to several tens of milliseconds, making jitter guarantee meaningless.

<Prior Art 2 (Non-Patent Document 2): Cycle Specified Queuing and Forwarding (CSQF)>
In CSQF of prior art 2, time synchronization and frequency synchronization are performed in the same way as in CQF, and then a transmission queue (SQ), a reception queue (RQ), and a jitter absorption queue (TQ) are prepared, which are switched every time slot (the number of TQs is adjusted depending on jitter, etc.). The switching of these queues is shown in Figure 4 (excerpt from Non-Patent Document 2).

In addition, the Segment Identifier (SID) of Segment Routing (SR) is used to assign queue information to packets sent at each node. This makes it possible to match receive queues with transmit queues by referencing the SID, even if the propagation delay and processing delay exceed the time slot time T, preventing the expansion of jitter.

The maximum delay d _max can be suppressed to P+2nT, and the maximum jitter j _max can be suppressed to 2T, the same as CQF.

<Prior Art 3 (Non-Patent Document 3): Deterministic IP (DIP)>
DIP, which is Prior Art 3, is an extension of CQF that guarantees jitter only with frequency synchronization (without time synchronization) for wide area networks where time synchronization is difficult.

Like CSQF, each node has at least three queues (two receive queues and one transmit queue) for each port, and also maintains a database of the time slot dependencies between adjacent nodes, and assigns information to each packet using SR or the like to indicate which time slot it was sent in. This makes it possible to correct for the difference in transmit and receive slots due to differences in the reference time, and transmit packets accordingly. Like CSQF, the maximum delay d _max is P + 2nT, and the maximum jitter j _max is 2T.

Figure 5 shows an example of packet forwarding in DIP. In the example shown in Figure 5, a time slot mapping is maintained such that a packet previously sent from node 0 in slot x is sent from node 1 in slot y+1, and a packet sent from node 1 in slot y+1 is sent from node 2 in slot z+1.

<Problems with conventional technology>
The conventional technologies disclosed in Non-Patent Documents 2 and 3 can solve the CQF problem. However, in addition to supporting protocols such as SR, they require support for proprietary headers, and time slot or transmission queue information must be added to each packet. This requires modification of packet headers in upper layers, which in itself causes processing delays and jitter.

(Outline of the embodiment)
Below, we will explain the technology that solves the above problems. The technology explained below is a technology that uses only IEEE802.1Q VLAN tags, which are required to be supported by the TSN standard, and suppresses E2E transmission and reception delays and maximum jitter. The outline is as follows:

In the communications system related to this technology, the propagation delay between nodes is calculated using PTP Delay_Req messages, etc., and the time-division transmission schedule between relay devices is corrected to achieve jitter compensation and reduce maximum delay, even when long-distance transmission is included between the transmitting and receiving terminals. There are the following variations of the transmission schedule correction method (details are explained in Examples 1 to 3):

- The reference time at which the repeatedly applied transmission schedule starts is corrected for each relay device (Example 1).

- The transmission schedule is corrected for each relay device based on the number of time slots corresponding to the propagation delay time (Examples 2 and 2').

- Each time the cumulative delay between the transmitting terminal and the receiving terminal exceeds a threshold, the transmission schedule is corrected by the number of time slots corresponding to the cumulative delay exceeded (Example 3).

In the above communication system, each node notifies adjacent nodes of a correction time, and the notified node calculates its own correction time from the calculated propagation delay and repeatedly notifies other nodes of the correction time, thereby autonomously (or through centralized control by a controller) correcting the transmission schedules of all devices in the network.

Note that the technology described below is one embodiment of the present invention, and the scope of application of this invention is not limited to the technology described below. For example, instead of the VLAN PCP value used in the technology below, a value other than the VLAN PCP value in the header may be used.

(Example of overall system configuration)
An example of the overall configuration of a communication system according to this embodiment is shown in Fig. 6. As shown in Fig. 6, this system includes a transmitting terminal 100, a relay device 200, a receiving terminal 400, a controller 300, and a TSN domain 500.

Multiple transmitting terminals 100-1 to 100-3 each transmit data. In addition, transmitting terminal 100 performs periodic communications with receiving terminal 400 that require jitter compensation. TSN domain 500 is a network consisting of multiple transmitting terminals 100-1 to 100-3, multiple relay devices 200-1 to 200-4 that support TSN (including IEEE 802.1AS, Qbv, Qci, etc.), and multiple receiving terminals 400-1 to 400-3. Each of the multiple receiving terminals 400-1 to 400-3 receives the data transmitted from transmitting terminal 100.

TSN domain 500 may have any network topology and may include long-distance transmissions spanning tens to hundreds of kilometers.

Management of schedules such as transmission schedules may be performed by each device (transmitting terminal, relay device, receiving terminal), or the controller 300 may be configured to manage the schedules of the devices included in the TSN domain 500.

The controller 300 may be capable of communicating with all devices included in the TSN domain 500 and may have the ability to remotely set the transmission schedule of each device.

In the example of Figure 6, for the sake of convenience, it is assumed that the transmitting terminal 100 transmits data and the receiving terminal 400 receives the data, but it is also possible to transfer data from the receiving terminal 400 to the transmitting terminal 100.

The transmitting terminal 100, relay device 200, receiving terminal 400, and controller 300 may all be physical machines or virtualized devices such as VMs or containers.

(Example of relay device configuration)
Fig. 7 shows an example configuration of relay device 200. As shown in Fig. 7, relay device 200 has multiple receiving ports 210-1 to 210-2, a forwarding information base (FIB) 220, a scheduler correction unit 230, a time synchronization processing unit 240, a transmission scheduler 250, a transmission schedule 255, and multiple transmitting ports 260-1 to 260-2.

Note that for convenience, Figure 7 shows a receiving port and a transmitting port, but both the receiving port and the transmitting port may be a transmitting/receiving port capable of transmitting and receiving. Furthermore, the transmitting terminal 100 and the receiving terminal 400 may each have the configuration shown in Figure 7.

The transmitting terminal 100, the relay device 200, and the receiving terminal 400 may be collectively referred to as "communication devices." The transmitting terminal 100, the relay device 200, and the receiving terminal 400 may also be collectively referred to as "nodes." The controller 300 may also be referred to as a "control device."

The received packet received by the receiving port 210 is sent to the corresponding transmitting port 260 after referencing the FIB 220 based on its destination information, etc.

Each transmission port 260 has multiple transmission queues, a transmission schedule 255 that determines which transmission queue to send from based on the time, and a transmission scheduler 250 that performs dequeue processing from each transmission queue based on the transmission schedule 255.

Time synchronization is performed between relay devices 200 (similarly between the transmitting terminal 100 and relay device 200, and between the receiving terminal 400 and relay device 200) by the time synchronization processing unit 240 using a time synchronization protocol such as PTP. Here, if the relay device 200 is a slave in PTP, it calculates the propagation delay with the opposing device from the Delay_Req/Resp message, or if it is a master, it calculates the propagation delay from the timestamp value in the Delay_Req message, and notifies the scheduler correction unit 230 of the calculated value. The scheduler correction unit 230 corrects the transmission schedule 255 based on the propagation delay.

Here, various methods can be used to obtain delay, such as obtaining it on a packet-by-packet basis using P4 INT, or using unique probe packets, without relying on PTP.

In addition, the transmission schedule 255 and the scheduler correction unit 230 can be managed by the controller 300, and the corrected transmission schedule 255 can be distributed from the controller 300 to the relay device 200.

The configuration shown in Figure 7 is just one example. For example, similar control can be performed using receive queues instead of transmit queues, as in PSFP, or control can be performed by combining receive queues and transmit queues, as in CQF.

(Configuration example in which the controller 300 determines the transmission schedule for the entire domain)
8 shows an example of a configuration in which the controller 300 performs scheduler correction for the entire TSN domain 500. FIG. 8 shows the configuration of the relay device 200 and the configuration of the controller 300.

The relay device 200 shown in Figure 8 differs from the configuration shown in Figure 7 in that it includes a controller transceiver unit 270 instead of the scheduler correction unit 230. The transmitting terminal 100 and the receiving terminal 400 may each have a configuration similar to that of the relay device 200 shown in Figure 8.

The controller 300 shown in Figure 8 has a TSN domain transceiver unit 310, a topology calculation unit 320, a topology DB 330, and a scheduler correction unit 340.

The relay device 200 (similar to the transmitting terminal and receiving terminal) transmits adjacent node information obtained using its own FIB, LLDP, etc., and calculated delay information between adjacent nodes (e.g., information on each delay shown in Figure 1) to the controller 300 via the controller transceiver unit 270.

In the controller 300, the topology calculation unit 320 calculates the topology of the entire TSN domain 500 from the FIB and adjacent node information and registers it in the topology DB 330. Thereafter, if any changes occur to the configuration, the topology DB 330 is updated. In addition, the delay time of each link in the topology is also registered in the topology DB 330 based on the delay information between adjacent nodes.

The scheduler correction unit 340 of the controller 300 corrects the transmission schedule of the relay device 200 (and the same for the transmitting terminal) based on the topology and delay information, and distributes the transmission schedule 255 for each transmitting port of each transmitting terminal and each relay device 200 to each transmitting terminal and each relay device 200 via the TSN domain transmission/reception unit 310. The TSN domain transmission/reception unit 310 may also be called the "distribution unit."

Regarding the transmission schedule described below, the relay device 200 (transmitting terminal 100) may use the transmission schedule 255 corrected (set, created) by its own scheduler correction unit 230, or may use the transmission schedule distributed by the controller 300.

(Basic transmission schedule)
Each transmitting terminal 100 and each relay device 200 sets the same transmission schedule except for the reference time in accordance with the following rules.

- Only one transmission queue is transmitted during each time slot.

- Transmit queues correspond to the Priority Code Point (PCP) value in the VLAN tag specified in IEEE802.1Q. That is, for example, a packet with a PCP value of 1 is queued in transmit queue 1.

- Schedule transmission so that adjacent time slots have different PCP values.

The amount of data transmitted in each time slot is limited in advance to the amount of data that can be transmitted within one time slot, or communications that require jitter compensation are guaranteed to be transmitted within one time slot using priority control such as IEEE802.1Qbu.

Figure 9 shows examples 1 and 2 of transmission schedules for a certain transmission port.

In Example 1, PCP values 0 to 7 are set in order for the time slots, and each transmit queue is always ready to transmit every 8 slots. For example, transmit queue 0 is ready to transmit in slot "0" in Example 1, and is ready to transmit again 8 slots later. This transmission period for each transmit queue is the maximum value that can absorb jitter.

In example 2, PCP value 0 is set every other slot, so the transmission period for PCP values 2-7 is every 13 slots.

Hereinafter, examples 1 to 3 will be explained as specific examples of how the scheduler correction unit 230/340 corrects the transmission schedule.

In the following, the time slot time T is set to the same value to make it easier to calculate the maximum delay and jitter, but the technology of the present invention can also be applied and the effects of the present invention can be obtained even if different time slot times are set for each time slot.

Figures 10 to 13 in Examples 1 to 3 show the time slots (arrangement) at each node. The number inside the frame (box) indicating the time slot indicates the PCP value set for that time slot. That PCP value corresponds to the transmission queue that transmits in that time slot. Also, in Figures 10 to 13, dotted lines indicate packet transfers with minimum delay, and solid lines indicate packet transfers with maximum delay.

It is also possible to combine any two or more of the embodiments in Examples 1, 2, 2', and 3.

(Example 1: Correcting the reference time by the hop delay)
First, a first embodiment will be described. In the first embodiment, the reference time of the transmission schedule is delayed for each hop based on the propagation delay between adjacent devices acquired by the relay device 200 and the forwarding process delay of the relay device 200 itself. Note that a method of delaying the reference time of the transmission schedule for each hop based only on the propagation delay may also be used.

An example is shown in Figure 10. In the example of Figure 10, for packets with a PCP value of 0, the transfer of packets with the minimum delay (dotted line) and the transfer of packets with the maximum delay (solid line) are shown. As shown in Figure 10, the reference time at node 1 is delayed from the reference time at node 0 by the propagation delay time between node 0 and node 1. The same is true for subsequent nodes.

Here, if time synchronization error, propagation delay jitter, and processing delay jitter can be ignored, packets with any PCP value can be transmitted as is within the received time slot. Because the transmission time slot is determined by the PCP value, relay device 200 does not need to assign transmission slot information, replace headers, or perform time slot mapping between devices.

Therefore, the minimum delay d _min is P, the maximum delay d _max is P+T, and the maximum jitter j _max =T can be guaranteed.

On the other hand, in the method of Example 1, time synchronization errors and various delay jitters (serialization delay differences due to differences in frame length, processing delay jitter, etc.) directly affect the maximum jitter, so if synchronization errors and jitter cannot be ignored, the following Example 2 is applied.

(Example 2: Correcting the transmission schedule by the time slot corresponding to the hop delay)
In the second embodiment, for a time slot time T and an ith hop delay p _i ,

The transmission schedule is shifted by the smallest time slot that is equal to or greater than the hop delay. For example, if the hop delay is 0.5T, the transmission schedule is shifted by T, and if the hop delay is 1.5T, the transmission schedule is shifted by 2T.

Figure 11 shows an example of a transmission schedule shift. In the example of Figure 11, the transmission schedule of node 1 is shifted by one slot relative to node 0, the transmission schedule of node 2 is shifted by two slots relative to node 1, and the transmission schedule of node 3 is shifted by three slots relative to node 2.

By shifting the transmission schedule by the minimum time slot that is equal to or greater than the hop delay, it becomes possible to absorb time synchronization errors and jitters by T-p _i mod T for each hop while suppressing E2E transmission and reception delays.

In addition, if the time slot times are evenly spaced, the reference time is unified across all nodes, so there is no need to adjust the reference time for each node.This can be achieved simply by applying the Gate Control List specified in the standard IEEE 802.1Qbv to each node.

The minimum delay, maximum delay and maximum jitter can be calculated as follows. The maximum delay is (n-1)T smaller than SCQF and DIP, and the maximum jitter is the same at 2T.

that's why

(Embodiment 2': Countermeasures against larger time synchronization errors and jitter)
In the second embodiment, a countermeasure against a larger time synchronization error and jitter than those in the second embodiment will be described.

In Example 2', the transmission time slot is allocated for each hop.

By shifting the time by 1, it is possible to further absorb errors and fluctuations. _{α i} is an integer and can be adjusted according to the magnitude of the error and fluctuation expected for each hop. For example, if the maximum value of the sum of the expected time synchronization error and propagation delay fluctuation (difference in propagation delay for each packet) at hop i can be expressed as e _i T, then α _i is the smallest integer equal to or greater than e _i .

Fig. 12 shows an example of shifting transmission time slots. Fig. 12 shows an example where α _i = 1. In the example of Fig. 12, the transmission schedule of node 1 is shifted by two slots relative to node 0, the transmission schedule of node 2 is shifted by three slots relative to node 1, and the transmission schedule of node 3 is shifted by four slots relative to node 2.

In Example 2', the maximum jitter remains less than 2T, and the maximum delay is d _max < P + nT + ΣαT. If an extra slot is shifted per hop, d _max < P + (2n + 1)T, which is equivalent to SCQF and DIP.

12, even if there is an error in time synchronization, if the error is within _αi slots, it is absorbed as a fixed queuing delay. Also, even if there is fluctuation in propagation delay, if the fluctuation is within _αi slots, the jitter is absorbed as a queuing delay.

(Example 3: Schedule correction only when cumulative hop delay exceeds threshold)
In the second embodiment, it is necessary to shift at least one time slot for each hop, and when a LAN or the like with a small propagation delay is included, the E2E transmission and reception delay increases.

Therefore, in Example 3, the transmission schedule correction of Example 2 is performed for each section where the total value of hop delay exceeds the threshold. In Example 3, since it is necessary to take into account the hop delay of each section from the transmitting terminal 100 to the receiving terminal 400, the transmission schedule correction is performed by the controller 300.

However, it is also possible to implement Example 3 without using the controller 300. For example, each relay device 200 acquires the propagation delay and processing delay from each of the other relay devices 200, and each relay device 200 calculates the total value of the hop delay described below, and the relay device 200 that determines that it should perform a schedule correction performs the schedule correction.

Here, the transmission schedule correction is performed by the controller 300. The scheduler correction unit 340 of the controller 300 acquires the propagation delay and processing delay calculated by each relay device 200 from each relay device 200, and adds the hop delay in order from the transmitting terminal 100 toward the receiving terminal 400. If the total hop delay exceeds a threshold, the schedule correction of Example 2 (or Example 2') is performed on the relay device 200 immediately before it exceeds the threshold. From the next relay device 200 after which the schedule correction was performed, the addition of the hop delay and the schedule correction are performed again until the receiving terminal 400 is reached.

In other words, in Example 3, when correcting the transmission schedule, each time the cumulative delay in the relay device between the transmitting terminal and the receiving terminal exceeds a threshold, the transmission schedule is corrected by a time length equal to the number of time slots corresponding to the cumulative delay exceeded.

The threshold can be determined based on whether the remaining transmission time (T-Σp _i ) excluding the hop delay is sufficient for packet transmission.

Figure 13 shows an example in which schedule correction is performed at nodes 2 and 3. In Figure 13, schedule correction is not performed at node 1, so the E2E transmission and reception delay is reduced by one time slot compared to Example 2.

(Schedule correction transmission method between relay devices)
An example of a method for correcting schedules only between relay devices 200 without using controller 300 will be described below.

First, a node (reference node) that will serve as the basis for the schedule is selected from among the nodes in the network. There are several ways to select a node, such as setting a priority for each node and selecting the node with the highest priority in the network, or selecting the node with the smallest total number of hops and propagation delay from all nodes. Any of these methods may be used.

Step 1)
The reference node notifies each adjacent node of a correction time (0 in the case of the reference node) which indicates the time to make a correction to the reference node.

Step 2)
After receiving the notification of the correction time, each adjacent node synchronizes time with the reference node using PTP, and then stores the propagation delay with the reference node obtained from a PTP_Delay_Req message or the like (Example 1), or the time slot that takes the propagation delay into account (Examples 2, 2', and 3), as the correction time for that adjacent node, and corrects the transmission schedule of the port connected to the reference node by the correction time. "Correcting by the correction time" means correcting to an earlier time. In addition, each adjacent node corrects the transmission schedule of the port connected to an adjacent node other than the reference node by the correction time.

Step 3)
After the schedule correction, the node notifies the adjacent node that did not have its schedule corrected of its own correction time. The node that received the correction time calculates the propagation delay in the same way as the adjacent node described in step 2, and stores the sum of the notified correction time and the correction time calculated from the propagation delay with the adjacent node as the correction time.

Step 4)
The node that performed the above-mentioned holding corrects the transmission schedule of the port connected to the node that notified the correction time to the earlier by the summed correction time, and corrects the transmission schedule of the port connected to other adjacent nodes to the later by the correction time.

From here on, the same procedures described in steps 1 to 4 are repeated until schedule correction is complete for all nodes.

When controller 300 corrects the schedule, it performs the above calculations internally based on the propagation delay obtained from each node, and distributes only the schedule correction results to each node.

<Setting example>
An example of setting a transmission schedule when node 1 of nodes 0 to 4 is set as the reference node will be explained with reference to Figures 14 to 17. In each figure, the numbers on the links indicate the number of time slots corresponding to the propagation delay. The step numbers below correspond to the step numbers above.

In step 1 (Figure 14), node 1 notifies node 0 and node 2 of correction time = 0.

In step 2 (Figure 15), nodes 0 and 2 each calculate a correction time from the propagation delay with node 1. The correction times are 1T and 5T, respectively. Nodes 0 and 2 each correct the transmission schedules of the ports connected to node 1 and the ports connected to nodes other than 1 according to the correction time.

Specifically, as shown in Figure 15, node 0 corrects the transmission schedule of the node 1 connected port to 1T forward, node 2 corrects the transmission schedule of the node 1 connected port to 5T forward, and corrects the transmission schedules of ports other than node 1 to 5T backward.

In step 3 (Figure 16), node 2 notifies node 3 and node 4 of its correction time 5T, and node 3 and node 4 each calculate their own correction time from the propagation delay with node 2 and the notified correction time.

In step 4 (Figure 17), node 3 and node 4 each correct the transmission schedule of each port according to the calculated correction time. Specifically, node 3 corrects the transmission schedule of the node 2 connection port to 8T later, and node 4 corrects the transmission schedule of the node 2 connection port to 7T later.

(Example of hardware configuration)
Any of the devices described in this embodiment (transmitting terminal, receiving terminal, relay device, and controller) can be realized by, for example, causing a computer to execute a program that describes the processing content described in this embodiment. The computer may be a physical machine or a virtual machine on the cloud.

The above program can be recorded on a computer-readable recording medium (such as portable memory) and stored or distributed. The above program can also be provided via a network, such as the Internet or email.

Figure 18 is a diagram showing an example of the hardware configuration of the above-mentioned computer. The computer in Figure 18 has a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, etc., which are all interconnected by bus B.

The program that realizes processing on the computer is provided by a recording medium 1001, such as a CD-ROM or memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001; it may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.

When an instruction to start a program is received, the memory device 1003 reads the program from the auxiliary storage device 1002 and stores it. The transmission queue is realized using the storage area of the memory device 1003. The CPU 1004 realizes the functions related to the device in accordance with the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network. The display device 1006 displays a GUI (Graphical User Interface) or the like according to the program. The input device 1007 is composed of a keyboard, mouse, buttons, touch panel, or the like, and is used to input various operational instructions. The output device 1008 outputs the results of calculations.

Note that using a device using CPU 1004 as described above is just one example. For example, the functions of the device described in this embodiment can also be implemented in an ASIC (or programmable ASIC) such as an L2 switch, or a Smart NIC with an FPGA built in.

(Effects of the embodiment)
The technology according to the present embodiment described above makes it possible to reduce transmission and reception delays and guarantee maximum jitter, even if the path includes a long-distance transmission route that exceeds the time slot time, using only the functions required for standard IEEE802.1Qbv, without requiring segment routing, implementation of a unique header, rewriting of headers for each packet, or maintaining time slot dependencies between relay devices.

<Additional Notes>
The following additional clauses are disclosed in relation to the above-described embodiment.
(Additional note 1)
A communication system including a plurality of communication devices, which performs data transfer between the communication devices based on a time-division transmission schedule,
In each communication device, a predetermined value in the data is associated with a time slot, and in at least one communication device, the transmission schedule is corrected based on a propagation delay between the communication devices.
(Additional note 2)
2. The communication system according to claim 1, wherein in each communication device, only one transmission queue is assigned to one time slot, and different transmission queues are assigned to adjacent time slots.
(Additional note 3)
3. The communication system according to claim 1, wherein the at least one communication device corrects a reference time at which a repeatedly applied transmission schedule starts based on a propagation delay between adjacent communication devices.
(Additional note 4)
3. The communication system according to claim 1, wherein the at least one communication device corrects the transmission schedule by a time length corresponding to the number of time slots based on a propagation delay between adjacent communication devices.
(Additional note 5)
3. The communication system according to claim 1, wherein, in the at least one communication device, when an accumulated delay exceeds a threshold, the transmission schedule is corrected by a time length equal to the number of time slots based on the accumulated delay.
(Additional note 6)
A control device used in a communication system that includes a plurality of communication devices and performs data transfer between the communication devices based on a time-division transmission schedule,
In each communication device, a predetermined value in the data is associated with a time slot;
a correction unit that corrects a transmission schedule for at least one communication device based on delay information between the communication devices collected from the communication system;
a distribution unit that distributes the corrected transmission schedule to the communication device.
(Supplementary Note 7)
A communication method in a communication system including a plurality of communication devices, the communication devices transferring data based on a time-division transmission schedule, the method comprising:
A communication method, wherein each communication device associates a predetermined value in the data with a time slot, and at least one communication device corrects the transmission schedule based on a propagation delay between the communication devices.
(Supplementary Note 8)
A non-transitory storage medium storing a program for causing a computer to function as each unit in the control device described in appended claim 6.

The present embodiment has been described above, but the present invention is not limited to such a specific embodiment, and various modifications and variations are possible within the scope of the gist of the present invention as described in the claims.

100: transmitting terminal 200: relay device 210: receiving port 220: FIB (Forwarding Information Base)
230 Scheduler correction unit 240 Time synchronization processing unit 250 Transmission scheduler 255 Transmission schedule 260 Transmission port 260
270 Controller Transmitter/Receiver 300 Controller 310 TSN Domain Transmitter/Receiver 320 Topology Calculator 330 Topology DB
340 Scheduler correction unit 400 Receiving terminal 500 TSN domain 1000 Drive device 1001 Recording medium 1002 Auxiliary storage device 1003 Memory device 1004 CPU
1005 Interface device 1006 Display device 1007 Input device 1008 Output device

Claims (8)

A communication system including a plurality of communication devices, which performs data transfer between the communication devices based on a time-division transmission schedule,
In each communication device, a predetermined value in the data is associated with a time slot, and in at least one communication device, the transmission schedule is corrected based on a propagation delay between the communication devices;
The predetermined value is a PCP value in a VLAN tag, and the transmission schedule is set so that adjacent time slots are associated with different PCP values.
Communication system. 2. The communication system according to claim 1, wherein in each communication device, only one transmission queue is assigned to one time slot, and adjacent time slots are assigned different transmission queues. The communication system according to claim 1 or 2, wherein in the at least one communication device, a reference time at which a repeatedly applied transmission schedule starts is corrected based on a propagation delay between adjacent communication devices. The communication system according to claim 1 or 2, wherein the at least one communication device corrects the transmission schedule by a time length corresponding to the number of time slots based on a propagation delay between adjacent communication devices. The communication system according to claim 1 or 2, wherein, when an accumulated delay exceeds a threshold in the at least one communication device, the transmission schedule is corrected by a time length corresponding to the number of time slots based on the accumulated delay. A control device used in a communication system that includes a plurality of communication devices and performs data transfer between the communication devices based on a time-division transmission schedule,
In each communication device, a predetermined value in the data is associated with a time slot;
a correction unit that corrects the transmission schedule for at least one communication device based on delay information between communication devices collected from the communication system;
a distribution unit that distributes the corrected transmission schedule to the communication device;
The predetermined value is a PCP value in a VLAN tag, and the transmission schedule is set so that adjacent time slots are associated with different PCP values.
Control device. A communication method in a communication system including a plurality of communication devices, the communication devices transferring data based on a time-division transmission schedule, the method comprising:
In each communication device, a predetermined value in the data is associated with a time slot, and in at least one communication device, the transmission schedule is corrected based on a propagation delay between the communication devices;
The predetermined value is a PCP value in a VLAN tag, and the transmission schedule is set so that adjacent time slots are associated with different PCP values.
Communication method. A program for causing a computer to function as each unit in the control device described in claim 6.