JP7803027B2 - Delay Time Measurement System - Google Patents

Description

This invention relates to a delay time measurement system that accurately measures the time required to perform a series of operations on a device under test, such as capturing, transmitting, and displaying video. It provides a means for determining the binary values of changes in brightness in the video, and measures the delay time based on the threshold value determined from these. The delay time measurement is not limited to video, but can also be applied to audio and haptics.

There are many ways to measure video delay time, including inserting a timestamp, capturing and displaying a clock, and measuring round-trip time using ICMP (Internet Control Message Protocol), one of the protocols for IP (Internet Protocol) networks.

In the method using timestamps, the video sending side sends information with an accurate time stamp along with the video signal, which is then transmitted through the system under test and analyzed on the receiving side. For example, details of this method are described in Patent Document 1. However, this method cannot measure the delay time that occurs inside the video cameras and displays included in the system under test, and can only measure the delay time of part of the system. Furthermore, in addition to the original video signal, a signal with a timestamp must be inserted on the video sending side and extracted on the receiving side, making the measurement system complicated.

A method for capturing and displaying a clock is described, for example, in Patent Document 2, in which an image of the clock is captured using a video camera in the system under test, and the image displayed on the display of the system under test after transmission is compared with the image of the clock at that time to calculate the delay time. While this system is capable of measuring delay times including the video camera and display, it has the drawback of being inaccurate because it cannot accurately capture changes in the clock display.

Measurements using ICMP, one of the IP network protocols, i.e., the so-called ping command, are widely used, and there is no need to cite any literature on this subject. However, this only targets round-trip delay time, and is limited to measurements transmitted over IP networks. Furthermore, it cannot measure delays including those from video cameras and displays.

Another method is the delay time measurement system shown in Non-Patent Document 1 and Non-Patent Document 2. In this method, a light source blinked by a measuring device is captured and transmitted by a video camera in the system under test, and the blinking image displayed on a display is then converted into an electrical signal by an optical sensor, and the delay time is measured from the response.

Figure 1 shows the configuration. In Figure 1, 1 is a light source, such as a light-emitting diode (LED), and 2 is an optical sensor, such as a photodiode (PD). 3 is a reference signal generator that outputs a predetermined rectangular wave under the control of controller 7. 4 is a light source driver that generates a signal to drive light source 1 from the signal from reference signal generator 3. 5 is a trigger signal generator that inputs the signal from reference signal generator 3 and generates a signal to be applied to waveform measurement unit 6. The timing of signal changes from light source driver 4 and trigger generator 5 is identical. Waveform measurement unit 6 measures the waveforms of the signals from optical sensors 2 and 5 and the signal from trigger generator 5. The waveforms acquired by waveform measurement unit 6 are sent to controller 7, where they are analyzed and the delay time is calculated. Controller 7 is, for example, a personal computer, and the delay time calculation results are displayed on its display unit (not shown). 10 is the system under test, which includes a video camera 11, a video transmission system 12, and a display 13.

This operation is shown in the timing chart of Figure 2. When the light source drive signal, i.e., the trigger signal, is low level, light source 1 is turned off, and when it is high level, light source 1 is turned on. The output of video camera 11 of system under test 10, which captures an image of light source 1, is displayed on display 13 via video transmission system 12. The delay time of this operation is a delay in the light intensity signal output from optical sensor 2, as shown in Figure 2, and the delay time can be measured by analyzing it in controller 7, such as the time indicated by T1 when light source 1 goes from off to on, and the time indicated by T2 when it goes from on to off.

In Figure 2, the optical signal intensity signal level when the LED image on the display 13 is on is Von, and the optical signal intensity signal level when it is off is Voff. The threshold voltage Vth is determined from these values, and when the LED image changes from off to on, the time from when the trigger signal changes to when the optical signal intensity signal exceeds Vth is measured.

The configuration in Figure 1 does not use timestamps, so the video signal of the system under measurement is left untouched, and it is possible to measure all delays from the video camera 11 to the display 13. Furthermore, it uses the binary value of on/off of the light source 1, so it is possible to simply calculate the delay time. Unlike the ICMP protocol, it is not limited to transmission using an IP network, and can evaluate the one-way delay time in any video transmission system.

Patent No. 3892844 Patent No. 5553409

Taro Ogawa and Hiroyuki Matsuura, "Zero-Delay Optical Video Transmission Technology and Its Applications," Large Display & Digital Signage Overview 2020, pp. 12-22, Japan Social Systems Laboratory, 2020. Hiroyuki Matsuura and Taro Ogawa, "Development of Transmission Technology for Zero Latency of 4K/8K Video," Optical Circuit Implementation and Device Development for Next-Generation High-Speed Communications, pp. 614-628, Information Technology Association, 2022.

Before measuring the delay time, Von and Voff in Figure 2 must be determined. These will vary depending on the relative positions of the light source 1 and video camera 11, the relative positions of the display 13 and optical sensor 2, and the conditions of the system under test 10. Therefore, Von and Voff must be known beforehand before measurement.

A simple way to find Von and Voff is to (a) measure the light intensity signal with Light Source 1 off and use this as Voff, and (b) measure the light intensity signal with Light Source 1 on and use this as Von, as shown in Figure 3. The above method works when the light intensity signal is limited to two values, off and on, but in reality it can have complex characteristics where fast and slow changes overlap and can take on intermediate values. When trying to measure the delay time for a fast change, the static method shown in Figure 3 can only determine the value after all fast and slow changes have finished.

It is also necessary to determine measurement parameters that match the video delay time of the object being measured, such as the repetition time for turning light source 1 on and off, and the data recording time for measurement. Furthermore, since the actual light intensity signal is superimposed with noise due to various factors, parameters for removing this noise must be determined. To achieve this, a pre-scan is performed before the actual delay time measurement, in which waveform data is acquired in advance by repeatedly turning the light on and off. Performing this process separately from the process for determining Von and Voff is time-consuming. This concludes the first challenge.

The second issue is that even if the Von and Voff values are obtained appropriately, if the system under test contains repetitive operational elements, errors will occur unless the synchronization relationship with the measurement side is handled appropriately. Specifically, errors of about one video frame may occur. In other words, if the measurement repetition interval is very close to an integer multiple of the measurement operation repetition rate of the system under test, the delay measurement results will fluctuate little from repetition to repetition, and may be biased toward either small or large. The error range is one frame, which cannot be ignored when evaluating low-latency video transmission systems.

An apparatus for driving an actuator installed at a first location on/off with a driving electric signal, sensing and transmitting the output of the actuator in a system to be measured at the first location, converting a physical quantity output from the system to be measured at a second location into an electric signal by a sensor installed at the second location, and measuring a time difference between a change in the driving electric signal and a change in the electric signal from the sensor, comprising:
Prior to measuring the time difference, the driving electric signal is controlled so that the actuator alternates between an off state and an on state;
a first value Voff of the electrical signal from the sensor before the timing of the actuator changing from an off state to an on state;
A second value Von of the electrical signal from the sensor after a predetermined guard time has elapsed after the timing is acquired;
In measuring the time difference, a delay time measurement system is used that is characterized by calculating the delay time from the difference between the timing of the change in the drive electrical signal of the actuator and the timing at which the electrical signal coincides with a threshold value based on a first value Voff of the electrical signal and a second value Von of the electrical signal.

When measuring the time difference, a delay time measurement system is used in which the repetition period when the actuator periodically repeats the off state and the on state is set to a non-integer multiple of the operation repetition period of the system under test or is determined by a random number.

Before measuring the delay time, the off and on states are repeatedly switched on and off, and electrical signals from the sensor before and after the change are obtained to obtain Voff and Von, thereby obtaining the light intensity signal value in use. In other words, values related to the fast signal changes mentioned above are obtained.

Voff and Von can be determined at the same time as the pre-scan, which determines measurement parameters to match the video delay time of the object being measured, reducing complexity and improving the efficiency of pre-measurement preparation work.

In the delay time measurement process after Von and Voff have been properly obtained, even if the system under test has repetitive operational elements, the measurement repetition interval is set to a non-integer multiple of the operational repetition interval of the system under test, or is determined by a random number. This prevents the measurement results from being biased toward short or long values, and by using the average value of the repeatedly measured delay time values, an appropriate numerical value can be obtained.

1 is a general diagram illustrating a delay time measurement system that is the subject of this specification. 2 is a timing chart showing the operation of FIG. 1; 1 is a diagram illustrating conventional calibration data acquisition. 1 is a diagram showing a first embodiment of the present invention; FIG. 10 is an explanatory diagram illustrating the effect of the first embodiment. FIG. 1 is a flowchart of a first embodiment. 1 is a general diagram for measuring delay times of a plurality of physical quantities. 8 is a timing chart showing the operation of FIG. 7. FIG. 10 is a diagram illustrating a second embodiment of the present invention. FIG. 10 is a first explanatory diagram for explaining a delay measurement operation. FIG. 10 is a second explanatory diagram for explaining the delay measurement operation. FIG. 10 is a third explanatory diagram for explaining the delay measurement operation.

The timing of the first embodiment is shown in Figure 4. This shows that light source 1 is turned off (off state) and turned on (on state) in sequence and repeated before measuring the delay time. The value of the trigger signal that drives light source 1 is Vlow when light source 1 is turned off, and Vhigh when light source 1 is turned on. Before the transition from Vlow to Vhigh, light source 1 is off, and the display 13 of the system under test is also off at that time. The level of the light intensity signal from light sensor 2 obtained before the above transition is obtained, and is taken as Voff, the value for the off state.

After the value of the trigger signal that drives the light source 1 transitions from Vlow to Vhigh, the display 13 does not immediately turn on. After the preset guard time Tguard has elapsed, the level of the light intensity signal from the optical sensor 2 is obtained and set to Von, the value for the on state.

The threshold value Vth is determined from Voff and Von. For example, the threshold value Vth is set to (Voff+Von)/2, which is the midpoint (50%) between on and off. As described below, Voff and Von can be weighted to determine the threshold value, or different values can be used for off → on and on → off. When measuring the delay time, the delay time is calculated from the timing of the change in light source 1 and the timing when the light intensity signal crosses this threshold.

Figure 5 is a diagram illustrating one of the effects of the first embodiment. In an actual system under test, changes in the light intensity signal may occur in which slow changes are superimposed on fast changes. This is the situation where, for example, the characteristics of a sensor device or display device change gradually due to temperature changes. Since the changes perceived by the human eye are the fast changes, in this case it is desirable to use the value of the light intensity signal Von after the fast change has ended after Tguard has elapsed. Figure 5 is more desirable than the Von value after a steady state has been reached as in Figure 3. The same is true for Voff, which is not shown in the figure.

Figure 6 shows a flowchart for measuring delay time using this embodiment. First, the parameters are initialized. The parameters here refer to the cycle at which light source 1 is repeatedly blinked and the recording time for acquiring the waveform. Next, light source 1 begins blinking, and after at least one cycle has passed, the output of light sensor 2 when it is off is obtained and designated as Voff. After the time Tguard has elapsed since light source 1 was turned on, the output of light sensor 2 when it is on is obtained and designated as Von. In practice, Voff and Von can be calculated by measuring the output of light sensor 2 continuously as a waveform in a time series, and then processing it in controller 7.

The obtained Von, Voff and waveforms are displayed in an appropriate way, and a human judges whether they are correct or not. Alternatively, an equivalent algorithm can be created and judged automatically. If the changes are as expected, that's fine; if they are unexpected, modify the parameters, the settings of the system being measured, and the positions of light source 1 and optical sensor 2 in the measurement system, and repeat the process. If there are no problems, perform a delay time measurement as shown in Figure 2 and display it as appropriate. When displaying, it is a good idea to display not only the delay time values described below, but also the waveforms at the time of measurement. Furthermore, the operating status of the system can be better understood by simultaneously displaying the raw data waveform and the waveform obtained by moving average on the time axis to remove noise.

The above explanation has been given on how to calculate Voff and Von when the light source 1 changes from off to on. In exactly the same way, when the light source 1 changes from on to off, it is also possible to calculate the light intensity level Von' when it is on and the light intensity level Voff' when it is off. However, it is also possible to simplify this so that Voff' = Voff and Von' = Von.

In Example 1, the measurement target was explained as video delay time. Other measurement targets besides video include sound and haptics. Figure 7 shows a configuration that can simultaneously measure multiple physical quantities.

When targeting audio, a sound source 41 is driven by a sound source driver 44 instead of the light source 1 used in the case of video. The system under test 10 controls whether or not to generate a sound wave with a frequency of, for example, 1000 Hz, according to the input trigger signal. The system under test 10 senses the sound with a microphone 45 and drives a speaker 47 via an audio transmission system 46. The sound from the speaker 47 is sensed by a microphone 42 in the measurement system, and in order to determine its intensity, the signal is subjected to detection processing, such as full-wave rectification and passing through a low-frequency filter, in a processing circuit 43, which then outputs an audio intensity signal to the waveform measurement unit 6.

Haptics can be used when the target physical quantity is vibration or displacement. When the haptics is vibration, it is similar to audio above, and here the target physical quantity is exemplified as displacement. In the case of displacement, instead of light source 1 as in the case of video, a displacement source 51 is driven by displacement source driver 54. The system under test 10 senses the displacement with displacement sensor 55 and drives displacement source 57 via displacement transmission system 56. The displacement from displacement source 57 is sensed by displacement sensor 52 of the measurement system and connected to waveform measurement unit 6 as a displacement intensity signal.

The time chart for measuring delay time is shown in Figure 8. For video, the delay time from lights off to lights on is represented as Tv1, and the delay from lights on to lights off is represented as Tv2. Similarly, for audio, the delay time from no audio to audio is represented as Ts1, and the delay from audio to no audio is represented as Ts2. Furthermore, for haptics, the delay time from no displacement to displacement is represented as Th1, and the delay from displacement to no displacement is represented as Th2.

The above is a general explanatory diagram, but based on this, Figure 9 shows a time chart for a second embodiment of the present invention. Video has already been explained in the first embodiment. For audio, data before and after the trigger signal changes from Vlow to Vhigh is acquired as a waveform to obtain Voff', the value when there is no sound, and Von', the value when there is sound. The guard time in this case is Tguard' for audio. The same is true for haptics, where data before and after the trigger signal changes from Vlow to Vhigh is acquired as a waveform to obtain Voff", the value when there is no displacement, and Von", the value when there is displacement. The guard time in this case is Tguard" for displacement.

As a variation, depending on the system under test 10, the input and output of the transmission system 59 of the system under test may not necessarily be the same. For example, the system under test may input a displacement and output light, i.e., pressing a switch causes a lamp to light up. Another example is a system that inputs video and issues an audible warning. In this case, the present invention can also be applied to find each Voff and each Von. Thus, the present invention is applicable even when what the system under test senses and what it outputs are different.

We will now explain the details of measuring delay time using Voff and Von. When measuring delay time, measurements are made repeatedly using the same parameters, and the obtained data is statistically processed to quantify the delay time. Care must be taken when handling measurement results, especially when the object being measured has periodic operating elements. As a concrete example, in a video transmission system, a video camera 11 typically captures and transmits a fixed number of frames of data per second. For example, it captures images at 60 frames per second, but it is common to scan the target screen sequentially rather than capturing them all at once. A schematic diagram is shown in Figure 10. This is what is known as raster scanning, and the scanning begins at the top left of Figure 10. Line 1 is scanned horizontally, and when it reaches the right edge, line 2 is scanned horizontally from the left edge. This scanning is repeated, and when it reaches the bottom right, it goes back to the top right and repeats 60 times per second.

At this time, there is a difference between the timing when light source 1 lights up and the timing when it is scanned; at the earliest, the light is sampled immediately, but at worst, it is sampled approximately one frame later. Because the system under test and the delay time measurement system described here operate asynchronously, it is not clear whether the error for one frame will appear as an approximately constant value, or whether it will shift with each repetition. Therefore, the repetition period is intentionally shifted so that it is not an integer multiple of the frame rate. The period of reference signal generator 3 is controlled under the control of controller 7.

This is shown in Figure 11. The light intensity signal at (1) changes quickly, as scanning is performed immediately after light source 1 is turned on. The start of scanning after lighting is delayed as shown in (2), (3), (4), and so on. The state at (N) is the state where the time from lighting to scanning is the slowest, and then it returns to (1). As a numerical example, let's assume that the imaging frame rate of the system under test is 60 frames per second, meaning that the frame repetition time is 1/60 second, and the delay time measurement is repeated every 1.01 seconds. From the state of the first delay time measurement, the timing of the second delay measurement is 1.01 ÷ (1/60) = 60.6 → 0.6 frame offset. The timing of the third delay measurement is 2.02 ÷ (1/60) = 61.2 → 0.2 frame offset. The timing of the fourth delay measurement is 3.03 ÷ (1/60) = 61.8 → 0.8 frame offset. The timing of the fifth delay measurement is 4.04 ÷ (1/60) = 62.4 → 0.4 frame offset. The timing of the sixth delay measurement is 5.05 ÷ (1/60) = 63.0 → 0.0 frame offset, and so on. It can be seen that this systematic frame offset occurs evenly between 0 and less than 1. Furthermore, if fractional parts are added to the repeated delay time measurements, an even finer distribution can be obtained.

By shifting the repetition period in this way and intentionally varying the timing of the change, it is possible to eliminate bias in the influence of the frame rate. If the shift is not intentionally done, the delay time measurement results may happen to be biased towards the faster or slower side.

In the above, we used a non-integer multiple to avoid the delay time measurement results being biased towards the faster or slower side, but it is also possible to add dither to the repetition period so that the next repetition time varies randomly. If you repeat the process a sufficient number of times, you can avoid the delay time measurement results being biased towards the faster or slower side.

When displaying the results, statistical processing is performed on the repeated results to display, for example, the average, minimum, and maximum values. The minimum value corresponds to (1) in Figure 11, and the maximum value corresponds to (N) in Figure 11. In other words, in the time axis direction (horizontal direction), three numbers are obtained when the light is off → on, and three numbers are obtained when the light is on → off.

Furthermore, three values are possible in the amplitude direction (vertical direction). When Voff is 0% and Von is 100%, one way to think about it is to set the threshold Vth for determining a change to, for example, the median value, 50%. However, if even the slightest change is to be recognized as a change, then it would be better to set Vth = 20% when the light goes from off to on, and Vth = 80% when the light goes from on to off. Conversely, if a change is only recognized when there is a complete change, then Vth = 80% when the light goes from off to on, and Vth = 20% when the light goes from on to off. Furthermore, larger and smaller Vths are also possible, but these values are given as examples because noise from various factors may be superimposed.

When displaying the results, for example, the 18 numerical values shown in Figure 12 are displayed on a display attached to the controller 7. The nine circular points represent when the light is off and when it is on, and the nine triangular points represent when it is on and when it is off. Each of the nine points is one of three types: {Vth=20%, Vth=50%, Vth=80%} x three types: {minimum, average, maximum}. The measurer can decide which of the displayed numbers to use depending on the application and how the changes are recognized. Because there are so many numbers, it is effective to make each of the nine points easier to see by instructing the measurer to display them larger, using a different text color, or changing the background color.

The symbols in FIGS. 1 to 12 are as follows:
1 --- Light source
2 --- Optical sensor
3 --- Reference signal generator
4 --- Light source driver
5 --- Trigger occurs
6 --- Waveform measurement section
7 --- Controller
10 --- System under test
11 --- Video camera
12 --- Video transmission system
13 --- Display

41 --- Sound source
42 --- Microphone
43 --- Processing circuit
44 --- Sound source drive unit
45 --- Microphone of the system under test
46 --- Audio transmission system
47 --- Speaker of the system under test
51 --- Displacement source
52 --- Displacement sensor
54 --- Displacement source drive unit
55 --- Displacement sensor of the system under test
56 --- Displacement transmission system
57 --- Displacement source of the system under test
59 --- Transmission system of the system under test

Claims (7)

An apparatus for driving an actuator installed at a first location on/off with a driving electric signal, sensing and transmitting the output of the actuator in a system to be measured at the first location, converting a physical quantity output from the system to be measured at a second location into an electric signal by a sensor installed at the second location, and measuring a time difference between a change in the driving electric signal and a change in the electric signal from the sensor, comprising:
Prior to measuring the time difference, the driving electric signal is controlled so that the actuator alternates between an off state and an on state;
a first value of the electrical signal from the sensor before the timing of the actuator changing between an off state and an on state;
and acquiring a second value of the electrical signal from the sensor after a predetermined guard time has elapsed after the timing.
In measuring the time difference, the delay time is calculated from the difference between the timing of the change in the actuator's drive electric signal and the timing at which the electric signal matches a threshold value based on a first value of the electric signal and a second value of the electric signal.
The delay time measurement system according to claim 1, wherein the physical quantity is an image.
The delay time measurement system according to claim 1, wherein the physical quantity is sound.
The delay time measurement system according to claim 1, wherein the physical quantity is haptics.
The delay time measurement system according to claim 1, wherein the physical quantities are at least two of the following: video, sound, and haptics, and what is sensed by the system under test is different from what is output by the system under test.
The delay time measurement system according to claim 1, wherein the repetition period when the actuator periodically alternates between the off state and the on state during measurement of the time difference is set to a non-integer multiple of the repetition period of the operation of the system under test.
The delay time measurement system according to claim 1, characterized in that when measuring the time difference, the repetition period when the actuator periodically switches between the off state and the on state is determined by random numbers so that it is not a constant value.