JP7063714B2 - Position estimation device, air conditioning system, position estimation method and program - Google Patents

Description

The present invention relates to a position estimation device, an air conditioning system, a position estimation method and a program.

When air-conditioning a space such as an office, it is often controlled so that the room temperature of the space becomes uniform. On the other hand, for example, there is provided a method of controlling air conditioning such as room temperature in an area where a user exists. When performing such control, the air conditioning system needs to accurately estimate the position where the user is present. For example, microphones that detect sound are attached to four locations on the ceiling, sound is output from a smartphone or the like owned by the user, and based on the arrival time difference of the sound, a method such as TDOA (Time Difference of Arrival) is used. A method of estimating the position where the user exists can be considered.

In Patent Document 1, in a space where a plurality of users exist, the position of each individual can be specified based on the radio wave strength of the environment setting terminal held by the user and the TDOA measurement, and each air conditioning request can be made. An air conditioning control device that performs control that satisfies the limit is disclosed.

Japanese Patent No. 4867836

However, the waveform of the sound emitted by the user using a smartphone or the like has a property that the amplitude gradually increases. Therefore, on the sensor side, the arrival time of the sound differs depending on the magnitude of the amplitude at which the sound is determined to be detected. In a method such as TDOA, position estimation is performed based on the difference in arrival time of sounds at a plurality of positions, but if the arrival time of sounds cannot be accurately detected, the accuracy of position estimation may decrease.

Therefore, an object of the present invention is to provide a position estimation device, an air conditioning system, a position estimation method and a program capable of solving the above-mentioned problems.

According to one aspect of the present invention, the position estimation device acquires the waveform signal of the sound detected by the sensor, estimates the detection end time of the sound based on the waveform signal, and is based on the detection end time. The output position of the sound is estimated based on the time difference between the arrival time estimation unit that estimates the arrival time of the sound to the sensor and the arrival time of the sound in the plurality of sensors estimated by the arrival time estimation unit. The arrival time estimation unit estimates the detection end time based on an approximate curve showing a change in the amplitude of the waveform signal over time, and the sound 1 from the detection end time. The time retroactive by the time obtained by multiplying the time of the cycle and the number of output cycles indicating how many cycles of waves are included in the output sound is estimated as the arrival time of the sound.

According to one aspect of the present invention, the position estimation device acquires the waveform signal of the sound detected by the sensor, estimates the detection start time and the detection end time of the sound based on the waveform signal, and the detection start time. And based on the time difference between the arrival time estimation unit that estimates the arrival time of the sound to the sensor based on the detection end time and the arrival time of the sound in the plurality of sensors estimated by the arrival time estimation unit. The sound output position is provided with a position estimation unit, and the arrival time estimation unit estimates the detection start time and the detection end time based on an approximate curve showing a change in the amplitude of the waveform signal over time. Then, the time and the time retroactive from the detection end time by the time obtained by multiplying the time of one cycle of the sound and the number of output cycles indicating how many cycles of waves are included in the output sound and the above. The arrival time of the sound is estimated by a weighted average with the detection start time.

According to one aspect of the present invention, the position estimation device acquires the waveform signal of the sound detected by the sensor, estimates the detection end time of the sound based on the waveform signal, and based on the detection end time, A position for estimating the output position of the sound based on the time difference between the arrival time estimation unit that estimates the arrival time of the sound to the sensor and the arrival time of the sound in the plurality of sensors estimated by the arrival time estimation unit. The arrival time estimation unit includes an estimation unit, and the arrival time estimation unit estimates the last time when the magnitude of the amplitude of the waveform signal exceeds a predetermined threshold value as the detection end time, and from the detection end time, 1 of the sound. The time retroactive by the time obtained by multiplying the time of the cycle and the number of output cycles indicating how many cycles of waves are included in the output sound is estimated as the arrival time of the sound.

According to one aspect of the present invention, the position estimation device acquires the waveform signal of the sound detected by the sensor, estimates the detection start time and the detection end time of the sound based on the waveform signal, and the detection start time. And based on the time difference between the arrival time estimation unit that estimates the arrival time of the sound to the sensor based on the detection end time and the arrival time of the sound in the plurality of sensors estimated by the arrival time estimation unit. The arrival time estimation unit includes a position estimation unit that estimates the output position of the sound, and the arrival time estimation unit sets a time when the number of times the magnitude of the amplitude of the waveform signal exceeds a predetermined threshold number exceeds a predetermined number of times. The last time when the amplitude exceeds the threshold value is estimated as the detection end time, and the detection end time is multiplied by the sound cycle and the number of output sound cycles. The arrival time of the sound is estimated from the weighted average of the time retroactive by the time and the detection start time.

According to one aspect of the present invention, the position estimation device further includes a communication unit that outputs an error signal when the estimation of the arrival time of the sound or the estimation of the output position of the sound fails.

According to one aspect of the present invention, the position estimation device includes a plurality of sensors for detecting sound, a filter for extracting a waveform signal having a frequency in a predetermined range from the waveform signals of the sound detected by the sensor, and the above-mentioned. Further, a converter for converting the waveform signal of the sound output by the filter into a digital signal is provided.

According to one aspect of the present invention, the position estimation device includes a plurality of sensors for detecting sound, a filter for extracting a waveform signal having a frequency in a predetermined range from the waveform signals of the sound detected by the sensors, and the above-mentioned. Further provided is a comparator that outputs whether or not the amplitude of the waveform signal of the sound output by the filter is larger than a predetermined threshold value.

According to one aspect of the present invention, the air conditioning system includes the position estimation device according to any one of the above, and air-conditions the sound output position estimated by the position estimation device.

According to one aspect of the present invention, the position estimation method acquires the waveform signal of the sound detected by the sensor, estimates the detection end time of the sound based on the waveform signal, and based on the detection end time, A step of estimating the arrival time of the sound, and a step of estimating the output position of the sound based on the time difference between the arrival times of the sound in the plurality of sensors. Then, the detection end time is estimated based on an approximate curve showing a change in the amplitude of the waveform signal over time, and the sound cycle is multiplied by the number of output sound cycles from the detection end time. The time retroactive by the time obtained is estimated as the arrival time of the sound.

According to one aspect of the present invention, the position estimation method acquires the waveform signal of the sound detected by the sensor, estimates the detection start time and the detection end time of the sound based on the waveform signal, and the detection start time. And a step of estimating the arrival time of the sound to the sensor based on the detection end time, and a step of estimating the output position of the sound based on the time difference of the arrival time of the sound in the plurality of the sensors. In the step of estimating the arrival time of the sound, the detection start time and the detection end time are estimated based on the approximate curve showing the temporal change of the amplitude of the waveform signal, and from the detection end time, By a weighted average of the time retroactive by the time obtained by multiplying the time of one cycle of the sound and the number of output cycles indicating how many cycles of the output wave are included, and the detection start time. Estimate the arrival time of the sound.

According to one aspect of the present invention, the position estimation method acquires the waveform signal of the sound detected by the sensor, estimates the detection end time of the sound based on the waveform signal, and based on the detection end time, A step of estimating the arrival time of the sound, and a step of estimating the output position of the sound based on the time difference between the arrival times of the sound in the plurality of sensors. Then, the last time when the magnitude of the amplitude of the waveform signal exceeds a predetermined threshold is estimated as the detection end time, and from the detection end time, the time of one cycle of the sound and what is output to the sound. The time retroactive by the time obtained by multiplying the number of output cycles indicating whether or not the wave for the cycle is included is estimated as the arrival time of the sound.

According to one aspect of the present invention, the position estimation method acquires the waveform signal of the sound detected by the sensor, estimates the detection start time and the detection end time of the sound based on the waveform signal, and the detection start time. And a step of estimating the arrival time of the sound to the sensor based on the detection end time, and a step of estimating the output position of the sound based on the time difference of the arrival time of the sound in the plurality of the sensors. The time when the magnitude of the amplitude of the waveform signal exceeds a predetermined threshold is estimated as the detection start time, and the last time when the amplitude exceeds the threshold is the detection end time. The arrival of the sound by a weighted average of the time retroactive by the time obtained by multiplying the sound cycle and the number of output sound cycles from the detection end time and the detection start time. Estimate the time.

According to one aspect of the present invention, the program acquires the waveform signal of the sound detected by the sensor, estimates the detection end time of the sound based on the waveform signal, and sets the detection end time to the detection end time. Based on this, it functions as a means for estimating the arrival time of the sound and a means for estimating the output position of the sound based on the time difference between the arrival times of the sound in the plurality of sensors, and estimates the arrival time of the sound. The means estimates the detection end time based on an approximate curve showing a change in the amplitude of the waveform signal over time, and from the detection end time, the sound cycle and the number of output sound cycles are calculated. The time retroactive by the time obtained by multiplying is estimated as the arrival time of the sound.
According to one aspect of the present invention, the program acquires the waveform signal of the sound detected by the sensor, estimates the detection start time and the detection end time of the sound based on the waveform signal, and starts the detection. As a means for estimating the arrival time of the sound to the sensor based on the time and the detection end time, and as a means for estimating the output position of the sound based on the time difference between the arrival times of the sound in the plurality of the sensors. The means for functioning and estimating the arrival time of the sound estimates the detection start time and the detection end time based on an approximate curve showing a change in the amplitude of the waveform signal over time, and from the detection end time, the said The weighted average of the time retroactive by the time obtained by multiplying the time of one cycle of the sound and the number of output cycles indicating how many cycles of the output wave are included and the detection start time. Estimate the arrival time of the sound.
According to one aspect of the present invention, the program acquires the waveform signal of the sound detected by the sensor, estimates the detection end time based on the waveform signal, and the sound is based on the detection end time. The means for estimating the arrival time of the sound, which functions as a means for estimating the arrival time of the sound and a means for estimating the output position of the sound based on the time difference between the arrival times of the sound in the plurality of sensors, is the waveform. The last time when the magnitude of the signal amplitude exceeds a predetermined threshold is estimated as the detection end time, and from the detection end time, the time of one cycle of the sound and the wave of several cycles to the output sound. The time retroactive by the time obtained by multiplying the number of output cycles indicating whether or not is included is estimated as the arrival time of the sound.
According to one aspect of the present invention, the program acquires the waveform signal of the sound detected by the sensor, estimates the detection start time and the detection end time of the sound based on the waveform signal, and starts the detection. As a means for estimating the arrival time of the sound to the sensor based on the time and the detection end time, and as a means for estimating the output position of the sound based on the time difference between the arrival times of the sound in the plurality of the sensors. The means for functioning and estimating the arrival time of the sound estimates the time when the magnitude of the amplitude of the waveform signal exceeds a predetermined threshold number of times exceeds a predetermined number of times as the detection start time, and the amplitude is the threshold value. The last time exceeding the above is estimated as the detection end time, and the time and the detection start are advanced by the time obtained by multiplying the sound cycle and the number of output sound cycles from the detection end time. The arrival time of the sound is estimated by a weighted average with the time.

According to the present invention, the position of the sound source can be estimated accurately.

It is a figure which shows an example of the air-conditioning system in 1st Embodiment of this invention. It is a figure which shows the arrangement example of the microphone in the 1st Embodiment of this invention. It is a figure explaining the sound signal in the 1st Embodiment of this invention. It is a figure explaining the method of estimating the arrival time of a sound in the 1st Embodiment of this invention. It is 1st flowchart of the position estimation process of a sound source in 1st Embodiment of this invention. It is a second flowchart of the position estimation process of the sound source in the 1st Embodiment of this invention. It is a 3rd flowchart of the position estimation process of a sound source in 1st Embodiment of this invention. It is a flowchart which shows an example of the air-conditioning control in 1st Embodiment of this invention. It is a figure which shows an example of the air-conditioning system in the 2nd Embodiment of this invention. It is a figure which shows an example of the hardware composition of the position estimation apparatus in each embodiment of this invention.

<First Embodiment>
Hereinafter, the air conditioning system according to the first embodiment of the present invention will be described with reference to FIGS. 1 to 8.
FIG. 1 is a diagram showing an example of an air conditioning system according to the first embodiment of the present invention.
As shown in the figure, the air conditioning system includes an air conditioner 10, a terminal device 40, and microphones M1 to M4. The air conditioner 10 includes a control device 20. The control device 20 controls the operation of the air conditioner 10. The control device 20 has a function of controlling the air conditioner 10 so as to perform air conditioning in a part of the space w. The control device 20 includes a position estimation device 30. The position estimation device 30 estimates the position where the sound s is emitted based on the sound s emitted by the terminal device 40. The control device 20 performs air conditioning control for, for example, the position estimated by the position estimation device 30.

The terminal device 40 outputs a sound s (ultrasonic wave) having a predetermined frequency for the purpose of notifying the air conditioner 10 of the position of the user. The terminal device 40 is, for example, an information processing device such as a smartphone. As the frequency of the sound s output by the terminal device 40, a frequency having a low noise level is selected so as not to be mixed with various originally existing noises. For example, a sound of 10 kHz or less is used as an audible sound, a wind noise associated with an air conditioner operation, or the like. Further, the upper limit of the frequency that can be output by a general smartphone is about 20 kHz. Therefore, it is desirable to select the frequency of the sound s from the range of 10 to 20 kHz. Further, the frequency of the sound s is preferably set so as to avoid the switching frequency (for example, around 17 kHz) of the inverter included in the air conditioner 10. Further, since the audible range may be unpleasant for humans, the frequency of the sound s may be set while avoiding the audible range (in the range of 10 to 20 kHz, the lower the frequency, the easier it is to hear). .. Further, there is a limit to the output near 20 kHz, which is the upper limit, and the amplitude tends to be attenuated as the frequency increases. Therefore, for example, it is preferable to avoid setting around 20 kHz. In consideration of these conditions, the frequency of the sound s is set to, for example, around 15 kHz.
The terminal device 40 and the position estimation device 30 are configured to be communicable by communication means such as wireless LAN, Bluetooth (registered trademark), and ultrasonic communication.
The microphones M1 to M4 are sensors that detect the sound s emitted by the terminal device 40.

The position estimation device 30 estimates the position of the sound source (terminal device 40) based on the difference in time when each of the microphones M1 to M4 detects the sound s emitted by the terminal device 40. The control device 20 performs air conditioning control for the position estimated by the position estimation device 30.

Next, the function of the position estimation device 30 will be described. The position estimation device 30 includes an amplifier 31, a filter 32, an A / D converter 33, and a microcomputer 34.
The amplifier 31 acquires an analog sound signal converted into an electric signal by detecting the sound s by the microphones M1 to M4 for each of the microphones M1 to M4, and amplifies the respective amplitudes. The amplifier 31 outputs the amplified analog sound signal (analog sound signal for each of the microphones M1 to M4) to the filter 32.
The filter 32 attenuates unnecessary high-frequency components and low-frequency components of the analog sound signal acquired from the amplifier 31, and outputs an analog sound signal in a predetermined frequency band to the A / D converter 33. The filter 32 is configured to pass the frequency of the sound s output by the terminal device 40 (for example, around 15 kHz).
The A / D converter 33 converts the analog sound signal acquired from the filter 32 into a digital sound signal. The A / D converter 33 outputs the converted digital sound signal to the microcomputer 34.

The microcomputer 34 is a computer provided with a CPU (central processing unit). The microcomputer 34 includes an arrival time estimation unit 35, a position estimation unit 36, and a communication unit 37.
The arrival time estimation unit 35 estimates the time (arrival time) at which each of the microphones M1 to M4 detects the sound s.
The position estimation unit 36 refers to the time when the reference microphone M1 detects the sound s and the other three microphones M2 to M4 use the sound s as the reference from one of the microphones M1 to M4 (for example, the microphone M1). The difference between the time when the sound was detected, that is, the time difference between the time when the sound s reaches the microphone M1 and the time when the sound s reaches the microphones M2 to M4 (arrival time difference) is calculated. The position estimation unit 36 estimates the position where the sound s is emitted (the position where the user exists) based on the calculated arrival time difference.
The communication unit 37 communicates with the terminal device 40 and transmits the result of success or failure of the arrival time estimation process and the position estimation process.

Next, an example of the position estimation method in the present embodiment will be described with reference to FIG.
FIG. 2 is a diagram showing an example of arrangement of microphones in the first embodiment of the present invention.
FIG. 2 shows an example of arrangement of microphones M1 to M4. The microphones M1 to M4 are arranged on the ceiling of the space w so that, for example, each of the microphones M1 to M4 forms the apex of a square having the center of the outlet of the air conditioner 10 as the center of gravity.
The position estimation unit 36 estimates the position of the terminal device 40 based on the time difference (arrival time difference) when the microphones M1 to M4 detect the sound s. The position can be estimated by using a known method such as TDOA (Time Difference Of Arrival). For example, the coordinates of the microphones M1 to M4 are set to (A1, B1, C1), (A2, B2, C2), (A3, B3, C3), (A4, B4, C4), respectively, and the coordinates of the position of the terminal device 40. Is (x, y, z), and the difference between the arrival time of the sound s to the microphone M1 and the arrival time of the sound s to the microphones M1 to M4 is t1, t2, t3, t4, respectively, and the terminal device 40 and the microphone M1 If the distance of is d and the speed of sound is c, the following simultaneous equations can be obtained.
(X-A1) ² + (y-B1) ² + (z-C1) ^2- (d + c · t1) ² = 0
(X-A2) ² + (y-B2) ² + (z-C2) ^2- (d + c · t2) ² = 0
(X-A3) ² + (y-B3) ² + (z-C3) ^2- (d + c · t3) ² = 0
(X-A4) ² + (y-B4) ² + (z-C4) ^2- (d + c · t4) ² = 0
According to the TDOA, the position of the terminal device 40 can be calculated by solving the above equation using an approximate calculation such as the Newton-Rapson method. In this way, in order to estimate the sound source position, it is necessary to accurately calculate the arrival time difference between the microphones. For that purpose, it is necessary to be able to detect the exact arrival time of the microphones M1 to M4. Here, the nature of the sound s output by the terminal device 40 such as a smartphone will be described.

FIG. 3 is a diagram illustrating a sound signal according to the first embodiment of the present invention.
FIG. 3A shows an example of an analog sound signal emitted by the terminal device 40. As shown in FIG. 3A, the amplitude of the sound pressure of the analog sound signal output by a smartphone or the like varies. Therefore, in order to detect the arrival of the sound s, a method is conceivable in which a threshold value is set for the sound pressure, and when the threshold value is exceeded a predetermined number of times, the time is determined as the arrival time of the sound s.
FIG. 3B shows a change over time in the magnitude (amplitude) of the sound pressure of the sound s detected by the microphone M1 or the like. FIG. 3B shows that the sound s output by a smartphone or the like gradually increases in the amplitude of the sound pressure from the start of the output, and the shape of the waveform becomes a trumpet shape. When the waveform changes in this way, the arrival time changes depending on how the threshold value of the sound pressure for determining the arrival of the sound s is provided, and the determination of the detection of the sound s becomes unclear. Further, since the rising method (inclination) of the sound pressure and the peak of the sound pressure (peak of the amplitude) differ depending on the model of the smartphone and the frequency of the sound s, it becomes impossible to determine the arrival of the sound s by one threshold value. Therefore, in the present embodiment, the arrival time of the sound s is estimated by utilizing the fact that the rising shape of the waveform of the sound s is a trumpet shape. Next, a method of estimating the arrival time of the present embodiment will be described with reference to FIG.

(Estimation at rising edge)
FIG. 4 is a diagram illustrating a method of estimating the arrival time of sound in the first embodiment of the present invention.
First, the arrival time estimation unit 35 calculates an approximate curve with respect to the change over time in the amplitude of the waveform signal of the sound s. Next, the approximate curve is extrapolated to calculate the intersection with the waveform signal. The broken line graph L1 in FIG. 4 is an example of a curve obtained by extrapolating the approximate curve connecting the peak positions of the sound pressure in the waveform signal of the sound s in the direction retroactively. The intersection (time t1) thus obtained is set as the detection start time of the sound s. The arrival time estimation unit 35 may estimate the time T1 as the arrival time of the sound s, or may estimate the time T2 after a predetermined time Δt from the time T1 as the arrival time of the sound s. The approximate curve may be calculated by fitting, for example, based on the arctangent function and the sigmoid function. Alternatively, in order to reduce the calculation load, approximation may be performed by a linear function or a quadratic function.

(Estimation at the fall)
Alternatively, the arrival time of the sound s may be estimated by paying attention to the detection end time of the sound s. In this case, it is determined in advance that the frequency f0 of the sound s output by the terminal device 40 and the sound s are output for N cycles per output. Then, for the time zone before the detection end time of the sound s, an approximate curve is calculated with respect to the time-dependent change in the amplitude of the waveform signal of the sound s in the same manner as described above. Next, the approximate curve is extrapolated in the direction in which time elapses, and the intersection with the waveform signal is calculated. The broken line graph L2 in FIG. 4 is a curve obtained by extrapolating an approximate curve connecting the peak positions of the sound pressure of the sound s in the time series. The intersection (time T3) thus obtained is set as the detection end time of the sound s. It is known that the slope of the approximate curve with respect to the waveform signal near the detection end time of the sound s is approximately Y-axis symmetric with the approximate curve with respect to the waveform signal near the detection start time of the sound s. Therefore, an approximate curve (broken line graph L1) for the rise of the sound s is calculated, and an approximate curve (broken line graph L2) for the waveform signal near the detection end time of the sound s is calculated using this approximate curve. Is also good. Alternatively, the approximate curve (broken line graph L2) may be calculated by fitting the waveform signal near the detection end time of the sound s.
When the arrival time estimation unit 35 calculates the time T2, the time TL from the start to the end of the output of the sound s is calculated by multiplying the time (1 / f0) of one cycle by N of the predetermined number of output cycles. Next, the arrival time estimation unit 35 estimates the time T4, which is retroactive by the time TL from the time T3, as the arrival time of the sound s.

(Estimation at rising and falling)
Alternatively, it may be estimated as the arrival time of the sound s by the weighted average of the time T1 or the time T2 obtained as described above and the time T4. The arrival time of the sound s can be calculated by, for example, the following formula.
Arrival time of sound s = α × time T1 + (1-α) × time T4
In the above, the calculation of the approximate curve for the waveform signal and the estimation of the arrival time of the sound s based on the approximate curve have been described by taking the waveform of the analog sound signal of the sound s as an example. In the configuration shown in FIG. 1, the arrival time estimation unit 35 stands up from the start of arrival of the sound s and stands at the end of the arrival of the sound s with respect to the digital sound signal of the sound s output by the A / D converter 33. An approximate curve showing the shape (inclination) of the descent is calculated, and the arrival time of the sound s is estimated. The estimation method is the same as the processing content described using the example of the analog sound signal of FIG.

FIG. 5 is a first flowchart of the sound source position estimation process according to the first embodiment of the present invention.
With reference to FIG. 5, the flow of the position estimation process using the waveform signal at the rising edge of the sound s will be described. As a premise, it is assumed that the position estimation unit 36 stores the position information of the microphones M1 to M4. Further, the frequency f0 of the sound s output by the terminal device 40 is defined, and the filter 32 is configured to pass through a frequency band in a predetermined range around the frequency f0.
First, the user operates the terminal device 40 to start outputting the sound s having a predetermined frequency. The microphones M1 to M4 detect the sound s (step S11). When the microphones M1 to M4 detect the sound s, they output an analog sound signal of the sound s to the amplifier 31. The amplifier 31 amplifies the analog sound signals of the sounds s output by the microphones M1 to M4, and outputs them to the filter 32. The filter 32 extracts the vicinity of a predetermined frequency from the four analog sound signals after amplification and outputs them to the A / D converter 33. The A / D converter 33 converts each of the four analog sound signals output by the filter 32 into a digital sound signal, and outputs the digital sound signal to the microcomputer 34.

In the microcomputer 34, the arrival time estimation unit 35 acquires four digital sound signals and estimates each detection start time from the rising shape of the waveform of the digital sound signal (step S12). Specifically, as described in FIG. 4, the arrival time estimation unit 35 sets points in the time series of the peak value of the sound pressure of the digital sound signal as an arctangent function, a sigmoid function, a linear function, a quadratic function, and the like. Approximate with to calculate the approximate curve. Then, the arrival time estimation unit 35 extrapolates the calculated approximate curve in the direction going back in time to calculate the intersection point (detection start time) of the sound s with the digital sound signal. The arrival time estimation unit 35 estimates the detection start time for each of the four digital sound signals.

Next, the arrival time estimation unit 35 estimates the arrival time of the sound s to each of the microphones M1 to M4 (step S13). For example, the arrival time estimation unit 35 estimates the detection start time (time T1 in FIG. 4) as the arrival time of the sound s. The arrival time estimation unit 35 outputs the estimated arrival times (4) of the sounds s to the microphones M1 to M4 to the position estimation unit 36.
The position estimation unit 36 is, for example, from the arrival time of the sound s to the microphones M1 to M4, for example, the arrival time difference of the sound s to the microphone M1 and the microphone M2, the arrival time difference of the sound s to the microphone M1 and the microphone M3, and the microphone. The difference in arrival time of the sound s to M1 and the microphone M4 is calculated, and the position of the sound source (terminal device 40) is estimated by TDOA (step S14).

FIG. 6 is a second flowchart of the sound source position estimation process according to the first embodiment of the present invention.
With reference to FIG. 6, the flow of the position estimation process using the waveform signal at the falling edge of the sound s will be described. In addition to the premise of FIG. 5, it is assumed that the terminal device 40 outputs sound s for N cycles at one output, and that information is shared between the terminal device 40 and the microcomputer 34. The description of the same processing as that described with reference to FIG. 5 will be omitted.
First, the user operates the terminal device 40 to start outputting the sound s having a predetermined frequency. The microphones M1 to M4 detect the sound s (step S11). The analog sound signal of the sound s detected by each of the microphones M1 to M4 passes through the amplifier 31 and the filter 32, and is converted into a digital sound signal by the A / D converter 33. The arrival time estimation unit 35 of the microcomputer 34 acquires four digital sound signals.

The arrival time estimation unit 35 estimates the detection end time from the shape of the falling edge of the waveform of the digital sound signal (step S12a). Specifically, as described in FIG. 4, the arrival time estimation unit 35 approximates the time-series points of the peak value of the sound pressure near the end of arrival of the digital sound signal by a linear function, a quadratic function, or the like. To calculate the approximate curve. Then, the arrival time estimation unit 35 extrapolates the calculated approximate curve in the direction in which time elapses, and calculates the intersection point (detection end time) of the sound s with the digital sound signal. The arrival time estimation unit 35 estimates the detection end time of the sound s for each of the four digital sound signals.

Next, the arrival time estimation unit 35 estimates the arrival time of the sound s retroactively from the detection end time of the sound s to each of the microphones M1 to M4 (step S13a). Specifically, the arrival time estimation unit 35 estimates the arrival time of the sound s as the time obtained by subtracting the time obtained by N / f0 from the detection end time. The arrival time estimation unit 35 outputs the estimated arrival times (4) of the sounds s to the microphones M1 to M4 to the position estimation unit 36. Next, the position estimation unit 36 estimates the position of the sound source (terminal device 40) by TDOA (step S14).

FIG. 7 is a third flowchart of the sound source position estimation process according to the first embodiment of the present invention.
With reference to FIG. 7, the flow of the position estimation process using the waveform signals at the rising and falling edges of the sound s will be described. The preconditions are the same as those described in FIG. The same processing as that described with reference to FIGS. 5 and 6 will be omitted.
First, the user operates the terminal device 40 to start outputting the sound s having a predetermined frequency. The microphones M1 to M4 detect the sound s (step S11). The analog sound signal of the sound s detected by each of the microphones M1 to M4 passes through the amplifier 31 and the filter 32, and is converted into a digital sound signal by the A / D converter 33. The arrival time estimation unit 35 of the microcomputer 34 acquires four digital sound signals.

The arrival time estimation unit 35 estimates the detection start time from the rising shape of the waveform of the digital sound signal (step S12). Next, the arrival time estimation unit 35 estimates the arrival time (first arrival time) of the sound s to each of the microphones M1 to M4 (step S13).

Next, the arrival time estimation unit 35 estimates the detection end time from the shape of the falling edge of the waveform of the digital sound signal (step S12a). Next, the arrival time estimation unit 35 estimates the arrival time (second arrival time) of the sound s by N / f0 from the detection end time of the sound s to each of the microphones M1 to M4 (step S13a).

Next, the arrival time estimation unit 35 estimates the arrival time from the first arrival time and the second arrival time (step S13b). For example, the arrival time estimation unit 35 may use the average of the first arrival time and the second arrival time of the microphone M1 as the estimated value of the arrival time of the sound s to the microphone M1. Further, for example, the arrival time estimation unit 35 sets a weighted average of the first arrival time and the second arrival time of the microphone M1 (weighted coefficients for each are predetermined) to the microphone M1. It may be an estimated value of the arrival time of s. Similarly, the arrival time estimation unit 35 estimates the arrival time of the sound s to the microphones M2 to M4 based on the first arrival time and the second arrival time, respectively. The arrival time estimation unit 35 outputs the estimated values of the arrival times (four) of the sounds s to the microphones M1 to M4 to the position estimation unit 36.
Next, the position estimation unit 36 estimates the position of the sound source (terminal device 40) by TDOA (step S14).

Next, the air conditioning control based on the position estimation by the position estimation device 30 will be described.
FIG. 8 is a flowchart showing an example of air conditioning control according to the first embodiment of the present invention.
First, the user operates the terminal device 40 to output the sound s having a predetermined frequency (step S21). In the air conditioner 10, the microphones M1 to M4 detect the sound s (step S22). The position estimation device 30 acquires the digital sound signal of the sound s output by the microphones M1 to M4, estimates the arrival time, and estimates the sound source position. For example, when the sound reception intensity is less than the threshold value, or when the sound for N cycles cannot be acquired, the arrival time estimation unit 35 of the position estimation device 30 determines that reception has failed (step S23; No) and communicates. The unit 37 transmits an error signal to the terminal device 40 (step S24). On the other hand, when the sound s can be normally received (step S23; Yes), the communication unit 37 transmits a signal indicating successful reception to the terminal device 40. The terminal device 40 notifies the user of the success of reception by displaying a message notifying the success of reception on the display or making a sound. Alternatively, the air conditioner 10 may make a sound or turn on the LED to notify the user of the success of reception. If the sound s is successfully received, the position estimation device 30 estimates the position of the sound source (step S25). The estimation method is as described with reference to FIGS. 5 to 7.

When the position estimation process is successful (step S26; Yes), the communication unit 37 transmits a signal indicating success in the position estimation to the terminal device 40. The terminal device 40 displays a message notifying the success of the position estimation on the display or makes a sound. Alternatively, the air conditioner 10 may make a sound or turn on the LED to notify the user of the success of the position estimation. Further, when the position estimation device 30 succeeds in estimating the position, the control device 20 performs air conditioning control for the estimated position of the sound source (step S28). For example, the control device 20 controls the louver (not shown) of the air conditioner 10 to direct the wind direction to the estimated sound source position.

On the other hand, when the position estimation process fails (step S26; No), the position estimation device 30 transmits an error signal to the terminal device 40 (step S27).
When an error signal is received in steps S24 and S27, the terminal device 40 displays a message on a display or the like, sounds an audible sound notifying that reception or position estimation has failed, or turns on an LED lamp. And notify the user of the error. Upon confirming these notifications, the user operates the terminal device 40 again to output the sound s (step S21). After that, the above process is repeated until the position estimation is successful.

According to the position estimation device 30 of the present embodiment, the sound s having the property that the amplitude of the sound pressure gradually increases with the arrival of the sound reaches the microphones M1 to M4 in an environment with various noises. The time can be detected accurately. This makes it possible to estimate the position of the sound source (user) with high accuracy. Further, the control device 20 is provided with the position estimation device 30 so that the air conditioning control for the area desired by the user can be performed.

In the above description, a case where position estimation is performed for one output of sound s by the terminal device 40 has been given as an example, but in order to perform one position estimation, sound s is performed a plurality of times from the terminal device 40. It may be output. In that case, the position estimation device 30 estimates the position with respect to the output position of the sound s each time the terminal device 40 outputs the sound s, and estimates, for example, the average value of the estimation results as the position of the terminal device 40 (user). You may do so. By determining the final estimated position based on the results of a plurality of estimations, it is possible to reduce the influence of variations in the position estimation process and improve the accuracy of the position estimation.

<Second embodiment>
Hereinafter, the air conditioning system according to the second embodiment of the present invention will be described with reference to FIG.
FIG. 9 is a diagram showing an example of an air conditioning system according to the second embodiment of the present invention.
Among the configurations according to the second embodiment of the present invention, the same components as those constituting the air conditioning system according to the first embodiment of the present invention are designated by the same reference numerals, and the description thereof will be omitted. In the air conditioning system according to the second embodiment, the position estimation device 30A includes a comparator 38 instead of the A / D converter 33, and an arrival time estimation unit 35A instead of the arrival time estimation unit 35. Further, in the second embodiment, it is defined that the terminal device 40 outputs the sound s for N cycles at the time of one output, and it is assumed that the information is shared between the terminal device 40 and the microcomputer 34.
The comparator 38 compares the amplitude of the analog sound signal of the sound s output by the filter 32 with a predetermined threshold value, and outputs "1" if the amplitude of the analog sound signal exceeds the threshold value, and the amplitude of the analog sound signal is increased. If it is equal to or less than the threshold value, "0" is output. Here, the threshold value is a value for determining whether or not the sound s output from the terminal device 40 has arrived. The comparator 38 outputs the pulse signal of "0" or "1" to the arrival time estimation unit 35A.

The arrival time estimation unit 35A, for example, changes the output from the comparator 38 from the state of "0" for a certain period of time, and when "1" is output a predetermined number of times in total or continuously a predetermined number of times, the sound s Is determined to have arrived. Further, in the arrival time estimation unit 35A, for example, the output from the comparator 38 changes from the state of "1" for a certain period, and "0" is a total of a predetermined number of times, or continuously a predetermined number of times (for example, 10 times). When it is output, it is determined that the arrival of the sound s is completed. When the arrival time estimation unit 35A determines that the arrival of the sound s has been completed, the last time when "1" is output is set as the detection end time, and the time retroactive by N / f0 from the detection end time is the sound s. Estimated as the arrival time of (second arrival time). Alternatively, the arrival time estimation unit 35A uses the time when it is determined that the sound s has arrived as the detection start time, and the time retroactive from the detection start time by a predetermined time (for example, 3 cycles, 3 / f0) as the arrival time of the sound s. It may be estimated (first arrival time). Estimating the time retroactive by a predetermined time from the time when the sound s is first determined to have arrived (detection start time) is estimated as the arrival time because the sound gradually rises as shown in FIG. 3 (b), so that the sound s rises. This is because the output from the comparator 38 becomes "0" for a while after the arrival. Alternatively, the arrival time estimation unit 35A may calculate the first arrival time and the second arrival time, and estimate the average of the two times or the weighted average as the arrival time of the sound s.

The arrival time estimation unit 35A estimates the arrival time for each of the sounds s detected by the microphones M1 to M4, and outputs the four arrival times to the position estimation unit 36.
From the estimated value of the arrival time of the sound s to the microphones M1 to M4, the position estimation unit 36 is, for example, the difference in the arrival time of the sound s to the microphone M1 and the microphone M2, and the arrival time of the sound s to the microphone M1 and the microphone M3. The difference, the arrival time difference of the sound s to the microphone M1 and the microphone M4 is calculated, and the position of the sound source (terminal device 40) is estimated by TDOA.

According to the present embodiment, the arrival time of the sound s can be accurately estimated based on the pulse signal output by the comparator 38 based on the threshold value. According to the second embodiment, the arrival time can be estimated with a smaller calculation load than that of the first embodiment.

Further, the method of estimating the arrival time of the second embodiment can also be executed in the air conditioning system provided with the configuration of the first embodiment. For example, the arrival time estimation unit 35 compares a predetermined threshold value with the peak value of the sound pressure output from the A / D converter 33. Then, when the peak value of the sound pressure changes from below the threshold value for a certain period of time and the peak value exceeds the threshold value for a predetermined number of times in total or continuously for a predetermined number of times, the sound s is generated by the arrival time estimation unit 35. Judge that it has arrived. The arrival time estimation unit 35 estimates the time when the sound s has arrived as the detection start time, and the time retroactive from the detection start time by a predetermined time (for example, 3 cycles, 3 / f0) as the arrival time of the sound s. It may be (first arrival time).
Further, the arrival time estimation unit 35 determines that the sound s has arrived when the peak value falls below the threshold value a predetermined number of times or continuously for a predetermined number of times (for example, 10 times). It is determined that the arrival has been completed. When it is determined that the arrival of the sound s is completed, the time when the peak value of the amplitude exceeds the threshold at the end is estimated as the detection end time, and the time retroactive by N / f0 from the detection end time is estimated as the arrival time of the sound s. (Second arrival time). Alternatively, the arrival time estimation unit 35 may calculate the first arrival time and the second arrival time, and estimate the two averages or the weighted average as the arrival time of the sound s.
In the case of estimating the arrival time based on the threshold value, the arrival time can be estimated with a smaller calculation load as compared with the case of calculating the approximate curve.

FIG. 10 is a diagram showing an example of the hardware configuration of the position estimation device according to each embodiment of the present invention.
The computer 900 includes a CPU 901, a main storage device 902, an auxiliary storage device 903, and an interface 904.
The position estimation devices 30 and 30A described above include a computer 900. The operation of each of the above-mentioned processing units is stored in the auxiliary storage device 903 in the form of a program. The CPU 901 reads the program from the auxiliary storage device 903, expands it to the main storage device 902, and executes the above processing according to the program. Further, the CPU 901 secures a storage area in the main storage device 902 according to the program.

In at least one embodiment, the auxiliary storage device 903 is an example of a non-temporary tangible medium. Other examples of non-temporary tangible media include magnetic disks, magneto-optical disks, optical disks, semiconductor memories, etc. connected via the interface 904. Further, when this program is distributed to the computer 900 by a communication line, the distributed computer 900 may expand the program to the main storage device 902 and execute the above processing.

The process of each process in the position estimation devices 30 and 30A described above is stored in a computer-readable recording medium in the form of a program, and the computer of the position estimation devices 30 and 30A reads and executes this program. The above processing is performed. Here, the computer-readable recording medium means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Further, the computer program may be distributed to the computer via a communication line, and the computer receiving the distribution may execute the program.

Further, the above program may be for realizing a part of the above-mentioned functions. Further, a so-called difference file (difference program) may be used, which can realize the above-mentioned function in combination with a program already recorded in the computer system.
The position estimation devices 30 and 30A may be composed of one computer or may be composed of a plurality of computers connected so as to be communicable.

In addition, it is possible to replace the components in the above-described embodiment with well-known components as appropriate without departing from the spirit of the present invention. Further, the technical scope of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention. The analog sound signal and the digital sound signal are examples of waveform signals.

For example, in the above embodiment, all of the microphones M1 to M4 are used at the same time, but the position of the sound source on the plane can be estimated if there are three microphones. When only the position estimation on the user's plane is performed, for example, the position estimation may be performed using only the microphones M1 to M3, and the remaining microphone M4 may be used when any of the microphones fails. Alternatively, any three microphones among the four microphones M1 to M4 may be selected each time, and the position may be estimated by the selected microphones.

Alternatively, when performing position estimation on a plane, estimation using microphones M1, M2, and M3, estimation using microphones M1, M3, and M4, and estimation using microphones M1, M2, and M4 for one position estimation. , The position estimation process of the estimation using the microphones M2, M3, and M4 may be performed four times, and the average of the estimation results of the four times may be determined as the final position of the terminal device 40.

10 ... Air conditioner 20 ... Control device 30 ... Position estimation device 31 ... Amplifier 32 ... Filter 33 ... A / D converter 34, 34A ... Microphone 35, 35A ... Arrival time estimation unit 36 ... Position estimation unit 37 ... Communication unit 38 ... Comparator 40 ... Terminal device M1, M2, M3, M4 ... Microphone w ... Space s ... Sound 900 ... Computer 901 ... CPU
902 ... Main storage device 903 ... Auxiliary storage device 904 ... Interface

Claims (16)

The waveform signal of the sound detected by the sensor is acquired, the detection end time of the sound is estimated based on the waveform signal, and the arrival time of the sound at the sensor is estimated based on the detection end time. Arrival time estimation unit and
A position estimation unit that estimates the output position of the sound based on the time difference between the arrival times of the sound in the plurality of sensors estimated by the arrival time estimation unit, and a position estimation unit.
Equipped with
The arrival time estimation unit is
The detection end time is estimated based on an approximate curve showing a change in the amplitude of the waveform signal over time, and from the detection end time, the time of one cycle of the sound and the wave of several cycles in the output sound. The time retroactive by the time obtained by multiplying the number of output cycles indicating whether or not is included is estimated as the arrival time of the sound.
Position estimation device. The waveform signal of the sound detected by the sensor is acquired, the detection start time and the detection end time of the sound are estimated based on the waveform signal, and the sensor of the sound is estimated based on the detection start time and the detection end time. Arrival time estimation unit that estimates the arrival time to
A position estimation unit that estimates the output position of the sound based on the time difference between the arrival times of the sound in the plurality of sensors estimated by the arrival time estimation unit, and a position estimation unit.
Equipped with
The arrival time estimation unit is
The detection start time and the detection end time are estimated based on an approximate curve showing a change in the amplitude of the waveform signal over time, and from the detection end time to the time of one cycle of the sound and the output sound. The arrival time of the sound is estimated by a weighted average of the time retroactive by the time obtained by multiplying the number of output cycles indicating how many cycles of waves are included and the detection start time.
Position estimation device. The arrival time that acquires the waveform signal of the sound detected by the sensor, estimates the detection end time of the sound based on the waveform signal, and estimates the arrival time of the sound to the sensor based on the detection end time. Estimator and
A position estimation unit that estimates the output position of the sound based on the time difference between the arrival times of the sound in the plurality of sensors estimated by the arrival time estimation unit, and a position estimation unit.
Equipped with
The arrival time estimation unit is
The last time when the magnitude of the amplitude of the waveform signal exceeds a predetermined threshold is estimated as the detection end time, and from the detection end time, the time of one cycle of the sound and the number of cycles of the output sound are included. The time retroactive by the time obtained by multiplying the number of output cycles indicating whether or not the sound is included is estimated as the arrival time of the sound.
Position estimation device. The waveform signal of the sound detected by the sensor is acquired, the detection start time and the detection end time of the sound are estimated based on the waveform signal, and the sensor of the sound is estimated based on the detection start time and the detection end time. Arrival time estimation unit that estimates the arrival time to
A position estimation unit that estimates the output position of the sound based on the time difference between the arrival times of the sound in the plurality of sensors estimated by the arrival time estimation unit, and a position estimation unit.
Equipped with
The arrival time estimation unit is
The time when the number of times the magnitude of the amplitude of the waveform signal exceeds the predetermined threshold value exceeds the predetermined number of times is estimated as the detection start time.
The last time when the amplitude exceeds the threshold value is estimated as the detection end time, and the time is advanced from the detection end time by the time obtained by multiplying the sound cycle and the number of output sound cycles . The arrival time of the sound is estimated by the weighted average of the time and the detection start time.
Position estimation device. The position estimation device according to any one of claims 1 to 4 , further comprising a communication unit that outputs an error signal when the estimation of the arrival time of the sound or the estimation of the output position of the sound fails. .. With multiple sensors to detect sound,
A filter that extracts a waveform signal having a frequency in a predetermined range from the waveform signals of the sound detected by the sensor, and a filter.
A converter that converts the waveform signal of the sound output by the filter into a digital signal,
The position estimation device according to any one of claims 1 to 2 , further comprising. With multiple sensors to detect sound,
A filter that extracts a waveform signal having a frequency in a predetermined range from the waveform signals of the sound detected by the sensor, and a filter.
A comparator that outputs whether or not the amplitude of the waveform signal of the sound output by the filter is larger than a predetermined threshold value, and
The position estimation device according to any one of claims 3 to 4 , further comprising. The position estimation device according to any one of claims 1 to 7 is provided.
Air conditioning is performed for the sound output position estimated by the position estimation device.
Air conditioning system. A step of acquiring the waveform signal of the sound detected by the sensor, estimating the detection end time of the sound based on the waveform signal, and estimating the arrival time of the sound based on the detection end time.
A step of estimating the output position of the sound based on the time difference of the arrival time of the sound in the plurality of sensors, and
Have ,
In the step of estimating the arrival time of the sound,
The detection end time is estimated based on an approximate curve showing a change in the amplitude of the waveform signal over time, and the detection end time is multiplied by the sound cycle and the number of output sound cycles. The time that goes back by the specified time is estimated as the arrival time of the sound.
Position estimation method. The waveform signal of the sound detected by the sensor is acquired, the detection start time and the detection end time of the sound are estimated based on the waveform signal, and the sensor of the sound is estimated based on the detection start time and the detection end time. And the step to estimate the arrival time to
A step of estimating the output position of the sound based on the time difference of the arrival time of the sound in the plurality of sensors, and
Have,
In the step of estimating the arrival time of the sound,
The detection start time and the detection end time are estimated based on an approximate curve showing a change in the amplitude of the waveform signal over time, and from the detection end time to the time of one cycle of the sound and the output sound. The arrival time of the sound is estimated by a weighted average of the time retroactive by the time obtained by multiplying the number of output cycles indicating how many cycles of waves are included and the detection start time.
Position estimation method. A step of acquiring the waveform signal of the sound detected by the sensor, estimating the detection end time of the sound based on the waveform signal, and estimating the arrival time of the sound based on the detection end time.
A step of estimating the output position of the sound based on the time difference of the arrival time of the sound in the plurality of sensors, and
Have,
In the step of estimating the arrival time of the sound,
The last time when the magnitude of the amplitude of the waveform signal exceeds a predetermined threshold is estimated as the detection end time, and from the detection end time, the time of one cycle of the sound and the number of cycles of the output sound are included. The time retroactive by the time obtained by multiplying the number of output cycles indicating whether or not the sound is included is estimated as the arrival time of the sound.
Position estimation method. The waveform signal of the sound detected by the sensor is acquired, the detection start time and the detection end time of the sound are estimated based on the waveform signal, and the sensor of the sound is estimated based on the detection start time and the detection end time. And the step to estimate the arrival time to
A step of estimating the output position of the sound based on the time difference of the arrival time of the sound in the plurality of sensors, and
Have,
The time when the magnitude of the amplitude of the waveform signal exceeds the predetermined threshold is estimated as the detection start time, and the last time when the amplitude exceeds the threshold is estimated as the detection end time. The arrival time of the sound is estimated from the detection end time by a weighted average of the time retroactive by the time obtained by multiplying the sound cycle and the number of output sound cycles and the detection start time. ,
Position estimation method. Computer,
A means for acquiring a waveform signal of a sound detected by a sensor, estimating the detection end time of the sound based on the waveform signal, and estimating the arrival time of the sound based on the detection end time.
A means for estimating the output position of the sound based on the time difference between the arrival times of the sound in the plurality of sensors.
To function as
The means for estimating the arrival time of the sound is
The detection end time is estimated based on an approximate curve showing a change in the amplitude of the waveform signal over time, and the detection end time is multiplied by the sound cycle and the number of output sound cycles. The time that goes back by the specified time is estimated as the arrival time of the sound.
program. Computer,
The waveform signal of the sound detected by the sensor is acquired, the detection start time and the detection end time of the sound are estimated based on the waveform signal, and the sensor of the sound is estimated based on the detection start time and the detection end time. A means of estimating the time of arrival at
A means for estimating the output position of the sound based on the time difference between the arrival times of the sound in the plurality of sensors.
To function as
The means for estimating the arrival time of the sound is
The detection start time and the detection end time are estimated based on an approximate curve showing a change in the amplitude of the waveform signal over time, and from the detection end time to the time of one cycle of the sound and the output sound. The arrival time of the sound is estimated by a weighted average of the time retroactive by the time obtained by multiplying the number of output cycles indicating how many cycles of waves are included and the detection start time.
program. Computer,
A means for acquiring a waveform signal of a sound detected by a sensor, estimating the detection end time of the sound based on the waveform signal, and estimating the arrival time of the sound based on the detection end time.
A means for estimating the output position of the sound based on the time difference between the arrival times of the sound in the plurality of sensors.
To function as
The means for estimating the arrival time of the sound is
The last time when the magnitude of the amplitude of the waveform signal exceeds a predetermined threshold is estimated as the detection end time, and from the detection end time, the time of one cycle of the sound and the number of cycles of the output sound are included. The time retroactive by the time obtained by multiplying the number of output cycles indicating whether or not the sound is included is estimated as the arrival time of the sound.
program. Computer,
The waveform signal of the sound detected by the sensor is acquired, the detection start time and the detection end time of the sound are estimated based on the waveform signal, and the sensor of the sound is estimated based on the detection start time and the detection end time. A means of estimating the time of arrival at
A means for estimating the output position of the sound based on the time difference between the arrival times of the sound in the plurality of sensors.
To function as
The means for estimating the arrival time of the sound is
The time when the number of times the magnitude of the amplitude of the waveform signal exceeds the predetermined threshold value exceeds the predetermined number of times is estimated as the detection start time.
The last time when the amplitude exceeds the threshold value is estimated as the detection end time, and the time is advanced from the detection end time by the time obtained by multiplying the sound cycle and the number of output sound cycles. The arrival time of the sound is estimated by the weighted average of the time and the detection start time.
program.

Images