JP7574879B2 - Data Acquisition Equipment - Google Patents

Description

The present invention relates to a data acquisition device .

Traditionally, in the fields of sports such as golf, tennis, and marathons, athletes have been able to learn proper form and improve their skills by filming their swing and running form and acquiring exercise data related to their body movements, and then reviewing and analyzing the footage and exercise data.

Systems that collect various data during such exercise include those that place imaging devices (digital cameras, action cameras, etc.) around the subject and attach various sensor devices (motion sensors, heart rate monitors, GPS receivers, etc.) to the body, and control the operation of these devices according to instructions from a host device (personal computer, smartphone, etc.).

Here, when acquiring various data during exercise, the internal clocks of each device are synchronized and the acquired data are temporally associated with each other, so that, for example, an image of the form and changes in the exercise data can be synchronized and played back. Regarding a method for synchronizing the internal clocks of multiple devices, for example, Patent Document 1 describes a method in a communication system equipped with a server and multiple clients, in which each client corrects their internal clock to synchronize with the server time based on the time notification information sent from the server and the error time in the communication path calculated by communication with the server.

JP 2007-163330 A

The above-mentioned Patent Document 1 describes a method of providing each client with a calculation means, calculating the time error in the communication path with the server, and correcting the time. When such a communication system is applied to a system that acquires various data during exercise as described above, each of the clients, such as the imaging device and sensor device, needs to be provided with a calculation means for correcting the time.

The present invention aims to provide a simple method for temporally associating various types of data when they are acquired by multiple devices.

A data acquisition device according to one embodiment of the present invention comprises a plurality of imaging devices that acquire video data by imaging , and a control device connected to each of the plurality of imaging devices via an individual communication path and controlling the plurality of imaging devices, wherein the control device calculates a signal delay time that occurs when communicating with each of the plurality of imaging devices via the communication path prior to the imaging by transmitting a predetermined transmission signal to each of the plurality of imaging devices via the communication path corresponding to that imaging device and receiving a response signal to the transmission signal, generates setting information based on the signal delay time for setting a common reference time when each of the video data acquired by each of the plurality of imaging devices is played back and output in a synchronized manner, and transmits the setting information to each of the plurality of imaging devices, and the plurality of imaging devices receive the setting information transmitted by the control device, and when the video data is acquired by imaging , associates the setting information with the acquired video data and stores it.

According to the present invention, when various types of data are acquired by multiple devices, the data can be temporally associated with each other using a simple method.

1 is a schematic configuration diagram showing an embodiment of a data acquisition device according to the present invention; 1 is a functional block diagram illustrating an example of an imaging device that is applied to a data acquisition device according to an embodiment. 1 is a functional block diagram illustrating an example of a host device that is applied to a data acquisition device according to an embodiment. 1 is a schematic diagram showing an example of a method for capturing a moving image in a data acquisition device according to an embodiment; 11 is a flowchart showing an example of a control method (device synchronization process, video data acquisition process) of a data acquisition device according to an embodiment. 11 is a time chart showing an example of a control method (device synchronization process, video data acquisition process) of a data acquisition device according to an embodiment. 11 is a flowchart showing a first example of a control method (synchronous playback process of video data) of a data acquisition device according to an embodiment. 4 is a time chart showing a first example of a control method of a data acquisition device according to an embodiment. 13 is a flowchart showing a second example of a control method (synchronous playback process of video data) of a data acquisition device according to an embodiment. 10 is a time chart showing a second example of a control method of a data acquisition device according to an embodiment. 13 is a time chart showing a third example of a control method of a data acquisition device according to an embodiment. 11 is a time chart showing a modified example of a control method (moving image data acquisition process) of a data acquisition device according to an embodiment. 10 is a time chart showing a modified example of a control method for a data acquisition device according to an embodiment.

The data acquisition device, its control method and control program, as well as the control device and data acquisition device according to the present invention will be described in detail below with reference to an embodiment. Note that in the following embodiment, in order to simplify the explanation, a case will be described in which a user performing a golf swing is filmed from multiple different directions using a digital camera.

<Data acquisition device>
Fig. 1 is a schematic diagram showing an embodiment of a data acquisition device according to the present invention, Fig. 2 is a functional block diagram showing an example of an imaging device applied to the data acquisition device according to the present embodiment, and Fig. 3 is a functional block diagram showing an example of a host device applied to the data acquisition device according to the present embodiment.

As shown in FIG. 1, one embodiment of the data acquisition device according to the present invention has a device group 100 including a plurality of imaging devices (data acquisition devices) 120A, 120B, 120C, ..., and a host device (control device) 200 that connects to each of the imaging devices 120A, 120B, 120C, ... of the device group 100 via wireless communication and controls the operation of each of the imaging devices 120A, 120B, 120C, .... Hereinafter, the plurality of imaging devices 120A, 120B, 120C, ... will be collectively referred to as "imaging devices 120".

(Imaging device 120)
1, the imaging device 120 is a device having a function of capturing moving images, such as a digital camera or an action camera, and in this embodiment, based on an instruction from the host device 200, continuously captures the body movements of a subject (a user) during exercise from a plurality of different directions and acquires the motion data as moving image data. Specifically, the imaging device 120 includes, for example, an imaging unit (data acquisition means) 122, an operation unit 124, a display unit 126, a communication function unit (receiving means) 128, an arithmetic circuit 130, a memory unit (storing means) 132, and an operating power supply 134, as shown in FIG.

The image capturing unit 122 generates video data by repeatedly acquiring image data of one screen (frame) of the subject at a predetermined cycle. The generated video data is stored in a predetermined storage area (data memory) of the memory unit 132 described later. The operation unit 124 is an input means such as a shooting button, menu switch, power switch, etc. attached to a digital camera or the like. The operation unit 124 accepts user operations to control the start and end of video image capture in the image capturing unit 122, the selection and setting of various menus such as the shooting mode, and the power on and off of the imaging device 120. Note that the operation unit 124 may be provided with a touch panel on the display unit 126 in addition to or instead of the various operation buttons described above.

The display unit 126 has a liquid crystal display panel or the like, and displays a live view image of the subject during shooting, video data stored in the memory unit 132, a menu screen when the operation unit 124 is operated, the operating status of the imaging device 120, etc. Note that the display unit 126 is not limited to being provided in the main body of the imaging device 120, and may be provided outside the imaging device 120.

The communication function unit 128 functions as an interface when transmitting and receiving signals (synchronization setting signals) related to the synchronization of the reference time during video capture with the host device 200, when receiving various control signals related to the operation of the imaging device 120, and when outputting video data stored in the memory unit to the outside of the imaging device 120. Here, as a method for transmitting and receiving synchronization setting signals, control signals, video data, etc. with the host device 200 via the communication function unit 128, a wireless communication method such as WiFi (registered trademark) communication or Bluetooth (registered trademark) communication is applied.

The arithmetic circuit 130 is an arithmetic processing device such as a CPU or MPU, and executes a predetermined control program based on an operation clock generated by an internal clock (or a timing circuit), thereby controlling the shooting operation in the shooting unit 122, the storage and reading operation of video data and the like in the memory unit 132, the sending and receiving operation of various control signals to and from the host device 200 in the communication function unit 128, and the playback and output operation of video data. The operation control executed in the arithmetic circuit 130 will be described in detail later in relation to the operation of the host device 200. The control program executed in the arithmetic circuit 130 may be one that is previously built into the arithmetic circuit 130, or one that is stored in the memory unit 132, which will be described later.

The memory unit 132 is roughly divided into a data memory, a program memory, and a working memory. The data memory stores the video data captured by the above-mentioned image capture unit 122 in association with time data. The program memory stores control programs for executing the image capture operation in the image capture unit 122, the storage and read operation of video data, etc. in the memory unit 132, the transmission and reception operation of various control signals with the host device 200 in the communication function unit 128, and the output operation of video data. The working memory temporarily stores data used when executing the control programs and data generated. Note that the data memory may be partially or entirely in the form of a removable storage medium such as a memory card, and may be configured to be detachable from the imaging device 120.

The operating power supply 134 supplies driving power to each component inside the imaging device 120 when the power switch of the operation unit 124 is operated. The imaging device 120 may be one in which the photographing unit 122 transitions to a photographing standby state when the operating power supply 134 supplies driving power (power on), or one in which photographing is started and ended immediately in conjunction with the power being turned on and off. The imaging device 120 may be one in which the communication function unit 128 automatically transitions to a standby state for a synchronization process executed with the host device 200 when the operating power supply 134 supplies driving power (power on), or one in which synchronization is executed immediately.

(Host device 200)
As shown in Fig. 1, the host device 200 is a general-purpose or dedicated information and communication device such as a smartphone 220A, a personal computer 220B, a tablet terminal (not shown), etc. In this embodiment, the host device 200 individually controls each of the imaging devices 120A, 120B, 120C, etc. of the device group 100 by wireless communication to obtain multiple video data capturing the user's body movements during exercise, and plays and outputs the video data in a synchronized state. Specifically, as shown in Fig. 3, the host device 200 includes an operation unit 224, a display unit 226, a communication function unit (transmission means) 228, an arithmetic circuit (calculation means, generation means) 230, a memory unit 232, and an operating power supply 234. Here, the description of the same configuration as the imaging device 120 described above will be simplified.

The operation unit 224 is an input means such as a touch panel, keyboard, mouse, etc. attached to the smartphone 220A, the personal computer 220B, etc. The operation unit 224 selects any icon or menu displayed on the display unit 226, or indicates any position on the screen display, thereby executing a function corresponding to the icon, menu, or display position. The display unit 226 has a liquid crystal display panel, etc., and plays and outputs at least multiple video data acquired by the above-mentioned imaging device 120 in a mutually synchronized state.

The communication function unit 228 functions as an interface when transmitting and receiving signals (synchronization setting signals) related to the synchronization of the reference time when shooting videos between multiple imaging devices 120 using the above-mentioned wireless communication method, when transmitting various control signals related to the operation of the imaging devices 120, and when receiving video data acquired by the imaging devices 120.

The arithmetic circuit 230 is an arithmetic processing device such as a CPU or MPU, and executes a predetermined control program based on an operation clock generated by an internal clock (or a timekeeping circuit), thereby controlling synchronization with multiple image capture devices 120, generation of control signals for instructing image capture operations and playback output operations in the image capture devices 120, display operations in the display unit 226, and transmission and reception of various control signals to and from the image capture devices 120 in the communication function unit 228. The operation control executed in the arithmetic circuit 230 will be described in detail later in relation to the operation of the host device 200. The control program executed in the arithmetic circuit 230 may be one that is previously built into the arithmetic circuit 230, or one that is stored in the memory unit 232 described later.

The memory unit 232 has a data memory, a program memory, and a working memory, similar to the imaging device 120 described above. The data memory stores data to be displayed on the display unit 226, such as video data output from the imaging device 120 described above. The program memory stores control programs for executing synchronization processing with the imaging device 120, display operations on the display unit 226, and transmission and reception operations of various control signals with the imaging device 120 in the communication function unit 228. Note that the data memory may be partially or entirely in the form of a removable storage medium such as a memory card, and configured to be detachable from the host device 200.

When a power switch (not shown) is operated, the operating power source 234 supplies driving power to each component inside the host device 200. Note that when the operating power source 234 supplies driving power (power on), the host device 200 may be one in which the communication function unit 228 automatically transitions to a standby state for synchronization processing executed between the multiple imaging devices 120, or may be one in which synchronization processing is executed immediately.

<Method of controlling data acquisition device>
Next, a method for controlling the data acquisition device according to the present embodiment will be described with reference to the drawings. Here, a series of operational controls from device synchronization processing in the data acquisition device according to the present embodiment, to video data acquisition processing by the imaging device 120, and video data synchronous playback processing by the host device 200 will be described.

Figure 4 is a schematic diagram showing an example of a method for capturing moving images in a data acquisition device according to this embodiment. Figure 5 is a flowchart showing an example of a control method (device synchronization processing, video data acquisition processing) of a data acquisition device according to this embodiment, and Figure 6 is a time chart showing an example of a control method (device synchronization processing, video data acquisition processing) of a data acquisition device according to this embodiment.

(Device synchronization process)
First, a method for capturing moving images in the data acquisition device according to this embodiment will be briefly described.
In this embodiment, as shown in Fig. 4, when the user US performs a golf swing, the image capturing device 120 is disposed at positions where the swing of the user US can be captured from a plurality of different directions, for example, the front side (abdominal side), rear side (back side), left and right lateral sides, upper side (head side), lower side (foot side), etc. of the user US, and the form during the swing is captured as a moving image. As a result, moving image data of the swing of the user US captured from a plurality of different directions is acquired, and time data (synchronous playback setting information) that specifies synchronous playback, which will be described later, is associated with this moving image data and saved.

In the control method of the data acquisition device according to this embodiment, first, as shown in the flowchart in FIG. 5, the host device 200 executes a device synchronization process for synchronizing the reference times of the imaging devices 120, and a video data acquisition process for capturing an image of the user US with each imaging device 120 to acquire video data. Here, the device synchronization process and the video data acquisition process are realized by executing a predetermined control program in the arithmetic circuit 230 of the host device 200 and the arithmetic circuit 130 of each imaging device 120.

In the device synchronization process in which the host device 200 synchronizes the reference time in each imaging device 120, the user US first turns on the host device 200 and the imaging devices 120 arranged around the user US to start each device. After starting each device, the user US operates the operation unit 224 of the host device 200 and the operation unit 124 of the imaging device 120 to instruct the start of the device synchronization process, or by turning on the power to start each device, a synchronization setting signal is transmitted and received between the host device 200 and each imaging device 120, and the reference time in each imaging device 120 is synchronized. Here, the host device 200 and each imaging device 120 are connected using a wireless communication method such as Wi-Fi communication or Bluetooth (registered trademark) communication. In addition, when transmitting and receiving a synchronization setting signal, packet communication is performed according to a communication protocol such as NTP (Network Time Protocol) or SNTP (Simple Network Time Protocol), thereby synchronizing the reference time between the host device 200 and multiple imaging devices 120 (i.e., a communication system between one to multiple data acquisition devices).

5 and the time chart shown in FIG. 6, the arithmetic circuit 230 of the host device 200 transmits a packet signal for time measurement (time measurement packet signal) to one of the multiple imaging devices 120 connected by the wireless communication (e.g., imaging device 120B) via a communication path formed by wireless communication by the communication function unit 228 (step S102). When the arithmetic circuit 130 of the imaging device 120B receives the time measurement packet signal transmitted from the host device 200, the communication function unit 128 transmits a response signal to the host device 200 via the communication path (step S122).

When the arithmetic circuit 230 of the host device 200 receives the response signal transmitted from the imaging device 120B (step S104), the arithmetic circuit 230 synchronizes the time between the host device 200 and the imaging device 120B based on the communication time required for transmitting and receiving the synchronization setting signal (time measurement packet signal and response signal) between the host device 200 and the imaging device 120B (step S106). For example, as shown in the time chart of FIG. 6, the transmission time of the time measurement packet signal in the host device 200 is set to T_H1, the time measurement packet signal is received and the response signal is transmitted at the imaging device 120B._B1, the time when the response signal is received by the host device 200 is T_H2In this case, the time is adjusted (T_H3= T_B1) is performed ((1) in FIG. 6). Here, since the host device 200 and the image capture device 120B transmit and receive the synchronization setting signal via the same communication path, the time required for transmission and the time required for reception can be considered to be approximately equal.
T_H3= T_H1+ (T_H2-T_H1)/2
=T_B1...(11)

The arithmetic circuit 230 of the host device 200 stores the time determined by the above-mentioned time adjustment as a provisional reference time T _B1 in the imaging device 120B, and transmits it as a time setting notification to the imaging device 120B (step S108; (3) in FIG. 6). When the arithmetic circuit 130 of the imaging device 120B receives the time transmitted from the host device 200 (step S124), it sets the time as the provisional reference time T _B1 (=T _H3 ) in the internal clock (step S126).

Then, the arithmetic circuit 230 of the host device 200 determines whether time adjustment has been completed for the other imaging devices 120 (e.g., imaging devices 120A, C, ...) other than the imaging device 120B for which the time adjustment has been completed (step S110). The arithmetic circuit 230 repeats the above series of processes (steps S102 to S108 and steps S122 to S126) for each imaging device 120 in sequence until time adjustment has been completed for all imaging devices 120 (No in step S110).

As a result, time synchronization is also performed between the host device 200 and the other imaging devices 120 (imaging devices 120A, C, ...). For example, as shown in the time chart in FIG. 6, in time synchronization for imaging device 120A, the transmission time of the time measurement packet signal in the host device 200 is set to T_H4, the time measurement packet signal is received and the response signal is transmitted at the imaging device 120A by T_A1, the time when the response signal is received by the host device 200 is T_H5In this case, the time can be adjusted (T_H6= T_A1) is carried out (see (2) in FIG. 6).
T_H6= T_H4+ (T_H5-T_H4)/2
=T_A1...(12)

The arithmetic circuit 230 of the host device 200 stores the time determined by the above-mentioned time adjustment as the provisional reference time T _A1 in the imaging device 120A and transmits it as a time setting notification to the imaging device 120A, and the arithmetic circuit 130 of the imaging device 120A sets the received time as the provisional reference time T _A1 (=T _H6 ) in its internal clock.

If the arithmetic circuit 230 of the host device 200 determines in step S110 that time adjustment has been completed for all of the imaging devices 120 (Yes in step S110), it calculates the difference between the provisional reference times stored for each of the multiple imaging devices 120 and corrects the reference times as follows:

Specifically, the arithmetic circuit 230 uses a provisional reference time defined by the earliest time adjustment performed between the host device 200 and each imaging device 120 (for example, provisional reference time T _B1 (=T _H3 ) defined by the time adjustment with imaging device 120B) as a reference and individually calculates the difference between this provisional reference time and the provisional reference times of the other imaging devices 120 (imaging devices 120A, C, ...).

Next, the arithmetic circuit 230 transmits the calculated difference in the provisional reference time as a correction time (difference time) t to the other imaging devices 120 individually (step S112). For example, as shown in the time chart of Fig. 6, if the imaging device 120 that has had its time adjusted earliest is the imaging device 120B, the difference between the provisional reference time _TB1 (= _TH3 ) of the imaging device 120B and the provisional reference time _TA1 (= _TH6 ) of the imaging device 120A is calculated using the following formula (13) and transmitted to the imaging device 120A as the correction time t ((3) in Fig. 6).
t=T _A1 - T _B1
=T _H6 -T _H3 ...(13)

Such calculation of the difference in the reference time and transmission of the correction time are also performed for the imaging devices 120 other than the imaging device 120B. Here, since the calculation of the difference in the reference time is calculated based on the tentative reference time T _B1 (=T _H3 ) of the imaging device 120B, the correction time t is not transmitted from the host device 200 to the imaging device 120B (or a correction time set to t=0 is transmitted).

When the arithmetic circuits 130 of the other imaging devices 120 (imaging devices 120A, C, ...) other than the imaging device 120B for which the provisional reference time has been defined receive the above-mentioned correction time (difference time) t transmitted from the host device 200, they execute a process of correcting the provisional reference time set in each internal clock of the other imaging devices 120 based on the correction time t (step S128).

Specifically, the arithmetic circuit 130 of the other imaging devices 120 corrects the provisional reference time set in step S126 to go back (advance) by the received correction time t. For example, as shown in the time chart in FIG. 6, when the provisional reference time of the imaging device 120A is T _A1 (=T _H6 ), a process of subtracting the correction time t from the provisional reference time T _A1 is performed as shown in the following formula (14), and the corrected new reference time T _A2 is set in the internal clock ((4) in FIG. 6 ). As a result, correction is performed so that the provisional reference times of all the other imaging devices 120 match the provisional reference time T _B1 (=T _H3 ) of the imaging device 120B, which has been adjusted earliest. That is, in this embodiment, the reference times of all the imaging devices 120 are corrected so that the provisional reference time of the imaging device 120, which has been adjusted earliest, becomes the regular reference time, and the synchronization of the reference times is performed between the host device 200 and the multiple imaging devices 120.
T _A2 = T _A1 - t (14)

In this embodiment, the case has been described in which the provisional reference time T _B1 (=T _H3 ) of the imaging device 120B, which is the earliest time to be synchronized between the host device 200 and the multiple imaging devices 120, is set as the regular reference time, but the present invention is not limited to this. That is, one of the provisional reference times stored for each of the multiple imaging devices 120 may be set as the reference (regular reference time), and the difference of the provisional reference time calculated by the above-mentioned difference calculation may be transmitted to each imaging device 120 as a correction time (difference time) t, and the provisional reference time set in the internal clock of the other imaging devices 120 may be corrected to go back (advance) or go back (delay) by the received correction time t, thereby synchronizing the reference time.

(Video data acquisition process)
Next, in the video data acquisition process in which each imaging device 120 photographs the user US to acquire video data, as shown in the flowchart of FIG. 5, prior to a golf swing, the user US operates the operation unit 224 of the host device 200 to instruct each imaging device 120 to start photographing, and a control signal (photographing start instruction) instructing each imaging device 120 to start photographing is simultaneously transmitted from the host device 200 to each imaging device 120 (step S114).

When the arithmetic circuit 130 of each imaging device 120 receives the shooting start instruction transmitted from the host device 200, the imaging unit 122 starts shooting a moving image with the user US as a subject (step S130). For convenience, as shown in the time chart in FIG. 6, among the multiple imaging devices 120, the communication time acquired when the host device 200 and the imaging device 120A are synchronized in the device synchronization process is the shortest (i.e., the signal delay is the smallest), and the communication time acquired when the host device 200 and the imaging device 120B are synchronized is the longest (i.e., the signal delay is the largest). As a result, among the multiple imaging devices 120, the imaging device 120A starts shooting a moving image the earliest (shooting start time T _A3 ), while the imaging device 120B starts shooting a moving image the latest (shooting start time T _B2 ).

The video data of the user US acquired by the image capturing unit 122 of the multiple image capturing devices 120 is sequentially stored in the data memory of the memory unit 132. At this time, the arithmetic circuit 130 of each image capturing device 120 acquires the elapsed time from the corrected reference time (normal reference time) set in each image capturing device 120 in the device synchronization process described above to the time when image capturing starts (step S132). For example, as shown in the time chart in FIG. 6, in the image capturing device 120A, the difference (T _A3 -T _A2 ) between the reference time T _A2 (=T _H3 ) and the image capturing start time T _A3 is acquired as the elapsed time, and in the image capturing device 120B, the difference (T _B2 -T _B1 ) between the reference time T _B1 (=T _H3 ) and the image capturing start time T _B2 is acquired as the elapsed time ((5) in FIG. 6).

Here, the elapsed time is calculated using the shooting start time of each imaging device 120 when shooting start instructions are simultaneously transmitted from the host device 200, using the normal reference time T _H3 defined by the above-mentioned device synchronization process as a common reference, and therefore each elapsed time includes a signal delay component caused by the communication path between the host device 200 and each imaging device 120. In addition, the signal delay time between the host device 200 and each imaging device 120 can be calculated as approximately 1/2 of the communication time (time required to transmit and receive signals) acquired when synchronizing the time between the host device 200 and each imaging device 120 in the above-mentioned device synchronization process.

Next, after the user US finishes his/her golf swing, he/she operates the operation unit 224 of the host device 200 to instruct each imaging device 120 to end image capture, and the host device 200 simultaneously transmits a control signal (image capture end instruction) to each imaging device 120 to instruct them to end image capture (step S116). When the arithmetic circuit 130 of each imaging device 120 receives the image capture end instruction transmitted from the host device 200, it ends image capture of the moving image in the image capture unit 122 (step S134).

At this time, the arithmetic circuit 130 of each imaging device 120, upon completion of shooting, stores the elapsed time including the above-mentioned signal delay component as synchronous playback setting information in the data memory of the memory unit 132 in association with the acquired video data (step S136; (6) in FIG. 6). Here, the elapsed time associated with the video data may be stored in a state where it is incorporated as part of the video data (e.g., header information), or may be stored in a storage area different from the video data in an associated state by linking or the like.

In the present embodiment, the user US operates the operation unit 224 of the host device 200 to transmit a control signal instructing each imaging device 120 to start and end shooting, thereby controlling the acquisition process of the video data. The present invention is not limited to this, and for example, each imaging device 120 may have a motion detection function, and after the above-mentioned device synchronization process is performed, each imaging device 120 may detect a specific movement of the user US and execute the acquisition process of the video data, or each imaging device 120 may automatically execute the acquisition process of the video data a predetermined time after the device synchronization process is performed.

(Synchronized playback processing)
Next, a description will be given of a synchronous playback process of video data executed in the control method of the data acquisition device according to the present embodiment. The video data acquired by each imaging device 120 through the above-described device synchronization process and video data acquisition process is synchronously played back using various control methods as follows.

(1) First Example of Synchronized Playback FIG. 7 is a flowchart showing a first example of a control method (synchronous playback processing of video data) of the data acquisition device according to this embodiment, and FIG. 8 is a time chart showing a first example of the control method of the data acquisition device according to this embodiment.

In a first example of the synchronous playback process of video data according to this embodiment, the video data acquired by each imaging device 120 by the above-mentioned video data acquisition process is moved to a single playback device and collected, and the video data is synchronously played back in the playback device. Here, the playback device may be, for example, the smartphone 220A or personal computer 220B shown as the host device 200 in this embodiment, a tablet terminal, or a general-purpose or dedicated video playback device. In this embodiment, a case will be described in which the host device 200 used when executing the above-mentioned device synchronization process and video data acquisition process is applied as the playback device. In this case, the synchronous playback of each video data collected in the host device 200 is realized, for example, by setting display areas corresponding to each video data individually on the display unit 126 of the host device 200 and displaying them in a divided manner.

As shown in the flowchart of FIG. 7, the video data synchronous playback process first copies or transfers the video data acquired by each imaging device 120 and stored in the memory unit 132 in the video data acquisition process described above, and moves it to the host device 200 for collection (step S152). Here, the elapsed time (synchronous playback setting information) stored in association with each video data is also moved to the host device 200 together with the video data. The video data and elapsed time collected from each imaging device 120 are stored in the data memory of the memory unit 232 of the host device 200.

Next, the user US operates the operation unit 224 of the host device 200 to instruct the start of playback of the collected video data (step S154), and the arithmetic circuit 230 acquires the elapsed time associated with each video data (step S156). The arithmetic circuit 230 sets the shooting start time of the video data with the longest elapsed time among the acquired elapsed times (i.e., the video data with the largest signal delay and the latest shooting start time) as the playback start reference time (step S158).

Next, for each video data other than the video data for which the playback start reference time is specified, the arithmetic circuit 230 fast-forwards the data from the shooting start time (i.e., the first frame) specified by the elapsed time to the playback start reference time, or skips playback (or skips forward) to exclude playback output (step S160). Here, the period during which playback output is excluded from the shooting start time to the playback start reference time is referred to as the "playback exclusion time". Then, the arithmetic circuit 230 simultaneously starts playback output of each video data from the frame of the playback start reference time, so that each video data is simultaneously displayed (synchronous playback) in a display area that has been previously divided and set on the display unit 126 (step S162).

Then, when the user US operates the operation unit 224 of the host device 200 to instruct the end of playback of the collected video data (step S164), or when each video data is played up to the final frame, the arithmetic circuit 230 ends the playback output of the video data to the display unit 226 (step S166).

In the synchronous playback process of the video data according to this embodiment, the process (steps S156, S158) in which the arithmetic circuit 230 of the host device 200 acquires the elapsed time from each collected video data and sets the playback start reference time is not limited to being executed with the user US's operation to start playback of the video data (step S154) as a trigger. For example, it may be executed automatically after the process (step S152) in which the video data of each imaging device 120 is moved to the host device 200 and collected is completed.

The process of setting the playback start reference time (step S158) may be performed by the calculation circuit 230 calculating the signal delay time based on the communication time acquired when synchronizing the time between the host device 200 and each imaging device 120 in the device synchronization process described above, and setting the shooting start time of the video data in the imaging device 120 with the largest signal delay as the playback start reference time. In addition, the process of excluding the playback output of each video data (step S160) may be performed by the calculation circuit 230 calculating the difference between the signal delay time in each imaging device 120 and the longest signal delay time in the device synchronization process described above, and setting the difference between the respective signal delay times as the playback exclusion time. These playback start reference times and playback exclusion times (or the signal delay times and their differences that define them) are transmitted from the host device 200 to each imaging device 120, and are stored in association with the video data acquired by each imaging device 120 as synchronous playback setting information.

Below, a control method for setting the playback start reference time and the playback exclusion time based on the signal delay time in each imaging device 120 calculated by the host device 200 will be specifically described with reference to the time chart in Figure 8.

8, in the control method for a data acquisition device having a synchronous playback process of video data according to this embodiment, first, in the above-mentioned device synchronization process, the signal delay times calculated based on the communication times acquired when synchronizing the time between the host device 200 and each of the imaging devices 120A, 120B, 120C, ... are respectively designated as T _H-A , T _H-B , T _H-C , .... Here, it is assumed that, among the multiple imaging devices 120, the signal delay time T _H-A in the imaging device 120A is the shortest, and the signal delay time T _H-B in the imaging device 120B is the longest.

In the device synchronization process, the arithmetic circuit 230 of the host device 200 extracts the longest signal delay time T _H-B from among the calculated signal delay times in each imaging device 120. Here, in the video data acquisition process, the shooting start time T _B2 of the video data acquired by the imaging device 120B having the longest signal delay time T _H-B is the latest shooting start time among the video data acquired by each imaging device 120. The arithmetic circuit 230 sets this latest shooting start time T _B2 as the playback start reference time T _max .

In addition, the calculation circuit 230 calculates the difference (T max - T _{A3 , T max - T B2 , T max - T C1 , ...) between the shooting start times T A3 , T B2 , T C1 ,} ... of each imaging device 120A, _120B , 120C, ... in the video data acquisition process _{and the playback start reference time T max defined} by the latest shooting start _time T _B2 , and sets this as the playback exclusion time T _skp to be applied when playing back each video data.

Here, the playback start reference time _Tmax and the playback exclusion time _Tskp are calculated using the shooting start time of each video data (or the elapsed time from a common reference time to the shooting start time) or the time difference between the shooting start times, and therefore can be calculated using the signal delay times T _H-A , T H-B, T _H-C , ... in each of the imaging devices 120A, _120B , 120C, ... as described above.

8, the playback start reference time Tmax common to each video data is defined by the shooting start time T _B2 of the video data in the imaging device 120B having the longest signal delay time T _H-B among the signal delay times in each imaging device 120. Also, the playback exclusion time T _skp in the video data acquired by the imaging device 120A is calculated from the difference (T _{max -T} H- _{A )} between the playback start reference time Tmax defined by the signal delay time T _H-B in the imaging device 120B, which is the longest signal delay time, and the signal delay time T _H-A in the imaging device 120A. Also, the playback exclusion time T _skp in the video data acquired by the imaging device 120C is calculated from the difference (T _{max -T H-C )} between the playback start reference time _Tmax and the signal delay time T _H-C in the imaging device 120C. Furthermore, since the playback exclusion time T _skp for each video data is calculated based on the signal delay time T _H-B in the imaging device 120B, the playback exclusion time T _skp for the video data acquired by the imaging device 120B is set to 0 (= T _max - _{T H-B} ). The individually calculated playback exclusion time T _skp for each video data is transmitted from the host device 200 to each imaging device 120 together with the above-mentioned playback start reference time T _max , and is stored in association with each video data acquired in the video data acquisition process as synchronous playback setting information.

In the synchronous playback process of the video data, the arithmetic circuit 230 acquires the playback start reference time _Tmax and the playback exclusion time _Tskp associated as synchronous playback setting information with the video data collected from each imaging device 120. Based on the synchronous playback setting information, the arithmetic circuit 230 excludes (fast-forwards or skips) the playback output of data for the playback exclusion time _Tskp set for each video data from the first frame of each video data, and simultaneously starts the playback output of each video data from the frame of the playback start reference time _Tmax .

Specifically, as shown in the time chart of Fig. 8, in the moving image data acquired by the imaging device 120A, for example, the reproduction output of data from the first frame to the fourth frame (circled number "4" in Fig. 8) is excluded, and the reproduction output starts from the fifth frame (circled number "5" in Fig. 8) of the reproduction start reference time _Tmax . In addition, in the moving image data acquired by the imaging device 120C, for example, the reproduction output of data from the first frame to the second frame (circled number "2" in Fig. 8) is excluded, and the reproduction output starts from the third frame (circled number "3" in Fig. 8) of the reproduction start reference time _Tmax . In addition, in the moving image data acquired by the imaging device 120B, the reproduction output of data is not excluded, and the reproduction output starts from the first frame (circled number "1" in Fig. 8) of the reproduction start reference time _Tmax .

In this embodiment, in a data acquisition device that acquires the movement of a subject (object) as video data using multiple imaging devices (data acquisition devices) connected by wireless communication, a device synchronization process is performed in which a common reference time is set for each imaging device by synchronizing the time between the host device and the imaging devices beforehand prior to the video data acquisition process. Next, during the video data acquisition process, synchronous playback setting information including a playback start reference time and a playback exclusion time calculated based on the elapsed time from the reference time to the start of shooting, or based on the shooting start time and signal delay time of the video data in each imaging device, is associated with each acquired video data and saved. Then, during playback output of each video data, the playback output of each playback exclusion time is excluded from the first frame of each video data based on the synchronous playback setting information associated with the video data, and then each video data is synchronously played and output from the frame of the common playback start reference time.

In this way, according to this embodiment, the video data acquired by the multiple imaging devices are played back and output with a common playback start reference time, and the synchronous playback setting information including the playback exclusion time for each video data is associated with each video data when the video data is acquired and saved, so that each video data can be played back and output in a synchronized manner based on the synchronous playback setting information. This makes it possible to eliminate the influence of signal delays that differ for each imaging device (data acquisition device) using a simple method and to play back multiple video data in a good synchronized manner. In other words, since the playback timing (playback start reference time) of the captured video images can be matched in multiple imaging devices, it is possible to effectively shoot synchronized video images using multiple imaging devices, and since the delay time during playback and output in this case can be matched to the video data of a specific imaging device, the shutter time lag during shooting corresponding to this delay time can be made the same as when shooting with a single imaging device.

In addition, in the device synchronization process, a common reference time can be set for each imaging device using a simple method based on the communication time acquired when synchronizing the time between the host device and multiple imaging devices, and synchronous playback setting information can be generated, so that the device synchronization process, which is executed in advance prior to the video data acquisition process, can be completed in a relatively short time.

In addition, in the synchronous playback process of video data shown in this embodiment, video data acquired by multiple imaging devices is moved to a host device, which is a single playback device, and is collected, and the host device (playback device) is controlled to execute the synchronous playback process of each video data. This means that the signal delays that each imaging device has in response to a playback command are not an influence, and multiple video data can be played back quickly and satisfactorily after a playback command is issued using a simple method.

(2) Second Synchronized Playback Example Fig. 9 is a flow chart showing a second example of the control method (synchronous playback processing of video data) of the data acquisition device according to this embodiment, and Fig. 10 is a time chart showing the second example of the control method of the data acquisition device according to this embodiment. Here, the explanation of the control method equivalent to the first synchronous playback example described above will be simplified.

In a second example of the synchronous playback process of video data according to this embodiment, the video data acquired by each imaging device 120 through the above-mentioned video data acquisition process is output to a display means connected to each imaging device 120 in response to an instruction from the host device 200, thereby performing synchronous playback of each video data. Here, the display means may be, for example, a plurality of display devices individually connected to each imaging device 120, or a single display device having a plurality of input ports corresponding to each imaging device 120. In the latter case, synchronous playback of each video data is realized by individually setting and splitting display areas corresponding to the video data input from each imaging device 120 via the individual input ports on a single display device.

As shown in the flowchart of FIG. 9, the synchronized playback process of the video data is performed when the user US operates the operation unit 224 of the host device 200 to instruct each imaging device 120 to start playing the acquired video data, and the host device 200 simultaneously transmits a control signal (playback start instruction) to each imaging device 120 to start playing (step S202).

When the arithmetic circuit 130 of each imaging device 120 receives the playback start instruction transmitted from the host device 200 (step S222), it acquires synchronous playback setting information associated with each video data (step S224). Here, the playback start instruction transmitted from the host device 200 is received by each imaging device 120 after the signal delay time calculated based on the communication time acquired by the host device 200 when synchronizing the time with each imaging device 120 in the device synchronization process described above has elapsed. In addition, as shown in the first synchronous playback example described above, the synchronous playback setting information includes the elapsed time from the reference time to the time when shooting was started in each imaging device 120, as well as the playback start reference time and playback exclusion time set based on the signal delay time in the device synchronization process and transmitted from the host device 200 to each imaging device 120. The arithmetic circuit 130 of each imaging device 120 sets the playback start reference time and playback exclusion time for each video data based on the acquired synchronous playback setting information (step S226).

The process of setting the playback start reference time and playback exclusion time of the video data in each imaging device 120 based on the synchronous playback setting information associated with each video data (steps S224 and S226) is not limited to being executed with the reception of a playback start instruction from the host device 200 (step S222) as a trigger, but may be executed automatically after the video data acquisition process in each imaging device 120 is completed.

Next, the arithmetic circuit 130 excludes the playback output of data from the first frame of each video data up to the playback start reference time (step S228), and then simultaneously starts the playback output of each video data from the frame of the playback start reference time (step S230), so that each video data is simultaneously displayed (synchronous playback) on the display means connected to each imaging device 120 (step S232).

Then, the user US operates the operation unit 224 of the host device 200 to instruct each imaging device 120 to end playback of the video data, and the host device 200 simultaneously transmits a control signal (playback end instruction) to each imaging device 120 to instruct them to end playback (step S204). When the arithmetic circuit 130 of each imaging device 120 receives the playback end instruction transmitted from the host device 200 (step S234), or when the video data is played up to the final frame in each imaging device 120, it ends the playback output of the video data to the display means (step S236).

A specific example of this embodiment will be described below with reference to the time chart of FIG.
10, in the control method for a data acquisition device having synchronous playback processing of video data according to this embodiment, first, in the device synchronization processing, similarly to the first synchronous playback example described above, the arithmetic circuit 230 of the host device 200 extracts the longest signal delay time T H-B from among the signal delay times T _H-A , T _H-B , T _H-C , ... in the imaging devices 120A, 120B, 120C, ... calculated based on the communication time acquired when synchronizing the time with each imaging device 120. Then, the arithmetic circuit 230 sets the shooting start time T _B2 of the video data in the imaging device 120B having this signal delay time T _{H-B as} the playback start reference time T _max . In addition, the calculation circuit 230 calculates the difference (T _max - _{T A3} , T _max - T _B2 , T _max - T _C1 , ...) between the shooting start time T _A3 , T _B2 , T _C1 , ... of each video data and the playback start reference time T _max , and sets this as the playback exclusion time T _skp to be applied when playing back each video data.

Here, as described above, the playback start reference time _Tmax is defined by the longest signal delay time _TH-B , and the playback exclusion time _Tskp for each video data is calculated from the difference between the longest signal delay time _TH-B that defines the playback start reference time _Tmax and the signal delay times _TH-A , _TH-B , _TH-C , ... for each imaging device 120. The calculated playback exclusion time _Tskp for each video data is transmitted from the host device 200 to each imaging device 120 together with the playback start reference time _Tmax , and is stored in association with each video data acquired in the video data acquisition process as synchronous playback setting information.

Then, in the synchronous playback process of video data, when each of the imaging devices 120A, 120B, 120C, ... receives a playback start instruction simultaneously transmitted from the host device 200 after the passage of each of the signal delay times T _H-A , T _H-B , T _H-C , ..., the arithmetic circuit 130 of each of the imaging devices 120 acquires the playback start reference time T _max and the playback exclusion time T _skp associated with each of the video data as synchronous playback setting information.

10, the arithmetic circuit 130 of each image capture device 120 sets the time when the playback start instruction is received from the host device 200 as the shooting start time (i.e., the first frame) of each video data, excludes (fast forwards or skips) the playback output of data for the playback exclusion time T _skp set for each from the first frame, and simultaneously starts playback output of each video data from the frame of the playback start reference time T _max , as in the first synchronous playback example described above. That is, the time when the longest signal delay time T _H-B has elapsed since the transmission of the playback start instruction by the host device 200 (corresponding to the time when the total time of the signal delay times T _H-A , T _H-B , T _H-C , ... and the playback exclusion time T _skp in each image capture device 120 has elapsed) is set as the common playback start reference time T _max for each video data, and playback output is started from the frame of the playback start reference time T _max .

In this manner, in this embodiment, as in the first synchronous playback example described above, synchronous playback setting information including the playback start reference time and playback exclusion time when playing and outputting video data is generated and stored in association with each video data acquired in the video data acquisition process, so that the video data stored in each imaging device can be played and output simultaneously from the frame of the playback start reference time based on the synchronous playback setting information. Therefore, a simple method can be used to suppress the effects of signal delays that differ for each imaging device (data acquisition device) and to perform good synchronous playback of multiple video data.

In addition, in this embodiment, when the host device controls the video data acquisition process and synchronous playback process in multiple imaging devices, the signal delay time of each imaging device 120 is taken into consideration. Therefore, by simply operating the host device and sending various instructions simultaneously, the effect of signal delay in each imaging device can be eliminated and the video data stored in each imaging device can be played back in good synchronization.

(3) Third Synchronous Playback Example Fig. 11 is a time chart showing a third example of a control method for the data acquisition device according to this embodiment. Here, the explanation of the control method equivalent to the second synchronous playback example described above will be simplified.

The third example of the synchronous playback process of video data according to this embodiment executes a video data acquisition process in response to an instruction from a host device for data acquisition, and outputs the video data acquired by each imaging device 120 to a display means connected to each imaging device 120 in response to an instruction from another host device for playback, thereby performing synchronous playback of each video data. Here, the display means has a configuration equivalent to that of the second synchronous playback example described above.

In the control method of the data acquisition device having the synchronous playback process of video data according to this embodiment, as shown in the time chart in Fig. 11, an arithmetic circuit (conveniently referred to as "arithmetic circuit 230A") of a data acquisition host device (conveniently referred to as "host device 200A") such as a smartphone 220A executes device synchronization processing in the same manner as in the second synchronous playback example described above. As a result, the arithmetic circuit 230A calculates signal delay times T _H-A , T H-B , T _{H-C ,} ... between the host device 200A and each of the imaging devices 120A, 120B, 120C, ... based on the communication time acquired when synchronizing the time with each imaging device 120.

Next, the arithmetic circuit 230A sets the shooting start time T B2 of the moving image data in the imaging device 120B having the longest signal delay time T _H-B among these signal delay times T _H-A , T _{H -B , T H-C} , ... as a playback start reference time T _max common to the moving image data acquired by each imaging device 120. In addition, the arithmetic circuit 230A calculates the differences T max -T _{H -A} , T _max -T H-B , T _max -T _H-C , ... between the signal delay time T H-B that defines the playback start reference time T _{max and the signal delay times T H- A} , _{T H-B} , _{T H-C ,} ... in each imaging device 120A, 120B, 120C, ..., and sets them as a first playback exclusion time T _skpA to be applied when playing back each moving image data. The playback start reference time _Tmax and the first playback exclusion time _TskpA are transmitted from the host device 200A to each imaging device 120, and are stored in association with each video data acquired in the video data acquisition process as synchronous playback setting information.

Next, in the video data synchronous playback process, the video data acquired by each imaging device 120 is synchronously played back in accordance with the following control method, in response to instructions from a playback host device (for convenience, referred to as "host device 200B"), such as a personal computer 220B, which is different from the host device 200A for data acquisition described above.

When the user US operates the host device 200B to instruct the start of playback of the video data acquired by each of the imaging devices 120, the arithmetic circuit of the host device 200B (for convenience, referred to as the "arithmetic circuit 230B") first connects the host device 200B and each of the imaging devices 120 by wireless communication, and synchronizes the time between the host device 200B and each of the imaging devices 120, in the same manner as the device synchronization process in the host device 200A described above. Then, the arithmetic circuit 230B calculates the signal delay times T _P-A , T _P-B , T _{P-C ,} ... between the host device 200B and each of the imaging devices 120A, 120B, 120C, ... based on the communication time acquired at that time.

Next, the calculation circuit 230B extracts the longest signal delay time T _P-C (=T _maxp ) from these signal delay times T _P-A , T _P-B , T _P- C, ..., calculates the differences T _maxp -T _P-A , T maxp -T _{P-B, T maxp -T P-C, ... from the signal delay times T P-A, T P- B , T P -C} , ... in each of the imaging devices 120A, 120B, 120C, _... , and sets them as a second playback exclusion time T _skpB to be applied when playing back each video data. The second playback exclusion time T _skpB is transmitted from the host device 200B to each imaging device 120, and is stored in association with each piece of video data acquired in the video data acquisition process as synchronous playback setting information, together with the above-mentioned playback start reference time T _max and the first playback exclusion time T _skpA .

Next, as shown in the time chart of FIG. 11, the arithmetic circuit 230B simultaneously transmits a playback start instruction from the host device 200B to each imaging device 120, and when each imaging device 120A, 120B, 120C, ... receives the playback start instruction after the elapse of each signal delay time T _P-A , T _P-B , T _P-C , ..., the arithmetic circuit 130 of each imaging device 120 acquires the playback start reference time T _max and the second playback exclusion time T _skpB associated with each video data as synchronous playback setting information.

As a result, the arithmetic circuit 130 of each image capture device 120, like the second synchronous playback example described above, sets the time when the playback start instruction is received from the host device 200B as the shooting start time (i.e., the first frame) for each video data, excludes (fast-forwards or skips) the playback output of the data for the playback exclusion time T _skpB set for each from the first frame, and simultaneously starts playback output of each video data from the frame of the playback start reference time T _max . That is, the time when the longest signal delay time T _P-C has elapsed since the transmission of the playback start instruction by the host device 200B (corresponding to the time when the total time of the signal delay times T _P-A , T _P-B , T _P-C , ... and the playback exclusion time T _skpB in each image capture device 120 has elapsed) is set as the common playback start reference time T _max for each video data, and playback output is started from the frame of the playback start reference time T _max .

In this way, in this embodiment, as in the second synchronous playback example described above, the video data of each imaging device 120 can be simultaneously played back from the frame of the playback start reference time based on the synchronous playback setting information stored in association with the video data acquired by each imaging device 120. Therefore, even if any host device is used for playing back video data, a simple method can be used to suppress the effects of signal delays that differ for each imaging device (data acquisition device) and to play back multiple video data in a good synchronous manner.

In addition, in this embodiment, when the host device controls the video data acquisition process and synchronous playback process in multiple imaging devices, the signal delay time of each imaging device 120 is taken into consideration. Therefore, even if any host device is used, the effect of signal delay in each imaging device can be eliminated by a simple method of simply sending various instructions simultaneously, and the video data stored in each imaging device can be played back synchronously with good results.

In this embodiment, in the device synchronization process and the video data acquisition process, synchronous playback setting information (playback start reference time T _max , first playback exclusion time T _skpA ) based on the signal delay time between the data acquisition host device 200A and each imaging device 120 is calculated, and stored in association with the video data acquired by each imaging device 120. After that, in the synchronous playback process, synchronous playback setting information (second playback exclusion time T _skpB ) based on the signal delay time between the playback host device 200B and each imaging device 120 is calculated, and further associated with and stored in association with each of the above video data, and a case has been described where each video data is synchronously played back based on this synchronous playback setting information. The present invention is not limited to this, and for example, when the host devices for data acquisition and playback are specified in advance, in the device synchronization process and the video data acquisition process, synchronous playback setting information (playback start reference time T _max , first playback exclusion time T _skpA , second playback exclusion time T _skpB ) based on the signal delay time between both the host devices for data acquisition and playback 200A, 200B and each imaging device 120 is calculated, and stored in association with the video data acquired by each imaging device 120. After that, in the synchronous playback process using the host device for playback 200B, each video data may be synchronously played back based on this synchronous playback setting information.

(Modification)
Next, a modified example of the control method for the data acquisition device according to this embodiment will be described.
Fig. 12 is a time chart showing a modified example of the control method (video data acquisition process) of the data acquisition device according to the present embodiment, and Fig. 13 is a time chart showing a modified example of the control method of the data acquisition device according to the present embodiment. Here, in this modified example, a case will be described in which one or more imaging devices (imaging device 120A) among the multiple imaging devices have the motion detection function as described above. In addition, the description of the control method equivalent to the embodiment including each of the above-mentioned examples will be simplified.

In a modified example of the control method for the data acquisition device according to this embodiment, in a video data acquisition process, one or more of the multiple imaging devices 120 start capturing video images before receiving a shooting start instruction transmitted from the host device 200, and the video data acquired at that time (for convenience, referred to as "stored captured data") is stored in association with synchronous playback setting information together with video data acquired after receiving the shooting start instruction. Then, in the synchronous playback process, by using the stored captured data, synchronous playback is performed by simultaneously outputting each video data to the display means from a common playback start reference time that goes back from the time when the shooting start instruction was received in each imaging device.

13, in the control method for a data acquisition device having synchronous playback processing of video data according to this modified example, first, in the device synchronization processing described above, the arithmetic circuit 230 of the host device 200 synchronizes the reference times in the image acquisition devices 120 and calculates signal delay times T _H-A , T _{H-B , T H-C , ... between the host device 200 and each of the image acquisition devices 120A, 120B} , 120C, .... Based on these reference times and signal delay times, the arithmetic circuit 230 sets the shooting start time T _A11 of the image acquisition device _120A , which starts shooting a moving image earliest in response to an instruction from the host device 200 in the video data acquisition processing, as the playback start reference time T _min . Here, the playback start reference time _Tmin is also defined by the shortest signal delay time TH-A among the signal delay times _TH-A , _TH-B , _{TH-C ,} ... in the image capture devices 120A, 120B, 120C, .... The arithmetic circuit 230 calculates the difference (TH _{-A-Tmin, TH-B-Tmin, TH-C-Tmin, ...) between the signal delay times TH-A , TH - B , TH -C ,} ... in the image capture devices 120A, 120B, 120C, ... and the shortest signal delay time _TH-A defining the playback start reference time _Tmin , and sets it as the playback addition time _Tadd to be applied when playing back each video data. The playback start reference time T _min and the playback additional time T _add (or the signal delay time and its difference that define them) are transmitted from the host device 200 to each image capture device 120 and stored therein.

Next, in the video data acquisition process, as shown in the time chart in FIG. 12, for example, the user US operates the host device 200 to instruct each imaging device 120 to start shooting, and a control signal (shooting standby instruction) instructing each imaging device 120 to wait for shooting is simultaneously transmitted to each imaging device 120, and each imaging device 120 is set to a shooting standby state.

In this standby state, when the imaging device 120A having a motion detection function detects a specific action of the user US as an event, it transmits an event notification to the host device 200, cancels the standby state, and starts imaging after measuring the passage of a predetermined time. At this time, the imaging device 120A acquires the elapsed time from the reference time to the imaging start time T _A11 , including the time lag from the event detection time T _A10 to the imaging start time T _A11 , and sets the imaging start time T _A11 to the playback start reference time T _min based on the signal delay time calculated in the device synchronization process, and associates it with the acquired video data as synchronous playback setting information and stores it. Here, the time lag from the event detection time T _A10 to the imaging start time T _A11 is set to be approximately the same as the time from the time when the host device 200 transmits an instruction to start imaging to the time when imaging is started in the imaging device 120A (i.e., the signal delay time T _H-A in the imaging device 120A).

On the other hand, the host device 200, which has received the event notification from the imaging device 120A, simultaneously transmits a shooting start instruction to each imaging device 120 other than the imaging device 120A. When each imaging device 120 other than the imaging device 120A receives the shooting start instruction after the respective signal delay times have elapsed, as shown in Fig. 12, it starts shooting the original moving image and acquires the elapsed time from the reference time to the shooting start time (time T _B12 for the imaging device 120B). Each imaging device 120 associates the playback start reference time T _min and the playback addition time T _add , which are calculated based on the shooting start time and the signal delay time, with each moving image data and stores them as synchronous playback setting information.

13, among the imaging devices 120 that receive the shooting start instruction from the host device 200, the imaging devices 120B and 120C start shooting a tentative moving image from a time T _B11 or T _C11 that is earlier than the shooting start times T _B12 or T _C12 , and temporarily store the acquired moving image data as shot data (temporarily stored data). This shot data is stored in association with the above-mentioned synchronous playback setting information together with the original moving image data acquired after the shooting start times T _B12 or T _C12 based on the shooting start instruction from the host device 200. Here, in the imaging devices 120B and 120C, the shooting start times T _B11 or T _C11 of the shot data executed prior to the shooting start instruction from the host device 200 are set, for example, immediately after the imaging devices 120 are set to a shooting standby state or after a predetermined time has elapsed, based on the time when each imaging device 120 is set to a shooting standby state, as shown in FIG. 12 and FIG. 13. In addition, it is desirable that the shooting start times T _B11 and T _C11 of the stored data are set to be the same time (same time) as the shooting start time T _A11 (i.e., the playback start reference time T _min ) specified in the image capture device 120A upon event detection, or to be earlier than the shooting start time T _A11 . In addition, the upper limit of the storage capacity of the stored data (i.e., the time to be stored as stored data) is specified in advance for each image capture device 120. Therefore, the image capture devices 120B and 120C have an endless recording function that sequentially stores the acquired stored data in units of frames for, for example, several seconds to several tens of seconds until a shooting start instruction is received from the host device 200, and sequentially erases the oldest frames when the upper limit of the storage capacity is reached. As a result, the video data acquired as stored data is only temporarily stored in an erasable state until it is saved together with the original video data, as described above, and is not associated with synchronous playback setting information.

Next, in the synchronous playback processing of video data, as shown in the time chart in Figure 13, when each imaging device 120A, 120B, 120C, ... receives a playback start instruction sent from the host device 200 after the elapse of its respective signal delay time T _H-A , T _H-B , T _H-C , ..., it acquires the playback start reference time T _min and playback additional time T _add associated with the video data of each imaging device 120 as synchronous playback setting information.

As a result, the imaging device 120A, which has a motion detection function and which starts capturing moving images the earliest among the multiple imaging devices 120 (i.e., which has the shortest signal delay time), sets the time at which it receives the playback start instruction as the capture start time of the moving image data that defines the playback start reference time T _min set in the moving image data acquisition process, and starts playback output from the first frame of the playback start reference time T _min set in the original moving image data.

On the other hand, each of the imaging devices 120 except the imaging device 120A sets the time when the playback start instruction was received as the time when the shooting start instruction was received in the video data acquisition process (i.e., the first frame of the original video data), sets the time back from that time by the playback addition time T _add set for each of the imaging devices 120 (i.e., the difference between the signal delay time in each imaging device 120 and the shortest signal delay time T _H-A that specifies the playback start reference time T _min ) as the playback start time, and simultaneously starts playback output from the frame at the playback start reference time T _min of each of the stored data. That is, at the playback start reference time T _min common to each of the video data, which is set at the time when the shortest signal delay time T _H-A has elapsed since the host device 200 sent the playback start instruction, the imaging device 120A starts playback from the first frame of the original video data acquired in the video data acquisition process. In addition, in each imaging device 120 except imaging device 120A, playback is started from a frame at a playback start reference time T _min , which is a point that goes back by the playback addition time T _add from the first frame of the original video data, among the captured data acquired prior to the original video data acquired in the video data acquisition process.

In this manner, in this modified example, the other imaging devices 120 acquire the stored data before the start of shooting the video data acquired by the imaging device 120A, which has the shortest signal delay time with the host device 200, so that the video data acquired by each imaging device 120 and the stored data can be simultaneously played back and output after the shortest signal delay time has elapsed from the time the host device 200 sends a playback start instruction (playback start reference time). Therefore, a simple method can be used to suppress the effects of signal delays that differ between imaging devices (data acquisition devices), allowing for rapid and excellent synchronous playback of multiple video data with the shortest signal delay time.

In addition, in this modified example, the captured data acquired by each imaging device 120 can be temporarily stored by endless recording up to a specified storage capacity, so that multiple video data can be played back synchronously without putting a strain on the storage capacity of the memory unit 132 of the imaging device 120.

In this modified example, the imaging devices 120, except for the imaging device 120A having the shortest signal delay time, execute the process of acquiring stored data before receiving the shooting start instruction from the host device 200. However, all imaging devices 120, except for the imaging device 120A, may not have the function of acquiring stored data, or may not execute the process of acquiring stored data. In this case, for imaging devices 120 that do not have stored data acquired before the playback start reference time, playback output is not performed until video data exists, and playback output is started from the original shooting start time of the video data after the playback start reference time (i.e., the first frame).

In each of the above-described embodiments, a case has been described in which the elapsed time, playback start reference time, playback exclusion time, and playback addition time are set as time data as the synchronous playback setting information associated with the video data acquired in the video data acquisition process. However, the present invention is not limited to this, and instead of time data, it is also possible to apply, for example, information indicating the elapsed time, playback start reference time, playback exclusion time, and playback addition time based on the number of frames or frame position of the video data.

In addition, in each of the above-mentioned embodiments, the host device 200 and the imaging device 120 have been described as having separate functions, but a specific imaging device among a plurality of imaging devices may have the function of a host device. In this case, the imaging device having the function of a host device performs synchronization processing and operation control with the other imaging devices, and video data acquired by the imaging device having the function of a host device can be synchronously played back together with video data acquired by the other imaging devices, as in the above-mentioned embodiments. In this case, the number of devices constituting the data acquisition device can be reduced to provide a simple configuration.

In addition, the present invention is not limited to the case where the movement of the object is captured as a video, and various sensor devices may be applied in addition to the above-mentioned imaging device 120 as a data acquisition device, or instead of the imaging device 120. In the present invention, sensor devices that can acquire parameters related to the movement and state of the object as numerical data (sensor data, position data, etc.), such as motion sensors that detect acceleration, angular velocity, tilt, etc., biosensors that detect heart rate, body temperature, myoelectric potential, etc., and GPS receivers that detect geographical position, movement speed, etc., can be applied. In this case, by performing processing equivalent to the above-mentioned embodiment in a state where the sensor device is attached to the object (for example, in a state where the user US wears it directly on the waist, limbs, head, chest, etc., or in a state where it is attached to sports equipment such as clothing, shoes, golf clubs, and tennis rackets), it is possible to acquire multiple types of numerical data, and the video data and numerical data acquired by the above-mentioned imaging device 120 in a state where they can be played back synchronously. This allows multiple types of numerical data, or video and numerical data to be linked together and played back on a display device or analytical device, making it possible to accurately grasp and analyze the movements and condition of the body during exercise, helping to learn proper form and improve skills.

In addition, in each of the above-described embodiments, a case has been described where a golf swing performed by a user US, who is a subject (object), is captured as a video, but the present invention is not limited to this, and may also capture the movements of other subjects, such as a tennis or baseball swing, running form, etc.

Although several embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and includes the inventions described in the claims and their equivalents.
The invention as originally claimed in the present application is set forth below.

(Additional Note)
[1]
A plurality of data acquisition devices for acquiring data;
a control device connected to each of the plurality of data acquisition devices via an individual communication path and controlling the plurality of data acquisition devices;
Equipped with
the control device calculates a signal delay time that occurs when communicating with each of the plurality of data acquisition devices via the communication path, generates setting information including a common reference time when each of the data acquired by the plurality of data acquisition devices is synchronously played and output based on the signal delay time, and transmits the setting information to each of the plurality of data acquisition devices;
the plurality of data acquisition devices receive the setting information transmitted by the control device, acquire the data, and store the acquired data in association with the setting information;
A data acquisition device comprising:

[2]
The data acquisition device according to claim 1, wherein the data acquired by the multiple data acquisition devices is each played back and output starting from the data position of the common reference time included in the setting information.

[3]
The data acquisition device according to claim 2, wherein the data acquired by the plurality of data acquisition devices excludes playback output from a beginning position of the data to a data position at the reference time.

[4]
The data acquisition device according to claim 3, wherein the reference time is set to the beginning of the data acquired by the data acquisition device having the longest signal delay time among the plurality of data acquisition devices.

[5]
The data acquisition device described in [2] is characterized in that at least one of the multiple data acquisition devices acquires temporarily stored data prior to the data acquisition instruction from the control device and before the data acquired based on the data acquisition instruction, and playback output of the data including the temporarily stored data begins from the data position of the reference time in the temporarily stored data acquired before the starting position of the acquired data.

[6]
The data acquisition device according to [5], characterized in that the reference time is set to the beginning position of the data acquired by the data acquisition device having the shortest signal delay time among the plurality of data acquisition devices.

[7]
The data acquisition device described in any of [2] to [6] is characterized in that the control device collects the data acquired by the multiple data acquisition devices, and based on a playback output instruction, plays and outputs each of the collected data in synchronization with the data position of the reference time.

[8]
the control device transmits a playback output instruction to the plurality of data acquisition devices;
The data acquisition device according to any one of [2] to [6], characterized in that the plurality of data acquisition devices, based on the playback output instruction, play and output each of the acquired data in synchronization with the data position of the reference time.

[9]
The data acquisition device according to any one of [1] to [8], wherein a specific data acquisition device among the plurality of data acquisition devices has a function as the control device.

[10]
the control device includes a data acquisition control device that controls the plurality of data acquisition devices to acquire the data, and a playback control device that plays and outputs each of the data acquired by the plurality of data acquisition devices,
The control device for playback includes:
excluding the reproduction output of the data acquired by the plurality of data acquisition devices for a predetermined time based on the signal delay time included in the setting information generated by the control device for reproduction;
The data acquisition device according to [1], characterized in that playback output of each of the data is started from the data position of the reference time included in the setting information generated by the control device for the data acquisition.

[11]
The data acquisition device according to any one of [1] to [10], characterized in that the multiple data acquisition devices include at least one of an imaging device that continuously captures the movement of an object, or a sensor device that continuously acquires parameters related to the movement or state of the object.

[12]
A method for controlling a data acquisition device including a plurality of data acquisition devices that acquire data, and a control device that is connected to each of the plurality of data acquisition devices via an individual communication path and controls the plurality of data acquisition devices, comprising:
In the control device,
calculating a signal delay time that occurs when communicating with each of the plurality of data acquisition devices via the communication path;
generating setting information including a common reference time when each of the data acquired by the plurality of data acquisition devices is synchronously reproduced and output based on the signal delay time, and transmitting the setting information to each of the plurality of data acquisition devices;
In the plurality of data acquisition devices,
receiving the setting information transmitted by the control device, acquiring the data, and storing the acquired data in association with the setting information;
23. A method for controlling a data acquisition device comprising:

[13]
The control method for a data acquisition device described in [12], characterized in that the data acquired by the multiple data acquisition devices each starts to be played back from the data position of the common reference time included in the setting information.

[14]
A control program for a data acquisition device including a plurality of data acquisition devices for acquiring data, and a control device connected to each of the plurality of data acquisition devices via an individual communication path and controlling the plurality of data acquisition devices,
The computer of the control device
calculating a signal delay time occurring when communicating with each of the plurality of data acquisition devices via the communication path;
generating setting information including a common reference time when each of the data acquired by the plurality of data acquisition devices is synchronously reproduced and output based on the signal delay time, and transmitting the setting information to each of the plurality of data acquisition devices;
A computer of the plurality of data acquisition devices,
receiving the setting information transmitted by the control device, acquiring the data, and storing the acquired data in association with the setting information;
A control program for a data acquisition device.

[15]
The control program for the data acquisition device described in [14], characterized in that the data acquired by the multiple data acquisition devices each starts to be played back from the data position of the common reference time included in the setting information.

[16]
A control device that is connected to each of a plurality of data acquisition devices via an individual communication path and controls the plurality of data acquisition devices,
a calculation means for calculating a signal delay time that occurs when communication is performed between each of the plurality of data acquisition devices via the communication path;
a generating means for generating setting information including a common reference time when each of the data acquired by the plurality of data acquisition devices is synchronously reproduced and output based on the signal delay time;
a transmitting means for transmitting the setting information to each of the plurality of data acquisition devices;
A control device comprising:

[17]
A data acquisition device that is connected to the control device via a communication path and acquires data,
the control device calculates a signal delay time that occurs when communicating with each of a plurality of data acquisition devices including the data acquisition device via the communication path, generates setting information including a common reference time when each of the data acquired by the plurality of data acquisition devices is synchronously played and output based on the signal delay time, and transmits the setting information to each of the plurality of data acquisition devices;
The data acquisition device is:
a receiving means for receiving the setting information transmitted by the control device;
A data acquisition means for acquiring the data;
a storage means for storing the acquired data in association with the setting information;
A data acquisition device comprising:

100 Device group 120 Imaging device (data acquisition device)
120A, 120B, 120C Imaging device 122 Photographing unit (data acquisition means)
128 Communication function unit (receiving means)
130 Arithmetic circuit 132 Memory section (storage means)
200 Host device (control device)
226 Display unit 228 Communication function unit (transmission means)
230 Arithmetic circuit (calculation means, generation means)
US User

Claims (1)

A plurality of imaging devices for capturing video data;
a control device connected to each of the plurality of image capture devices via an individual communication path and controlling the plurality of image capture devices;
Equipped with
The control device includes:
a signal delay time occurring when communicating with each of the plurality of imaging devices via the communication path is calculated prior to the imaging by transmitting a predetermined transmission signal via the communication path corresponding to each of the plurality of imaging devices and receiving a response signal to the transmission signal,
generating setting information based on the signal delay time for setting a common reference time when each of the moving image data acquired by each of the plurality of image capture devices is synchronously played back and output, and transmitting the setting information to each of the plurality of image capture devices;
The plurality of imaging devices include
receiving the setting information transmitted by the control device;
When the moving image data is acquired by the imaging , the setting information is associated with the acquired moving image data and stored.
A data acquisition device comprising:

Images