JP6485238B2 - Tagging device, tagging system, tagging method and program - Google Patents

Description

  The present invention relates to a tagging device, a tagging system, a tagging method, and a program.

  Tagging (tagging) the moving image data, the moving image data is edited later, or only the tagged frames in the moving image data are recorded. For example, Patent Document 1 describes that in an image recording apparatus, only a frame in which a flag F1 is set based on biometric information, that is, a frame tagged based on biometric information is recorded in a recording unit.

JP 2014-192755 A

In the image recording apparatus according to the background art, the importance of moving image data can be clarified by tagging, but the number of frames is fixed, and important frames may be lost.
There is a tagging method in which a user presses a button or the like to tag during recording or viewing, but with such a method, the range from where to where is important can be clarified. There wasn't.

  The present invention has been made to solve such a problem, and a tagging device, a tagging system, a tagging method, and a program capable of accurately setting a tag range and attaching a tag to an important scene of moving image data. The purpose is to provide.

Accordingly, the present invention provides an image capturing unit that captures an image of a subject and outputs moving image data, and a data communication unit that continuously receives vital data of a user who records or reproduces moving image data and receives timing data of tag input of the user. And a tag range processing unit that sets the time range as a tag range when the time range corresponding to the timing data is included in a time range in which the vital data received by the data communication unit is equal to or greater than a preset value. A tagging device is provided.
The present invention also provides a data communication step for continuously receiving vital data of a user who performs recording or reproduction of moving image data and receiving timing data of tag input of the user, and vital data received in the data communication step is set in advance. And a tag range processing step for setting the time range as a tag range when the time range corresponding to the timing data is included in the time range equal to or greater than the set value.

  In addition, the present invention provides a data communication procedure for continuously receiving vital data of a user who performs recording or reproduction of moving image data to a computer and receiving timing data of tag input of the user, and vital data received by the data communication procedure. When a time range corresponding to the timing data is included in a time range that is equal to or greater than a preset value, a program for executing a tag range processing procedure for setting the time range as a tag range is provided.

  According to the present invention, it is possible to provide a tagging device, a tagging system, a tagging method, and a program for accurately setting a tag range in an important scene of moving image data and attaching a tag.

1 is a block diagram showing a schematic configuration of a tagging system 1 according to an embodiment. It is a figure which shows the relationship between the time change of the heart rate which concerns on embodiment, and a tag range. It is a figure which shows the detailed relationship between the time change of the heart rate which concerns on embodiment, and a tag range. It is a flowchart which shows the detailed process sequence of the tag range process part 50 which concerns on embodiment.

  Hereinafter, a tagging system according to the present embodiment will be described with reference to the drawings. The tagging system according to the present embodiment uses the vital data (biological information) of the user and the tag timing data input in an important scene when the user is shooting a sports game or the like. The tag range is set with high accuracy to the important range of the photographing data and the important frame.

First, the configuration of the tagging system according to the present embodiment will be described.
FIG. 1 is a block diagram showing a schematic configuration of a tagging system 1 according to the present embodiment.
The tagging system 1 includes a sensor device 2, a tag input device 3, an imaging device 4, and the like. Communication between the sensor device 2 and the imaging device 4 and between the tag input device 3 and the imaging device 4 can be performed by wire or wireless.

  The sensor device 2 measures a heart rate, which is vital data of the user, when a user (not shown) is photographing with the imaging device 4. The sensor device 2 includes data communication that transmits the measured heart rate to the imaging device 4 in real time in addition to the configuration of a general sensor device such as the sensor main body 21, the sensor control unit 22, the sensor data processing unit 23, and the CPU 24. The unit 25 is provided.

  The tag input device 3 is a tag that is pressed by pressing a button or the like at the moment when the user is shooting with the imaging device 4 and thinks that it is better to tag the shooting data, for example, at the goal of a soccer game. Is input. The tag input device 3 includes a key input unit 31 such as a button and a screen, a key control unit 32, an input tag data processing unit 33, a configuration provided in a general tag input device such as a CPU 34, and a tag input by a user. A data communication unit 35 that transmits input tag data, which is data, to the imaging device 4 in real time is provided.

  The imaging device (tagging device) 4 can capture a moving image such as a video camera or a digital camera. The imaging device 4 includes a data communication unit 47, sensor data, in addition to a configuration provided in a general imaging device such as an image sensor 41, an imaging unit 42, an image data processing unit 43, a data output unit 44, a recording medium 45, and a CPU 46. A recording unit 48, a tag control unit 49, a tag range processing unit 50, a tag recording unit 51, and the like are provided.

The data communication unit 47 receives sensor data transmitted by the data communication unit 25 of the sensor device 2, input tag data transmitted by the data communication unit 35 of the tag input device 3, and the like. In addition, the data communication unit 47 may transmit a signal for controlling the operation start timing to the sensor device 2, the tag input device 3, and the like. That is, the data communication unit 25 of the sensor device 2 and the data communication unit 35 of the tag input device 3 communicate with the data communication unit 47 of the imaging device 4 in one direction or in both directions such as a control signal and a status notification signal. Also good.
The sensor data recording unit 48 records sensor data received from the sensor device 2.
The tag control unit 49 controls the entire tag providing operation (tagging method) in the imaging device 4.

The tag range processing unit 50 uses the received sensor data and input tag data, and an appropriate range of the moving image data captured by the imaging device 4, for example, at a timing when the tag is recorded, vital data exceeds a preset threshold value If it is, a period that is equal to or greater than the threshold is set in the tag range. The tag range processing unit 50 can set such a tag range during shooting. Furthermore, the tag range may be set when playing back a moving image.
The tag recording unit 51 records the received sensor data, input tag data, and set tag range in association with moving image data during shooting or during moving image reproduction. The tag recording unit 51 adds a tag within the set tag range and records the timing data.
The received sensor data, input tag data, set tag range data, and moving image data associated therewith may be recorded on the recording medium 45.

The input tag data includes, for example, the time when the user inputs the tag and the time when the imaging device 4 receives the tag input. Even if it is not the time, if the sensor device 2, the tag input device 3, and the imaging device 4 can share some synchronized timing such as the start of recording, the information of the elapsed time from that point can be substituted.
If the sensor device 2, the tag input device 3 and the imaging device 4 have a built-in clock, the sensor device 2 and the tag input device 3 transmit the measured heart rate and input tag data to the imaging device 4 in real time. do not have to. Later, it is possible to synchronize the timing of the moving image data, vital data, and input tag data based on the respective clock information.

  Each component realized by the imaging device 4 can be realized by executing a program under the control of the CPU 46. More specifically, the imaging device 4 is realized by loading a program stored in a storage unit (not shown) into a main storage device (not shown) and executing the program under the control of the CPU 46. Each component is not limited to being realized by software by a program, and may be realized by any combination of hardware, firmware, and software.

  The above-described program can be stored using various types of non-transitory computer readable media and supplied to a computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / W and semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)) are included.

  Further, the program may be supplied to the imaging device 4 by various types of temporary computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the imaging device 4 via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

Subsequently, an operation of the tagging system 1 according to the present embodiment, that is, a tagging method will be described. Here, the operation of the tag range processing unit 50 will be mainly described.
First, an outline of the tag range setting operation of the tag range processing unit 50 will be described. As described above, the tag range processing unit 50 uses the received sensor data and input tag data to set a tag in an appropriate range of the moving image data captured by the imaging device 4.

  FIG. 2 is a diagram showing a relationship between a change in heart rate with time and a tag range according to the present embodiment. A graph showing the input tag data in the heart rate graph is shown above, and a tag range set from the input tag data is related to the moving image data below. In the heart rate graph, the horizontal axis represents time, and the vertical axis represents heart rate. A horizontal line near the heart rate 75 indicates a threshold for setting the tag range. Further, horizontal arrows and characters such as “normal” and “tension” written in the vicinity of the heart rate 40 are for explaining the state of the user at that time.

In the example of the heart rate shown in FIG. 2, the state of the user changes from a change in heart rate from the start of the game to the end of the game to a normal state, a tension state or an excitement state, and a normal (relaxed) state. Four peaks (1-4) of the heart rate occur in the tension state or the excitement state, and the user inputs the tags F1-F4 at each peak.
The tag range processing unit 50 is a time range when the heart rate is equal to or higher than a predetermined value (here, 75 as a threshold), and when the user inputs the tag F within the time range, the time range Is set as the range of the tag F input by the user. A tag F may be attached to a frame in the time range.

  For example, for the first peak, since the heart rate exceeds a predetermined value in the time range t1 to t2, and the user inputs the tag F1, the time range t1 to t2 is set as the tag range of the tag F1. To do. For the second peak, since the heart rate exceeds a predetermined value in the time range t3 to t4 and the user inputs the tag F2, the time range t3 to t4 is set as the tag range of the tag F2. To do. The same applies to the third and fourth peaks.

Note that the tag range processing unit 50 does not set the tag range even if the heart rate is a time range equal to or greater than a predetermined value if the user does not input the tag F within the time range.
Moreover, even if the user inputs the tag F, if the heart rate is not equal to or greater than the predetermined value, the tag range processing unit 50 sets the tag range by default or the time or the number of frames set by the user.

Next, a detailed processing procedure of the tag range processing unit 50 will be described.
FIG. 3 is a diagram showing a detailed relationship between the time change of the heart rate and the tag range according to the present embodiment. The heart rate graph is shown at the top and the tag range at the bottom. Horizontal dotted lines in the graph indicate threshold values, and equally spaced points on the horizontal dotted line indicate heart rate sampling points, that is, sampling times. The heart rate is sampled at intervals of 1 second, for example.
FIG. 4 is a flowchart showing a detailed processing procedure of the tag range processing unit 50 according to the present embodiment.
A detailed processing procedure of the tag range processing unit 50 will be described with reference to FIGS. 3 and 4.

  First, when processing is started, the status indicating the status of vital data is set to “below threshold” (step S10), and the “previous vital data time value” is initialized to the recording start time t0 (step S20).

At the sampling time t0, it is determined whether the status is “threshold or more” (step S30). At time t0, the status remains “less than threshold value” set in step S10 (No in step S30), so it is determined whether there is vital data received at time t0 (step S40).
Since there is vital data received at time t0 (Yes in step S40), it is determined whether the value of the vital data at time t0 is equal to or greater than a threshold value (step S50). As shown in FIG. 3, since the value of the vital data at time t0 is less than the threshold value (No in step S50), the current vital data time t0 is stored as the “previous vital data time value” (step S80). Then, the process returns to step S30. Until t11, since the normal state, that is, the vital data is less than the threshold value, the same processing is repeated.

  At the sampling time t12, since the status is “less than threshold” and there is vital data at t12 (No at step S30, Yes at step S40), it is determined whether the value of the vital data at time t12 is equal to or greater than the threshold (step S50). . As shown in FIG. 3, since the value of the vital data at time t12 is equal to or greater than the threshold (Yes in step S50), the “previous vital data time value”, here, the time t11 stored in step S80 is “tag range start time value”. (Step S60), the status is set to "above threshold" (step S70), the current vital data time t12 is saved as "previous vital data time value" (step S80), and the process returns to step S30. .

Next, at the sampling time t13, since the status is “above threshold value” as set in step S70 (Yes in step S30), it is determined whether there is vital data at time t13 (step S90).
Since there is vital data received at time t13 (Yes in step S90), it is determined whether the value of vital data at time t13 is equal to or greater than a threshold value (step S100). As shown in FIG. 3, since the value of the vital data at time t13 is equal to or greater than the threshold value (Yes in step S100), the current vital data time t13 is stored as “previous vital data time value” (step S80), and the process proceeds to step S30. Return.
Until the sampling time t21, the same processing as that at time t13 is continued.

  Next, at the sampling time t22, since the status is “above threshold” and there is vital data received at time t22 (Yes at step S30, Yes at step S90), is the vital data value at time t22 greater than or equal to the threshold? Judgment is made (step S100). As shown in FIG. 3, since the vital data value at time t22 is less than the threshold (No in step S100), the current vital data time t22 is stored as a “tag range end time value” (step S110), and a tag recording request is made. In other words, it is determined whether a tag is input when the value of the vital data is equal to or greater than a threshold value (step S120). As shown in FIG. 3, since the user inputs the tag F between time t16 and time t17 (Yes in step S120), the “tag range start time value of the tag range obtained in steps S60 and S110” The tag F is recorded at “t11 to“ tag range end time value ”t22 (step S130), the status is set to“ less than threshold ”(step S140), and the current vital data time t22 is set to“ previous vital data time value ”. (Step S80) and return to step S30.

Next, at the sampling time t25, since the status is “less than threshold” and there is no vital data at time t25 (No in step S30, No in step S40), the process ends.
The current vital data time t is stored as a “tag range end time value” (step S110). When there is no tag recording request, that is, when the user has not input a tag (No in step S120), The status is set to “less than threshold value” (step S140). That is, as described above, even if there is a time range in which the heart rate is equal to or greater than the predetermined value, if the user does not input a tag within the time range, the time range is not set as the tag range.

If the status is “above threshold” and there is no vital data for the time t (Yes in step S30, No in step S90), the recording end time is stored as a “tag range end time value” (step S150). As in steps S120 and S130, when there is a tag recording request, the tag F is recorded within the obtained tag range and the process is terminated. When there is no tag recording request, the process is terminated as it is.
With such a configuration, the tagging system 1 according to the present embodiment can add tags to the important range with high accuracy while capturing moving image data.

In the tagging system 1 according to the present embodiment, the time t11 to the time t22 are selected as the time range in which the heart rate exceeds the threshold value, and this time range is set as the tag range. Time t12 to time t21 corresponding to the sampling time may be selected, and this time range may be set as the tag range. Of course, it is also possible to select time t11 to time t21, time t12 to time t22 as the time range, and it is also possible to select an intersection to an intersection where the heart rate exceeds the threshold value. In that case, the intersection may be calculated, and the number of moving image samplings and the number of vital data samplings may be the same.
Further, in the tagging system 1 according to the present embodiment, the heart rate threshold value for setting the tag range is fixed, but it may be set according to the situation, such as for each user or for each scene. good. Further, the threshold value may be changed at any time so that the tag range can be adjusted even after moving image data is recorded.

  Further, in the tagging system 1 according to the present embodiment, when the heart rate is a time range equal to or greater than a predetermined value and the user inputs a tag within the time range, the time range is set as the tag range. However, when the heart rate is in a time range equal to or greater than a predetermined value and the user inputs a tag within the time range, a predetermined time before and after the tag input by the user, for example, before and after the tag. Three seconds may be set as the tag range. Of course, the user may be able to select a time range in which the heart rate is equal to or greater than a predetermined value or a predetermined time before and after the tag.

  In the tagging system 1 according to the present embodiment, the heart rate is used as vital data. However, data other than the heart rate and data indicating the degree of user excitement may be used as vital data. May be set as a tag range when the vital data is a time range such as a predetermined value or less or less than a predetermined value and the user inputs a tag within the time range.

In addition, the tagging system 1 according to the present embodiment includes the sensor device 2, the tag input device 3, the imaging device 4, and the like. However, the imaging device serves as both the sensor device and the tag input device, or the imaging device. However, it may be configured to serve as either a sensor device or a tag input device. That is, the tagging system may be configured as a tagging device.
Further, the tagging system 1 according to the present embodiment is configured so that the user's vital data and input tag data when the user is shooting using the imaging device 4, that is, when the user is recording moving image data. Although the tag range is set based on the user's vitality when the user is viewing or editing a moving image using an image processing device or the like, that is, when the user is reproducing the moving image data, A tag range may be set based on the data and input tag data. For this reason, the tagging apparatus according to the present embodiment is not limited to the imaging apparatus, and may be an image processing apparatus or the like.

As described above, the tagging device 4 according to the present embodiment continuously receives the vital data of the user who records or reproduces the moving image data and receives the timing data of the tag input of the user, A tag range processing unit 50 for setting the time range as a tag range when the time range corresponding to the timing data is included in the time range in which the vital data received by the data communication unit 47 is equal to or greater than a preset value; It is to be prepared.
The tagging system 1 according to the present embodiment includes a sensor device 2 that measures user vital data and outputs the data to the data communication unit 47, a tag input device 3 that outputs timing data to the data communication unit 47, and a tagging device. 4.

  In addition, the tagging method according to the present embodiment accepts the vital data of the user who records or reproduces the video data continuously, and accepts the tag input timing data of the user at the data communication step and the data communication step. Tag range processing steps S120 and S130 for setting the time range as a tag range when the time range corresponding to the timing data is included in a time range in which the vital data is equal to or greater than a preset value. .

  In addition, the program according to the present embodiment includes a data communication procedure for continuously receiving vital data of a user who performs recording or reproduction of moving image data and receiving timing data of the tag F input of the user, and the data Step S120 for setting the time range of the video data as the range of the tag F when the time range corresponding to the timing data is included in the time range in which the vital data received in the communication procedure is equal to or greater than a preset value. This is for executing S130.

1 Tagging system 2 Sensor device 3 Tag input device 4 Imaging device (tagging device)
50 Tag range processing section

Claims (5)

A data communication unit for the vital data of a user for recording or reproduction of the moving image data together with the accepted continuously receives timing data tag input of said user,
A tag range processing unit that sets the time range as a tag range when the time range corresponding to the timing data is included in a time range in which the vital data received by the data communication unit is greater than or equal to a preset value;
Tagging device comprising: The tagging device according to claim 1, further comprising an imaging unit that images a subject and outputs moving image data . A sensor device for measuring the vital data of the user and outputting it to the data communication unit;
A tag input device that outputs the timing data to the data communication unit;
A tagging device according to claim 1 or 2,
Tagging system with A data communication step of continuously receiving vital data of a user who performs recording or reproduction of moving image data and receiving timing data of the tag input of the user;
A tag range processing step of setting the time range as a tag range when the time range corresponding to the timing data is included in a time range in which the vital data received in the data communication step is greater than or equal to a preset value;
Tagging method comprising: On the computer,
A data communication procedure for continuously receiving vital data of a user who performs recording or reproduction of moving image data and receiving timing data of the tag input of the user;
A tag range processing procedure for setting the time range as a tag range when the time range corresponding to the timing data is included in a time range in which the vital data received in the data communication procedure is equal to or greater than a preset value;
A program for running

Images