JP7779372B2 - Information processing device, information processing method, and program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and a program.

There is a need for technology that can automatically determine which process out of a set of processes is being carried out from video footage of that process being carried out.

Patent Document 1 describes a work management device that identifies work based on the characteristic quantities of a worker's work detected by a sensor and estimates the progress of the work being performed by the worker. In the process of identifying work, the work management device described in Patent Document 1 breaks down the work into multiple elemental tasks, and if a characteristic action is present in the extracted characteristic quantities, the elemental task corresponding to the characteristic task is identified. If no characteristic action is present in the extracted characteristic quantities, the task is identified as unknown.

Japanese Patent Application Publication No. 2018-163556

The work management device described in Patent Document 1 identifies component tasks based on whether or not a characteristic action is present in the extracted features, i.e., by binary input. Therefore, the work management device described in Patent Document 1 has the problem of being unable to flexibly identify tasks based on the features extracted from the video.

One aspect of the present invention has been made in consideration of the above problems, and one of its objectives is to provide a technology that flexibly determines the process represented by an image.

An information processing device according to one aspect of the present invention comprises an acquisition means for acquiring an image, a calculation means for calculating a degree of certainty as to whether the image satisfies one or more predetermined conditions, and a discrimination means for discriminating the process represented by the image based on the degree of certainty for each of the conditions and a score indicating the degree to which the condition contributes to the discrimination of the process.

An information processing method according to one aspect of the present invention includes an information processing device acquiring video, calculating a degree of certainty as to whether the video satisfies one or more predetermined conditions, and identifying the process represented by the video based on the degree of certainty for each of the conditions and a score indicating the degree to which the condition contributes to identifying the process.

A program according to one aspect of the present invention causes a computer to function as an information processing device, the program causing the computer to function as: an acquisition means for acquiring video; a calculation means for calculating a degree of certainty as to whether the video satisfies one or more predetermined conditions; and a discrimination means for discriminating the process represented by the video based on the degree of certainty for each of the conditions and a score indicating the degree to which the condition contributes to the discrimination of the process.

According to one aspect of the present invention, the process represented by the image can be flexibly determined.

1 is a block diagram showing a configuration of an information processing device according to a first exemplary embodiment of the present invention; FIG. 1 is a flowchart showing the flow of an information processing method according to a first exemplary embodiment of the present invention. FIG. 10 is a block diagram showing a configuration of an information processing device according to a second exemplary embodiment of the present invention. FIG. 10 is a flowchart showing an example of the flow of processes executed by a receiving unit and a generating unit in exemplary embodiment 2 of the present invention. FIG. 10 is a diagram showing an example of one or more predetermined elements in exemplary embodiment 2 of the present invention. FIG. 10 is a diagram showing another example of one or more predetermined elements in the second exemplary embodiment of the present invention. FIG. 10 is a flowchart showing an example of the flow of processes executed by a receiving unit and a generating unit in exemplary embodiment 2 of the present invention. FIG. 11 is a diagram showing an example of a screen for accepting input of a score in the second exemplary embodiment of the present invention. FIG. 10 is a diagram showing an example of a score table in the second exemplary embodiment of the present invention. FIG. 10 is a flowchart showing the flow of an information processing method according to a second exemplary embodiment of the present invention. FIG. 10 is a diagram showing an example of an image including a determination result displayed in the second exemplary embodiment of the present invention. FIG. 10 is a diagram showing another example of an image including a discrimination result displayed in the second exemplary embodiment of the present invention. FIG. 2 is a block diagram illustrating an example of a hardware configuration of an information processing device according to each exemplary embodiment of the present invention.

[Example Embodiment 1]
A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described below.

(Overview of information processing device 1)
The information processing device 1 according to this exemplary embodiment is a device that acquires an image and determines the process represented by the image.

The video acquired by the information processing device 1 is video captured of at least one process in a series of processes being carried out. A series of processes refers to multiple processes each performed in a predetermined order. Examples of a series of processes include, but are not limited to, a series of tasks at a construction site, a series of tasks on a production line in a factory, and exercises that involve multiple predetermined movements.

(Configuration of information processing device 1)
The configuration of an information processing device 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of an information processing device 1 according to this exemplary embodiment.

As shown in FIG. 1, the information processing device 1 is configured to include an acquisition unit 11, a calculation unit 12, and a discrimination unit 13. In this exemplary embodiment, the acquisition unit 11, the calculation unit 12, and the discrimination unit 13 are configured to respectively realize an acquisition means, a calculation means, and a discrimination means.

The acquisition unit 11 acquires the video. The acquisition unit 11 supplies the acquired video to the calculation unit 12.

The calculation unit 12 calculates the degree of certainty as to whether the video supplied from the acquisition unit 11 satisfies one or more predetermined conditions. As an example, the calculation unit 12 is a learning model that has been trained to estimate whether the video satisfies one or more predetermined conditions and calculate the degree of certainty of the estimation result. Examples of the one or more predetermined conditions include, but are not limited to, that an element is included in the video, that an element included in the video is present in a predetermined position, that an element included in the video is in a predetermined state, and that a predetermined sound is included in the video.

The specific configuration of the calculation unit 12 does not limit this exemplary embodiment, but as an example, a convolution neural network (CNN), a recurrent neural network (RNN), or a combination thereof may be used. Non-neural network models such as random forests and support vector machines may also be used.

The discrimination unit 13 discriminates the process represented by the video acquired by the acquisition unit 11 based on the confidence level for each of the conditions supplied from the calculation unit 12 and a score indicating the degree to which the condition contributes to process discrimination. The score indicating the degree to which the condition contributes to process discrimination is an index indicating the degree of relevance between the condition and the process. For example, if the score indicates a high relevance between the condition and the process, the likelihood of the process being determined to be that process increases when the condition is met. On the other hand, if the score indicates a high relevance between the condition and the process, the likelihood of the process being determined to be that process decreases even when the condition is met. For example, if the score for a certain process is higher than a predetermined threshold, the discrimination unit 13 discriminates that the video represents that process.

As described above, the information processing device 1 according to this exemplary embodiment is configured to include an acquisition unit 11 that acquires video, a calculation unit 12 that calculates the degree of certainty as to whether the video satisfies one or more specified conditions, and a discrimination unit 13 that discriminates the process represented by the video based on the degree of certainty for each condition and a score indicating the degree to which the condition contributes to the discrimination of the process.

Accordingly, the information processing device 1 according to this exemplary embodiment can distinguish processes based on the degree of certainty and the score. For example, when the information processing device 1 determines whether a certain video represents process A, if the score indicates that condition A has a low degree of contribution to the determination of process A, the information processing device 1 may determine that the certain video does not represent process A even if the degree of certainty for condition A is high. On the other hand, if the score indicates that condition B has a high degree of contribution to the determination of process A, the information processing device 1 may determine that the certain video represents process A even if the degree of certainty for condition B is low. In other words, the information processing device 1 has the advantage of being able to flexibly distinguish processes represented by video.

(Flow of information processing method S1)
The flow of the information processing method S1 according to this exemplary embodiment will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the information processing method S1 according to this exemplary embodiment.

(Step S11)
In step S11, the acquisition unit 11 acquires an image. The acquisition unit 11 supplies the acquired image to the calculation unit 12.

(Step S12)
In step S12, the calculation unit 12 calculates a degree of certainty as to whether or not the video supplied from the acquisition unit 11 in step S11 satisfies one or more predetermined conditions. The calculation unit 12 supplies the calculated degree of certainty to the determination unit 13.

(Step S13)
In step S13, the discrimination unit 13 discriminates the process represented by the video acquired by the acquisition unit 11 in step S11 based on the degree of certainty for each of the conditions supplied from the calculation unit 12 in step S12 and the score indicating the degree to which the condition contributes to the discrimination of the process.

As described above, in the information processing method S1 according to this exemplary embodiment, the acquisition unit 11 acquires video in step S11, the calculation unit 12 calculates the degree of certainty as to whether the video satisfies one or more predetermined conditions in step S12, and the discrimination unit 13 discriminates the process represented by the video in step S13 based on the degree of certainty for each of the conditions and a score indicating the degree to which the condition contributes to the discrimination of the process. Therefore, the information processing method S1 according to this exemplary embodiment achieves the same effects as the information processing device 1 described above.

Exemplary Embodiment 2
A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are denoted by the same reference numerals, and their description will be omitted as appropriate.

(Configuration of information processing device 2)
The configuration of the information processing device 2 according to this exemplary embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the information processing device 2 according to this exemplary embodiment.

As shown in Figure 3, the information processing device 2 has a communication unit 21, an input unit 22, an output unit 23, a display panel 24, a memory unit 25, and a control unit 26.

The communication unit 21 is a communication module that communicates with other devices via a network (not shown). As an example, the communication unit 21 outputs data supplied from the control unit 26 (described below) to other devices via the network, and acquires data output from other devices via the network and supplies the data to the control unit 26.

The specific configuration of the network through which the communication unit 21 communicates with other devices does not limit this embodiment, but as an example, a wireless LAN (Local Area Network), a wired LAN, a WAN (Wide Area Network), a public line network, a mobile data communication network, or a combination of these networks can be used.

The input unit 22 is an interface that acquires data from other connected devices. As an example, the input unit 22 acquires input information indicating video and operation input received from the user, and supplies this to the control unit 26.

The output unit 23 is an interface that outputs data to other connected devices. As an example, if a display device is connected to the output unit 23, the output unit 23 outputs video information indicating the video supplied from the control unit 26 to the display device.

The display panel 24 is a device that displays images. As an example, the display panel 24 acquires image information supplied from the control unit 26 and displays the image indicated by the image information.

The memory unit 25 stores data referenced by the control unit 26. Examples of data stored in the memory unit 25 include video, input information, a score table, and a discrimination result indicating the discrimination result. The video and input information are as described above. The score table will be described later.

(Functions of the control unit 26)
The control unit 26 controls each component of the information processing device 2. As shown in FIG. 3 , the control unit 26 also functions as an acquisition unit 11, a reception unit 261, a generation unit 262, a calculation unit 12, a determination unit 13, and a display unit 263. In this exemplary embodiment, the acquisition unit 11, the calculation unit 12, and the determination unit 13 are configured to realize an acquisition means, a calculation means, and a determination means, respectively. In this exemplary embodiment, the reception unit 261 is configured to realize a first reception means, a second reception means, and a third reception means. In this exemplary embodiment, the generation unit 262 is configured to realize a first generation means, a second generation means, and a third generation means.

The acquisition unit 11 acquires data supplied from the communication unit 21 or the input unit 22. Examples of data acquired by the acquisition unit 11 include video and sound related to the video. Examples of sound related to the video include sound of a specific frequency emitted by an object included in the video, and audio of specific words uttered by a person included in the video. The acquisition unit 11 stores the acquired data in the memory unit 25.

The reception unit 261 receives input from the user. As an example, the reception unit 261 receives input indicated by input information acquired from the input unit 22 as input from the user. The reception unit 261 stores the acquired input information in the memory unit 25.

An example of input from a user that the receiving unit 261 receives is input regarding one or more predetermined elements. The one or more predetermined elements are elements contained in the video, and examples include objects, actions, positions, and colors.

Another example of input from a user that the receiving unit 261 accepts is input regarding the selection of one or more predetermined conditions. Examples of one or more predetermined conditions are as described above. Yet another example of input from a user that the receiving unit 261 accepts is input of a score regarding the selected condition, which indicates the degree to which the condition contributes to process identification.

An example of the process by which the reception unit 261 receives these inputs from the user will be described later.

The generation unit 262 generates data in response to input from the user. As an example, the generation unit 262 references the input information stored in the memory unit 25 and generates one or more predetermined conditions or a score table. Here, a score table is a table that includes scores for each of one or more predetermined conditions and each score for each of multiple processes. Examples of score tables will be described later.

Furthermore, when the input information stored in the memory unit 25 indicates input regarding one or more specified elements (in other words, when the receiving unit 261 receives input regarding one or more specified elements), the generation unit 262 may generate a condition including that the video contains one or more specified elements.

The generation unit 262 stores the generated data in the memory unit 25. An example of the process by which the generation unit 262 generates one or more specified conditions or score tables will be described later.

The calculation unit 12 calculates the degree of certainty as to whether or not the video satisfies one or more predetermined conditions. As an example, the calculation unit calculates the degree of certainty as to whether or not the video stored in the memory unit 25 satisfies one or more predetermined conditions stored in the memory unit 25. An example configuration of the calculation unit 12 is as described above.

The calculation unit 12 stores the calculated confidence levels in the memory unit 25. Furthermore, when the calculation unit 12 calculates confidence levels for each of a plurality of conditions, the calculation unit 12 associates each calculated confidence level with the condition used to calculate the confidence level and stores it in the memory unit 25. The time intervals at which the calculation unit 12 calculates the confidence levels will be described later.

The discrimination unit 13 discriminates the process represented by the video based on the confidence level for each condition and a score indicating the degree to which the condition contributes to the discrimination of the process. As an example, the discrimination unit 13 refers to the confidence levels and score table stored in the memory unit 25, and calculates a score for each of one or more processes. The discrimination unit 13 then discriminates the process represented by the video based on the score. The discrimination unit 13 stores the discrimination result in the memory unit 25. An example of the process by which the discrimination unit 13 discriminates the process represented by the video will be described later.

The display unit 263 generates video information indicating the video to be displayed and supplies the video information. As an example, the display unit 263 supplies the video information generated by referring to the video stored in the memory unit 25 and the discrimination result to at least one of the output unit 23 and the display panel 24. Hereinafter, the display unit 263 supplying the video information to at least one of the output unit 23 and the display panel 24 will also be expressed as the display unit 263 displaying the video.

(Time interval for which the calculation unit 12 calculates the confidence factor)
There is no particular limitation on when the calculation unit 12 calculates the certainty factor, and one example is a configuration in which the calculation is performed at a predetermined time interval. The predetermined time interval in which the calculation unit 12 calculates the certainty factor is not particularly limited, and is preferably the same as the time interval in which the discrimination unit 13 makes a discrimination, or shorter than the time interval in which the discrimination unit 13 makes a discrimination. As one example, the calculation unit 12 may calculate the certainty factor at a time interval set by a user. As another example, the calculation unit 12 may be configured to calculate the certainty factor at a predetermined interval (one frame interval, ten minutes interval, three hours interval, etc.). As yet another example, the calculation unit 12 may change the time interval in which the calculation unit 12 calculates the certainty factor depending on the discrimination result by the discrimination unit 13.

As an example of a process in which the calculation unit 12 changes the time interval at which it calculates the confidence level depending on the determination result by the determination unit 13, the calculation unit 12 may be configured to change the time interval at which it calculates the confidence level for each process indicated by the determination result. For example, if the determination result by the determination unit 13 is "Process A," the calculation unit 12 may calculate the confidence level at one-hour intervals, and if the determination result by the determination unit 13 is "Process B," the calculation unit 12 may calculate the confidence level at five-hour intervals.

As another example of a process in which the calculation unit 12 changes the time interval at which it calculates the confidence level in accordance with the determination result by the determination unit 13, the calculation unit 12 may be configured to shorten the time interval at which it calculates the confidence level for a predetermined period of time after the process indicated by the determination result by the determination unit 13 changes. As an example, in a configuration in which the calculation unit 12 calculates the confidence level at five-hour intervals, if the determination result by the determination unit 13 changes from "Process A" to "Process B," the calculation unit 12 may shorten the time interval at which it calculates the confidence level from five hours to one hour for three hours after the determination result by the determination unit 13 becomes "Process B."

As yet another example of a process in which the calculation unit 12 changes the time interval at which it calculates the confidence level in accordance with the determination result by the determination unit 13, the calculation unit 12 may be configured to lengthen the time interval at which it calculates the confidence level if the determination result by the determination unit 13 does not change for a predetermined period of time. One example is a configuration in which, in a state in which the calculation unit 12 calculates the confidence level at one-hour intervals, if the determination result by the determination unit 13 is "Process A" for a period of one day, the calculation unit 12 lengthens the interval at which it calculates the confidence level from one hour to five hours. In this configuration, if the determination result by the determination unit 13 changes (for example, if the determination result by the determination unit 13 changes from "Process A" to "Process B"), the calculation unit 12 may return the time interval at which it calculates the confidence level to one hour.

Here, the determination unit 13 may be configured to change the time interval at which it performs the process of determining the process represented by the video, in accordance with the time interval at which the calculation unit 12 calculates the confidence level. In other words, the determination unit 13 may be configured to change the time interval at which it performs the determination process depending on the determination result by the determination unit 13. In other words, the determination unit 13 may be configured to change the time interval at which it performs the determination process in the future depending on the determination result performed in the past. With this configuration, the information processing device 2 can appropriately reduce processing depending on the determination result.

(Example of processing executed by the reception unit 261 and the generation unit 262)
An example of processing executed by the reception unit 261 and the generation unit 262 will be described with reference to Fig. 4. Fig. 4 is a flow diagram showing an example of the flow of processing S2 executed by the reception unit 261 and the generation unit 262 in this exemplary embodiment. In processing S2, the reception unit 261 receives input related to one or more predetermined elements, and the generation unit 262 generates one or more predetermined conditions.

(Step S21)
In step S21, the receiving unit 261 receives an input of a condition name. For example, when the input unit 22 receives an operation input from a user indicating that the condition name is “condition a,” the receiving unit 261 stores input information indicating that the condition name is “condition a” in the storage unit 25.

(Step S22)
In step S22, the receiving unit 261 receives a selection of a target video that is the subject of the generation conditions. As an example, when the input unit 22 receives an operation input from the user indicating that the target video is "video A," the receiving unit 261 stores input information indicating that the target video is "video A" in the storage unit 25.

(Step S23)
In step S23, the receiving unit 261 receives input related to one or more predetermined elements. An example of the process in which the receiving unit 261 receives input related to one or more predetermined elements will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of one or more predetermined elements in this exemplary embodiment.

First, the display unit 263 displays the target video selected in step S22. As an example, the display unit 263 references the input information stored in the memory unit 25 and acquires the target video indicated by the input information from the memory unit 25. The display unit 263 displays video information indicating the acquired target video on the display panel 24. For example, as shown in FIG. 5, the display unit 263 displays the target video including the object OBJ1 as an element in the region R1 on the display panel 24.

Next, the reception unit 261 receives input regarding one or more specified elements. For example, in the diagram shown in FIG. 5, the reception unit 261 acquires input information indicating that the one or more specified elements are object OBJ1 included in region R1. The reception unit 261 stores the acquired input information in the memory unit 25.

Furthermore, another example of the process in which the reception unit 261 receives input regarding one or more predetermined elements will be described using Figure 6. Figure 6 is a diagram showing another example of one or more predetermined elements in this exemplary embodiment.

For example, the display unit 263 displays a target image including object OBJ2 as an element on the display panel 24, as shown in the upper part of Fig. 6. Next, the reception unit 261 acquires input information indicating that the specified one or more elements in the target image shown in the upper part of Fig. 6 are object OBJ2 and the color of object OBJ2. For example, if the color of object OBJ2 is red, the reception unit 261 acquires input information indicating that the specified one or more elements are red object OBJ2.

(Step S24)
In step S24, the generating unit 262 generates a condition. For example, the generating unit 262 references the input information stored in the storage unit 25 by the receiving unit 261 in step S23 to generate the condition.

As an example, if the input information referenced by the generation unit 262 indicates that a specified element or elements are object OBJ1 contained in region R1 described using Figure 5, the generation unit 262 generates the object OBJ1 with the condition that the object OBJ1 is contained in region R1.

As another example of a condition generated by the generation unit 262, if the input information referenced by the generation unit 262 indicates that a specific element or elements are the red object OBJ2 described using the upper part of Figure 6, the generation unit 262 generates an image with the condition that the red object OBJ2 is included in the target image.

The generation unit 262 stores the generated conditions in the memory unit 25.

(Step S25)
In step S25, the receiving unit 261 determines whether or not input regarding other conditions has been received.

If it is determined in step S25 that input regarding another condition has been received (step S25: YES), the reception unit 261 executes step S21 again to receive input of the condition name.

On the other hand, if it is determined in step S25 that input regarding other conditions has not been received (step S25: NO), processing S2 ends.

(Another example of processing executed by the reception unit 261 and the generation unit 262)
Another example of the processing executed by the receiving unit 261 and the generating unit 262 will be described with reference to Fig. 7. Fig. 7 is a flow diagram showing an example of the flow of processing S3 executed by the receiving unit 261 and the generating unit 262 in this exemplary embodiment. In processing S3, the receiving unit 261 receives input regarding the selection of one or more predetermined conditions and input of scores regarding the selected conditions, and the generating unit 262 generates a score table.

(Step S31)
In step S31, the receiving unit 261 receives a selection of a process. As an example, when the input unit 22 receives an operation input from the user indicating that “Process A” has been selected, the receiving unit 261 stores input information indicating that the selected process is “Process A” in the storage unit 25.

(Step S32)
In step S32, the receiving unit 261 receives a selection of a condition. For example, when the input unit 22 receives an operation input from the user indicating that “condition a” has been selected, the receiving unit 261 stores input information indicating that the selected condition is “condition a” in the storage unit 25.

As another example, when the input unit 22 receives an operation input from the user indicating that "condition a" and "condition c" have been selected, the reception unit 261 stores input information in the memory unit 25 indicating that the selected conditions are "condition a" and "condition c."

(Step S33)
In step S33, the receiving unit 261 determines whether a plurality of conditions are selected in step S32.

As an example, the reception unit 261 refers to the input information stored in the memory unit 25, and if the input information indicates that the selected condition is "condition a," the reception unit 261 determines in step S33 that there are not multiple conditions selected in step S32 (step S33: NO).The reception unit 261 then executes step S36, which will be described later.

(Step S34)
As another example, the receiving unit 261 refers to the input information stored in the memory unit 25, and if the input information indicates that the selected conditions are "condition a" and "condition c," then in step S33 the receiving unit 261 determines that multiple conditions were selected in step S32 (step S33: YES).

In this case, in step S34, the reception unit 261 receives input regarding the selection of multiple conditions. As an example, if the input unit 22 receives operational input from the user indicating "condition a" and "condition c," the reception unit 261 stores input information indicating that the input regarding the selection of multiple conditions is "condition a" and "condition c" in the storage unit 25.

In addition, in step S34, the reception unit 261 may further receive a condition name as input regarding the selection of multiple conditions. As an example, if the input unit 22 receives operational input from the user indicating that the condition name of "condition a" and "condition c" is "condition α," the reception unit 261 stores input information indicating that the condition name of "condition a" and "condition c" is "condition α" in the memory unit 25.

(Step S35)
In step S35, the generating unit 262 generates new conditions based on the selected conditions. For example, the generating unit 262 generates new conditions by referring to the input information stored in the storage unit 25 by the receiving unit 261 in step S34.

As an example, if the input information referenced by the generation unit 262 indicates a condition name of "condition α" and condition contents of "condition a" and "condition c," the generation unit 262 generates a condition that the condition name "condition α" satisfies "condition a" and "condition c."

(Step S36)
In step S36, the reception unit 261 receives the input of the score. The reception unit 261 stores the received input information indicating the score in the storage unit 25. An example of the process in which the reception unit 261 receives the input of the score will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of a screen for receiving the input of the score in this exemplary embodiment.

First, the display unit 263 refers to the input information stored in the memory unit 25 and displays on the display panel 24 image information showing an image prompting the user to input the degree to which the condition selected in step S32 contributes to the determination of the process selected in step S31. As an example, if the process selected in step S31 is "process B" and the condition selected in step S32 is "condition b," an image prompting the user to input the degree to which "condition b" contributes to the determination of "process B," as shown in Figure 8, is displayed on the display panel 24.

Here, the display unit 263 may be configured to display a table ST1 in which each process, each condition, and the entered score are associated, as shown in FIG. 8.

(Step S37)
In step S37, the generation unit 262 generates a score table. As an example, the generation unit 262 generates the score table by referring to the input information accepted by the acceptance unit 261 in step S36. An example of the score table generated by the generation unit 262 will be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of the score table in this exemplary embodiment.

As shown in the upper part of Figure 9, the generation unit 262 generates a score table that includes scores for each of one or more conditions (condition a, condition b, and condition c) and each score for each of multiple processes (process A, process B, and process C). In the score table shown in the upper part of Figure 9, for example, the degree to which "condition a" contributes to the discrimination of "process A" is "0.5," and the degree to which "condition b" contributes to the discrimination of "process B" is "-0.2."

Furthermore, when a condition is generated from multiple conditions, the generation unit 262 may generate a score table that also includes the condition name and the content of the condition for that condition name, as shown in the lower part of Figure 9. In the score table shown in the lower part of Figure 9, the condition name "Condition α" indicates "Condition a" and "Condition c," and the degree to which "Condition α" contributes to the discrimination of "Process A" is "0.5," and the degree to which "Condition α" contributes to the discrimination of "Process B" is "0."

(Step S38)
In step S38, the receiving unit 261 determines whether or not an input related to another process has been received.

If it is determined in step S38 that input regarding another process has been received (step S38: YES), the reception unit 261 executes step S31 again to receive the selection of a process.

On the other hand, if it is determined in step S38 that no input regarding other processes has been received (step S38: NO), processing S3 terminates.

(Example of processing executed by the determination unit 13)
An example of the process executed by the discrimination unit 13 will be described with reference to the score table shown in Fig. 9. The discrimination unit 13 calculates a discrimination value, which will be described later, and determines the process represented by the video according to the discrimination value.

As an example, if the certainty factor for "condition a" calculated by the calculation unit 12 is "0.2," the discrimination unit 13 calculates, as a judgment value, the product of the certainty factor "0.2" and the score of each process associated with "condition a" in the score table shown in the upper part of Figure 9, as follows.
Process A: 0.2×0.5=0.1
Process B: 0.2×0.7=0.14
Process C: 0.2×0=0
The discrimination unit 13 then discriminates the process represented by the video based on the calculated judgment value. As an example, the discrimination unit 13 discriminates the process with the highest judgment value among the calculated judgment values as the process represented by the video. In the example described above, the process with the highest judgment value is "0.14" for "Process B," so the discrimination unit 13 determines that the process represented by the video is "Process B."

As another example, the discrimination unit 13 may be configured to determine that a process with the highest judgment value is the process represented by the image if the judgment value for that process is higher than a threshold value.

For example, if the threshold is set to "0.12," the discrimination unit 13 determines whether the highest judgment value "0.14" is higher than the threshold value in the example described above. Because the judgment value "0.14" is higher than the threshold value "0.12," the discrimination unit 13 determines that the process represented by the video is "Process B," with a judgment value of "0.14." On the other hand, if the threshold is set to "0.15," the highest judgment value "0.14" is lower than the threshold value in the example described above, so the discrimination unit 13 determines that there is no process represented by the video.

(Another example of processing executed by the determination unit 13)
As another example, a case where there are multiple conditions in the score table will be described using the score table also shown in the upper part of Fig. 9. When there are multiple conditions, the discrimination unit 13 discriminates the process represented by the video based on a weighted sum of the certainty factors for each condition, where the score for that condition is used as a weighting coefficient. In other words, the discrimination unit 13 calculates, as a judgment value, a weighted sum of the certainty factors for each condition, where the score for that condition is used as a weighting coefficient.

As an example, if the certainty factors calculated by the calculation unit 12 are "0.2" for "condition a,""0" for "condition b," and "0.8" for "condition c," the discrimination unit 13 calculates a weighted sum using the score as a weighting coefficient as follows:
Process A: 0.2 x 0.5 + 0 x 0 + 0.8 x 1 = 0.9
Process B: 0.2 x 0.7 + 0 x (-0.2) + 0.8 x 0 = 0.14
Process C: 0.2 x 0 + 0 x 0 + 0.8 x 0.5 = 0.4
The discrimination unit 13 then uses the calculated weighted sum as a judgment value and, as in the example described above, discriminates the process represented by the video based on the judgment value. That is, the discrimination unit 13 determines that the process represented by the video is "Process A," which has the highest judgment value of "0.9." Alternatively, as in the example described above, the discrimination unit 13 may set a threshold value and discriminate the process represented by the video that has the highest judgment value higher than the threshold value.

(Information processing method S4)
The flow of the information processing method S4 in which the information processing device 2 according to this exemplary embodiment determines the process represented by the video will be described with reference to Fig. 10. Fig. 10 is a flow chart showing the flow of the information processing method S4 according to this exemplary embodiment.

(Step S41)
In step S41, the receiving unit 261 receives the selection of a model. Here, a model is a collection of conditions and scores (score table) referenced to distinguish between predetermined processes. For example, "Model A" is a model that collects conditions and scores referenced to distinguish between "Process A,""ProcessB," and "Process C," and "Model B" is a model that collects conditions and scores referenced to distinguish between "Process A,""ProcessB," and "Process D."

(Step S42)
In step S42, the receiving unit 261 determines whether or not an input for changing the model has been received. Examples of changing the model include adding a process to the model, adding a condition, changing the content of a condition, and changing the score.

(Step S43)
If it is determined in step S42 that an input to change the model has been received (step S42: YES), the receiving unit 261 receives the content of the change to the model in step S43. Then, the generation unit 262 changes the model by referring to the content of the change received by the receiving unit 261. The generation unit 262 may use the processes described in FIGS. 4 and 7 as the process of changing the model.

(Step S44)
In step S42, if it is determined that an input to change the model has not been received (step S42: NO), the receiving unit 261 determines whether or not a request to execute a determination process to determine the process represented by the video has been received.

If it is determined in step S44 that execution of the discrimination process has not been accepted (step S44: NO), processing in information processing method S4 is terminated.

(Step S45)
If it is determined in step S44 that the execution of the determination process has been accepted (step S44: YES), in step S45 the accepting unit 261 accepts a selection of which video to use as the target video. The accepting unit 261 stores input information indicating the selected target video in the storage unit 25.

(Step S46)
In step S46, the calculation unit 12 refers to the input information stored in the storage unit 25 and acquires the target video from the storage unit 25. Then, the calculation unit 12 refers to the conditions stored in the storage unit 25 and calculates a certainty factor as to whether the acquired target video satisfies one or more predetermined conditions. The calculation unit 12 stores the calculated certainty factor in the storage unit 25. Here, as described above, when the calculation unit 12 calculates a certainty factor for each of a plurality of conditions, the calculation unit 12 stores each of the calculated certainty factors in the storage unit 25 in association with the condition used to calculate the certainty factor.

As an example of the confidence level calculated by the calculation unit 12, as shown in the upper part of Figure 6, if the condition is that the color of object OBJ2 included as an element in the video is red, the confidence level calculated by the calculation unit 12 will be high if the color of object OBJ2 is red. On the other hand, as shown in the lower part of Figure 6, if the color of object OBJ2 is black (other than red), the confidence level calculated by the calculation unit 12 will be low.

The calculation unit 12 may also be configured to calculate a degree of certainty as to whether or not the sound associated with the target video satisfies one or more predetermined conditions. As an example, if the condition is that the audio included as an element in the video contains a predetermined phrase, the degree of certainty calculated by the calculation unit 12 will be high if the audio associated with the target video contains the predetermined phrase. With this configuration, the information processing device 2 can flexibly determine the process represented by the video, even when sound associated with the video is used.

(Step S47)
In step S47, the discrimination unit 13 refers to the certainty factor stored in the memory unit 25 and discriminates the process represented by the video based on the certainty factor for each condition and a score indicating the degree to which the condition contributes to the discrimination of the process. As an example, the discrimination unit 13 discriminates the process represented by the video by referring to a score table. The process by which the discrimination unit 13 discriminates the process represented by the video is as described above. The discrimination unit 13 stores the discrimination result in the memory unit 25.

(Example of video including the discrimination result)
An example of an image including the determination result displayed by the display unit 263 will be described with reference to Fig. 11. Fig. 11 is a diagram showing an example of an image including the determination result displayed in this exemplary embodiment.

As an example, the display unit 263 displays a target image including the discrimination result DR and the judgment value JV on the display panel 24, as shown in FIG. 11.

Furthermore, when an element is included in the target video, the display unit 263 may further display a confidence level as to whether or not the condition for including the element is satisfied. As an example, the display unit 263 may display on the display panel 24 a target video that further includes a confidence level CF1 as to whether or not the condition for including object OBJ4, an element included in the target video, is satisfied, as shown in FIG. 11 . For this configuration, the calculation unit 12 may associate the calculated confidence level with one or more predetermined elements referenced in each of one or more predetermined conditions for calculating the confidence level.

The display unit 263 may further display the judgment value. As an example, the display unit 263 may display on the display panel 24 a target image that further includes a judgment value JV1 calculated with reference to the confidence factor CF1, as shown in FIG. 11. For this configuration, the discrimination unit 13 may associate the calculated judgment value with one or more predetermined elements referenced in each of one or more predetermined conditions for calculating the judgment value.

With these configurations, the information processing device 2 can present to the user the confidence level calculated by the calculation unit 12 and the judgment value calculated by the discrimination unit 13. Therefore, the information processing device 2 can allow the user to confirm whether the confidence level calculated by the calculation unit 12 and the judgment value calculated by the discrimination unit 13 are valid.

Furthermore, as shown in FIG. 11, the display unit 263 may similarly display a target image for object OBJ5 on the display panel 24, further including a confidence factor CF2 and a judgment value JV2.

(Another example of video containing the classification results)
Another example of an image including a determination result displayed by the display unit 263 will be described with reference to Fig. 12. Fig. 12 is a diagram showing another example of an image including a determination result displayed in this exemplary embodiment.

The display unit 263 may display any multiple images included in the target video and the discrimination results for the any multiple images. As an example, as shown in FIG. 12, the display unit 263 displays an image including any multiple images PIC1, PIC2, and PIC3 included in the target video on the display panel 24. Images PIC1, PIC2, and PIC3 are not particularly limited, but may be images at the time the discrimination results changed, or may be images of the first frame in the divided video when the target video is divided at a predetermined time interval, or any images included in the divided video.

Then, the display unit 263 displays an image including the discrimination results DR1, DR2, and DR3 of the multiple images PIC1, PIC2, and PIC3, respectively, on the display panel 24.

In addition, the display unit 263 may display on the display panel 24 an image further including time information TI1, time information TI2, and time information TI3 indicating the times at which the multiple images PIC1, image PIC2, and image PIC3 were captured, respectively.

With these configurations, the information processing device 2 can present to the user which process each of the multiple images PIC1, PIC2, and PIC3 has been identified as belonging to. Furthermore, if images PIC1, PIC2, and PIC3 are images at the time when the respective identification results have changed, the information processing device 2 can present to the user that the identification results have changed for images PIC1, PIC2, and PIC3.

As described above, the information processing device 2 according to this exemplary embodiment is configured to include an acquisition unit 11 that acquires video, a calculation unit 12 that calculates the degree of certainty as to whether or not a condition is met, provided that the video contains one or more specified elements, and a discrimination unit 13 that determines the process represented by the video by referring to a score table that includes scores for each of one or more conditions, each of which includes a score for each of a plurality of processes.

For this reason, the information processing device 2 according to this exemplary embodiment can appropriately identify the process by using one or more predetermined elements contained in the video as conditions. Furthermore, the information processing device 2 according to this exemplary embodiment references a score table that includes scores for each of one or more conditions and each of multiple processes, so that even if there are multiple conditions and processes, the process can be appropriately identified.

[Software implementation example]
Some or all of the functions of the information processing devices 1 and 2 may be realized by hardware such as an integrated circuit (IC chip), or by software.

In the latter case, information processing devices 1 and 2 are realized, for example, by a computer that executes program instructions, which are software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in Figure 13. Computer C has at least one processor C1 and at least one memory C2. Memory C2 stores program P for operating computer C as information processing devices 1 and 2. In computer C, processor C1 reads and executes program P from memory C2, thereby realizing each function of information processing devices 1 and 2.

The processor C1 may be, for example, a CPU (Central Processing Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating Point Number Processing Unit), PPU (Physics Processing Unit), microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), or a combination thereof.

Computer C may further include RAM (Random Access Memory) for expanding program P during execution and for temporarily storing various data. Computer C may also include a communications interface for sending and receiving data to and from other devices. Computer C may also include an input/output interface for connecting input/output devices such as a keyboard, mouse, display, and printer.

The program P can also be recorded on a non-transitory, tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communications network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[Additional Note 1]
The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention.

[Additional Note 2]
Some or all of the above-described embodiments can also be described as follows, but the present invention is not limited to the following described aspects.

(Appendix 1)
An information processing device comprising: an acquisition means for acquiring an image; a calculation means for calculating a degree of certainty as to whether the image satisfies one or more predetermined conditions; and a determination means for determining the process represented by the image based on the degree of certainty for each of the conditions and a score indicating the degree to which the condition contributes to the determination of the process.

(Appendix 2)
2. The information processing device according to claim 1, wherein each of the conditions includes that the video contains one or more predetermined elements.

(Appendix 3)
3. The information processing device according to claim 2, further comprising: a first receiving means for receiving input relating to the specified one or more elements; and a first generating means for generating the condition by referring to the input received by the first receiving means.

(Appendix 4)
The information processing device according to claim 2 or 3, wherein the calculation means associates the calculated certainty with the one or more predetermined elements referenced in each of the one or more predetermined conditions for calculating the certainty.

(Appendix 5)
The information processing device described in any one of appendices 1 to 4, wherein the discrimination means discriminates the process represented by the video based on a weighted sum of certainty for each of the conditions, the weighted sum using the score for the condition as a weighting coefficient.

(Appendix 6)
An information processing device described in any one of appendices 1 to 5, wherein the discrimination means discriminates the process represented by the image by referring to a score table including scores for each of one or more conditions and each score for each of a plurality of processes.

(Appendix 7)
The information processing device described in Appendix 6 further comprises a second receiving means for receiving input regarding the selection of one or more conditions, a third receiving means for receiving input of the score regarding the selected condition, and a second generating means for generating the score table based on the input of the score.

(Appendix 8)
8. The information processing device according to claim 7, wherein the second accepting means accepts input regarding selection of a plurality of conditions, and further comprises third generating means that generates new conditions based on the selected plurality of conditions.

(Appendix 9)
9. The information processing device according to any one of claims 1 to 8, wherein the determining means changes a time interval for a determination process to be performed in the future depending on a result of a determination performed in the past.

(Appendix 10)
10. The information processing device according to any one of claims 1 to 9, wherein the acquisition means further acquires sound related to the video, and the determination means further refers to the sound to determine the process represented by the video.

(Appendix 11)
An information processing method including the steps of: an information processing device acquiring an image; calculating a degree of certainty as to whether the image satisfies one or more predetermined conditions; and determining a process represented by the image based on the degree of certainty for each of the conditions and a score indicating the degree to which the condition contributes to determining the process.

(Appendix 12)
A program that causes a computer to function as an information processing device, the program causing the computer to function as: an acquisition means that acquires video; a calculation means that calculates a degree of certainty as to whether the video satisfies one or more specified conditions; and a discrimination means that discriminates a process represented by the video based on the degree of certainty for each of the conditions and a score that indicates the degree to which the condition contributes to discriminating the process.

[Appendix 3]
Some or all of the above-described embodiments can also be expressed as follows.

An information processing device comprising at least one processor that executes an acquisition process for acquiring video, a calculation process for calculating the degree of certainty as to whether the video satisfies one or more specified conditions, and a determination process for determining the process represented by the video based on the degree of certainty for each of the conditions and a score indicating the degree to which the condition contributes to the determination of the process.

The information processing device may further include a memory, and this memory may store a program for causing the processor to execute the acquisition process, the calculation process, and the determination process. This program may also be recorded on a computer-readable, non-transitory, tangible recording medium.

REFERENCE SIGNS LIST 1, 2 Information processing device 11 Acquisition unit 12 Calculation unit 13 Discrimination unit 26 Control unit 261 Reception unit 262 Generation unit 263 Display unit

Claims (9)

an acquisition means for acquiring a video;
a calculation means for calculating a degree of certainty as to whether the video satisfies one or more predetermined conditions;
a determination means for determining the process represented by the image based on a degree of certainty for each of the conditions and a score indicating the degree to which the condition contributes to the determination of the process ;
The determining means changes the time interval for a determination process to be performed in the future in accordance with a result of a determination performed in the past.
Information processing device. The above conditions are:
The information processing device according to claim 1 , wherein the video includes one or more predetermined elements. a first receiving means for receiving an input relating to the predetermined one or more elements;
a first generating means for generating the condition by referring to the input received by the first receiving means;
The information processing device according to claim 2 , further comprising: 4. The information processing apparatus according to claim 2, wherein the calculation means associates the calculated certainty factor with the one or more predetermined elements referenced in each of the one or more predetermined conditions for calculating the certainty factor. 5. The information processing device according to claim 1, wherein the discrimination means discriminates the process represented by the video based on a weighted sum of certainty factors for each of the conditions, the weighted sum using a score for the condition as a weighting coefficient. The information processing device according to claim 1 , wherein the discrimination means discriminates the process represented by the image by referring to a score table including scores for each of one or more conditions and each of a plurality of processes. a second receiving means for receiving an input regarding the selection of one or more conditions;
a third receiving means for receiving an input of the score related to the selected condition;
7. The information processing apparatus according to claim 6, further comprising: second generating means for generating the score table based on input of the scores. The information processing device
Acquiring video and
calculating a confidence level as to whether the video satisfies one or more predetermined conditions;
Identifying the process represented by the video based on a confidence level for each of the conditions and a score indicating the degree to which the condition contributes to the process identification;
Including,
Change the time interval for future discrimination processes depending on past discrimination results
Information processing methods. A program that causes a computer to function as an information processing device,
The program causes the computer to:
an acquisition means for acquiring a video;
a calculation means for calculating a degree of certainty as to whether the video satisfies one or more predetermined conditions;
a determination means for determining the process represented by the video based on a degree of certainty for each of the conditions and a score indicating the degree to which the condition contributes to the determination of the process;
It functions as
The determining means changes the time interval for a determination process to be performed in the future depending on the result of the determination performed in the past.
program.