JP7596697B2 - Information processing device, generation method, and generation program - Google Patents

Description

The present invention relates to an information processing device, etc.

In recent years, with the spread of smartphones and tablet devices, the number of opportunities to watch promotional videos among video content has increased. For this reason, commercial stores and other businesses are creating promotional videos to promote their products (public relations), etc.

When there are multiple stores to be promoted, it is a heavy burden for users to manually create videos (promotional videos), so video generation devices are used to automatically create videos.

Figure 15 is a diagram for explaining a motion generation device of the prior art. As shown in Figure 15, this motion generation device 10 has a still image DB (Data Base) 11, a model generation unit 12, a material analysis unit 13, and an image insertion unit 14.

The video generating device 10 uses web scraping or the like to collect multiple still image data related to the promotional video and stores it in the still image DB 11. The video generating device 10 also extracts multiple still image data from video data posted from a user's terminal device or the like and stores it in the still image DB 11. The video generating device 10 generates training data by assigning correct answer labels to the multiple still image data stored in the still image DB 11.

The model generation unit 12 generates a learning model by performing machine learning using the teacher data stored in the still image DB 11 as input. For example, by inputting still image data into such a learning model, a classification class of the still image data is output. The classification class corresponds to the name of an object contained in the still image data, and is used as a tag for the still image data. The model generation unit 12 outputs the learned learning model to the material analysis unit 13.

The material analysis unit 13 acquires video data posted from a user's terminal device, etc., and inputs multiple still image data included in the video data into a learning model in order to identify tags (classification classes) for the still image data. The material analysis unit 13 outputs the tagged still image data to the image insertion unit 14.

The image insertion unit 14 generates promotional video data based on a template and multiple still image data with tags. The template defines the duration of the video and the type of tag for the still image data to be placed during that duration. The image insertion unit 14 places the still image data with the corresponding tag according to the template. For example, the image insertion unit 14 places multiple still image data with the tag "store clerk" during a time period corresponding to the tag "store clerk," thereby generating a video of the store clerk during that time period.

JP 2017-111731 A JP 2017-173098 A

Conventional technology involves using web scraping or other methods to collect still image data to be used as training data, but the collected still image data may contain multiple copies of the same still image data or may contain still image data that is not suitable as training data.

When training a learning model with such still image data included in the training data, over-learning or under-learning will occur, resulting in poor accuracy of the learning model. Furthermore, when tagging is performed using such a learning model, appropriate tags cannot be assigned to the still image data, resulting in poor accuracy of the video.

It is possible for users to check still image data collected using web scraping, etc., one by one and remove unnecessary still image data, but this is not realistic due to the enormous amount of still image data that needs to be checked.

In one aspect, the present invention aims to provide an information processing device, a generation method, and a generation program that can improve the accuracy of videos.

In the first proposal, the information processing device has a storage unit, a determination unit, and a generation unit. The storage unit stores multiple image data. The determination unit classifies the multiple image data into multiple groups based on the feature amounts of the multiple image data, and for each group, determines whether or not to discard the group based on the degree of aggregation of the feature amounts of the multiple image data belonging to the group. The generation unit generates teacher data using multiple image data belonging to the remaining groups after discarding the groups to be discarded from the multiple groups.

This can improve the accuracy of the video.

FIG. 1 is a diagram for explaining the process of an information processing device according to the present embodiment. FIG. 2 is a diagram illustrating an example of a system according to the present embodiment. FIG. 3 is a functional block diagram showing the configuration of an information processing apparatus according to the present embodiment. FIG. 4 is a diagram showing an example of the data structure of the still image DB. FIG. 5 is a diagram illustrating an example of a data structure of the teacher data table. FIG. 6 is a diagram illustrating an example of a data structure of the posted video DB. FIG. 7 is a diagram illustrating an example of a data structure of the analysis result table. FIG. 8 is a diagram illustrating an example of the data structure of a template. FIG. 9 is a diagram for explaining the process of the determination unit. FIG. 10 is a flowchart showing a processing procedure of the information processing device according to the present embodiment. FIG. 11 is a flowchart illustrating a procedure of the discarding process according to the present embodiment. FIG. 12 is a diagram for explaining other processes of the information processing device. FIG. 13 is a flowchart showing other processing procedures of the information processing device. FIG. 14 is a diagram illustrating an example of a hardware configuration of a computer that realizes the same functions as the information processing apparatus of the embodiment. FIG. 15 is a diagram for explaining a conventional motion generating device.

Below, examples of the information processing device, generation method, and generation program disclosed in the present application are described in detail with reference to the drawings. Note that the present invention is not limited to these examples.

Figure 1 is a diagram for explaining the processing of the information processing device according to this embodiment. For example, the processing of the information processing device will be explained in the order of steps S10 to S12.

The process of step S10 executed by the information processing device will be described. The information processing device executes web scraping using keywords and collects multiple still image data 20.

The process of step S11 executed by the information processing device will be described. The information processing device executes clustering (unsupervised classification) on the multiple still image data 20 based on the feature amounts of the still image data. As a result, the multiple still image data 20 are classified into multiple groups. Here, it is assumed that the multiple still image data 20 are classified into groups Gr1 to Gr8. In the following description, any one of the multiple groups Gr1 to Gr8 will be referred to as "group Grn."

The process of step S12 executed by the information processing device will be described. The information processing device selects a group Grn, and determines whether or not to discard the group Grn based on the degree of aggregation of the feature space of the multiple still image data included in the selected group Grn.

If the still image data included in group Grn are distributed over a wide area or concentrated in a narrow area, the information processing device determines to discard group Grn.

The information processing device stores all still image data included in the group Grn that is determined not to be discarded in the still image DB 141. The information processing device discards all still image data included in the group Grn that is determined to be discarded.

The information processing device repeats the above process for other groups. The information processing device generates training data using the multiple still image data remaining in the still image DB 141, and uses the training data to perform machine learning of the learning model.

As described above, the information processing device according to this embodiment clusters multiple still image data, and for each group, determines whether to discard the group based on the degree of aggregation of the feature space of the still image data. For example, if the still image data included in the group is distributed over a wide range, the still image data included in the group can be said to be unnecessary still image data. Also, if the still image data included in the group is distributed over a narrow range, the still image data included in the group can be said to be duplicated still image data.

Therefore, the information processing device can remove unnecessary and duplicated still image data by discarding groups of still image data from still image DB141 based on the degree of aggregation of the feature space of the still image data. The information processing device can generate teacher data using the still image data remaining in still image DB141 and perform machine learning of the learning model, thereby improving the accuracy of the learning model and improving the accuracy of videos using such learning models.

FIG. 2 is a diagram showing an example of a system according to this embodiment. As shown in FIG. 2, this system has a terminal device 30 and an information processing device 100. The terminal device 30 and the information processing device 100 are connected to each other via a network 50. Although FIG. 2 shows only the terminal device 30, this system may also have other terminal devices.

The terminal device 30 is a terminal device operated by a user, and corresponds to a PC (Personal Computer), a notebook PC, a tablet terminal, a smartphone, etc. The user operates the terminal device 30 to transmit the posted video data that he/she has shot to the information processing device 100 and to request the creation of a promotional video. The terminal device 30 may transmit multiple types of posted video data to the information processing device 100.

The information processing device 100 generates a learning model by executing the processes described in steps S10 to S12 in FIG. 1. When the information processing device 100 receives a request to create a promotional video from the terminal device 30, the information processing device 100 creates the promotional video using the video data received from the terminal device 30.

FIG. 3 is a functional block diagram showing the configuration of an information processing device according to this embodiment. As shown in FIG. 3, the information processing device 100 has a communication unit 110, an input unit 120, a display unit 130, a storage unit 140, and a control unit 150.

The communication unit 110 transmits and receives information to and from the terminal device 30 and other external devices via the network 50. For example, the communication unit 110 is realized by a network interface card (NIC) or the like.

The input unit 120 is realized using input devices such as a keyboard and a mouse, and inputs various instruction information to the control unit 150 in response to input operations by an operator. For example, an administrator of the information processing device 100 operates the input unit 120 to input various keywords to be used in web scraping.

The display unit 130 is a display device that displays information output from the control unit 150. The display unit 130 corresponds to an organic EL (Electro Luminescence) display, a liquid crystal display, a touch panel, etc.

The memory unit 140 has a still image DB 141, a teacher data table 142, a learning model 143, a posted video DB 144, an analysis result table 145, templates 146, and video data 147. The memory unit 140 corresponds to a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a HDD (Hard Disk Drive).

Still image DB141 is a DB that stores still image data collected by web scraping or the like. FIG. 4 is a diagram showing an example of the data structure of the still image DB. As shown in FIG. 4, this still image DB141 associates keywords, identification information, and still image data. The keywords are keywords specified when performing web scraping. The identification information is information that identifies the still image data. The still image data is still image data collected by web scraping.

The example shown in FIG. 4 shows a case where still image data of identification information IM1-1 to IM1-n1 is collected by specifying the keyword "KW01 (e.g., miso ramen)." It shows a case where still image data of identification information IM2-1 to IM2-n2 is collected by specifying the keyword "KW01 (e.g., ramen bowl)."

The teacher data table 142 is a table that stores a teacher data set for performing machine learning of the learning model 143. FIG. 5 is a diagram showing an example of the data structure of the teacher data table. As shown in FIG. 5, the teacher data table 142 associates classification classes with still image data. The classification classes correspond to correct labels when performing machine learning. The still image data is still image data stored in the still image DB 141 that has not been discarded by the process described in FIG. 1.

The learning model 143 is a learning model such as a Neural Network (NN) that is machine-learned based on the teacher data set stored in the teacher data table 142. By inputting still image data to the learning model 143, the classification class to which the input still image data belongs is output.

The posted video DB 144 is a DB that stores posted video data (time-series still image data) sent from the terminal device 30. FIG. 6 is a diagram showing an example of the data structure of the posted video DB. As shown in FIG. 6, the posted video DB 144 associates posted video identification information with posted video data. The posted video identification information is information that uniquely identifies the posted video data. The posted video data has time-series still image data.

The analysis result table 145 is a table that holds the analysis results of a plurality of still image data stored in the posted video DB 144. FIG. 7 is a diagram showing an example of the data structure of the analysis result table. As shown in FIG. 7, the analysis result table 145 associates posted video identification information, time, still image data, and tags. The posted video identification information is information that uniquely identifies the posted video data. The time is the time of the still image data. For example, the start time of the posted video data is time _t1 . The still image data is still image data included in the posted video data. The tag corresponds to the classification class output from the learning model 143 when the still image data is input to the learning model 143.

In template 146, a scenario is defined by a number of scenes, and video data 147 is generated based on the scenario defined in template 146. FIG. 8 is a diagram showing an example of the data structure of a template. In the example shown in FIG. 8, a store scene, a ramen scene, and a store clerk scene are set in that order in template 146.

For example, the start time of the store scene is "T1" and the end time is "T2", indicating that the tag corresponding to the store scene is "classification class C1". The start time of the ramen scene is "T2" and the end time is "T3", indicating that the tag corresponding to the ramen scene is "classification class C2". The start time of the store clerk scene is "T3" and the end time is "T4", indicating that the tag corresponding to the store scene is "classification class C3".

Returning to the explanation of FIG. 3, video data 147 is video data generated according to the scenario of template 146.

The control unit 150 has a collection unit 151, a determination unit 152, a generation unit 153, a learning unit 154, a reception unit 155, a material analysis unit 156, and an image insertion unit 157. The control unit 150 can be realized by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), etc. The control unit 150 can also be realized by hardwired logic such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The collection unit 151 receives keywords from the input unit 120, and executes web scraping using the received keywords to collect still image data corresponding to the keywords from external devices on the network. The collection unit 151 associates the collected still image data with the keywords and stores them in the still image DB 141. The collection unit 151 assigns unique identification information to each still image data in the still image DB.

The administrator may set multiple keywords in the collection unit 151 in advance, and the collection unit 151 may perform web scraping using the pre-set keywords.

The determination unit 152 classifies still image data registered in the still image DB 141 into a plurality of groups based on the feature amounts of the still image data. For each group, the determination unit 152 determines whether or not to discard the group based on the degree of aggregation of the feature amounts of the plurality of image data belonging to the group. For example, the feature amounts of the still image data may be feature amounts such as brightness, luminance, saturation, SIFT, HOG, etc., or may be feature amounts of other images.

Figure 9 is a diagram for explaining the processing of the determination unit. For convenience of explanation, in Figure 9, a two-dimensional feature amount space 40 is used for explanation, but the present invention is not limited to this. The feature amount space 40 is a space in which the horizontal axis represents the first feature amount and the vertical axis represents the second feature amount. A plot based on the feature amounts of the still image data in the still image DB 141 is indicated by black circles.

The determination unit 152 classifies the multiple still image data into multiple groups by clustering based on the feature amounts of the multiple still image data in the still image DB 141. In the example shown in FIG. 9, the multiple still image data are classified into groups Gr1, Gr2, and Gr3. The determination unit 152 may perform clustering using k-means or the like.

The determination unit 152 selects a group Grn, and determines whether or not to discard the group Grn based on the degree of aggregation of the feature space 40 of the multiple still image data included in the selected group Grn. The determination unit 152 calculates the degree of aggregation based on the distance between the center of gravity of the group Grn and each feature of the group Grn. In FIG. 9, the centers of gravity of groups Gr1 to 3 are indicated by open circles.

For example, the determination unit 152 calculates the degree of aggregation S based on formula (1). Here, as an example, the feature space is assumed to be two-dimensional. The coordinates of the center of gravity are (X, Y), and the coordinates of the i-th still image data in the group Grn are (x _i , y _i ). The number of still image data included in the group Grn is assumed to be n.

The determination unit 152 also calculates a first threshold value Th1 and a second threshold value Th2. The determination unit 152 calculates the distance between the coordinates of the center of gravity and the coordinates of each still image data of the group Grn, and identifies the maximum distance among the calculated distances. The determination unit 152 sets the value obtained by dividing the identified maximum distance by Na as the first threshold value Th1. Na is a preset value, and may be 2, 3, 4, or another numerical value.

The determination unit 152 calculates the distance between the coordinates of the center of gravity and the coordinates of each still image data of the group Grn, and identifies the minimum value of the calculated distances. The determination unit 152 sets the value obtained by multiplying the identified minimum value by Nb as the first threshold value Th2. Nb is a preset value, and may be 2, 3, 4, or another numeric value.

When the degree of aggregation S is equal to or greater than the first threshold Th1, the determination unit 152 determines that the still image data belonging to group Grn are widely distributed and determines to discard group Grn. For example, since the degree of aggregation S of group Gr1 in FIG. 9 is equal to or greater than the first threshold Th1, the determination unit 152 determines to discard group Gr1.

When the degree of aggregation S is less than the second threshold Th2, the determination unit 152 determines that the still image data belonging to group Grn is concentrated in a small range and determines to delete group Grn. For example, since the degree of aggregation S of group Gr3 in FIG. 9 is less than the first threshold Th2, the determination unit 152 determines to discard group Gr3.

In contrast, the aggregation degree S of group Gr2 is less than the first threshold Th1 and is equal to or greater than the second threshold Th2, so the determination unit 152 determines not to discard group Gr2.

The determination unit 152 repeatedly executes the above process for each classified group, determines which groups are to be discarded and which are not, and outputs the determination result to the generation unit 153. For example, the determination result output by the determination unit 152 to the generation unit 153 includes identification information of the still image data included in the group to be discarded.

The generation unit 153 generates the teacher data table 142 based on the judgment result of the judgment unit 152 and the still image DB 141. The generation unit 153 deletes the still image data included in the group to be discarded from the still image DB 141 based on the judgment result of the judgment unit 152, and obtains a pair of the still image data remaining in the still image DB 141 and the keyword of this still image data.

The generation unit 153 converts the keywords into generic classification classes (tags) using a tag conversion table (not shown). The tag conversion table is a table that associates keywords with classification classes that correspond to the keywords. The generation unit 153 associates the still image data with the converted classification classes and registers them in the teacher data table 142.

The generation unit 153 repeatedly performs the above process for pairs of still image data and keywords remaining in the still image DB 141.

The learning unit 154 performs learning (machine learning using backpropagation) of the learning model 143 using the teacher data set stored in the teacher data table 142 as input. For example, the learning unit 154 adjusts the parameters of the learning model 143 so that the output when the still image data (input data) in the teacher data table 142 is input to the learning model 143 approaches the corresponding classification class.

The reception unit 155 receives a request to create a video from the terminal device 30. The reception unit 155 receives posted video data from the terminal device 30, and stores the received posted video data in the posted video DB 144. When storing the posted video data in the posted video DB 144, the reception unit 155 assigns unique posted video identification information.

The material analysis unit 156 inputs the time-series still image data contained in the posted video data stored in the posted video DB 144 into the learning model 143 to determine the tag (classification class) of the still image data. The material analysis unit 156 associates the posted video identification information, the time of the still image data (or frame number), the still image data, and the tag, and stores them in the analysis result table 145.

The material analysis unit 156 repeatedly performs the above process on the time-series still image data contained in the posted video data stored in the posted video DB 144.

The image insertion unit 157 generates video data based on the analysis result table 145 and the template 146. For example, the image insertion unit 157 identifies the tag, start time, and end time of each scenario in the template 146. The image insertion unit 157 calculates the scenario time from the start time to the end time. The image insertion unit 157 refers to the analysis result table 145, acquires time-series still image data for the scenario time from the still image data corresponding to the scenario tag, and sets the acquired time-series still image data in the video data 147.

The image insertion unit 157 generates video data 147 by repeatedly executing the above process for each scenario of the template 146.

Next, an example of the processing procedure of the information processing device 100 according to this embodiment will be described. FIG. 10 is a flowchart showing the processing procedure of the information processing device according to this embodiment. As shown in FIG. 10, the collection unit 151 of the information processing device 100 collects still image data and stores it in the still image DB 141 (step S101).

The determination unit 152 of the information processing device 100 performs clustering based on the feature quantities of the still image data in the still image DB 141, and classifies the data into a plurality of groups (step S102). The determination unit 152 and the generation unit 153 of the information processing device 100 perform a discard process (step S103).

The generation unit 153 generates a teacher data set based on the still image data remaining in the still image DB and stores it in the teacher data table (step S104). The learning unit 154 of the information processing device 100 learns the learning model 143 based on the teacher data table 142 (step S105).

Next, an example of the processing procedure of the discarding process shown in step S103 of FIG. 10 will be described. FIG. 11 is a flowchart showing the processing procedure of the discarding process according to this embodiment. As shown in FIG. 11, the determination unit 152 of the information processing device 100 selects an unselected group (step S201).

The determination unit 152 calculates the center of gravity of the features in the group (step S202). The determination unit 152 calculates the distance between the feature of each still image data and the center of gravity in the feature space (step S203). The determination unit 152 identifies the maximum distance, the minimum distance, and the degree of accumulation (step S204).

The determination unit 152 determines whether the accumulation level is 1/Na times or more of the maximum value (step S205). If the accumulation level is 1/Na times or more of the maximum value (step S205, Yes), the generation unit 153 of the information processing device discards the still image data of the selected group from the still image DB 141 (step S207) and proceeds to step S208.

If the accumulation level is not 1/Na times the maximum value or more (step S205, No), the determination unit 152 determines whether the accumulation level is less than Nb times the minimum value (step S206). If the accumulation level is less than Nb times the minimum value (step S206, Yes), the determination unit 152 proceeds to step S207.

On the other hand, if the accumulation level is not less than Nb times the minimum value (step S206, No), the determination unit 152 proceeds to step S208. If the determination unit 152 has not selected all groups (step S208, No), the determination unit 152 proceeds to step S201. If the determination unit 152 has selected all groups (step S208, Yes), the determination unit 152 ends the discarding process.

Next, the effect of the information processing device 100 according to this embodiment will be described. The information processing device 100 clusters a plurality of still image data, and for each group, determines whether or not to discard the group based on the degree of aggregation of the feature space of the still image data. For example, if the still image data included in a group is distributed over a wide range, the still image data included in the group can be said to be unnecessary still image data. Also, if the still image data included in a group is distributed over a narrow range, the still image data included in the group can be said to be duplicated still image data.

Therefore, the information processing device 100 can remove unnecessary and duplicated still image data by discarding groups of still image data from the still image DB 141 based on the degree of aggregation of the feature space of the still image data. The information processing device 100 generates teacher data using the still image data remaining in the still image DB 141 and performs machine learning of the learning model, thereby improving the accuracy of the learning model and improving the accuracy of videos using such learning models.

The information processing device 100 calculates the center of gravity of the features of multiple image data belonging to the same group, and calculates the degree of aggregation based on the average value of the difference between the center of gravity and the features of multiple image data belonging to the same group. This makes it possible to quantitatively identify the degree of aggregation of multiple still image data included in each group.

The information processing device 100 calculates the first threshold Th1 and the second threshold Th2, and identifies the group to be discarded by comparing the degree of aggregation with the first threshold Th1 and the second threshold Th2. This makes it possible to remove unnecessary and duplicate still image data.

The information processing device 100 removes unnecessary and duplicated still image data from the still image DB 141, and generates a teacher data table 142 based on the remaining still image data. The information processing device 100 then learns a learning model based on the teacher data table 142. This prevents overlearning and underlearning of the learning model 143, and improves the accuracy of the learning model 143. By improving the accuracy of the learning model 143, the accuracy of the video data using the learning model 143 can also be improved.

The information processing device 100 described above clusters multiple still image data and determines whether or not to discard each group based on the degree of aggregation of the feature space of the still image data, but this is not limited to this. Other processes of the information processing device 100 will be described below.

Figure 12 is a diagram for explaining other processes of the information processing device. For example, the other processes of the information processing device 100 will be explained in the order of steps S20 to S22.

The process of step S20 executed by the information processing device 100 will be described. The information processing device 100 executes web scraping using keywords to collect multiple still image data 20a. The information processing device 100 also acquires multiple still image data captured by the user himself. The still image data captured by the user himself is referred to as multiple local image data 20b. The local image data 20b can be said to be still image data that well represents the characteristics of the still image data that the user wants to include in the video data.

The process of step S21 executed by the information processing device 100 will be described. The determination unit 152 of the information processing device 100 performs clustering (unsupervised classification) on the multiple still image data 20a and multiple local image data 20b based on the feature amounts of the still image data. As a result, the multiple still image data 20a and multiple local image data 20b are classified into multiple groups. Here, it is assumed that the multiple still image data 20a and multiple local image data 20b are classified into groups Gr1 to Gr8. In the following description, any one of the multiple groups Gr1 to Gr8 will be referred to as "group Grn."

The process of step S21 executed by the information processing device 100 will be described. The determination unit 152 of the information processing device 100 selects a group Grn, and determines whether or not to discard the group Grn depending on whether or not the selected group Grn includes local image data. The determination unit 152 selects a group Grn, and discards the Grn if the selected group Grn does not include local image data. On the other hand, the determination unit 152 selects a group Grn, and does not discard the Grn if the selected group Grn includes local image data.

The information processing device 100 repeats the above process for the other groups. The information processing device 100 generates a teacher data table 142 using the multiple still image data remaining in the still image DB 141, and uses the teacher data table 142 to perform machine learning of the learning model 143.

Here, the processing procedure of other processes executed by the information processing device 100 will be described. FIG. 13 is a flowchart showing the processing procedure of other processes of the information processing device. The process shown in FIG. 13 is a process that replaces the discard process shown in FIG. 11.

As shown in FIG. 13, the determination unit 152 of the information processing device 100 selects an unselected group (step S301).

The determination unit 152 determines whether or not local image data is included in the group (step S302). If local image data is not included in the group (step S303, No), the determination unit 152 discards the still image data of the selected group from the still image DB 141 (step S304) and proceeds to step S305.

On the other hand, if the group contains local image data (step S303, Yes), the determination unit 152 determines whether or not all groups have been selected (step S305). If all groups have not been selected (step S305, No), the determination unit 152 proceeds to step S301. If all groups have been selected (step S305, Yes), the determination unit 152 ends the process.

As described above, the information processing device 100 selects a group Grn, and determines whether to discard the group Grn depending on whether the selected group Grn contains local image data. This allows the user to keep still image data that belongs to the same group as the local image data captured by the user.

The information processing device 100 according to this embodiment may determine the group to be discarded by combining the process described in FIG. 11 and the process described in FIG. 13. For example, the information processing device 100 may execute the process shown in FIG. 13 to narrow down a plurality of groups to those that include local image data, and determine which of the narrowed down groups to discard based on the degree of aggregation.

Next, an example of the hardware configuration of a computer that realizes the same functions as the information processing device 100 shown in the above embodiment will be described. FIG. 14 is a diagram showing an example of the hardware configuration of a computer that realizes the same functions as the information processing device of the embodiment.

As shown in FIG. 14, computer 200 has a CPU 201 that executes various arithmetic processes, an input device 202 that accepts data input from the user, and a display 203. Computer 200 also has a reading device 204 that reads programs and the like from a storage medium, and a communication device 205 that transmits and receives data to and from terminal device 30 and other external devices via a wired or wireless network. Computer 200 also has a RAM 206 that temporarily stores various information, and a hard disk device 207. Each of devices 201 to 207 is connected to a bus 208.

The hard disk device 207 has a collection program 207a, a judgment program 207b, a generation program 207c, a learning program 207d, a reception program 207e, a material analysis program 207f, and an image insertion program 207f. The CPU 201 also reads out each of the programs 207a to 207f and expands them in the RAM 206.

The collection program 207a functions as a collection process 206a. The judgment program 207b functions as a judgment process 206b. The generation program 207c functions as a generation process 206c. The learning program 207d functions as a learning process 206d. The reception program 207e functions as a reception process 206e. The material analysis program 207f functions as a material analysis process 206f. The image insertion program 207g functions as an image insertion process 206g.

The processing of the collection process 206a corresponds to the processing of the collection unit 151. The processing of the judgment process 206b corresponds to the processing of the judgment unit 152. The processing of the generation process 206c corresponds to the processing of the generation unit 153. The processing of the learning process 206d corresponds to the processing of the learning unit 154. The processing of the reception process 206 corresponds to the processing of the reception unit 155. The processing of the material analysis process 206 corresponds to the processing of the material analysis unit 156. The processing of the image insertion process 206g corresponds to the processing of the image insertion unit 157.

Note that each of the programs 207a to 207g does not necessarily have to be stored in the hard disk device 207 from the beginning. For example, each program may be stored in a "portable physical medium" such as a flexible disk (FD), CD-ROM, DVD, magneto-optical disk, or IC card that is inserted into the computer 200. Then, the computer 200 may read and execute each of the programs 207a to 207g.

The following notes are further provided with respect to the embodiments including the above examples.

(Supplementary Note 1) A storage unit that stores a plurality of image data;
a determination unit that classifies the plurality of image data into a plurality of groups based on feature amounts of the plurality of image data, and determines for each group whether or not the group is to be discarded based on a degree of aggregation of the feature amounts of the plurality of image data belonging to the group;
a generation unit that generates teacher data using a plurality of image data belonging to remaining groups after discarding groups to be discarded from the plurality of groups.

(Appendix 2) The information processing device described in Appendix 1 is characterized in that the determination unit calculates the center of gravity of the features of multiple image data belonging to the same group, and calculates the degree of aggregation based on the difference between the center of gravity and the features of the multiple image data belonging to the same group.

(Supplementary Note 3) The information processing device described in Supplementary Note 2 is characterized in that the determination unit determines that the group is to be discarded when the degree of aggregation is equal to or greater than a first threshold or is less than a second threshold that is smaller than the first threshold.

(Appendix 4) The information processing device described in Appendix 1 is characterized in that the plurality of image data stored in the storage unit includes a pre-specified local image, and the determination unit further executes a process of determining, among each group, a group that does not include the local image as a group to be discarded.

(Appendix 5) The information processing device according to any one of appendices 1 to 4, further comprising a learning unit that performs machine learning of a learning model based on the teacher data.

(Supplementary Note 6) A computer-implemented generation method, comprising:
storing a plurality of image data in a storage device;
classifying the plurality of image data into a plurality of groups based on feature amounts of the plurality of image data;
For each group, it is determined whether or not the group is to be discarded based on the degree of aggregation of the feature amounts of the multiple image data belonging to the group;
A generation method comprising: generating teacher data using a plurality of image data belonging to remaining groups after discarding groups to be discarded from the plurality of groups.

(Appendix 7) The generation method described in Appendix 6 is characterized in that the determination process calculates the center of gravity of the features of multiple image data belonging to the same group, and calculates the degree of aggregation based on the difference between the center of gravity and the features of the multiple image data belonging to the same group.

(Appendix 8) The generation method described in Appendix 7, characterized in that the determination process determines that the group is to be discarded if the degree of aggregation is equal to or greater than a first threshold or is less than a second threshold that is smaller than the first threshold.

(Appendix 9) The generation method described in Appendix 6, characterized in that the plurality of image data stored in the storage device includes a pre-specified local image, and the determination process further includes a process of determining, among each group, a group that does not include the local image as a group to be discarded.

(Appendix 10) The generation method described in any one of appendices 6 to 9, further comprising a process for executing machine learning of a learning model based on the training data.

(Appendix 11) A computer includes:
storing a plurality of image data in a storage device;
classifying the plurality of image data into a plurality of groups based on feature amounts of the plurality of image data;
For each group, it is determined whether or not the group is to be discarded based on the degree of aggregation of the feature amounts of the multiple image data belonging to the group;
A generation program for executing a process of generating teacher data using a plurality of image data belonging to remaining groups after discarding a group to be discarded from the plurality of groups.

(Appendix 12) The generation program described in Appendix 11, characterized in that the process of determining calculates the center of gravity of the features of multiple image data belonging to the same group, and calculates the degree of aggregation based on the difference between the center of gravity and the features of the multiple image data belonging to the same group.

(Appendix 13) The generation program described in appendix 12, characterized in that the process of determining determines that the group is to be discarded if the degree of aggregation is equal to or greater than a first threshold or is less than a second threshold that is smaller than the first threshold.

(Appendix 14) The generation program described in appendix 11, characterized in that the multiple pieces of image data stored in the storage device include a pre-specified local image, and the determining process further includes a process of determining, among each group, a group that does not include the local image as a group to be discarded.

(Appendix 15) The generation program described in any one of appendices 11 to 14, further comprising a process for executing machine learning of a learning model based on the training data.

REFERENCE SIGNS LIST 100 Information processing device 110 Communication unit 120 Input unit 130 Display unit 140 Storage unit 141 Still image DB
142 Teacher data table 143 Learning model 144 Posted video DB
145 Analysis result table 146 Template 147 Video data 150 Control unit 151 Collection unit 152 Determination unit 153 Generation unit 154 Learning unit 155 Reception unit 156 Material analysis unit 157 Image insertion unit

Claims (9)

A storage unit that stores a plurality of image data;
a determination unit that classifies the plurality of image data into a plurality of groups based on feature amounts of the plurality of image data, calculates a degree of aggregation for each group based on feature amounts of the plurality of image data belonging to the group, and determines that the group is to be discarded when the degree of aggregation is equal to or greater than a first threshold or less than a second threshold that is smaller than the first threshold ;
a generation unit that generates teacher data using a plurality of image data belonging to remaining groups after discarding a group to be discarded from the plurality of groups. The information processing device according to claim 1, characterized in that the determination unit calculates the center of gravity of the features of the multiple image data belonging to the same group, and calculates the degree of aggregation based on the difference between the center of gravity and the features of the multiple image data belonging to the same group. The information processing device according to claim 1, characterized in that the plurality of image data stored in the storage unit includes a pre-specified local image, and the determination unit further executes a process of determining, among each group, a group that does not include the local image as a group to be discarded. 4. The information processing device according to claim 1, further comprising a learning unit that executes machine learning of a learning model based on the teacher data. 1. A computer-implemented generation method comprising:
storing a plurality of image data in a storage device;
classifying the plurality of image data into a plurality of groups based on feature amounts of the plurality of image data;
For each classified group, a degree of aggregation is calculated based on feature amounts of the multiple image data belonging to the group, and if the degree of aggregation is equal to or greater than a first threshold or less than a second threshold that is smaller than the first threshold, the group is determined to be discarded ;
A generation method comprising: generating teacher data using a plurality of image data belonging to remaining groups after discarding groups to be discarded from the plurality of groups. On the computer,
storing a plurality of image data in a storage device;
classifying the plurality of image data into a plurality of groups based on feature amounts of the plurality of image data;
For each classified group, a degree of aggregation is calculated based on feature amounts of the multiple image data belonging to the group, and if the degree of aggregation is equal to or greater than a first threshold or less than a second threshold that is smaller than the first threshold, the group is determined to be discarded ;
A generation program for executing a process of generating teacher data using a plurality of image data belonging to remaining groups after discarding a group to be discarded from the plurality of groups. A storage unit that stores a plurality of image data;
a determination unit that classifies the plurality of image data into a plurality of groups based on feature amounts of the plurality of image data, calculates a degree of aggregation for each group based on feature amounts of the plurality of image data belonging to the group, and identifies, from among the plurality of groups, a group whose calculated degree of aggregation is less than a first threshold and equal to or greater than a second threshold that is smaller than the first threshold;
A generation unit that generates teacher data using a plurality of image data belonging to the identified group;
13. An information processing device comprising: 1. A computer-implemented generation method comprising:
storing a plurality of image data in a storage device;
classifying the plurality of image data into a plurality of groups based on feature amounts of the plurality of image data;
For each classified group, a degree of aggregation is calculated based on the feature amounts of the multiple image data belonging to the group, and a group is identified in which the degree of aggregation is less than a first threshold and is equal to or greater than a second threshold that is smaller than the first threshold;
The training data is generated using the plurality of image data belonging to the identified group.
A generating method comprising: On the computer,
storing a plurality of image data in a storage device;
classifying the plurality of image data into a plurality of groups based on feature amounts of the plurality of image data;
For each classified group, a degree of aggregation is calculated based on the feature amounts of the multiple image data belonging to the group, and a group is identified in which the degree of aggregation is less than a first threshold and is equal to or greater than a second threshold that is smaller than the first threshold;
The training data is generated using the plurality of image data belonging to the identified group.
A generating program for executing a process.

Images