JP7541635B2 - Training data generation program, training data generation device, and training data generation method - Google Patents

Description

The present invention relates to a learning data generation program, a learning data generation device, and a learning data generation method.

In recent years, the use of electric carts (hereafter referred to as senior cars) designed to support the daily activities of the elderly has become widespread. For example, by riding in a senior car when going out for shopping, elderly people can reduce the physical burden associated with going out.

Here, there is a possibility that such a senior car may fall over while being driven due to, for example, bad roads. In this case, the elderly person may not be able to stand up by himself.

For this reason, the senior car detects the presence of obstacles along the travel route that could cause the car to fall, and travels while avoiding the detected obstacles. This enables the senior car to automatically avoid the risk of falling (see Patent Documents 1 and 2 and Non-Patent Documents 1 and 2).

JP 2016-186703 A JP 2018-156408 A

Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi, “You Only Look Once: Unified, Real-Time Object Detection”, arXiv:1506.02640v5. Vijay Badrinarayanan, Alex Kendall, Roberto Cipolla, “SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation”, arXiv:1511.00561v3.

In such a senior car, detection of obstacles while driving is performed by a learning model that has learned multiple pieces of learning data, including image data captured by an imaging device such as a camera and information indicating the position and type of obstacle contained in the image data (hereinafter also referred to as label information). Therefore, before generating the learning model, the worker needs to generate in advance the number of learning data required to generate the learning model.

However, when generating training data as described above, an operator must manually input, for example, label information corresponding to each image data. Therefore, depending on the amount of training data required to generate a learning model, the workload of the operator required to generate the training data may become enormous.

The object of the present invention is to provide a learning data generation program, a learning data generation device, and a learning data generation method that can reduce the workload required to generate learning data.

The learning data generation program of the present invention for achieving the above object is characterized in that it causes a computer to execute the following processes: extracting a plurality of partial image data corresponding to each of a plurality of fixed positions from first image data among a plurality of image data included in video data captured by an imaging device, outputting the plurality of partial image data corresponding to the first image data, accepting input of first identification information indicating the type of object shown in each of the plurality of partial image data corresponding to the first image data, extracting a plurality of partial image data corresponding to each of the plurality of fixed positions from second image data captured after the first image data among the plurality of image data, outputting the plurality of partial image data corresponding to the second image data such that the output positions of the corresponding plurality of partial image data in the second image data match the output positions of the corresponding plurality of partial image data corresponding to the same fixed position in the first image data, and accepting input of second identification information indicating the type of object shown in at least a part of the plurality of partial image data corresponding to the second image data.

In addition, in one aspect, the learning data generation program of the present invention for achieving the above object is characterized in that the multiple partial image data are extracted from the first image data such that the size of each of the multiple partial image data is larger than the size of each pixel constituting the first image data, and the sizes of each of the multiple partial image data are the same as each other.

In addition, in one aspect, the learning data generation program of the present invention for achieving the above object is characterized in that it outputs the plurality of partial image data corresponding to the second image data in association with the first identification information corresponding to the same fixed position in the first image data.

In one aspect, the learning data generation program of the present invention for achieving the above object is characterized in that it causes a computer to execute a process of generating first learning data by adding the first identification information to color information in each partial image data and distance information from the imaging device to an object shown in each partial image data for each partial image data corresponding to the first image data, and generating second learning data by adding the second identification information to color information in each partial image data and distance information from the imaging device to an object shown in each partial image data for each partial image data corresponding to the second image data.

In one aspect, the learning data generation program of the present invention for achieving the above object is characterized in that, for each piece of partial image data corresponding to the second image data, for which the input of the second identification information has been accepted, the second learning data is generated by adding the second identification information to color information in each piece of partial image data and distance information from the imaging device to an object depicted in each piece of partial image data, and, for each piece of partial image data corresponding to the second image data, for which the input of the second identification information has not been accepted, the first identification information corresponding to partial image data corresponding to the same fixed position as each piece of partial image data is added to color information in each piece of partial image data and distance information from the imaging device to an object depicted in each piece of partial image data, to generate the second learning data.

In one aspect, the learning data generation program of the present invention for achieving the above object is characterized in that it causes a computer to execute a process of identifying, for each of the plurality of fixed positions, one or more coordinates corresponding to each fixed position, generating, for each of the plurality of partial image data corresponding to the first image data, third learning data including the one or more coordinates corresponding to each partial image data and the first identification information, and generating, for each of the plurality of partial image data corresponding to the second image data, fourth learning data including the one or more coordinates corresponding to each partial image data and the second identification information.

In order to achieve the above object, the learning data generating device of the present invention includes a partial image extraction unit that extracts a plurality of partial image data corresponding to each of a plurality of fixed positions from first image data among a plurality of image data included in video data captured by an imaging device, an image output unit that outputs the plurality of partial image data corresponding to the first image data, and an input receiving unit that receives an input of first identification information indicating the type of object appearing in each of the plurality of partial image data corresponding to the first image data, and the partial image extraction unit further extracts a plurality of partial image data corresponding to each of the plurality of fixed positions from second image data captured after the first image data among the plurality of image data, and the image output unit further outputs the plurality of partial image data corresponding to the second image data such that the output positions of the corresponding plurality of partial image data in the second image data match the output positions of the corresponding plurality of partial image data corresponding to the same fixed position in the first image data, and the input receiving unit further receives an input of second identification information indicating the type of object appearing in at least a part of the plurality of partial image data corresponding to the second image data.

The learning method of the present invention for achieving the above object is characterized in that it causes a computer to execute the following processes: extracting a plurality of partial image data corresponding to each of a plurality of fixed positions from first image data among a plurality of image data included in video data captured by an imaging device, outputting the plurality of partial image data corresponding to the first image data, accepting input of first identification information indicating the type of object shown in each of the plurality of partial image data corresponding to the first image data, extracting a plurality of partial image data corresponding to each of the plurality of fixed positions from second image data captured after the first image data among the plurality of image data, outputting the plurality of partial image data corresponding to the second image data such that the output positions of the corresponding plurality of partial image data in the second image data match the output positions of the corresponding plurality of partial image data corresponding to the same fixed position in the first image data, and accepting input of second identification information indicating the type of object shown in at least a part of the plurality of partial image data corresponding to the second image data.

The training data generation program, training data generation device, and training data generation method of the present invention make it possible to reduce the workload required to generate training data.

FIG. 1 is a diagram showing an example of a configuration of an information processing device 1 according to the first embodiment. FIG. 2 is a diagram for explaining an outline of the learning data generation process in the first embodiment. FIG. 3 is a diagram for explaining an outline of the learning data generation process in the first embodiment. FIG. 4 is a flowchart illustrating details of the learning data generation process according to the first embodiment. FIG. 5 is a flowchart illustrating details of the learning data generation process according to the first embodiment. FIG. 6 is a diagram illustrating a specific example of the learning data generation process. FIG. 7 is a diagram illustrating the output result of the label information. FIG. 8 is a diagram illustrating the output result of the label information.

Below, an embodiment of the present invention will be described with reference to the drawings. However, such an embodiment does not limit the technical scope of the present invention.

First, a configuration example of an information processing device 1 (hereinafter also referred to as a learning data generating device 1) in the first embodiment will be described. FIG. 1 is a diagram showing a configuration example of the information processing device 1 in the first embodiment.

The information processing device 1 is a computer device, for example, a general-purpose PC (Personal Computer). The information processing device 1 performs a process (hereinafter also simply referred to as a learning data generation process) for generating learning data used to generate a learning model to be installed in a senior car, for example.

The information processing device 1 has the hardware configuration of a general-purpose computer device, and for example, as shown in FIG. 1, has a processor, a CPU 101, a memory 102, a communication interface 103, and a storage medium 104. Each part is connected to each other via a bus 105.

The storage medium 104 has, for example, a program storage area (not shown) that stores a program (not shown) for performing the learning data generation process.

The storage medium 104 also has a storage unit 110 (hereinafter also referred to as storage area 110) that stores information used when performing the learning data generation process. The storage medium 104 may be, for example, a hard disk drive (HDD) or a solid state drive (SSD).

The CPU 101 executes the program loaded from the storage medium 104 to the memory 102 to perform the learning process.

The communication interface 103 communicates with the operation terminal 2 via a network NW such as the Internet. Note that the transfer of information between the information processing device 1 and the operation terminal 2 may be performed manually by an operator using a storage medium such as a USB memory.

The operation terminal 2 is, for example, a computer device used by an operator when generating learning data, such as a general-purpose PC (Personal Computer).

[Overview of the first embodiment]
Next, an outline of the learning process in the first embodiment will be described with reference to Fig. 2. The learning process in the first embodiment will be described with reference to Fig. 2.

The image acquisition unit 111 of the information processing device 1 acquires, for example, image data used to generate a learning model. Specifically, the image acquisition unit 111 may acquire, for example, a plurality of image data constituting video data (video data captured in advance by an imaging device not shown) that has been stored in advance in the storage area 110 by an operator.

Then, the partial image extraction unit 112 of the information processing device 1 extracts multiple partial image data (hereinafter also referred to as first partial image data) corresponding to each of the multiple fixed positions in one image data (hereinafter also referred to as first image data) from one of the image data acquired by the image acquisition unit 111.

Then, the image output unit 113 of the information processing device 1 outputs, for example, the multiple first partial image data extracted by the partial image extraction unit 112 to the operation terminal 2.

Then, the input receiving unit 114 of the information processing device receives input of identification information (hereinafter also referred to as first identification information) indicating the type of object depicted in each of the multiple first partial image data output by the image output unit 113.

Specifically, for example, in response to the output of multiple first partial image data to the operation terminal 2, the worker inputs first identification information (label information) corresponding to each of the multiple first partial image data.

Then, for each of the multiple first partial image data extracted by the partial image extraction unit 112, the learning data generation unit 115 generates learning data (hereinafter also referred to as first learning data) including each partial image data and first identification information corresponding to each partial image data accepted as input by the input acceptance unit 114.

Furthermore, the learning data output unit 116 outputs, for example, the multiple first learning data generated by the learning data generation unit 115 to the operation terminal 2.

Next, the partial image extraction unit 112 extracts a plurality of partial image data (hereinafter also referred to as second partial image data) corresponding to each of a plurality of fixed positions in the second image data from the image data (hereinafter also referred to as second image data) following the first image data among the image data acquired by the image acquisition unit 111.

Specifically, the partial image extraction unit 112 extracts multiple second partial image data from, for example, second image data captured after the first image data.

Then, the image output unit 113 outputs, for example, the multiple second partial image data extracted by the partial image extraction unit 112 to the operation terminal 2.

Specifically, the image output unit 113 outputs, for example, each of the multiple second partial image data extracted by the partial image extraction unit 112 such that the output position of each partial image data matches the output position of partial image data corresponding to the same fixed position among the multiple first partial image data extracted by the partial image extraction unit 112.

More specifically, the image output unit 113 outputs, for example, each of the multiple second partial image data extracted by the partial image extraction unit 112 in a form in which the multiple second partial image data are associated with each of the first identification information (the first identification information accepted as input by the input acceptance unit 114) corresponding to each of the multiple first partial image data extracted from the first image data.

Then, the input receiving unit 114 receives input of identification information (hereinafter also referred to as second identification information) indicating the type of object appearing in at least a portion of the multiple second partial image data output by the image output unit 113.

Specifically, for example, in response to each of the multiple second partial image data being output together with the first identification information, the worker identifies, among the multiple second partial image data, those for which the first identification information output together with each partial image data is not appropriate as identification information corresponding to each partial image data. In other words, the worker identifies, among the multiple second partial image data, those for which the type of object depicted in each partial image data has changed between the first partial image data and the second partial image data corresponding to the same fixed position. The worker then inputs the second identification information corresponding to the identified second partial image data (appropriate identification information corresponding to the second partial image data).

In other words, for example, if the second image data is image data captured immediately after the first image data, it can be determined that the first partial image data and the second image data are a combination of image data that are similar to each other and have few differences.

Therefore, the information processing device 1 outputs, for example, each of the multiple second partial image data in a form in which it corresponds to the first identification information of the first partial image data corresponding to the same fixed position. Then, the worker, for example, modifies only the identification information among the first identification information output together with the multiple second partial image data that is not appropriate as identification information corresponding to the second partial image data.

Then, for other image data constituting the video data stored in the memory area 110, the information processing device 1 outputs each of the multiple partial image data constituting each image data in association with identification information corresponding to each of the multiple partial image data constituting the image data captured immediately before each image data. Then, for each of the other image data, the worker changes only the identification information that needs to be changed (inappropriate identification information) among the identification information output together with each of the multiple partial image data constituting each image data.

This enables the information processing device 1 to reduce the workload of the worker involved in inputting identification information (label information) when generating training data. Therefore, the worker can manually generate training data even when, for example, a huge amount of training data is required to generate a learning model.

In addition, the information processing device 1 becomes able to generate learning data that can generate a learning model that is smaller in scale than other learning models (e.g., learning models using YOLO (You Only Live Once) and SSD (Single Shot Multibox Detector)).

Then, for each of the multiple second partial image data extracted by the partial image extraction unit 112, the learning data generation unit 115 generates learning data (hereinafter also referred to as second learning data) including each partial image data and second identification information corresponding to each partial image data accepted as input by the input acceptance unit 114.

Furthermore, the learning data output unit 116 outputs, for example, the multiple second learning data generated by the learning data generation unit 115 to the operation terminal 2.

[Details of the First Embodiment]
Next, the learning data generation process in the first embodiment will be described in detail. Figures 3 and 4 are flow charts for explaining the learning data generation process in the first embodiment. Figures 5 to 8 are diagrams for explaining the learning data generation process in the first embodiment.

As shown in FIG. 3, the image acquisition unit 111 waits, for example, until it is time to generate learning data (NO in S11). The timing to generate learning data may be, for example, the timing when the worker inputs information to start the learning data generation process via the operation terminal 2. The timing to generate learning data may also be, for example, the timing when the worker stores video data captured by an imaging device (not shown) in the memory area 110.

Then, when it is time to generate learning data (YES in S11), the image acquisition unit 111 acquires, for example, video data stored in the memory area 110 (S12).

Next, the image acquisition unit 111 acquires one image data from the image data acquired in the processing of S12 (S13).

Specifically, the image acquisition unit 111 acquires, in chronological order, one image data piece that constitutes the video data acquired in the processing of S12.

Then, the partial image extraction unit 112 extracts a plurality of partial image data corresponding to each fixed position in the image data acquired in the process of S13 (S14). A specific example of the process of S14 will be described below.

[Specific example of processing in S14]
Fig. 5 is a diagram for explaining a specific example of the process of S14. Specifically, Fig. 5 is a diagram for explaining a plurality of partial image data corresponding to a plurality of regions of interest ROI in the image data DT1. In the following description, the numbers of the regions of interest ROI will be assumed to correspond to the numbers of the regions shown in Fig. 5.

For example, before starting the learning data generation process by the information processing device 1, the worker determines the fixed positions and sizes of the regions of interest ROI in advance. Specifically, for example, as shown in FIG. 5, the worker determines six regions of interest ROI (region of interest ROI1 to region of interest ROI6) each consisting of a rectangle of a first size, and 16 regions of interest ROI (region of interest ROI7 to region of interest ROI22) each consisting of a rectangle of a second size larger than the first size.

Then, in the process of S14, the partial image extraction unit 112 extracts, for example, from the image data DT1, each of the partial image data corresponding to the 22 regions of interest ROI determined by the operator.

Specifically, the partial image extraction unit 112 extracts partial image data (eight partial image data consisting of rectangles of the second size) obtained by dividing the lower end area of the image data DT1 horizontally into eight equal parts, as shown in FIG. 5, as partial image data corresponding to the regions of interest ROI15, ROI16, ROI17, ROI18, ROI19, ROI20, ROI21, and ROI22 (hereinafter, these are collectively referred to simply as the first regions of interest ROI).

The partial image extraction unit 112 also extracts partial image data (eight partial image data pieces each consisting of a rectangle of the second size) obtained by dividing the area adjacent to the upper end of the first region of interest ROI in the image data DT1 into eight equal parts along the horizontal direction, as shown in FIG. 5, as partial image data corresponding to the regions of interest ROI7, ROI8, ROI9, ROI10, ROI11, ROI12, ROI13, and ROI14 (hereinafter, these are collectively referred to simply as the second region of interest ROI).

Furthermore, as shown in FIG. 5, the partial image extraction unit 112 extracts partial image data (six partial image data pieces each consisting of a rectangle of the first size) obtained by dividing an area adjacent to the upper end of the second region of interest ROI in the image data DT1 and near the center in the horizontal direction into six equal parts along the horizontal direction, as partial image data corresponding to the regions of interest ROI1, ROI2, ROI3, ROI4, ROI5, and ROI6 (hereinafter, these are collectively referred to simply as the third region of interest ROI).

In the example shown in FIG. 5, the worker determines the second size so that the area 1 m to 2 m ahead of the senior car corresponds to the second region of interest ROI and the third region of interest ROI. That is, for example, when the senior car travels at 6 km/h, the second size is determined so that the area on the route where the senior car will travel in about 1 second (hereinafter also referred to as the danger area) corresponds to the second region of interest ROI and the third region of interest ROI. Also, in the example shown in FIG. 5, the worker determines the first size so that the area 2 m to 4 m ahead of the senior car corresponds to the third region of interest ROI. That is, for example, when the senior car travels at 6 km/h, the first size is determined so that the area on the route where the senior car will travel in about 2 seconds (hereinafter also referred to as the notice area) corresponds to the third region of interest ROI.

This enables the information processing device 1 to easily compare the identification information corresponding to each of the multiple partial image data extracted from each image data with the identification information corresponding to each of the multiple partial image data extracted from image data captured after each image data, as described below.

The following description will be given of a case where an operator determines two sizes of ROIs, but the operator may determine three or more sizes of ROIs. The following description will be given of a case where an operator determines a rectangular ROI, but the operator may determine a ROI of a shape other than a rectangle (e.g., a regular hexagon).

Returning to FIG. 3, the image output unit 113 outputs the multiple partial image data extracted in the processing of S14 (S15).

Here, for example, when processing S15 is performed for the second or subsequent times, the image output unit 113 outputs each of the multiple partial image data extracted by processing S14 such that the output position of each partial image data matches the output position of the partial image data corresponding to the same fixed position among the multiple partial image data extracted by processing S14 performed previously.

Furthermore, when the process of S15 is performed for the second or subsequent time, the image output unit 113 outputs each of the multiple partial image data extracted by the process of S14 in a manner that associates the multiple partial image data with the label information (the label information inputted in the process of S16 performed previously) corresponding to each of the multiple partial image data extracted in the process of S14 performed previously.

Then, the input receiving unit 114 receives input of label information (type of object shown in each of the multiple partial image data) corresponding to each of the multiple partial image data extracted in the process of S14 (S16).

Specifically, for example, in response to the multiple partial image data extracted in the process of S14 being output to the operation terminal 2, the worker inputs label information corresponding to each partial image data.

More specifically, for example, when the process of S16 is performed for the first time in the learning data generation process, the operator inputs label information corresponding to each of the multiple partial image data extracted in the process of S14.

On the other hand, for example, when processing S16 is performed for the second or subsequent time in the learning data generation process, the operator inputs only label information corresponding to each of the multiple partial image data extracted in processing S14 that differs from the label information input in the previous processing of S16.

Next, when label information corresponding to each of the multiple partial image data extracted in the process of S14 is received (YES in S16), the learning data generation unit 115 generates learning data including each partial image data and label information corresponding to each partial image data received in the process of S16 performed immediately before, for each partial image data received in the process of S16 performed immediately before, as shown in FIG. 4 (S21). Then, the learning data generation unit 115 stores the learning data generated in the process of S21 in the memory area 110, for example.

In this case, the learning data generation unit 115 generates learning data including each partial image data and label information corresponding to each partial image data that was last input in the process of S16, for each partial image data that was not input in the process of S16 that was performed immediately before (S22). Then, the learning data generation unit 115 stores the learning data generated in the process of S22 in the memory area 110, for example.

Then, the learning data generation unit 115 determines whether or not all of the image data has been acquired in the processing of S13 (S23).

As a result, if it is determined that not all of the image data has been acquired (NO in S23), the image acquisition unit 111 performs the processes from S13 onwards again.

On the other hand, if it is determined that all of the image data has been acquired (YES in S23), the information processing device 1 ends the data generation process.

Specifically, in this case, the information processing device 1 generates a learning model by performing machine learning using the learning data generated in the processes of S21 and S22. A specific example of the learning data generation process is described below.

[Specific example of learning data generation process]
Fig. 6 is a diagram for explaining a specific example of the learning data generation process, specifically, Fig. 6 is a diagram for explaining a specific example of an input screen for label information.

Image data DT2a shown in FIG. 6 is image data regarding RGB information generated from image data acquired in the first processing of S13, and image data DT3a is image data regarding distance information generated from image data acquired in the first processing of S13. The input screen IN shown in FIG. 6 has an image data designation section INa for designating image data DT2a and image data DT3a, and a label information input section INb for inputting label information (type of obstacle shown in each partial image data), etc.

In the example shown in FIG. 6, when an operator inputs an instruction to load image data DT2a and image data DT3a by pressing "Load RGB Images" and "Load Depth Images" included in the image data designation unit INa, the input reception unit 114 outputs each of the image data DT2a and image data DT3a with the frames corresponding to each region of interest ROI (region of interest ROI1 to region of interest ROI22) superimposed on each other.

In this case, the worker specifies (selects) the type of object reflected in each region from among "Bump" or "Dent" in the label information input section INb, for example, for each region corresponding to each region of interest ROI in the image data DT2a (each partial image data extracted from the image data DT2a).

Next, the operator may, for example, press "Next Images" in the image data designation section INa to display image data DT2b (image data for the next RGB information after image data DT2a) about RGB information generated from the image data obtained in the second processing of S13, and image data DT3b (image data for the next distance information after image data DT3a) about distance information generated from the image data obtained in the second processing of S13, superimposed on frames corresponding to each region of interest ROI.

Then, the worker identifies, for example, areas in the image data DT2b corresponding to each region of interest ROI where the type of object reflected in each area has changed from that in the image data DT2a. Furthermore, for each identified area, the worker specifies (selects) the type of object reflected in each area from among "Bump" or "Dent" in the label information input section INb, etc.

Then, the worker repeats the process of identifying the type of object appearing in the area corresponding to each region of interest ROI in each of the other image data included in the video data acquired in the process of S12.

This enables the information processing device 1 to reduce the workload of the worker required to input label information when generating training data. Therefore, the worker can manually generate training data even when, for example, a huge amount of training data is required to generate a learning model.

In the above example, we have described the generation of learning data to be used to generate a learning model for detecting obstacles on the driving path of a senior car. However, the learning data generation process in this embodiment can also be applied to the generation of learning data to be used to generate learning models for other purposes.

[Output of label information received]
Next, the output of the label information input by the worker will be described below. Figures 7 and 8 are diagrams for explaining the output result of the label information.

In the process of S16, when the information processing device 1 receives input of label information corresponding to each of the multiple partial image data constituting each image data, the information included in the received input label information may be converted into a pre-specified format and output.

Specifically, the information processing device 1 may output, for example, as shown in FIG. 7, in the form of a text file, a value indicating the type of object depicted in each of the partial image data, and the top left coordinate, height, and width of each of the partial image data for each of the multiple partial image data included in the image data DT2a.

In addition, as shown in FIG. 8, the information processing device 1 may output, for each of a plurality of partial image data included in the image data DT2a, the type of object (name) depicted in each partial image data, and the minimum value (xmin) on the X axis, the maximum value (xmax) on the X axis, the minimum value (ymin) on the Y axis, and the maximum value (ymax) on the Y axis of each partial image data in the form of an XML (Extensible Markup Language) file.

This allows the worker to generate training data to be used in generating other training models (e.g., training models using YOLO or SSD) by using a file that converts the information contained in the label information received in the processing of S16. This allows the worker to reduce the workload associated with generating training data for other training models.

The label information input in the process of S16 is label information input for each region of interest ROI described in FIG. 5, and is, for example, in smaller units than when a bounding box is used, but larger units than when it is input for each pixel.

Therefore, by using a file that converts the information contained in the label information received in the process of S16, the worker can generate training data that can generate a training model with high judgment accuracy while reducing the workload required to generate the training data.

1: Information processing device 2: Operation terminal 101: CPU
102: Memory 103: Communication interface 104: Storage medium 105: Bus

Claims (8)

extracting a plurality of partial image data corresponding to the plurality of fixed positions from a first image data among a plurality of image data included in the video data captured by the imaging device;
outputting the plurality of partial image data corresponding to the first image data;
accepting an input of first identification information indicating a type of an object shown in each of the plurality of partial image data corresponding to the first image data;
extracting a plurality of partial image data corresponding to the plurality of fixed positions from second image data captured after the first image data among the plurality of image data;
outputting the plurality of partial image data corresponding to the second image data such that output positions of the plurality of partial image data corresponding to the second image data coincide with output positions of the plurality of partial image data corresponding to the same fixed position in the first image data;
accepting an input of second identification information indicating a type of an object shown in at least a part of the plurality of partial image data corresponding to the second image data;
A learning data generation program for causing a computer to execute a process. In claim 1,
In the process of extracting a plurality of partial image data from the first image data, the plurality of partial image data are extracted from the first image data such that each of the plurality of partial image data has a size larger than each pixel constituting the first image data, and each of the plurality of partial image data has a size equal to each other.
A learning data generation program comprising: In claim 1,
In the process of outputting the plurality of partial image data corresponding to the second image data, the plurality of partial image data corresponding to the second image data and the first identification information corresponding to the same fixed position in the first image data are output in association with each other.
A learning data generation program comprising: In claim 1, further comprising:
generating first learning data by adding the first identification information to color information in each partial image data and distance information from the imaging device to an object shown in each partial image data for each partial image data corresponding to the first image data;
generating second learning data by adding the second identification information to color information in each partial image data and distance information from the imaging device to an object shown in each partial image data for each partial image data corresponding to the second image data;
A learning data generation program for causing a computer to execute a process. In claim 4,
In the process of generating the second learning data,
generating the second learning data by adding the second identification information to color information in each partial image data and distance information from the imaging device to an object shown in each partial image data for each partial image data for which the input of the second identification information has been accepted among the partial image data corresponding to the second image data;
the second learning data is generated by adding, to color information in each partial image data and distance information from the imaging device to an object shown in each partial image data, the first identification information corresponding to partial image data corresponding to the same fixed position as each partial image data, among the partial image data corresponding to the second image data, to the first identification information.
A learning data generation program comprising: In claim 4, further comprising:
identifying, for each of the plurality of fixed locations, one or more coordinates corresponding to each fixed location;
generating third learning data including the one or more coordinates corresponding to each of the plurality of partial image data corresponding to the first image data and the first identification information;
generating fourth learning data including the one or more coordinates corresponding to each of the plurality of partial image data corresponding to the second image data and the second identification information;
A learning data generation program for causing a computer to execute a process. a partial image extracting unit that extracts a plurality of partial image data corresponding to a plurality of fixed positions from a first image data among a plurality of image data included in the video data captured by the imaging device;
an image output unit that outputs the plurality of partial image data corresponding to the first image data;
an input receiving unit that receives an input of first identification information indicating a type of an object shown in each of the plurality of partial image data corresponding to the first image data,
The partial image extraction unit further extracts a plurality of partial image data corresponding to the plurality of fixed positions from second image data captured after the first image data among the plurality of image data,
the image output unit further outputs the plurality of partial image data corresponding to the second image data such that output positions of the plurality of partial image data corresponding to the second image data coincide with output positions of the plurality of partial image data corresponding to the same fixed position in the first image data;
The input receiving unit further receives input of second identification information indicating a type of an object appearing in at least a part of the plurality of partial image data corresponding to the second image data.
A learning data generating device comprising: extracting a plurality of partial image data corresponding to the plurality of fixed positions from a first image data among a plurality of image data included in the video data captured by the imaging device;
outputting the plurality of partial image data corresponding to the first image data;
accepting an input of first identification information indicating a type of an object shown in each of the plurality of partial image data corresponding to the first image data;
extracting a plurality of partial image data corresponding to the plurality of fixed positions from second image data captured after the first image data among the plurality of image data;
outputting the plurality of partial image data corresponding to the second image data such that output positions of the plurality of partial image data corresponding to the second image data coincide with output positions of the plurality of partial image data corresponding to the same fixed position in the first image data;
accepting an input of second identification information indicating a type of an object appearing in at least a part of the plurality of partial image data corresponding to the second image data;
A learning data generating method comprising the steps of: causing a computer to execute a process.

Images