JP7452149B2 - Learning device and computer program for block noise detection - Google Patents

Description

The present invention relates to a learning device and a computer program for detecting block noise occurring in an image.

Conventionally, in various devices equipped with image display devices (liquid crystal displays, organic EL displays, etc.) for displaying images, such as navigation devices, smartphones, and personal computers, image data generated by a calculation means such as a CPU is temporarily processed. The image is displayed by writing the image data into the memory and then drawing the image data read out from the memory on the image display device by a calculation means (which may be performed by the CPU) that performs image processing such as a GPU. However, when displaying images on such an image display device, an abnormality may occur in the generation of image data to be written to the memory, for example, because the timing of writing the image data to the memory and reading the image data do not match. It is known that block noise may occur in the image displayed on the image display device.

In order to prevent the occurrence of block noise, it is necessary to be able to detect an abnormality that causes block noise in generated image data before drawing an image. In order to detect such abnormalities in image data before rendering, it is effective to introduce a learning model. For example, as a learning model for displaying images on an image display device, Japanese Patent Application Laid-Open No. 2018-136537 describes data corresponding to images actually displayed on the display as learning data, and data corresponding to the image actually displayed on the display as learning data, and data as learning data for the image that is intended to be displayed on the display. A technique has been disclosed that corrects the gradation of pixels and improves the quality of the video by performing learning using a neural network using data corresponding to a typical video as teacher data.

Japanese Patent Application Publication No. 2012-192878 (pages 15-17)

Here, in the learning model of Patent Document 1, data corresponding to an ideal video intended to be displayed on the display unit is used as the teacher data. However, when actually constructing a learning model, it is relatively easy for the learning model to acquire data of the image actually displayed on the display, while it is relatively easy to obtain the data of the image actually displayed on the display. In many cases, it is not easy to obtain data for images (videos scheduled to be displayed). Therefore, the learning model of Patent Document 1 has limited cases where it can be applied, and has problems in practical use.

The present invention has been made to solve the above-mentioned problems in the conventional art, and it is possible to obtain training data by generating an image in which pseudo block noise is generated from a normal image and using it as training data. It is an object of the present invention to provide a learning device and a computer program for block noise detection that can be easily and widely applied.

In order to achieve the above object, a learning device for detecting block noise according to the present invention includes a normal image acquisition means for acquiring a normal image for learning in which block noise does not occur, and a predetermined area of a block shape as a pseudo noise region. a pseudo block noise image generating means that generates a pseudo block noise image which is an image of block noise set to a teacher data generating means for generating a pseudo abnormal image in which block noise is generated in the image as teacher data used for learning a learning model for detecting block noise; The image generating means generates the pseudo block noise image by setting the size and position of the block shape to be the noise area, the color and transmittance of the noise area, respectively .

Further, a computer program according to the present invention is a program related to learning for block noise detection. Specifically, the computer is equipped with a normal image acquisition means that acquires a normal image for learning without block noise, and a pseudo block noise image that is a block noise image in which a predetermined block-shaped area is set as a pseudo noise area. pseudo block noise image generation means for generating an image; and superimposing and compositing the pseudo block noise image on the normal learning image to generate pseudo block noise in the normal learning image. A computer program for operating a computer program as a teacher data generation means for generating pseudo abnormal images as teacher data used for learning a learning model for detecting block noise, the pseudo block noise image generation means comprising: , the size and position of the block shape to be the noise region, the color and transmittance of the noise region are set respectively, and the pseudo block noise image is generated .

According to the learning device and computer program for block noise detection according to the present invention having the above-described configuration, an image in which block noise is generated in a pseudo normal image is generated and used as training data, and the block noise is detected. Since learning is performed for detection, it is easier to obtain training data than in the past. As a result, it becomes possible to construct a learning model that is widely applicable in terms of improving the quality of images displayed on image display devices. Furthermore, even in the case of generating a pseudo abnormal image in which block noise is generated in a pseudo manner, it is possible to reproduce an abnormal image in which block noise has actually occurred and use it as training data.

1 is a schematic configuration diagram showing a learning system according to this embodiment. FIG. 1 is a block diagram showing a learning device according to the present embodiment. FIG. 1 is a block diagram showing a navigation device according to the present embodiment. 3 is a flowchart of a learning processing program according to the present embodiment. FIG. 6 is a diagram showing a mode in which duplicate images are excluded from learning images. FIG. 2 is a diagram showing an example of a pseudo block noise image. FIG. 3 is a diagram showing an example of a pseudo-abnormal image. FIG. 3 is a diagram showing a distribution of area values of a block noise region. FIG. 7 is a diagram showing a mode of relearning accompanying evaluation of learning results. 3 is a flowchart of an image display processing program according to the present embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a learning device according to the present invention will be described in detail below with reference to the drawings. First, a schematic configuration of a learning system 2 including a learning device 1 according to the present embodiment will be described using FIG. 1. FIG. 1 is a schematic configuration diagram showing a learning system 2 according to this embodiment.

As shown in FIG. 1, the learning system 2 according to the present embodiment includes a learning device 1 for learning a learning model for detecting block noise occurring in images, and means for displaying various images. The navigation device 3 basically includes a navigation device 3 that is a target for reducing block noise in a displayed image. In this embodiment, the navigation device 3, which is an on-vehicle device, is used as an example of a target for reducing block noise in a displayed image, but any device other than the navigation device may be used as long as it is equipped with means for displaying an image. It may be. For example, a smartphone, a tablet terminal, a personal computer, etc. may be used instead of the navigation device 3.

Further, in this embodiment, the learning device 1 which is the main body that performs learning is provided separately from the navigation device 3 which displays images, but the learning device 1 is a part of the navigation device 3 (i.e., a navigation device with a learning function). It may also be used as device 3). In that case, the navigation device 3 having a learning function corresponds to the learning device according to the present invention.

Here, the learning device 1 is a device that performs learning of a learning model for detecting block noise occurring in images displayed on the navigation device 3. The learning device 1 also includes a DB in which various images used for learning the learning model and tests for evaluating the learning results are stored, and uses the images stored in the DB to learn the learning model as described below. Evaluate learning results. In particular, in this embodiment, since learning for block noise detection is performed for images displayed on the navigation device 3, the DB contains images to be displayed on the navigation device 3 (for example, map images, movement guide images, facility images, etc.). guide images, etc.) are stored. Further, by finally incorporating the learning model learned in the learning device 1 into the navigation device 3, the learning results are reflected in the navigation device 3.

On the other hand, the navigation device 3 is installed in the vehicle 4 and displays a map around the vehicle's location based on stored map data and map data acquired through communication, and displays the current location of the vehicle on a map image. , is an in-vehicle device that provides movement guidance along a set guidance route. In particular, in this embodiment, the learning model learned by the learning device 1 is incorporated, and processing for reducing the occurrence of block noise in images to be displayed is also performed. The details will be described later.

Next, a more specific configuration of the learning device 1 included in the learning system 2 will be described using FIG. 2. FIG. 2 is a block diagram showing the learning device 1 according to this embodiment.

As shown in FIG. 2, the learning device 1 includes a control section 11, a teacher image DB 12 as an information recording means connected to the control section 11, and an input operation section 13 composed of a touch panel, a keyboard, and the like.

The control unit 11 is a control unit (MCU, MPU, etc.) that controls the entire learning device 1, and is used as a CPU 21 as an arithmetic device and a control device, and as a working memory when the CPU 21 performs various arithmetic processes. In addition to a RAM 22 and a control program, it also includes an internal storage device such as a ROM 23 in which a learning processing program (see FIG. 4), which will be described later, is recorded, and a flash memory 24 that stores programs read from the ROM 23. Note that the control unit 11 has various means as processing algorithms. For example, the normal image acquisition means acquires a normal image for learning without block noise. The pseudo block noise image generation means generates a pseudo block noise image that is a block noise image in which a predetermined block-shaped area is pseudo-set as a noise region. The training data generation means superimposes and synthesizes the pseudo-block noise image on the normal learning image, thereby generating a pseudo-abnormal image in which block noise is generated in a pseudo-normal image for learning. It is generated as training data used for learning the learning model for detection.

Further, the teacher image DB 12 is a storage means for storing various images used for learning of the learning model and tests for evaluating the learning results, and is configured by, for example, a non-volatile memory, an HDD, or the like. Here, the images stored in the teacher image DB 12 are roughly divided into three types: a learning image 25 used for learning, a primary test image 26 used for a primary test to evaluate learning results, and a secondary test image 26 used for a primary test to evaluate learning results. There is a secondary test image 27 to be used in the next test. The number of images stored in the teacher image DB 12 includes, for example, approximately 90,000 learning images 25, approximately 20,000 primary test images 26, and approximately 790,000 secondary test images 27 used in the secondary test. . Note that the learning image 25, the primary test image 26, and the secondary test image 27 may partially overlap.

Here, since the learning device 1 according to the present embodiment learns a learning model for detecting block noise in images displayed on the navigation device 3, the learning image 25, the primary test image 26 and the secondary test image 27 are all images to be displayed on the navigation device 3 (for example, a map image, a movement guide image, a facility guide image, etc.). Further, the learning image 25 may be an image actually displayed on the navigation device 3 in the past, or may be an image pseudo-generated for learning. Furthermore, the learning images 25 basically include only normal images without block noise. However, as will be described later, during learning, an abnormal image with block noise is generated from the normal image of the learning image 25, and learning is performed using the normal image and the pseudo-generated abnormal image as a teacher image. It will be done.

On the other hand, the primary test image 26 and the secondary test image 27 may be pseudo-generated images, but in order to confirm the learning effect by assuming a more realistic situation, it is necessary to It is desirable to use the image actually displayed in step 3. The primary test image 26 and the secondary test image 27 include in advance a normal image without block noise and an abnormal image with block noise. For example, the ratio is set to 50% for normal images and 50% for abnormal images, but the ratio can be changed as appropriate.

Further, the input operation unit 13 is configured to include, for example, a keyboard, a mouse, a touch panel provided on the front of the display, etc., and is configured to be able to accept various operations by an operator. Then, the control unit 11 performs control based on operations input via the input operation unit 13 to perform various corresponding operations. For example, in this embodiment, it is operated when setting the parameters of pseudo-generated block noise or when setting the target value for learning, as will be described later.

The learning model learned by the learning device 1 according to the present embodiment is a learning model for block noise detection targeting images displayed on the navigation device 3 as described above, but more specifically, This is a learning model that determines whether the image data generated in the device 3 corresponds to an abnormal image (an image that causes block noise when displayed) before displaying the image data on the screen. Although the details will be described later, the navigation device 3 determines that the image is an abnormal image when the area value of the area where block noise is predicted to occur is equal to or greater than the threshold value, and in particular, the learning model calculates the optimal value by learning for the threshold value. calculate.

Next, a schematic configuration of the navigation device 3 included in the learning system 2 will be described using FIG. 3. FIG. 3 is a block diagram showing the navigation device 3 according to this embodiment.

As shown in FIG. 3, the navigation device 3 according to the present embodiment includes a current position detection unit 31 that detects the current position of the vehicle in which the navigation device 3 is mounted, a data recording unit 32 in which various data are recorded, A navigation ECU 33 that performs various arithmetic processing based on input information, an operation unit 34 that accepts operations from the user, and information regarding maps around the vehicle and guidance routes set in the navigation device 3 for the user. a liquid crystal display 35 that displays route guidance, a speaker 36 that outputs audio guidance regarding route guidance, warnings about risk factors, etc., a DVD drive 37 that reads a DVD as a storage medium, and a probe center and VICS (registered trademark: Vehicle Information and Communication). The communication module 38 communicates with an information center such as a system center. Further, the navigation device 3 is connected to sensors installed in the vehicle in which the navigation device 3 is mounted, an ECU for vehicle control, etc. via an in-vehicle network such as CAN.

Below, each component included in the navigation device 3 will be explained in order.
The current position detection unit 31 includes a GPS 41, a vehicle speed sensor 42, a steering sensor 43, a gyro sensor 44, etc., and is capable of detecting the current vehicle position, direction, vehicle speed, current time, etc. . Here, in particular, the vehicle speed sensor 42 is a sensor for detecting the travel distance and vehicle speed of the vehicle, generates pulses according to the rotation of the drive wheels of the vehicle, and outputs the pulse signal to the navigation ECU 33. Then, the navigation ECU 33 calculates the rotation speed and travel distance of the drive wheels by counting the generated pulses. Note that it is not necessary for the navigation device 3 to include all of the four types of sensors described above, and the navigation device 3 may be configured to include only one or more types of sensors.

The data recording unit 32 also has a hard disk (not shown) as an external storage device and a recording medium, and reads map information DB 45, learning information DB 46, predetermined programs, etc. recorded on the hard disk, and also writes predetermined data to the hard disk. It is equipped with a recording head (not shown) which is a driver for writing. Note that the data recording section 32 may include a memory card, an optical disk such as a CD, or a DVD instead of a hard disk. Alternatively, the map information DB 45 may be stored in an external server and acquired by the navigation device 3 through communication.

Here, the map information DB 45 includes, for example, link data related to roads (links), node data related to node points, facility data related to facilities, search data used for route search processing, map display data for displaying a map, and each intersection. Contains intersection data related to, search data for searching points, etc.

Further, the learning information DB 46 is a storage means in which information regarding learning models learned by the learning device 1 is stored. In this embodiment, as will be described later, the navigation device 3 determines whether or not the generated image data before being displayed on the liquid crystal display 35 corresponds to an abnormal image (an image that causes block noise when displayed), and When it is determined that this applies, the image data is regenerated, and the threshold value used in the determination is included in the learning model as a learning value. Note that it is desirable that the information regarding the learning model stored in the learning information DB 46 be acquired from the learning device 1 via communication or a storage medium, and can be updated as appropriate based on the acquired information.

On the other hand, the navigation ECU (electronic control unit) 33 is an electronic control unit that controls the entire navigation device 3, and includes a CPU 51 as an arithmetic unit and a control device, and a working memory when the CPU 51 performs various arithmetic processes. A RAM 52 stores route data when a route is searched, and a ROM 53 stores control programs as well as an image display processing program (see FIG. 10), which will be described later. It is equipped with an internal storage device such as a flash memory 54 for storing programs. Furthermore, it also includes a GPU 55 that performs calculations particularly related to image drawing, and a VRAM (video memory) 56 that stores information related to image drawing. However, the GPU 55 and VRAM 56 are not essential, and the CPU 51 and RAM 52 may perform these operations instead.

The operation unit 34 is operated when inputting a departure point as a travel start point and a destination as a travel end point, and has a plurality of operation switches (not shown) such as various keys and buttons. Then, the navigation ECU 33 performs control to execute various corresponding operations based on switch signals output by pressing each switch. Note that the operation unit 34 may include a touch panel provided in front of the liquid crystal display 35. Further, it may include a microphone and a voice recognition device.

The liquid crystal display 35 also displays map images including roads, traffic information, operation guidance, operation menus, key guidance, the guidance route set by the navigation device 3, guidance information along the guidance route, news, and weather forecasts. , time, email, TV programs, etc. are displayed. Note that a HUD or HMD may be used instead of the liquid crystal display 35. Furthermore, in this embodiment, using a learning model built into the navigation device 3, processing for reducing block noise is performed on an image displayed on the liquid crystal display 35 as described later.

Further, the speaker 36 outputs audio guidance for guiding travel along the guide route and traffic information based on instructions from the navigation ECU 33.

Further, the DVD drive 37 is a drive that can read data recorded on recording media such as DVDs and CDs. Then, based on the read data, music and video are played back, and the map information DB 45 and learning information DB 46 are updated. Note that a card slot for reading and writing a memory card may be provided in place of the DVD drive 37.

Further, the communication module 38 is a communication device for receiving traffic information transmitted from a traffic information center, such as a VICS center or a probe center, and is, for example, a mobile phone or a DCM. Further, communication with the learning device 1 may be made possible.

Next, a learning processing program executed by the CPU 21 in the learning device 1 according to the present embodiment having the above configuration will be described based on FIG. 4. FIG. 4 is a flowchart of the learning processing program according to this embodiment. Here, the learning processing program is executed after a certain period of time has passed since the previous processing was executed, and uses the images stored in the teacher image DB 12 to detect block noise occurring in the image displayed on the navigation device 3. This is a program that performs training on a learning model and generates a more optimal learning model. Further, a program shown in a flowchart in FIG. 4 below is stored in the RAM 22, ROM 23, etc. included in the control unit 11, and is executed by the CPU 21.

First, in step (hereinafter abbreviated as S) 1 of the learning processing program, the CPU 21 extracts the learning image 25 used for learning the learning model from the teacher image DB 12. The learning image 25 is an image to be displayed on the navigation device 3 (for example, a map image, a movement guide image, a facility guide image, etc.), and may be an image actually displayed on the navigation device 3 in the past. The images may be pseudo-generated for learning purposes. The means for collecting images actually displayed in the navigation device 3 in the past is, for example, by capturing an image displayed on the liquid crystal display 35 and visually extracting only normal images from the captured image. Further, during the learning process, a part of the secondary test image 27 may be added as the learning image 25 as described later (S13). The learning images 25 extracted in S1 are basically only normal images without block noise, and the number of images is approximately 90,000.

Note that the approximately 90,000 learning images 25 extracted in S1 include many substantially identical images (hereinafter referred to as duplicate images) that are completely the same or differ only in a small portion. Even if learning is performed using all of these multiple duplicate images as the target, there is a large difference in the learning effect even if learning is performed using only one duplicate image selected from the multiple duplicate images. There isn't. Therefore, in order to reduce the processing load, as shown in FIG. 5, only one of the duplicate images is left and the other duplicate images are excluded from subsequent processing (S2). In particular, in S2, the CPU 21 uses a hash function to search for the duplicated images from among the approximately 90,000 learning images 25 extracted in S1. Note that duplicate images may be determined by means other than searching using a hash function. As a result of performing the process in S2, the number of learning images 25 to be processed in subsequent processes is reduced to approximately 45,000.

Subsequently, in S3, the CPU 21 generates a block noise image (hereinafter referred to as a pseudo block noise image) in which a predetermined block-shaped area is pseudo-set as a noise region. Here, FIG. 6 is a diagram showing an example of the pseudo block noise image 61 generated in S3. The size of the pseudo block noise image 61 is set to be the same size as the learning image 25. Further, the noise area is an area in which one or more square blocks are connected, but the size and position of the block shapes that become the noise area are set randomly within the image. For example, use the following method to set it.
(1) Obtain frequencies at random and generate a composite wave of sine waves and cosine waves.
(2) Sampling values are sampled from the composite wave in steps of block size in the vertical direction of the image.
(3) The above processes (1) and (2) are also performed in the horizontal direction of the image.
(4) The sampling values sampled in the vertical and horizontal directions are classified into either white (255) or black (0) based on the threshold value. The area set to black becomes the noise area.

In addition, in S3, the CPU 21 generates pseudo blocks corresponding to the number of learning images 25 (for example, about 45,000 images) extracted from the teacher image DB 12 in S1 and after removing duplicate images in S2. A noise image 61 is generated. For each of the plurality of generated pseudo block noise images 61, the size and position of the block shape serving as the noise area are randomly set using the means (1) to (4) above.

Thereafter, in S4, the CPU 21 randomly sets the color of the noise region for each of the plurality of pseudo block noise images 61 for the pseudo block noise images generated in S3. For example, a color is randomly selected from the HSV color space and the color of the noise region is converted to the selected color. Furthermore, the transmittance of the noise area is also randomly set. Note that the color and transmittance of the noise area may be the same within one image, or may be set differently for each area even within one image.

Subsequently, in S5, the CPU 21 extracts the pseudo blocks generated in S3 as shown in FIG. The noise image 61 is superimposed and synthesized. Furthermore, in S6, the CPU 21 converts the superimposed image into a gray scale. As a result, as shown in FIG. 7, a pseudo-abnormal image 62 is generated in which block noise is artificially generated in the learning image 25, which is a normal image. Note that the position, shape, transmittance, and color (grayscale color) of the block noise occurring in the pseudo-abnormal image 62 are randomly set for each image by the processing in S3 and S4. Furthermore, if the number of learning images 25 to be processed is approximately 45,000, approximately 45,000 pseudo abnormal images 62 will also be generated.

Thereafter, in S7, the CPU 21 converts the pseudo block noise image generated in S6 (i.e., the abnormal image in which block noise has been pseudo-generated) and the learning image 25 (i.e., the normal image) extracted from the teacher image DB 12 into the teacher image. is randomly input to the learning model and learning is performed. Note that the ratio of normal images and abnormal images input to the learning model is, for example, normal image: abnormal image = 4:6. Although the ratio can be changed as appropriate, it is desirable to increase the ratio of abnormal images in order to detect all abnormal images. Learning using the learning model is performed until all the pseudo block noise images 61 generated in S6 are input. Further, as the learning model, for example, the U-Net model is used.

As for the learning using the learning model in S7, for example, it is predicted whether a randomly input image corresponds to a "normal image" or an "abnormal image." Then, the learning model is trained by comparing the results predicted by the learning model with a previously prepared answer as to whether the image is a normal image or an abnormal image. In addition to predicting "normal images" or "abnormal images," the learning model is trained to predict in which areas within the image block noise occurs, and the results predicted by the learning model are combined with the predictions prepared in advance. The learning model may be trained by comparing the answer with the answer of the area where block noise occurs.

After that, in S8, the CPU 21 trains the learning model using all the generated pseudo block noise images 61, and then uses the primary test image stored in the teacher image DB 12 to evaluate the learning results of the learning model. 26 to calculate the evaluation value mIoU. Specifically, the primary test image 26 is input to the learned model after learning, and the area where block noise occurs in the primary test image 26 is predicted. Then, the result predicted by the learning model is compared with a previously prepared answer for the area where block noise occurs, and an evaluation value mIoU is calculated. Note that the evaluation value mIoU is 1.0 if the prediction result and the correct answer completely match, and a value closer to 0 is calculated as the deviation becomes larger. For example, the evaluation value mIoU is calculated by using a confusion matrix for each pixel.

The primary test image 26 is an image actually displayed on the navigation device 3 in the past, and includes a normal image without block noise and an abnormal image with block noise. The number of primary test images 26 is approximately 20,000, and the average value of the evaluation values mIoU calculated for each of the 20,000 primary test images 26 is the final evaluation value of this learning model. Let it be mIoU. However, the median value or the mode may be used instead of the average value.

Thereafter, the CPU 21 repeats the processes of S1 to S8 described above a predetermined number of times (for example, 100 times). Each time the process is repeated, the position, shape, color, and transmittance of the noise region of the pseudo block noise image 61 generated in S3 change randomly, that is, the teacher image to be learned by the learning model changes, so the learning model The evaluation value mIoU used to evaluate the learning result will also change each time it is repeated.

Next, in S9, the CPU 21 selects the learning model that has the highest evaluation value mIoU calculated in S8 (hereinafter referred to as the best learning model) in the learning that has been repeated a predetermined number of times. Then, in order to reevaluate the learning results of the selected best learning model, the secondary test image 27 stored in the teacher image DB 12 is input to the best learning model, and the input secondary test image 27 The area value of a region where block noise is predicted to occur (hereinafter referred to as a block noise region) is calculated.

The secondary test image 27 uses images actually displayed on the navigation device 3 in the past, similar to the primary test image 26, and includes a normal image with no block noise and an abnormal image with block noise. Include each image. The number of secondary test images 27 is approximately 790,000, and the area value of the block noise region is calculated for each of the 790,000 secondary test images 27.

Next, in S10, the CPU 21 divides the input secondary test image 27 into a normal image and an abnormal image, and calculates the area value distribution (histogram) of the block noise region calculated in S9. ) is generated. For example, FIG. 8 is a diagram showing an example of the distribution of area values of the block noise region generated in S10. As shown in FIG. 8, the distribution of the area values of the block noise region generated in S10 is determined by dividing the calculated area values into predetermined ranges, with the horizontal axis representing the area value and the vertical axis representing the frequency. A histogram showing the included frequencies.

Basically, in the distribution of the area values of the block noise region when the inputted secondary test image 27 is a normal image, the frequency is concentrated around 0. On the other hand, when the inputted secondary test image 27 is an abnormal image, the area values of the block noise region will be distributed in a somewhat dispersed frequency distribution.

Next, in S11, the CPU 21 sets the first condition that the recall rate (Recall) for determining an abnormal image as an abnormal image is 100%, that is, that no abnormal image is overlooked, and The threshold value is calculated with the second condition being that the value (also referred to as ``(also referred to as ``(also referred to as ``(also referred to as ``(also referred to as "(also referred to as "))" is the highest" is the second condition. In this embodiment, the learning model learns whether the image data generated in the navigation device 3 corresponds to an abnormal image (an image that causes block noise when displayed) before displaying the image data on the screen. This is learning to judge. Then, the learning model determines that the image is an abnormal image when the area value of a block noise region where block noise is predicted to occur is equal to or larger than a threshold value, and in S11, an optimal value for the threshold value is calculated by learning.

Here, what is important in calculating the threshold value is that no abnormal images are overlooked when reflected on the navigation device 3, that is, that abnormal images are not determined to be normal images. Furthermore, it is desirable to make the precision rate, that is, the percentage of images determined to be abnormal images by the navigation device 3 as actually abnormal images, as high as possible. Therefore, in the distribution of the area values of the block noise region generated as shown in FIG. 8, the lowest area value that has the frequency when the input secondary test image 27 is an abnormal image is set as the threshold value.

Further, in S11, the CPU 21 calculates the matching rate at the set threshold value. Here, the precision rate is the proportion (accuracy rate) of images that are determined to be abnormal images when reflected on the navigation device 3 and are actually abnormal images, as described above. In the distribution of the area values of the block noise region generated in S10, the ratio of the frequency of the abnormal image to the combined frequency of the normal image and the abnormal image for area values equal to or higher than the threshold value is the matching rate. In other words, if there is no frequency of normal images in area values greater than or equal to the threshold, the precision will be the ideal value of 100%, but since it is difficult to reach 100%, the target value is set to 99%, for example, as described below. Set.

After that, in S12, the CPU 21 determines whether or not the values of the recall rate (Recall) and the precision rate (Precision) when the threshold value set in S11 is reflected in the navigation device 3 reach the target values. Note that the target value for the recall rate is 100%, and the target value for the precision rate is 99%. However, the target value can be changed as appropriate. For example, the target value of the precision rate may be set to 98% or 97%. In addition, in this embodiment, the threshold value is set on the condition that the recall rate is 100% as described above, so in S12, it is determined that the recall rate always reaches the target value. A threshold value may be set on the condition that the rate is 99%, and it may be determined in S12 whether the recall rate reaches the target value.

Then, if it is determined that the recall rate (Recall) and precision rate (Precision) when the threshold value set in S11 is reflected in the navigation device 3 reaches the target value (S12: YES), S14 transition to. On the other hand, if it is determined that the recall rate (Recall) and precision rate (Precision) when the threshold value set in S11 is reflected in the navigation device 3 does not reach the target value (S12: NO). Then, the process moves to S13.

In S13, in order to improve the precision rate, the CPU 21 uses the normal image in which the block noise region could not be correctly predicted using the current learning model, that is, the distribution of area values of the block noise region generated in S10. , normal images in which the area value of the block noise region is greater than or equal to the threshold are added as learning targets. Specifically, as shown in FIG. 9, in the area value distribution of the block noise region, a normal image having an area value equal to or greater than a threshold value is added to the learning image 25. That is, a part of the secondary test image 27 is newly added to the learning image 25.

Thereafter, the process returns to S1 and the CPU 21 performs learning of the learning model based on the newly added learning image 25. By adding normal images that could not correctly predict block noise regions as learning targets, it is expected that the precision of the learning model that performs new learning after that will improve.

On the other hand, in S14, the CPU 21 incorporates the best learning model including the threshold set in S11 into the navigation device 3 as a learning result. Incidentally, the best learning model is incorporated by, for example, causing the navigation device 3 to acquire necessary information via communication or a storage medium. As a result, the navigation device 3 can use the incorporated best learning model to perform processing to reduce the occurrence of block noise in images to be displayed as described later.

Next, an image display processing program executed by the CPU 51 and the GPU 55 in the navigation device 3 according to this embodiment will be described based on FIG. 10. FIG. 10 is a flowchart of the image display processing program according to this embodiment. Here, the image display processing program is executed after the ACC power supply (accessory power supply) of the vehicle is turned on, and by incorporating the learning model learned by the learning device 1 in the above-mentioned learning processing program (see FIG. 4), This is a program that reduces the occurrence of block noise in displayed images. Further, a program shown in a flowchart in FIG. 10 below is stored in the RAM 52, ROM 53, etc. included in the navigation ECU 33, and is executed by the CPU 51 or GPU 55.

First, in the image display processing program, the CPU 51 generates image data of an image to be displayed in the navigation device 3 in S21. Note that the image to be displayed in the navigation device 3 varies depending on the current status of the navigation device 3, but basically a map image around the current position of the vehicle is displayed. Additionally, when the vehicle approaches a guidance intersection while the vehicle is being guided, an enlarged view of the intersection, etc. is also displayed. Furthermore, when the user performs a facility search, a list of searched facilities, facility information, etc. are also displayed.

Subsequently, in S22, the CPU 51 uses the learning model stored in the learning information DB 46 to detect block noise before displaying the image data generated in S21 on the liquid crystal display 35. Specifically, the CPU 51 uses the learning model to specify the area value of an area where block noise is predicted to occur, and determines that block noise will occur in the image data if the area value is equal to or greater than a threshold value. Note that the optimal value of the threshold value is calculated in the aforementioned learning processing program (see FIG. 4) (S11).

Thereafter, in S23, the CPU 51 determines whether it is determined that block noise occurs in the image data generated in S21 based on the detection result of S22, and if it is determined that block noise occurs (S23: If YES), the process moves to S24.

In S24, the CPU 51 estimates that there is an abnormality in the generated image data, and performs processing to generate the image data again. After that, the process moves to S22, and block noise is detected again on the regenerated image data.

On the other hand, if it is determined that block noise does not occur in the image data generated in S21 (S23: NO), the process moves to S25. In S25, the generated image data is temporarily stored in a memory such as the RAM 52 or VRAM 56.

Thereafter, the GPU 55 performs a drawing process to draw the image data stored in the memory on the liquid crystal display 35. As a result, it is possible to prevent block noise from occurring in the image displayed on the liquid crystal display 35. Note that the CPU 51 may perform image drawing processing.

As explained in detail above, in the learning device 1 according to the present embodiment and the computer program executed by the learning device 1, a normal image for learning without block noise is acquired (S1), and a predetermined area of a block shape is acquired. A pseudo block noise image is generated (S3), which is an image of block noise that is set in a pseudo noise region, and the pseudo block noise image is superimposed and synthesized on the normal image for learning, thereby creating the normal image for learning. A pseudo abnormal image in which block noise is generated is generated as training data for use in learning a learning model for detecting block noise (S6), and the generated training data is used to train the learning model. Since learning is performed (S1 to S13), it is easier to obtain teacher data than in the past. As a result, it is possible to construct a learning model that is widely applicable in terms of improving the quality of images displayed on image display devices (eg, navigation devices, smartphones, tablet terminals, personal computers).

It should be noted that the present invention is not limited to the embodiments described above, and it goes without saying that various improvements and modifications can be made without departing from the gist of the present invention.
For example, in the embodiment described above, recall and precision are used as means for evaluating the learning results of the learning model, but the learning results may be evaluated using other evaluation means.

Furthermore, in this embodiment, when generating the pseudo block noise image 61, which is an image of block noise, each parameter of the size and position of the block shape to be the noise region, the color and transmittance of the noise region is changed at random. However, only a part of each parameter (for example, the size and position of the block shape used as the noise region) may be randomly set. Further, the operator who operates the learning device 1 may be able to set the parameters.

Further, in this embodiment, the pseudo abnormal image in which block noise is generated in the learning image 25 is subjected to gray scale conversion, but the gray scale conversion may not be performed.

Further, in the above embodiment, the learning processing program (FIG. 4) is executed by the learning device 1, but it may be executed by the navigation device 3. In that case, the navigation device 3 having a learning function corresponds to the learning device according to the present invention.

Furthermore, in the above embodiment, the navigation device 3, which is an on-vehicle device, is used as an example to reduce block noise in a displayed image. It's okay. For example, a smartphone, a tablet terminal, a personal computer, etc. may be used instead of the navigation device 3.

Moreover, although the embodiment embodying the learning device for detecting block noise according to the present invention has been described above, the learning device for detecting block noise can also have the following configuration, and in that case, has the following effects.

For example, the first configuration is as follows.
A normal image acquisition means (21) that acquires a normal image for learning (25) in which block noise does not occur, and a pseudo block noise image (21) that is an image of block noise in which a predetermined block-shaped area is pseudo-set as a noise region. 61), and a pseudo block noise image generating means (21) that generates a pseudo block noise image on the learning normal image by superimposing and composing the pseudo block noise image on the learning normal image. It has a teacher data generating means (21) that generates a pseudo abnormal image (62) in which noise is generated as teacher data used for learning a learning model for detecting block noise.
According to the learning device for block noise detection having the above configuration, an image in which block noise is generated in a pseudo normal image is generated and used as training data, and learning for detecting block noise is performed. Therefore, it is easier to obtain training data than in the past. As a result, it becomes possible to construct a learning model that is widely applicable in terms of improving the quality of images displayed on image display devices.

Further, the second configuration is as follows.
The pseudo block noise image generation means (21) generates the pseudo block noise image (61) by setting the size and position of a block shape to be a noise region, the color and transmittance of the noise region, respectively.
According to the learning device for block noise detection having the above configuration, even when generating a pseudo abnormal image in which block noise is generated, it is possible to reproduce an abnormal image in which block noise actually occurs. This makes it possible to use the data as training data.

Moreover, the third configuration is as follows.
The pseudo block noise image generation means (21) generates a plurality of pseudo block noise images (61) by randomly changing at least one of the size and position of the block shape to be a noise region, the color and transmittance of the noise region as parameters. ), and the teacher data generation means (21) generates a plurality of pseudo abnormal images (62), which are synthesized by superimposing a plurality of pseudo block noise images (61) on the learning normal image (25). ), and the learning model is trained using the plurality of pseudo abnormal images as training data.
According to the learning device for detecting block noise having the above configuration, when generating a pseudo-abnormal image in which block noise is generated, the learning device randomly changes the parameters and generates a plurality of images based on various block noise patterns. By generating pseudo-abnormal images and using a plurality of generated pseudo-abnormal images as training data, a high learning effect can be expected assuming various block noises that may actually occur.

Further, the fourth configuration is as follows.
a learning result evaluation means (21) that performs learning of the learning model multiple times by changing the pseudo abnormal images (62) used as training data, and evaluates the learning results of the learning model for each of the multiple learnings; learning model employing means (21) for employing the learning model that has shown a high learning result.
According to the learning device for block noise detection having the above configuration, learning is performed multiple times by changing the pseudo abnormal images used as teaching data, and the learning model after learning with the highest evaluation is adopted. Even if a pseudo-abnormal image that artificially generates block noise contains unnatural abnormal images that cannot actually exist, the learning model that learned them as training data can eliminate them, making it more practical. It becomes possible to employ a learning model that is trained using pseudo-abnormal images close to the abnormal images of , as training data.

Further, the fifth configuration is as follows.
The learning result evaluation means (21) uses the learning model learned for each of the plurality of learnings to evaluate the first test image (26) including the abnormal test image in which block noise has occurred. A predictive determination of the area where the noise is occurring is made, and the degree of coincidence between the area where the block noise is predicted to occur and the area where the block noise is actually occurring is evaluated.
According to the learning device for block noise detection having the above configuration, since the learning results are evaluated by predictive judgment of the area where block noise occurs, a learning model with a high learning evaluation for detecting block noise is adopted. It becomes possible to do so.

Further, the sixth configuration is as follows.
Using the learning model adopted by the learning model adoption means (21), a second test image ( 27) Area prediction means (21) that causes the group to predict areas where block noise has occurred; Area distribution generation means (21) that separately generates a normal image and the test abnormal image; and a learning model re-evaluation means that re-evaluates the learning model adopted by the learning model adoption means based on the area distribution. (21).
According to the learning device for block noise detection having the above configuration, the area where block noise occurs is predicted for a normal test image in which block noise does not occur and an abnormal test image in which block noise occurs, respectively. Since the learning result is evaluated based on the area distribution of areas predicted to have block noise, normal images without block noise are detected as normal images, and normal images with block noise are detected as normal images. It becomes possible to evaluate a learning model that targets detection of abnormal images as abnormal images.

Moreover, the seventh configuration is as follows.
The learning model re-evaluation means (21) evaluates whether or not the recall rate and precision rate have reached preset target values, and if the recall rate and precision rate have not reached the target values, , has a relearning means (21) for learning the learning model again.
According to the learning device for detecting block noise having the above configuration, it is possible to adopt a learning model whose recall rate and precision rate have reached preset target values, while the recall rate and precision rate are set in advance. By relearning learning models that do not reach the set target value, it is possible to improve recall and precision.

Further, the eighth configuration is as follows.
The relearning means (21) selects a normal test image for which block noise areas could not be predicted correctly by the learning model, among the normal test images included in the second test image (27) group. It is added to the normal image for learning (25) and the learning model is trained again.
According to the learning device for block noise detection having the above configuration, for the learning model whose recall rate and precision rate do not reach the preset target values, the test normality that cannot predict the block noise area correctly is determined. By adding images to the learning target and performing relearning, it is possible to improve recall and precision.

Further, the ninth configuration is as follows.
An image data generation means (51) that generates image data to be displayed on an image display device (35); and before displaying an image based on the image data generated by the image data generation means, the image data block noise detection means (51) for detecting block noise using a learning model learned based on the teacher data; A block noise reduction means (51) that performs processing for reducing block noise.
According to the learning device for block noise detection having the above configuration, by using the learned learning model, it is possible to determine whether or not block noise is generated in image data before displaying it on the image display device. It becomes possible to detect. Then, by performing processing for reducing block noise on image data in which block noise has been detected, it is possible to prevent images in which block noise has occurred from being displayed on the image display device.

Moreover, the tenth configuration is as follows.
The block noise detection means (51) predicts an area where block noise will occur in the image data, and adds the block noise to the image data when the area value of the area where block noise is predicted to occur is greater than or equal to a threshold value. is detected when it occurs.
According to the learning device for detecting block noise having the above configuration, the learned learning model is used to predict areas where block noise will occur in image data before displaying it on an image display device. It becomes possible to accurately detect whether the image data before being displayed on the device is image data that causes block noise.

1 Learning device 2 Learning system 3 Navigation device 12 Teacher image DB
21 CPU
22 RAM
23 ROM
25 Learning image 26 Primary test image 27 Secondary test image 33 Navigation ECU
35 LCD display 51 CPU
55 GPU
61 Pseudo block noise image 62 Pseudo abnormal image

Claims (10)

normal image acquisition means for acquiring a normal image for learning without block noise;
pseudo block noise image generation means for generating a pseudo block noise image that is a block noise image in which a predetermined block-shaped area is pseudo-set as a noise region;
In order to detect block noise, a pseudo-abnormal image is generated in which block noise is pseudo-generated in the normal image for learning by superimposing and composing the pseudo-block noise image on the normal image for learning. a teacher data generation means for generating teacher data used for learning the learning model ;
The pseudo block noise image generation means is a learning device that generates the pseudo block noise image by setting the size and position of a block shape to be a noise region, the color and transmittance of the noise region, respectively . The pseudo block noise image generation means generates a plurality of pseudo block noise images by randomly changing at least one of the size and position of a block shape to be a noise region, the color and transmittance of the noise region as parameters,
The teacher data generating means generates a plurality of pseudo abnormal images obtained by superimposing and synthesizing a plurality of pseudo block noise images on the learning normal image,
The learning device according to claim 1 , wherein the learning model is trained using a plurality of the pseudo abnormal images as training data. Learning the learning model multiple times by changing the pseudo abnormal images used as training data,
learning result evaluation means for evaluating the learning results of the learning model every time the learning is performed a plurality of times;
3. The learning device according to claim 1, further comprising a learning model employing unit that employs the learning model that has shown the highest evaluation learning result. The learning result evaluation means includes:
Using a learning model learned each time a plurality of learnings are performed, a predictive judgment of an area where block noise is occurring is performed on a first test image including an abnormal test image where block noise is occurring;
4. The learning device according to claim 3 , wherein the learning device evaluates the degree of coincidence between an area where block noise is predicted to occur and an area where block noise actually occurs. Using the learning model adopted by the learning model adoption means, a second test image group including a normal test image in which no block noise occurs and an abnormal test image in which block noise occurs. Area prediction means for predicting an area where block noise is occurring;
Area distribution generating means for generating an area distribution of an area predicted to have block noise by the area predicting means, dividing it into the normal test image and the abnormal test image;
5. The learning device according to claim 3 , further comprising learning model re-evaluation means for re-evaluating the learning model adopted by the learning model adoption means based on the area distribution. The learning model re-evaluation means evaluates whether the recall rate and the precision rate have reached preset target values,
6. The learning device according to claim 5 , further comprising a relearning means for learning the learning model again when the recall rate and the precision rate have not reached the target values. The relearning means selects a normal test image for which the block noise area could not be correctly predicted by the learning model from among the normal test images included in the second test image group, into the normal learning image. The learning device according to claim 6 , wherein the learning model is added and the learning model is learned again. image data generation means for generating image data to be displayed on an image display device;
Before displaying an image based on the image data generated by the image data generation means, block noise is detected using a learning model trained on the image data based on the teacher data. detection means;
8. A block noise reduction unit that performs processing to reduce block noise on image data in which block noise has been detected by the block noise detection unit. learning device. The block noise detection means predicts an area where block noise will occur in the image data, and detects that the block noise will occur in the image data if the area value of the area where block noise is predicted to occur is greater than or equal to a threshold value. The learning device according to claim 8 , which detects. computer,
normal image acquisition means for acquiring a normal image for learning without block noise;
pseudo block noise image generation means for generating a pseudo block noise image that is a block noise image in which a predetermined block-shaped area is pseudo-set as a noise region;
In order to detect block noise, a pseudo-abnormal image is generated in which block noise is pseudo-generated in the normal image for learning by superimposing and composing the pseudo-block noise image on the normal image for learning. a teacher data generation means for generating teacher data to be used for learning the learning model;
A computer program for functioning as
The pseudo block noise image generation means is a computer program that generates the pseudo block noise image by setting the size and position of a block shape to be a noise region, the color and transmittance of the noise region, respectively.

Images