JP7537753B2 - Management device and management method - Google Patents

Description

The present invention relates to a management device and a management method.

Patent document 1 discloses a product monitoring device that monitors display disarray and other issues based on image data captured in a display area.

International Publication No. 2016/038774

Customers may place objects (foreign objects) in the display areas that are different from the products assigned to each display area. For example, they may place products in each display area that are different from the products assigned to each display area, or place objects other than products. This situation not only negatively impacts the store's image, but may also affect the quality of the products. Patent Document 1 does not provide a means to solve this problem. The present invention aims to solve this problem.

According to the present invention,
an image analysis means for analyzing image data obtained by photographing a display area and detecting from the image data other products than the products allocated to the display area;
A determination means for determining an urgency of an operation to collect the other products based on the other products and the display area;
an output means for outputting the urgency of the other product together with information indicating the other product;
A management device is provided.

Further, according to the present invention,
The computer
an image analysis step of analyzing image data of a display area and detecting from the image data products other than the products allocated to the display area;
a determination step of determining an urgency of an operation to collect the other products based on the other products and the display area;
an output step of outputting the urgency of the other product together with information indicating the other product;
A management method for performing the above is provided.

According to the present invention, it is possible to detect when an object (foreign object) different from the product assigned to each display area is placed in the display area.

FIG. 2 is a diagram conceptually illustrating an example of a hardware configuration of a management device according to the present embodiment. FIG. 2 is a functional block diagram of a management device according to the present embodiment; FIG. 4 is a diagram for explaining the process of the management device of the present embodiment. FIG. 2 is a diagram illustrating an example of information processed by a management device according to the embodiment. FIG. 2 is a diagram illustrating an example of information processed by a management device according to the embodiment. 10 is a flowchart showing an example of a process flow of a management device of the present embodiment. FIG. 4 is a diagram illustrating an example of information output by a management device according to the present embodiment. FIG. 2 is a functional block diagram of a management device according to the present embodiment; FIG. 4 is a diagram illustrating an example of information output by a management device according to the present embodiment. 10 is a flowchart showing an example of a process flow of a management device of the present embodiment. FIG. 2 is a functional block diagram of a management device according to the present embodiment; FIG. 2 is a diagram illustrating an example of information processed by a management device according to the embodiment. FIG. 2 is a diagram illustrating an example of information processed by a management device according to the embodiment. FIG. 4 is a diagram illustrating an example of information output by a management device according to the present embodiment. FIG. 2 is a functional block diagram of a management device according to the present embodiment; FIG. 2 is a diagram illustrating an example of information processed by a management device according to the embodiment. FIG. 4 is a diagram illustrating an example of information output by a management device according to the present embodiment. FIG. 2 is a diagram illustrating an example of information processed by a management device according to the embodiment.

First Embodiment
First, an overview of the management device of this embodiment will be described. The management device of this embodiment analyzes image data captured of a display area, and detects foreign objects that are objects different from the products assigned to the display area. If a foreign object is detected, an alert is output.

According to the management device of this embodiment, the store clerk can quickly recognize that an object (foreign object) different from the product assigned to each display area has been placed in the display area. The store clerk can then remove the foreign object or move it to an appropriate location. As a result, the undesirable situation of "an object (foreign object) different from the product assigned to each display area has been placed in the display area" can be quickly resolved.

Next, the configuration of the management device will be described. First, an example of the hardware configuration of the management device will be described. The management device is realized by any combination of hardware and software, centered on the CPU (Central Processing Unit) of any computer, memory, programs loaded into the memory, a storage unit such as a hard disk that stores the programs (in addition to programs that are stored before the device is shipped, it can also store programs downloaded from storage media such as CDs (Compact Discs) or servers on the Internet), and a network connection interface. Those skilled in the art will understand that there are many variations in the methods and devices for implementing the device.

Figure 1 is a block diagram illustrating the hardware configuration of a management device. As shown in Figure 1, the management device has a processor 1A, a memory 2A, an input/output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules.

The bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input/output interface 3A to send and receive data to each other. The processor 1A is an arithmetic processing device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The memory 2A is a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The input/output interface 3A includes interfaces for acquiring information from input devices (e.g., keyboard, mouse, microphone, physical keys, touch panel display, code reader, etc.), external devices, external servers, external sensors, etc., and interfaces for outputting information to output devices (e.g., display, speaker, printer, mailer, etc.), external devices, external servers, etc. The processor 1A can issue commands to each module and perform calculations based on the results of those calculations.

Figure 2 shows an example of a functional block diagram of the management device 10. As shown in the figure, the management device 10 has an image analysis unit 11 and an output unit 12.

First, one or more cameras (hereafter referred to as store cameras) are installed in a store. The store cameras are installed to capture images of the display area. The store cameras may capture moving images, or may capture still images at a predetermined time interval. The orientation and shooting conditions of the store cameras may be fixed, and the cameras may continue to capture images under the same conditions.

Figure 3 shows a schematic example of an image F shown in image data generated by a store camera. The image F shown shows a display shelf with three shelves. Since the display shelf is installed in a fixed position, if photographing is continued under the same conditions, the same part of the display shelf will be photographed continuously.

In this embodiment, one or more areas are specified in advance for each store camera within the image F captured by each store camera, and information identifying the display area (e.g., display area code) is associated with each specified area and registered. As described above, when continuing to capture images under the same conditions, the same locations on the display shelves are continually captured, so the position of each display area within image F is fixed.

In image F shown in FIG. 3, three areas are specified on the top row, and each is associated with a display area code of T1-1-1 through T1-1-3. Two areas are specified on the second row from the top, and each is associated with a display area code of T1-2-1 and T1-3-2. Two areas are specified on the bottom row, and each is associated with a display area code of T1-3-1 and T1-3-2.

In addition, in this embodiment, as shown in FIG. 4, information identifying the products assigned to each of the multiple display areas (e.g., product codes) is registered in advance in association with each of the multiple display areas. "Products assigned to each display area" are products that are displayed in each display area.

Furthermore, in this embodiment, as shown in FIG. 5, the external appearance features of each of the multiple products are registered in advance in association with each of the multiple products. For each product, multiple patterns of feature values that appear in the image when photographed from multiple angles may be registered.

The image analysis unit 11 analyzes image data of the display areas (image data generated by the store cameras described above) and detects from the image data any foreign objects that are objects different from the products assigned to each display area. The image analysis unit 11 detects foreign objects present in each of the multiple display areas. It is preferable for the image analysis unit 11 to acquire and analyze image data generated by the store cameras with as little time loss as possible. The image analysis unit 11 can detect foreign objects by image analysis using the external features of each of the multiple products as shown in Figure 5. An example of the process for detecting foreign objects is illustrated below, but is not limited to this.

"Processing Example 1"
For example, the image analysis unit 11 detects one or more enclosed areas (objects) in the image F by a contour extraction process. Note that the object in the image F may be detected by other processes. After that, the image analysis unit 11 determines, for each detected area (area in which the detected object is shown), in which display area each area is included.

Then, for each detected region, the image analysis unit 11 determines whether the "image features of each region" match with the "appearance features of the products assigned to the display area that includes each region" (the degree of similarity is equal to or greater than a predetermined level).

If there is a match, the image analysis unit 11 determines that no foreign object is present in that area. On the other hand, if there is no match, the image analysis unit 11 determines that a foreign object is present in that area.

In this way, the image analysis unit 11 can detect an object from image data. Then, the image analysis unit 11 can determine whether or not the object is a foreign object based on the degree of similarity between the appearance of the object detected in the display area and the appearance of the product assigned to the display area.

"Processing Example 2"
For example, the image analysis unit 11 detects one or more enclosed areas (objects) in the image F by a contour extraction process. Note that the object in the image F may be detected by other processes. After that, the image analysis unit 11 determines, for each detected area (area in which the detected object is shown), in which display area each area is included.

In addition, the image analysis unit 11 performs a matching process for each detected area using the "image features of each area" and the "appearance features of each of the multiple products."

If the image does not match any of the products (if the similarity to any of the products is less than a predetermined level), the image analysis unit 11 determines that a foreign object is present in the area. If the image matches any of the products (if the similarity to any of the products is equal to or greater than a predetermined level), the image analysis unit 11 determines whether the matched product is a product assigned to the display area that includes the area.

If the area is a product assigned to a display area that includes that area, the image analysis unit 11 determines that no foreign object is present in that area. On the other hand, if the area is not a product assigned to a display area that includes that area, the image analysis unit 11 determines that a foreign object is present in that area.

In this way, the image analysis unit 11 can detect an object from image data. Furthermore, the image analysis unit 11 can determine which product the object corresponds to based on the degree of similarity between the appearance of the object detected in the display area and the appearance of each of a plurality of products. Then, the image analysis unit 11 can determine whether or not the object is a foreign object based on the determination result and/or the products assigned to the display area.

Returning to FIG. 2, the output unit 12 outputs an alert if a foreign object is detected. The output unit 12 can output the alert using any output device, such as a display, speaker, mailer, printer, warning lamp, or vibration device.

Next, an example of the processing flow of the management device 10 of this embodiment will be described using the flowchart in FIG. 6.

The image analysis unit 11 continues to acquire and analyze image data of the display areas. If the analysis detects a foreign object in any of the display areas that is different from the product assigned to that display area (Yes in S10), the output unit 12 outputs an alert (S11). If there is no instruction to end the process thereafter (No in S12), the process returns to S10 and continues.

According to the management device 10 of the present embodiment described above, image analysis can be used to quickly detect that an object (foreign object) different from the product assigned to each display area has been placed in the display area. When the management device 10 detects a foreign object, it can output an alert.

In such cases, the store staff can quickly grasp the situation and can then remove the foreign object or move it to an appropriate location. As a result, the undesirable situation of "an object (foreign object) different from the product assigned to each display area being placed in the display area" can be quickly resolved.

Second Embodiment
The management device 10 of this embodiment differs from that of the first embodiment in that it further outputs information indicating the display area in which the foreign object was detected. This will be described in detail below.

An example of the hardware configuration of the management device 10 in this embodiment is the same as that in the first embodiment.

Figure 2 shows an example of a functional block diagram of the management device 10 of this embodiment. As shown in the figure, the management device 10 has an image analysis unit 11 and an output unit 12. The configuration of the image analysis unit 11 is the same as in the first embodiment.

When a foreign object is detected, the output unit 12 outputs, in addition to an alert, information indicating the display area in which the foreign object was detected. The "information indicating the display area" output by the output unit 12 may be a display area code, or may be information expressing the display area such as the "aisle name," "corner name," or "which shelf level," etc. In the following, the information indicating the display area will be described on the assumption that it is a display area code. Figure 7 shows a schematic example of information output by the output unit 12 of this embodiment. The information is output using an output device such as a display or mailer.

An example of the processing flow of the management device 10 in this embodiment is similar to that in the first embodiment.

The management device 10 of this embodiment described above can achieve the same effects as the first embodiment.

In addition, according to the management device 10 of this embodiment, when a foreign object is detected, in addition to an alert, information indicating the display area in which the foreign object was detected (e.g., a display area code) can be output.

In such a case, the store clerk can quickly grasp the situation in which the foreign object is present, as well as the location where the foreign object is located. The clerk can then quickly go to that location and remove the foreign object or move it to an appropriate location. As a result, the undesirable situation of "an object (foreign object) different from the product assigned to each display area being placed in the display area" can be quickly resolved.

Third Embodiment
The management device 10 of this embodiment differs from the first and second embodiments in that it determines whether the detected foreign object is another product (a product that is not assigned to the display area in which the foreign object was detected) and, if it is another product, further outputs information indicating the display area assigned to that product. This will be described in detail below.

An example of the hardware configuration of the management device 10 in this embodiment is similar to that in the first and second embodiments.

Figure 8 shows an example of a functional block diagram of the management device 10 of this embodiment. As shown in the figure, the management device 10 has an image analysis unit 11, an output unit 12, and an identification unit 13.

As in the first and second embodiments, the image analysis unit 11 analyzes image data captured of a display area and detects from the image data a foreign object that is an object different from the product assigned to the display area. Then, when a foreign object is detected in a certain display area, the image analysis unit 11 determines whether the foreign object is another product. An "other product" is a product different from the product assigned to that display area.

In the case of "Processing Example 1" described in the first embodiment, after detecting a foreign object, the image analysis unit 11 performs a matching process using "image features of the area where the foreign object was detected" and "feature values of the appearance of each of the multiple products." If there is no match with any product, the image analysis unit 11 determines that the foreign object present in that area is not another product. On the other hand, if there is a match with any product, the image analysis unit 11 determines that the foreign object present in that area is the matched product (another product).

In the case of "Processing Example 2" described in the first embodiment, the image analysis unit 11 can perform a matching process for each extracted region using the "image feature amount of each region" and the "appearance feature amount of each of the multiple products." If there is no match with any product, the image analysis unit 11 determines that a foreign object is present in that region, and that the foreign object present in that region is not another product.

If the image matches any of the products, the image analysis unit 11 determines whether the matched product is a product assigned to the display area that includes the region. If the matched product is a product assigned to the display area that includes the region, the image analysis unit 11 determines that no foreign object is present in the region. On the other hand, if the matched product is not a product assigned to the display area that includes the region, the image analysis unit 11 determines that a foreign object is present in the region and that the foreign object is a matched product (another product).

When the detected foreign object is another product, the identification unit 13 identifies the display area assigned to the other product. For example, the identification unit 13 identifies the display area assigned to the other product using correspondence information such as that shown in FIG. 4.

When a foreign object is detected and the foreign object is another product, the output unit 12 outputs, in addition to an alert, information indicating the other product (e.g., product name) and information indicating the display area assigned to the other product (e.g., display area code). The output unit 12 may further output information indicating the display area in which the foreign object was detected (e.g., display area code). Figure 9 shows a schematic example of information output by the output unit 12 of this embodiment. The information is output using an output device such as a display or mailer.

Next, an example of the processing flow of the management device 10 of this embodiment will be described using the flowchart in FIG. 10.

The image analysis unit 11 continues to acquire and analyze image data of the display areas. If the analysis detects a foreign object in any of the display areas that is different from the product assigned to that display area (Yes in S20), the process proceeds to S21.

In S21, the image analysis unit 11 determines whether the detected foreign object is another product. If the detected foreign object is another product, the identification unit 13 identifies the display area allocated to the other product.

In S22, the output unit 12 outputs an alert. The output unit 12 also outputs information indicating the display area in which the foreign object was detected and information indicating the display areas assigned to other detected products (foreign objects). After this, if there is no instruction to end the process (No in S23), the process returns to S20 and continues.

The management device 10 of this embodiment described above can achieve the same effects as the first and second embodiments.

In addition, according to the management device 10 of this embodiment, when a foreign object is detected, it is possible to determine whether the foreign object is another product (a product that is not assigned to the display area in which the foreign object was detected). If the foreign object is another product, it is possible to identify the display area assigned to the other product and output information indicating that display area.

In such a case, the store clerk can quickly grasp the situation in which the foreign object is present, and can quickly grasp the location (display area) to return the foreign object to. The other products (foreign objects) can then be quickly moved to the appropriate location. As a result, the undesirable situation of "objects (foreign objects) different from the products assigned to each display area being placed in the display areas" can be quickly and appropriately resolved.

Fourth Embodiment
The management device 10 of this embodiment differs from the first to third embodiments in that, when a detected foreign object is another product (a product not assigned to the display area in which the foreign object was detected), the management device 10 calculates and outputs the urgency of an operation to remove the other product from the display area. This will be described in detail below.

An example of the hardware configuration of the management device 10 in this embodiment is similar to that in the first to third embodiments.

Figure 10 shows an example of a functional block diagram of the management device 10 of this embodiment. As shown in the figure, the management device 10 has an image analysis unit 11, an output unit 12, an identification unit 13, and an urgency calculation unit 14. The configurations of the image analysis unit 11 and the identification unit 13 are the same as those of the first to third embodiments.

When the foreign object detected by the image analysis unit 11 is another product, the urgency calculation unit 14 calculates the urgency of the task of recalling the other product based on the other product and the display area in which the other product was detected as a foreign object.

The urgency calculation unit 14 calculates a high urgency in a situation where the quality of other detected products will deteriorate if the product is left unattended for a long time. On the other hand, the urgency calculation unit 14 calculates a low urgency in a situation where the quality of other detected products will not be affected even if the product is left unattended for a long time.

In this embodiment, storage conditions are registered in advance in association with each of a plurality of products (see FIG. 12). The storage conditions are the optimal conditions for storing each product. For example, the storage conditions may be indicated as conditions such as "refrigerated storage," "frozen storage," "room temperature storage," or may be indicated as a temperature range, or may be other conditions.

In addition, in this embodiment, environmental conditions are registered in advance in association with each of the multiple display areas (see FIG. 13). The environmental conditions are the environmental conditions set for each display area. For example, they may be indicated as conditions such as "refrigerated," "frozen," or "room temperature," or may be indicated as a temperature range, or may be other conditions.

The urgency calculation unit 14 can calculate the urgency based on, for example, the storage conditions of each product shown in FIG. 12 and the environmental conditions of each display area shown in FIG. 13.

For example, the urgency calculation unit 14 may calculate a "low" urgency when the storage conditions of the other product (foreign object) match the environmental conditions of the display area in which the other product (foreign object) was detected. On the other hand, the urgency calculation unit 14 may calculate a "high" urgency when the storage conditions of the other product (foreign object) do not match the environmental conditions of the display area in which the other product (foreign object) was detected.

For example, if the value of the storage conditions of the other product (foreign object) (e.g., refrigerated) does not match the value of the environmental conditions of the display area in which the other product (foreign object) was detected (e.g., room temperature), a "high" urgency level may be calculated. If they match, a "low" urgency level may be calculated.

In addition, if the temperature range of the storage conditions of the other product (foreign object) (e.g., 10°C to 30°C) includes the temperature range of the environmental conditions of the display area in which the other product (foreign object) was detected (e.g., 20°C to 30°C), a "low" urgency may be calculated. If it does not, a "high" urgency may be calculated.

When a foreign object is detected and the foreign object is another product, the output unit 12 outputs the urgency calculated by the urgency calculation unit 14 in addition to an alert. The output unit 12 may further output information indicating the display area in which the foreign object was detected (e.g., display area code), information indicating the other product (e.g., product name), and information indicating the display area assigned to the other product (e.g., display area code). Figure 14 shows a schematic example of information output by the output unit 12 of this embodiment. The information is output using an output device such as a display or mailer.

Next, an example of the processing flow of the management device 10 of this embodiment will be described using the flowchart in FIG. 10.

The image analysis unit 11 continues to acquire and analyze image data of the display areas. If the analysis detects a foreign object in any of the display areas that is different from the product assigned to that display area (Yes in S20), the process proceeds to S21.

In S21, the image analysis unit 11 determines whether the detected foreign object is another product. If the detected foreign object is another product, the urgency calculation unit 14 calculates the urgency of the task of recalling the other product. The identification unit 13 may also identify the display area assigned to the other product.

In S22, the output unit 12 outputs an alert. The output unit 12 also outputs the above-mentioned urgency level. The output unit 12 may further output information indicating the display area in which the foreign object was detected, information indicating the display area allocated to other detected products (foreign objects), etc. After this, if there is no instruction to end the process (No in S23), the process returns to S20 and continues.

The management device 10 of this embodiment described above can achieve the same effects as the first to third embodiments.

Furthermore, according to the management device 10 of this embodiment, when a foreign object is detected, it can be determined whether the foreign object is another product (a product that is not assigned to the display area in which the foreign object was detected). If the foreign object is another product, the urgency of the task of collecting the other product can be calculated. Specifically, in a situation where the quality of the other detected product will deteriorate if the product is left unattended for a long period of time, a high urgency can be calculated. On the other hand, if the quality of the other detected product is not affected even if the product is left unattended for a long period of time, a low urgency can be calculated.

In such a case, the store clerk can quickly grasp the situation in which the foreign object is present, and can quickly grasp the urgency of the work to remove the foreign object. Based on the urgency, the store clerk can then decide whether or not to interrupt the work he is currently doing and perform the removal work. As a result, even if the removal work occurs while the store clerk is in the middle of other work, the store clerk can quickly determine the preferred order in which to perform those work and perform them.

Fifth embodiment
The management device 10 of this embodiment differs from the first to fourth embodiments in that, when a detected foreign object is another product (a product not assigned to the display area in which the foreign object was detected), the management device 10 determines whether or not to recommend discarding the other product and outputs the result. This will be described in detail below.

An example of the hardware configuration of the management device 10 in this embodiment is similar to that in the first to fourth embodiments.

Figure 15 shows an example of a functional block diagram of the management device 10 of this embodiment. As shown in the figure, the management device 10 has an image analysis unit 11, an output unit 12, an identification unit 13, and a discard determination unit 15. Although not shown, the management device 10 may further have an urgency calculation unit 14. The configurations of the image analysis unit 11, the identification unit 13, and the urgency calculation unit 14 are the same as those of the first to fourth embodiments.

If the detected foreign object is another product, the discard decision unit 15 decides whether to recommend discarding the other product.

For example, the discard determination unit 15 can determine whether or not to recommend discarding another product (foreign object) based on the time that has elapsed since the detection of the other product. In this case, the discard determination unit 15 may determine whether or not to recommend discarding the other product based on the other product and the display area in which the other product was detected as a foreign object.

For example, as shown in FIG. 16, for each combination of product storage conditions and display area environmental conditions, the allowable time (allowable leaving time) for leaving a product with each storage condition in the display area with each environmental condition may be registered in advance.

The discard determination unit 15 may determine the allowable time (allowable leaving time) for the other detected products to be left in the display area based on the storage conditions of the other products detected as foreign objects, the environmental conditions of the display area in which the foreign object was detected, and the corresponding information shown in FIG. 16. The discard determination unit 15 may then determine that it is recommended to discard the other products if the leaving time exceeds the allowable leaving time. On the other hand, the discard determination unit 15 may determine that it is not recommended to discard the other products if the leaving time does not exceed the allowable leaving time.

Additionally, the discard decision unit 15 may determine whether to recommend discarding the other product based on the accumulated time that the detected foreign object (other product) is in the blind spot of the store camera after detection (the accumulated time that the other product is not detected in the image after detection until collection). The discard decision unit 15 may determine to recommend discarding the other product if the accumulated time exceeds a predetermined value. On the other hand, the discard decision unit 15 may determine not to recommend discarding the other product if the accumulated time does not exceed the predetermined value.

If a foreign object is detected and the foreign object is another product, the output unit 12 outputs the result of the determination made by the discard determination unit 15.

For example, when another product (foreign object) is detected, the discard determination unit 15 may continue to monitor the elapsed time and the accumulated time in the blind spot, and whether or not these values satisfy the conditions for recommending the discard of the other product (foreign object) (e.g., the time left unattended exceeds the allowable time for leaving, the accumulated time in the blind spot exceeds a predetermined value, etc.). Then, when the conditions for recommending the discard of the other product (foreign object) are satisfied, the output unit 12 may output a determination result recommending the discard of the other product accordingly. FIG. 17 shows a schematic example of information output by the output unit 12 of this embodiment. The information is output using an output device such as a display or a mailer.

Another example of a method for outputting the determination result by the discard determination unit 15 will be described. The management device 10 may accumulate foreign object detection events as shown in FIG. 18. The information shown in FIG. 18 associates the detection number (serial number of the detection event) with values of various items such as the display area code, foreign object type, collection status, elapsed time, blind spot time, and discard determination result.

The display area code field records the code of the display area where the foreign object was detected. The foreign object type field records information indicating whether the foreign object is another product, and if it is another product, the product code. The collection status field records information indicating whether the collection work has been completed. The elapsed time field records the time elapsed until the collection work is completed. The blind spot time field records the accumulated time that the foreign object was in the blind spot of the store camera until the collection work was completed. The discard decision result field records the decision made by the discard decision unit 15.

For example, when a new foreign object is detected, the management device 10 registers the new detection number in association with the display area code and the value of the foreign object type (the analysis result by the image analysis unit 11). A value indicating that the object has not been collected is registered as the collection status. The elapsed time and blind spot time are left blank.

When the collection of the detected foreign object is complete, the store clerk operates the terminal device, specifies the detection number, and inputs that the collection has been completed. The information shown in FIG. 18 is updated accordingly. Specifically, the value in the collection status column becomes a value indicating that the object has been collected. The elapsed time and blind spot time columns record the time that has elapsed since the foreign object was detected until the input that the collection has been completed, and the accumulated time that the foreign object was in the blind spot of the store camera during that time. The result of the determination made by the discard determination unit 15 is entered in the discard determination result column.

When another product (foreign object) is detected, the discard decision unit 15 may continue to monitor the elapsed time and the accumulated time in the blind spot, and to monitor whether these values satisfy the conditions for recommending the discard of the other product (foreign object) (e.g., the time left unattended exceeds the allowable time left unattended, the accumulated time in the blind spot exceeds a predetermined value, etc.). Then, when the conditions for recommending the discard of the product (foreign object) are satisfied, the value in the discard decision result column shown in FIG. 18 may be updated accordingly.

The store clerk may operate the terminal device at any time, specify the detection number, and request the determination result of the discard determination unit 15 regarding other products (foreign objects) detected in the detection event. Then, the output unit 12 may output the determination result of the discard determination unit 15 in response to the request.

In addition, the output unit 12 may output list information such as that shown in FIG. 18 in response to a request from the user.

The output unit 12 may further output information similar to that of the first to fourth embodiments.

The management device 10 of this embodiment described above can achieve the same effects as the first to fourth embodiments.

Furthermore, according to the management device 10 of this embodiment, when a foreign object is detected, it can be determined whether the foreign object is another product (a product that is not assigned to the display area in which the foreign object was detected). If the foreign object is another product, it can be determined whether to recommend discarding the product. Specifically, products that have been left in an undesirable display area for a long time, products that have been in the blind spot of a store camera for a long time and whose safety is uncertain can be determined as products recommended for discarding.

In such a case, the store clerk can quickly and accurately determine whether to discard the other products (foreign objects) based on the information provided by the management device 10. As a result, it is possible to avoid the inconvenience of selling products that have been left in undesirable display areas for a long time and are likely to have deteriorated in quality, or products that have been in the blind spot of the store cameras for a long time and are therefore unsafe.

Below, examples of reference forms are given.
1. An image analysis means for analyzing image data of a display area and detecting a foreign object, which is an object different from the product allocated to the display area, from the image data;
an output means for outputting an alert when the foreign object is detected;
A management device having the above configuration.
2. In the management device according to claim 1,
The output means is a management device that further outputs information indicating the display area in which the foreign object was detected.
3. In the management device according to 1 or 2,
When the image analysis means detects the foreign object, it determines whether the foreign object is another product,
If the foreign object is the other product, the display area is further specified by specifying the display area assigned to the other product.
The output means is a management device that further outputs information indicating the display area allocated to the other products.
4. In the management device according to claim 3,
If the foreign object is the other product, an urgency calculation means is further provided for calculating an urgency of an operation to collect the other product based on the other product and the display area in which the other product is detected as the foreign object,
The output means is a management device that further outputs the urgency level.
5. In the management device according to claim 4,
The urgency calculation means is a management device that calculates the urgency based on storage conditions of each of a plurality of products and environmental conditions maintained in each of the plurality of display areas.
6. In the management device according to any one of 3 to 5,
If the foreign object is the other product, a discard determination means is further provided for determining whether or not to recommend discarding the other product,
The output means is a management device that further outputs a result of the determination made by the discard determination means.
7. In the management device according to claim 6,
The discard determination means is a management device that determines whether or not to recommend discarding the other products based on the elapsed time since the foreign object was detected.
8. In the management device according to 6 or 7,
The discard determination means is
A management device that determines whether or not to recommend discarding the other products based on the other products and the display area in which the other products are detected as the foreign object.
9. In the management device according to claim 8,
The discard determination means is a management device that determines whether or not to recommend discarding the other products based on the storage conditions of each of the multiple products and the environmental conditions maintained in each of the multiple display areas.
10. In the management device according to any one of 6 to 9,
The discard determination means is a management device that determines whether or not to recommend discarding the other products based on an accumulated time that the detected foreign object is in a blind spot of a camera.
11. In the management device according to any one of 1 to 10,
The image analysis means
Detecting an object from the image data;
A management device that determines whether the object is a foreign object based on the degree of similarity between the appearance of the object detected in the display area and the appearance of a product assigned to the display area.
12. In the management device according to any one of 1 to 10,
The image analysis means
Detecting an object from the image data;
determining which product the object corresponds to based on a degree of similarity between an appearance of the object detected in the display area and an appearance of each of a plurality of products;
A management device that determines whether the object is a foreign object based on at least one of the determination result and the products assigned to the display area.
13. The computer:
an image analysis step of analyzing image data of a display area and detecting a foreign object, which is an object different from the product allocated to the display area, from the image data;
an output step of outputting an alert when the foreign object is detected;
A management method for carrying out the above.
14. The computer
an image analysis means for analyzing image data obtained by photographing a display area and detecting a foreign object, which is an object different from the product allocated to the display area, from the image data;
an output means for outputting an alert when the foreign object is detected;
A program that functions as a

1A Processor 2A Memory 3A Input/Output I/F
Reference Signs List 4A Peripheral circuit 5A Bus 10 Management device 11 Image analysis unit 12 Output unit 13 Identification unit 14 Urgency calculation unit 15 Discard determination unit

Claims (9)

an image analysis means for analyzing image data obtained by photographing a display area and detecting from the image data other products than the products allocated to the display area;
A determination means for determining an urgency of an operation to collect the other products based on the other products and the display area;
an output means for outputting the urgency of the other product together with information indicating the other product ;
an acquisition means for acquiring information on storage conditions of the other products and environmental conditions including information on at least one of a temperature and a temperature range of the display area;
The determination means is a management device that determines the urgency based on storage conditions of the other products and the environmental conditions of the display area . 2. The management device according to claim 1 ,
The determination means determines the level of urgency to be low when the storage conditions of the other products include the environmental conditions of the display area, and determines the level of urgency to be high when the storage conditions of the other products do not include the environmental conditions of the display area. 3. The management device according to claim 2 ,
The storage conditions of the other products include a storage temperature range of the other products;
The determining means determines that the urgency is low when the storage temperature range of the other product includes the temperature of the display area or includes the entire temperature range of the display area;
The determination means is a management device that determines that the urgency is high when the storage temperature range of the other products does not include the temperature of the display area or does not include at least a portion of the temperature range of the display area. 4. The management device according to claim 1 ,
The image analysis means is a management device that determines whether or not to recommend disposal of the other products based on whether or not the storage conditions of the other products include the environmental conditions of the display area. 5. The management device according to claim 1 ,
The image analysis means refers to information regarding the appearance of products assigned to each of the multiple display areas that make up the display shelf, which is stored in a memory means, and determines whether the product included in the image data is a product assigned to the display area based on the appearance of the product included in the image data and the appearance of the product assigned to the photographed display area. 6. The management device according to claim 5 ,
The output means is a management device that outputs an alert when a product included in the image data is not a product assigned to the display area. 7. The management device according to claim 1 ,
The image analysis means is a management device that determines whether or not to recommend discarding the other products based on an accumulated time that the detected other products are in a blind spot of the camera. 8. The management device according to claim 1 ,
The image analysis means is a management device that determines whether or not to recommend discarding the other product based on the elapsed time since the other product was detected. The computer
an image analysis step of analyzing image data of a display area and detecting from the image data products other than the products allocated to the display area;
a determination step of determining an urgency of an operation to collect the other products based on the other products and the display area;
an output step of outputting the urgency of the other product together with information indicating the other product;
Run
In the determining step,
acquiring information about storage conditions of the other products and environmental conditions including information about at least one of a temperature and a temperature range of the display area;
A management method for determining the urgency based on storage conditions of the other products and the environmental conditions of the display area .

Images