JP6813808B2 - POS terminal device - Google Patents

Description

The present invention relates to a POS (Point Of Sale) system, and more particularly to a POS terminal device, a server, a POS system, a PLU identification method, and a program for recognizing an object by image recognition.

Recently, a POS system incorporating image recognition technology has been developed.
In the POS system using this image recognition technology, the imaged object (image) is compared with the contrast data in the database, and the degree of agreement and the likelihood are calculated. Next, the POS system finds one or more candidates corresponding to the PLU (Price Look Up) code based on the calculation results of the degree of agreement and the likelihood, and obtains the target PLU code or the target PLU code depicted in the image. Find PLU code candidates. After that, it is used for payment and inventory management like the existing POS system.

A POS system that incorporates such image recognition technology enables product management that does not rely on barcodes.

Examples of the related technique include Patent Document 1.

Patent Document 1 describes a POS system having a product type identification unit that recognizes an object of a target product from a captured image of the product. Further, in this POS system, after the product type identification process, an image recognition process for individual identification of the identified target product is executed. The individual identification result is associated with information prepared in advance for this individual (for example, place of origin, grade, producer, harvest date, etc.) as necessary, and is used at the time of accounting.

Japanese Unexamined Patent Publication No. 2013-210971

There are some problems in recognizing the product type by image recognition of the product image.

One of them is that it is difficult to distinguish the size of the product and the grade of the product from the image. In the method described in Patent Document 1, for this problem, a mark (characteristic symbol) indicating a grade attached to a product in advance is searched for by an image matching technique to prepare in advance for this individual. The information that was there is associated with the article and resolved. On the other hand, in this method, it is possible to point out the problem that the grade and the like cannot be determined without the mark.

From another point of view, there are many articles in which it is difficult to properly fit the entire product in the imaging range. The size and length of the product, the arrangement of the imaging means (imaging range), and the like limit the use of the POS system. As a result, humans select the product type with a keyboard, a barcode scanner, or the like for these products.

In addition, it is difficult to identify the product type and grade even when the products are small or bundled.

In view of the above problems, an object of the present invention is to provide a POS terminal device, a server, a POS system, a PLU identification method, and a program for accurately identifying and / or classifying an object depicted in a captured image.

In the POS terminal device according to the present invention, a size classification range (size), a code for specifying a PLU code, and a size item for classification (designated item) are linked according to an article name prepared in advance. An object recognition means for recognizing an object from an image including an object imaged by an imaging means, which is configured to be accessible to the attached size determination table, and a classification for each object prepared in advance in the size determination table. With reference to the size item, the captured image, and the recognition result of the object recognition means, the image area range of the target included in the image, the measured distance to the target, and the like. An identification means for identifying the size of the object by calculation based on a size item for classification of a recognition result by object recognition, and an article or an article candidate that needs to be classified, the object obtained from the object recognition means. and recognition result, the size of the object obtained from the identification means, with reference the size classification range of the hit, and to either of the prepared divided in advance the size determination table, is depicted in the image It is characterized by including a classification means for determining the classification of the articles that have been used and outputting the result to the PLU collation means.

In the server according to the present invention, the size classification range (size), the code for specifying the PLU code, and the size item for classification (designated item) are associated with each other in accordance with the article name prepared in advance. The object recognition means for recognizing the object from the image including the object captured by the imaging means, which is configured to be accessible to the size determination table, and the size for classification for each object prepared in advance in the size determination table. With reference to the item, the captured image, and the recognition result of the object recognition means, the image area range of the target included in the image, the measured distance to the target, and the object recognition. Recognition means for identifying the size of the object by calculation based on the size item for classification of the recognition result by the above, and recognition of the object obtained from the object recognition means for an article or an article candidate requiring classification. result, the size of the object obtained from the identification means, with reference to a size classification range of the hit, in any segment that has been prepared in advance size determination table, has been depicted in the image It is characterized by having a classification means for determining the classification of articles and outputting the result.

In the POS system according to the present invention, the size classification range (size), the code for specifying the PLU code, and the size item for classification (designated item) are associated with each other in accordance with the article name prepared in advance. From a POS terminal device having a size determination table, an image pickup means for capturing an image of an object, and a PLU collation means, and an image including an object imaged by the image pickup means, which is configured to be accessible to the size determination table. With reference to the object recognizing means for recognizing the object, the size item for classification for each object prepared in advance in the size determination table, the captured image, and the recognition result of the object recognizing means. The size of the target is identified by calculation based on the image area range of the target included in the image, the measured distance to the target, and the size item for classification of the recognition result by object recognition. an identification unit, the classification for the required article or articles candidates, the recognition result of the object obtained from the object recognition unit, and the size of the object obtained from the identification means, the hit size classification range And, in any of the categories prepared in advance in the size determination table, there is a classification means for determining the classification of the article depicted in the image and outputting the result to the PLU collation means. It is characterized by being provided with an information processing device.

The PLU identification method according to the present invention corresponds to the PLU data and the article name prepared in advance, and corresponds to the size classification range (size), the code for specifying the PLU code, and the size item for classification (designated item). The information processing system that can access the size determination table associated with) acquires the target image captured by the imaging unit, and recognizes the target from the target image by the object recognition means. While acquiring the result, the distance to the object captured by the imaging unit is identified, and the size item for classification for each object prepared in advance in the size determination table, the captured image, and the above. With reference to the recognition result of the object recognition means, it is based on the image area range of the target included in the image, the measured distance to the target, and the size item for classification of the recognition result by the object recognition. The size of the object is identified by the identification means , and for an article or an article candidate that needs to be classified, the recognition result of the object obtained from the object recognition means , the size of the object obtained from the identification means, and the size of the object. Referring to a size classification range of the hit, and confirm to any segment that has been prepared in advance size determination table, a classification of the articles which have been depicted in the image to one of PLU division, as a result It is characterized in that the target PLU code is specified from the PLU data based on the above.

In the program according to the present invention, the size classification range (size), the code for specifying the PLU code, and the classification size item (designated item) are associated with the article name prepared in advance. An object recognition means for recognizing an object from an image including an object captured by the imaging means and an object prepared in advance in the size determination table by using an information processing system configured to be accessible to the size determination table. The image area range of the object included in the image and the measurement up to the object were measured with reference to the size item for each division, the captured image, and the recognition result of the object recognition means. An identification means for identifying the size of the object by calculation based on a distance and a size item for classification of a recognition result by object recognition, and an article or an article candidate that needs to be classified are obtained from the object recognition means. and a recognition result of the object, and the size of the object obtained from the identification means, with reference to a size classification range of the hit, in any segment that has been prepared in advance size determination table, the image It is characterized in that it operates as a classification means for determining the classification of the articles described in the above and outputting the result to the PLU collation means.

According to the present invention, it is possible to provide a POS terminal device, a server, a POS system, a PLU identification method, and a program that accurately identify and / or classify an object depicted in a captured image.

It is a functional block diagram which showed the POS system which concerns on embodiment. It is a flowchart which showed the processing operation example which concerns on the classification of the object in the POS system which concerns on embodiment. It is explanatory drawing which showed the example of the table information for classification.

Embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a functional block diagram showing a POS system according to an embodiment. In this embodiment, the POS system arranged in the supermarket will be described in mind.

The POS system includes an imaging unit 10 and an object recognition unit 20, an identification unit 30 that identifies the size of the target, and a classification unit 40 that classifies the target based on the size of the target and the recognition result of the target. There is. The identification unit 30 and the classification unit 40 operate as a division derivation means for deriving the division.

It should be noted that known mechanisms (for example, various calculation units, various databases, communication units, drawers, receipt printers, etc.) and processing operations (for example, PLU reference processing, settlement processing, cooperation with drawers, receipt ejection processing, etc.) possessed by the POS system, etc. ) Will be simplified or omitted. The POS system according to the present embodiment also controls each unit by various programs in which one or a plurality of control units (CPUs: Central Processing Units) are deployed in RAM.

The imaging unit 10 images an identification target (article) within the imaging range. Further, if necessary, a compound eye camera, a multispectral camera, or the like may be used as the imaging unit 10.

The object recognition unit 20 recognizes an object to be identified (article) from the image imaged by the image pickup unit 10. As the object recognition method, an image matching method, a general object recognition method based on a feature amount, a two-dimensional code image identification method using a barcode or a QR code (registered trademark), or a composite method thereof can be used. The arithmetic resources and comparison data of the object recognition unit 20 do not necessarily have to be in the store system. Computational resources and comparison data may be appropriately installed in a housing, a store, an in-house network, a cloud, or the like, if necessary.

The identification unit 30 identifies the size of the target from the image based on the distance from the target.
The distance to the target may be identified by arranging a physical device near the imaging unit as a distance calculating means, or by image analysis from an image in which a product image is depicted as a distance calculating means.

For example, distance measurement using laser light or ultrasonic waves, distance calculation analysis based on parallax from a stereo visual image, and distance calculation analysis based on differences on continuous images of the same object at different times are used. Further, the distance to the target may be acquired by back calculation processing based on symbols, patterns, and colors of known dimensions having distinctiveness, or distance measurement using a TOF (Time of Flight) sensor.

The size of the target is derived based on the image area range and the distance of the target included in the image. As the size derivation method and items, an algorithm that automatically changes as appropriate according to the type and shape of the target may be adopted. This change may be made according to the object recognition result, or an item or algorithm suitable for image feature analysis may be selected at the initial stage of image analysis for determining the size.

The size derivation processing method is, for example, if the entire identification target is contained in the image, the contour line of the target is extracted from the image and rounded to the contour estimation line as necessary, and the diameter, radius, and volume (volume) are used. ), Vertical width, horizontal width, thickness, etc. If it is not possible to obtain all the values from the image of one surface at a time, the missing values may be obtained by imaging the other surface (another angle image) by issuing an announcement to rotate the target. At this time, in the target area identified by each of the plurality of images, common points and surfaces may be determined, and image recognition for obtaining a missing value may be appropriately executed.

For example, in the method of finding the volume of a box-shaped object, the surface area (calculated by the image range (x × y) and the distance) is obtained from one image in which only the surface is depicted, and it is different from another image to which rotation is applied. Find the surface area (x × z or y × z) of the surface. The volume may be estimated from the area and distance on the images of these two surfaces.

Further, for example, in the method of obtaining the volume of a spherical object, the cross-sectional area, diameter, circumference, flatness, vertical length, horizontal length, etc. may be obtained from one image, and the volume may be estimated. In addition, the volume may be estimated using a plurality of images.

Further, for example, a plurality of products put in a bag may be individually image-analyzed to obtain the quantity and size of the articles in the bag, and the total amount in the bag may be derived. In the case of an article such as "carrot", one of the number and the total volume may be estimated as a required value.

In addition, for a product packed in a pack such as "strawberry", one or more items for individually determining the size or size of a plurality of objects depicted in the image are obtained, and a set of "strawberry" is set to 1. You may ask for the size and grade of the pack as a single product. Note that the size identification may include an algorithm that identifies several individuals without performing the size identification on all the individuals and reflects the result on the pack or all the individuals in the bag.

Further, even if the entire object to be identified does not fit in the image, appropriate measures may be taken for each type and shape.

For example, in the case of a long object such as "leek" or "burdock", the entire image may be acquired as a continuous image over the imaging range to determine the size. Specifically, a plurality of images may be joined together to form a single image, and then the length may be obtained. If necessary, the lengths of a plurality of long objects depicted in the image may be obtained in parallel, and the total volume may be obtained together with the number of pieces.

Further, for example, in the case of a large object such as a "watermelon", it may occur that only a part of the image can be captured by setting the imaging range. For such an object, a part of the area captured in the image of the object captured by the imaging unit 10 is identified, the curvature of the contour of the part of the area is obtained, and another element (distance or one) is obtained. The size may be derived based on the length of the region, etc.). The curvature is obtained from, for example, the contrast ratio of an image to obtain the contour, and the transition thereof. Further, it may be obtained from the feature quantity, or may be obtained by any method. From the obtained curvature and the length of a part of the object depicted in the image, the estimated contour of the undepicted part can be derived. It is also possible to obtain a plurality of images by copying the whole as in the case of the above-mentioned long object.

The identification unit 30 may adopt a configuration in which many image analyzes as illustrated are simultaneously performed in parallel when the size of the target is identified from the image. On the other hand, there are many products that do not require size identification and products that do not require all the factors related to size when selling products. As one measure against this, the identification unit 30 may determine the necessity of obtaining the size and its items by referring to the identification result (candidate) output from the object recognition unit 20. Items to calculate for each article, previously classified for size items in a database accessible may be set as (specified item). For example, a designated item may be prepared in advance in the PLU file (PLU master data), or table information (classification size determination table) linked with the PLU file may be separately prepared.
For example, in the case of an article close to a sphere such as "tomato" or "orange", "L", "M", "S", etc. can be obtained by one element such as diameter, radius, and circumference, without necessarily finding the volume. Can be classified.

When the reference table is configured separately for the PLU file and the classification size determination table, the general names indicating "tomato" and "orange" and their designated items may be described in the classification size determination table. By limiting this designated item, it is possible to reduce the amount of calculation and improve the reading speed of the article.

In the following examples of the size classification of the articles, the designated items may be defined by the table information as in the above description. Further, an algorithm for automatically determining a specified size item from a rough object shape may be adopted.

For example, in the case of a spherical article such as "grapefruit", the size classification may be easily obtained only from the length of the long side and the length (diameter) of the short side.

Also, for example, even for an article whose grade changes depending on the size (weight) such as "egg", the weight may be estimated from the long side, the short side and the assumed specific gravity, but the length of the long side and the length of the long side can be easily estimated. The classification such as "L", "M", "S" may be obtained only from the length (diameter) of the short side.

Further, in the case of an article having various shapes such as "sweet potato", the volume may be obtained from various item elements. Further, the division may be obtained by determining the size by an original calculation formula instead of the volume.

Further, in the case of a rod-shaped article having a curved line such as a "banana", the length based on the curved line may be obtained.

On the other hand, in the case of an article having a size and other classification elements (mesh density in the case of melon) such as "melon", the identification unit 30 calculates and outputs each classification element together with the volume by image analysis. It may be configured. At the time of this image processing, solid identification processing may be performed from the "pattern" if necessary.

As the above-mentioned classification element, "hue", "glossiness", "number", "number", "with scratches" and the like can be used. As the "hue", one or more images obtained by additionally operating a multispectral camera may be used. This classification element can also be used as information for automatically calculating when to eat "fresh fish" and "fruit" and when to expire. Further, by outputting the number of articles as a classification element, a discount code for bulk selling can be automatically set. The division element can also be used as complementary information for the object recognition unit 20. For example, in the case of the item of "glossiness", it can be used to distinguish between "matte products" and "glossy products" of articles having the same shape that do not need to be determined in size.

The classification unit 40 divides the object into preset categories based on the object recognition result (candidate) obtained from the object recognition unit 20, the object size item, and the category element item. For example, the grade of fruit, the size of vegetables, the presence / absence of a bulk sale flag, the presence / absence of scratches, etc. are classified for each target. Next, the classification unit 40 outputs the target object recognition result and its classification to the PLU collation unit (not shown). At this time, when there are a plurality of candidates in the object recognition result of the target, the classification unit 40 may output the object recognition result of the target and its classification for each candidate. The classification unit 40 may be configured to perform the PLU collation process.

The PLU collation unit reads out the price corresponding to the goods and the classification in the PLU data for each received goods, and adds the goods to the unsettled list. When there are a plurality of candidates or when a measure such as a discount is taken, the POS system may appropriately receive selection from the operator and determine the unsettled list. The calculation resources and comparison data of the PLU collation unit do not necessarily have to be in the store system. Computational resources and comparison data may be appropriately installed in a housing, a store, an in-house network, a cloud, or the like, if necessary.

Subsequent accounting processing may be appropriately configured according to the style of the store.

The POS system may be configured to execute the pricing process on the terminal according to the target classification result. For example, in an article corresponding to the PLU code, pricing formulas such as internal quantity, unit price per volume, grade item and unit price per volume, division element item and unit price per volume are registered in advance, and based on the formula. The terminal may calculate the price of the imaged object.

According to this mechanism, for example, when "vegetable bagging" and the price per volume of each vegetable are registered in the PLU code, the customer puts "carrot", "onion" and "potato" in the same bag. Even if you put it in the box and take images at the same time, you can set the appropriate price on the terminal.

With the configuration described above, the POS system can accurately identify and / or classify the objects depicted in the captured image to create an unsettled list.

FIG. 2 is a flowchart showing an example of processing operation related to the classification of the target in the POS system according to the embodiment.

The POS system first captures an article to be purchased in the imaging range of the imaging unit 20, and outputs the captured image to the object recognition unit 20 and the identification unit 30 (step S101). The image of the article output to the object recognition unit 20 and the identification unit 30 may be acquired from different image pickup devices.

Next, the POS system (object recognition unit 20) executes object recognition of the object depicted in the captured image, and outputs the result to the classification unit 40 (step S102).

At the same time, the POS system (identification unit 30) executes a size identification process for obtaining the size of the object depicted in the captured image, and outputs the result to the classification unit 40 (step S103). At this time, the identification unit 30 may extract the classification element from the image and output it to the classification unit 40. Further, the object recognition result may be received from the object recognition unit 20, and whether or not the size is identified and the item type requiring analysis may be identified.

Next, the POS system (classification unit 40) determines the articles or article candidates that need to be classified from the articles depicted in the captured image and the classification based on the size of the object, the recognition result of the object, and the classification element. Then, the result is output to the PLU collation unit (step S104).

After that, the POS system acquires the price of the product from the PLU file and shifts to the process of adding to the unsettled list or the process of selecting candidates.

By operating the POS system in this way, it is possible to accurately identify and / or classify the objects depicted in the captured image and create an unsettled list.
For example, in the existing POS system that executes object recognition, the machine could not accurately classify "tomatoes" and "mini tomatoes", "strawberry pack" price classification, "watermelon" and "green onions". It is possible to classify the size such as ".
As a result, the labor of workers can be reduced and the cash register work can be speeded up.

In addition, the degree of freedom in pricing the goods is increased.
In addition, self-checkout can reduce complicated operations for customers.
In addition, when pricing individual products at stores, etc., pricing is possible with this mechanism, which simplifies pricing work using a handler beller, barcode printer, or the like.

Next, a configuration example of the classification size determination table will be described. FIG. 3 is an explanatory diagram showing an example of table information for classification.

In the PLU master shown in FIG. 3A, the PLU code (JAN (Japanese Article Number) code in the figure) is associated with a list of division elements to be additionally calculated together with the product name and price.
In the classification size determination table shown in FIG. 3B, a flag indicating whether or not to obtain the size, a desired item, and a classification element to be additionally calculated are configured in a list corresponding to the article name. In addition, the PLU code can be specified using the classification size determination table.

Each part of the POS system may be realized by using a combination of hardware and software. In the form of combining hardware and software, a POS system program (PLU classification identification program) is developed in RAM, and hardware such as a control unit (CPU) is operated based on the program. As a result, each part is operated as various means. Further, this program may be temporarily recorded on a recording medium and distributed. The program recorded on the recording medium is read into the memory via wired, wireless, or the recording medium itself, and operates the control unit or the like. Examples of recording media include optical disks, magnetic disks, semiconductor memory devices, hard disks, and the like.

Although the present invention has been described with reference to embodiments, various configurations can be taken when embodying the present invention. For example, a POS terminal device having an imaging means, an object recognition means, an identification means, and a classification means all in one housing and one control unit executing up to a PLU identification process can be mentioned. Another aspect is a POS terminal device that has one control unit for imaging means, object recognition means, identification means, and classification means dispersed in a plurality of housings. Further, even in a mode in which control units are provided on the register side and the scanner side, respectively, as in the existing POS terminal device, the imaging means, the object recognition means, the identification means, and the classification means can be appropriately separated in the POS terminal device. Just do it.

It is also conceivable to consider a POS system in which an image is acquired by a large number of POS terminal devices, an object recognition or identification process is executed by a server in the store, and the result is returned to each POS terminal device.
To explain the features of the POS system of the above embodiment in another expression, an information processing device having one or a plurality of POS terminal devices having imaging means, an object recognition means, an identification means, and a classification means. Will be prepared.

Further, to explain the above embodiment in another expression, an information processing device that operates as a POS server is provided with a control unit as an object recognition means, an identification means, and a classification means based on a POS program developed in a RAM. It can be realized by operating it.

Although the embodiments and examples have been illustrated and described above, changes such as separation and merging of block configurations and replacement of procedures are free as long as the gist of the present invention and the functions described are satisfied, and the above description describes the present invention. It is not limited.

As described above, the POS system to which the present invention is applied can accurately identify and / or classify the object depicted in the captured image.

That is, according to the present invention, it is possible to provide a POS terminal device, a server, a POS system, a PLU identification method, and a program that accurately identify and / or classify an object depicted in a captured image.

In addition, some or all of the above embodiments may also be described as follows. The following notes do not limit the present invention in any way.
[Appendix 1]
An imaging means for capturing an image of an object,
An object recognition means for recognizing an object and
An identification means for identifying the size of the object and
A classification means for classifying the object based on the size of the object and the recognition result of the object, and
A POS terminal device characterized by having.

[Appendix 2]
It has a distance calculation means that identifies the distance to the target,
The identification means
The POS terminal device according to the above description, wherein the size of the target is derived based on the screen area range of the target included in the captured image of the target and the distance obtained by the distance calculating means. ..

[Appendix 3]
The imaging means acquires a plurality of images having parallax and obtains a plurality of images.
The identification means
The distance from the difference between the images of each object having parallax to the object is identified and processed.
The POS terminal device according to the above description, wherein the size of the target is derived based on the distance obtained from the difference between the image region range of the target included in the captured image of the target and the image. ..

[Appendix 4]
The identification means
From the difference between the images of the same object acquired at different times by the imaging means, the distance of the image to recognize the object to the object is identified.
The POS terminal device according to the above description, wherein the size of the target is derived based on a distance obtained from a difference between an image and a region range of the target included in the captured image of the target.

[Appendix 5]
The identification means obtains one or a plurality of items for individually determining the size or size of a plurality of objects depicted in an image, and derives the size of the plurality of objects as one grouped object. The POS terminal device described in the above appendix as a feature.

[Appendix 6]
The POS terminal device according to the above description, wherein the identification means joins a plurality of consecutive images of the target captured by the imaging means to estimate the size of the target.

[Appendix 7]
The identification means identifies a part of the target image captured by the imaging means and estimates the size of the target from the curvature of the contour of the part of the target. The POS terminal device described in the above appendix as a feature.

[Appendix 8]
The identification means records table information in which items used as a standard for determining the size of an identification target prepared in advance are recorded in correspondence with an object name output as an object recognition result, and an object recognition result of the object recognition means. The POS terminal device according to the above description, wherein an item used as a reference for determining the size of the object is specified with reference to the object, and the size of the object is identified based on the specified item.

[Appendix 9]
The identification means refers to the table information prepared in advance corresponding to the object name output as the object recognition result and the object recognition result of the object recognition means, and the object recognition result of the object recognition means. The POS terminal device according to the above description, wherein a target classification element item is obtained by image analysis, and the result is output to the classification means.

[Appendix 10]
The classification means is characterized in that the object-recognized object is classified into any of the categories corresponding to the PLU code based on at least the value of each item reflecting the size of the object. The POS terminal device described in the appendix.

[Appendix 11]
The POS terminal device according to the above description, wherein the POS terminal device enables execution of a pricing process according to a classification result based on the size of the target.

[Appendix 12]
An object recognition means for recognizing an object and
An identification means for identifying the size of the object and
A classification means for classifying the target based on the size of the target and the recognition result of the target, and
A server characterized by having.

[Appendix 13]
The identification means
The server according to the above appendix, wherein the size of the target is derived based on the screen area range of the target included in the captured image of the target and the distance from the notified imaging position.

[Appendix 14]
The identification means
A plurality of images having parallax are acquired, and the distance from the difference between the images of each object having parallax to the object is identified.
The server according to the above appendix, wherein the size of the target is derived based on a distance obtained from a difference between an image region range of the target included in the captured image of the target and the image.

[Appendix 15]
The identification means
From the difference between the images of the same object acquired at different times, the distance of the image to recognize the object to the object is identified.
The server according to the above appendix, wherein the size of the target is derived based on a distance obtained from a difference between an image and a region range of the target included in the captured image of the target.

[Appendix 16]
The identification means obtains one or a plurality of items for individually determining the size or size of a plurality of objects depicted in an image, and derives the size of the plurality of objects as one grouped object. The server described in the above appendix as a feature.

[Appendix 17]
The server according to the above appendix, wherein the identification means joins a plurality of consecutive images of the target and estimates the size of the target.

[Appendix 18]
The above-mentioned appendix, wherein the identification means identifies a part of the target area in the image of the target and estimates the size of the target from the curvature of the contour of the part of the target area. server.

[Appendix 19]
The identification means records table information in which items used as a standard for determining the size of an identification target prepared in advance are recorded in correspondence with an object name output as an object recognition result, and an object recognition result of the object recognition means. The server according to the above appendix, which is characterized in that an item used as a criterion for determining the size of the object is specified with reference to the object, and the size of the object is identified based on the specified item.

[Appendix 20]
The identification means refers to the table information prepared in advance corresponding to the object name output as the object recognition result and the object recognition result of the object recognition means, and the object recognition result of the object recognition means. The server according to the above appendix, characterized in that a target classification element item is obtained by image analysis and the result is output to the classification means.

[Appendix 21]
The classification means is characterized in that the object-recognized object is classified into any of the categories corresponding to the PLU code based on at least the value of each item reflecting the size of the object. The server described in the appendix.

[Appendix 22]
The server according to the above appendix, wherein a price is specified according to a classification result based on the size of the target, and the price is output as data for pricing processing.

[Appendix 23]
A POS terminal device having an imaging means for capturing an image of an object,
An object recognition means for recognizing an object and
An identification means for identifying the size of the object and
A classification means for classifying the target based on the size of the target and the recognition result of the target, and
Information processing device with
POS system equipped with.

[Appendix 24]
An imaging means for capturing an image of an object,
An identification means for identifying the size of the object and
With a POS terminal device that has
An object recognition means for recognizing an object and
A classification means for classifying the target based on the size of the target and the recognition result of the target, and
Information processing device with
POS system equipped with.

[Appendix 25]
An information processing device having an object recognition means for recognizing an object,
An imaging means for capturing an image of the target and
An identification means for identifying the size of the object and
A classification means for classifying the object based on the size of the object and the recognition result of the object, and
With a POS terminal device that has
POS system equipped with.

[Appendix 26]
An object recognition means that recognizes an object and
An identification means for identifying the size of the object and
Information processing device with
An imaging means for capturing an image of the target and
A classification means for classifying the target based on the size of the target and the recognition result of the target, and
With a POS terminal device that has
POS system equipped with.

[Appendix 27]
The information processing device is a POS system which is a server according to any one of the above appendices 12 to 22.

[Appendix 28]
The information processing system that can access the PLU data is
The target image captured by the imaging unit and the recognition result obtained by subject recognition processing of the target image are acquired.
The size of the object is identified and processed,
Based on the size of the target and the recognition result of the target, the target is classified into one of the PLU categories, and the target is classified into one of the PLU categories.
A PLU identification method, characterized in that the PLU code of the target is specified from the PLU data based on the recognition result and the classification result of the target.

[Appendix 29]
The information processing system is used as a process for identifying the size of the target.
The feature is that the size of the target is derived based on the screen area range of the target included in the captured image of the target and the distance obtained by the distance calculation unit that identifies the distance to the target. The PLU identification method described in the above appendix.

[Appendix 30]
The information processing system is used as a process for identifying the size of the target.
Using a plurality of images having parallax acquired by the imaging unit, the distance from the difference between the images of each object having parallax to the object is identified.
The PLU identification method according to the above appendix, wherein the size of the target is derived based on the distance obtained from the difference between the image region range of the target included in the captured image of the target and the image. ..

[Appendix 31]
The information processing system is used as a process for identifying the size of the target.
From the difference between the images of the same object acquired at different times by the imaging unit, the distance of the image to recognize the object to the object is identified.
The PLU identification method according to the above description, wherein the size of the target is derived based on a distance obtained from a difference between an image and a region range of the target included in the captured image of the target.

[Appendix 32]
The information processing system is used as a process for identifying the size of the target.
The above-mentioned appendix, which comprises obtaining one or a plurality of items for individually determining the size or size of a plurality of objects depicted in an image, and deriving the size of the plurality of objects as one grouped object. The PLU identification method described.

[Appendix 33]
The information processing system is used as a process for identifying the size of the target.
The PLU identification method according to the above description, wherein a plurality of consecutive images of the target captured by the imaging unit are joined together to estimate the size of the target.

[Appendix 34]
The information processing system is used as a process for identifying the size of the target.
The above-mentioned appendix, characterized in that a part of the target area captured by the image pickup unit is identified and the size of the target is estimated from the curvature of the contour of the part of the target area. The PLU identification method described.

[Appendix 35]
The information processing system is used as a process for identifying the size of the target.
The size of the object is determined by referring to the table information recorded in advance corresponding to the object name output as the object recognition result and the item used as the standard for determining the size of the identification object, and the object recognition result. The PLU identification method according to the above description, wherein an item used for a specified standard is specified, and the size of the object is identified based on the specified item.

[Appendix 36]
The information processing system refers to the table information prepared in advance corresponding to the object name output as the object recognition result for the classification element item of the identification target and the object recognition result, and refers to the classification element of the target. The PLU identification method according to the above appendix, wherein items are obtained by image analysis and the results are classified.

[Appendix 37]
The information processing system is characterized in that the object-recognized object is classified into any of the categories corresponding to the PLU code, based on at least the value of each item reflecting the size of the object. The PLU identification method described in the above appendix.

[Appendix 38]
The PLU identification method according to the above description, wherein the information processing system enables execution of a pricing process according to a classification result based on the size of the target.

[Appendix 39]
Information system,
An image recognition means that recognizes an object from an image of an object captured by the image pickup unit,
An identification means for identifying the size of the object and
A classification means for classifying the target based on the size of the target and the recognition result of the target.
A program characterized by operating as.

[Appendix 40]
Information processing device,
An imaging means for capturing an image of an object,
An identification means for identifying the size of the object,
A program characterized by operating as.

[Appendix 41]
Information processing device,
An object recognition means that recognizes an object and
A classification means for classifying the object based on the size of the object and the recognition result of the object.
A program characterized by operating as.

[Appendix 42]
Information processing device,
An imaging means for capturing an image of an object,
An identification means for identifying the size of the object and
A classification means for classifying the object based on the size of the object and the recognition result of the object.
A program characterized by operating as.

[Appendix 43]
Information processing device,
An object recognition means that recognizes an object and
An identification means for identifying the size of the object,
A program characterized by operating as.

[Appendix 44]
Information processing device,
An imaging means for capturing an image of an object,
A classification means for classifying the object based on the size of the object and the recognition result of the object.
A program characterized by operating as.

[Appendix 45]
A program characterized by operating as the POS terminal device according to any one of the above-mentioned appendices 1 to 11.

[Appendix 46]
A program characterized by operating as the server according to any one of the above appendices 12 to 22.

The present invention can be applied to POS terminal devices arranged in supermarkets, convenience stores, department stores, and the like. It can also be applied to tablet POS systems and smartphone POS systems.

10 Imaging unit (imaging means)
20 Object recognition unit (object recognition means)
30 Identification unit (identification means, image analysis means)
40 Classification section (classification means)

Claims (8)

It is possible to access the size judgment table prepared in advance, in which the size classification range (size), the code for specifying the PLU code, and the size item for classification (designated item) are associated with each other. Consists of
An object recognition means that recognizes an object from an image including an object captured by the imaging means,
The image of the object included in the image with reference to the size item for classification for each object prepared in advance in the size determination table, the captured image, and the recognition result of the object recognition means. An identification means for identifying the size of the target by calculation based on the area range, the measured distance to the target, and the size item for classification of the recognition result by object recognition.
Classification for the required article or articles candidate reference and recognition result of the object obtained from the object recognition unit, and the size of the object obtained from the identification means, the size classification range of the hit, the , A classification means that determines the classification of the article depicted in the image in any of the classifications prepared in the size determination table in advance and outputs the result to the PLU collation means.
A POS terminal device comprising. The identification means individually obtains the value of the classification size item for the plurality of objects depicted in the image, and the value of the classification size item of each of the obtained multiple objects and the measured distance to each object. The POS terminal device according to claim 1, wherein the POS terminal device is included in a parameter used when deriving and processing the size of the plurality of objects as one collective object. The identification means joins a plurality of consecutive images of the object captured by the imaging means to form a single image, and measures the image region range of the target and the target as depicted in the monoimaged image. The POS terminal device according to claim 1 or 2, wherein the size of the object is estimated by including the distance as a parameter. The identification means identifies an image region range and a part of the image region of the target image captured by the imaging means, and the curvature of the contour of the target image region range and the partial region. The POS terminal device according to any one of claims 1 to 3, wherein the size of the object is estimated by including the object and the measured distance to the object as parameters. It is possible to access the size judgment table prepared in advance, in which the size classification range (size), the code for specifying the PLU code, and the size item for classification (designated item) are associated with each other. Consists of
An object recognition means for recognizing an object from an image including an object captured by the image pickup means,
The image of the object included in the image with reference to the size item for classification for each object prepared in advance in the size determination table, the captured image, and the recognition result of the object recognition means. An identification means for identifying the size of the target by calculation based on the area range, the measured distance to the target, and the size item for classification of the recognition result by object recognition.
Classification for the required article or articles candidate reference and recognition result of the object obtained from the object recognition unit, and the size of the object obtained from the identification means, the size classification range of the hit, the , A classification means for determining the classification of the article depicted in the image in any of the classifications prepared in the size determination table in advance and outputting the result.
A server characterized by having. A size determination table prepared in advance, in which a size classification range (size), a code for specifying a PLU code, and a classification size item (designated item) are associated with each other,
A POS terminal device having an imaging means for capturing an image of an object and a PLU matching means,
The size determination table is configured to be accessible and
An object recognition means for recognizing an object from an image including an object captured by the image pickup means,
The image of the object included in the image with reference to the size item for classification for each object prepared in advance in the size determination table, the captured image, and the recognition result of the object recognition means. An identification means for identifying the size of the target by calculation based on the area range, the measured distance to the target, and the size item for classification of the recognition result by object recognition.
Classification for the required article or articles candidate reference and recognition result of the object obtained from the object recognition unit, and the size of the object obtained from the identification means, the size classification range of the hit, the , A classification means that determines the classification of the article depicted in the image in any of the classifications prepared in advance in the size determination table, and outputs the result to the PLU collation means.
Information processing device with
A POS system characterized by being equipped with. Size determination in which the PLU data prepared in advance, the size classification range (size) corresponding to the article name, the code for specifying the PLU code, and the size item for classification (designated item) are linked. Information processing systems that can access the table
Acquire the target image captured by the imaging unit and
The recognition result obtained by recognizing the object by the object recognition means from the object image is acquired, the distance to the object imaged by the imaging unit is identified, and the object is prepared in advance in the size determination table. With reference to the classification size item for each object, the captured image, and the recognition result of the object recognition means, the image area range of the object included in the image and the measurement up to the object. The size of the object is identified by the identification means based on the distance obtained and the size item for classification of the recognition result by object recognition.
Classification for the required article or articles candidate reference and recognition result of the object obtained from the object recognition unit, and the size of the object obtained from the identification means, the size classification range of the hit, the , The category of the article depicted in the image is determined to be one of the PLU categories in any of the categories prepared in advance in the size determination table.
A PLU identification method, characterized in that the target PLU code is specified from the PLU data based on the result. It is possible to access the size judgment table prepared in advance, in which the size classification range (size), the code for specifying the PLU code, and the size item for classification (designated item) are associated with each other. Information processing system configured in
An object recognition means for recognizing an object from an image including an object captured by the image pickup means,
The image of the object included in the image with reference to the size item for classification for each object prepared in advance in the size determination table, the captured image, and the recognition result of the object recognition means. An identification means for identifying the size of the target by calculation based on the area range, the measured distance to the target, and the size item for classification of the recognition result by object recognition.
Classification for the required article or articles candidate reference and recognition result of the object obtained from the object recognition unit, and the size of the object obtained from the identification means, the size classification range of the hit, the , A classification means that determines the classification of the article depicted in the image in any of the classifications prepared in advance in the size determination table, and outputs the result to the PLU collation means.
A program characterized by operating as.

Images