JP2021044019A - system - Google Patents

Abstract

To properly determine a kind and a price of a commodity by image recognition of an image of the commodity by a POS.SOLUTION: An information processing system includes: object recognition means for recognizing as an object an article from an image captured by imaging means; identification means for identifying a size of the article on the basis of the image; classification means for referring to an information table in which a size of the article recognized as an object is associated with a section of the article to determine the section of the article recognized as an object on the basis of a size of the article identified by the identification means; and collation means for reading a price associated with the section of the article in commodity data relating to the article determined by the classification means.SELECTED DRAWING: Figure 1

Description

The present invention relates to a POS (Point Of Sale) system, and more particularly to a system for recognizing an object by image recognition and a method for identifying a product.

Recently, a POS system incorporating image recognition technology has been developed.
In the POS system using this image recognition technology, the degree of agreement and the likelihood are calculated by comparing the captured object (image) with the contrast data in the database. Next, the POS system finds one or more candidates corresponding to the PLU (Price Look Up) code based on the calculation results of the degree of agreement and the likelihood, and obtains the target PLU code or the target PLU code depicted in the image. Find PLU code candidates. After that, it is used for payment and inventory management like the existing POS system.

A POS system incorporating such image recognition technology enables product management that does not rely on barcodes.

Examples of the related technique include Patent Document 1.

Patent Document 1 describes a POS system having a product type identification unit that recognizes an object of a target product from a captured image of the product. Further, in this POS system, after the product type identification process, an image recognition process for individual identification of the identified target product is executed. The individual identification result is associated with information prepared in advance for this individual (for example, place of origin, grade, producer, harvest date, etc.) as necessary, and is used at the time of accounting.

Japanese Unexamined Patent Publication No. 2013-210971

There are some problems in recognizing a product image and recognizing a product type.

One of them is that it is difficult to distinguish the size of the product and the grade of the product from the image. In the method described in Patent Document 1, for this problem, a mark (characteristic symbol) indicating a grade attached to a product in advance is searched for by an image matching technique, so that the product is prepared in advance for this individual. The information that was there is associated with the article and resolved. On the other hand, in this method, it is possible to point out the problem that the grade and the like cannot be determined without the mark.

From another point of view, there are many articles in which it is difficult to properly fit the entire product in the imaging range. The size and length of the product, the arrangement of the imaging means (imaging range), and the like limit the use of the POS system. As a result, humans select product types using keyboards, barcode scanners, and the like for these products.

In addition, it is difficult to identify the product type and grade even when the products are small or bundled.

In view of the above problems, an object of the present invention is to provide a system for accurately identifying and / or classifying an object depicted in a captured image, and a method for identifying a product.

The system according to the present invention includes an object recognizing means that recognizes an article from an image captured by an imaging means, an identification means that identifies the size of the article based on the image, and a size of the article that is recognized as an object. With reference to the information table associated with the classification of the article, the classification means for determining the classification of the article recognized as an object based on the size of the article identified by the identification means, and the classification means for determining the classification of the article. It is characterized by including a collating means for reading out the price corresponding to the classification of the article in the product data with respect to the article.

The product identification method according to the present invention is obtained by an information processing system having access to product data, acquiring an image of an article captured by an imaging means, and recognizing the article from the image by an object recognition means. In addition to acquiring the recognition result, the size of the article is identified by the identification means based on the image, and the identification is referred to by referring to the information table in which the size of the article recognized as an object and the classification of the article are associated with each other. Based on the size of the article identified by the means, the category of the article recognized as an object is determined to be one of the product categories, and based on the result, the category of the article in the product data with respect to the determined article. It is characterized by reading the price corresponding to.

According to the present invention, it is possible to provide a system for accurately identifying and / or classifying an object depicted in a captured image, and a method for identifying a product.

It is a functional block diagram which showed the POS system which concerns on embodiment. It is a flowchart which showed the processing operation example which concerns on the classification of the object in the POS system which concerns on embodiment. It is explanatory drawing which showed the example of the table information for classification.

Embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a functional block diagram showing a POS system according to an embodiment. In this embodiment, the POS system arranged in the supermarket will be described in mind.

The POS system includes an imaging unit 10 and an object recognition unit 20, an identification unit 30 for identifying the size of the target, and a classification unit 40 for classifying the target based on the size of the target and the recognition result of the target. There is. The identification unit 30 and the classification unit 40 operate as a division derivation means for deriving the division.

It should be noted that known mechanisms (for example, various arithmetic units, various databases, communication units, drawers, receipt printers, etc.) and processing operations (for example, PLU reference processing, settlement processing, cooperation with drawers, receipt ejection processing, etc.) possessed by the POS system, etc. ) Will be simplified or omitted. The POS system according to the present embodiment also controls each unit by various programs in which one or a plurality of control units (CPUs: Central Processing Units) are deployed in RAM.

The imaging unit 10 images an identification target (article) within the imaging range. Further, if necessary, a compound eye camera, a multispectral camera, or the like may be used as the imaging unit 10.

The object recognition unit 20 recognizes an object to be identified (article) from the image imaged by the image pickup unit 10. As the object recognition method, an image matching method, a general object recognition method based on a feature amount, a two-dimensional code image identification method using a barcode or a QR code (registered trademark), or a composite method thereof can be used. The arithmetic resources and comparison data of the object recognition unit 20 do not necessarily have to be in the store system. Computational resources and comparison data may be appropriately installed in a housing, a store, an in-house network, a cloud, or the like, if necessary.

The identification unit 30 identifies the size of the target from the image based on the distance from the target.
The distance to the target may be identified by arranging a physical device in the vicinity of the imaging unit as the distance calculating means, or by image analysis from the image on which the product image is drawn as the distance calculating means.

For example, distance measurement using laser light or ultrasonic waves, distance calculation analysis based on parallax from a stereo visual image, and distance calculation analysis based on differences on continuous images of the same object at different times are used. Further, the distance to the target may be acquired by back calculation processing based on symbols, patterns, and colors of known dimensions having distinctiveness, or distance measurement using a TOF (Time of Flight) sensor.

The size of the target is derived based on the image area range and the distance of the target included in the image. As the size derivation method and items, an algorithm that automatically changes as appropriate according to the type and shape of the target may be adopted. This change may be made according to the object recognition result, or an item or algorithm suitable for image feature analysis may be selected at the initial stage of image analysis for determining the size.

The size derivation processing method is, for example, if the entire identification target is contained in the image, the contour line of the target is extracted from the image and rounded to the contour estimation line as necessary, and the diameter, radius, and volume (volume) are used. ), Length, width, thickness, etc. If it is not possible to obtain all the values from the image of one surface at a time, the missing values may be obtained by imaging the other surface (another angle image) by issuing an announcement to rotate the target. At this time, in the target area identified by each of the plurality of images, common points and surfaces may be determined, and image recognition for obtaining a missing value may be appropriately executed.

For example, in the method of finding the volume of a box-shaped object, the surface area (calculated by the image range (x × y) and the distance) is obtained from one image in which only the surface is depicted, and it is different from another image to which rotation is applied. Find the surface area (x × z or y × z) of the surface. The volume may be estimated from the area and distance on the images of these two surfaces.

Further, for example, in the method of obtaining the volume of a spherical object, the cross-sectional area, diameter, circumference, flatness, vertical length, horizontal length, etc. may be obtained from one image, and the volume may be estimated. In addition, the volume may be estimated using a plurality of images.

Further, for example, a plurality of products put in a bag may be individually image-analyzed to obtain the quantity and size of the articles in the bag, and the total amount in the bag may be derived. In the case of an article such as "carrot", one of the number and the total volume may be estimated as a required value.

In addition, for products packed in packs such as "strawberry", one or more items that individually determine the size or size of multiple objects depicted in the image are obtained, and a set of "strawberry" is set to 1. You may ask for the size and grade of the pack as a single product. It should be noted that the size identification may include an algorithm that identifies several individuals without performing the size identification on all the individuals and reflects the result on the pack or all the individuals in the bag.

Further, even if the entire object to be identified does not fit in the image, appropriate measures may be taken for each type and shape.

For example, in the case of a long object such as "leek" or "burdock", the entire image may be acquired as a continuous image over the imaging range to determine the size. Specifically, the length may be obtained after connecting a plurality of images into one image. If necessary, the lengths of a plurality of long objects depicted in the image may be obtained in parallel, and the total volume may be obtained together with the number of objects.

Further, for example, in the case of a large object such as a "watermelon", it may occur that only a part of the image can be captured by setting the imaging range. For such an object, a part of the area captured in the image of the object captured by the imaging unit 10 is identified, the curvature of the contour of the part of the area is obtained, and another element (distance or one) is obtained. The size may be derived based on the length of the region, etc.). The curvature is obtained from, for example, the contrast ratio of an image to obtain the contour, and the transition thereof. Further, it may be obtained from the feature quantity, or may be obtained by any method. From the obtained curvature and the length of a part of the object depicted in the image, the estimated contour of the undepicted part can be derived. It is also possible to obtain a plurality of images by copying the whole as in the case of the above-mentioned long object.

The identification unit 30 may adopt a configuration in which many image analyzes as illustrated are simultaneously performed in parallel when the size of the target is identified from the image. On the other hand, there are many products for which it is not necessary to identify the size, and there are many products that do not require all the elements related to the size when selling the product. As one measure against this, the identification unit 30 may determine the necessity of obtaining the size and its items by referring to the identification result (candidate) output from the object recognition unit 20. The items to be calculated for each article may be set in advance as classification size items (designated items) in the accessible database. For example, a designated item may be prepared in advance in the PLU file (PLU master data), or table information (classification size determination table) linked with the PLU file may be separately prepared.
For example, in the case of an article close to a sphere such as "tomato" or "orange", it is not always necessary to obtain the volume, but one element such as diameter, radius, and circumference can be used as "L", "M", "S", etc. Can be classified.

When the reference table is configured separately for the PLU file and the classification size determination table, the general names indicating "tomato" and "orange" and their designated items may be described in the classification size determination table. By limiting this designated item, it is possible to reduce the amount of calculation and improve the reading speed of the article.

In the following examples of the size classification of articles, the designated items may be defined by the table information as in the above description. Further, an algorithm for automatically determining a specified size item from a rough object shape may be adopted.

For example, in the case of a spherical article such as "grapefruit", the size classification may be easily obtained only from the length of the long side and the length (diameter) of the short side.

Further, for example, even in the case of an article whose grade changes depending on the size (weight) such as "egg", the weight may be estimated from the long side, the short side and the assumed specific gravity, but the length of the long side and the length of the long side can be easily estimated. The classification such as "L", "M", "S" may be obtained only from the length (diameter) of the short side.

Further, in the case of an article having various shapes such as "sweet potato", the volume may be obtained from various item elements. Further, the division may be obtained by determining the size by an original calculation formula instead of the volume.

Further, in the case of a rod-shaped article having a curved line such as "banana", the length based on the curved line may be obtained.

On the other hand, in the case of an article having a size and other classification elements (mesh density in the case of melon) such as "melon", the identification unit 30 calculates and outputs each classification element together with the volume by image analysis. It may be configured. At the time of this image processing, solid identification processing may be performed from the "pattern" if necessary.

As the above-mentioned classification element, "hue", "glossiness", "number", "number", "with scratches" and the like can be used. As the "hue", one or more images obtained by additionally operating a multispectral camera may be used. This classification element can also be used as information for automatically calculating the time to eat and the best-by date of "fresh fish" and "fruit". In addition, by outputting the number of articles as a classification element, a discount code for bulk sales can be automatically set. The division element can also be used as complementary information for the object recognition unit 20. For example, in the case of the item of "glossiness", it can be used to classify "matte products" and "glossy products" of articles of the same shape that do not need to determine the size.

The classification unit 40 divides the object into preset categories based on the object recognition result (candidate) obtained from the object recognition unit 20, the object size item, and the category element item. For example, the grade of fruit, the size of vegetables, the presence / absence of a bulk sale flag, the presence / absence of scratches, etc. are classified for each target. Next, the classification unit 40 outputs the target object recognition result and its classification to the PLU collation unit (not shown). At this time, when there are a plurality of candidates in the target object recognition result, the classification unit 40 may output the target object recognition result and its classification for each individual candidate. The classification unit 40 may be configured to perform the PLU collation process.

The PLU collation unit reads out the price corresponding to the goods and the classification in the PLU data for each received goods, and adds the goods to the unsettled list. When there are a plurality of candidates or when a measure such as a discount is to be taken, the POS system may appropriately receive selection from the operator and determine the unsettled list. The calculation resources and comparison data of the PLU collation unit do not necessarily have to be in the store system. Computational resources and comparison data may be appropriately installed in a housing, a store, an in-house network, a cloud, or the like, if necessary.

Subsequent accounting processing may be appropriately configured according to the style of the store.

The POS system may be configured to execute the pricing process on the terminal according to the target classification result. For example, for goods corresponding to the PLU code, pricing formulas such as internal quantity, unit price per volume, grade item and unit price per volume, division element item and unit price per volume are registered in advance, and based on the formula. The terminal may calculate the price of the imaged object.

According to this mechanism, for example, when "vegetable bagging" and the price per volume of each vegetable are registered in the PLU code, the customer puts "carrot", "onion" and "potato" in the same bag. Even if you put it in the bag and take an image at the same time, you can set an appropriate price on the terminal.

With the configuration described above, the POS system can accurately identify and / or classify the objects depicted in the captured image to create an unsettled list.

FIG. 2 is a flowchart showing an example of processing operation related to the classification of the target in the POS system according to the embodiment.

The POS system first captures an article to be purchased in the imaging range of the imaging unit 20, and outputs the captured image to the object recognition unit 20 and the identification unit 30 (step S101). The image of the article output to the object recognition unit 20 and the identification unit 30 may be acquired from different image pickup devices.

Next, the POS system (object recognition unit 20) executes object recognition of the object depicted in the captured image, and outputs the result to the classification unit 40 (step S102).

At the same time, the POS system (identification unit 30) executes a size identification process for obtaining the size of the object depicted in the captured image, and outputs the result to the classification unit 40 (step S103). At this time, the identification unit 30 may extract the classification element from the image and output it to the classification unit 40. Further, the object recognition result may be received from the object recognition unit 20, and whether or not the size is identified and the item type that needs to be analyzed may be identified.

Next, the POS system (classification unit 40) determines the articles or article candidates that need to be classified from the articles depicted in the captured image and the classification based on the size of the target, the recognition result of the target, and the classification element. Then, the result is output to the PLU collation unit (step S104).

After that, the POS system acquires the price of the product from the PLU file and shifts to the process of adding to the unsettled list or the process of selecting a candidate.

By operating the POS system in this way, it is possible to accurately identify and / or classify the objects depicted in the captured image and create an unsettled list.
For example, in the existing POS system that executes object recognition, the machine was not able to accurately classify "tomatoes" and "mini tomatoes", the price of "strawberry packs", "watermelon" and "green onions". It is possible to classify the size such as ".
As a result, the labor of the worker is reduced and the cash register work can be speeded up.

In addition, the degree of freedom in pricing goods is increased.
In addition, self-checkout can reduce complicated operations for customers.
In addition, when pricing individual products at stores, etc., pricing is possible with this mechanism, which simplifies pricing work using a handler beller, barcode printer, or the like.

Next, a configuration example of the size determination table for classification will be described. FIG. 3 is an explanatory diagram showing an example of table information for classification.

In the PLU master shown in FIG. 3A, the PLU code (JAN (Japanese Article Number) code in the figure) is associated with a list of division elements to be additionally calculated together with the product name and price.
In the classification size determination table shown in FIG. 3B, a flag indicating whether or not to obtain the size, a desired item, and a classification element to be additionally calculated are configured in a list corresponding to the article name. In addition, the PLU code can be specified using the classification size determination table.

Each part of the POS system may be realized by using a combination of hardware and software. In the form of combining hardware and software, a POS system program (PLU classification identification program) is developed in RAM, and hardware such as a control unit (CPU) is operated based on the program. As a result, each part is operated as various means. In addition, this program may be temporarily recorded on a recording medium and distributed. The program recorded on the recording medium is read into the memory via wire, wireless, or the recording medium itself, and operates the control unit or the like. Examples of recording media include optical disks, magnetic disks, semiconductor memory devices, hard disks, and the like.

Although the present invention has been described using embodiments, various configurations can be taken when embodying the present invention. For example, a POS terminal device having an image pickup means, an object recognition means, an identification means, and a classification means all in one housing and one control unit executing up to a PLU identification process can be mentioned. Another aspect is a POS terminal device that has one control unit for imaging means, object recognition means, identification means, and classification means dispersed in a plurality of housings. Further, even in a mode in which control units are provided on the register side and the scanner side, respectively, as in the existing POS terminal device, the imaging means, the object recognition means, the identification means, and the classification means can be appropriately separated in the POS terminal device. Just do it.

It is also conceivable to consider a POS system in which an image is acquired by a large number of POS terminal devices, an object recognition or identification process is executed by a server in the store, and the result is returned to each POS terminal device.
To explain the features of the POS system of the above embodiment in another expression, an information processing device having one or more POS terminal devices having imaging means, an object recognition means, an identification means, and a classification means. Will be prepared.

Further, to explain the above embodiment in another expression, an information processing device that operates as a POS server is provided with a control unit as an object recognition means, an identification means, and a classification means based on a POS program developed in a RAM. It can be realized by operating it.

Although the embodiments and examples have been illustrated and described above, changes such as separation and merging of block configurations and replacement of procedures are free as long as the gist of the present invention and the functions described are satisfied, and the above description describes the present invention. It is not limited.

As described above, the POS system to which the present invention is applied can accurately identify and / or classify the object depicted in the captured image.

That is, according to the present invention, it is possible to provide a POS terminal device, a server, a POS system, a PLU identification method, and a program that accurately identify and / or classify an object depicted in a captured image.

In addition, some or all of the above embodiments may also be described as follows. The following notes do not limit the present invention in any way.
[Appendix 1]
An imaging means for capturing an image of an object,
An object recognition means for recognizing an object and
An identification means for identifying the size of the object and
A classification means for classifying the object based on the size of the object and the recognition result of the object, and
A POS terminal device characterized by having.

[Appendix 2]
It has a distance calculation means that identifies the distance to the target,
The identification means
The POS terminal device according to the above description, wherein the size of the target is derived based on the screen area range of the target included in the captured image of the target and the distance obtained by the distance calculation means. ..

[Appendix 3]
The imaging means acquires a plurality of images having parallax and obtains a plurality of images.
The identification means
The distance from the difference between the images of each object having parallax to the object is identified and processed.
The POS terminal device according to the above description, wherein the size of the target is derived based on the distance obtained from the difference between the image region range of the target included in the captured image of the target and the image. ..

[Appendix 4]
The identification means
From the difference between the images of the same object acquired at different times by the imaging means, the distance of the image to recognize the object to the object is identified.
The POS terminal device according to the above description, wherein the size of the target is derived based on a distance obtained from a difference between an image and a region range of the target included in the captured image of the target.

[Appendix 5]
The identification means obtains one or a plurality of items for individually determining the size or size of a plurality of objects depicted in an image, and derives the size of the plurality of objects as one grouped object. The POS terminal device according to the above description, which is a feature.

[Appendix 6]
The POS terminal device according to the above description, wherein the identification means joins a plurality of consecutive images of the target captured by the imaging means to estimate the size of the target.

[Appendix 7]
The identification means identifies a part of the target image captured by the imaging means and estimates the size of the target from the curvature of the contour of the part of the target. The POS terminal device according to the above description, which is a feature.

[Appendix 8]
The identification means records table information in which items used as a standard for determining the size of an identification target prepared in advance are recorded in correspondence with an object name output as an object recognition result, and an object recognition result of the object recognition means. The POS terminal device according to the above description, wherein an item used as a reference for determining the size of the object is specified with reference to the object, and the size of the object is identified based on the specified item.

[Appendix 9]
The identification means refers to the table information prepared in advance corresponding to the object name output as the object recognition result and the object recognition result of the object recognition means, and the object recognition result of the object recognition means. The POS terminal device according to the above description, wherein a target classification element item is obtained by image analysis, and the result is output to the classification means.

[Appendix 10]
The classification means is characterized in that the object-recognized object is classified into any of the categories corresponding to the PLU code based on at least the value of each item reflecting the size of the object. The POS terminal device described in the appendix.

[Appendix 11]
The POS terminal device according to the above description, wherein the POS terminal device enables execution of a pricing process according to a classification result based on the size of the target.

[Appendix 12]
An object recognition means for recognizing an object and
An identification means for identifying the size of the object and
A classification means for classifying the object based on the size of the object and the recognition result of the object, and
A server characterized by having.

[Appendix 13]
The identification means
The server according to the above appendix, wherein the size of the target is derived based on the screen area range of the target included in the captured image of the target and the distance from the notified imaging position.

[Appendix 14]
The identification means
A plurality of images having parallax are acquired, and the distance from the difference between the images of each object having parallax to the target is identified.
The server according to the above appendix, wherein the size of the target is derived based on a distance obtained from a difference between an image region range of the target included in the captured image of the target and the image.

[Appendix 15]
The identification means
From the difference between the images of the same object acquired at different times, the distance of the image to recognize the object to the object is identified.
The server according to the above appendix, wherein the size of the target is derived based on a distance obtained from a difference between an image and a region range of the target included in the captured image of the target.

[Appendix 16]
The identification means obtains one or a plurality of items for individually determining the size or size of a plurality of objects depicted in an image, and derives the size of the plurality of objects as one grouped object. The server described in the above appendix as a feature.

[Appendix 17]
The server according to the above appendix, wherein the identification means connects a plurality of consecutive images of the target and estimates the size of the target.

[Appendix 18]
The above-mentioned appendix, wherein the identification means identifies a part of the target area in the image of the target and estimates the size of the target from the curvature of the contour of the part of the target area. server.

[Appendix 19]
The identification means records table information in which items used as a reference for determining the size of an identification target prepared in advance are recorded in correspondence with an object name output as an object recognition result, and an object recognition result of the object recognition means. The server according to the above appendix, which is characterized in that an item used as a criterion for determining the size of the object is specified with reference to the object, and the size of the object is identified based on the specified item.

[Appendix 20]
The identification means refers to the table information prepared in advance corresponding to the object name output as the object recognition result and the object recognition result of the object recognition means, and the object recognition result of the object recognition means. The server according to the above appendix, characterized in that a target classification element item is obtained by image analysis and the result is output to the classification means.

[Appendix 21]
The classification means is characterized in that the object-recognized object is classified into any of the categories corresponding to the PLU code based on at least the value of each item reflecting the size of the object. The server described in the appendix.

[Appendix 22]
The server according to the above appendix, wherein a price is specified according to a classification result based on the size of the target, and the price is output as data for pricing processing.

[Appendix 23]
A POS terminal device having an imaging means for capturing an image of an object,
An object recognition means for recognizing an object and
An identification means for identifying the size of the object and
A classification means for classifying the object based on the size of the object and the recognition result of the object, and
Information processing device with
POS system equipped with.

[Appendix 24]
An imaging means for capturing an image of an object,
An identification means for identifying the size of the object and
With a POS terminal device
An object recognition means for recognizing an object and
A classification means for classifying the object based on the size of the object and the recognition result of the object, and
Information processing device with
POS system equipped with.

[Appendix 25]
An information processing device having an object recognition means for recognizing an object,
An imaging means for capturing an image of the target and
An identification means for identifying the size of the object and
A classification means for classifying the object based on the size of the object and the recognition result of the object, and
With a POS terminal device
POS system equipped with.

[Appendix 26]
An object recognition means that recognizes an object and
An identification means for identifying the size of the object and
Information processing device with
An imaging means for capturing an image of the target and
A classification means for classifying the object based on the size of the object and the recognition result of the object, and
With a POS terminal device
POS system equipped with.

[Appendix 27]
The information processing device is a POS system which is a server according to any one of the above appendices 12 to 22.

[Appendix 28]
An information processing system that can access PLU data is
The target image captured by the imaging unit and the recognition result obtained by subject recognition processing of the target image are acquired.
The size of the object is identified and processed,
Based on the size of the object and the recognition result of the object, the object is classified into one of the PLU categories, and the object is classified into one of the PLU categories.
A PLU identification method, characterized in that the PLU code of the target is specified from the PLU data based on the recognition result and the classification result of the target.

[Appendix 29]
The information processing system is used as a process for identifying the size of the target.
The feature is that the size of the target is derived based on the screen area range of the target included in the captured image of the target and the distance obtained by the distance calculation unit that identifies the distance to the target. The PLU identification method described in the above appendix.

[Appendix 30]
The information processing system is used as a process for identifying the size of the target.
Using a plurality of images having parallax acquired by the imaging unit, the distance from the difference between the images of each object having parallax to the object is identified.
The PLU specifying method according to the above appendix, wherein the size of the target is derived based on the distance obtained from the difference between the image region range of the target included in the captured image of the target and the image. ..

[Appendix 31]
The information processing system is used as a process for identifying the size of the target.
From the difference between the images of the same object acquired at different times by the imaging unit, the distance of the image to recognize the object to the object is identified.
The PLU identification method according to the above description, wherein the size of the target is derived based on a distance obtained from a difference between an image and a region range of the target included in the captured image of the target.

[Appendix 32]
The information processing system is used as a process for identifying the size of the target.
The above-mentioned appendix, which comprises obtaining one or a plurality of items for individually determining the size or size of a plurality of objects depicted in an image, and deriving the size of the plurality of objects as one grouped object. The PLU identification method described.

[Appendix 33]
The information processing system is used as a process for identifying the size of the target.
The PLU identification method according to the above description, wherein a plurality of consecutive images of the target captured by the imaging unit are joined together to estimate the size of the target.

[Appendix 34]
The information processing system is used as a process for identifying the size of the target.
The above-mentioned appendix, characterized in that a part of the target area captured by the imaging unit is identified and the size of the target is estimated from the curvature of the contour of the part of the target area. The PLU identification method described.

[Appendix 35]
The information processing system is used as a process for identifying the size of the target.
The size of the object is determined by referring to the table information prepared in advance and the items used as the criteria for determining the size of the identification object, which are recorded corresponding to the object name output as the object recognition result, and the object recognition result. The PLU identification method according to the above description, wherein an item used for a specified standard is specified, and the size of the object is identified based on the specified item.

[Appendix 36]
The information processing system refers to the table information prepared in advance corresponding to the object name output as the object recognition result of the classification element item of the identification target and the object recognition result, and refers to the classification element of the target. The PLU identification method according to the above appendix, wherein items are obtained by image analysis and the results are classified.

[Appendix 37]
The information processing system is characterized in that the object-recognized object is classified into any of the categories corresponding to the PLU code based on at least the value of each item reflecting the size of the object. The PLU identification method described in the above appendix.

[Appendix 38]
The PLU identification method according to the above description, wherein the information processing system makes it possible to execute a pricing process according to a classification result based on the size of the target.

[Appendix 39]
Information processing system,
An image recognition means that recognizes an object from an image of an object captured by the image pickup unit,
An identification means for identifying the size of the object and
A classification means for classifying the object based on the size of the object and the recognition result of the object.
A program characterized by operating as.

[Appendix 40]
Information processing device,
An imaging means for capturing an image of an object,
An identification means for identifying the size of the object,
A program characterized by operating as.

[Appendix 41]
Information processing device,
An object recognition means that recognizes an object and
A classification means for classifying the object based on the size of the object and the recognition result of the object.
A program characterized by operating as.

[Appendix 42]
Information processing device,
An imaging means for capturing an image of an object,
An identification means for identifying the size of the object and
A classification means for classifying the object based on the size of the object and the recognition result of the object.
A program characterized by operating as.

[Appendix 43]
Information processing device,
An object recognition means that recognizes an object and
An identification means for identifying the size of the object,
A program characterized by operating as.

[Appendix 44]
Information processing device,
An imaging means for capturing an image of an object,
A classification means for classifying the object based on the size of the object and the recognition result of the object.
A program characterized by operating as.

[Appendix 45]
A program characterized by operating as the POS terminal device according to any one of the above-mentioned appendices 1 to 11.

[Appendix 46]
A program characterized by operating as the server according to any one of the above appendices 12 to 22.

The present invention can be applied to POS terminal devices arranged in supermarkets, convenience stores, department stores, and the like. It can also be applied to tablet POS systems and smartphone POS systems.

10 Imaging unit (imaging means)
20 Object recognition unit (object recognition means)
30 Identification unit (identification means, image analysis means)
40 Classification section (classification means)

Claims (10)

An object recognition means that recognizes an article from an image captured by the imaging means, and an object recognition means.
An identification means for identifying the size of the article based on the image, and
A classification means for determining the classification of an object-recognized article based on the size of the article identified by the identification means by referring to an information table in which the size of the article recognized as an object and the classification of the article are associated with each other. When,
A collation means for reading out the price corresponding to the classification of the goods in the product data for the goods confirmed by the classification means, and
A system characterized by including. The identification means refers to the classification size item for each article prepared in the information table, the image area range of the article included in the image, the measured distance to the article, and the like. The system according to claim 1, wherein the size of the article is identified by calculation based on the classification size item of the recognition result. The classification means includes, for an article or an article candidate that needs to be classified, a recognition result of the article obtained from the object recognition means, a size of the article obtained from the identification means, and a size classification range of the article. The second aspect of the present invention is characterized in that the classification of the article depicted in the image is determined in any of the classifications included in the information table, and the result is output to the collation means. System. The identification means individually obtains the value of the size item for classification of a plurality of objects depicted in the image, and the value of the size item for classification of each of the obtained multiple objects and the measured distance to each article. The system according to any one of claims 1 to 3, wherein the above-mentioned is included in the parameters used in the derivation process of the size of the plurality of objects as one bundled article. The identification means joins a plurality of consecutive images of the article captured by the imaging means to form a single image, and measures the image region range of the article and the article as depicted in the monoimaged image. The system according to any one of claims 1 to 4, wherein the size of the article is estimated by including the distance as a parameter. The identification means identifies an image region range and a part of the image region captured in the image of the article captured by the imaging means, and the curvature of the contour of the target image region range and the partial region. The system according to any one of claims 1 to 5, wherein the size of the article and the measured distance to the article are included in a parameter to estimate the size of the article. The system according to any one of claims 1 to 6, wherein the collation means is provided on a cloud. The system
Including one or more POS terminal devices having the imaging means,
As the one or more POS terminal devices, a tablet terminal or a smartphone terminal is used.
The system according to any one of claims 1 to 6, wherein the collation means is provided on a cloud. An information processing system that can access product data
Acquire an image of an article captured by an imaging means,
The recognition result obtained by recognizing the article by the object recognition means by the object recognition means is acquired from the image, and the recognition result is obtained.
The size of the article is identified by the identification means based on the image, and the size of the article is identified by the identification means.
Refer to the information table in which the size of the article recognized as an object and the classification of the article are associated with each other, and based on the size of the article identified by the identification means, the classification of the article recognized as an object is selected as one of the product categories. Confirmed
A product identification method, characterized in that, based on the result, the price corresponding to the classification of the goods in the goods data is read out with respect to the confirmed goods. The product identification method according to claim 9, wherein the information processing system uses PLU (Price Look Up) data as the product data.

Images