CN111753837A - Learning completion model generation method and device, and commodity discrimination method and device - Google Patents

Learning completion model generation method and device, and commodity discrimination method and device Download PDF

Info

Publication number
CN111753837A
CN111753837A CN202010203532.0A CN202010203532A CN111753837A CN 111753837 A CN111753837 A CN 111753837A CN 202010203532 A CN202010203532 A CN 202010203532A CN 111753837 A CN111753837 A CN 111753837A
Authority
CN
China
Prior art keywords
product
image
divided
commodity
images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010203532.0A
Other languages
Chinese (zh)
Inventor
堤弘法
府中谷洸介
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ishida Co Ltd
Original Assignee
Ishida Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ishida Co Ltd filed Critical Ishida Co Ltd
Publication of CN111753837A publication Critical patent/CN111753837A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0283—Price estimation or determination
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0036—Checkout procedures
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/60—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to nutrition control, e.g. diets
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/68—Food, e.g. fruit or vegetables

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Business, Economics & Management (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Development Economics (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Strategic Management (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • Quality & Reliability (AREA)
  • Evolutionary Biology (AREA)
  • Marketing (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Game Theory and Decision Science (AREA)
  • Probability & Statistics with Applications (AREA)
  • Economics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Business, Economics & Management (AREA)
  • Nutrition Science (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)

Abstract

The invention relates to a learning completion model generation method and device, and a commodity discrimination method and device. A weighing system (1) is provided with a server (200) for identifying the type of a product from a product image including the product, and a weighing device (100) for identifying the type of the product from a target image including the product. A server (200) is provided with: an acquisition unit that acquires a product image and product information related to the type of product; a dividing unit that divides the product image into a plurality of regions and acquires a plurality of divided images; an extraction unit that extracts a plurality of divided images that satisfy a predetermined condition relating to an amount of captured merchandise from the plurality of divided images; and a generation unit that performs machine learning based on the plurality of divided images extracted by the extraction unit and generates a discrimination model. The meter (100) is provided with a determination unit (33) that acquires the processing result of the processing by the determination model and determines the type of the product based on the processing result.

Description

Learning completion model generation method and device, and commodity discrimination method and device
Technical Field
One aspect of the present invention relates to a learned model generation method, a learned model generation device, a commodity discrimination method, a commodity discrimination device, and a weighing device.
Background
Conventionally, a method of discriminating a product from an image containing the product is known. For example, in a product discrimination method described in patent document 1 (japanese patent application laid-open No. 2015-64628), a feature amount of an image of a product is extracted, and the feature amount is compared with a feature amount in learning data to identify the type of the product.
Disclosure of Invention
Examples of products sold in supermarkets and the like include salad, non-staple food, and the like. Salad, non-staple food, etc. contain a plurality of food items, and the shape is not always the same. In this way, when the shape of the product is irregular, it is difficult to obtain the feature amount of the product. Therefore, the accuracy of product discrimination may be reduced.
An object of one aspect of the present invention is to provide a learned model generation method, a learned model generation device, a product discrimination method, a product discrimination device, and a weighing device that achieve an improvement in the discrimination accuracy of a product.
A learning-completed model generation method according to an aspect of the present invention is a method for generating a discrimination model for discriminating a type of a commodity from a commodity image including the commodity, including: an acquisition step of acquiring a commodity image and commodity information related to the kind of a commodity; a segmentation step of segmenting the commodity image into a plurality of regions and acquiring a plurality of segmented images; an extraction step of extracting a predetermined divided image from the plurality of divided images on the basis of a predetermined condition relating to an amount of captured merchandise; and a generation step of associating the plurality of divided images extracted in the extraction step with the product information and performing machine learning to generate a discrimination model.
In the learning-completed model generation method according to one aspect of the present invention, a predetermined divided image is extracted from a plurality of divided images on the basis of a predetermined condition relating to the shot size of the product. The divided images include an image in which only the product is captured, an image in which the product and the non-product (container, background, etc.) are captured, and an image in which only the non-product is captured. In the learned model generation method, a segmented image is extracted from the segmented images according to a predetermined condition, and machine learning is performed based on the extracted segmented image. Thus, in the learning-completed model generation method, machine learning based on an appropriate teaching image can be performed. Therefore, in the learned model generation method, a discrimination model for improving the discrimination accuracy of the product can be generated.
In one embodiment, in the extracting step, a plurality of divided images satisfying a predetermined condition may be extracted from the plurality of divided images. In this method, machine learning can be performed based on an appropriate teaching image. Therefore, in the learned model generation method, a discrimination model for improving the discrimination accuracy of the product can be generated.
In one embodiment, in the extracting step, the divided image in which the shot amount other than the commodity is equal to or less than the threshold value may be extracted. In other words, in the extraction step, the divided image in which the shot size of the product is larger than the threshold value is extracted. In this method, a segmented image having a high commodity occupancy rate is extracted. That is, in the learned model generation method, the divided images of the container or the background captured in addition to the commodity can be excluded. Therefore, in the learning-completed model generation method, machine learning can be performed from the segmented image of the captured product. Therefore, in the learned model generation method, a discrimination model for improving the discrimination accuracy of the product can be generated.
In one embodiment, in the extracting step, a non-product likelihood indicating a probability that the divided image does not include the product may be acquired for each of the plurality of divided images using an extraction model generated by machine learning based on the image not including the product, and the divided images having the non-product likelihood of not more than a threshold value may be extracted. In this method, only the divided image in which the product is captured can be appropriately extracted.
In one embodiment, the generation of the extraction model may also include: a first step of acquiring a non-commodity image not containing a commodity; a second step of dividing the non-commodity image into a plurality of regions to obtain a plurality of non-commodity divided images; and a third step of performing machine learning from the plurality of non-commodity segmented images to generate an extraction model. Thus, in the learned model generation method, the extraction model can be appropriately generated.
In one embodiment, in the dividing step, the product image may be divided so that the regions of the plurality of divided images are rectangular and all have the same size; in the second step, the non-product image is divided so that each of the plurality of non-product divided images has the same shape and the same size as the divided image. In this method, since the divided image and the non-product divided image have the same shape and the same size, it is not necessary to perform conversion processing for converting the shape and size of the image. Therefore, in the learned model generation method, the processing load is reduced.
In one embodiment, in the extracting step, one divided image in which the shot amount of the non-product is equal to or less than the threshold value and the other divided images in which the shot amount of the non-product is greater than the threshold value are extracted from the plurality of divided images, the product information may be associated with the one divided image, and the non-product information indicating that the non-product is not the product may be associated with the other divided images; in the generation step, machine learning is performed on the basis of the one divided image extracted in the extraction step and the other divided images, and a discrimination model is generated. In this method, machine learning is performed based on a divided image marked with product information and another image marked with non-product information. In this way, since the divided image in which the product is captured and the divided image in which the captured amount of the product is small (not captured) are distinguished (classified) and machine learning is performed, it is possible to generate the discrimination model for improving the discrimination accuracy of the product.
In one embodiment, in the extracting step, a divided image in which the inclusion ratio of pixels is equal to or less than a predetermined threshold may be extracted from the plurality of divided images based on the feature values of the pixels of the image not including the product. In this method, only the divided image in which the product is captured can be appropriately extracted.
In one embodiment, in the dividing step, the product image may be divided such that one divided image overlaps at least a part of the other divided image. In this method, when dividing the product image, even when one article included in the product is cut in one divided image, the article may be completely included in the other divided images. Therefore, in the learned model generation method, since machine learning can be performed based on an appropriate teaching image, a discrimination model can be generated which can improve the discrimination accuracy of a commodity.
In one embodiment, in the dividing step, the area for dividing the product image may be set such that the other area is moved by a predetermined amount with respect to one area in a first direction, which is an arrangement direction of pixels in the product image, or in a second direction orthogonal to the first direction. In this method, by moving the region by a predetermined amount, the product image can be divided so that the number of times each pixel is included in each region arranged in the moving direction is equal.
In one embodiment, in the dividing step, the product image may be divided so that each of the plurality of divided images has a rectangular region and all of the regions have the same size. In this method, since the divided images are all of the same shape and the same size, it is not necessary to perform conversion processing for converting the shape and size of the image. Therefore, in the learned model generation method, the processing load is reduced.
In one embodiment, the method may further include a cutting step of comparing the product image with a base image not including the product, and cutting out a product area including at least the entire product and smaller than the product image from the product image; in the dividing step, the product region is divided into a plurality of regions, and a plurality of divided images are acquired. In this method, since the product image is divided using the product region excluding the product, it is not necessary to divide the unnecessary region. Therefore, in the learned model generation method, the processing load is reduced.
A learned model generation device according to an aspect of the present invention is a determination model generation device that determines a type of a product from a product image including the product, and includes: an acquisition unit that acquires a product image and product information related to the type of product; a dividing unit that divides the product image into a plurality of regions and acquires a plurality of divided images; an extraction unit that extracts a plurality of divided images from the plurality of divided images on the basis of a predetermined condition relating to an amount of captured merchandise; and a generation unit that performs machine learning from the plurality of divided images extracted by the extraction unit to generate a discrimination model.
In the learned model generation device according to one aspect of the present invention, a predetermined divided image is extracted from the plurality of divided images on the basis of a predetermined condition relating to the shot size of the product. The divided images include an image in which only the product is captured, an image in which the product and the non-product (container, background, etc.) are captured, and an image in which only the non-product is captured. The learned model generation device extracts a segmented image from the segmented images according to a predetermined condition, and performs machine learning from the extracted segmented image. Thus, the learning-completed model generation device can perform machine learning from an appropriate teaching image. Therefore, the learned model generation device can generate a discrimination model that can improve the discrimination accuracy of the product.
A commodity discrimination method according to an aspect of the present invention is a commodity discrimination method for discriminating a type of a commodity from an object image including the commodity using a discrimination model generated by the learned model generation method, including: a first acquisition step of acquiring a plurality of divided object images by dividing the object image into a plurality of regions; a second acquisition step of acquiring, from the plurality of divided object images, a plurality of divided object images that satisfy a predetermined condition relating to an amount of captured merchandise; and a determination step of obtaining a processing result of the processing by the determination model for the plurality of divided object images obtained in the second obtaining step, and determining the type of the commodity based on the processing result.
In the product discrimination method according to one aspect of the present invention, the discrimination model is used to discriminate the type of the product. Therefore, in the commodity discrimination method, the accuracy of discrimination of the commodity is improved. In the product discrimination method, a plurality of divided object images satisfying a predetermined condition relating to the captured amount of the product are acquired from the plurality of divided object images. The division target image includes an image in which only the product is captured, an image in which the product and the article other than the product (container, background, etc.) are captured, and an image in which only the article other than the product is captured. In the product discrimination method, a division target image satisfying a predetermined condition is acquired from the division target image, and the acquired division target image is processed by a discrimination model. Thus, the product discrimination method can perform processing based on an appropriate segmentation target image. Therefore, in the commodity discrimination method, the accuracy of discrimination of the commodity is improved.
In one embodiment, in the second acquisition step, the image to be segmented in which the shot size other than the product is equal to or less than the threshold value may be acquired. In this method, an image with a high commodity occupancy rate is acquired. In other words, the image of the container or the background other than the commodity can be excluded. Therefore, in the commodity discrimination method, the accuracy of discrimination of the commodity is improved.
In one embodiment, in the second acquisition step, a non-product likelihood indicating a probability that the divided object image does not include the product may be acquired for each of the plurality of divided object images using an acquisition model generated by machine learning based on an image not including the product, and the divided object images having the non-product likelihood of not more than a threshold value may be acquired. In this method, only the image to be segmented in which the product is captured can be appropriately acquired.
In one embodiment, in the first acquisition step, the target image may be divided so that the respective regions of the plurality of divided target images are rectangular and all have the same size. In this method, since all the images to be segmented have the same shape and the same size, it is not necessary to perform conversion processing for converting the shape and size of the image. Therefore, in the commodity discrimination method, the processing load is reduced.
A commodity discrimination method according to an aspect of the present invention is a commodity discrimination method for discriminating a type of a commodity from an object image including the commodity using a discrimination model generated by the learned model generation method, including: a divided image acquisition step of dividing the target image into a plurality of regions and acquiring a plurality of divided target images; and a discrimination step of acquiring, from the plurality of divided target images acquired in the divided image acquisition step, a product likelihood indicating a probability of being a product and a non-product likelihood indicating a probability of being other than a product in a processing result of processing by the discrimination model, and discriminating the type of the product based on the product likelihood.
In the product discrimination method according to one aspect of the present invention, the discrimination model is used to discriminate the type of the product. Therefore, in the commodity discrimination method, the accuracy of discrimination of the commodity is improved. In the product discrimination method, the product likelihood indicating the probability of being a product and the non-product likelihood indicating the probability of being a product are acquired from the processing result of the processing by the discrimination model, and the type of the product is discriminated from the product likelihood. Thus, in the commodity discrimination method, the discrimination accuracy of the commodity is improved.
In one embodiment, the determination step may be configured to process each of the plurality of divided images by a determination model to obtain a plurality of processing results, and determine the type of the product based on the plurality of processing results. In this method, since each of the divided object images is processed by the discrimination model to obtain a plurality of processing results, it is possible to further improve the discrimination accuracy of the product.
In one embodiment, in the determining step, the processing result may be weighted according to a degree of likelihood of the product indicating a probability that the product is of one type, and the type of the product may be determined according to a majority determination method of weighting the processing result. In this method, the product can be discriminated with higher accuracy.
In one embodiment, in the determination step, when a plurality of products are included in the target image, the type of each of the plurality of products may be determined based on the processing result of each of the regions including each of the plurality of products. In this method, the type of each of the plurality of products included in the target image can be determined.
A commodity discrimination device according to an aspect of the present invention discriminates a type of a commodity from a target image including the commodity using a discrimination model generated by the learned model generation method, and includes: a first acquisition unit configured to acquire a plurality of divided target images by dividing a target image into a plurality of regions; a second acquisition unit that acquires, from among the plurality of divided object images, a plurality of divided object images that satisfy a predetermined condition relating to an amount of captured merchandise; and a determination unit configured to acquire a processing result of the processing by the determination model for the plurality of divided images acquired by the second acquisition unit, and determine the type of the product based on the processing result.
In the product discrimination device according to one aspect of the present invention, the discrimination model is used to discriminate the type of the product. Therefore, the commodity discrimination device can improve the discrimination accuracy of the commodity. In addition, the product discrimination device acquires, from among the plurality of divided object images, a plurality of divided object images that satisfy a predetermined condition relating to the captured amount of the product. The division target image includes an image in which only the product is captured, an image in which the product and the article other than the product (container, background, etc.) are captured, and an image in which only the article other than the product is captured. In the product discrimination device, a division target image satisfying a predetermined condition is acquired from the division target image, and the acquired division target image is processed by a discrimination model. Thus, the product discrimination device can perform processing based on an appropriate division target image. Therefore, the commodity discrimination device can improve the discrimination accuracy of the commodity.
A product discrimination system according to an aspect of the present invention includes a generation device of a discrimination model for discriminating a type of a product from a product image including the product, and a discrimination device for discriminating the type of the product from a target image including the product, wherein the generation device includes: an acquisition unit that acquires a product image and product information related to the type of product; a dividing unit that divides the product image into a plurality of regions and acquires a plurality of first divided images; an extraction unit that extracts a plurality of first divided images that satisfy a predetermined condition relating to an amount of captured merchandise from the plurality of first divided images; and a generation unit configured to perform machine learning by associating the plurality of first divided images extracted by the extraction unit with the product information to generate a discrimination model, wherein the discrimination device includes: a first acquisition unit configured to acquire a plurality of second divided images by dividing a target image into a plurality of regions; a second acquisition unit that acquires, from among the plurality of second divided images, a plurality of second divided images that satisfy a predetermined condition relating to an amount of capture of the commodity; and a determination unit configured to acquire a processing result of the processing by the determination model for the plurality of second divided images acquired by the second acquisition unit, and determine the type of the product based on the processing result.
In the product discrimination system generating device according to one aspect of the present invention, the plurality of first divided images satisfying a predetermined condition relating to the captured amount of the product are acquired from the plurality of divided images. The first divided images include an image in which only the product is captured, an image in which the product and the non-product (container, background, etc.) are captured, and an image in which only the non-product is captured. The generation device extracts a first divided image satisfying a predetermined condition from the first divided images, and performs machine learning based on the extracted first divided image. Thus, the generation device can perform machine learning based on an appropriate teaching image. Therefore, the generation device can generate a discrimination model for improving the discrimination accuracy of the product.
In the discrimination device of the product discrimination system, the plurality of divided images satisfying a predetermined condition relating to the captured amount of the product are acquired from the plurality of second divided images. The determination device acquires a second divided image satisfying a predetermined condition from the second divided image, and processes the acquired second divided image using a determination model. Thus, the discrimination device can perform processing based on an appropriate second divided image. Therefore, in the commodity discrimination system, the accuracy of discrimination of commodities is improved.
A weighing apparatus according to an aspect of the present invention is a weighing apparatus that determines a type of a commodity from a target image including the commodity using a determination model generated by the learned model generation method, and calculates a price of the commodity, the weighing apparatus including: a measuring unit for measuring the weight of the commodity; an imaging unit that images a commodity; a determination unit for acquiring a processing result of the processing by the determination model with respect to the object image captured by the imaging unit, and determining the type of the commodity based on the processing result; and a calculating unit for calculating the price of the commodity based on the weight of the commodity measured by the measuring unit and the type of the commodity determined by the determining unit.
In the weighing apparatus according to one aspect of the present invention, the discrimination model is used to discriminate the type of the product. Therefore, the accuracy of discriminating a product is improved in the weighing apparatus.
According to one aspect of the present invention, it is realized to improve the discrimination accuracy of a commodity.
Drawings
Fig. 1 is a diagram showing a structure of a meter according to an embodiment.
Fig. 2 is a perspective view showing the meter.
Fig. 3 is a diagram showing a configuration of the measuring apparatus.
Fig. 4 is a diagram showing a configuration of the control device.
Fig. 5 is a diagram showing a product discrimination method in the discrimination unit.
Fig. 6 is a diagram showing an object image including a product.
Fig. 7A is a diagram showing a segmentation target image.
Fig. 7B is a diagram showing a segmentation target image.
Fig. 7C is a diagram showing a segmentation target image.
Fig. 7D is a diagram showing a segmentation target image.
Fig. 8 is a diagram showing a neural network.
Fig. 9 is a diagram showing a neural network.
Fig. 10 is a diagram showing a method of generating an extraction model by the learned model generating unit.
Fig. 11 is a diagram showing a non-product image containing no product.
Fig. 12 is a diagram showing a method of generating the discrimination model by the learned model generating unit.
Fig. 13 is a diagram showing a product image including a product.
Fig. 14A is a diagram showing a divided image.
Fig. 14B is a diagram showing a divided image.
Fig. 14C is a diagram showing a divided image.
Fig. 14D is a diagram showing a divided image.
Fig. 15 is a diagram showing an object image including a product.
Detailed Description
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or corresponding elements are denoted by the same reference numerals, and redundant description is omitted.
As shown in fig. 1, the weighing system (product discrimination system) 1 includes a meter (product discrimination device) 100 and a server (learned model generation device) 200. The meter 100 is connected to the server 200 in a manner capable of communicating.
The meter 100 includes a metering device 2, a control device 3, a display device 4, and a camera (imaging unit) 5. The metering device 2 and the control device 3 are connected so as to be able to communicate with each other. The control device 3 is connected to the camera 5 so as to be able to communicate with each other. The metering device 2 is connected to the display device 4 in a communicable manner.
The metering device 2 is an electronic scale. The weighing device 2 has a function of weighing the weight of the product S (see fig. 6) and dispensing the label LC attached to the weighed product S. In the present embodiment, the product S is a food (salad, non-staple food) or the like placed in the container P. Here, the container P has a transparent lid. The cover portion is colored or colorless and is formed of a material that transmits light. The camera 5 images the product S in the container P by imaging the light transmitted through the lid portion of the container P.
As shown in fig. 2, in the present embodiment, the weighing device 2 is housed in a housing 10. The housing 10 includes a housing 11, a holding portion 12, and a connecting portion 13. The storage unit 11 stores the measuring device 2. The housing 11 is box-shaped. The housing 11 is provided with a first opening 11a for exposing the measuring table 21a of the measuring section 21 of the measuring device 2 and a second opening 11b for exposing the operating section 22 of the measuring device 2.
The control device 3, the camera 5, the first illumination unit 6, and the second illumination unit 7 are disposed in the holding unit 12. The holding portion 12 is disposed on the housing portion 11. The control device 3 is disposed on the holding portion 12. The connecting portion 13 connects the housing portion 11 and the holding portion 12. The coupling portion 13 extends in the vertical direction. The display device 4 is disposed on the rear surface of the connection portion 13. The display device 4 is, for example, a liquid crystal display. The display device 4 performs display for the customer.
The camera 5 is disposed above a measuring table 21a (described later) of the measuring apparatus 2 at a position facing the measuring table 21 a. The camera 5 outputs captured image data to the control device 3. Further, the camera 5 outputs captured image data to the server 200.
The first illumination unit 6 and the second illumination unit 7 are LED illumination, for example. The first illumination unit 6 and the second illumination unit 7 are disposed above the measuring table 21a of the measuring device 2 at positions facing the measuring table 21 a. The first illumination unit 6 and the second illumination unit 7 are disposed at a predetermined interval in the width direction of the housing 10. Specifically, the first illumination unit 6 and the second illumination unit 7 are disposed at positions sandwiching the camera 5 in the width direction of the housing 10.
As shown in fig. 3, the measuring apparatus 2 includes a control unit 20, a measuring unit 21, an operating unit 22, and a label issuing unit 23.
The control Unit 20 is a part that controls various operations in the measuring apparatus 2, and includes a CPU (central processing Unit), a ROM (Read Only Memory), a RAM (random access Memory), and the like. The control unit 20 controls display of a touch panel display 22a described later.
The measuring unit 21 includes a measuring table 21a, a load cell not shown, a signal processing circuit, and a transmission module. The weighing table 21a mounts the product S. The load cell is disposed below the measurement table 21 a. The load cell converts mechanical strain generated by placing the object on the measurement table 21a into an electric signal. The signal processing circuit amplifies and converts the electric signal output from the load cell into a digital signal. The transmission module outputs the digital signal to the control unit 20.
The operation unit 22 includes a touch panel display 22a and a fixed key 22 b. The touch panel display 22a displays information on the product S measured by the measuring device 2, basic information necessary for the operation of the measuring device 2, and the like under the control of the control unit 20. The fixed keys 22b include a "unit price" key, a "rating" key, a "tare" key, a "print" key, and a "call" key, which are necessary for the charge scale, and they are appropriately arranged together with the numeric keys.
The label issuing section 23 issues the label LC. The label issuing unit 23 prints the product information based on the print information output from the output unit 29 described later, and issues the label LC. In the present embodiment, the label issuing section 23 issues a so-called base-paper-free label.
The control unit 20 includes an input unit 25, a storage unit 26, a calculation unit 27, a control unit 28, and an output unit 29.
The input unit 25 inputs the number information output from the control device 3. The input unit 25 outputs the input number information to the calculation unit 27.
The storage unit 26 stores a product master. The commodity master file stores commodity-related information related to the commodity S for each commodity S. The product master is a table in which product numbers are associated with product names, unit prices, denominations, and the like. The product master can be updated (changed).
The calculation unit 27 calculates the price of the product S. Upon receiving the number information output from the input unit 25, the calculation unit 27 refers to the product master based on the product number included in the number information. The calculation unit 27 acquires the unit price of the product S corresponding to the product number from the product master. The calculation unit 27 calculates the price (cost) of the product S based on the measurement value output from the measurement unit 21 and the unit price of the product. Upon receiving the identification information from the control unit 28, the calculation unit 27 identifies the price and outputs the print information to the label issuing unit 23. The print information includes at least information indicating a product name, a weight, and a price. Further, the calculation unit 27 outputs display information for displaying the calculated price on the touch-panel display 22a to the touch-panel display 22 a.
When the "print" key is pressed in the fixed key 22b of the operation section 22, the control section 28 causes the price calculated by the calculation section 27 to be determined. When determining that the "print" key is pressed, the control unit 28 outputs the identification information to the output unit 29. The specifying information is information indicating a specified price in the calculating unit 27. The output unit 29 outputs the determination information to the calculation unit 27.
As shown in fig. 4, the control device 3 includes a control unit 30 and a touch panel display 31. The control unit 30 controls display of the touch panel display 31. The control device 3 may be a tablet terminal or the like in which the control unit 30 and the touch panel display 31 are integrated, or may be a computer.
The control Unit 30 is a part that controls various operations in the control device 3, and includes a CPU (central processing Unit), a ROM (Read Only Memory), a RAM (random access Memory), and the like. The control unit 30 includes an input unit 32, a determination unit (first acquisition unit, second acquisition unit) 33, a storage unit 34, a control unit 35, and an output unit 36.
The input unit 32 inputs image data output from the camera 5. The input unit 32 outputs the input image data to the determination unit 33. The input unit 32 inputs the learned model (discrimination model, acquisition model) transmitted from the server 200 and stores the learned model in the storage unit 34.
The determination unit 33 determines the type of the product S placed on the weighing device 2 based on the target image (target image data) captured by the camera 5. The determination unit 33 detects that the product S is placed on the weighing table 21a of the weighing device 2 based on the target image data output from the input unit 32. Specifically, the determination unit 33 detects that the product S is placed on the basis of a difference (background difference) between the target image data output from the input unit 32 and a base image (background image) stored in advance. Specifically, the determination unit 33 determines that the product S is placed when the degree of change in the target image data from the base image is equal to or greater than the threshold value.
When determining that the product S is placed, the determination unit 33 determines whether the placement of the product S is stable. That is, the determination unit 33 determines whether or not the position of the product S is specified. The determination unit 33 determines whether or not the placement of the commodity S is stable by, for example, an inter-frame difference method of consecutive target image data. Specifically, the determination unit 33 determines that the placement of the product S is stable when the difference between frames (for example, the number of pixels in which a change in pixel value equal to or greater than a certain value occurs) in the continuous target image data is equal to or less than a threshold value. When determining that the placement of the product S is stable, the determination unit 33 determines the product S from the image in the target image data determined to be stable.
In the present embodiment, the determination unit 33 determines the type of the product S using the acquisition model and the determination model. Hereinafter, a description will be given of a product discrimination method in which the discrimination unit 33 discriminates the type of the product S. As shown in fig. 5, the discrimination unit 33 performs the first acquisition step S01, the second acquisition step S02, and the discrimination step S03 as the commodity discrimination method.
The determination unit 33 divides the target image data into a plurality of regions, and acquires a plurality of divided target images (second divided image (first acquisition step S01: fig. 5). as shown in fig. 6, the determination unit 33 divides the target image data G1 into a plurality of regions a. specifically, the determination unit 33 divides the target image data G1. such that the regions a of the plurality of divided target images are rectangular and all have the same size, the determination unit 33 divides the target image data G1. such that one divided target image overlaps with at least a part of the other divided target images, and the determination unit 33 sets the region a of the divided target image data G1 such that the other region a moves by a predetermined amount in the X direction (first direction) or the Y direction (second direction orthogonal to the first direction), which is the arrangement direction of the pixels of the target image data G1, relative to one region a, and the predetermined amount is a natural number obtained by dividing the width of the region a in the movement direction by 2 or more The latter widths (1/2, 1/3, etc.) correspond to the amount. The natural number of 2 or more is a value of the number of pixels in the moving direction of the region a in the divisible target image data G1.
The determination unit 33 divides the object image data G1 to acquire divided object images G11, G12, G13, G14, and the like as shown in fig. 7A, 7B, 7C, and 7D. The division target image G11 is an image in which only the background B is captured, i.e., the product S is not captured. The segmentation target image G12 is an image in which the container P and the background B are captured. The segmentation target image G13 is an image in which the product S, the container P, and the background B are captured. The division target image G14 is an image in which only the product S is captured.
The determination unit 33 acquires a plurality of divided object images satisfying a predetermined condition relating to the shot size of the product S from among the plurality of divided object images (second acquisition step S02: fig. 5). Specifically, the determination unit 33 acquires the divided object image in which the captured amount of the product S other than the product S is equal to or less than the threshold value. In other words, the determination unit 33 acquires the division target image in which the shot amount of the product S is larger than the threshold value. The determination unit 33 acquires the division target image in which the captured amount of the product S other than the product S is equal to or less than the threshold value by the acquisition model. The acquisition model includes a neural network NW 1. The acquisition model is the same model as the extraction model described later.
As shown in fig. 8, the neural network NW1 of the acquisition model is composed of, for example, a first layer as an input layer, a second layer, a third layer, and a fourth layer as intermediate layers (hidden layers), and a fifth layer as an output layer. The first layer directly outputs an input value x (x0, x1, x2, … … xp) having p parameters as elements to the second layer. The second, third and fourth layers convert the total input into output by activating functions and pass the output to the next layer. The fifth layer also converts the total input into an output by means of an activation function, the output being the output value y-y of a neural network with a parameter as an element0。
In the present embodiment, the neural network NW1 inputs the pixel values of the pixels in the division target images G11 to G14, and outputs information indicating the processing result. The input layer of the neural network NW1 is provided with neurons corresponding to the number of pixels of the segmentation target images G11 to G14. The output layer of the neural network NW1 is provided with neurons for outputting information related to the processing result. The segmentation target image in which the capture amount other than the commodity S is equal to or less than the threshold value can be acquired from the output value (non-commodity likelihood) of the neuron element of the output layer. The output value of the neuron element is, for example, a value of 0 to 1. In this case, the larger the value of the neuron (the closer to 1), the lower the possibility that the segmented target image is one in which the captured amount of the image other than the product S is equal to or less than the threshold value, and the smaller the value of the neuron (the closer to 0), the higher the possibility that the segmented target image is one in which the captured amount of the image other than the product S is equal to or less than the threshold value. That is, a high neuron value indicates that the proportion of the background G or the like in the image to be segmented is large, and a low neuron value indicates that the proportion of the product S in the image to be segmented is large. The determination unit 33 acquires the neuron value output from the neural network NW1, and acquires a segmentation target image in which the neuron value is equal to or less than a threshold value.
The determination unit 33 determines the type of the product S using the determination model for the plurality of divided images (determination step S03: fig. 5). The discriminant model includes a neural network NW 2.
As shown in fig. 9, the neural network NW2 is configured from, for example, a first layer as an input layer, a second layer, a third layer, and a fourth layer as intermediate layers (hidden layers), and a fifth layer as an output layer. The first layer directly outputs an input value x (x0, x1, x2, … … xp) having p parameters as elements to the second layer. The second, third and fourth layers convert the total input into output by activating functions and pass the output to the next layer. The fifth layer also converts the total input into an output by an activation function, which is the output value y (y) of the neural network NW2 with q parameters as elements0、y1、……、yq)。
In the present embodiment, the neural network NW2 inputs the pixel value of each pixel of each division target image, and outputs information indicating the result of discrimination of the product S for each division target image. The input layer of the neural network NW2 is provided with neurons in an amount corresponding to the number of pixels of an image. The output layer of the neural network NW2 is provided with neurons for outputting information related to the discrimination result of the product S. The type of the product S can be discriminated from the output values (product likelihood) of the neurons of the output layer. The output value y corresponds to the product likelihood of the product S. For example, the output value y1 and the product S1Corresponding to the commodity likelihood, and outputting the value yiWith the commodity SiCorresponds to the product likelihood of (1). Neuron and its useThe output value of (b) is, for example, a value of 0 to 1. For example, at the output value y1When the value of (A) is "0.8", the product S1The product likelihood of (2) is "0.8", and the output value y is2When the value of (A) is "0.2", the product S2The commodity likelihood of (2) is "0.2". In this case, the larger the value of the neuron element (the closer to 1), the higher the possibility of indicating that the product S is the target image data G1, and the smaller the value of the neuron element (the closer to 0), the lower the possibility of indicating that the product S is the target image data G1. That is, when the neuron value is large, the probability of indicating that the product S is high, and when the neuron value is small, the probability of indicating that the product S is low.
The determination unit 33 inputs the segmentation target image to the determination model. The determination unit 33 acquires a determination result including an output value output from the neural network NW2 for each of the divided target images, based on the input of the divided target image to the neural network NW2 of the determination model. The discrimination result includes all kinds of commodities registered in the commodity master.
The determination unit 33 sorts the candidate products based on the determination result. Specifically, the determination unit 33 performs weighting according to the magnitude of the neuron value of each divided target image, and sorts the products according to a majority determination method of the weight given to the determination result. The discrimination unit 33 generates discrimination information in which the product numbers are associated with the ranks for all types of products. The determination unit 33 outputs the image information and the determination information of the image data used for the determination process to the control unit 35.
After the image information and the discrimination information are output from the discrimination unit 33, the control unit 35 causes the touch panel display 31 to display the image information and the discrimination information. Control unit 35 controls display on touch-panel display 31 based on an input received by touch-panel display 31. Specifically, the control unit 35 displays an image of the product S based on the image information on one screen displayed on the touch panel display 31. The control unit 35 displays the product name of the highest ranked product among the product candidates in the discrimination information on one screen. The control unit 35 outputs number information indicating the product number to the output unit 36 based on the determination information or the input received by the touch panel display 31. The output unit 36 outputs the number information to the measuring device 2.
The storage unit 34 stores a product master. The product master file includes the same contents as those of the product master file stored in the storage unit 26 of the weighing device 2. The storage unit 34 stores the learned model.
As shown in fig. 1, the server 200 includes a communication unit 201 and a learned model generation unit (acquisition unit, division unit, extraction unit, generation unit) 202. The server 200 is a device that generates a learned model by machine learning. The server 200 is composed of a cpu (central Processing unit), a rom (read Only memory), a ram (random Access memory), and the like.
The communication section 201 communicates with the meter 100. The communication unit 201 receives the image data transmitted from the meter 100 and outputs the image data to the learned model generation unit 202. The communication unit 201 transmits the learned model output from the learned model generation unit 202 to the meter 100.
The learned model generation unit 202 acquires learning data used for machine learning, and performs machine learning using the acquired learning data to generate a learned model. In the present embodiment, the learning data is a teaching image. The teaching image is, for example, image data acquired by the camera 5 of the meter 100.
The learned model generation unit 202 generates an extraction model and a discrimination model. First, a method of generating an extraction model will be described. The extraction model is the same model as the acquisition model described above. The extraction model is generated by machine learning based on an image not containing a commodity. A method of generating the extraction model by the learned model generation unit 202 will be described below. As shown in fig. 10, the learned model generation unit 202 performs the first step S11, the second step S12, and the third step S13 as a method of generating an extraction model.
The learned model generation unit 202 acquires a non-product image including no product as a teaching image (first step S11: fig. 10). The non-commodity image may include a background B and a container P. The learned model generation unit 202 divides the non-commodity image into a plurality of regions, and acquires a plurality of non-commodity divided images (second step S12: fig. 10). As shown in fig. 11, the learned model generation unit 202 divides the non-product image data G2 into a plurality of regions a 1. Specifically, the learned model generation unit 202 divides the non-commodity image data G2 so that the areas a1 of the plurality of non-commodity divided images are rectangular and all have the same size. The learned model generation unit 202 divides the non-product image data G2 so that one non-product divided image overlaps at least a part of the other non-product divided images. Specifically, the learned model generation unit 202 sets the region a1 that divides the non-commodity image data G2 such that the other region a1 is shifted by a predetermined amount in the X direction or the Y direction, which is the arrangement direction of the pixels of the non-commodity image data G2, with respect to one region a 1. The predetermined amount is an amount corresponding to a width obtained by dividing the width of the area a1 in the moving direction by a natural number equal to or greater than 2. The natural number of 2 or more is a value that can divide the number of pixels in the moving direction of the area a in the non-product image data G2.
The learned model generation unit 202 performs machine learning from the plurality of non-product segmented images thus obtained to generate an extraction model (third step S13: fig. 10). The machine learning itself can be performed using known machine learning algorithms. The extraction model includes a neural network NW1 (refer to fig. 8).
The learned model generation unit 202 generates a discrimination model using the extraction model. A method of generating the discrimination model by the learned model generation unit 202 (learned model generation method) will be described below. As shown in fig. 12, the learned model generator 202 performs the acquisition step S21, the segmentation step S22, the extraction step S23, and the generation step S24 as a method of generating the discriminant model.
The learned model generation unit 202 acquires the product image and the product information on the type of the product (acquisition step S21: fig. 12). The learned model generation unit 202 acquires, for example, product image data acquired by the camera 5 and product information indicating the type of a product (for example, salad) included in the product image data. The product information corresponds to information included in the product master, and includes a product number, a product name, and the like. The product image may be a sample image or the like in addition to the image data acquired by the camera 5.
The learned model generation unit 202 divides the product image data into a plurality of regions and acquires a plurality of divided images (first divided images) (dividing step S22: fig. 12). As shown in fig. 13, the learned model generation unit 202 divides the product image data G3 into a plurality of regions a 2. Specifically, the learned model generation unit 202 divides the product image data G3 so that the regions a2 of the plurality of divided images are rectangular and all have the same size. The learned model generation unit 202 divides the product image data G3 so that the region a2 has the same shape and the same size as the region a 1. The learned model generation unit 202 divides the product image data G3 so that one divided image overlaps at least a part of the other divided images. Specifically, the learned model generation unit 202 sets the region a2 of the divided product image data G3 such that the other region a2 is shifted by a predetermined amount in the X direction or the Y direction, which is the arrangement direction of the pixels of the product image data G3, with respect to the one region a 2. The predetermined amount is an amount corresponding to a width obtained by dividing the width of the area a in the moving direction by a natural number of 2 or more. The natural number of 2 or more is a value that can divide the number of pixels in the moving direction of the area a in the product image data G3.
The learned model generation unit 202 divides the product image data G3, and acquires the divided images G31, G32, G33, G34, and the like as shown in fig. 14A, 14B, 14C, and 14D. The divided image G31 is an image in which the product S or the like is not captured, that is, only the background B is captured. The divided image G32 is an image in which the container P and the background B are captured. The divided image G33 is an image in which the product S, the container P, and the background B are captured. The divided image G34 is an image in which only the product S is captured.
The learned model generation unit 202 acquires a plurality of divided images satisfying a predetermined condition relating to the shot size of the product S from among the plurality of divided images (extraction step: S23). Specifically, the learned model generation unit 202 acquires a divided image in which the shot amount of the product S other than the product S is equal to or less than a threshold value. In other words, the learned model generation unit 202 acquires the divided images in which the shot size of the product S is larger than the threshold value. The learned model generation unit 202 acquires a divided image in which the shot amount of the product S other than the product S is equal to or less than a threshold value by using the extraction model. The learned model generation unit 202 acquires a neuron value output from the neural network from which the model is extracted, and acquires a segmented image in which the neuron value is equal to or less than a threshold value. The learned model generation unit 202 associates the acquired divided image with product information (the type of product (product name, product number, etc.)). In other words, the learned model generation unit 202 marks the acquired divided images with product information.
The learned model generation unit 202 associates the extracted plurality of divided images with the product information, performs machine learning, and generates a discrimination model (generation step S24: fig. 12). The machine learning itself may be performed using known machine learning algorithms. As shown in fig. 9, the learned model generation unit 202 generates a discrimination model including the neural network NW 2. The discriminant model may also comprise a convolutional neural network. Furthermore, the discriminant model may include a plurality of hierarchical (for example, 8 or more) neural networks. That is, the discriminant model may be generated by deep learning. The learned model generation unit 202 outputs the acquisition model (extraction model) and the discrimination model to the communication unit 201 at a predetermined timing.
As described above, the weighing system 1 according to the present embodiment includes the server 200 that generates the discrimination model for discriminating the type of the product from the product image including the product, and the meter 100 that discriminates the type of the product from the target image including the product. Server 200 includes learning model generation unit 202. The learned model generation unit 202 acquires a product image and product information related to the type of product (acquisition step), and divides the product image into a plurality of regions to acquire a plurality of divided images (division step). The learned model generation unit 202 extracts a plurality of divided images satisfying a predetermined condition on the captured amount of the product from the plurality of divided images (extraction step). The learned model generation unit 202 associates the extracted plurality of divided images with the product information and performs machine learning to generate a discrimination model (generation step).
As described above, in the weighing system 1 according to the present embodiment, the learned model generation unit 202 of the server 200 extracts a plurality of divided images satisfying a predetermined condition relating to the shot amount of the product from among the plurality of divided images. The divided images include an image in which only the product is captured, an image in which the product and the non-product (container, background, etc.) are captured, and an image in which only the non-product is captured. The learned model generation unit 202 extracts a segmented image satisfying a predetermined condition from the segmented images, and performs machine learning based on the extracted segmented image. Thus, the learned model generation unit 202 can perform machine learning from an appropriate teaching image. Therefore, the server 200 can generate a discrimination model for improving the discrimination accuracy of the product.
In the weighing system 1 according to the present embodiment, the control device 3 includes the determination unit 33. The determination unit 33 divides the target image into a plurality of regions, acquires a plurality of divided target images (first acquisition step), and acquires a plurality of divided target images satisfying a predetermined condition relating to the shot amount of the product from the plurality of divided target images (second acquisition step). The determination unit 33 acquires the processing results processed by the determination model for the plurality of acquired images to be divided, and determines the type of the product based on the processing results (determination step).
As described above, in the weighing system 1 according to the present embodiment, the determination unit 33 of the control device 3 extracts a plurality of divided object images satisfying a predetermined condition relating to the shot amount of the product from among the plurality of divided object images. The determination unit 33 acquires a division target image satisfying a predetermined condition from among the division target images, and processes the acquired division target image using a determination model. Thus, the meter 100 can perform processing based on an appropriate segmentation target image. Therefore, in the weighing system 1, the accuracy of discriminating a product is improved.
In the measurement system 1 according to the present embodiment, the learning-completed model generation unit 202 of the server 200 extracts a divided image in which the shot amount other than the commodity is equal to or less than a threshold value. In other words, the learned model generation unit 202 extracts the divided images in which the shot size of the product is larger than the threshold value. In this method, a segmented image having a high commodity occupancy rate is extracted. That is, in the weighing system 1, the divided images of the container and the background other than the commodity taken in can be excluded. Therefore, the measurement system 1 can perform machine learning using the divided image of the captured product. Therefore, the weighing system 1 can generate a discrimination model for improving the discrimination accuracy of the product.
In the measurement system 1 according to the present embodiment, the learned model generation unit 202 acquires a neuron value indicating a probability that the segmented image does not include a product for each of the plurality of segmented images using an extraction model generated by machine learning based on an image not including a product, and extracts the segmented image in which the neuron value is equal to or less than a threshold value. In this method, only the divided image in which the product is captured can be appropriately extracted.
In the weighing system 1 according to the present embodiment, the learned model generation unit 202 acquires a non-commodity image including no commodity, divides the non-commodity image into a plurality of regions, and acquires a plurality of non-commodity divided images. The learned model generation unit 202 performs machine learning from the plurality of non-product segmented images to generate an extraction model. Thus, the metrology system 1 can appropriately generate an extraction model.
In the weighing system 1 according to the present embodiment, the learned model generation unit 202 divides the product image so that each of the plurality of divided images has a rectangular region and all of the regions have the same size. The learned model generation unit 202 divides the non-product image so that each of the plurality of non-product divided images has the same shape and the same size as the divided image. In this method, since the divided image and the non-product divided image have the same shape and the same size, it is not necessary to perform conversion processing for converting the shape and size of the image. Therefore, in the weighing system 1, the processing load is reduced.
In the weighing system 1 according to the present embodiment, the learned model generation unit 202 divides the product image so that one divided image overlaps at least a part of the other divided images. In this method, when dividing the product image, even when one article included in the product is cut in one divided image, the article may be completely included in the other divided images. Therefore, in the weighing system 1, since machine learning can be performed based on an appropriate teaching image, a discrimination model for improving the discrimination accuracy of the commodity can be generated.
In the weighing system 1 according to the present embodiment, the learned model generation unit 202 sets the regions for dividing the product image so that one region is shifted by a predetermined amount in the X direction or the Y direction, which is the arrangement direction of the pixels in the product image, with respect to the other region. In this method, by moving the region by a predetermined amount, the product image can be divided so that the number of times each pixel is included in each region arranged in the moving direction is equal.
In the weighing system 1 according to the present embodiment, the learned model generation unit 202 divides the product image so that each of the plurality of divided images has a rectangular region and all of the regions have the same size. In this method, since the divided images are all of the same shape and the same size, it is not necessary to perform conversion processing for converting the shape and size of the image. Therefore, in the weighing system 1, the processing load is reduced.
In the weighing system 1 according to the present embodiment, the determination unit 33 acquires a division target image in which the shot amount other than the commodity is equal to or less than a threshold value. In this method, an image with a high commodity occupancy rate is acquired. In other words, the image of the container or the background other than the commodity can be excluded. Therefore, in the weighing system 1, the accuracy of discriminating a product is improved.
In the weighing system 1 according to the present embodiment, the determination unit 33 acquires a neuron value indicating a probability that the segmented target image does not contain a product for each of the plurality of segmented target images using an acquisition model generated by machine learning based on an image not containing a product, and acquires the segmented target image in which the neuron value is equal to or less than a threshold value. In this method, only the image to be segmented in which the product is captured can be appropriately acquired.
In the weighing system 1 according to the present embodiment, the determination unit 33 performs processing on each of the plurality of divided images by the determination model to obtain a plurality of processing results, and determines the type of the product from the plurality of processing results. In this method, since each of the divided object images is processed by the discrimination model to obtain a plurality of processing results, the discrimination accuracy of the product can be further improved.
In the weighing system 1 according to the present embodiment, the determination unit 33 determines the type of the product by weighting the processing result according to the degree of the neuron value indicating the probability that the product is of one type and determining the majority of the weights given to the processing result. In this method, the product can be discriminated with higher accuracy.
In the weighing system 1 according to the present embodiment, the determination unit 33 divides the target image so that the respective regions of the plurality of divided target images are rectangular and all have the same size. In this method, since all the images to be segmented have the same shape and the same size, it is not necessary to perform conversion processing for converting the shape and size of the image. Therefore, in the weighing system 1, the processing load is reduced.
While the embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the scope of the present invention.
In the above embodiment, the case where the determination unit 33 divides the target image data G1 into the plurality of areas a has been described as an example. However, the determination unit 33 may compare the product image with the base image not including the product, cut out a product region smaller than the product image including the entire product from the product image (cutting-out step), divide the product region into a plurality of regions, and acquire a plurality of divided images. Specifically, as shown in fig. 15, the determination unit 33 compares the object image data G1, which is a product image, with the base image, and extracts a product area SA, which includes the product S at the minimum, from the object image data G1. The base image is an image in which only the background B is captured. The determination unit 33 divides the product area SA into a plurality of areas and acquires a plurality of divided images. In this method, since the target image data G1 is divided using the product area SA excluding the product S, there is no need to divide an unnecessary area. Thus, reduction in processing load is achieved.
In the above embodiment, the case where the predetermined amount by which the determination unit 33 moves the other region a with respect to one region a in order to set the region a of the division target image data G1 corresponds to the width (1/2, 1/3, etc.) obtained by dividing the width of the region a in the moving direction by a natural number of 2 or more has been described as an example. However, the determination unit 33 may set the predetermined amount to one pixel. That is, the determination unit 33 may set the region a of the divided target image data G1 such that the other region a is shifted by one pixel in the X direction or the Y direction, which is the arrangement direction of the pixels in the target image data G1, with respect to the one region a. The same applies to the case where the learned model generation unit 202 sets the region a1 of the divided non-product image data G2 and the case where the region a2 of the divided product image data G3 is set.
In the above embodiment, an example of a mode in which the neural network NW2 of the discrimination model inputs the pixel value of each pixel of each division target image and outputs information indicating the discrimination result of the product S for each division target image has been described. However, the discrimination model may be configured to include a neural network NW2 that receives the pixel values of the pixels of each of the divided target images and outputs information indicating the discrimination result of the product S for each pixel.
In the above embodiment, an example of a mode in which the discrimination unit 33 performs weighting according to the magnitude of the neuron value of each divided target image and determines the product ranking according to the number of the weighted products assigned to the discrimination result has been described. However, the sorting (discrimination) of the commodities in the discrimination unit 33 may be performed by another method. For example, the determination unit 33 may sort the products using only neuron elements equal to or larger than a threshold value, or may sort the products by preferentially using the determination result of the divided target image in the central region of the target image data. The determination unit 33 may not use neuron values equal to or lower than a threshold value (product likelihood equal to or lower than the threshold value) among neuron values of each divided target image.
In addition, discriminationFor example, the unit 33 may be configured to divide the neuron value (output value) y of one image to be divided2(product S)2The product likelihood of (2) is "0.9", and the neuron value y of the other segmentation object image2Is "0.4", and the neuron value y of the above one divided object image5(product S)5Product likelihood of) is "0.7", and the product S2When the variance of the neuron value is large, the product S is reduced2The weight of (c). That is, the determination unit 33 reduces the weight of one product when the neuron value corresponding to one product in one divided target image is different (deviated) from the neuron value corresponding to one product in another divided target image (the neuron values are dispersed), and the neuron value corresponding to another product in one divided target image is larger than a predetermined value.
In the above embodiment, the description has been given taking as an example a mode in which the determination unit 33 acquires a divided object image in which the shot size of the non-commodity is equal to or less than the threshold value from the plurality of divided object images by the acquisition model. However, the determination unit 33 may extract a division target image in which the inclusion ratio of pixels is equal to or less than a predetermined threshold value from the plurality of division target images based on the feature values of the pixels of the image not including the product. In this method, only the image to be segmented in which the product is captured can be appropriately extracted without using an acquisition model.
In the above embodiment, the description has been given taking as an example the form in which the determination unit 33 acquires the image to be divided in which the shot size of the non-commodity is equal to or smaller than the threshold value. However, the determination unit 33 may acquire the presence or absence of an article other than a commodity (container, background, or the like) in the outer edge portion (outer edge region) of the divided object image, and acquire the divided object image when the article other than the commodity is not captured. This enables extraction of a segmentation target image having a high commodity occupancy rate.
In the above embodiment, the description has been given taking an example in which one product S is included in the target image data and the discrimination unit 33 discriminates the type of the product S. However, the object image data may include a plurality of products. In this case, the determination unit 33 determines the type of each of the plurality of products. Specifically, the determination unit 33 extracts a contour line (edge) from a difference between the target image data and a base image stored in advance, and acquires a region including the product from the edge. In this case, even when there are 2 or more commodities due to a situation such as overlapping of a part of the commodities, the commodities may be acquired as one area. Therefore, the determination unit 33 obtains the region for each commodity (divides the region for each commodity) by comparing the area of the region with the area of the container, for example. As a method of acquiring the area of each product, other methods may be used. When a plurality of areas are acquired, the determination unit 33 determines the type of product for each area.
In addition, for example, when a plurality of commodities are contained in one tray or the like, the discrimination unit 33 discriminates the types of the commodities by the following processing. The determination unit 33 determines a plurality of processing results in one divided target image and a plurality of divided target images in the vicinity of the one divided target image, and sets the result as a result (product type) at the center coordinates of the region including the one divided target image and the plurality of divided target images (smoothes the processing result of each divided target image). The determination unit 33 performs the above-described processing on all the images to be segmented to obtain the result of the center coordinates of each region. The determination unit 33 determines the type of each of the plurality of products by grouping the same result (product) dense areas among the obtained results into one area. When a plurality of products are included, the meter 100 is not used as a product discrimination device for measuring and selling products as in the above-described embodiment. For example, when a plurality of food items are placed on a tray, the product identification device can identify the type of each of the plurality of food items or can inspect the contents thereof.
In the above embodiment, the description has been given by taking as an example a configuration in which the learned model generation unit 202 divides the product image so that the respective regions of the plurality of divided images are rectangular and all have the same size, and divides the non-product image so that each of the plurality of non-product divided images has the same shape and the same size as the divided image. However, the product image and the non-product-divided image may have different shapes and sizes. For example, the resolution of the segmented image may be higher than that of the non-commodity segmented image. In this case, when the shape is a rectangle, it is preferable that the aspect ratio (aspect ratio) is uniform. That is, the divided image and the non-product divided image are preferably formed in a similar shape. This eliminates the need for processing for converting the ratio of images, thereby reducing the processing load.
In the above embodiment, the description has been given taking as an example the form in which the learned model generating unit 202 acquires a divided image having an imaging amount of the non-commodity amount equal to or smaller than the threshold value from the plurality of divided images by using the extraction model. However, the learned model generation unit 202 may extract a divided image having a pixel inclusion ratio of a predetermined threshold value or less from the plurality of divided images based on the feature amount of the pixel of the image not including the product. In this method, only the segmented image in which the product is captured can be appropriately extracted without using an extraction model.
In the above embodiment, the description has been given taking as an example the form in which the learned model generation unit 202 acquires the divided image in which the shot size of the non-commodity is equal to or smaller than the threshold value. However, the learned model generation unit 202 may acquire the presence or absence of an article other than a commodity (container, background, or the like) in the outer edge portion (outer edge region) of the captured divided image, and acquire the divided image when the article other than the commodity is not captured. This enables extraction of a divided image having a high commodity occupancy rate.
In the above embodiment, a mode in which the learned model generation unit 202 acquires a plurality of divided images satisfying a predetermined condition relating to the captured amount of the product S from among the plurality of divided images, and performs machine learning by associating the extracted plurality of divided images with product information to generate a discrimination model has been described as an example. However, the learned model generation unit 202 may generate the discrimination model by extracting one divided image in which the shot amount of the non-commodity is equal to or less than the threshold value and another divided image in which the shot amount of the non-commodity is greater than the threshold value from the plurality of divided images, associating the commodity information with the one divided image, associating the non-commodity information indicating the non-commodity with the other divided image, and performing machine learning from the extracted one divided image and the other divided image. The determination unit 33 extracts, for example, another divided image having a larger shot size than the threshold (the shot size of the product is equal to or smaller than the threshold) using the extraction model.
Specifically, the learned model generation unit 202 acquires a neuron value output from the neural network that extracts the model, and extracts one divided image in which the shot amount of the product S other than the product S is equal to or less than a threshold value and other divided images in which the shot amount of the product S other than the product S is greater than the threshold value from the neuron value. That is, the learned model generation unit 202 classifies one divided image in which the shot amount of the product S other than the product S is equal to or less than the threshold value and another divided image in which the shot amount of the product S other than the product S is greater than the threshold value, and extracts one divided image and the other divided image. The learned model generation unit 202 marks commodity information (commodity name) on one divided image, and marks non-commodity information indicating an article other than a commodity (background, container, or the like) on the other divided image. The learned model generation unit 202 performs machine learning from one divided image and the other divided images to generate a discrimination model. The machine learning itself may be performed using known machine learning algorithms.
In the above embodiment, the description has been given taking as an example the form in which the determination unit 33 acquires a plurality of divided object images satisfying a predetermined condition relating to the shot amount of the product S from among the plurality of divided object images, inputs the divided object images into the determination model, acquires the processing result of the processing by the determination model, and determines the type of the product S from the processing result. However, the determination unit 33 may divide the target image data into a plurality of regions (divided image acquisition step), and input the plurality of divided target images to the determination model. That is, the determination unit 33 inputs all the divided object images including the background B and the like other than the product S into the determination model. The determination unit 33 acquires a determination result including an output value output from the neural network NW2 for each of the divided target images, based on the input of the divided target image to the neural network NW2 of the determination model.
The determination unit 33 acquires, from the processing result of the processing by the determination model, a product likelihood indicating a probability of being a product and a non-product likelihood indicating a probability of being a product other than the product. As a processing result, for example, for each divided image, the ratio of 1: (0.02; 0.20, 0.31, 089, … …), 2: (0.89; 0.12, 0.04, 0.23, … …), and the like. In the processing result, a (non-commodity likelihood; commodity likelihood) is shown. That is, as a result of the processing of the divided image of "1", the non-product likelihood is "0.02", and the product likelihoods are "0.20, 0.31, 0.89, and … …". In this case, the result of processing the divided image of "1" indicates that the product has a high likelihood of having a product likelihood of "0.89". In the processing result of the divided image of "2", the non-product likelihood is "0.89", and the product likelihood is "0.12, 0.04, 0.23, … …". In this case, the processing result of the divided image of "2" indicates that it is highly likely to be a background. The discrimination unit 33 extracts the product likelihood of the two kinds of likelihoods, and discriminates the kind of product from the product likelihood. Specifically, first, the determination unit 33 removes the divided image having the non-product likelihood of not less than the threshold (not used for determination). Next, the determination unit 33 extracts the product likelihood having the largest value for each of the divided images among the processing results of the remaining divided images. The determination unit 33 determines the type of the product based on the manner of determining the number of the products indicated by the extracted product likelihood.
In the above embodiment, the description has been given taking an example in which the meter 100 includes the control device 3. However, the meter 100 may not include the control device 3. In this case, the measuring device 2 may have the function of the control device 3. Alternatively, one device having the functions of the measuring device 2 and the control device 3 may be provided.
In the above embodiment, the control device 3 of the meter 100 is provided with the determination unit 33, and the type of the product is determined by the control device 3. However, the control device 3 may not include the determination unit 33. For example, the server 200 may determine the type of the product. In this case, the server 200 transmits the determination result to the control device 3.
In the above embodiment, the description has been given taking an example in which the metering system 1 includes the meter 100 and the server 200. However, the server 200 may not be provided. In this case, the meter 100 may include a learned model generation unit. Alternatively, the meter 100 may acquire a learning completion model generated by another device (computer) and store the learning completion model in the storage unit.
In the above embodiment, the metering system 1 including the meter 100 and the server 200 is described. However, the present invention may be the server 200 only. That is, the present invention may be a device for generating a discrimination model for discriminating a type of a product from a product image including the product.
In the above embodiment, the description has been given taking as an example a configuration in which the touch panel display 31 of the control device 3 is disposed on the holding portion 12 of the casing 10. However, touch-panel display 31 may be disposed at a position other than holding portion 12. Preferably, touch-panel display 31 is disposed in the vicinity of measuring device 2.
In the above embodiment, the description has been given taking an example in which the meter 100 includes the display device 4. However, for example, when the customer operates the meter 100, the display device 4 may not be provided.
In the above embodiment, the description has been given taking as an example a configuration in which the measuring device 2 and the control device 3 are provided in the housing 10. However, the form of the meter 100 is not limited to this. The weighing device 2 (weighing table 21a) may be provided separately from the control device 3, and may be in the form of a so-called separation scale.
In the above embodiment, an example of measuring the weight of the product S stored in the container P is described. However, the product S may be fruit or vegetable not stored in the container P.

Claims (23)

1. A method for generating a learning completion model for discriminating a type of a product from a product image including the product, comprising:
an acquisition step of acquiring the commodity image and commodity information related to the kind of the commodity;
a segmentation step of segmenting the commodity image into a plurality of regions and acquiring a plurality of segmented images;
an extraction step of extracting a predetermined divided image from the plurality of divided images on the basis of a predetermined condition relating to an amount of capture of the product; and
a generation step of performing machine learning based on the plurality of segmented images extracted in the extraction step to generate the discrimination model.
2. The learned model generation method according to claim 1,
in the extracting step, a plurality of the divided images satisfying the predetermined condition are extracted from the plurality of the divided images,
in the generation step, the segmented image extracted in the extraction step is associated with the product information and machine-learned to generate the discrimination model.
3. The learned model generation method according to claim 2,
in the extracting step, the divided image in which the shot amount of the non-commodity is equal to or less than a threshold value is extracted.
4. The learned model generation method according to claim 2 or 3,
in the extracting step, a non-product likelihood indicating a probability of being the divided image not including the product is acquired for each of the plurality of divided images using an extraction model generated by machine learning based on an image not including the product, and the divided images in which the non-product likelihood is equal to or less than a threshold value are extracted.
5. The learned model generation method according to claim 4,
the generation of the extraction model comprises:
a first step of acquiring a non-commodity image not including the commodity;
a second step of dividing the non-commodity image into a plurality of regions to obtain a plurality of non-commodity divided images; and
and a third step of performing machine learning from the plurality of non-commodity segmented images to generate the extraction model.
6. The learned model generation method according to claim 5,
in the dividing step, the product image is divided so that the regions of the plurality of divided images are rectangular and all have the same size;
in the second step, the non-product image is divided so that the plurality of non-product divided images have the same shape and the same size as the divided image.
7. The learned model generation method according to claim 1,
in the extracting step, one of the divided images in which the shot amount of the non-product is equal to or less than a threshold value and the other of the divided images in which the shot amount of the non-product is greater than the threshold value are extracted from the plurality of divided images, the product information is associated with the one of the divided images, and the non-product information indicating that the non-product is not the product is associated with the other of the divided images,
in the generating step, the discrimination model is generated by performing machine learning from the one of the divided images extracted in the extracting step and the other of the divided images.
8. The learned model generation method according to any one of claims 1 to 7,
in the dividing step, the product image is divided so that one of the divided images overlaps at least a part of the other divided images.
9. The learned model generation method according to claim 8,
in the dividing step, the area of the product image is set to be divided such that the other area is moved by a predetermined amount in a first direction, which is an arrangement direction of pixels of the product image, or in a second direction orthogonal to the first direction, with respect to one area.
10. The learned model generation method according to any one of claims 1 to 5,
in the dividing step, the product image is divided so that the regions of the plurality of divided images are rectangular and all have the same size.
11. The learned model generation method according to any one of claims 1 to 10, comprising:
a cutting-out step of comparing the product image with a base image not including the product, and cutting out a product area including at least the entire product and smaller than the product image from the product image,
in the dividing step, the product region is divided into a plurality of regions, and a plurality of divided images are acquired.
12. A learning completion model generation device that is a determination model generation device that determines a type of a product from a product image including the product, the learning completion model generation device comprising:
an acquisition unit that acquires the product image and product information related to the type of the product;
a dividing unit that divides the product image into a plurality of regions and acquires a plurality of divided images;
an extraction unit that extracts the plurality of divided images from the plurality of divided images on the basis of a predetermined condition relating to an amount of capture of the product; and
and a generation unit configured to perform machine learning based on the plurality of divided images extracted by the extraction unit, and generate the discrimination model.
13. A commodity discrimination method for discriminating a type of a commodity from an object image including the commodity using a discrimination model generated by the learned model generation method according to any one of claims 1 to 11, comprising:
a first acquisition step of acquiring a plurality of divided object images by dividing the object image into a plurality of regions;
a second acquisition step of acquiring, from among the plurality of divided target images, a plurality of divided target images that satisfy a predetermined condition relating to an amount of capture of the product; and
a determination step of obtaining a processing result of processing by the determination model for the plurality of divided object images obtained in the second acquisition step, and determining the type of the product based on the processing result.
14. The product discrimination method according to claim 13,
in the second acquisition step, the division target image in which the shot size of the non-commodity is equal to or less than a threshold value is acquired.
15. The product discrimination method according to claim 13 or 14,
in the second acquisition step, a non-product likelihood indicating a probability of being the division target image not including the product is acquired for each of the plurality of division target images using an acquisition model generated by machine learning based on an image not including the product, and the division target images in which the non-product likelihood is equal to or less than a threshold value are acquired.
16. The product discrimination method according to any one of claims 13 to 15,
in the first acquisition step, the object image is segmented in such a manner that the regions of the respective plurality of segmented object images are rectangular and all have the same size.
17. A commodity discrimination method for discriminating a type of a commodity from an object image including the commodity using a discrimination model generated by the learned model generation method according to claim 7, comprising:
a divided image acquisition step of dividing the target image into a plurality of regions to acquire a plurality of divided target images; and
a determination step of obtaining, from the plurality of divided images obtained in the divided image obtaining step, a product likelihood indicating a probability of being the product and a non-product likelihood indicating a probability of being the product, in a processing result of processing by the determination model, and determining a type of the product based on the product likelihood.
18. The product discrimination method according to any one of claims 13 to 17,
in the discrimination step, the discrimination model is used to process each of the plurality of divided images to obtain a plurality of processing results, and the type of the product is discriminated from the plurality of processing results.
19. The product discrimination method according to any one of claims 13 to 18,
in the determining step, the processing result is weighted according to the degree of the product likelihood indicating the probability that the product is of one type, and the type of the product is determined according to the majority of the weights given to the processing result.
20. The product discrimination method according to any one of claims 13 to 19,
in the determining step, when a plurality of the products are included in the target image, the type of each of the plurality of products is determined based on the processing result of each of the regions including each of the plurality of products.
21. A product discrimination apparatus that discriminates a type of a product from an object image including the product using a discrimination model generated by the learned model generation method according to any one of claims 1 to 11, the apparatus comprising:
a first acquisition unit configured to acquire a plurality of divided target images by dividing the target image into a plurality of regions;
a second acquisition unit configured to acquire, from among the plurality of divided target images, a plurality of divided target images that satisfy a predetermined condition relating to an amount of capture of the product; and
and a determination unit configured to acquire a processing result of the processing by the determination model for the plurality of divided images acquired by the second acquisition unit, and determine the type of the product based on the processing result.
22. A commodity discrimination system includes: a generation device of a discrimination model for discriminating the type of a commodity from a commodity image containing the commodity; and a discrimination device for discriminating a type of the product from an object image including the product,
the generation device is provided with:
an acquisition unit that acquires the product image and product information related to the type of the product;
a dividing unit that divides the product image into a plurality of regions and acquires a plurality of first divided images;
an extraction unit that extracts a plurality of first divided images that satisfy a predetermined condition relating to an amount of capture of the product from the plurality of first divided images; and
a generation unit configured to perform machine learning by associating the plurality of first divided images extracted by the extraction unit with the product information to generate the discrimination model,
the discrimination device includes:
a first acquisition unit configured to acquire a plurality of second divided images by dividing the target image into a plurality of regions;
a second acquisition unit configured to acquire, from among the plurality of second divided images, a plurality of second divided images that satisfy a predetermined condition relating to an amount of capture of the product; and
and a determination unit configured to acquire a processing result of the processing by the determination model for the plurality of second divided images acquired by the second acquisition unit, and determine the type of the product based on the processing result.
23. A weighing apparatus that discriminates a type of a commodity from an object image including the commodity using a discrimination model generated by the learned model generation method according to any one of claims 1 to 11, and calculates a price of the commodity, the weighing apparatus comprising:
a measuring unit that measures the weight of the commodity;
an imaging unit that images the commodity;
a determination unit configured to acquire a processing result of the processing by the determination model with respect to the target image captured by the imaging unit, and determine the type of the product based on the processing result; and
and a calculation unit that calculates the price of the product based on the weight of the product measured by the measurement unit and the type of the product determined by the determination unit.
CN202010203532.0A 2019-03-28 2020-03-20 Learning completion model generation method and device, and commodity discrimination method and device Pending CN111753837A (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019-062949 2019-03-28
JP2019062949A JP7374453B2 (en) 2019-03-28 2019-03-28 Trained model generation method, trained model generation device, product discrimination method, product discrimination device, product discrimination system, and weighing device

Publications (1)

Publication Number Publication Date
CN111753837A true CN111753837A (en) 2020-10-09

Family

ID=70289523

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010203532.0A Pending CN111753837A (en) 2019-03-28 2020-03-20 Learning completion model generation method and device, and commodity discrimination method and device

Country Status (4)

Country Link
US (1) US11669948B2 (en)
EP (1) EP3726430A3 (en)
JP (1) JP7374453B2 (en)
CN (1) CN111753837A (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7374453B2 (en) * 2019-03-28 2023-11-07 株式会社イシダ Trained model generation method, trained model generation device, product discrimination method, product discrimination device, product discrimination system, and weighing device
DE102020111955A1 (en) * 2020-05-04 2021-11-04 Airbus Defence and Space GmbH Face recognition with neural networks
US11954932B2 (en) * 2020-10-16 2024-04-09 Bluebeam, Inc. Systems and methods for automatic detection of features on a sheet
US11720650B2 (en) * 2020-10-30 2023-08-08 Tiliter Pty Ltd. Methods and apparatus for training a classification model based on images of non-bagged produce or images of bagged produce generated by a generative model

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1912932A (en) * 2005-08-08 2007-02-14 株式会社东芝 Sheet identifying apparatus and sheet identifying method
US20130027424A1 (en) * 2011-07-26 2013-01-31 Sony Corporation Information processing apparatus, information processing method, and program
US20160026900A1 (en) * 2013-04-26 2016-01-28 Olympus Corporation Image processing device, information storage device, and image processing method
CN105373938A (en) * 2014-08-27 2016-03-02 阿里巴巴集团控股有限公司 Method for identifying commodity in video image and displaying information, device and system
CN108303170A (en) * 2018-02-01 2018-07-20 京东方科技集团股份有限公司 The weighing method and smart electronics that smart electronics claim claim

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0942939A (en) * 1995-07-28 1997-02-14 Tec Corp Article recognition device
JP2002329188A (en) * 2001-04-27 2002-11-15 Fuji Xerox Co Ltd Data analyzer
JP2013117861A (en) * 2011-12-02 2013-06-13 Canon Inc Learning device, learning method and program
JP6177068B2 (en) 2013-09-24 2017-08-09 株式会社ブレイン Store system
JP6495764B2 (en) * 2015-06-29 2019-04-03 株式会社東芝 Image recognition apparatus, image recognition program, and image recognition method
JP6702716B2 (en) * 2015-12-21 2020-06-03 キヤノン株式会社 Image processing device, image processing method, and program
KR102433391B1 (en) * 2016-01-14 2022-08-17 한국전자통신연구원 Apparatus and Method for Food Search Service
JP6751882B2 (en) * 2016-03-31 2020-09-09 パナソニックＩｐマネジメント株式会社 Product monitoring device, product monitoring system and product monitoring method
CN107818554B (en) * 2016-09-12 2023-04-21 索尼公司 Information processing apparatus and information processing method
JP2018169752A (en) * 2017-03-29 2018-11-01 パナソニックＩｐマネジメント株式会社 Product recognition system, learned model, and product recognition method
WO2018179361A1 (en) * 2017-03-31 2018-10-04 日本電気株式会社 Image-processing device, image-processing method, and recording medium
US11430154B2 (en) * 2017-03-31 2022-08-30 Nec Corporation Classification of change related to display rack
JP7010641B2 (en) * 2017-09-27 2022-01-26 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Abnormality diagnosis method and abnormality diagnosis device
US10643320B2 (en) * 2017-11-15 2020-05-05 Toyota Research Institute, Inc. Adversarial learning of photorealistic post-processing of simulation with privileged information
JP7374453B2 (en) * 2019-03-28 2023-11-07 株式会社イシダ Trained model generation method, trained model generation device, product discrimination method, product discrimination device, product discrimination system, and weighing device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1912932A (en) * 2005-08-08 2007-02-14 株式会社东芝 Sheet identifying apparatus and sheet identifying method
US20130027424A1 (en) * 2011-07-26 2013-01-31 Sony Corporation Information processing apparatus, information processing method, and program
US20160026900A1 (en) * 2013-04-26 2016-01-28 Olympus Corporation Image processing device, information storage device, and image processing method
CN105373938A (en) * 2014-08-27 2016-03-02 阿里巴巴集团控股有限公司 Method for identifying commodity in video image and displaying information, device and system
CN108303170A (en) * 2018-02-01 2018-07-20 京东方科技集团股份有限公司 The weighing method and smart electronics that smart electronics claim claim

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
YE HE等: "Food image analysis: Segmentation, identification and weight estimation", 2013 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO (ICME), pages 1 - 6 *

Also Published As

Publication number Publication date
JP7374453B2 (en) 2023-11-07
EP3726430A3 (en) 2021-01-27
JP2020161077A (en) 2020-10-01
EP3726430A2 (en) 2020-10-21
US20200311903A1 (en) 2020-10-01
US11669948B2 (en) 2023-06-06

Similar Documents

Publication Publication Date Title
CN111753837A (en) Learning completion model generation method and device, and commodity discrimination method and device
JP6709862B2 (en) Accounting method and equipment by convolutional neural network image recognition technology
CN108922026B (en) Replenishment management method and device for vending machine and user terminal
US7416119B1 (en) Methods and apparatus for automatically determining and deducting weight of containers for products
EP2533217A1 (en) Commodity search device and commodity information processing device
JP2009163331A (en) Merchandise sales data processor and computer program
EP3696518B1 (en) Weighing machine
CN105608808A (en) Information processing apparatus, store system and method
CN110909698A (en) Electronic scale recognition result output method, system, device and readable storage medium
EP3901818A1 (en) Method and system for analyzing commodities on supermarket shelf
CN107533783A (en) Product calling mechanism, program and control method
JP2009163330A (en) Merchandise sales data processor and computer program
CN112232334B (en) Intelligent commodity selling identification and detection method
JP2021128388A (en) Data generation device, learned model generation device, weighing machine and data generation method
JP2009163332A (en) Merchandise sales data processor and computer program
DE112019006422T5 (en) AUTOMATED POS SYSTEMS AND PROCEDURES
US11599737B1 (en) System for generating tags
US20240029017A1 (en) Information processing device, information processing method, and recording medium
CN109584481A (en) Meal platform and system are sold in the self-service weighing of one kind
CN110868476A (en) Push information determining method and system
CN113256919B (en) Commodity candidate presentation system and settlement processing system
WO2022260148A1 (en) Recognizing device, terminal device, recognizer constructing device, recognizer correcting device, construction method, and correction method
JP7398779B2 (en) weighing machine
WO2023248676A1 (en) Inference method and inference device
WO2021095152A1 (en) Information processing device, information processing method, and program

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination