US20190362182A1 - Article recognition device - Google Patents
Article recognition device Download PDFInfo
- Publication number
- US20190362182A1 US20190362182A1 US16/416,702 US201916416702A US2019362182A1 US 20190362182 A1 US20190362182 A1 US 20190362182A1 US 201916416702 A US201916416702 A US 201916416702A US 2019362182 A1 US2019362182 A1 US 2019362182A1
- Authority
- US
- United States
- Prior art keywords
- distance
- article
- distance sensor
- image
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G06K9/6215—
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
- G07G1/00—Cash registers
- G07G1/0036—Checkout procedures
- G07G1/0045—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader
- G07G1/0054—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader with control of supplementary check-parameters, e.g. weight or number of articles
- G07G1/0063—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader with control of supplementary check-parameters, e.g. weight or number of articles with means for detecting the geometric dimensions of the article of which the code is read, such as its size or height, for the verification of the registration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q20/00—Payment architectures, schemes or protocols
- G06Q20/08—Payment architectures
- G06Q20/20—Point-of-sale [POS] network systems
- G06Q20/203—Inventory monitoring
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q20/00—Payment architectures, schemes or protocols
- G06Q20/08—Payment architectures
- G06Q20/20—Point-of-sale [POS] network systems
- G06Q20/208—Input by product or record sensing, e.g. weighing or scanner processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/12—Details of acquisition arrangements; Constructional details thereof
- G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
- G06V10/147—Details of sensors, e.g. sensor lenses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- G06K2209/40—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/12—Acquisition of 3D measurements of objects
Definitions
- Embodiments described herein relate generally to an article recognition device.
- the article recognition device can recognize an article based on information obtained by capturing an image of the article, measuring a distance with respect to the article, and the like. Such an article recognition device can extract an area having a predetermined height based on the distance information as an article area in which the article appears from the image.
- the article recognition device cannot specify the article area when the height of the article is lower than a height known as the minimum discrimination distance recognizable by the distance sensor of the article recognition device.
- FIG. 1 is a diagram schematically illustrating a configuration example of a checkout device according to an embodiment
- FIG. 2 is a block diagram illustrating a configuration example
- FIG. 3 is a diagram illustrating an operation example of a distance sensor
- FIG. 4 is a flowchart illustrating an operation example
- FIG. 5 is a diagram illustrating a modification example of the operation example of the distance sensor.
- An exemplary embodiment provides an article recognition device capable of recognizing an article having a height lower than a minimum discrimination distance of a distance sensor.
- an article recognition device includes an image interface, a distance information interface, a support unit, and a processor.
- the image interface acquires a captured image.
- the distance information interface acquires distance information from a distance sensor.
- the support unit supports an article at a height equal to or greater than a minimum discrimination distance of the distance sensor from a reference plane of the distance sensor and includes a member of which a distance cannot be measured by the distance sensor.
- the processor acquires the captured image capturing the article supported by the support unit through the image interface, acquires the distance information measured by the distance sensor toward the reference plane through the distance information interface, extracts an area having a height equal to or greater than the minimum discrimination distance from the reference plane as an article area of the article from the captured image based on the distance information, and recognizes the article from an image of the article area.
- a checkout device (article recognition device) according to an embodiment settles an item (article).
- the checkout device captures an image of the item set in the checkout device.
- the checkout device recognizes the item from the captured image and performs settlement.
- the checkout device is set up at a store or the like selling items. For example, the checkout device performs settlement processing on an item when the item is placed at a predetermined position or when checkout device accepts a predetermined operation.
- the checkout device may be installed as a self-register by which a user performs settlement processing. Also, the checkout device may be installed as an ordinary register for a shop clerk of a shop to perform settlement processing.
- FIG. 1 is a block diagram schematically illustrating a configuration example of a checkout device 1 according to the embodiment.
- the checkout device 1 includes a housing 2 , a camera 3 , a distance sensor 4 , an operation panel 5 , an installation base 6 , a control unit 10 , and the like.
- the housing 2 is a frame forming an outline of the checkout device 1 .
- the housing 2 is formed so that the installation base 6 can be installed thereon.
- the housing 2 is formed in a U-shape.
- the camera 3 is installed on an upper portion of the housing 2 so as to be directed downward.
- the camera 3 captures an image of an item A on the installation base 6 . That is, the camera 3 is installed so as to capture the image of the item A from above.
- the camera 3 may be installed so as to capture the image of the item A from obliquely above.
- the position and direction in which the camera 3 is installed are not limited to a specific configuration.
- the camera 3 transmits the captured image to the control unit 10 .
- the distance sensor 4 is installed on an upper portion of the housing 2 so as to be directed downward.
- the distance sensor 4 measures a distance from the item A placed on the installation base 6 to the distance sensor 4 or a distance from the item A to a plane horizontal to the distance sensor 4 .
- the distance sensor 4 generates distance information indicating a distance from a predetermined reference plane based on the measurement result. That is, the distance sensor 4 generates the distance information measured toward the predetermined reference plane. For example, the distance information indicates the coordinates of each point in a predetermined three-dimensional coordinate system.
- the distance sensor 4 transmits the generated distance information to the control unit 10 . Also, based on the measurement result, a processor 11 in the control unit 10 may generate the distance information indicating the distance from the predetermined reference plane.
- the distance sensor 4 measures the distance with a predetermined resolution. That is, the distance sensor 4 measures a height equal to or greater than a predetermined threshold value (minimum discrimination distance). In this case, the distance sensor 4 calculates the distance from a predetermined reference plane to an area having a height equal to or greater than the minimum discrimination distance.
- a predetermined threshold value minimum discrimination distance
- the distance sensor 4 includes a light source and a sensor for detecting reflected light rays of light rays irradiated from the light source.
- the distance sensor 4 measures a distance based on the reflected light ray of the light ray (visible light ray or invisible light ray) irradiated from the light source.
- the distance sensor 4 may perform a Time-of-Flight (ToF) method in which the distance to a measurement target object is measured based on the time until the irradiated light ray is reflected by the measurement target object and reaches the distance sensor 4 .
- ToF Time-of-Flight
- the distance sensor 4 may calculate the distance based on the parallax of each image captured by two cameras (stereo cameras).
- the configuration of the distance sensor 4 is not limited to a specific configuration.
- the operation panel 5 is an interface for accepting input of an instruction from an operator and displaying various pieces of information to an operator.
- the operation panel 5 includes an operation unit for accepting an instruction input and a display unit for displaying information.
- the operation panel 5 transmits a signal indicating the operation accepted from an operator to the control unit 10 as an operation of the operation unit.
- the operation unit includes a touch panel.
- the operation panel 5 displays the image from the control unit 10 as an operation of the display unit.
- the display unit includes a liquid crystal monitor.
- the display unit is integrally formed with a touch panel as the operation unit.
- the operation unit may further include a keyboard or a numeric keypad.
- the installation base 6 (support portion) is installed at a lower portion of the housing 2 .
- the installation base 6 is a base which supports the item A.
- the installation base 6 is formed in a rectangular shape having a predetermined size.
- the installation base 6 has a thickness equal to or greater than the minimum discrimination distance of the distance sensor 4 . That is, the installation base 6 is formed at a height equal to or greater than the minimum discrimination distance towards the distance sensor 4 .
- the installation base 6 includes members in which the distance sensor 4 cannot measure the distance.
- the installation base 6 is formed from a material which transmits radio waves from the distance sensor 4 .
- the installation base 6 is formed from a transparent material which does not appear in the stereo camera.
- the installation base 6 is made of an acrylic plate, a glass plate, or the like. Further, the installation base 6 may include a plurality of sharp pointed support members. The configuration of the installation base 6 is not limited to a specific configuration.
- the control unit 10 controls the entire checkout device 1 .
- the control unit 10 settles the item A installed in the installation base 6 based on an instruction from an operator or the like.
- control unit 10 receives various inputs from an operator through the operation panel 5 . Further, the control unit 10 displays various pieces of information to an operator through the operation panel 5 .
- FIG. 2 is a block diagram illustrating a configuration example of the checkout device 1 .
- the checkout device 1 includes the control unit 10 , the camera 3 , the distance sensor 4 , the operation panel 5 , and the like.
- the control unit 10 , the camera 3 , the distance sensor 4 , and the operation panel 5 are electrically connected.
- the camera 3 , the distance sensor 4 , and the operation panel 5 are as described above.
- the control unit 10 includes the processor 11 , a ROM 12 , a RAM 13 , an NVM 14 , a camera interface 15 , a distance sensor interface 16 , an input/output interface 17 , and the like.
- the processor 11 , the ROM 12 , the RAM 13 , the NVM 14 , the camera interface 15 , the distance sensor interface 16 , and the input/output interface 17 are electrically connected.
- the camera interface 15 is electrically connected to the camera 3 .
- the distance sensor interface 16 is electrically connected to the distance sensor 4 .
- the input/output interface 17 is electrically connected to the operation panel 5 .
- the processor 11 controls the overall operation of the control unit 10 . That is, the processor 11 controls the operation of the entire checkout device 1 .
- the processor 11 may be provided with an internal cache, various interfaces, and the like.
- the processor 11 realizes various processes by executing a program stored in the internal cache, the ROM 12 , or the NVM 14 in advance.
- processor 11 controls the function executed by the hardware circuit.
- the ROM 12 is a nonvolatile memory in which a control program, control data, and the like are stored in advance.
- the control program and the control data stored in the ROM 12 are incorporated in advance according to the specification of the control unit 10 .
- the ROM 12 stores, for example, a program for controlling a circuit board of the control unit 10 or the like.
- the RAM 13 is a volatile memory.
- the RAM 13 temporarily stores data being processed by the processor 11 and the like.
- the RAM 13 stores various application programs based on instructions from the processor 11 . Further, the RAM 13 may store data necessary for executing the application program, an execution result of the application program, and the like.
- the NVM 14 is a nonvolatile memory capable of writing and rewriting data.
- the NVM 14 includes, for example, a Hard Disk Drive (HDD), a Solid State Drive (SSD), an Electrically Erasable Programmable Read-Only Memory (Registered Trademark) (EEPROM), or a flash memory.
- the NVM 14 stores control programs, applications, various data, and the like according to the operational purpose of the control unit 10 .
- the NVM 14 stores an item database on the items.
- the item database stores an item name, a price, external dimensions, dictionary information for recognizing the item, and the like in association with each other.
- the NVM 14 stores the item database in advance.
- the item database may be updated as appropriate.
- the camera interface 15 is an interface for sending and receiving data to and from the camera.
- the camera interface 15 transmits a signal instructing image-capturing to the camera 3 based on the control of processor 11 .
- the camera interface 15 acquires a captured image obtained by image-capturing from the camera 3 .
- the camera interface 15 may support USB connection or may support connection via Ethernet (registered trademark) or camera link.
- the distance sensor interface 16 (distance information interface) is an interface for sending and receiving data with the distance sensor 4 .
- the distance sensor interface 16 transmits a signal which causes the distance sensor 4 to acquire the distance information based on the control of the processor 11 .
- the distance sensor interface 16 obtains the distance information from the distance sensor 4 .
- the distance sensor interface 16 may support USB connection or may support connection via Ethernet (registered trademark) or a camera link.
- the input/output interface 17 is an interface for sending and receiving data to and from the operation panel 5 .
- the input/output interface 17 receives a signal indicating an operation received from an operator from the operation panel 5 . Further, the input/output interface 17 transmits information indicating a screen to be displayed to an operator to the operation panel 5 based on the control of the processor 11 .
- the input/output interface 17 may support USB connection or may support connection via Ethernet or a parallel I/F.
- the checkout device 1 may have a configuration according to necessity besides the configuration as illustrated in FIGS. 1 and 2 or a specific configuration may be excluded from the checkout device 1 .
- processor 11 executing programs stored in the ROM 12 , the NVM 14 , or the like.
- the processor 11 has a function of acquiring an image (captured image) obtained by capturing an image of the installation base 6 on which the item A is placed.
- the processor 11 determines whether to start settlement processing. For example, when it is detected that the item A is placed by a user on the installation base 6 , the processor 11 determines to start settlement processing. The processor 11 detects that the item A is placed on the installation base 6 based on the image from the camera 3 . In addition, the processor 11 may accept an operation indicating that settlement processing is started from a user.
- the processor 11 captures an image including the item A. For example, the processor 11 transmits a signal for capturing an image to the camera 3 . The processor 11 acquires a captured image from the camera 3 . In order to capture an image, the processor 11 may set a capturing parameter to the camera 3 .
- the processor 11 may acquire a captured image from an external device.
- the processor 11 has a function of acquiring distance information from the distance sensor 4 .
- the processor 11 acquires the distance information from the distance sensor 4 .
- the processor 11 sends a signal for measuring the distance to the distance sensor 4 .
- the processor 11 obtains the distance information from the distance sensor 4 .
- the processor 11 may acquire the distance information after acquiring the captured image. Further, the processor 11 may acquire the distance information with the captured image at the same time. Further, the processor 11 may acquire the distance information before acquiring the captured image.
- FIG. 3 is a diagram illustrating an operation example in which the distance sensor 4 acquires the distance information.
- the distance sensor 4 sets a measurement range 51 .
- the measurement range 51 is an area where the distance sensor 4 measures the distance. That is, the distance sensor 4 measures the distance of each portion in the measurement range 51 .
- the measurement range 51 is formed in a conical shape with the distance sensor 4 as an apex.
- the distance sensor 4 sets a reference plane 101 on an end surface (bottom surface of the cone) of the measurement range 51 .
- the distance sensor 4 measures the distance from the distance sensor 4 to an object in the measurement range 51 .
- the processor 11 sets the reference plane 101 to the distance sensor 4 .
- the reference plane 101 is set to a plane obtained by extending the bottom surface of the installation base 6 . That is, the reference plane 101 is set on the housing 2 where the installation base 6 is installed. When the height of the installation base 6 is high enough, the reference plane 101 does not need to be on the housing 2 where the installation base 6 is installed.
- the distance sensor 4 calculates the distance from the housing 2 where the installation base 6 is set.
- the installation base 6 is formed to have a thickness equal to or greater than the minimum discrimination distance. Therefore, the installation base 6 supports the item A in a position which is close to the distance sensor 4 by a distance equal to or greater than the minimum discrimination distance from the reference plane 101 .
- the distance sensor 4 measures the height (sum of the thickness of the installation base 6 and the height of the item A) of an upper surface of the item A supported at a height equal to or greater than the minimum discrimination distance.
- the processor 11 has a function of extracting an item area (item area) in which an item appears from the captured image based on the distance information.
- the area where the upper surface of the item appears is higher than the reference plane 101 by the height of the item and the thickness of the installation base 6 . Therefore, the processor 11 can extract an area higher than the reference plane 101 as an item area even when the height of the item is low.
- the processor 11 extracts the item area of each item.
- the processor 11 has a function of recognizing the item in the item area.
- the processor 11 identifies the item based on the identification information.
- the identification information may be a barcode, a QR code (registered trademark), a letter, a number, or a mark.
- the processor 11 retrieves the identification information from the image of the item area and reads out the found identification information. Based on the reading result, the processor 11 acquires the item code indicating the item of the item image.
- the processor 11 may recognize the item in the item area by object recognition.
- the processor 11 acquires the dictionary information of each piece of item information from the item database stored in the NVM 14 and the like.
- the processor 11 compares the dictionary information with the image in the item area.
- the processor 11 specifies the item code of the item information corresponding to the dictionary information matching the image of the item area as the item code indicating the item of the item image.
- the processor 11 recognizes the item of each item area.
- the method by which the processor 11 recognizes the item is not limited to a specific method.
- the processor 11 may determine whether item recognition is properly performed based on the outer dimensions of the item stored in advance, the area of the item area of the item, and the like. The processor 11 may determine whether item recognition is properly performed based on the outer dimensions of the item stored in advance, the height of the item area of the item, and the like.
- the processor 11 has a function of settling the recognized item.
- the processor 11 refers to the item database and acquires the price based on the item code of the recognized item and the like. Further, the processor 11 may acquire the price of the item specified from an external device.
- the processor 11 settles the item based on the price of the item. For example, the processor 11 accepts input of credit card information from a user. For example, the processor 11 may acquire credit card information using a credit card reader or the like. The processor 11 settles the item based on the credit card information.
- the processor 11 may settle items using cash, debit cards, electronic money, or the like.
- the way in which processor 11 settles items is not limited to a specific method.
- FIG. 4 is a flowchart for describing the operation example of the checkout device 1 .
- the processor 11 of the checkout device 1 determines whether to start settlement processing (ACT 11 ). When it is determined that settlement processing is not started (ACT 11 , NO), the processor 11 returns to ACT 11 .
- the processor 11 acquires the captured image using the camera 3 (ACT 12 ).
- the processor 11 acquires the distance information using the distance sensor 4 (ACT 13 ).
- the processor 11 extracts the item area based on the distance information (ACT 14 ). When the item area is extracted, the processor 11 recognizes the item in the item area (ACT 15 ).
- the processor 11 settles the recognized item (ACT 16 ).
- the processor 11 ends the operation.
- FIG. 5 is a diagram illustrating another example of the operation example in which the distance sensor 4 acquires the distance information.
- the distance sensor 4 sets a measurement range 52 .
- the measurement range 52 is an area where the distance sensor 4 measures the distance. That is, the distance sensor 4 measures the distance of each portion in the measurement range 52 .
- the measurement range 52 is formed in a conical shape with the distance sensor 4 as an apex.
- the distance sensor 4 sets the reference plane 102 on an end surface (bottom surface of the cone) of the measurement range 52 . That is, the distance sensor 4 measures the distance from the reference plane 102 .
- the processor 11 sets reference plane 102 to the distance sensor 4 .
- the reference plane 102 is set at a position apart from the upper surface of the installation base 6 by a distance equal to or greater than the minimum discrimination distance. Further, the reference plane 102 is set at a position closer to the distance sensor 4 than the bottom surface of the installation base 6 . That is, the reference plane 102 is set between the upper surface and the bottom surface of the installation base 6 and on a plane separated from the upper surface of the installation base 6 by a distance equal to or greater than the minimum discrimination distance. Therefore, the item A installed on the installation base 6 is supported at a height equal to or greater than the minimum discrimination distance from the reference plane 102 .
- the distance sensor 4 measures the height (sum of the thickness of the installation base 6 and the height of the item A) of the upper surface of the item A supported at a height equal to or greater than the minimum discrimination distance.
- the checkout device 1 may include a wire or the like for supporting items at a height equal to or greater than the minimum discrimination distance from the reference plane.
- the checkout device 1 may include a plurality of horizontally extending wires.
- the checkout device 1 may include a support plate or the like for supporting items at a height equal to or greater than the minimum discrimination distance from the reference plane.
- the support plate is suspended at a predetermined height by a wire extending downward from an upper portion of the checkout device 1 .
- the support plate is formed from a material of which the height cannot be detected by the distance sensor 4 .
- the checkout device configured as described above has an installation base which supports an item at a height equal to or greater than the minimum discrimination distance of the distance sensor.
- the checkout device acquires a captured image of the item placed in the installation base.
- the checkout device obtains the distance information indicating the height of the item placed in the installation base by using the distance sensor.
- the checkout device extracts the item area from the captured image based on the distance information. That is, the checkout device extracts the area higher than the reference plane as the item area. Since the thickness of the installation base is equal to or greater than the minimum discrimination distance, even when the height of the item is lower than the minimum discrimination distance, the upper surface of the item is located at the height equal to or greater than the minimum discrimination distance. As a result, the checkout device can extract the upper surface of the item as an area higher than the reference plane. Therefore, the checkout device can appropriately extract the item area from the captured image using the distance information even when the height of the item is lower than the minimum discrimination distance.
Abstract
An article recognition device includes an image interface, a distance information interface, a support unit, and a processor. The support unit supports an article at a height equal to or greater than a minimum discrimination distance of a distance sensor from a reference plane of the distance sensor and includes a member of which a distance cannot be measured by the distance sensor. The processor acquires a captured image capturing the article supported by the support unit through the image interface, acquires distance information measured by the distance sensor toward the reference plane through the distance information interface, extracts an area having a height equal to or greater than the minimum discrimination distance from the reference plane as an article area of the article from the captured image based on the distance information, and recognizes the article from an image of the article area.
Description
- This application is based upon and claims the benefit of priority from Japanese Patent Application No. 2018-098182, filed on May 22, 2018, the entire contents of which are incorporated herein by reference.
- Embodiments described herein relate generally to an article recognition device.
- In an article recognition device that recognizes an article such as an item, the article recognition device can recognize an article based on information obtained by capturing an image of the article, measuring a distance with respect to the article, and the like. Such an article recognition device can extract an area having a predetermined height based on the distance information as an article area in which the article appears from the image.
- In the related art, however, the article recognition device cannot specify the article area when the height of the article is lower than a height known as the minimum discrimination distance recognizable by the distance sensor of the article recognition device.
-
FIG. 1 is a diagram schematically illustrating a configuration example of a checkout device according to an embodiment; -
FIG. 2 is a block diagram illustrating a configuration example; -
FIG. 3 is a diagram illustrating an operation example of a distance sensor; -
FIG. 4 is a flowchart illustrating an operation example; and -
FIG. 5 is a diagram illustrating a modification example of the operation example of the distance sensor. - An exemplary embodiment provides an article recognition device capable of recognizing an article having a height lower than a minimum discrimination distance of a distance sensor.
- In general, according to one embodiment, an article recognition device includes an image interface, a distance information interface, a support unit, and a processor. The image interface acquires a captured image. The distance information interface acquires distance information from a distance sensor. The support unit supports an article at a height equal to or greater than a minimum discrimination distance of the distance sensor from a reference plane of the distance sensor and includes a member of which a distance cannot be measured by the distance sensor. The processor acquires the captured image capturing the article supported by the support unit through the image interface, acquires the distance information measured by the distance sensor toward the reference plane through the distance information interface, extracts an area having a height equal to or greater than the minimum discrimination distance from the reference plane as an article area of the article from the captured image based on the distance information, and recognizes the article from an image of the article area.
- Hereinafter, embodiments will be described with reference to the drawings.
- A checkout device (article recognition device) according to an embodiment settles an item (article). The checkout device captures an image of the item set in the checkout device. The checkout device recognizes the item from the captured image and performs settlement. The checkout device is set up at a store or the like selling items. For example, the checkout device performs settlement processing on an item when the item is placed at a predetermined position or when checkout device accepts a predetermined operation. The checkout device may be installed as a self-register by which a user performs settlement processing. Also, the checkout device may be installed as an ordinary register for a shop clerk of a shop to perform settlement processing.
-
FIG. 1 is a block diagram schematically illustrating a configuration example of acheckout device 1 according to the embodiment. As illustrated inFIG. 1 , thecheckout device 1 includes ahousing 2, acamera 3, adistance sensor 4, anoperation panel 5, aninstallation base 6, acontrol unit 10, and the like. - The
housing 2 is a frame forming an outline of thecheckout device 1. Thehousing 2 is formed so that theinstallation base 6 can be installed thereon. In an example illustrated inFIG. 1 , thehousing 2 is formed in a U-shape. - The
camera 3 is installed on an upper portion of thehousing 2 so as to be directed downward. Thecamera 3 captures an image of an item A on theinstallation base 6. That is, thecamera 3 is installed so as to capture the image of the item A from above. Thecamera 3 may be installed so as to capture the image of the item A from obliquely above. The position and direction in which thecamera 3 is installed are not limited to a specific configuration. Thecamera 3 transmits the captured image to thecontrol unit 10. - The
distance sensor 4 is installed on an upper portion of thehousing 2 so as to be directed downward. Thedistance sensor 4 measures a distance from the item A placed on theinstallation base 6 to thedistance sensor 4 or a distance from the item A to a plane horizontal to thedistance sensor 4. - The
distance sensor 4 generates distance information indicating a distance from a predetermined reference plane based on the measurement result. That is, thedistance sensor 4 generates the distance information measured toward the predetermined reference plane. For example, the distance information indicates the coordinates of each point in a predetermined three-dimensional coordinate system. Thedistance sensor 4 transmits the generated distance information to thecontrol unit 10. Also, based on the measurement result, aprocessor 11 in thecontrol unit 10 may generate the distance information indicating the distance from the predetermined reference plane. - The
distance sensor 4 measures the distance with a predetermined resolution. That is, thedistance sensor 4 measures a height equal to or greater than a predetermined threshold value (minimum discrimination distance). In this case, thedistance sensor 4 calculates the distance from a predetermined reference plane to an area having a height equal to or greater than the minimum discrimination distance. - For example, the
distance sensor 4 includes a light source and a sensor for detecting reflected light rays of light rays irradiated from the light source. Thedistance sensor 4 measures a distance based on the reflected light ray of the light ray (visible light ray or invisible light ray) irradiated from the light source. For example, thedistance sensor 4 may perform a Time-of-Flight (ToF) method in which the distance to a measurement target object is measured based on the time until the irradiated light ray is reflected by the measurement target object and reaches thedistance sensor 4. - The
distance sensor 4 may calculate the distance based on the parallax of each image captured by two cameras (stereo cameras). - The configuration of the
distance sensor 4 is not limited to a specific configuration. - The
operation panel 5 is an interface for accepting input of an instruction from an operator and displaying various pieces of information to an operator. Theoperation panel 5 includes an operation unit for accepting an instruction input and a display unit for displaying information. - The
operation panel 5 transmits a signal indicating the operation accepted from an operator to thecontrol unit 10 as an operation of the operation unit. In this case, the operation unit includes a touch panel. - The
operation panel 5 displays the image from thecontrol unit 10 as an operation of the display unit. For example, the display unit includes a liquid crystal monitor. The display unit is integrally formed with a touch panel as the operation unit. - The operation unit may further include a keyboard or a numeric keypad.
- The installation base 6 (support portion) is installed at a lower portion of the
housing 2. Theinstallation base 6 is a base which supports the item A. For example, theinstallation base 6 is formed in a rectangular shape having a predetermined size. Theinstallation base 6 has a thickness equal to or greater than the minimum discrimination distance of thedistance sensor 4. That is, theinstallation base 6 is formed at a height equal to or greater than the minimum discrimination distance towards thedistance sensor 4. - The
installation base 6 includes members in which thedistance sensor 4 cannot measure the distance. - When the
distance sensor 4 measures the distance using the ToF method, theinstallation base 6 is formed from a material which transmits radio waves from thedistance sensor 4. - Further, when the
distance sensor 4 includes a stereo camera, theinstallation base 6 is formed from a transparent material which does not appear in the stereo camera. - For example, the
installation base 6 is made of an acrylic plate, a glass plate, or the like. Further, theinstallation base 6 may include a plurality of sharp pointed support members. The configuration of theinstallation base 6 is not limited to a specific configuration. - The
control unit 10 controls theentire checkout device 1. Thecontrol unit 10 settles the item A installed in theinstallation base 6 based on an instruction from an operator or the like. - For example, the
control unit 10 receives various inputs from an operator through theoperation panel 5. Further, thecontrol unit 10 displays various pieces of information to an operator through theoperation panel 5. -
FIG. 2 is a block diagram illustrating a configuration example of thecheckout device 1. As illustrated inFIG. 2 , thecheckout device 1 includes thecontrol unit 10, thecamera 3, thedistance sensor 4, theoperation panel 5, and the like. Thecontrol unit 10, thecamera 3, thedistance sensor 4, and theoperation panel 5 are electrically connected. Thecamera 3, thedistance sensor 4, and theoperation panel 5 are as described above. - The
control unit 10 includes theprocessor 11, aROM 12, aRAM 13, anNVM 14, acamera interface 15, adistance sensor interface 16, an input/output interface 17, and the like. Theprocessor 11, theROM 12, theRAM 13, theNVM 14, thecamera interface 15, thedistance sensor interface 16, and the input/output interface 17 are electrically connected. Thecamera interface 15 is electrically connected to thecamera 3. Thedistance sensor interface 16 is electrically connected to thedistance sensor 4. The input/output interface 17 is electrically connected to theoperation panel 5. - The
processor 11 controls the overall operation of thecontrol unit 10. That is, theprocessor 11 controls the operation of theentire checkout device 1. Theprocessor 11 may be provided with an internal cache, various interfaces, and the like. Theprocessor 11 realizes various processes by executing a program stored in the internal cache, theROM 12, or theNVM 14 in advance. - Further, some of the various functions realized by the
processor 11 executing the program may be realized by a hardware circuit. In this case, theprocessor 11 controls the function executed by the hardware circuit. - The
ROM 12 is a nonvolatile memory in which a control program, control data, and the like are stored in advance. The control program and the control data stored in theROM 12 are incorporated in advance according to the specification of thecontrol unit 10. TheROM 12 stores, for example, a program for controlling a circuit board of thecontrol unit 10 or the like. - The
RAM 13 is a volatile memory. TheRAM 13 temporarily stores data being processed by theprocessor 11 and the like. TheRAM 13 stores various application programs based on instructions from theprocessor 11. Further, theRAM 13 may store data necessary for executing the application program, an execution result of the application program, and the like. - The
NVM 14 is a nonvolatile memory capable of writing and rewriting data. TheNVM 14 includes, for example, a Hard Disk Drive (HDD), a Solid State Drive (SSD), an Electrically Erasable Programmable Read-Only Memory (Registered Trademark) (EEPROM), or a flash memory. TheNVM 14 stores control programs, applications, various data, and the like according to the operational purpose of thecontrol unit 10. - The
NVM 14 stores an item database on the items. For example, the item database stores an item name, a price, external dimensions, dictionary information for recognizing the item, and the like in association with each other. TheNVM 14 stores the item database in advance. The item database may be updated as appropriate. - The camera interface 15 (image interface) is an interface for sending and receiving data to and from the camera. For example, the
camera interface 15 transmits a signal instructing image-capturing to thecamera 3 based on the control ofprocessor 11. Also, thecamera interface 15 acquires a captured image obtained by image-capturing from thecamera 3. For example, thecamera interface 15 may support USB connection or may support connection via Ethernet (registered trademark) or camera link. - The distance sensor interface 16 (distance information interface) is an interface for sending and receiving data with the
distance sensor 4. For example, thedistance sensor interface 16 transmits a signal which causes thedistance sensor 4 to acquire the distance information based on the control of theprocessor 11. Also, thedistance sensor interface 16 obtains the distance information from thedistance sensor 4. For example, thedistance sensor interface 16 may support USB connection or may support connection via Ethernet (registered trademark) or a camera link. - The input/
output interface 17 is an interface for sending and receiving data to and from theoperation panel 5. For example, the input/output interface 17 receives a signal indicating an operation received from an operator from theoperation panel 5. Further, the input/output interface 17 transmits information indicating a screen to be displayed to an operator to theoperation panel 5 based on the control of theprocessor 11. For example, the input/output interface 17 may support USB connection or may support connection via Ethernet or a parallel I/F. - Further, the
checkout device 1 may have a configuration according to necessity besides the configuration as illustrated inFIGS. 1 and 2 or a specific configuration may be excluded from thecheckout device 1. - Next, the functions realized by the
processor 11 will be described. The following functions are realized by theprocessor 11 executing programs stored in theROM 12, theNVM 14, or the like. - First, the
processor 11 has a function of acquiring an image (captured image) obtained by capturing an image of theinstallation base 6 on which the item A is placed. - The
processor 11 determines whether to start settlement processing. For example, when it is detected that the item A is placed by a user on theinstallation base 6, theprocessor 11 determines to start settlement processing. Theprocessor 11 detects that the item A is placed on theinstallation base 6 based on the image from thecamera 3. In addition, theprocessor 11 may accept an operation indicating that settlement processing is started from a user. - When it is determined that settlement processing is to be started, the
processor 11 captures an image including the item A. For example, theprocessor 11 transmits a signal for capturing an image to thecamera 3. Theprocessor 11 acquires a captured image from thecamera 3. In order to capture an image, theprocessor 11 may set a capturing parameter to thecamera 3. - Further, the
processor 11 may acquire a captured image from an external device. - In addition, the
processor 11 has a function of acquiring distance information from thedistance sensor 4. - When it is determined that settlement processing is to be started, the
processor 11 acquires the distance information from thedistance sensor 4. For example, theprocessor 11 sends a signal for measuring the distance to thedistance sensor 4. Theprocessor 11 obtains the distance information from thedistance sensor 4. - The
processor 11 may acquire the distance information after acquiring the captured image. Further, theprocessor 11 may acquire the distance information with the captured image at the same time. Further, theprocessor 11 may acquire the distance information before acquiring the captured image. -
FIG. 3 is a diagram illustrating an operation example in which thedistance sensor 4 acquires the distance information. As illustrated inFIG. 3 , thedistance sensor 4 sets ameasurement range 51. Themeasurement range 51 is an area where thedistance sensor 4 measures the distance. That is, thedistance sensor 4 measures the distance of each portion in themeasurement range 51. Themeasurement range 51 is formed in a conical shape with thedistance sensor 4 as an apex. - Further, the
distance sensor 4 sets areference plane 101 on an end surface (bottom surface of the cone) of themeasurement range 51. Thedistance sensor 4 measures the distance from thedistance sensor 4 to an object in themeasurement range 51. For example, theprocessor 11 sets thereference plane 101 to thedistance sensor 4. - The
reference plane 101 is set to a plane obtained by extending the bottom surface of theinstallation base 6. That is, thereference plane 101 is set on thehousing 2 where theinstallation base 6 is installed. When the height of theinstallation base 6 is high enough, thereference plane 101 does not need to be on thehousing 2 where theinstallation base 6 is installed. - As a result, the
distance sensor 4 calculates the distance from thehousing 2 where theinstallation base 6 is set. - As described above, the
installation base 6 is formed to have a thickness equal to or greater than the minimum discrimination distance. Therefore, theinstallation base 6 supports the item A in a position which is close to thedistance sensor 4 by a distance equal to or greater than the minimum discrimination distance from thereference plane 101. - The
distance sensor 4 measures the height (sum of the thickness of theinstallation base 6 and the height of the item A) of an upper surface of the item A supported at a height equal to or greater than the minimum discrimination distance. - In addition, the
processor 11 has a function of extracting an item area (item area) in which an item appears from the captured image based on the distance information. - The area where the upper surface of the item appears is higher than the
reference plane 101 by the height of the item and the thickness of theinstallation base 6. Therefore, theprocessor 11 can extract an area higher than thereference plane 101 as an item area even when the height of the item is low. - When a plurality of items appears in the captured image, the
processor 11 extracts the item area of each item. - In addition, the
processor 11 has a function of recognizing the item in the item area. - For example, when the item has identification information identifying itself, the
processor 11 identifies the item based on the identification information. For example, the identification information may be a barcode, a QR code (registered trademark), a letter, a number, or a mark. Theprocessor 11 retrieves the identification information from the image of the item area and reads out the found identification information. Based on the reading result, theprocessor 11 acquires the item code indicating the item of the item image. - Further, the
processor 11 may recognize the item in the item area by object recognition. Theprocessor 11 acquires the dictionary information of each piece of item information from the item database stored in theNVM 14 and the like. - For example, the
processor 11 compares the dictionary information with the image in the item area. Theprocessor 11 specifies the item code of the item information corresponding to the dictionary information matching the image of the item area as the item code indicating the item of the item image. - When a plurality of item areas is extracted, the
processor 11 recognizes the item of each item area. - The method by which the
processor 11 recognizes the item is not limited to a specific method. - Further, the
processor 11 may determine whether item recognition is properly performed based on the outer dimensions of the item stored in advance, the area of the item area of the item, and the like. Theprocessor 11 may determine whether item recognition is properly performed based on the outer dimensions of the item stored in advance, the height of the item area of the item, and the like. - Further, the
processor 11 has a function of settling the recognized item. - The
processor 11 refers to the item database and acquires the price based on the item code of the recognized item and the like. Further, theprocessor 11 may acquire the price of the item specified from an external device. - The
processor 11 settles the item based on the price of the item. For example, theprocessor 11 accepts input of credit card information from a user. For example, theprocessor 11 may acquire credit card information using a credit card reader or the like. Theprocessor 11 settles the item based on the credit card information. - The
processor 11 may settle items using cash, debit cards, electronic money, or the like. The way in whichprocessor 11 settles items is not limited to a specific method. - Next, an operation example of the
checkout device 1 will be described.FIG. 4 is a flowchart for describing the operation example of thecheckout device 1. - First, the
processor 11 of thecheckout device 1 determines whether to start settlement processing (ACT11). When it is determined that settlement processing is not started (ACT11, NO), theprocessor 11 returns to ACT11. - When it is determined that settlement processing is to be started (ACT11, YES), the
processor 11 acquires the captured image using the camera 3 (ACT12). When the captured image is acquired, theprocessor 11 acquires the distance information using the distance sensor 4 (ACT13). - When the distance information is acquired, the
processor 11 extracts the item area based on the distance information (ACT14). When the item area is extracted, theprocessor 11 recognizes the item in the item area (ACT15). - When the item is recognized, the
processor 11 settles the recognized item (ACT16). When the recognized item is settled, theprocessor 11 ends the operation. - Next, a modification example of the operation example in which the
distance sensor 4 acquires distance information will be described.FIG. 5 is a diagram illustrating another example of the operation example in which thedistance sensor 4 acquires the distance information. - As illustrated in
FIG. 5 , thedistance sensor 4 sets ameasurement range 52. Themeasurement range 52 is an area where thedistance sensor 4 measures the distance. That is, thedistance sensor 4 measures the distance of each portion in themeasurement range 52. Themeasurement range 52 is formed in a conical shape with thedistance sensor 4 as an apex. - Further, the
distance sensor 4 sets thereference plane 102 on an end surface (bottom surface of the cone) of themeasurement range 52. That is, thedistance sensor 4 measures the distance from thereference plane 102. For example, theprocessor 11sets reference plane 102 to thedistance sensor 4. - The
reference plane 102 is set at a position apart from the upper surface of theinstallation base 6 by a distance equal to or greater than the minimum discrimination distance. Further, thereference plane 102 is set at a position closer to thedistance sensor 4 than the bottom surface of theinstallation base 6. That is, thereference plane 102 is set between the upper surface and the bottom surface of theinstallation base 6 and on a plane separated from the upper surface of theinstallation base 6 by a distance equal to or greater than the minimum discrimination distance. Therefore, the item A installed on theinstallation base 6 is supported at a height equal to or greater than the minimum discrimination distance from thereference plane 102. - The
distance sensor 4 measures the height (sum of the thickness of theinstallation base 6 and the height of the item A) of the upper surface of the item A supported at a height equal to or greater than the minimum discrimination distance. - Further, instead of the
installation base 6, thecheckout device 1 may include a wire or the like for supporting items at a height equal to or greater than the minimum discrimination distance from the reference plane. For example, thecheckout device 1 may include a plurality of horizontally extending wires. - Instead of the
installation base 6, thecheckout device 1 may include a support plate or the like for supporting items at a height equal to or greater than the minimum discrimination distance from the reference plane. For example, the support plate is suspended at a predetermined height by a wire extending downward from an upper portion of thecheckout device 1. Also, the support plate is formed from a material of which the height cannot be detected by thedistance sensor 4. - The checkout device configured as described above has an installation base which supports an item at a height equal to or greater than the minimum discrimination distance of the distance sensor. The checkout device acquires a captured image of the item placed in the installation base. In addition, the checkout device obtains the distance information indicating the height of the item placed in the installation base by using the distance sensor.
- The checkout device extracts the item area from the captured image based on the distance information. That is, the checkout device extracts the area higher than the reference plane as the item area. Since the thickness of the installation base is equal to or greater than the minimum discrimination distance, even when the height of the item is lower than the minimum discrimination distance, the upper surface of the item is located at the height equal to or greater than the minimum discrimination distance. As a result, the checkout device can extract the upper surface of the item as an area higher than the reference plane. Therefore, the checkout device can appropriately extract the item area from the captured image using the distance information even when the height of the item is lower than the minimum discrimination distance.
- While certain embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the inventions. Indeed, the novel embodiments described herein may be embodied in a variety of other forms; furthermore, various omissions, substitutions and changes in the form of the embodiments described herein may be made without departing from the spirit of the inventions. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the inventions.
Claims (20)
1. An article recognition device, comprising:
an image interface configured to acquire a captured image;
a distance information interface configured to acquire distance information from a distance sensor;
a support configured to support an article at a height equal to or greater than a minimum discrimination distance of the distance sensor from a reference plane of the distance sensor and includes a member of which a distance cannot be measured by the distance sensor; and
a processor configured to:
acquire the captured image of the article supported by the support through the image interface;
acquire the distance information measured by the distance sensor toward the reference plane through the distance information interface;
extract an area of the article from the captured image, wherein the area has a height equal to or greater than the minimum discrimination distance from the reference plane, and wherein the area is an article area of the captured image; and
recognize the article from a portion of the captured image corresponding to the article area extracted based on the distance information.
2. The article recognition device according to claim 1 , wherein
the support is an installation base having a thickness equal to or greater than the minimum discrimination distance and formed from a material of which a distance cannot be measured by the distance sensor.
3. The article recognition device according to claim 2 , wherein
the reference plane is set to a plane obtained by expanding a bottom surface of the installation base.
4. The article recognition device according to claim 2 , wherein
the reference plane is set to a plane which is located between an upper surface and a bottom surface of the installation base and apart from the upper surface of the installation base by a distance equal to or greater than the minimum discrimination distance.
5. The article recognition device according to claim 1 , further comprising:
a camera connected to the image interface; and
the distance sensor.
6. The article recognition device according to claim 1 , wherein the processor is configured to ignore an area having a height less than the minimum discrimination distance from the reference plane.
7. The article recognition device according to claim 1 , wherein the support includes a plate suspended at a predetermined height by a wire extending downward from an upper portion of the article recognition device.
8. The article recognition device according to claim 7 , wherein the plate is made of a material not detectable by the distance sensor.
9. A device for identifying a digital processing region and recognizing one or more items therein, the device comprising:
an image sensor having a first field of view;
a distance sensor having a second field of view no less than the first field of view, the distance sensor is configured to ignore objects having heights less than a minimum discrimination distance;
a support for receiving an article having a height less than the minimum discrimination distance, wherein the support raises the article toward the distance sensor such that the distance sensor registers an area covered by the article, and
a processor configured to:
determine the digital processing region based on the first field of view, the second field of view, and the area covered by the article,
process a portion of image data of an image of the article captured by the image sensor, within the area covered by the article, and corresponding to the digital processing region, and
identify commodity information of the article.
10. The device of claim 9 , wherein the area covered by the article is a subset of the second field of view and wherein the processor excludes image data of the image of the article captured by the image sensor outside of the area covered by the article from processing
11. The device of claim 9 , wherein the commodity information of the article includes a name and a price of the article for checkout.
12. The device of claim 9 , wherein the support includes a base resting on a reference plane, wherein the base has a thickness equal to or greater than the minimum discrimination distance and formed from a material of which a distance cannot be measured by the distance sensor.
13. The device of claim 9 , wherein the support includes a plate suspended at a predetermined height by a wire extending downward from an upper portion of the article recognition device, the plate made of a material not detectable by the distance sensor.
14. A method for recognizing thin items during check out, the method comprising:
capturing at least one image in a range;
detecting, by a distance sensor, a distance between an article and the distance sensor when the article is placed within the range, wherein the distance sensor ignores objects having heights less than the minimum discrimination distance, the article has a thickness less than a minimum discrimination distance;
supporting the article using a support to reach a height equal to or greater than the minimum discrimination distance;
extracting a processing area of the article based on the detected distance; and
recognizing commodity information associated with the article from image data of a portion, corresponding to the processing area, of the at least one image.
15. The method of claim 14 , wherein capturing at least one image in a range comprises acquiring the at least one image using a digital camera.
16. The method of claim 14 , further comprising including the support in the range.
17. The method of claim 16 , wherein the support includes a base resting on a reference plane, wherein the base has a thickness equal to or greater than the minimum discrimination distance and formed from a material of which a distance cannot be measured by the distance sensor.
18. The method of claim 16 , wherein the support includes a plate suspended at a predetermined height by a wire extending downward from an upper portion of the article recognition device, the plate made of a material not detectable by the distance sensor.
19. The method of claim 16 , further comprising:
searching a database having records of article features and their associated commodity information; and
matching one or more of the article features to the image data of the portion, corresponding to the processing area, of the at least one image for identifying relevant commodity information.
20. The method of claim 16 , further comprising accepting a payment to a price included in the commodity information associated with the article.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/212,631 US11651663B2 (en) | 2018-05-22 | 2021-03-25 | Article recognition device |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2018098182A JP6999493B2 (en) | 2018-05-22 | 2018-05-22 | Article recognition device |
JP2018-098182 | 2018-05-22 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/212,631 Continuation US11651663B2 (en) | 2018-05-22 | 2021-03-25 | Article recognition device |
Publications (1)
Publication Number | Publication Date |
---|---|
US20190362182A1 true US20190362182A1 (en) | 2019-11-28 |
Family
ID=66625824
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/416,702 Abandoned US20190362182A1 (en) | 2018-05-22 | 2019-05-20 | Article recognition device |
US17/212,631 Active 2039-09-09 US11651663B2 (en) | 2018-05-22 | 2021-03-25 | Article recognition device |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/212,631 Active 2039-09-09 US11651663B2 (en) | 2018-05-22 | 2021-03-25 | Article recognition device |
Country Status (3)
Country | Link |
---|---|
US (2) | US20190362182A1 (en) |
EP (1) | EP3572978B1 (en) |
JP (1) | JP6999493B2 (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20230252542A1 (en) | 2022-02-04 | 2023-08-10 | Ncr Corporation | Multi-item product recognition for checkouts |
Family Cites Families (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7191943B2 (en) | 2004-07-28 | 2007-03-20 | Caterpillar Inc | Robust barcode and reader for rod position determination |
EP2065849A1 (en) * | 2007-11-30 | 2009-06-03 | Sap Ag | Detection of counterfeited products based on sensor measurements |
JP4900302B2 (en) | 2008-03-31 | 2012-03-21 | 沖電気工業株式会社 | Automatic delivery device |
US8582803B2 (en) | 2010-10-15 | 2013-11-12 | International Business Machines Corporation | Event determination by alignment of visual and transaction data |
JP6318557B2 (en) | 2013-11-11 | 2018-05-09 | 日本電気株式会社 | POS terminal device, POS system, image processing method and program |
US9256775B1 (en) | 2014-12-23 | 2016-02-09 | Toshiba Tec Kabushiki Kaisha | Image recognition apparatus and commodity information processing apparatus |
US10332066B1 (en) * | 2015-03-30 | 2019-06-25 | Amazon Technologies, Inc. | Item management system using weight |
US9747512B2 (en) | 2015-06-25 | 2017-08-29 | Toshiba Tec Kabushiki Kaisha | Article recognition apparatus and image processing method for article recognition apparatus |
WO2017011125A1 (en) | 2015-07-13 | 2017-01-19 | Rambus Inc. | Optical systems and methods supporting diverse optical and computational functions |
US10007991B2 (en) * | 2016-01-29 | 2018-06-26 | International Business Machines Corporation | Low-cost method to reliably determine relative object position |
SG11201807829RA (en) * | 2016-03-14 | 2018-10-30 | Sys Tech Solutions Inc | Methods and a computing device for determining whether a mark is genuine |
-
2018
- 2018-05-22 JP JP2018098182A patent/JP6999493B2/en active Active
-
2019
- 2019-05-20 US US16/416,702 patent/US20190362182A1/en not_active Abandoned
- 2019-05-21 EP EP19175560.2A patent/EP3572978B1/en active Active
-
2021
- 2021-03-25 US US17/212,631 patent/US11651663B2/en active Active
Also Published As
Publication number | Publication date |
---|---|
US20210209409A1 (en) | 2021-07-08 |
EP3572978A1 (en) | 2019-11-27 |
EP3572978B1 (en) | 2023-10-18 |
JP2019204233A (en) | 2019-11-28 |
JP6999493B2 (en) | 2022-01-18 |
US11651663B2 (en) | 2023-05-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6999404B2 (en) | Article recognition device and article recognition method | |
US9256775B1 (en) | Image recognition apparatus and commodity information processing apparatus | |
EP3038015A2 (en) | Image recognition apparatus and commodity information processing apparatus | |
JP2021057054A (en) | Image recognition apparatus | |
EP2897110A1 (en) | Commodity reading apparatus, sales data processing apparatus having the same and method for recognizing commodity | |
JP6957300B2 (en) | Image processing device and image processing method | |
EP3367298A1 (en) | Article recognition apparatus, settlement apparatus and article recognition method | |
US11651663B2 (en) | Article recognition device | |
US20190073880A1 (en) | Article recognition apparatus, article recognition method, and non-transitory readable storage medium | |
EP3767538A1 (en) | Sequential classification for commodity identification | |
US10192136B2 (en) | Image processing apparatus and image processing method | |
US10963726B2 (en) | Article recognition device | |
EP3518143B1 (en) | Article recognition device and commodity settlement device | |
US10977512B2 (en) | Article recognition device | |
JP7274644B2 (en) | Article recognition device | |
EP3742335A1 (en) | Commodity recognition device and commodity recognition method | |
EP3358503A1 (en) | Article recognition apparatus and article recognition method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: TOSHIBA TEC KABUSHIKI KAISHA, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ARIGA, NORIMASA;YASUNAGA, MASAAKI;NOBUOKA, TETSUYA;REEL/FRAME:049230/0282 Effective date: 20190515 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |