WO2020179480A1 - 物品推定装置、物品推定方法、及びプログラム - Google Patents

物品推定装置、物品推定方法、及びプログラム Download PDF

Info

Publication number
WO2020179480A1
WO2020179480A1 PCT/JP2020/006860 JP2020006860W WO2020179480A1 WO 2020179480 A1 WO2020179480 A1 WO 2020179480A1 JP 2020006860 W JP2020006860 W JP 2020006860W WO 2020179480 A1 WO2020179480 A1 WO 2020179480A1
Authority
WO
WIPO (PCT)
Prior art keywords
article
shelf
person
weight
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/006860
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
元亨 仲村
尚幸 村
晋哉 山崎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to US17/434,813 priority Critical patent/US11922391B2/en
Priority to JP2021503960A priority patent/JP7435587B2/ja
Publication of WO2020179480A1 publication Critical patent/WO2020179480A1/ja
Anticipated expiration legal-status Critical
Priority to US18/231,605 priority patent/US12147962B2/en
Priority to US18/231,583 priority patent/US12141776B2/en
Priority to US18/233,623 priority patent/US12141777B2/en
Priority to JP2024017129A priority patent/JP7694737B2/ja
Priority to JP2025094722A priority patent/JP2025123254A/ja
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47F—SPECIAL FURNITURE, FITTINGS, OR ACCESSORIES FOR SHOPS, STOREHOUSES, BARS, RESTAURANTS OR THE LIKE; PAYING COUNTERS
    • A47F10/00—Furniture or installations specially adapted to particular types of service systems, not otherwise provided for
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G1/00—Storing articles, individually or in orderly arrangement, in warehouses or magazines
    • B65G1/02—Storage devices
    • B65G1/04—Storage devices mechanical
    • B65G1/137—Storage devices mechanical with arrangements or automatic control means for selecting which articles are to be removed
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01G—WEIGHING
    • G01G19/00—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups
    • G01G19/40—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups with provisions for indicating, recording, or computing price or other quantities dependent on the weight
    • G01G19/42—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups with provisions for indicating, recording, or computing price or other quantities dependent on the weight for counting by weighing
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01G—WEIGHING
    • G01G19/00—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups
    • G01G19/52—Weighing apparatus combined with other objects, e.g. furniture
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/08—Systems determining position data of a target for measuring distance only
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/08—Payment architectures
    • G06Q20/20—Point-of-sale [POS] network systems
    • G06Q20/203—Inventory monitoring
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/08—Payment architectures
    • G06Q20/20—Point-of-sale [POS] network systems
    • G06Q20/208—Input by product or record sensing, e.g. weighing or scanner processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0036—Checkout procedures
    • G07G1/0045—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader
    • G07G1/0054—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader with control of supplementary check-parameters, e.g. weight or number of articles
    • G07G1/0063—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader with control of supplementary check-parameters, e.g. weight or number of articles with means for detecting the geometric dimensions of the article of which the code is read, such as its size or height, for the verification of the registration
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0036—Checkout procedures
    • G07G1/0045—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader
    • G07G1/0054—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader with control of supplementary check-parameters, e.g. weight or number of articles
    • G07G1/0072—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader with control of supplementary check-parameters, e.g. weight or number of articles with means for detecting the weight of the article of which the code is read, for the verification of the registration
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/12—Cash registers electronically operated
    • G07G1/14—Systems including one or more distant stations co-operating with a central processing unit
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47F—SPECIAL FURNITURE, FITTINGS, OR ACCESSORIES FOR SHOPS, STOREHOUSES, BARS, RESTAURANTS OR THE LIKE; PAYING COUNTERS
    • A47F10/00—Furniture or installations specially adapted to particular types of service systems, not otherwise provided for
    • A47F10/02—Furniture or installations specially adapted to particular types of service systems, not otherwise provided for for self-service type systems, e.g. supermarkets
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30232—Surveillance

Definitions

  • the present invention relates to an article estimation device, an article estimation method, and a program.
  • Patent Document 1 in the work of packing a plurality of types of articles taken out from an inventory shelf into a box, the total weight of the articles 5 stored in the inventory shelf is measured, and a warning is given using the measurement results. It is described to determine whether or not to do so.
  • Patent Document 2 describes that in order to manage the handling of goods, inventory data is generated using the result of processing an image of a bar code or QR code (registered trademark) of the goods.
  • the inventory data is information indicating the handling status of the article, and includes, for example, the identification information of the photographed article, the date and time when the article was collated, the place where the article was installed, and the user who handled the article. It is information in which the identification information and the like are associated with each other.
  • One of the objects of the present invention is to improve the discrimination accuracy of articles taken out from shelves.
  • weight change data which is data based on a change in a detection value of a weight sensor provided on a shelf on which a plurality of articles can be placed, and a position in a shelf front space that is a space in front of the shelf.
  • Acquisition means for acquiring motion data indicating the movement of a person's hand An output unit that outputs, using the weight change data and the operation data, the item identification information of the item estimated to be taken out by the hand,
  • an article estimation device including:
  • the computer Weight change data which is data based on a change in a detection value of a weight sensor provided on a shelf on which a plurality of articles can be placed, and movement of a person's hand located in the front shelf space which is a space in front of the shelf.
  • the operation data indicating An article estimation method is provided which outputs article identification information of the article estimated to have been taken out by the hand using the weight change data and the operation data.
  • Weight change data which is data based on a change in a detection value of a weight sensor provided on a shelf on which a plurality of articles can be placed, and movement of a person's hand located in the front shelf space which is a space in front of the shelf.
  • the function to obtain the operation data A function of using the weight change data and the operation data to output the article identification information of the article estimated to be taken out by the hand;
  • the accuracy of distinguishing an item taken out of a shelf is improved.
  • FIG. 1 It is a figure which shows the function structure of the article information estimation apparatus which concerns on this embodiment with the usage environment of an article information estimation apparatus. It is a figure which shows an example of the data which the shelving allocation information storage part has memorize
  • step S104 of FIG. 4 It is a top view for explaining the layout of the weight sensor concerning a 3rd embodiment. It is a flowchart for demonstrating the detail of the article identification process (step S104 of FIG. 4) in 3rd Embodiment. It is a figure which shows the functional structure of the article information estimation apparatus which concerns on 4th Embodiment together with the use environment of the article information estimation apparatus. It is a flowchart for demonstrating the detail of step S104 in 4th Embodiment. It is a figure which shows the functional structure of the article information estimation apparatus which concerns on 5th Embodiment together with the use environment of the article information estimation apparatus. It is a flow chart for explaining an example of operation of an article information estimating device concerning a 5th embodiment.
  • FIG. 1 is a diagram showing a functional configuration of the article information estimation device 10 according to the present embodiment together with a usage environment of the article information estimation device 10.
  • the article information estimation device 10 according to the embodiment is an device that estimates the article 200 taken out from the shelf 20 by a person, and is used together with the weight sensor 30 and the depth sensor 40.
  • the shelf 20 is shown as viewed from the side.
  • a plurality of articles 200 can be placed on the shelf 20.
  • the shelves 20 are product shelves
  • the goods 200 are goods
  • the person who takes out the goods 200 is a customer or a clerk (employee).
  • the shelf 20 is arranged in a pharmacy
  • the article 200 is a drug shelf
  • the article 200 is a drug
  • the person who takes out the article 200 is a pharmacist.
  • the articles 200 are arranged on each of the plurality of shelves 20.
  • a plurality of types of articles 200 are placed on the plurality of shelves 20.
  • the shelf 20 on which the article 200 is placed is predetermined for each article 200. Therefore, if the shelf 20 from which the article 200 is taken out is known, the type of the article 200 can be estimated.
  • the shelf 20 may be a single stage.
  • the depth sensor 40 includes the space in front of the shelf 20 (hereinafter referred to as the space in front of the shelf) in the detection range, and generates data indicating the movement of the hand of a person located in the space in front of the shelf.
  • the depth sensor 40 is arranged above the space in front of the shelf, it may be arranged in both of the space in front of the shelf or below the space in front of the shelf. Then, the depth sensor 40 generates data indicating the position of the hand in the xy plane (that is, the horizontal plane) and the position of the hand in the z direction (that is, the height direction), and outputs this data to the article information estimation device 10. ..
  • the article information estimation device 10 can identify the shelf 20 by using the data generated by the depth sensor 40.
  • the depth sensor 40 may be, for example, one using a stereo camera or one using LiDAR (Light Detection and Ringing). Further, the article information estimation device 10 may generate data indicating the position of the hand by processing the output data from the depth sensor 40.
  • the articles taken out from the shelf 20. 200 can be estimated.
  • the weight sensor 30 detects the total weight of the shelf 20.
  • the detection value of the weight sensor 30 is output to the article information estimation device 10 together with the weight sensor identification information assigned to the weight sensor 30. Then, by using this weight sensor identification information, the article information estimation device 10 can estimate the type of the article 200 taken out.
  • the article information estimation device 10 includes an acquisition unit 110 and an output unit 120.
  • the acquisition unit 110 acquires data based on the change in the detected value of the weight sensor 30 (hereinafter referred to as weight change data). For example, the acquisition unit 110 generates weight change data by arranging the data acquired from the weight sensor 30 in chronological order.
  • a data processing device that generates weight change data using the data generated by the weight sensor 30 may be provided outside the article information estimation device 10. In this case, the acquisition unit 110 acquires the weight change data from this data processing device.
  • the acquisition unit 110 acquires data indicating the movement of the hand of a person located in the space in front of the shelf (hereinafter referred to as operation data).
  • the acquisition unit 110 generates operation data by, for example, arranging the data output from the depth sensor 40 to the article information estimation device 10 in chronological order.
  • the output unit 120 uses the weight change data and the operation data to output the article identification information of the article presumed to have been taken out by a person located in the space in front of the shelf.
  • the article information estimation device 10 has a shelving allocation information storage unit 130.
  • the shelving allocation information storage unit 130 stores, for each of the shelves 20, article specifying information that specifies an article placed on the shelf 20.
  • the output unit 120 identifies the shelf 20 on which the taken-out product is placed, reads the article identification information corresponding to the identified shelf 20 from the shelf allocation information storage unit 130, and outputs the read article identification information.
  • the item identification information is, for example, an ID (which may be code information) assigned to the item, or an item name (for example, a product name) of the item.
  • FIG. 2 is a diagram showing an example of data stored in the shelf allocation information storage unit 130.
  • the shelving allocation information storage unit 130 stores, for each piece of information (hereinafter, referred to as shelf position information) indicating the position of the shelf 20, the weight sensor identification information of the weight sensor 30 attached to the shelf 20 and the position thereof.
  • the article identification information and the threshold information of the article 200 placed on the article 200 are stored.
  • the shelf position information includes information that identifies the height of the shelf 20 (for example, the height from the floor or the number of steps from the bottom).
  • the threshold information is a value assumed as a decrease in the detected value of the weight sensor 30 when one article 200 is taken out from the shelf, and is, for example, a value of 90% or more and 110% or less of the weight of the article 200. It is set.
  • the threshold indicated by this threshold information is used by the output unit 120, as described later.
  • FIG. 3 is a block diagram illustrating a hardware configuration of the article information estimation device 10 shown in FIG.
  • the article information estimation device 10 includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.
  • the bus 1010 is a data transmission path for the processor 1020, the memory 1030, the storage device 1040, the input/output interface 1050, and the network interface 1060 to send and receive data to and from each other.
  • the method of connecting the processors 1020 and the like to each other is not limited to bus connection.
  • the processor 1020 is a processor realized by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and the like.
  • the memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.
  • the storage device 1040 is an auxiliary storage device realized by an HDD (Hard Disk Drive), an SSD (Solid State Drive), a memory card, a ROM (Read Only Memory), or the like.
  • the storage device 1040 stores a program module that realizes each function of the article information estimation device 10 (for example, the acquisition unit 110 and the output unit 120).
  • the processor 1020 reads each of these program modules into the memory 1030 and executes them, each function corresponding to the program module is realized.
  • the input / output interface 1050 is an interface for connecting the article information estimation device 10 and various input / output devices.
  • the network interface 1060 is an interface for connecting the article information estimation device 10 to a network.
  • This network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network).
  • the method for connecting the network interface 1060 to the network may be a wireless connection or a wired connection.
  • the article information estimation device 10 is connected to necessary devices (for example, a sensor group such as the weight sensor 30 and the depth sensor 40) via the input/output interface 1050 or the network interface 1060.
  • necessary devices for example, a sensor group such as the weight sensor 30 and the depth sensor 40
  • FIG. 4 is a flowchart for explaining an operation example of the article information estimation device 10.
  • the weight sensor 30 constantly transmits data and weight sensor identification information to the article information estimation apparatus 10.
  • the depth sensor 40 also continuously transmits data to the article information estimation device 10.
  • the acquisition unit 110 continues to acquire these data, that is, the weight change data and the motion data. Further, the acquisition unit 110 keeps the acquired data stored in the storage as necessary.
  • the output unit 120 analyzes the detection value of the weight sensor 30 acquired by the acquisition unit 110 and identifies the weight sensor identification information of the weight sensor 30 whose detection value (that is, weight) has decreased by a reference value or more (step S102). This reference value is stored in the shelf allocation information storage unit 130, for example, as shown in FIG. Then, the output unit 120 performs the article identification process using the weight sensor identification information identified in step S102 and the detection value of the depth sensor 40 (step S104).
  • FIG. 5 is a flowchart for explaining the details of the processing performed in step S104.
  • the output unit 120 reads out the shelf position information corresponding to the weight sensor identification information specified in step S102 from the shelf allocation information storage unit 130 (step S202).
  • the output unit 120 analyzes the data output by the depth sensor 40 and detects the height of the hand inserted into the shelf 20 (step S204).
  • the output unit 120 determines whether or not the relationship between the hand height detected in step S204 and the shelf position information read in step S202 satisfies the criteria (step S206). For example, the output unit 120 determines that the reference is satisfied when the height of the hand detected in step S204 is between the height indicated by the shelf position information and the height of the shelf 20 above it.
  • the standard may be stored in the shelving allocation information storage unit 130 for each shelf 20. In this case, the output unit 120 reads out and uses the reference associated with the shelf position information specified in step S102. Further, the shelf allocation information storage unit 130 may store a range that can be taken by the height of the hand inserted into the shelf 20 instead of the shelf position information. In this case, the output unit 120 determines whether or not the height acquired by the depth sensor 40 is within this range.
  • the output unit 120 stores the article identification information corresponding to the weight sensor identification information identified in step S102 in the shelving allocation information storage unit 130.
  • the article 200 taken out by a person is estimated by reading from (step S208). Then, the output unit 120 outputs the read item specifying information.
  • the output unit 120 performs alert processing.
  • This alert processing is, for example, causing a terminal of an administrator of the article 200 (for example, a store clerk when the shelf 20 is a store) to display a predetermined screen (step S210).
  • the data generated by the depth sensor 40, the image captured by the first imaging unit 70 and the image captured by the second imaging unit 80 described in the embodiments described later are displayed on the terminal of the administrator. May be sent to.
  • the administrator may estimate the article 200 taken out by checking the image or the like, and transmit the result to the output unit 120 via the terminal.
  • the output unit 120 first detects that the position of the hand has reached a height corresponding to one of the shelves 20, and then when the weight change of the shelf 20 satisfies the standard. Then, the article specifying information corresponding to the shelf 20 may be read.
  • FIG. 6 is a diagram showing a layout example of the shelf 20 and the weight sensor 30 according to the modification.
  • the shelf 20 is shown as viewed from the front.
  • the shelf 20 has a plurality of partial regions 22 in at least one stage.
  • An article 200 different from the other partial areas 22 is placed on at least one of the plurality of partial areas 22.
  • the weight sensor 30 is provided for each partial region 22.
  • the shelving allocation information storage unit 130 stores the information shown in FIG. 2, that is, the shelving position information, the weight sensor identification information, the article specifying information, and the threshold information for each of the plurality of partial areas 22.
  • the space in front of the shelf is set for each partial area 22, and the depth sensor 40 is also provided for each partial area 22.
  • Each of the plurality of depth sensors 40 stores depth sensor identification information that identifies the depth sensor 40 from the other depth sensors 40. Then, the depth sensor 40 transmits the depth sensor identification information together with the data to the article information estimation device 10.
  • the shelf allocation information storage unit 130 stores the depth sensor identification information of the depth sensor 40 corresponding to the shelf position for each shelf position information.
  • the article information estimation device 10 uses the combination of the weight sensor identification information and the depth sensor identification information stored in the shelving allocation information storage unit 130 to transmit the data transmitted from the depth sensor 40 and the data transmitted from the weight sensor 30. The combination of data
  • the article information estimation device 10 performs the processing shown in FIGS. 4 and 5 for each combination of the data, that is, in units of 22 subregions. According to this modification, even when a plurality of types of articles 200 are arranged on the shelves 20 having the same height, it is possible to estimate the articles 200 taken out by a person.
  • the article 200 on the shelf 20 is the hand. Judge that it was taken out by. Therefore, the discrimination accuracy of the article 200 taken out from the shelf 20 is improved.
  • FIG. 7 is a diagram showing the functional configuration of the article information estimation device 10 according to the present embodiment together with the usage environment of the article information estimation device 10, and corresponds to FIG. 1 of the first embodiment.
  • the article information estimation device 10 according to the present embodiment has the same configuration as the article information estimation device 10 according to the first embodiment except for the following points.
  • the article information estimation device 10 acquires the person identification information of a person existing in the space in front of the shelf 20 from the person tracking device 50.
  • the person tracking device 50 generates, for example, flow line information indicating the flow line of each person by analyzing images sent from a plurality of image pickup units whose image pickup ranges are different from each other. Then, the person tracking device 50 stores the flow line information in association with the person identification information.
  • the person identification information is, for example, a feature amount obtained from an image of a person. Further, when the shelf 20 is installed in the store, the person identification information may be a customer ID such as a membership number. Then, when a person stays in the space in front of the shelf 20 for a certain period of time, the person tracking device 50 outputs the person identification information of the person to the article information estimation device 10.
  • the article information estimation device 10 has a storage processing unit 140.
  • the storage processing unit 140 stores the article identification information acquired by the output unit 120 in step S208 of FIG. 5 in the registered article storage unit 60 in association with the person identification information acquired from the person tracking device 50.
  • the article information estimation device 10 and the registered article storage unit 60 can be used as a product registration device or a store server of a POS (Point of sale system). Then, the POS settlement device performs payment processing using the information stored in the registered article storage unit 60.
  • POS Point of sale system
  • the person tracking device 50 stores the feature amount of the face of the customer who entered the store.
  • the person tracking device 50 acquires, for example, an image from an imaging device including the entrance of a store in the imaging range and processes the image to acquire and store the feature amount of the customer's face.
  • the person tracking device 50 uses this feature amount to generate customer flow line information.
  • the flow line information is associated with the feature amount or the customer ID associated with the feature amount.
  • the storage processing unit 140 of the article information estimation device 10 associates the product identification information of the product taken out by the customer with the registered product storage unit 60 in association with the customer's feature amount (or the customer ID associated with this feature amount). To memorize. Since this process is repeated until the customer performs the payment process, when the customer takes out a plurality of products, the registered product storage unit 60 stores the product identification information of the plurality of products in the feature amount of the customer (or Store it by associating it with the customer ID) associated with the feature amount.
  • the customer can read the information stored in the registered article storage unit 60 using the customer's terminal.
  • the customer's terminal transmits the customer's feature amount (or customer ID) to the storage processing unit 140.
  • the storage processing unit 140 reads out the item specifying information associated with the transmitted characteristic amount (or customer ID) from the registered item storing unit 60, and transmits this item specifying information as a list of products to the customer's terminal.
  • the article-specific information may be converted into a product name using a database.
  • the price of the product may be sent together with the product identification information (or the product name). In the latter case, the total amount of the registered goods may be further transmitted to the customer's terminal.
  • This screen includes, for example, an input button for making a payment.
  • the customer operates the customer's terminal, for example, to transmit information indicating that the product is to be settled to the settlement device together with the customer's feature amount (or customer ID).
  • the settlement device reads the article identification information corresponding to the received feature amount (or customer ID) from the registered article storage unit 60, and performs the settlement process using the read information.
  • the checkout device then generates an electronic receipt and sends it to the customer's terminal.
  • the settlement device may be incorporated in the article information estimation device 10.
  • the information indicating that the product is to be settled may be input from the terminal installed in the store.
  • the terminal may capture the face of the customer to generate a characteristic amount and transmit the characteristic amount to the settlement device.
  • the shelf 20 When the shelf 20 is installed in a distribution center or a pharmacy, the person who took out the article 200 can be confirmed by using the information stored in the registered article storage unit 60.
  • the registered article storage unit 60 is outside the article information estimation apparatus 10 in the example shown in FIG. 7, the registered article storage unit 60 may be a part of the article information estimation apparatus 10. Further, the person identification information may be input by a person using, for example, an input device (for example, a card reader) attached to the shelf 20.
  • an input device for example, a card reader
  • the present embodiment also improves the discrimination accuracy of the article 200 taken out from the shelf 20. Further, the registered article storage unit 60 stores the article identification information of the article 200 taken out by a person in association with the person identification information of the person. Therefore, it is possible to confirm who took out which article 200.
  • FIG. 8 is a plan view for explaining the layout of the weight sensor 30 according to this embodiment.
  • a plurality of weight sensors 30 are provided separately from each other on one shelf 20 or partial area 22 (hereinafter, referred to as shelf 20).
  • the shelf 20 has a rectangular shape, and the weight sensors 30 are provided at the four corners of the shelf 20, respectively.
  • the weight change data used by the output unit 120 is based on changes in the detection values of the plurality of weight sensors 30.
  • the weight change data indicates the time transition of the detection values of the plurality of weight sensors 30.
  • the output unit 120 of the article information estimation device 10 determines that the article 200 on the shelf 20 has been taken out when the change of the detected values of the plurality of weight sensors 30 satisfies the standard.
  • the output unit 120 determines that the article 200 is taken out when the total value of the reduction amounts of the detection values of the plurality of weight sensors 30 satisfies the standard.
  • the output unit 120 determines at which position of the shelf 20 the article 200 is taken out by using the amount of decrease in the detected values of the plurality of weight sensors 30.
  • the weight sensor identification information of the plurality of weight sensors 30 provided on the same shelf 20 is associated with each other in the shelving allocation information storage unit 130 and managed as a set of weight sensors 30.
  • the weight sensor identification information of a plurality of weight sensors 30 provided on the same shelf 20 is associated with information for identifying the shelf 20 from other shelves 20, for example, shelf position information. Therefore, the output unit 120 can perform the above-mentioned processing by using the information stored in the shelf allocation information storage unit 130.
  • the article information estimation device 10 first identifies a set of weight sensors 30 whose changes in the detected values satisfy the criteria (step S102 in FIG. 4). Next, the identification process of the article is executed using the detection result of the identified set of the weight sensors 30 (step S104 in FIG. 4 ).
  • FIG. 9 is a flowchart for explaining the details of the article identification process (step S104 in FIG. 4) in this embodiment.
  • the output unit 120 reads out shelf position information corresponding to the weight sensor identification information of the weight sensor 30 specified in step S102 (step S222).
  • the output unit 120 estimates the position of the shelf 20 where the weight change has occurred, that is, the position where the taken-out article 200 has been placed, by using the change of the detected values of the plurality of weight sensors 30. For example, the output unit 120 regards the amount of change of each weight sensor 30 as a weight, and estimates the position serving as the center of gravity of these weights as the above-mentioned position (step S224).
  • the output unit 120 specifies the height of the hand and the direction in which the hand is extended by using the data transmitted from the depth sensor 40. For example, when the depth sensor 40 outputs a depth map in which the height information is two-dimensionally displayed, the output unit 120 uses the depth map to identify the height and direction of the hand (step S226).
  • the output unit 120 determines whether or not the relationship between the hand height and the shelf position information satisfies the standard, and the relationship between the hand direction and the position of the article 200 specified in step S224 satisfies the standard.
  • the determination as to whether or not the relationship between the height of the hand and the shelf position information satisfies the criteria is the same as the determination described in step S206 of FIG.
  • the relationship between the direction of the hand and the position of the article 200 for example, when the direction of the hand overlaps the position of the article 200 or when the shortest distance is equal to or less than the reference value, it is determined that the criterion is satisfied (step S228).
  • step S230 the output unit 120 estimates the article 200 taken out by the person.
  • step S230 is the same as the process performed in step S208 of FIG.
  • step S232 the output unit 120 performs alert processing (step S232).
  • the process performed in step S232 is similar to the process performed in step S210 of FIG.
  • the present embodiment also improves the discrimination accuracy of the article 200 taken out from the shelf 20.
  • the article information estimation device 10 not only has the relationship between the hand height and the shelf position information (that is, the relationship in the height direction) but also the relationship between the hand direction and the position of the article 200 (that is, the relationship in the horizontal plane). It is used when estimating the article 200. Therefore, the discrimination accuracy of the article 200 taken out from the shelf 20 is further improved.
  • FIG. 10 is a diagram showing a functional configuration of the article information estimation device 10 according to the present embodiment together with a usage environment of the article information estimation device 10.
  • the article information estimation apparatus 10 according to the present embodiment repeatedly acquires images from the first imaging unit 70 (hereinafter referred to as the first image), and uses the first images to identify the article 200, except that the article 200 is specified.
  • the article information estimating apparatus 10 according to any one of the first to third embodiments has the same configuration.
  • FIG. 10 shows a case similar to that of the first embodiment.
  • the first imaging unit 70 includes at least a part of the space in front of the shelf, which is a space in front of the shelf 20, in the imaging area. Therefore, the first image generated by the first imaging unit 70 includes at least a part of the space in front of the shelf and includes the article 200 taken out from the shelf 20.
  • the output unit 120 estimates the article 200 taken out from the shelf 20 by a person using the image of the article 200 included in the first image.
  • the shelf allocation information storage unit 130 stores the feature amount on the image of the article 200 together with the article identification information. Then, the output unit 120 estimates the article 200 by using the result of collating this feature amount with the first image.
  • FIG. 11 is a flowchart for explaining the details of step S104 in this embodiment.
  • the processing performed in steps S202, S204, S206, S208, and S210 is as described with reference to FIG.
  • the output unit 120 reads the feature amount of the article 200 together with the article specifying information.
  • the output unit 120 processes the first image within the reference time (for example, within 10 seconds) after the detection value of the weight sensor 30 changes, and extracts the feature amount of the article 200 included in the first image.
  • the extracted feature amount and the feature amount read in step S208 match, for example, when the score is equal to or higher than the reference value (step S209: Yes)
  • the article specifying information read in step S208 is kept as it is. Output.
  • alert processing is performed (step S210).
  • step S209 is performed after step S230 in FIG.
  • the estimation accuracy of the article 200 taken out by the person from the shelf 20 is improved.
  • the first image includes the article 200 taken out by the person.
  • the output unit 120 of the article information estimation device 10 further verifies the article 200 estimated by the detected values of the depth sensor 40 and the weight sensor 30 by using the first image. Therefore, the estimation accuracy of the article 200 is further improved.
  • FIG. 12 is a diagram showing a functional configuration of the article information estimation device 10 according to the present embodiment together with a usage environment of the article information estimation device 10.
  • the article information estimation device 10 according to the present embodiment repeatedly acquires an image (hereinafter referred to as a second image) from the second image capturing unit 80, except that the article 200 is identified using the plurality of second images.
  • the article information estimating apparatus 10 according to any one of the first to fourth embodiments has the same configuration.
  • FIG. 12 shows a case similar to the fourth embodiment.
  • the second imaging unit 80 images the shelf 20 from the front (for example, diagonally above the front). Therefore, the second image includes the article 200 placed on the shelf 20. Further, when the second image capturing unit 80 captures an image obliquely above the shelf 20 from the front, the article 200 located at the back of the shelf 20 can be captured. Then, the output unit 120 of the article information estimating apparatus 10 estimates the article 200 taken out by the person from the shelf 20 by further using the change in the second image. Specifically, the output unit 120 does not detect the second image before the depth sensor 40 detects the human hand (that is, before the person comes to the space in front of the shelf) and the depth sensor 40 does not detect the human hand. The article 200 is estimated using the difference between the second images from (that is, after a person leaves the front shelf space).
  • FIG. 13 is a flowchart for explaining an operation example of the article information estimation device 10 according to this embodiment. The processing performed in steps S202, S204, S206, S208, S209, and S210 is as described with reference to FIG.
  • step S209 Yes
  • step S212 the output unit 120 of the article information estimation device 10 matches the feature amount of the article 200 included in the first image with the feature amount of the article read from the shelf allocation information storage unit 130.
  • step S212 the output unit 120 executes the correction (step S214).
  • the output unit 120 may output the second image before the depth sensor 40 detects the hand of the person (that is, before the person comes to the space in front of the shelf) and after the depth sensor 40 stops detecting the hand of the person (that is, the shelf).
  • the output unit 120 may output the second image before the depth sensor 40 detects the hand of the person (that is, before the person comes to the space in front of the shelf) and after the depth sensor 40 stops detecting the hand of the person (that is, the shelf).
  • the output unit 120 does not output the article identification information.
  • the article specific information of the article 200 is not stored in the registered article storage unit 60 (step S214).
  • the output unit 120 stores in advance a combination of the detection result of the weight sensor 30, the detection result of the depth sensor 40, and the processing result of the second image for each movement pattern of the article 200 by a person. Then, when the result corresponding to this combination is detected, the output unit 120 estimates that a movement pattern corresponding to the combination has occurred.
  • step S212 No
  • the output unit 120 outputs the article specifying information read in step S208.
  • this article identification information is used for the settlement processing of the product in the store, for example, as described in the second embodiment.
  • the estimation accuracy of the article 200 taken out by the person from the shelf 20 is improved. Further, the output unit 120 of the article information estimation device 10 identifies the article 200 moved in the shelf 20. For this reason, when a person moves the article 200 in the shelf 20, it is possible to prevent the person from erroneously recognizing that the article 200 is taken out.
  • Weight change data which is data based on a change in a detection value of a weight sensor provided on a shelf on which a plurality of articles can be placed, and movement of a person's hand located in the front shelf space which is a space in front of the shelf.
  • Operation data indicating, acquisition means for acquiring, and An output unit that outputs, using the weight change data and the operation data, the item identification information of the item estimated to be taken out by the hand,
  • An article estimation device including.
  • the acquisition unit generates the operation data by using a detection value of a depth sensor including the front shelf space in a detection range, When the relationship between the height of the shelf in which the detection value of the weight sensor has changed and the detection value of the depth sensor that can acquire the operation data satisfies a criterion, the output unit is associated with the shelf.
  • An article estimation device that outputs the article identification information. 3.
  • There are multiple shelves with different heights, At least one of the article identification information is associated with each of the plurality of shelves.
  • the weight change data is an article estimation device showing the weight change for each of the plurality of shelves. 4. In the article estimation device according to any one of 1 to 3 above.
  • the output device is an article estimation device that outputs the article specifying information associated with the partial areas in which a change in weight in a plurality of partial areas of the shelf satisfies a reference. 5.
  • the article estimation device further comprising a storage processing unit that stores the article identification information output by the output unit in association with the person identification information that identifies the person in the storage unit. 6.
  • the storage processing means is an article estimation device that acquires the person identifying information from a person tracking device that tracks the movement of the person. 7. In the article estimation device according to any one of 1 to 6 above.
  • the acquisition unit repeatedly acquires a first image that is an image including at least a part of the front shelf space, The output means further estimates the article taken out by the hand by using the image of the article included in the first image.
  • the acquisition means repeatedly acquires a second image, which is an image of the shelf captured from the front.
  • the output unit further uses the change in the second image to specify the item specifying information of the item taken out by the hand.
  • a plurality of weight sensors are provided on the shelf at a distance from each other.
  • the acquisition means is an article estimation device that generates the weight change data using detection values of the plurality of weight sensors. 10.
  • the shelves are installed in stores, The person is a customer, Settlement processing means for performing settlement processing using the article specifying information output by the output means, An electronic receipt output means that outputs an electronic receipt based on the settlement process, An article estimation device including.
  • Computer Weight change data which is data based on a change in a detection value of a weight sensor provided on a shelf on which a plurality of articles can be placed, and movement of a person's hand located in the front shelf space which is a space in front of the shelf. And the operation data indicating An article estimation method for outputting the article identification information of the article estimated to be taken out by the hand using the weight change data and the motion data.
  • the computer is The operation data is generated by using the detection value of the depth sensor including the space in front of the shelf in the detection range.
  • the article specifying information associated with the shelf is displayed.
  • At least one of the article identification information is associated with each of the plurality of shelves.
  • the weight change data is an article estimation method showing a weight change for each of the plurality of shelves. 14.
  • the computer according to claim 1, wherein the computer outputs the product identification information associated with the partial regions in which a change in weight in a plurality of partial regions of the shelf satisfies a criterion.
  • the computer is an article estimating method for storing the output article specifying information in a storage unit in association with the person specifying information for identifying the person. 16.
  • the article estimating method wherein the computer acquires the person identifying information from a person tracking device that tracks the movement of the person. 17.
  • the computer is The first image, which is an image including at least a part of the space in front of the shelf, is repeatedly acquired. Furthermore, the article estimation method of estimating the article taken out by the hand using the image of the article included in the first image. 18. In the article estimation method according to any one of 12 to 16 above, The computer repeatedly acquires a second image that is an image of the shelf taken from the front, and further uses a change in the second image to specify the product identification information of the product taken out by the hand. Method. 19. In the article estimation method according to any one of 11 to 18 above, A plurality of weight sensors are provided on the shelf at a distance from each other.
  • the computer is an article estimating method for generating the weight change data by using detection values of the plurality of weight sensors.
  • the shelves are installed in stores, The person is a customer, The computer according to claim 1, wherein the computer performs a settlement process using the output item specifying information, and outputs an electronic receipt based on the settlement process.
  • Weight change data which is data based on a change in a detection value of a weight sensor provided on a shelf on which a plurality of articles can be placed, and movement of a person's hand located in the front shelf space which is a space in front of the shelf.
  • a function to obtain the operation data A function of using the weight change data and the operation data to output the article identification information of the article estimated to be taken out by the hand; A program to have. 22.
  • Output function A program to have. 23.
  • At least one of the article identification information is associated with each of the plurality of shelves.
  • the weight change data is a program showing the weight change for each of the plurality of shelves. 24.
  • the computer has a function of repeatedly acquiring a second image that is an image of the shelf taken from the front, and further using the change of the second image to specify the item specifying information of the item taken out by the hand.
  • a plurality of weight sensors are provided on the shelf at a distance from each other.
  • the shelves are installed in stores, The person is a customer, A program for causing the computer to have a function of performing a settlement process using the output article specifying information and outputting an electronic receipt based on the settlement process.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Accounting & Taxation (AREA)
  • Multimedia (AREA)
  • Finance (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Human Computer Interaction (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Electromagnetism (AREA)
  • Development Economics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Geometry (AREA)
  • Mathematical Physics (AREA)
  • Mechanical Engineering (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Image Analysis (AREA)
PCT/JP2020/006860 2019-03-01 2020-02-20 物品推定装置、物品推定方法、及びプログラム Ceased WO2020179480A1 (ja)

Priority Applications (7)

Application Number Priority Date Filing Date Title
US17/434,813 US11922391B2 (en) 2019-03-01 2020-02-20 Article deduction apparatus, article deduction method, and program
JP2021503960A JP7435587B2 (ja) 2019-03-01 2020-02-20 物品推定装置、物品推定方法、及びプログラム
US18/231,605 US12147962B2 (en) 2019-03-01 2023-08-08 Article deduction apparatus, article deduction method, and program
US18/231,583 US12141776B2 (en) 2019-03-01 2023-08-08 Article deduction apparatus, article deduction method, and program
US18/233,623 US12141777B2 (en) 2019-03-01 2023-08-14 Article deduction apparatus, article deduction method, and program
JP2024017129A JP7694737B2 (ja) 2019-03-01 2024-02-07 物品推定装置、物品推定方法、及びプログラム
JP2025094722A JP2025123254A (ja) 2019-03-01 2025-06-06 物品推定装置、物品推定方法、及びプログラム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019-037829 2019-03-01
JP2019037829 2019-03-01

Related Child Applications (4)

Application Number Title Priority Date Filing Date
US17/434,813 A-371-Of-International US11922391B2 (en) 2019-03-01 2020-02-20 Article deduction apparatus, article deduction method, and program
US18/231,583 Continuation US12141776B2 (en) 2019-03-01 2023-08-08 Article deduction apparatus, article deduction method, and program
US18/231,605 Continuation US12147962B2 (en) 2019-03-01 2023-08-08 Article deduction apparatus, article deduction method, and program
US18/233,623 Continuation US12141777B2 (en) 2019-03-01 2023-08-14 Article deduction apparatus, article deduction method, and program

Publications (1)

Publication Number Publication Date
WO2020179480A1 true WO2020179480A1 (ja) 2020-09-10

Family

ID=72337272

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/006860 Ceased WO2020179480A1 (ja) 2019-03-01 2020-02-20 物品推定装置、物品推定方法、及びプログラム

Country Status (3)

Country Link
US (4) US11922391B2 (https=)
JP (3) JP7435587B2 (https=)
WO (1) WO2020179480A1 (https=)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022065282A1 (ja) * 2020-09-28 2022-03-31 日本電気株式会社 情報処理装置、システム、情報処理方法、および記録媒体
JP2022127090A (ja) * 2021-02-19 2022-08-31 トヨタ自動車株式会社 棚在庫管理システム、棚在庫管理方法、及びプログラム
JP2022127124A (ja) * 2021-02-19 2022-08-31 トヨタ自動車株式会社 棚在庫管理システム、棚在庫管理方法、及びプログラム
WO2022195752A1 (ja) 2021-03-17 2022-09-22 日本電気株式会社 情報処理装置、情報処理方法、及び、記録媒体
JP2023122059A (ja) * 2022-02-22 2023-09-01 富士通株式会社 情報処理プログラム、情報処理方法および情報処理装置
JP2024025061A (ja) * 2022-08-10 2024-02-26 株式会社日立エルジーデータストレージ 物品特定システム、物品特定方法、物品特定プログラム及び物品取得判断システム

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10885642B1 (en) * 2019-10-25 2021-01-05 7-Eleven, Inc. Scalable position tracking system for tracking position in large spaces
WO2021054266A1 (ja) * 2019-09-17 2021-03-25 日本電気株式会社 画像処理装置、画像処理方法、及びプログラム
US11514766B1 (en) * 2020-12-10 2022-11-29 Amazon Technologies, Inc. Detecting interactions with storage units based on RFID signals and auxiliary signals
GB202202511D0 (en) * 2022-02-23 2022-04-06 Henry Dean Charles Communicating weight sensor units, asset tags and dispensing units and enclosures, and techniques for using same

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014194732A (ja) * 2013-03-01 2014-10-09 Toshiba Tec Corp 電子レシート管理サーバ、情報処理装置及びプログラム
US20150039458A1 (en) * 2013-07-24 2015-02-05 Volitional Partners, Inc. Method and system for automated retail checkout using context recognition
JP2018160107A (ja) * 2017-03-23 2018-10-11 日本電気株式会社 決済処理装置、方法およびプログラム
JP2018206372A (ja) * 2018-05-15 2018-12-27 株式会社 ディー・エヌ・エー 商品を管理するためのシステム、方法、及びプログラム

Family Cites Families (40)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10430798B2 (en) * 2002-10-23 2019-10-01 Matthew Volpi System and method of a media delivery services platform for targeting consumers in real time
US20140179231A1 (en) * 2012-12-26 2014-06-26 Cellco Partnership D/B/A Verizon Wireless Smart vending machine
US20150073925A1 (en) * 2013-05-23 2015-03-12 Gavon Augustus Renfroe System and Method for Integrating Business Operations
US10229414B2 (en) * 2013-06-25 2019-03-12 Square, Inc. Mirroring a storefront to a social media site
US10176456B2 (en) * 2013-06-26 2019-01-08 Amazon Technologies, Inc. Transitioning items from a materials handling facility
JP6217373B2 (ja) * 2013-12-13 2017-10-25 富士通株式会社 動作判定方法、動作判定装置および動作判定プログラム
JP2015141572A (ja) * 2014-01-29 2015-08-03 富士通株式会社 商品情報提供方法、商品情報提供装置および商品情報提供プログラム
US10832310B2 (en) * 2014-03-31 2020-11-10 Monticello Enterprises LLC System and method for providing a search entity-based payment process
US10726472B2 (en) * 2014-03-31 2020-07-28 Monticello Enterprises LLC System and method for providing simplified in-store, product-based and rental payment processes
US10078003B2 (en) * 2014-06-04 2018-09-18 Nectar, Inc. Sensor device configuration
US10037662B2 (en) * 2014-09-18 2018-07-31 Indyme Solutions, Inc. Merchandise activity sensor system and methods of using same
US10937289B2 (en) * 2014-09-18 2021-03-02 Indyme Solutions, Llc Merchandise activity sensor system and methods of using same
US9911290B1 (en) * 2015-07-25 2018-03-06 Gary M. Zalewski Wireless coded communication (WCC) devices for tracking retail interactions with goods and association to user accounts
WO2017163909A1 (ja) 2016-03-22 2017-09-28 日本電気株式会社 画像表示装置、画像表示システム、画像表示方法及びプログラム
US10198710B1 (en) * 2016-03-28 2019-02-05 Amazon Technologies, Inc. System to determine a change in weight at a load cell
EP4410155A1 (en) * 2016-05-09 2024-08-07 Grabango Co. System and method for computer vision driven applications within an environment
JP2017210310A (ja) 2016-05-24 2017-11-30 日本電信電話株式会社 物品管理装置、および、物品管理方法
JP2017218289A (ja) 2016-06-08 2017-12-14 株式会社関電工 管理システム
US10339656B1 (en) * 2016-09-29 2019-07-02 Amazon Technologies, Inc. Inferring count of items using image
WO2018071096A1 (en) * 2016-10-13 2018-04-19 Walmart Apollo, Llc Shelf system and associated methods
US20180139570A1 (en) * 2016-11-14 2018-05-17 Symbol Technologies, Llc Arrangement for, and method of, associating an identifier of a mobile device with a location of the mobile device
US20190149725A1 (en) * 2017-09-06 2019-05-16 Trax Technologies Solutions Pte Ltd. Using augmented reality for image capturing a retail unit
JP7019357B2 (ja) * 2017-09-19 2022-02-15 東芝テック株式会社 棚情報推定装置及び情報処理プログラム
US10963704B2 (en) * 2017-10-16 2021-03-30 Grabango Co. Multiple-factor verification for vision-based systems
US10516982B2 (en) * 2017-10-27 2019-12-24 Hewlett Packard Enterprise Development Lp Match Bluetooth low energy (BLE) moving patterns
US11030442B1 (en) * 2017-12-13 2021-06-08 Amazon Technologies, Inc. Associating events with actors based on digital imagery
US11284041B1 (en) * 2017-12-13 2022-03-22 Amazon Technologies, Inc. Associating items with actors based on digital imagery
US10318569B1 (en) * 2017-12-29 2019-06-11 Square, Inc. Smart inventory tags
US10885336B1 (en) * 2018-01-13 2021-01-05 Digimarc Corporation Object identification and device communication through image and audio signals
JP7088281B2 (ja) * 2018-03-09 2022-06-21 日本電気株式会社 商品分析システム、商品分析方法および商品分析プログラム
GB2585800B (en) * 2018-03-27 2022-05-18 Teledyne FLIR LLC People counting and tracking systems and methods
US10872221B2 (en) * 2018-06-21 2020-12-22 Amazon Technologies, Inc Non-contact biometric identification system
US11017238B2 (en) * 2018-06-25 2021-05-25 Shopify Inc. Capturing transactional context
US11301984B1 (en) * 2018-06-28 2022-04-12 Amazon Technologies, Inc. System to determine user interaction with fixture
US11468681B1 (en) * 2018-06-28 2022-10-11 Amazon Technologies, Inc. Associating events with actors using digital imagery and machine learning
CN108921098B (zh) 2018-07-03 2020-08-18 百度在线网络技术（北京）有限公司 人体运动分析方法、装置、设备及存储介质
US10282852B1 (en) * 2018-07-16 2019-05-07 Accel Robotics Corporation Autonomous store tracking system
US11085809B1 (en) * 2018-12-03 2021-08-10 Amazon Technologies, Inc. Multi-channel weight sensing system
JP7391513B2 (ja) * 2019-01-17 2023-12-05 東芝テック株式会社 商品登録装置及び情報処理プログラム
US10805556B1 (en) * 2019-01-22 2020-10-13 Amazon Technologies, Inc. Storage units with shifted-lens cameras

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014194732A (ja) * 2013-03-01 2014-10-09 Toshiba Tec Corp 電子レシート管理サーバ、情報処理装置及びプログラム
US20150039458A1 (en) * 2013-07-24 2015-02-05 Volitional Partners, Inc. Method and system for automated retail checkout using context recognition
JP2018160107A (ja) * 2017-03-23 2018-10-11 日本電気株式会社 決済処理装置、方法およびプログラム
JP2018206372A (ja) * 2018-05-15 2018-12-27 株式会社 ディー・エヌ・エー 商品を管理するためのシステム、方法、及びプログラム

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022065282A1 (ja) * 2020-09-28 2022-03-31 日本電気株式会社 情報処理装置、システム、情報処理方法、および記録媒体
JPWO2022065282A1 (https=) * 2020-09-28 2022-03-31
JP7670064B2 (ja) 2020-09-28 2025-04-30 日本電気株式会社 情報処理装置、方法、およびプログラム
JP7533272B2 (ja) 2021-02-19 2024-08-14 トヨタ自動車株式会社 棚在庫管理システム、棚在庫管理方法、及びプログラム
JP7533273B2 (ja) 2021-02-19 2024-08-14 トヨタ自動車株式会社 棚在庫管理システム、棚在庫管理方法、及びプログラム
JP2022127124A (ja) * 2021-02-19 2022-08-31 トヨタ自動車株式会社 棚在庫管理システム、棚在庫管理方法、及びプログラム
JP2022127090A (ja) * 2021-02-19 2022-08-31 トヨタ自動車株式会社 棚在庫管理システム、棚在庫管理方法、及びプログラム
WO2022195752A1 (ja) 2021-03-17 2022-09-22 日本電気株式会社 情報処理装置、情報処理方法、及び、記録媒体
US12039510B2 (en) 2021-03-17 2024-07-16 Nec Corporation Information processing apparatus, information processing method, and storage medium
US12293346B2 (en) 2021-03-17 2025-05-06 Nec Corporation Information processing apparatus, information processing method, and storage medium
JP2023122059A (ja) * 2022-02-22 2023-09-01 富士通株式会社 情報処理プログラム、情報処理方法および情報処理装置
JP7760931B2 (ja) 2022-02-22 2025-10-28 富士通株式会社 情報処理プログラム、情報処理方法および情報処理装置
JP2024025061A (ja) * 2022-08-10 2024-02-26 株式会社日立エルジーデータストレージ 物品特定システム、物品特定方法、物品特定プログラム及び物品取得判断システム
JP7777507B2 (ja) 2022-08-10 2025-11-28 株式会社日立エルジーデータストレージ 物品特定システム、物品特定方法、物品特定プログラム及び物品取得判断システム

Also Published As

Publication number Publication date
JP7694737B2 (ja) 2025-06-18
US20230385799A1 (en) 2023-11-30
JP2025123254A (ja) 2025-08-22
US20220101296A1 (en) 2022-03-31
JP7435587B2 (ja) 2024-02-21
JP2024040297A (ja) 2024-03-25
US20230385798A1 (en) 2023-11-30
US12141776B2 (en) 2024-11-12
US20230401552A1 (en) 2023-12-14
JPWO2020179480A1 (https=) 2020-09-10
US12141777B2 (en) 2024-11-12
US11922391B2 (en) 2024-03-05
US12147962B2 (en) 2024-11-19

Similar Documents

Publication Publication Date Title
JP7694737B2 (ja) 物品推定装置、物品推定方法、及びプログラム
JP7586161B2 (ja) 情報処理システム、商品推薦方法、およびプログラム
JP5731766B2 (ja) 販売機会損失の分析システム及び分析方法
JP2018206372A (ja) 商品を管理するためのシステム、方法、及びプログラム
JP2020017253A (ja) 商品管理システム及び商品管理方法
JP5571633B2 (ja) 健康度報知装置、プログラム及び健康度報知方法
US20250362171A1 (en) Footfall detection method and apparatus
US20200320552A1 (en) Sales analysis apparatus, sales management system, sales analysis method, and program recording medium
JP7130945B2 (ja) 在庫検出プログラム、在庫検出方法及び在庫検出装置
US20170330206A1 (en) Motion line processing system and motion line processing method
JP2021080087A (ja) 情報処理システムと情報処理装置と情報処理プログラムと情報処理方法
JP2016219065A (ja) 滞留分析システム及び方法
KR20230046978A (ko) 컴퓨터 판독 가능한 기억 매체에 저장된 통지 프로그램, 통지 방법 및 정보 처리 장치
JP7619427B2 (ja) 処理装置、処理方法及びプログラム
CN112154488B (zh) 信息处理装置、控制方法和程序
JP2015133131A (ja) 販売機会損失分析用データ出力システム及び方法
JP7697587B2 (ja) 商品数特定装置、商品数特定方法、及びプログラム
JP2019207717A (ja) 販売分析装置、販売管理システム、販売分析方法、及びプログラム
JP6734891B2 (ja) 販売実績の分析システムおよび分析方法
WO2025017920A1 (ja) 情報処理装置、支援方法、非一時的なコンピュータ可読記録媒体および店舗システム
WO2026038304A1 (ja) 情報処理装置、情報処理方法、及び記録媒体
JP2026050409A (ja) 精算装置、精算方法、及びプログラム
WO2022269755A1 (ja) 情報処理装置、死に筋商品判定支援方法およびプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20765971

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021503960

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20765971

Country of ref document: EP

Kind code of ref document: A1