US20240020672A1 - Intelligent weighing apparatus based on computer vision technology, and intelligent weighing method - Google Patents

Intelligent weighing apparatus based on computer vision technology, and intelligent weighing method Download PDF

Info

Publication number
US20240020672A1
US20240020672A1 US18/254,408 US202118254408A US2024020672A1 US 20240020672 A1 US20240020672 A1 US 20240020672A1 US 202118254408 A US202118254408 A US 202118254408A US 2024020672 A1 US2024020672 A1 US 2024020672A1
Authority
US
United States
Prior art keywords
goods
information
identification
weight
visual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
US18/254,408
Other languages
English (en)
Inventor
Ping Wei
Hao Li
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ronsson Technology Co Ltd
Original Assignee
Ronsson Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ronsson Technology Co Ltd filed Critical Ronsson Technology Co Ltd
Publication of US20240020672A1 publication Critical patent/US20240020672A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0009—Details of the software in the checkout register, electronic cash register [ECR] or point of sale terminal [POS]
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0036—Checkout procedures
    • G07G1/0045—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader
    • G07G1/0054—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader with control of supplementary check-parameters, e.g. weight or number of articles
    • G07G1/0072—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader with control of supplementary check-parameters, e.g. weight or number of articles with means for detecting the weight of the article of which the code is read, for the verification of the registration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/08—Payment architectures
    • G06Q20/20—Point-of-sale [POS] network systems
    • G06Q20/208—Input by product or record sensing, e.g. weighing or scanner processing
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01G—WEIGHING
    • G01G19/00—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups
    • G01G19/40—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups with provisions for indicating, recording, or computing price or other quantities dependent on the weight
    • G01G19/413—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups with provisions for indicating, recording, or computing price or other quantities dependent on the weight using electromechanical or electronic computing means
    • G01G19/414—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups with provisions for indicating, recording, or computing price or other quantities dependent on the weight using electromechanical or electronic computing means using electronic computing means only
    • G01G19/4144—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups with provisions for indicating, recording, or computing price or other quantities dependent on the weight using electromechanical or electronic computing means using electronic computing means only for controlling weight of goods in commercial establishments, e.g. supermarket, P.O.S. systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]
    • G06F30/20—Design optimisation, verification or simulation
    • G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/08—Payment architectures
    • G06Q20/18—Payment architectures involving self-service terminals [SST], vending machines, kiosks or multimedia terminals
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/38—Payment protocols; Details thereof
    • G06Q20/387—Payment using discounts or coupons
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/02—Marketing; Price estimation or determination; Fundraising
    • G06Q30/0207—Discounts or incentives, e.g. coupons or rebates
    • G06Q30/0238—Discounts or incentives, e.g. coupons or rebates at point-of-sale [POS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/94—Hardware or software architectures specially adapted for image or video understanding
    • G06V10/945—User interactive design; Environments; Toolboxes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/68—Food, e.g. fruit or vegetables
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0018—Constructional details, e.g. of drawer, printing means, input means
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/12—Cash registers electronically operated
    • G07G1/14—Systems including one or more distant stations co-operating with a central processing unit

Definitions

  • the application relates to the technical field of information processing, and in particular, relates to an intelligent weighing apparatus and an intelligent weighing method based on computer vision technology.
  • the main settlement method is to identify goods by scanning goods bar codes, and then perform goods settlement.
  • This goods identification and settlement method has relatively low efficiency, which leads to a long queue in supermarkets, convenience stores and other places with large traffic.
  • the embodiments of the present disclosure provides an intelligent weighing apparatus and method based on computer vision technology, which can conveniently settle goods according to goods identification results and weighing results, thereby improving the efficiency of goods settlement and reducing the labor cost consumed in goods settlement.
  • an intelligent weighing apparatus based on computer vision technology which includes:
  • system configuration component is configured to set hardware parameters including different hardware model selections of the weighing platform and the price tag printer.
  • the system configuration component is configured to set hardware parameters for setting the visual sensor, including a preview resolution, whether to cut a main body area out, and parameters for cutting the main body area out.
  • the system configuration component is configured to set the hardware parameters including tare setting for the weighing platform, including three modes of not subtracting tare weight, one-time subtracting tare weight and continuously subtracting tare weight.
  • system configuration component is configured to set software parameters including merchant selection, store selection and goods management, herein the goods management includes goods information of name, code, price and illustrative figures for browsing and editing.
  • the first trigger includes automatically triggering the visual sensor to collect the visual information of the goods placed on the weighing platform when the weighing platform senses that a weight changes and the weight is not zero
  • the second trigger includes automatically triggering the identifier to identify goods when the weighing platform senses that a weight changes and the weight is not zero
  • the third trigger includes the weight signal being stable so that a stable weight signal is taken as goods weight information for calculating a total price of goods and generating a goods weighing bar code.
  • the visual sensor is configured to collect visual information of goods placed on the weighing platform, herein the visual information includes at least one of images and videos, and the visual sensor is configured to collect visual information for building models for new goods, and to collect visual information for goods to be identified, herein when collecting visual information for new goods, the visual sensor is configured to carry out capturing modes including a single-capturing-a-photo-mode, a continuous-capturing-multiple-photos-mode and a video-capturing-mode, and when collecting visual information for goods to be identified, the visual sensor is configured to carry out capturing modes including a single-capturing-a-photo-mode, a continuous-capturing-multiple-photos-mode and a video-capturing-mode.
  • the second trigger includes a changes of a content of a visual signal collected by the visual sensor to trigger an identifier to identify goods
  • the third trigger includes stabilization of the visual signal to terminate goods identification.
  • the visual sensor is located above or on a side of the weighing platform, so as to be able to perceive visual information of goods on the weighing platform with a definition higher than a predetermined definition.
  • the hardware parameters of the visual sensor are set to cut a main body area out
  • the main body area of the collected visual information is cut out according to parameters for cutting the main body area out.
  • the second trigger includes a manual selection.
  • the identification model is configured to perform a conditional filtering or statistical regression to filter out candidate identification results whose weight is greater than a maximum goods weight or less than a minimum goods weight by using a computer vision technology based on deep learning, by considering goods weight information.
  • the identifier is configured to complete identification calculation of the goods identification by using a computing unit in the intelligent weighing apparatus, an adjacent edge computing unit, or a cloud computing unit.
  • the cashing POS system includes a POS system of a traditional cash register and a POS system of a self-help cash register.
  • the identification feedback device is configured to adopt an adaptive display strategy based on an identification confidence when displaying candidate goods information, herein when the identification confidences of goods identification results are higher than a predetermined confidence, only goods identification results of a number lower than a predetermined number with a confidence higher than the predetermined confidence are displayed as candidate goods information, and when the identification confidences of goods identification results are lower than the predetermined confidence, the number of displayed goods identification results is increased as candidate goods information.
  • identification recall rate is guaranteed, and the difficulty of selection is reduced.
  • interactive behaviors of receiving user feedback by using the identification feedback device include clicking to confirm, re-identifying, searching, screening, modifying the price and returning to the system configuration; the interactive behaviors of user feedback include automatically confirming the goods identification results under predetermined conditions, and the user feedback is automatically transmitted to the price tag printer or cashing POS system and fed back to the modeling platform, herein the predetermined conditions include that the identification confidence exceeds an automatic publishing threshold, or a user feedback time exceeds a waiting threshold.
  • the identification feedback device includes a search component, which is configured for the user to search goods by using initial letters of goods names or codes, and simultaneously displays trending search candidate goods.
  • the identification feedback device includes a screening component, the screening component is configured to preferentially display a list of clearance goods of the same or related categories according to the goods identification results.
  • the identification feedback device includes a price modification component
  • the price modification component is configured to receive an interactive action of long-pressing a goods information display card of candidate goods information or an element of goods icon and name in the card, and trigger a pop-up price modification tag, so that the user can modify a unit price, a total price or a discount coefficient of the goods.
  • the selected good is a piece-based goods
  • the quantity of the goods is fed back through an automatic pop-up window.
  • the confirmed goods information when the confirmed goods information needs to be transmitted to the cashing POS system, the confirmed goods information is transmitted to the POS system through a wired or wireless network, or when the confirmed goods information needs to be transmitted to the price tag printer, the confirmed goods information is transmitted to the price tag printer through a wired network, or when the confirmed goods information needs to be fed back to the modeling platform, the confirmed goods information is uploaded to the modeling platform in a cloud through a wired or wireless network.
  • the user when it is required to return to a system setting interface, the user can return to the system setting interface by sliding on the screen from left to right or from top to bottom.
  • the modeling platform is configured to use a computer vision technology based on deep learning to train the identification model based on the collected visual information, the weight information and the data of the user feedback in use, herein the goods weight information is applied to set a weight filtering rule to filter out the identification results of goods whose weight is greater than the maximum goods weight or less than the minimum goods weight, or is applied to statistical goods weight distribution models to assist goods identification; the data of the user feedback in use is applied to the continuous iterative optimization of the identification model; and the trained identification model is distributed to different computing platforms, such as terminal, edge computing or cloud computing platforms.
  • the goods information synchronization component is configured to update the updated goods information into the intelligent weighing apparatus to support the display of candidate goods information
  • the goods information synchronization component including two forms of a Windows® program and a mobile phone APP, completes the update of goods information through database connecting, file uploading and manual editing, and supports the update of goods information in the intelligent weighing apparatus through two connection modes of a merchant intranet and an extranet.
  • the intelligent weighing apparatus does not include a cashing POS system and a price tag printer, and is connected as a whole with an external cashing POS system; or the intelligent weighing apparatus includes a cashing POS system; or the intelligent weighing apparatus includes a price tag printer.
  • an intelligent weighing method for an intelligent weighing apparatus based on computer vision technology including:
  • system configuration component is configured to set hardware parameters including selection of different hardware models of the weighing platform and the price tag printer.
  • the system configuration component is configured to set hardware parameters of the visual sensor, including a preview resolution, whether to cut a main body area out, and parameters for cutting the main body area out.
  • the system configuration component is configured to set hardware parameters for tare weight setting of the weighing platform, including three modes of not subtracting tare weight, one-time subtracting tare weight and continuously subtracting tare weight.
  • system configuration component is configured to set software parameters including merchant selection, store selection and goods management, herein the goods management includes goods information of name, code, price and illustrative figures for browsing and editing.
  • the first trigger includes automatically triggering the visual sensor to collect the visual information of the goods placed on the weighing platform when the weighing platform senses that a weight changes and the weight is not zero
  • the second trigger includes automatically triggering the identifier to identify goods when the weighing platform senses that a weight changes and the weight is not zero
  • the third trigger includes the weight signal being stable so that a stable weight signal is taken as goods weight information for calculating a total price of goods and generating a goods weighing bar code.
  • the visual sensor is configured to collect visual information of goods placed on the weighing platform, herein the visual information includes at least one of images and videos, and the visual sensor is configured to collect visual information for building models for new goods; and collect visual information for the goods to be identified, herein the visual sensor is configured to carry out a capturing mode including a single-capturing-a-photo-mode, a continuous-capturing-multiple-photos-mode and a video-capturing-mode when collecting visual information for building models for new goods, and is configured to carry out a capturing mode including a single-capturing-a-photo-mode, a continuous-capturing-multiple-photos-mode and a video-capturing-mode when collecting visual information for the goods to be identified.
  • a capturing mode including a single-capturing-a-photo-mode, a continuous-capturing-multiple-photos-mode and a video-capturing-mode when collecting visual information for the goods to be identified.
  • the second trigger includes a changes of a content of a visual signal collected by the visual sensor to trigger the identifier to identify the goods
  • the third trigger includes stabilization of the visual signals to terminate the identification of the goods
  • the visual sensor is located above or on a side of the weighing platform, so as to be able to perceive visual information of goods on the weighing platform with a definition higher than a predetermined definition, and when the hardware parameters of the visual sensor are set to cut the main body area out, the main body area of the collected visual information is cut according to the parameters for cutting the main body area out.
  • the second trigger includes a manual selection.
  • the identification model is configured to perform a conditional filtering or statistical regression to filter out candidate identification results whose weight is greater than a maximum goods weight or less than a minimum goods weight by using a computer vision technology based on deep learning, combined with goods weight information.
  • the identifier is configured to complete identification calculation of the goods identification by using a computing unit in the intelligent weighing apparatus, an adjacent edge computing unit, or a cloud computing unit.
  • the cashing POS system includes a POS system of a traditional cash register and a POS system of a self-help cash register.
  • the identification feedback device is configured to adopt an adaptive display strategy based on an identification confidence when displaying candidate goods information, herein when the identification confidences of goods identification results are higher than a predetermined confidence, only goods identification results of a number lower than a predetermined number with a confidence higher than the predetermined confidence are displayed as candidate goods information, and when the identification confidences of goods identification results are lower than the predetermined confidence, the number of displayed goods identification results is increased as candidate goods information.
  • interactive behaviors of receiving user feedback by using the identification feedback device include clicking to confirm, re-identifying, searching, screening, modifying the price and returning to the system configuration; the interactive behaviors of user feedback include automatically confirming the goods identification results under predetermined conditions, and the user feedback is automatically transmitted to the price tag printer or cashing POS system and fed back to the modeling platform, herein the predetermined conditions include that the identification confidence exceeds an automatic publishing threshold, or a user feedback time exceeds a waiting threshold.
  • the identification feedback device includes a search component, which enables users to search goods by using initials letters of goods names or codes, and simultaneously displays trending search candidate goods.
  • the identification feedback device includes a screening component, and the screening component is configured to preferentially display a list of clearance goods of the same or related categories according to the goods identification results.
  • the identification feedback device includes a price modification component
  • the price modification component is configured to receive an interactive action of long-pressing a goods information display card of candidate goods information or an element of goods icon and name in the card, and trigger a pop-up price modification tag, so that the user can modify a unit price, a total price or a discount coefficient of the goods.
  • the selected good is a piece-based goods
  • the quantity of the goods is fed back through an automatic pop-up window.
  • the confirmed goods information when the confirmed goods information needs to be transmitted to the cashing POS system, the confirmed goods information is transmitted to the POS system through a wired or wireless network, or when the confirmed goods information needs to be transmitted to the price tag printer, the confirmed goods information is transmitted to the price tag printer through a wired network, or when the confirmed goods information needs to be fed back to the modeling platform, the confirmed goods information is uploaded to the modeling platform in a cloud through a wired or wireless network.
  • the user when it is required to return to a system setting interface, the user can return to the system setting interface by sliding on the screen from left to right or from top to bottom.
  • the modeling platform is configured to use a computer vision technology based on deep learning to train the identification model based on the collected visual information, the weight information and the data of the user feedback in use, herein the goods weight information is applied to set a weight filtering rule to filter out the identification results of goods whose weight is greater than the maximum goods weight or less than the minimum goods weight, or is applied to statistical goods weight distribution models to assist goods identification; the data of the user feedback in use is applied to the continuous iterative optimization of the identification model; the trained identification model is distributed to different computing platforms, such as terminal, edge computing or cloud computing platforms.
  • the goods information synchronization component is configured to update the updated goods information into the intelligent weighing apparatus to support the display of candidate goods information
  • the goods information synchronization component including two forms of a Windows® program and a mobile phone APP, completes the update of goods information through database connecting, file uploading and manual editing, and supports the update of goods information in the intelligent weighing apparatus through two connection modes of a merchant intranet and an extranet.
  • the intelligent weighing apparatus does not include a cashing POS system and a price tag printer, and is connected as a whole with an external cashing POS system; or the intelligent weighing apparatus includes a cashing POS system; or the intelligent weighing apparatus includes a price tag printer.
  • an intelligent weighing method which includes the following steps: system configuration, for setting the system configuration of software parameters, hardware parameters and goods information; transmission synchronization, for uploading one or more kinds of goods information including obtaining goods name, goods code, weighing apparatus code, price, pricing method and illustrative figure; goods database update, for updating the uploaded information to a database; information collection, for collecting visual information and goods weight information; training an identification model by combining the visual information and goods weight information; delivering the identification model to an identification service; weighing a goods to be weighed that are placed on a weighing platform to obtain weight information; collecting the visual information of goods; identifying types of goods by using the identification model in the identification service; displaying goods information of the candidate identification result; identification feedback, for receiving user feedback information, transmitting a confirmation result to a cashing POS system or a bar code printer, and updating it to a cloud training platform to support iterative updating of the model; and the cashing POS system adding goods information, or the bar code printer printing
  • FIG. 1 schematically illustrates a block diagram of an intelligent weighing apparatus according to one embodiment of the present disclosure
  • FIG. 2 schematically illustrates an external structural diagram of an intelligent weighing apparatus according to an embodiment of the present disclosure
  • FIG. 3 schematically illustrates a flowchart of an intelligent weighing method according to one embodiment of the present disclosure.
  • FIG. 4 schematically illustrates a flowchart of an intelligent weighing method according to another embodiment of the present disclosure.
  • the technology disclosed by the invention can automatically identify and weigh a goods and obtain goods settlement data without manually identifying the goods, weighing and pasting bar codes identifying the goods and the weight on the goods, and without making customers scan bar codes at a cash register.
  • the goods can be intelligently identified by using the image of the goods, the weight of the goods and the goods identification model, and the goods settlement result can be generated.
  • the whole process can be done without manual operations of a full-time weigher, which improves the efficiency of goods weighing and settlement and reduces labor costs consumed in goods weighing and settlement.
  • the technical scheme disclosed by the invention can be used for selling goods in various goods sales places, including but not limited to shopping malls, supermarkets, vegetable markets, bakeries, various retail food stores and the like.
  • the same goods sold in different goods sales places and in different periods may have their own uniqueness, such as oranges, but different varieties of oranges sold in different goods sales places or even oranges of the same variety may be different in appearance, size and weight, or oranges sold in different periods in the same goods sales place may be different in appearance, size and weight.
  • an existing or unchanged goods identification model is used, it may not be able to identify these goods accurately and differently.
  • the existing or unchanged goods identification model may be simple and coarse, such as using some ordinary orange photos to train the model. If this goods identification model is directly applied to specific goods sales places, it may also get wrong goods identification results.
  • FIG. 1 schematically illustrates a block diagram of an intelligent weighing apparatus according to one embodiment of the present disclosure.
  • the intelligent weighing apparatus 100 includes: a system configuration component 101 , configured to set software parameters, hardware parameters and goods information of the intelligent weighing apparatus; a weighing platform 102 , configured to place a goods to be identified and weighed, sense a weight and a weight change, and obtain goods weight information of the goods; a visual sensor 103 , configured to collect visual information of goods placed on the weighing platform in response to a first trigger; an identifier 104 , configured to perform goods identification based on the visual information, the weight information and the identification model in response to a second trigger, and to terminate goods identification in response to a third trigger to obtain goods identification results; an identification feedback device 105 , configured to display candidate goods information by combining the goods identification result with goods information, receive user feedback, and transmit the goods information confirmed by a user feedback to a price tag printer 106 or a cashing POS system 107 , and feed it back to a modeling platform
  • the goods identification model can be gradually established and improved in the actual operation process through the actually collected image of the goods and the confirmation of the goods as user feedback without initially training the image and establishing the goods identification model or with only establishing a simple goods identification model, and the process of weighing and using is the process of learning, thus simplifying the process of machine learning.
  • User feedback is provided to confirm whether the identification result of the goods identification model is accurate and fed back to the goods identification model to further optimize the model, the use process is very simple, and the feeling of customers and goods sales places is better.
  • the goods images collected for the actual goods and the confirmation of the goods may also be used as training images, and further learning can be carried out by using the training mages, so as to strengthen the learning of the goods identification model, improve the accuracy of goods identification, and save the time and cost of learning the training images separately and establishing the goods identification model.
  • the goods sales place can directly train and improve the goods identification model that is adapted to characteristics of the goods sales place while using the product for weighing and cashong.
  • the weight result of weighing may be added to further optimize the goods identification model itself or the identification result of the goods identification model.
  • system configuration component is configured to set hardware parameters including different hardware model selections of the weighing platform and the price tag printer.
  • the system configuration component is configured to set hardware parameters for setting the visual sensor, including a preview resolution, whether to cut the main body area out, and parameters for cutting the main body area out.
  • the preview resolution indicates the resolution of the preview camera image.
  • the main body area is cut out, for example, the area outside the main body in the image of the goods is cut off.
  • the parameters for cutting the main body area out can be used to cut the main body area of the collected image out, for example, the parameters for cutting the main body area out are used to define coordinates of the main body area to be cut out.
  • the parameters for cutting the main body area out are used to define coordinates of the main body area to be cut out.
  • by cutting off the area outside the weighing platform from the image of the goods it is possible to reduce errors of the identified image or the image for training caused by interference outside the weighing platform, such as hands and plastic bags.
  • by identifying foreground of the goods and surrounding background in the image of the goods, and cutting off the surrounding background of the goods from the image of the goods it is possible to filter out the image errors caused by interference around the goods (not just outside the weighing platform) such as hands and plastic bags.
  • the images of goods packed with plastic bags or other packages may also be used as a whole for identification or training.
  • the system configuration component is configured to set hardware parameters of tare setting for the weighing platform, including not subtracting tare weight, one-time subtracting tare weight and continuously subtracting tare weight.
  • Not subtracting tare weight means that the current weighing does not need to subtract the tare weight (the weight of the goods' tare).
  • One-time tare weight refers to subtracting a configured tare weight in current weighing.
  • Continuously subtracting tare weight means subtracting the configured tare weight for each subsequent weighing for the configured goods.
  • system configuration component is configured to set software parameters including merchant selection, store selection and goods management, herein the goods management includes goods information of name, code, price and illustrative figures for browsing and editing.
  • the first trigger includes automatically triggering the visual sensor to collect the visual information of the goods placed on the weighing platform when the weighing platform senses that a weight changes and the weight is not zero
  • the second trigger includes automatically triggering the identifier to identify goods when the weighing platform senses that a weight changes and the weight is not zero
  • the third trigger includes the weight signal being stable so that a stable weight signal is taken as goods weight information for calculating a total price of goods and generating a goods weighing bar code.
  • the visual sensor starts to collect the visual information of the goods
  • the identifier starts to identify the goods. This can avoid the visual sensor from collecting the interference images of things that do not need to be weighed, such as shaking hands, and save the power of the visual sensor and the identifier.
  • the visual sensor is configured to collect visual information of goods placed on the weighing platform, herein the visual information includes at least one of images and videos, and the visual sensor is configured to collect visual information for building models for new goods, and to collect visual information for goods to be identified, herein when collecting visual information for new goods, the visual sensor is configured to carry out capturing modes including a single-capturing-a-photo-mode, a continuous-capturing-multiple-photos-mode and a video-capturing-mode, and when collecting visual information for goods to be identified, the visual sensor is configured to carry out capturing modes including a single-capturing-a-photo-mode, a continuous-capturing-multiple-photos-mode and a video-capturing-mode.
  • the second trigger includes a change of a content of the collected visual signal to trigger the identifier to identify goods
  • the third trigger includes stabilization of the visual signal to terminate the identification of goods.
  • the visual sensor is located above or on a side of the weighing platform, so as to sense the visual information of goods on the weighing platform with a definition higher than a predetermined definition.
  • the hardware parameters of the visual sensor are set to cut the main body area out, the main body area of the collected visual information is cut out according to the parameters for cutting the main body area out.
  • the second trigger includes a manual selection. That is, the goods identification of the identifier is manually started.
  • the identification model is configured to filter out candidate identification results whose weight is greater than the maximum goods weight or less than the minimum goods weight, combining with the goods weight information by using a computer vision technology based on deep learning, and the identifier is configured to use a computing unit in an intelligent weighing apparatus, an adjacent edge computing unit, or a cloud computing unit to complete identification calculation of goods identification.
  • ordinary watermelons and small Kirin melons are two different species. If the identification results of the goods identification model are ordinary watermelons and small Kirin melons, but the weight obtained by the weighing platform is obviously smaller than the minimum weight of ordinary watermelons, the large watermelons in the goods identification results can be filtered out, leaving the identification results of small Kirin melons. Or if the identification result of the goods identification model is the ordinary big watermelon, the goods identification result is changed from the ordinary big watermelon to the small Kirin melon, so that two watermelon species with different sizes can be distinguished more accurately.
  • the filtered or changed goods identification results alone or together with the currently collected image of the goods (and the weight obtained by the weighing platform) can also be input to the goods identification model, so as to further train or update the goods identification model so as to identify the goods more accurately in the next goods identification.
  • the cashing POS system includes a traditional cash register and a self-service cashing POS system
  • the identification feedback device is configured to adopt an adaptive display strategy based on an identification confidence when displaying candidate goods information, herein when the identification confidences of goods identification results are higher than the predetermined confidence, only goods identification results of a number lower than a predetermined number (for example, a few) with a confidence higher than the predetermined confidence are displayed as candidate goods information, and when the identification confidences of goods identification results are lower than the predetermined confidence, the number of identification results displayed is increased as candidate goods information. While ensuring an identification recall rate, it reduces the difficulty of selection.
  • interactive behaviors of the identification feedback device receiving user feedback include clicking to confirm, re-identifying, searching, screening, modifying the price and returning to the system configuration; the interactive behaviors of user feedback include automatically confirming the goods identification results under predetermined conditions, and the user feedback is automatically transmitted to the price tag printer or cashing POS system and fed back to the modeling platform, herein the predetermined conditions include that the identification confidence exceeds an automatic publishing threshold, or a user feedback time exceeds a waiting threshold.
  • the identification feedback device displays the candidate goods identification results identified by the goods identification model.
  • the goods identification model has not been established and improved yet, or it is simply trained, more goods identification results or inaccurate goods identification results may be identified through identification, for example, ordinary watermelons with seeds and seedless watermelons, etc.
  • the cashier or customer can select the actual goods category through an interactive behavior on the identification feedback device to confirm the goods.
  • the identification feedback device may have a touch screen, including resistive, capacitive, infrared, surface acoustic wave, etc. It may also be other intelligent human-computer interactors, such as voice identification interface, and it may also include human-computer interactors being developed or appearing in the future. If there is no actual goods category in the goods identification result at this time, the cashier or customer may also confirm the goods by searching the database. Cashiers or customers can also use the identification feedback to modify the price of goods, return to the system configuration and so on.
  • the identification feedback device includes a search component configured for a user to search goods by the initials of goods names or codes, and at the same time, to show trending search candidate goods.
  • the identification feedback device includes a screening component configured to preferentially display a list of clearance goods of the same or related categories according to the goods identification results.
  • the list of clearance goods is a list of discount and other sales promotions of goods.
  • the identification feedback device includes a price modification component configured to receive an interactive action of long-pressing a goods information display card of candidate goods information or an element of goods icon and name in the card, and trigger a pop-up of a price modification label, so that the user can modify a unit price, a total price or a discount coefficient of the goods; and when the selected good is a piece-based goods, a quantity of the goods is fed back through an automatic pop-up window.
  • the unit price, total price, discount coefficient, quantity, etc. of the goods can be easily changed through simple interaction.
  • the confirmed goods information when the confirmed goods information needs to be transmitted to the cashing POS system, the confirmed goods information is transmitted to the POS system through a wired or wireless network, or when the confirmed goods information needs to be transmitted to the price tag printer, the confirmed goods information is transmitted to the price tag printer through a wired or wireless network, or when the confirmed goods information needs to be fed back to the modeling platform, the confirmed goods information is uploaded to the modeling platform in a cloud through a wired or wireless network.
  • the user when it is required to return to a system setting interface, the user can return to the system setting interface by sliding on the screen from left to right or from top to bottom. In this way, it can be returned to the system setting interface through simple interaction.
  • the modeling platform is configured to use a computer vision technology based on deep learning to train the identification model based on the collected visual information, weight information and data of user feedback in use, herein the goods weight information is applied to set a weight filtering rule to filter out the identification results of goods whose weight is greater than a goods maximum weight or less than a goods minimum weight, or is applied to statistical goods weight distribution models to assist goods identification; the data of user feedback in use is applied to continuous iterative optimization of the identification model; the trained identification model is distributed to different computing platforms, such as terminal, edge computing or cloud computing platforms.
  • the goods information synchronization component is configured to update the updated goods information into the intelligent weighing apparatus to support display of candidate goods information
  • the goods information synchronization component includes two forms of a windows program and a mobile phone APP, completes the update of goods information through database connecting, file uploading and manual editing, and supports the update of goods information in the intelligent weighing apparatus through two connection modes of a merchant intranet and an extranet.
  • the intelligent weighing apparatus does not include a cashing POS system and a price tag printer, and is connected as a whole with an external cashing POS system.
  • This can be used as a visual weighing plug-in to be plugged and used with the merchant's cashing POS system.
  • the intelligent weighing apparatus includes a cashing POS system, so as to be used as an integrated visual weighing and collecting machine.
  • the intelligent weighing apparatus includes a price tag printer, so as to print the price tag, and customers can take the price tag to the cashier POS machine for scanning the price tag for payment after weighing.
  • the identification feedback device in the intelligent weighing apparatus may be a touch screen, or the intelligent weighing apparatus may not include the touch screen, but be connected with other devices with touch screens or other user interfaces as plug-ins.
  • a complete iterative closed loop of the identification model is established for the first time to ensure the continuous iteration and optimization of the identification model in use and realizes the efficient and accurate identification of weighed goods.
  • the intelligent weighing is combined with the cashing machine, and an efficient solution of integrating cashing and weighing is realized.
  • FIG. 2 schematically illustrates the external structural diagram of an intelligent weighing apparatus according to an embodiment of the present disclosure.
  • the intelligent weighing apparatus 200 may include: a chassis 205 ; a weighing platform 201 , which is connected with the chassis 205 and used for weighing a goods 206 (such as an apple) placed on it; a support rod 204 connected with the chassis 205 ; a visual sensor 202 arranged on or outside the weighing platform 201 , or on the support rod 204 connected with the chassis 205 of the weighing platform 201 , for example, as shown in FIG. 2 , on the upper part of the support rod 204 , and its collecting area covers at least one part of the weighing platform 201 or its surrounding area, for example, as shown in FIG. 2 where the collecting area covers a top view plane of the weighing platform 201 .
  • the intelligent goods identification and weighing structure 200 may further include: a man-machine interactor 203 , which is removably connected with the support rod 204 , herein the man-machine interactor 203 's position can be adjusted relative to the support rod, so as to facilitate users who use the intelligent goods identification and weighing structure 200 to view contents displayed on the man-machine interactor 203 and operate the man-machine interactor 203 .
  • a man-machine interactor 203 which is removably connected with the support rod 204 , herein the man-machine interactor 203 's position can be adjusted relative to the support rod, so as to facilitate users who use the intelligent goods identification and weighing structure 200 to view contents displayed on the man-machine interactor 203 and operate the man-machine interactor 203 .
  • the positions, shapes and sizes of the chassis 205 , the weighing platform 201 , the support rod 204 and the man-machine interactor 203 shown in FIG. 2 are only examples, not limitations, and may be changed according to actual situations.
  • the size of the chassis 205 may be larger than the size of the weighing platform 201 , so as to obtain a better center of gravity and maintain the stabilization of the whole structure.
  • the support rod 204 may be a vertical rod instead of an L-shape.
  • the position of an image collecting component 202 may be located at a right angle of the support rod 204 instead of the top of the support rod 204 , and the human-computer interactor 203 may also not be located on one side of the support rod 204 , instead it may be located with its center on the support rod 204 so that the human-computer interactor is located in the center.
  • the connection modes between these structures may also adopt the existing connection modes, which may be a fixed connection, removable or rotatable connection.
  • Chips, integrated circuits or functional modules that can realize the functions and methods of intelligent goods identification and weighing together with the weighing platform, the image collecting component 202 and the human-computer interactor 203 may also be arranged inside or outside the chassis 205 or the support rod 204 or at the parts connected with it.
  • assemblies are schematic, and some assemblies may be deleted or other assemblies may be added as needed.
  • a microphone, a voice identification device and a speaker may be added, and are configured to perform voice interaction with users to perform various controls.
  • a price tag printer may be added to support printing price tags.
  • USB cable, serial data cable and network cable may be added to support data transmission.
  • FIG. 3 schematically illustrates a flowchart of an intelligent weighing method 300 according to one embodiment of the present disclosure.
  • the intelligent weighing method 300 includes the following steps: at step 301 , setting software parameters, hardware parameters and goods information of the intelligent weighing apparatus by using a system configuration component; at step 302 , placing a goods to be identified and weighed by using a weighing platform, sensing a weight and a weight change, and obtaining goods weight information of the goods; at step 303 , using a visual sensor to collect the visual information of the goods placed on the weighing platform in response to a first trigger; at step 304 , using an identifier to identify the goods in response to a second trigger based on the visual information, the weight information and an identification model, and to terminate the goods identification in response to a third trigger to obtain goods identification results; at step 305 , displaying candidate goods information by using the identification feedback device, by combining the goods identification result with goods information and receive user feedback, and transmit the goods information confirmed in the user feedback to a price tag printer or a cashing POS system, and feed it back to a modeling platform, herein the price tag printer is configured to print a
  • system configuration component is configured to set the hardware parameters including different hardware model selection of the weighing platform and the price tag printer.
  • system configuration component is configured to set hardware parameters of the visual sensor, including a preview resolution, whether to cut the main body area out, and parameters for cutting the main body area out.
  • the system configuration component is configured to set hardware parameters of tare setting for the weighing platform, including three modes of not subtracting tare weight, one-time subtracting tare weight and continuously subtracting tare weight.
  • system configuration component is configured to set software parameters including merchant selection, store selection and goods management, herein the goods management includes goods information of name, code, price and illustrative figures for browsing and editing.
  • the first trigger includes automatically triggering the visual sensor to collect the visual information of the goods placed on the weighing platform when the weighing platform senses that a weight changes and the weight is not zero
  • the second trigger includes automatically triggering the identifier to identify goods when the weighing platform senses that a weight changes and the weight is not zero
  • the third trigger includes the weight signal being stable so that a stable weight signal is taken as goods weight information for calculating a total price of goods and generating a goods weighing bar code.
  • the visual sensor is configured to collect visual information of goods placed on the weighing platform, herein the visual information includes at least one of images and videos, and the visual sensor is configured to collect visual information for building models for new goods; and collect visual information for the goods to be identified, herein the visual sensor is configured to carry out a capturing mode including a single-capturing-a-photo-mode, a continuous-capturing-multiple-photos-mode and a video-capturing-mode when collecting visual information for building models for new goods, and is configured to carry out a capturing mode including a single-capturing-a-photo-mode, a continuous-capturing-multiple-photos-mode and a video-capturing-mode when collecting visual information for the goods to be identified.
  • a capturing mode including a single-capturing-a-photo-mode, a continuous-capturing-multiple-photos-mode and a video-capturing-mode when collecting visual information for the goods to be identified.
  • the second trigger includes a changes of a content of a visual signal collected by the visual sensor to trigger the identifier to identify the goods
  • the third trigger includes stabilization of the visual signals to terminate the identification of the goods
  • the visual sensor is located above or on a side of the weighing platform, so as to be able to perceive visual information of goods on the weighing platform with a definition higher than a predetermined definition, and when the hardware parameters of the visual sensor are set to cut the main body area out, the main body area of the collected visual information is cut out according to the parameters for cutting the main body area out.
  • the second trigger includes a manual selection.
  • the identification model is configured to perform a conditional filtering or statistical regression to filter out candidate identification results whose weight is greater than a maximum goods weight or less than a minimum goods weight by using a computer vision technology based on deep learning, combined with goods weight information.
  • the identifier is configured to complete identification calculation of the goods identification by using a computing unit in the intelligent weighing apparatus, an adjacent edge computing unit, or a cloud computing unit.
  • the cashing POS system includes a POS system of a traditional cash register and a POS system of a self-help cash register.
  • the identification feedback device is configured to adopt an adaptive display strategy based on an identification confidence when displaying candidate goods information, herein when the identification confidences of goods identification results are higher than a predetermined confidence, only goods identification results of a number lower than a predetermined number with a confidence higher than the predetermined confidence are displayed as candidate goods information, and when the identification confidences of goods identification results are lower than the predetermined confidence, the number of displayed goods identification results is increased as candidate goods information.
  • interactive behaviors of receiving user feedback by using the identification feedback device include clicking to confirm, re-identifying, searching, screening, modifying the price and returning to the system configuration; the interactive behaviors of user feedback include automatically confirming the goods identification results under predetermined conditions, and the user feedback is automatically transmitted to the price tag printer or cashing POS system and fed back to the modeling platform, herein the predetermined conditions include that the identification confidence exceeds an automatic publishing threshold, or a user feedback time exceeds a waiting threshold.
  • the identification feedback device includes a search component, which enables users to search goods by using initials letters of goods names or codes, and simultaneously displays trending search candidate goods.
  • the identification feedback device includes a screening component, and the screening component is configured to preferentially display a list of clearance goods of the same or related categories according to the goods identification results.
  • the identification feedback device includes a price modification component
  • the price modification component is configured to receive an interactive action of long-pressing a goods information display card of candidate goods information or an element of goods icon and name in the card, and trigger a pop-up price modification tag, so that the user can modify a unit price, a total price or a discount coefficient of the goods.
  • the selected good is a piece-based goods
  • the quantity of the goods is fed back through an automatic pop-up window.
  • the confirmed goods information when the confirmed goods information needs to be transmitted to the cashing POS system, the confirmed goods information is transmitted to the POS system through a wired or wireless network, or when the confirmed goods information needs to be transmitted to the price tag printer, the confirmed goods information is transmitted to the price tag printer through a wired network, or when the confirmed goods information needs to be fed back to the modeling platform, the confirmed goods information is uploaded to the modeling platform in a cloud through a wired or wireless network.
  • the user when it is required to return to a system setting interface, the user can return to the system setting interface by sliding on the screen from left to right or from top to bottom.
  • the modeling platform is configured to use a computer vision technology based on deep learning to train the identification model based on the collected visual information, the weight information and the data of the user feedback in use, herein the goods weight information is applied to set a weight filtering rule to filter out the identification results of goods whose weight is greater than the maximum goods weight or less than the minimum goods weight, or is applied to statistical goods weight distribution models to assist goods identification; the data of the user feedback in use is applied to the continuous iterative optimization of the identification model; the trained identification model is distributed to different computing platforms, such as terminal, edge computing or cloud computing platforms.
  • the goods information synchronization component is configured to update the updated goods information into the intelligent weighing apparatus to support the display of candidate goods information
  • the goods information synchronization component including two forms of a Windows® program and a mobile phone APP, completes the update of goods information through database connecting, file uploading and manual editing, and supports the update of goods information in the intelligent weighing apparatus through two connection modes of a merchant intranet and an extranet.
  • the intelligent weighing apparatus does not include a cashing POS system and a price tag printer, and is connected as a whole with an external cashing POS system; or the intelligent weighing apparatus includes a cashing POS system; or the intelligent weighing apparatus includes a price tag printer.
  • a complete iterative closed loop of the identification model is established for the first time to ensure the continuous iteration and optimization of the identification model in use, thereby realizing the efficient and accurate identification of weighed goods.
  • the intelligent weighing is combined with the cashier, and an efficient solution of integrating cashing and weighing is realized.
  • FIG. 4 schematically illustrates a flowchart of an intelligent weighing method 400 according to another embodiment of the present disclosure.
  • the intelligent weighing method 400 includes the following steps: S 401 , system configuring, for setting software and hardware parameters and merchant information; S 402 , synchronously transmitting, for uploading goods information, including one or more of the obtained goods information such as goods name, goods code, weighing apparatus code, price, pricing method, illustrative figure, etc.; S 403 , goods database updating, for uploading the information uploaded at S 402 into a database; S 404 , information collecting, for collecting visual information and goods weight information; S 405 , training a identification model by combining the visual information with the weight information of the goods; S 406 , delivering the identification model to the identification service; S 407 , placing the goods to be weighed on the weighing platform and to get weighed to obtain weight information; S 408 , collecting visual information of goods; S 409 , identifying the goods types by using the identification model in an identification service
  • S 401 system configuring includes hardware parameter setting, software parameter setting and merchant information setting.
  • Hardware parameter setting includes different hardware model selection of weighing platform and price tag printer.
  • the hardware parameter setting also includes visual sensor setting, including a preview resolution, whether to cut a main body area out, parameters for cutting the main body area out and so on.
  • Hardware parameter setting also includes tare setting of weighing platform, including three modes of not subtracting tare weight, one-time subtracting tare weight and continuously subtracting tare weight.
  • Software parameter setting includes merchant selection, store selection and goods management. Among them, goods management also includes browsing and editing goods information such as name, code, price and pictures.
  • S 402 synchronously transmitting includes two forms of: a Windows® program and a mobile phone APP, and the change of goods information is completed through database connecting, file uploading and manual editing. At the same time, it supports updating the goods information in the intelligent weighing apparatus through two connection modes of: a merchant intranet and an extranet.
  • the synchronized information includes one or more of goods information such as goods name, goods code, weighing apparatus code, price, pricing method, illustrative figure, etc.
  • S 403 is goods database updating, in which the goods database may be deployed in the merchant intranet, and the goods information in the intelligent weighing apparatus is updated through the merchant intranet.
  • S 403 is goods updating, in which the goods database may be deployed in a cloud, and the goods information in the intelligent weighing apparatus is updated through the external network.
  • goods images are collected through a single-capturing-photo-mode; in one embodiment, at S 404 of information collecting, goods images are collected through a single-capturing-photo-mode; in one embodiment, at S 404 of information collecting, goods images are collected through a continuous-capturing-multiple-photos-mode; in one embodiment, at S 404 of information collecting, goods images are collected through a video-capturing-mode; in one embodiment, at S 404 of information collecting, the main body area of visual information is cut out based on the system configuration; in one embodiment, at S 404 of information collecting, at the same time, goods weight information is collected.
  • the goods identification model is trained through computer vision technology based on deep learning.
  • the visual information of goods may come from collected data, refluxed data or their combination.
  • the goods weight information is configured to build a conditional filtering model to filter out the candidate identification results whose weight is greater than a maximum goods weight or less than a minimum goods weight.
  • the model is delivered to a computing unit on the intelligent weighing apparatus, and the identification calculation is completed by a terminal computing unit; in one embodiment, at S 406 , the model is delivered to an edge computing unit, and the edge computing unit of the neighboring merchants is configured to complete identification calculation; in one embodiment, at S 406 , the model is delivered to a cloud server, and the cloud server is configured to complete identification calculation.
  • the weight change is sensed, and the weight of the goods is obtained.
  • the change of weight signal is used as an identification trigger.
  • the weight changes and the weight is not zero, it can automatically trigger the visual sensor to collect a visual signal, and the identifier can identify the goods.
  • the stabilization of the weight signal is one of the judging conditions for the termination of the identification, and a stable weight signal is used as the weight of the goods to calculate the total price of the goods and generate the bar code information of the goods.
  • At S 408 of goods visual information collecting collecting goods images starts when the weight changes, and the goods images and weights are transmitted into S 409 for goods identifying.
  • the goods category is predicted by the visual model and the weight model.
  • an adaptive display strategy based on an identification confidence is adopted.
  • the identification result has a high confidence
  • only a few high confidence results are displayed, and when the identification result has a low confidence, the number of displayed goods is increased. While ensuring the identification recall rate, it reduces the difficulty of selection.
  • the user's interactive behaviors on the identification results include clicking to confirm, re-recognizing, searching, screening, modifying the price, and returning to the system configuration.
  • the search component is configured so that users can search goods by using the initials of goods names or codes, and at the same time, trending search candidates are displayed, which improves the search efficiency of users.
  • the screening component is configured to give priority to displaying the list of clearance goods of similar categories according to the goods identification results, so as to improve the screening efficiency of users.
  • the price modification component is configured to display the card or the goods icon, name and other elements in the card by long pressing the goods information, and triggering the pop-up price modification label, which can enable users to modify a unit price, a total price or a discount coefficient of the goods.
  • the goods are goods counted by pieces, the quantity of the goods is fed back through an automatic pop-up window.
  • the system setting interface can be returned to by sliding on the screen from left to right or from top to bottom.
  • the identification result can be automatically confirmed.
  • the identification confidence exceeds the automatic publishing threshold, or the user feedback time exceeds the waiting threshold, the corresponding result is automatically transmitted to the price tag printer or POS system and fed back to the modeling platform.
  • the confirmed goods information is transmitted to the POS system through a wired network.
  • the confirmed goods information is transmitted to the POS system through software communication.
  • the confirmed goods information is transmitted to the price tag printer through a wired network.
  • At S 411 of identification feedback when the user feedback information needs to be uploaded to the modeling platform, it is uploaded to the cloud platform through a wireless network. In one embodiment, at S 411 of identification feedback, when the user feedback information needs to be uploaded to the modeling platform, it is uploaded to the cloud platform through a wired network.
  • the POS system adds goods information to support further cashier actions; in one embodiment, at S 412 , the bar code printer prints a price tag, which supports users to record a price into the POS system by scanning the code.
  • a complete iterative closed loop of the identification model is established for the first time to ensure the continuous iteration and optimization of the identification model in use and realize the efficient and accurate identification of weighed goods.
  • intelligent weighing is combined with the cashier, and an efficient solution of integrating cashing and weighing is realized.
  • steps and devices in various embodiments herein are not limited to a certain embodiment.
  • some related steps and devices in various embodiments herein may be combined to conceive new embodiments according to the concepts of this disclosure, and these new embodiments are also included in the scope of this disclosure.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Accounting & Taxation (AREA)
  • Finance (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Mathematical Physics (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Development Economics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Hardware Design (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Game Theory and Decision Science (AREA)
  • Geometry (AREA)
  • Entrepreneurship & Innovation (AREA)
  • General Engineering & Computer Science (AREA)
  • Cash Registers Or Receiving Machines (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
US18/254,408 2020-11-26 2021-10-07 Intelligent weighing apparatus based on computer vision technology, and intelligent weighing method Pending US20240020672A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
CN202011348099.6A CN112466068B (zh) 2020-11-26 2020-11-26 基于计算机视觉技术的智能称重装置与智能称重方法
CN202011348099.6 2020-11-26
PCT/CN2021/122531 WO2022111059A1 (zh) 2020-11-26 2021-10-07 基于计算机视觉技术的智能称重装置与智能称重方法

Publications (1)

Publication Number Publication Date
US20240020672A1 true US20240020672A1 (en) 2024-01-18

Family

ID=74808506

Family Applications (1)

Application Number Title Priority Date Filing Date
US18/254,408 Pending US20240020672A1 (en) 2020-11-26 2021-10-07 Intelligent weighing apparatus based on computer vision technology, and intelligent weighing method

Country Status (4)

Country Link
US (1) US20240020672A1 (zh)
EP (1) EP4239607A4 (zh)
CN (2) CN114550385A (zh)
WO (1) WO2022111059A1 (zh)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114550385A (zh) * 2020-11-26 2022-05-27 融讯伟业（北京）科技有限公司 基于计算机视觉技术的智能称重装置与智能称重方法
CN113624314B (zh) * 2021-07-06 2023-03-07 盒马(中国)有限公司 称重处理方法、装置及称重设备
CN113485993A (zh) * 2021-07-13 2021-10-08 浙江网商银行股份有限公司 数据识别方法以及装置
CN113269935B (zh) * 2021-07-16 2021-11-30 融讯伟业（北京）科技有限公司 一种基于无屏称重装置的视觉称重方法及称重系统
CN113903127A (zh) * 2021-09-30 2022-01-07 厦门顶尖电子有限公司 带多秤台的人工智能识别秤管理系统
CN114333183A (zh) * 2022-01-17 2022-04-12 上海岩伸信息科技有限公司 水产品识别检测智能销售系统一体机
CN114543962A (zh) * 2022-02-22 2022-05-27 深圳进化动力数码科技有限公司 一种智能识别非标商品的称重设备及其识别方法
CN115048355B (zh) * 2022-06-13 2023-06-06 烟台创迹软件有限公司 一种识别模型的更新方法、装置、设备及介质
CN115099752B (zh) * 2022-07-18 2023-08-15 融讯伟业（北京）科技有限公司 一种基于视觉识别的商品盘点方法及装置
AU2022470820A1 (en) * 2022-11-16 2024-05-30 Hanshow Technology Co., Ltd. System and identification method for artificial intelligence identification scale based on autonomous incremental learning
CN116403339B (zh) * 2023-06-08 2023-08-11 美恒通智能电子(广州)股份有限公司 一种基于rfid标签识别的打印机智能管理系统及方法

Family Cites Families (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5799593B2 (ja) * 2011-06-07 2015-10-28 株式会社寺岡精工 商品検索装置、商品情報処理装置及びラベル発行装置
JP6141208B2 (ja) * 2014-01-08 2017-06-07 東芝テック株式会社 情報処理装置及びプログラム
JP6193136B2 (ja) * 2014-01-21 2017-09-06 東芝テック株式会社 画像情報処理装置及びプログラム
US10282722B2 (en) * 2015-05-04 2019-05-07 Yi Sun Huang Machine learning system, method, and program product for point of sale systems
US9911290B1 (en) * 2015-07-25 2018-03-06 Gary M. Zalewski Wireless coded communication (WCC) devices for tracking retail interactions with goods and association to user accounts
JP6679344B2 (ja) * 2016-02-29 2020-04-15 東芝テック株式会社 計量システム
CN107679850A (zh) * 2017-09-15 2018-02-09 苏衍杰 一种商品结算方法、装置及系统
CN108269369A (zh) * 2017-09-27 2018-07-10 中山市宾哥网络科技有限公司 结算箱及其结算方法
US20190236360A1 (en) * 2018-01-30 2019-08-01 Mashgin Inc. Feedback loop for image-based recognition
CN108303170A (zh) * 2018-02-01 2018-07-20 京东方科技集团股份有限公司 智能电子称的称重方法和智能电子称
CN108537994A (zh) * 2018-03-12 2018-09-14 深兰科技（上海）有限公司 基于视觉识别及重量感应技术的智能商品结算系统及方法
CN108550229A (zh) * 2018-04-08 2018-09-18 珠海博明视觉科技有限公司 一种人工智能自动收银方法
US20190333039A1 (en) * 2018-04-27 2019-10-31 Grabango Co. Produce and bulk good management within an automated shopping environment
JP2019200533A (ja) * 2018-05-15 2019-11-21 パナソニックＩｐマネジメント株式会社 計数装置、会計システム、学習装置、及び、制御方法
CN109118200A (zh) * 2018-07-26 2019-01-01 上海凯景信息技术有限公司 一种基于图像识别的商品识别与收银系统
CN109147170A (zh) * 2018-07-26 2019-01-04 上海凯景信息技术有限公司 一种基于图像识别的无人售货柜
KR101960900B1 (ko) * 2018-10-12 2019-03-21 주식회사 에스피씨네트웍스 제품 식별 방법
CN111222870B (zh) * 2019-01-24 2024-02-27 图灵通诺（北京）科技有限公司 结算方法、装置和系统
CN111554057A (zh) * 2019-02-12 2020-08-18 株式会社石田 计量机
CN110363185A (zh) * 2019-08-09 2019-10-22 融讯伟业（北京）科技有限公司 智能商品识别设备和方法、电子设备以及智能结算台
CN110987140A (zh) * 2019-11-28 2020-04-10 浙江由由科技有限公司 一种商品称重辅助设备及称重设备
CN110956459A (zh) * 2019-11-28 2020-04-03 浙江由由科技有限公司 一种商品处理方法及系统
CN111814614A (zh) * 2020-06-28 2020-10-23 袁精侠 智能识物电子秤称重方法及系统
CN114550385A (zh) * 2020-11-26 2022-05-27 融讯伟业（北京）科技有限公司 基于计算机视觉技术的智能称重装置与智能称重方法

Also Published As

Publication number Publication date
WO2022111059A1 (zh) 2022-06-02
CN112466068B (zh) 2021-07-09
CN112466068A (zh) 2021-03-09
EP4239607A4 (en) 2024-03-06
CN114550385A (zh) 2022-05-27
EP4239607A1 (en) 2023-09-06

Similar Documents

Publication Publication Date Title
US20240020672A1 (en) Intelligent weighing apparatus based on computer vision technology, and intelligent weighing method
CN212675641U (zh) 基于计算机视觉技术的智能称重装置
CN108230086B (zh) 一种商品售卖调整的方法及存储介质
CN107808469B (zh) 自助超市系统
US20100253787A1 (en) Method for Object Recognition and Communication of Associated Label and Other Information
CN107578537A (zh) 一种自助售货机及自助售货机的数据推送方法
CN108140209B (zh) 信息处理装置、信息处理方法和其中存储有程序的记录介质
US10948338B2 (en) Digital product label generation using modular scale device
WO2017056429A1 (ja) 情報処理装置、情報処理方法、およびプログラムが格納された記録媒体
US9076170B2 (en) Self-service checkout pay station located remote from a produce weighing scale and methods of operating such a self-service checkout pay station
CN107293049A (zh) 一种商品自助销售方法及其装置
RU2565005C1 (ru) Касса самообслуживания
CN110782314B (zh) 一种基于边缘计算技术的新型散货零售平台
CN110942293A (zh) 物品信息的处理方法、设备、存储介质及系统
CN110119915A (zh) 对象入库处理方法、装置和系统
JP6393987B2 (ja) 商品販売データ処理システム、および携帯端末
CN111598561B (zh) 称重信息处理方法、装置及系统
CN108492490A (zh) 一种基于商品重量的自助购物系统及其控制方法
JP6647485B1 (ja) 商品代金精算システム
WO2023280124A1 (zh) 称重处理方法、装置及称重设备
WO2016136082A1 (ja) 情報処理装置、プログラム、及び制御方法
US20210090059A1 (en) Weighing device, weighing device with label printer, and remote order processing system
JP2011227645A (ja) 注文データ管理システム及びメニューブック
US11727470B2 (en) Optical scanning for weights and measures
CN219223902U (zh) 一种智能秤

Legal Events

Date Code Title Description
STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION