CN110815235B - Intelligent shopping service method and system based on data matching - Google Patents
Intelligent shopping service method and system based on data matching Download PDFInfo
- Publication number
- CN110815235B CN110815235B CN201910901250.5A CN201910901250A CN110815235B CN 110815235 B CN110815235 B CN 110815235B CN 201910901250 A CN201910901250 A CN 201910901250A CN 110815235 B CN110815235 B CN 110815235B
- Authority
- CN
- China
- Prior art keywords
- information
- article information
- article
- image
- controlling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 230000008014 freezing Effects 0.000 claims abstract description 30
- 238000007710 freezing Methods 0.000 claims abstract description 30
- 239000000463 material Substances 0.000 claims abstract description 24
- 238000000605 extraction Methods 0.000 claims description 33
- 230000005540 biological transmission Effects 0.000 claims description 8
- 239000003550 marker Substances 0.000 claims description 6
- 238000005057 refrigeration Methods 0.000 claims description 5
- 238000010330 laser marking Methods 0.000 claims description 4
- 239000000284 extract Substances 0.000 description 9
- 235000011194 food seasoning agent Nutrition 0.000 description 6
- 239000004278 EU approved seasoning Substances 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J11/00—Manipulators not otherwise provided for
- B25J11/008—Manipulators for service tasks
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/12—Target-seeking control
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Automation & Control Theory (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Manipulator (AREA)
Abstract
An intelligent shopping service method based on data matching and a system thereof comprise: receiving information sent by a user terminal and analyzing whether the information contains a food material consultation request, if so, controlling the recognition robot to start and control the first camera to start and shoot a first image in real time, controlling the recognition robot to move to a refrigerator area planned in a house and control the grabbing mechanical arm to open a refrigerating door body and a freezing door body of the refrigerator, controlling the second camera to start and shoot a second image in real time, extracting first article information in the refrigerating space of the refrigerator and second article information in the freezing space of the refrigerator, extracting a shopping cart article image contained in the information sent by the user terminal and extracting third article information contained in the shopping cart article image, comparing the third article information with the first article information and the second article information respectively, analyzing whether matched articles exist in real time, and if so, transmitting the analyzed matched article information to the user terminal.
Description
Technical Field
The invention relates to the field of supermarket shopping, in particular to an intelligent shopping service method and system based on data matching.
Background
Many people experience the experience in life, when people return to home to cook by opening the refrigerator, the food materials are all used up, and when people visit a supermarket, the people do not know how many food materials are left in the home and do not need to buy food materials.
Therefore, how to combine big data, article identification and a robot, when food materials in a refrigerator and a kitchen at home are forgotten during supermarket shopping, how to control the robot to open the refrigerator to obtain corresponding cold storage and freezing food material information and obtain food material information in a cabinet and a drawer of the kitchen, and then comparing the obtained food materials with the food materials of a shopping cart of a user to remove repeated food materials is a problem which is urgently needed to be solved at present.
Disclosure of Invention
The purpose of the invention is as follows: in order to overcome the disadvantages in the background art, embodiments of the present invention provide an intelligent shopping service method and system based on data matching, which can effectively solve the problems related to the background art.
The technical scheme is as follows:
an intelligent shopping service method based on data matching, the method comprising the steps of:
s1, receiving information sent by the user terminal maintaining the long connection relation in real time and analyzing whether the information contains the food material consultation request according to the information sent by the user terminal;
s2, if yes, controlling the recognition robot arranged at the inner position of the house to start and controlling a first camera arranged at the outer position of the recognition robot to start to capture a first image in real time;
s3, controlling the recognition robot to move to a refrigerator area of the planned house according to the first image, and controlling a grabbing mechanical arm arranged at the outer position of the recognition robot to open a cold storage door body and a freezing door body of the refrigerator according to the first image;
s4, controlling a second camera arranged outside the recognition robot to start to capture a second image in real time, and extracting first article information inside a refrigerating space of the refrigerator and second article information inside a freezing space of the refrigerator according to the second image;
s5, extracting a shopping cart article image contained in the received information sent by the user terminal and extracting third article information contained in the shopping cart article image;
s6, comparing the extracted third article information with the first article information and the second article information respectively to analyze whether matched articles exist in real time;
and S7, if yes, transmitting the analyzed matched article information to the user terminal.
As a preferred mode of the present invention, the method further comprises the steps of:
s8, extracting the shopping cart number contained in the received information sent by the user terminal and establishing a connection relation with the supermarket shopping cart with the same number as the extracted shopping cart number;
s9, controlling a third camera arranged at the position of the push handle of the supermarket shopping cart for connection to start, shooting a third image in real time and controlling a laser identifier arranged at the position of the push handle of the supermarket shopping cart to start;
and S10, controlling the laser marker in real time to carry out laser marking on the articles in the supermarket shopping cart matched with the matched article information according to the third image and the analyzed matched article information.
As a preferred mode of the present invention, after S5, the method further includes the steps of:
s50, controlling the recognition robot to move to a kitchen area where the house is planned according to the first image, and controlling the grabbing mechanical arm to open a cabinet door and a drawer of the kitchen area according to the first image;
s51, controlling the recognition robot to patrol for a preset number of turns in the kitchen area according to the first image and extracting fourth article information contained in the first image;
s52, comparing the extracted third article information with the first article information, the second article information and the fourth article information respectively, and analyzing whether a matched article exists;
and S53, if yes, transmitting the analyzed matched article information to the user terminal.
As a preferred mode of the present invention, after S51, the method further includes the steps of:
s510, inputting the extracted first item information, second item information, third item information and fourth item information into a connected recipe database for recipe query, and extracting a recipe containing the first item information and/or the second item information and/or the third item information and/or the fourth item information in real time;
s511, transmitting all the extracted recipe information to the user terminal and receiving the recipe information fed back by the user terminal in real time;
s512, matching the recipe information fed back by the user terminal with the first article information, the second article information, the third article information and the fourth article information to obtain the article information required by the remaining recipe information, and transmitting the obtained article information required by the remaining recipe information to the user terminal.
As a preferred mode of the present invention, the method further comprises the steps of:
s100, after the first article information is extracted, controlling the grabbing mechanical arm to close a refrigeration door body opened by the refrigerator according to a first image;
s101, after the information of the second article is extracted, controlling the grabbing mechanical arm to close a freezing door body opened by the refrigerator according to the first image;
s101, after the fourth article information is extracted, controlling the grabbing mechanical arm to close a cabinet door and a drawer which are opened in a kitchen area according to the first image.
An intelligent shopping service system based on data matching uses an intelligent shopping service method based on data matching, and comprises an identification robot, a supermarket shopping cart and a cloud control center, wherein the identification robot is provided with a first camera, a second camera and a grabbing mechanical arm, and the first camera is arranged at an external position above the identification robot and used for shooting an environment image outside the identification robot; the second cameras are respectively arranged at the upper position and the lower position outside the identification robot, correspond to the refrigerating area and the freezing area of the refrigerator inside the house, and are provided with up-down adjusting functions for respectively shooting environment images of the refrigerating area and the freezing area of the refrigerator; the grabbing mechanical arm is arranged at the outer position of the recognition robot and used for opening and closing a refrigerator door body, a kitchen cabinet door and a kitchen drawer;
the supermarket shopping cart is provided with a scanning shell, a third camera and a laser identifier, and the scanning shell is arranged in the middle of a pushing handle of the supermarket shopping cart; the third camera is arranged at the surface position of the scanning shell facing the shopping basket and used for shooting information of articles placed in the supermarket shopping cart; the laser marker is arranged at the surface position of the scanning shell facing the shopping basket and used for marking articles;
the cloud control center comprises:
the wireless module is used for being wirelessly connected with the first camera, the second camera, the grabbing mechanical arm, the scanning shell, the third camera, the laser identifier, the user terminal and external equipment of a supermarket management department respectively;
the information receiving module is used for receiving information and/or requests and/or instructions;
the information analysis module is used for processing and analyzing the information according to the specified information;
the robot control module is used for controlling the recognition robot to execute set operation according to the set steps;
the first shooting module is used for controlling the first camera to be started or closed;
the mechanical arm control module is used for controlling the mechanical arm to execute set operation according to set steps;
the second shooting module is used for controlling the second camera to be started or closed;
the refrigerator extraction module is used for extracting first article information in the refrigerating space of the refrigerator and second article information in the freezing space of the refrigerator according to the image shot by the second camera;
the image extraction module is used for extracting the shopping cart object image contained in the information sent by the user terminal;
the shopping extraction module is used for extracting third article information contained in the shopping cart article image;
the first comparison module is used for comparing the third article information with the first article information and the second article information respectively and acquiring comparison results;
and the information transmission module is used for transmitting the specified information to the specified object.
As a preferable mode of the present invention, the cloud control center further includes:
the number extraction module is used for extracting the shopping cart number contained in the information sent by the user terminal;
the connection establishing module is used for establishing a connection relation with the extracted supermarket shopping carts with the same serial numbers as the shopping cart numbers;
the third shooting module is used for controlling the third camera to be started or closed;
and the laser identification module is used for controlling the laser identifier to carry out laser tube identification on the specified articles in the supermarket shopping cart.
As a preferable mode of the present invention, the cloud control center further includes:
the kitchen extraction module is used for extracting fourth article information contained in the first image;
and the second comparison module is used for comparing the third article information with the first article information, the second article information and the fourth article information respectively and acquiring comparison results.
As a preferable mode of the present invention, the cloud control center further includes:
the recipe database is used for storing recipe information;
the recipe query module is used for inputting the first article information, the second article information, the third article information and the fourth article information into the recipe database for recipe query;
the recipe extraction module is used for extracting a recipe containing the first article information and/or the second article information and/or the third article information and/or the fourth article information;
and the third comparison module is used for matching the recipe information fed back by the user terminal with the first article information, the second article information, the third article information and the fourth article information to obtain the article information required by the rest of the recipe information.
The invention realizes the following beneficial effects:
1. after the intelligent shopping service system is started, food material consultation requests sent by a user are acquired in real time, after the food material consultation requests are acquired, the recognition robot is controlled to go to a refrigerator area to recognize article information of a refrigerator refrigerating area and a freezing area and recognize article information inside a shopping cart of the user, then matching articles existing in the supermarket shopping cart are recognized through comparison between the article information in the shopping cart and the article information of the refrigerator refrigerating area and the freezing area, and the existing matching articles are transmitted to a user terminal to be prompted.
2. After the existing matched articles are transmitted to the user terminal for prompting, the third camera of the supermarket shopping cart identifies repeated information of the articles in the supermarket shopping cart and the articles in the house, and then the laser identifier of the supermarket shopping cart identifies the repeated articles in the house by laser.
3. After the article information of the refrigerator refrigerating area and the freezing area inside the house is extracted, the recognition robot is controlled to patrol inside the kitchen and open a cabinet door and a drawer of the kitchen by using the grabbing mechanical arm, then the article information existing inside the kitchen is scanned, then the matching articles existing in the supermarket shopping cart are recognized by comparing the article information in the shopping cart with the article information of the refrigerator refrigerating area, the freezing area and the article information inside the kitchen, and the existing matching articles are transmitted to the user terminal for prompting.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 is a flow chart of an intelligent shopping service method provided by one example of the present invention;
FIG. 2 is a flow chart of a laser marking method according to one embodiment of the present invention;
fig. 3 is a flow chart of a kitchen item identification method according to an example of the present invention;
FIG. 4 is a flow chart of a recipe selection method according to an exemplary embodiment of the present invention;
FIG. 5 is a flow chart of a method for completing the identification of an item according to one embodiment of the present invention;
FIG. 6 is a connection diagram of an intelligent shopping service system provided by one example of the present invention;
fig. 7 provides a schematic diagram of a recognition robot for one example of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments.
Example one
Referring to fig. 1, fig. 6-7.
Specifically, the embodiment provides an intelligent shopping service method based on data matching, which is characterized by comprising the following steps:
s1, receiving information sent by the user terminal maintaining the long connection relation in real time and analyzing whether the information contains the food material consultation request according to the information sent by the user terminal;
s2, if yes, controlling the recognition robot 1 arranged at the inner position of the house to start and controlling the first camera 10 arranged at the outer position of the recognition robot 1 to start to capture a first image in real time;
s3, controlling the recognition robot 1 to move to a refrigerator area of the planned house according to the first image and controlling the grabbing mechanical arm 12 arranged at the outer position of the recognition robot 1 to open a cold storage door body and a freezing door body of the refrigerator according to the first image;
s4, controlling a second camera 11 arranged outside the recognition robot 1 to start to capture a second image in real time and extracting first article information inside a refrigerating space of the refrigerator and second article information inside a freezing space of the refrigerator according to the second image;
s5, extracting a shopping cart article image contained in the received information sent by the user terminal and extracting third article information contained in the shopping cart article image;
s6, comparing the extracted third article information with the first article information and the second article information respectively to analyze whether matched articles exist in real time;
and S7, if yes, transmitting the analyzed matched article information to the user terminal.
In S1, specifically, after the cloud control center 3 is started, the information receiving module 31 included in the cloud control center 3 receives, in real time, information sent by the user terminal maintaining the long connection relationship and sends the received information to the information analysis module 32, after the information analysis module 32 of the cloud control center 3 receives the information sent by the user terminal, the information analysis module 32 analyzes whether the information is information including a food material consultation request according to the information sent by the user terminal, that is, analyzes whether the user forgets the food material information at home.
In S2, specifically, after the information analysis module 32 analyzes that the information sent by the user terminal is information including a food material consultation request, the robot control module 33 included in the cloud control center 3 controls the recognition robot 1 disposed at an internal position of the house to start up, and after the recognition robot 1 is started up, the first camera module 34 included in the cloud control center 3 controls the first camera 10 disposed at an external position of the recognition robot 1 to start up to capture a first image in real time, where the first image is an environmental image around the recognition robot 1.
In S3, specifically after the first camera 10 is started, the robot control module 33 included in the cloud control center 3 controls the recognition robot 1 to move to the refrigerator area where the house is planned according to the first image, and the second camera 11 of the robot faces the refrigerator in the refrigerator area, after the recognition robot 1 finishes moving, the robot arm control module 35 included in the cloud control center 3 controls the grabbing robot arm 12 arranged at the position outside the recognition robot 1 to extend out of the refrigerator door and the refrigerator door for grabbing the refrigerator according to the first image, and then controls the grabbing robot arm 12 to retract and open the refrigerator area and the refrigerator area.
Specifically, after the refrigeration door body and the freezing door body of the refrigerator are opened, the second camera 11 arranged outside the identification robot 1 is controlled by the second shooting module 36 included in the cloud control center 3 to start to shoot a second image in real time and transmit the second image to the refrigerator extraction module 37, and after the refrigerator extraction module 37 included in the cloud control center 3 receives the second image, the refrigerator extraction module 37 extracts first article information inside the refrigeration space of the refrigerator and second article information inside the freezing space of the refrigerator according to the second image and transmits the extracted first article information and second article information to the first comparison module 40; the second image is an image of a cold storage area and a freezing area of the refrigerator captured by the second camera 11.
In S5, specifically after the refrigerator extraction module 37 extracts the first item information and the second item information, the image extraction module 38 included in the cloud control center 3 extracts the shopping cart item image included in the information sent by the user terminal and received by the information receiving module 31 and transmits the extracted shopping cart item image to the shopping extraction module 39, and after the shopping extraction module 39 included in the cloud control center 3 receives the shopping cart item image, the shopping extraction module 39 extracts the third item information included in the shopping cart item image and transmits the extracted third item information to the first comparison module 40.
In S6, specifically, after the first comparison module 40 included in the cloud control center 3 receives the first item information, the second item information, and the third item information, the first comparison module 40 compares the third item information with the first item information and the second item information respectively to obtain comparison results, and transmits the obtained comparison results to the information analysis module 32, and the information analysis module 32 analyzes whether there is a matching item in real time according to the comparison results, that is, analyzes whether there is a same item in a supermarket shopping cart corresponding to the user at home.
In S7, specifically, after the information analysis module 32 analyzes that the items in the supermarket shopping cart corresponding to the user have the same items at home, the information transmission module 41 included in the cloud control center 3 transmits the analyzed information of the matched items to the user terminal, so as to remind the user of the existence of the items with the same items in the shopping cart at home, and pay attention to check and prevent the user from buying useless repeated food materials or seasonings.
The article information comprises food material information and seasoning information, the food material information comprises the name and the type of food material, and the seasoning information comprises the name and the type of seasoning; the first article information refers to article information in a refrigerating area of the refrigerator, the second article information refers to article information in a refrigerating area of the refrigerator, and the third article information refers to article information in a shopping basket of a supermarket shopping cart.
Example two
Referring to fig. 2, fig. 6-7.
Specifically, the present embodiment is substantially the same as the first embodiment, and the difference is that in the present embodiment, the method further includes the following steps:
s8, the shopping cart number contained in the received information sent by the user terminal is extracted, and the connection relation is established between the shopping cart number and the supermarket shopping cart with the same number as the extracted shopping cart number.
Specifically, when the user is reminded that the same article exists in the supermarket shopping cart at home, the number extraction module 42 included in the cloud control center 3 extracts the shopping cart number included in the received information sent by the user terminal and transmits the extracted shopping cart number to the connection establishment module 43, after the connection establishment module 43 included in the cloud control center 3 receives the shopping cart number, the connection establishment module establishes a connection relationship with the supermarket shopping cart with the same number as the extracted shopping cart number, and the supermarket shopping cart keeps a long connection relationship with the operation control center.
And S9, controlling the third camera 21 arranged at the position of the push handle of the supermarket shopping cart for connection to start, shooting a third image in real time and controlling the laser identifier 22 arranged at the position of the push handle of the supermarket shopping cart to start.
Specifically, after the connection establishing module 43 establishes a connection relationship with the supermarket shopping cart, the third camera 21 arranged at the push handle position of the supermarket shopping cart which establishes the connection is controlled by the third shooting module 44 included in the cloud control center 3 to start shooting a third image in real time, the third image is an environment image in the shopping basket of the supermarket shopping cart which is shot by the third camera 21, and meanwhile, the laser identifier module 45 included in the cloud control center 3 controls the laser identifier 22 arranged at the push handle position of the supermarket shopping cart to start.
And S10, controlling the laser marker 22 to carry out laser marking on the articles in the supermarket shopping cart matched with the matched article information in real time according to the third image and the analyzed matched article information.
Specifically, after the third camera 21 is started and the laser identifier 22 is started, the laser identification module 45 controls the laser identifier 22 to perform laser identification on the article matched with the matched article information in the supermarket shopping cart according to the third image and the analyzed matched article information in real time, that is, the safety laser beam is emitted to identify the article in the supermarket shopping cart which is overlapped with the article in the supermarket shopping cart.
EXAMPLE III
As shown with reference to fig. 3-7.
Specifically, this embodiment is substantially the same as the first embodiment, except that in this embodiment, after S5, the method further includes the following steps:
and S50, controlling the recognition robot 1 to move to a kitchen area where the house is planned according to the first image and controlling the grabbing mechanical arm 12 to open a cabinet door and a drawer of the kitchen area according to the first image.
Specifically, after the shopping extraction module 39 extracts the information of the third article, the robot control module 33 controls the recognition robot 1 to move to the kitchen area where the house is planned according to the first image, after the recognition robot 1 moves to the kitchen area, the robot control module 35 controls the grabbing robot 12 to open all doors and drawers in the kitchen area according to the first image, when the doors and drawers are opened, the robot control module 33 and the robot control module 35 work in a matching manner, that is, the robot control module 33 controls the recognition robot 1 to move to the unopened cabinet or drawer, then the robot control module 35 controls the grabbing robot 12 to open the cabinet or drawer, and so on until all the cabinets and drawers are opened.
And S51, controlling the recognition robot 1 to patrol the kitchen area for a preset number of turns according to the first image and extracting fourth article information contained in the first image.
Wherein, the preset number of turns can be 1-10 turns, and is preferably 3 turns in the embodiment; the fourth item information is item information in the kitchen.
Specifically, after the cabinet and the drawer in the kitchen area are opened, the robot control module 33 controls the recognition robot 1 to patrol the kitchen area for a preset number of turns according to the first image, when the robot moves for patrol, the kitchen extraction module 46 included in the cloud control center 3 extracts the fourth item information included in the first image and transmits the extracted fourth item information to the second comparison module 47, and the information of each item in the kitchen is extracted only once.
And S52, comparing the extracted third article information with the first article information, the second article information and the fourth article information respectively, and analyzing whether a matched article exists.
Specifically, after the second comparison module 47 included in the cloud control center 3 receives the fourth article information, the second comparison module 47 compares the extracted third article information with the first article information, the second article information, and the fourth article information respectively to obtain a comparison result, and transmits the obtained comparison result to the information analysis module 32, and the information analysis module 32 analyzes whether there is a matching article in real time according to the comparison result, that is, analyzes whether there is a same article in a home of an article in the supermarket shopping cart corresponding to the user.
And S53, if yes, transmitting the analyzed matched article information to the user terminal.
Specifically, after the information analysis module 32 analyzes that the items in the supermarket shopping cart corresponding to the user have the same items at home, the information transmission module 41 transmits the analyzed matched item information to the user terminal to remind the user of the existence of the items in the supermarket shopping cart which are consistent, so that the user can pay attention to check the items and prevent the user from buying useless repeated food materials or seasonings.
As a preferred mode of the present invention, after S51, the method further includes the steps of:
s510, inputting the extracted first item information, second item information, third item information and fourth item information into a connected recipe database 48 for recipe query and extracting recipes containing the first item information and/or the second item information and/or the third item information and/or the fourth item information in real time.
Specifically, after the kitchen extraction module 46 extracts the fourth item information, the recipe query module 49 included in the cloud control center 3 inputs the extracted first item information, second item information, third item information and fourth item information into the connected recipe database 48 for recipe query, that is, the recipe information including any one or more of the first item information, the second item information, the third item information and the fourth item information is inquired and the inquired recipe information is transmitted to the recipe extraction module 50, after the recipe extraction module 50 included in the cloud control center 3 receives the recipe information, the recipe extraction module 50 extracts the recipe including the first item information and/or the second item information and/or the third item information and/or the fourth item information in real time and transmits the extracted recipe to the information transmission module 41.
And S511, transmitting all the extracted recipe information to the user terminal and receiving the recipe information fed back by the user terminal in real time.
Specifically, after the information transmission module 41 receives the recipe, the information transmission module 41 transmits all the received recipe information to the user terminal, and the information receiving module 31 receives the recipe information fed back by the user terminal in real time.
S512, matching the recipe information fed back by the user terminal with the first article information, the second article information, the third article information and the fourth article information to obtain the article information required by the remaining recipe information, and transmitting the obtained article information required by the remaining recipe information to the user terminal.
Specifically, if the information receiving module 31 receives recipe information fed back by the user terminal, the recipe information is transmitted to the third comparing module 51, the third comparing module 51 included in the cloud control center 3 matches the recipe information fed back by the user terminal with the first item information, the second item information, the third item information and the fourth item information to obtain the remaining required item information of the recipe information, that is, to obtain food materials and/or seasonings that are not available in the user shopping cart and the home of the recipe record, and after the remaining required item information of the recipe information is obtained, the information transmitting module 41 transmits the remaining required item information of the recipe information to the user terminal. To remind the user to make a purchase.
As a preferred mode of the present invention, the method further comprises the steps of:
and S100, after the first article information is extracted, controlling the grabbing mechanical arm 12 to close a refrigeration door body opened by the refrigerator according to the first image.
Specifically, after all the article information in the refrigerating area of the refrigerator is acquired, the mechanical arm control module 35 controls the grabbing mechanical arm 12 to close the refrigerating door body opened by the refrigerator according to the first image.
S101, after the information of the second article is extracted, controlling the grabbing mechanical arm 12 to close the freezing door body opened by the refrigerator according to the first image.
Specifically, after all the article information in the freezing area of the refrigerator is acquired, the mechanical arm control module 35 controls the grabbing mechanical arm 12 to close the freezing door body opened by the refrigerator according to the first image.
S101, after the fourth article information is extracted, controlling the grabbing mechanical arm 12 to close the cabinet door and the drawer opened in the kitchen area according to the first image.
Specifically, after all the article information in the kitchen area is acquired, the mechanical arm control module 35 controls the grabbing mechanical arm 12 to close a cabinet door or a drawer opened in the kitchen according to a first image, the robot control module 33 controls the recognition robot 1 to cooperate according to the first image, namely the robot control module 33 controls the recognition robot 1 to move to the opened cabinet or drawer, then the mechanical arm control module 35 controls the grabbing mechanical arm 12 to close the cabinet or the drawer, and so on until all the cabinets and the drawers are closed.
Example four
As shown with reference to fig. 6-7.
Specifically, the embodiment provides an intelligent shopping service system based on data matching, and an intelligent shopping service method based on data matching is used, the intelligent shopping service system comprises an identification robot 1, a supermarket shopping cart and a cloud control center 3, wherein the identification robot 1 is provided with a first camera 10, a second camera 11 and a grabbing mechanical arm 12, and the first camera 10 is arranged at an external position above the identification robot 1 and used for shooting an environment image outside the identification robot 1; the second cameras 11 are respectively arranged at the upper position and the lower position outside the recognition robot 1, correspond to the refrigerating area and the freezing area of the refrigerator inside the house, and have an up-down adjusting function for respectively capturing the environment images of the refrigerating area and the freezing area of the refrigerator; the grabbing mechanical arm 12 is arranged at the outer position of the recognition robot 1 and used for opening and closing a refrigerator door body, a kitchen cabinet door and a kitchen drawer;
the supermarket shopping cart is provided with a scanning shell 20, a third camera 21 and a laser identifier 22, wherein the scanning shell 20 is arranged in the middle of a pushing handle of the supermarket shopping cart; the third camera 21 is arranged at a position of the scanning shell 20 facing the surface of the shopping basket, and is used for capturing information of articles placed in the supermarket shopping cart; the laser marker 22 is arranged on the surface of the scanning shell 20 facing the shopping basket and is used for marking articles;
the cloud control center 3 includes:
the wireless module 30 is used for being wirelessly connected with the first camera 10, the second camera 11, the grabbing mechanical arm 12, the scanning shell 20, the third camera 21, the laser identifier 22, the user terminal and external equipment of a supermarket management department respectively;
an information receiving module 31, configured to receive information and/or requests and/or instructions;
an information analysis module 32, configured to perform information processing and analysis according to the specified information;
a robot control module 33 for controlling the recognition robot 1 to perform a set operation according to the set steps;
the first shooting module 34 is used for controlling the first camera 10 to be started or closed;
a robot arm control module 35 for controlling the robot arm to execute the set operation according to the set steps;
the second shooting module 36 is used for controlling the second camera 11 to be started or closed;
the refrigerator extraction module 37 is used for extracting first article information inside the refrigerating space of the refrigerator and second article information inside the freezing space of the refrigerator according to the images shot by the second camera 11;
the image extraction module 38 is used for extracting the shopping cart object images contained in the information sent by the user terminal;
a shopping extraction module 39, configured to extract third item information included in the shopping cart item image;
the first comparison module 40 is configured to compare the third item information with the first item information and the second item information respectively and obtain a comparison result;
an information transmission module 41 for transmitting the designation information to the designation object.
As a preferred embodiment of the present invention, the cloud control center 3 further includes:
a number extraction module 42, configured to extract a shopping cart number included in the information sent by the user terminal;
the connection establishing module 43 is used for establishing a connection relation with the extracted supermarket shopping carts with the same serial numbers;
the third shooting module 44 is used for controlling the third camera 21 to be started or closed;
and the laser identification module 45 is used for controlling the laser identifier 22 to carry out laser pipe identification on the specified articles in the supermarket shopping cart.
As a preferred embodiment of the present invention, the cloud control center 3 further includes:
a kitchen extraction module 46, configured to extract fourth item information included in the first image;
the second comparing module 47 is configured to compare the third article information with the first article information, the second article information, and the fourth article information, respectively, and obtain a comparing result.
As a preferred embodiment of the present invention, the cloud control center 3 further includes:
a recipe database 48 for storing recipe information;
the recipe query module 49 is used for inputting the first item information, the second item information, the third item information and the fourth item information into the recipe database 48 for recipe query;
a recipe extraction module 50 for extracting a recipe containing the first item information and/or the second item information and/or the third item information and/or the fourth item information;
the third comparing module 51 is configured to match the recipe information fed back by the user terminal with the first item information, the second item information, the third item information, and the fourth item information to obtain the remaining required item information of the recipe information.
It should be understood that, in the fourth embodiment, the specific implementation process of each module described above may correspond to the description of the above method embodiments (the first to the third embodiments), and is not described in detail here.
The system provided in the fourth embodiment is only illustrated by dividing the functional modules, and in practical applications, the above-mentioned functions may be distributed by different functional modules according to needs, that is, the internal structure of the system is divided into different functional modules to complete all or part of the functions described above.
The above embodiments are merely illustrative of the technical ideas and features of the present invention, and are intended to enable those skilled in the art to understand the contents of the present invention and implement the present invention, and not to limit the scope of the present invention. All equivalent changes or modifications made according to the spirit of the present invention should be covered within the protection scope of the present invention.
Claims (9)
1. An intelligent shopping service method based on data matching is characterized by comprising the following steps:
s1, receiving information sent by the user terminal maintaining the long connection relation in real time and analyzing whether the information contains the food material consultation request according to the information sent by the user terminal;
s2, if yes, controlling the recognition robot arranged at the inner position of the house to start and controlling a first camera arranged at the outer position of the recognition robot to start to capture a first image in real time;
s3, controlling the recognition robot to move to a refrigerator area of the planned house according to the first image, and controlling a grabbing mechanical arm arranged at the outer position of the recognition robot to open a cold storage door body and a freezing door body of the refrigerator according to the first image;
s4, controlling a second camera arranged outside the recognition robot to start to capture a second image in real time, and extracting first article information inside a refrigerating space of the refrigerator and second article information inside a freezing space of the refrigerator according to the second image;
s5, extracting a shopping cart article image contained in the received information sent by the user terminal and extracting third article information contained in the shopping cart article image;
s6, comparing the extracted third article information with the first article information and the second article information respectively to analyze whether matched articles exist in real time;
and S7, if yes, transmitting the analyzed matched article information to the user terminal.
2. The intelligent shopping service method based on data matching as claimed in claim 1, further comprising the steps of:
s8, extracting the shopping cart number contained in the received information sent by the user terminal and establishing a connection relation with the supermarket shopping cart with the same number as the extracted shopping cart number;
s9, controlling a third camera arranged at the position of the push handle of the supermarket shopping cart for connection to start, shooting a third image in real time and controlling a laser identifier arranged at the position of the push handle of the supermarket shopping cart to start;
and S10, controlling the laser marker in real time to carry out laser marking on the articles in the supermarket shopping cart matched with the matched article information according to the third image and the analyzed matched article information.
3. The intelligent shopping service method based on data matching as claimed in claim 1, wherein after S5, the method further comprises the steps of:
s50, controlling the recognition robot to move to a kitchen area where the house is planned according to the first image, and controlling the grabbing mechanical arm to open a cabinet door and a drawer of the kitchen area according to the first image;
s51, controlling the recognition robot to patrol for a preset number of turns in the kitchen area according to the first image and extracting fourth article information contained in the first image;
s52, comparing the extracted third article information with the first article information, the second article information and the fourth article information respectively, and analyzing whether a matched article exists;
and S53, if yes, transmitting the analyzed matched article information to the user terminal.
4. The intelligent shopping service method based on data matching as claimed in claim 3, wherein after S51, the method further comprises the steps of:
s510, inputting the extracted first item information, second item information, third item information and fourth item information into a connected recipe database for recipe query, and extracting a recipe containing the first item information and/or the second item information and/or the third item information and/or the fourth item information in real time;
s511, transmitting all the extracted recipe information to the user terminal and receiving the recipe information fed back by the user terminal in real time;
s512, matching the recipe information fed back by the user terminal with the first article information, the second article information, the third article information and the fourth article information to obtain the article information required by the remaining recipe information, and transmitting the obtained article information required by the remaining recipe information to the user terminal.
5. The intelligent shopping service method based on data matching as claimed in claim 3, further comprising the steps of:
s100, after the first article information is extracted, controlling the grabbing mechanical arm to close a refrigeration door body opened by the refrigerator according to a first image;
s101, after the information of the second article is extracted, controlling the grabbing mechanical arm to close a freezing door body opened by the refrigerator according to the first image;
s101, after the fourth article information is extracted, controlling the grabbing mechanical arm to close a cabinet door and a drawer which are opened in a kitchen area according to the first image.
6. An intelligent shopping service system based on data matching, which uses the intelligent shopping service method based on data matching as claimed in any one of claims 1-5, and comprises an identification robot, a supermarket shopping cart and a cloud control center, wherein the identification robot is provided with a first camera, a second camera and a grabbing mechanical arm, the first camera is arranged at an external position above the identification robot and is used for shooting an environment image outside the identification robot; the second cameras are respectively arranged at the upper position and the lower position outside the identification robot, correspond to the refrigerating area and the freezing area of the refrigerator inside the house, and are provided with up-down adjusting functions for respectively shooting environment images of the refrigerating area and the freezing area of the refrigerator; the grabbing mechanical arm is arranged at the outer position of the recognition robot and used for opening and closing a refrigerator door body, a kitchen cabinet door and a kitchen drawer;
the supermarket shopping cart is provided with a scanning shell, a third camera and a laser identifier, and the scanning shell is arranged in the middle of a pushing handle of the supermarket shopping cart; the third camera is arranged at the surface position of the scanning shell facing the shopping basket and used for shooting information of articles placed in the supermarket shopping cart; the laser marker is arranged at the surface position of the scanning shell facing the shopping basket and used for marking articles;
the cloud control center comprises:
the wireless module is used for being wirelessly connected with the first camera, the second camera, the grabbing mechanical arm, the scanning shell, the third camera, the laser identifier, the user terminal and external equipment of a supermarket management department respectively;
the information receiving module is used for receiving information and/or requests and/or instructions;
the information analysis module is used for processing and analyzing the information according to the specified information;
the robot control module is used for controlling the recognition robot to execute set operation according to the set steps;
the first shooting module is used for controlling the first camera to be started or closed;
the mechanical arm control module is used for controlling the mechanical arm to execute set operation according to set steps;
the second shooting module is used for controlling the second camera to be started or closed;
the refrigerator extraction module is used for extracting first article information in the refrigerating space of the refrigerator and second article information in the freezing space of the refrigerator according to the image shot by the second camera;
the image extraction module is used for extracting the shopping cart object image contained in the information sent by the user terminal;
the shopping extraction module is used for extracting third article information contained in the shopping cart article image;
the first comparison module is used for comparing the third article information with the first article information and the second article information respectively and acquiring comparison results;
and the information transmission module is used for transmitting the specified information to the specified object.
7. The intelligent shopping service system based on data matching as claimed in claim 6, wherein the cloud control center further comprises:
the number extraction module is used for extracting the shopping cart number contained in the information sent by the user terminal;
the connection establishing module is used for establishing a connection relation with the extracted supermarket shopping carts with the same serial numbers as the shopping cart numbers;
the third shooting module is used for controlling the third camera to be started or closed;
and the laser identification module is used for controlling the laser identifier to carry out laser tube identification on the specified articles in the supermarket shopping cart.
8. The intelligent shopping service system based on data matching as claimed in claim 6, wherein the cloud control center further comprises:
the kitchen extraction module is used for extracting fourth article information contained in the first image;
and the second comparison module is used for comparing the third article information with the first article information, the second article information and the fourth article information respectively and acquiring comparison results.
9. The intelligent shopping service system based on data matching as claimed in claim 6, wherein the cloud control center further comprises:
the recipe database is used for storing recipe information;
the recipe query module is used for inputting the first article information, the second article information, the third article information and the fourth article information into the recipe database for recipe query;
the recipe extraction module is used for extracting a recipe containing the first article information and/or the second article information and/or the third article information and/or the fourth article information;
and the third comparison module is used for matching the recipe information fed back by the user terminal with the first article information, the second article information, the third article information and the fourth article information to obtain the article information required by the rest of the recipe information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910901250.5A CN110815235B (en) | 2019-09-23 | 2019-09-23 | Intelligent shopping service method and system based on data matching |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910901250.5A CN110815235B (en) | 2019-09-23 | 2019-09-23 | Intelligent shopping service method and system based on data matching |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110815235A CN110815235A (en) | 2020-02-21 |
| CN110815235B true CN110815235B (en) | 2021-05-18 |
Family
ID=69548192
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910901250.5A Active CN110815235B (en) | 2019-09-23 | 2019-09-23 | Intelligent shopping service method and system based on data matching |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110815235B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112068576B (en) * | 2020-07-21 | 2022-07-12 | 华中科技大学 | Task-hierarchical timing optimization-based underwater unmanned ship-double mechanical arms cooperative control method |
| CN113494745B (en) * | 2021-02-27 | 2022-10-11 | 北京威势优创科技有限公司 | Intelligent mobile air conditioner and control method thereof |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| SE521047C2 (en) * | 1999-02-19 | 2003-09-23 | Rutger Roseen | Device and method for providing information about the contents of a cabinet |
| CN108418734B (en) * | 2013-09-27 | 2021-06-01 | 夏普株式会社 | Control device, control method, control system, and notification device |
| CN103822431B (en) * | 2014-02-25 | 2016-02-10 | 四川长虹电器股份有限公司 | A kind of refrigerator |
| CN204893945U (en) * | 2015-09-11 | 2015-12-23 | 哈尔滨理工大学 | Imitative articular housekeeping robot of human arm |
| KR20170076992A (en) * | 2015-12-26 | 2017-07-05 | 삼성전자주식회사 | Refrigerator |
| CN105897886A (en) * | 2016-04-07 | 2016-08-24 | 美的集团股份有限公司 | Food material purchase reminding method and device, intelligent terminal, cloud server and automobile |
| JP6904349B2 (en) * | 2016-07-13 | 2021-07-14 | ソニーグループ株式会社 | Agent robot control system, agent robot system, agent robot control method, recording medium, and agent robot control program |
| CN108775764A (en) * | 2018-06-12 | 2018-11-09 | 芜湖乐创电子科技有限公司 | A kind of refrigerator storing analysis robot control system based on monitoring analysis transmission |
-
2019
- 2019-09-23 CN CN201910901250.5A patent/CN110815235B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110815235A (en) | 2020-02-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10192197B2 (en) | Automated asset management system with multiple sensing technologies | |
| CN110815235B (en) | Intelligent shopping service method and system based on data matching | |
| US10621543B2 (en) | System and method to enable delivery and pick up of packages using pods and unmanned vehicles | |
| US9741010B1 (en) | System and method for securely delivering packages to different delivery recipients with a single vehicle | |
| WO2020143179A1 (en) | Article identification method and system, and electronic device | |
| WO2020001288A1 (en) | Storage cabinet control system, storage cabinet control method and cabinet lock | |
| CN108257320A (en) | Picking method and marketing system based on warehousing cabinet | |
| CN106846750A (en) | A kind of automatically deploying troop is withdrawn a garrison system and method | |
| US10438033B1 (en) | Object tracking system having first and second RFID readers with display | |
| CN105404201B (en) | The intelligent control method of the intelligence control system of refrigerator | |
| CN105575009A (en) | Cabinet door opening method and device for intelligent cabinet | |
| US11414895B2 (en) | Systems and methods for automated capture and recovery of tag and tack | |
| CN111564005A (en) | Storage device control method and storage device | |
| WO2018099930A1 (en) | System and method for securely delivering packages to different delivery recipients with a single vehicle | |
| CN112036790A (en) | Unmanned distribution method and device | |
| US9836936B1 (en) | Object tracking system | |
| JP7030092B2 (en) | Information generation method and equipment and equipment for human-computer interaction | |
| US20180137495A1 (en) | Layaway system | |
| CN109911498A (en) | A kind of intelligent element locker | |
| CN112835356A (en) | Conveyance control system, conveyance control program, and conveyance control method | |
| CN116184892B (en) | AI identification control method and system for robot object taking | |
| KR102510066B1 (en) | system for receiving orders through terminals in the store | |
| CN113700414A (en) | Valuables safety control system | |
| CN109801018B (en) | Control method of express cabinet, express cabinet and computer readable medium | |
| CN110852410A (en) | Intelligent tool management system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210427 Address after: 518000 unit 01-02, 7 / F, Changhong science and technology building, South 12 Keji Road, high tech Zone, Yuehai street, Nanshan District, Shenzhen City, Guangdong Province Applicant after: Shenzhen Lianzhong Interactive Technology Co.,Ltd. Address before: Room 1307, Building 168, East Shanghai Road, Loudong Street, Taicang City, Suzhou City, Jiangsu Province Applicant before: SUZHOU SHANGXINBAO INFORMATION TECHNOLOGY Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A Data Matching Based Intelligent Shopping Service Method and System Effective date of registration: 20231129 Granted publication date: 20210518 Pledgee: Shenzhen Rural Commercial Bank Co.,Ltd. Nanshan Sub branch Pledgor: Shenzhen Lianzhong Interactive Technology Co.,Ltd. Registration number: Y2023980068411 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right |