CN110352116A - Intellect service robot and related systems and methods - Google Patents
Intellect service robot and related systems and methods Download PDFInfo
- Publication number
- CN110352116A CN110352116A CN201780075073.9A CN201780075073A CN110352116A CN 110352116 A CN110352116 A CN 110352116A CN 201780075073 A CN201780075073 A CN 201780075073A CN 110352116 A CN110352116 A CN 110352116A
- Authority
- CN
- China
- Prior art keywords
- robot
- product
- computer
- inventory
- facility
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims description 29
- 230000000007 visual effect Effects 0.000 claims abstract description 30
- 238000004891 communication Methods 0.000 claims description 14
- 230000035807 sensation Effects 0.000 claims description 13
- 238000003556 assay Methods 0.000 claims description 6
- 238000007726 management method Methods 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 claims description 3
- 238000000151 deposition Methods 0.000 claims description 2
- 238000003860 storage Methods 0.000 claims description 2
- 238000002604 ultrasonography Methods 0.000 claims 2
- 238000009434 installation Methods 0.000 abstract description 5
- 230000004888 barrier function Effects 0.000 description 15
- 238000010586 diagram Methods 0.000 description 14
- 230000033001 locomotion Effects 0.000 description 14
- 238000001514 detection method Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 10
- 238000005286 illumination Methods 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 8
- 238000003032 molecular docking Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 230000009471 action Effects 0.000 description 6
- 235000013339 cereals Nutrition 0.000 description 6
- 239000003795 chemical substances by application Substances 0.000 description 6
- 210000003128 head Anatomy 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 238000000605 extraction Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 4
- 230000017531 blood circulation Effects 0.000 description 4
- 230000001737 promoting effect Effects 0.000 description 4
- 230000029058 respiratory gaseous exchange Effects 0.000 description 4
- 238000004140 cleaning Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 230000005611 electricity Effects 0.000 description 3
- 210000001508 eye Anatomy 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000002452 interceptive effect Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 210000000120 body fluid compartment Anatomy 0.000 description 2
- 238000013481 data capture Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000001815 facial effect Effects 0.000 description 2
- 230000008921 facial expression Effects 0.000 description 2
- 239000008267 milk Substances 0.000 description 2
- 210000004080 milk Anatomy 0.000 description 2
- 235000013336 milk Nutrition 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000036651 mood Effects 0.000 description 2
- 230000000877 morphologic effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002207 retinal effect Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 230000005236 sound signal Effects 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 239000000725 suspension Substances 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 206010037660 Pyrexia Diseases 0.000 description 1
- 206010070834 Sensitisation Diseases 0.000 description 1
- 239000002253 acid Substances 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000037237 body shape Effects 0.000 description 1
- 235000008429 bread Nutrition 0.000 description 1
- 210000005252 bulbus oculi Anatomy 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 235000013569 fruit product Nutrition 0.000 description 1
- 239000002828 fuel tank Substances 0.000 description 1
- 230000037308 hair color Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000007788 liquid Substances 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000008450 motivation Effects 0.000 description 1
- NJPPVKZQTLUDBO-UHFFFAOYSA-N novaluron Chemical compound C1=C(Cl)C(OC(F)(F)C(OC(F)(F)F)F)=CC=C1NC(=O)NC(=O)C1=C(F)C=CC=C1F NJPPVKZQTLUDBO-UHFFFAOYSA-N 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000035755 proliferation Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 230000035939 shock Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
- 230000005641 tunneling Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/087—Inventory or stock management, e.g. order filling, procurement or balancing against orders
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J11/00—Manipulators not otherwise provided for
- B25J11/0005—Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J11/00—Manipulators not otherwise provided for
- B25J11/008—Manipulators for service tasks
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/0011—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement
- G05D1/0038—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
- G05D1/0248—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means in combination with a laser
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Economics (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Quality & Reliability (AREA)
- General Business, Economics & Management (AREA)
- Automation & Control Theory (AREA)
- Development Economics (AREA)
- Mechanical Engineering (AREA)
- Entrepreneurship & Innovation (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Aviation & Aerospace Engineering (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- Robotics (AREA)
- Theoretical Computer Science (AREA)
- Optics & Photonics (AREA)
- Accounting & Taxation (AREA)
- General Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Health & Medical Sciences (AREA)
- Finance (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Electromagnetism (AREA)
- Manipulator (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
Robot for providing intelligent Service in facility includes lower sensor, display and the robot computer communicated with mobile platform, upper sensor and lower sensor of mobile platform, the upper sensor of object in the top visual field for detecting robot, object in the lower part visual field for detecting robot.Robot computer be configured to make an inventory the product in commercial facility and at computer installation user provide inventory variance data.
Description
Technical field
The disclosure relates generally to robot device, and relate more particularly to intellect service robot and its related system and
Method.
Background technique
For business environment (such as consumer stores, market and shopping center) provide high quality service robot at
For the significant challenge in industry.This challenge is partly due to that robot needs to interact with human interaction and with business environment,
It and is not usually to be designed for robot application.In terms of customer service, to succeed, robot allow for
The specific position that shopper links up, provides information to shopper and be directed to shopper in shop, while ensuring to do shopping
Person not will receive the existing negative effect of robot.For robot, physical operations is also one and chooses in business environment
War.Channel, displaying and the environment in shop seek to fill that the demand of people and obtain business success, rather than robot interacts.Cause
This, robot allows for completing its given task using the existing capability in the shop designed for the mankind.
Other than customer service, many responsibilities are also needed in business environment, these responsibilities are usually handled by the mankind.Wherein
One responsibility is to be constantly needed to the inventory of product in cleaning shop.Store clerk spends numerous hour to comment by channel of passing by
Estimate the inventory in shop and product associates with clerking；Robot can not usually execute such work.There are many classes
The known service robot of type；However, this robot is not suitable for customer service or adaptive inventory management services mesh
, thus it is usually inefficient, not income, it is unattractive and not satisfactory in other respects.In addition, working as machine
Device people is in commercial facility when successful implementation, and initial cost and daily operating cost may be very big, therefore robot allows for
Execute task as much as possible.
Therefore, so far, there is the demands for solving drawbacks described above and shortcoming in industry.
Summary of the invention
Embodiment of the present disclosure is provided for providing the robot and system, robot of intelligent Service in commercial facility
System.Robot includes mobile platform, the upper sensor of object in the top visual field for detecting robot, for detection machine
Lower sensor, display and the robot computer of object in the lower part visual field of device people.Robot computer and movement are flat
Platform, upper sensor and lower sensor communication, and robot computer includes processor and computer-readable memory.Machine
Device people's allocation of computer is the product made an inventory in commercial facility.
In another embodiment, it provides for providing the system, robot of intelligent inventory service in commercial facility
System.Robot includes mobile platform, the upper sensor of object in the top visual field for detecting robot, for detection machine
The lower sensor of object in the lower part visual field of device people, display and with mobile platform, upper sensor and lower sensor
The robot computer of communication, computer have processor and computer-readable memory.Support erect-position in robot distant place, and
And the support station computer with station display is supported and with supporting station video camera and support station display to communicate, support station to calculate
Machine has processor and computer-readable memory.Robot is configured to visually capture the number of item for sale in commercial facility
According to, and at least part for capturing data is transmitted to support station.
In another embodiment, the disclosure can also be considered as offer and be made an inventory the method for commercial facility with robot.Wherein
In this respect, an embodiment of this method can substantially be summarized by following steps: machine is provided in commercial facility
People, wherein the top sensing of object of the robot with mobile platform, in the top visual field for detecting the robot
Device, display, is configured to capture commercial facility at the lower sensor of object in the lower part visual field for detecting the robot
It the product scanning means of the reported visual sensation of interior product and is swept with mobile platform, upper sensor, lower sensor and product
The robot computer of imaging apparatus communication；Use the reported visual sensation of the product in product scanning means capture commercial facility；And locate
The reported visual sensation of product is managed to determine the inventory data of product in commercial facility.
Detailed description of the invention
By examining the following drawings and detailed description, other systems, method, the feature and advantage of the disclosure will be to this fields
It is become apparent for technical staff.All these spare systems, method, feature and advantage are intended to be included in this specification
It is interior, within the scope of this disclosure, and protected by the appended claims.
Many aspects of the disclosure may be better understood with reference to the following drawings.Component in attached drawing is drawn not necessarily to scale
System, but focus on and clearly demonstrate on the principle of the disclosure.In addition, in the accompanying drawings, it is identical in several views
Appended drawing reference indicates corresponding part.
Fig. 1 is the robot for the offer customer service in facility according to the first illustrative embodiments of the disclosure
The schematic diagram of device.
Fig. 2 is according to Fig. 1 of the first illustrative embodiments of the disclosure for providing customer service in facility
The block diagram of the subsystem of robot device.
Fig. 3 be according to the first illustrative embodiments of the disclosure for the robot device using Fig. 1 in facility
The schematic diagram of the robot system of customer service is provided.
Fig. 4 is the active path rule used according to the robot device of Fig. 1 of the first illustrative embodiments of the disclosure
The schematic diagram drawn.
Fig. 5 is the machine with article and customer interaction in facility according to the first illustrative embodiments of the disclosure
The schematic diagram of people's device.
Fig. 6 is the offer customer service in facility for Fig. 1 according to the first illustrative embodiments of the disclosure
The block diagram for the vision algorithm that robot device uses.
Fig. 7 is the machine in the facility using the robot device of Fig. 1 according to the first illustrative embodiments of the disclosure
The schematic diagram of device people's system.
Fig. 8 is the signal according to the mobile platform of the robot device of Fig. 1 of the first illustrative embodiments of the disclosure
Figure.
Fig. 9-22 is according to the view of digital inventory's administration interface of the first illustrative embodiments of the disclosure, the number
Stock control interface can operate together with the software of Fig. 3, for providing intelligent Service in facility.
Specific embodiment
In the following description, with reference to forming part thereof of attached drawing, and the disclosure is shown by way of explanation
Various embodiments.It should be appreciated that without departing from the scope of the disclosure, can use other embodiments and go forward side by side
Row changes.
Many aspects of the invention can use the form of computer executable instructions, including be executed by programmable calculator
Algorithm.Those skilled in the relevant art will be understood that the present invention can also be realized with other computer system configurations.The present invention
Some aspects can be embodied in the special purpose computer or data processor specifically programmed, at the special purpose computer or data
The configuration of reason device is configured to execute one or more computer executable algorithms described below.Therefore, usually used herein
Term " computer " refers to any data processor, and (including palm PC, can wear including internet device, handheld device
Wear computer, honeycomb or mobile phone, multicomputer system, processor-based or programmable consumer electronics, net
Network computer, minicomputer) etc..
Some aspects of the invention can also be implemented in a distributed computing environment, and wherein task or module are by passing through communication
The remote processing devices of network connection execute.In a distributed computing environment, program module or subprogram, which can be located locally, deposits
It stores up in equipment and remote storage device.It can store or be distributed in computer-readable medium in terms of invention described below
On, including magnetic and optical readable and moveable computer disk, fixed disk, floppy disk drive, CD drive, magneto-optic
Disk drive, tape, disc driver (HDD), solid state drive (SSD), compact flash or nonvolatile memory and
Distributed by network electronic.Transmission specific to data structure and data in terms of the present invention is also included within the scope of the present invention
It is interior.
Fig. 1 is the robot for the offer intelligent Service in facility according to the first illustrative embodiments of the disclosure
The schematic diagram of device 1.(it is properly termed as intellect service robot to robot device 1 for providing intelligent Service herein
1) it can be any class in robot, tele-robotic (telebot) or all fours or the similar robot device in part
Type can automatically navigate, and sense or detect certain elements and communicated with remote support station.Intellect service robot 1
May include mobile platform 11, robot head 23, wireless telecom equipment 13, robot computer 14, lower part 3D sensor 12,
Top 3D sensor 15, more array microphones 16, loudspeaker 18, touch panel device 19 and back displays device 20.In addition, Intellectual garment
Business robot 1 may include the structure 21 that mobile platform 11 is such as connected to the frame of robot head 23.Although herein
Illustrative embodiments in be described as " 3D " sensor, those skilled in the relevant art will readily appreciate that, upper sensor
One or two of 15 and lower sensor 12 can be 2D sensor, and have 2D sensor as upper sensor 15
Think within the scope of this disclosure with the embodiment of the robot of one or two of lower sensor 12.
Mobile platform 11 allows robot 1 mobile.Mobile platform 11 can have such as two or more wheels and foot
Wheel 30 allows robot to move in any direction.Robot 1 may include that any kind of drive system is mobile flat to use
Platform 11 is mobile, such as electromechanical driving engine.Robot 1 can have various height, it is preferred that at least 5 feet of height with
Just optimal convenience is provided for client.The height of structure 21 can be adjustable, and robot head 23 can move
Dynamic and inclination is otherwise moved, thus allow robot 1 height and robot head 23 angulation change to mention
For any desired height and from upper sensor 15.
Robot computer 14, which can be, to be made of various hardware and software components using any of operating system
Any calculating equipment.In one embodiment, robot computer 14 is using Ubuntu operating system and including single 12V
The microcomputer of power supply.Robot computer 14 can have enough processing capacities to run various softwares, including for example
Robot operating system (ROS), video processing with OpenCV etc..Any computation module known in the art can be with machine
People's computer 14 is used together.
Multiple sensors can be set on intellect service robot 1.Sensor may include any kind of sensor,
Including three-dimensional (3D) sensor, two dimension (2D) sensor or any other sensor known in the art.Due to 3D sensor pair
It may be preferred in most of purposes, therefore sensor is described as 3D sensor by the disclosure.Top 3D sensor 15 allows
Robot 1 perceives 1 outrunner of robot and object, and (it can be 2D, 3D to additional upper sensor 30,31 and 32
Or other) robot 1 is allowed to perceive the side for being located at robot 1 and subsequent people and object.Lower part on mobile platform
3D sensor 12 allows the barrier on the ground in front of detection robot, robot 1, and additional lower sensor 34,35
Robot 1 is allowed to detect the barrier on the side and subsequent ground of robot 1 with 36 (it can be 2D, 3D or other).
Robot 1 can also include panorama 3D sensor 17, and robot 1 is able to detect and is located at robot 1 into any angle
Object.3D sensor 12,15,17,30,31,32,34,35 and 36 can be any kind of 2D or 3D video camera, Laser Measuring
Distance meter, 2D or 3D scanner, structured light scanner, stereo camera, ultrasonic wave, radar, laser radar or for detect 2D or
Any other sensing or scanning device that 3D is statically and/or dynamically moved.In one embodiment, upper sensor is Microsoft
(Microsoft) Kinect device；Lower sensor is northern positive (Hokuyo) laser radar apparatus；And panorama 3D sensor is
Power steps on (Velodyne) laser radar apparatus.
Robot 1 may include that there is the touch screen of touch-screen display 50, computer 51 and eye-tracking device 29 to set
Standby 19.Touch-screen display 50 may include information being presented to client and to allow client to interact with touch-screen display 50 any
The interactive graphical user interface (GUI) of type.Computer 51 includes that enough processing capacities are soft to run various touch compatibilities
Part, including such as 8 operating system of Windows.Back displays device 20 may be coupled to touch panel device computer 51, and can
To include the touch screen feature similar to touch panel device 19.The use of touch panel device 19 and back displays device 20 allows machine
People 1 and the client connection from multiple directions.For example, the first client can be used the touch panel device 19 in the front of robot 1 with
Robot 1 interacts, and the second client watches the information being shown on back displays device 20.
More array microphones 16 be connected to touch panel device computer 51 and can make robot 1 carry out auditory localization and
Ambient noise inhibits.Using microphone 16, robot 1 can understand the speech of multiple people, and the area from ambient noise simultaneously
Divide the speech of particular person.One or more speakers 18 are also included in robot 1, allow robot 1 and client's term
Sound exchange.Loudspeaker 18 is connected to computer 51, and computer 51 will detect ambient noise by microphone 16, and determine loudspeaker
18 correct speaker volume.The voice of 1 exchange of robot may include any language.
Robot 1 can also include electric system 22.Electric system 22 may include battery 24 and charging system 25.Battery
24 can be rechargeable lead-acid battery or the battery of any other type.Charging system 25 may include allowing 1 electricity of robot
Be connected to Docking station (not shown) for charging interface.Electric system 22 may include power distribution circuit and component, including adjust
Device, radiator, fuse and/or breaker.In addition, electric system 22 may include emergency cut-off circuit, it can be at certain
The electric power from robot 1 is automatically or manually cut off in a little situations, for example, battery is lower than a certain minimum if battery is too warm
Threshold value charge, or if robot is moved to except presumptive area.According to the degree of user's mobile robot, battery life can
It can be very different.Preferably, battery types, size and capacity allow to charge twice between using eight hours.
Robot 1 can also include embedded system 26, have one of the control for being used for robot 1 and sensing interface
Or multiple processors 27 and associated circuit 28.Processor 27 can be such as Arduino Mega microcontroller, allow
It is easily developed together with the Serial output for control platform 11, and can serve as to robot computer 14 and interface is provided
Serial (for example, passing through USB) equipment.Processor 27 can be any processor, microprocessor or microcontroller, and can be with
It is PIC microcontroller, is usually that high speed USB that is powerful and being allowed for data to transmit is connected with Ethernet.Processor
27 may include same amount of computer-readable memory or associated with same amount of computer-readable memory.
Robot 1 can also include robot location's detector 33.Robot location's detector 33 can use it is a variety of
Know that any one of location detecting technology is used to detect the position of robot 1, including global positioning system (GPS), interior are determined
Position system (IPS) and inertial navigation system (INS).Robot location's detector 33 can also be with any amount of map, plane
The similar principles figure co-ordination of the layout of the facility of layout drawing or use robot 1.Robot 1 can also include that can receive
At least one RFID antenna 56 of RFID signal is sended and received to robot 1 and/or from robot 1 with transmission RFID signal.
Additional sensor may include in robot 1.These may include following any one: thermal camera 37
(for example, personal blood flow distribution for sensing Environmental Heat Source and client)；Radar sensor or Doppler radar sensing
Device 38 (personal breathing and heart rate for motion detection and detection client)；It can be may be quick to touch, power and/or pressure
Piezoresistance sensor, piezoelectric transducer, capacitance sensor and/or the electric resistance sensor of sense；Steep cliff sensor 40 is (" outstanding for sensing
The presence of precipice " or stair or other featuress of terrain)；Crash sensor 41 can be and be used for when robot 1 collides object
The force snesor of sensing；Humidity sensor 42 can be the hygrometer for sensing the humidity in surrounding air；Chemical sensitisation
Device 43 can be the potentiometric sensor of the chemical constituent for sensing environment；Vibrating sensor 44, can be for sensing
The piezoelectric transducer or accelerometer of vibration；For sensing the UV sensor 45 of ultraviolet radiation；Ultrasonic sensor 46,
It can be used for detecting people, the degree of filling for detecting the liquid in tank (for example, fuel tank etc.), for measuring wind speed and side
To, or it is used for any other known application；And/or for sense ambient temperature and/or for sense robot computer 14 or
The temperature sensor 47 of any other fever or sensing unit temperature.Each of these additional sensors can be with robot
Computer 14 and touch panel device 19 are communicated.
Any of above sensor and the other sensors and/or sense including such as biometric sensor 52 can be used
Survey technology carrys out the ability of expanding machinery people 1 to sense blood flow, breathing and the heart rate of the client observed, retinal pattern, and
Body fluid compartments.For example, video camera 39 can detecte the mamillary expansion and retinal pattern of people.Thermal camera 37 or other heat
Imaging device can sense the blood flow and body fluid compartments of people.For example, video camera 39 can sense the breathing of people, and can be with base
The breathing mould moved to determine people in the nostril of people when the visible expansion and contraction or air-breathing and expiration of the chest of people
Formula.Also, for example, because heart rate variation based on skin brightness, the variation of blood flow and/or can use capacity coupled displacement
Electric current determines, can be sensed and be determined based on the information sensed by video camera 39, thermal camera 37 or potentiometric sensor
The heart rate of people.These parameters facilitate " psychological condition " of assessment observation individual.Similarly, these skills can be used in robot 1
Art monitors client to identify whether customer action shows that s/he is felt confused or dejected, and in this case, robot can be with
It approaches and help is provided.
Fig. 2 is according to Fig. 1 of the first illustrative embodiments of the disclosure for providing customer service in facility
The block diagram of the subsystem of robot device 1.Therefore, each frame of Fig. 2/module indicates the subsystem of robot 1.Video input and
Export 201 subsystems, audio input and output 202 subsystems, wireless communication 203 subsystems, 204 subsystem of 3D sensor and its
He outputs and inputs 205 subsystems and communicates with calculating, data storage, order and control subsystem 200.Subsystem 200 with it is micro-
Controller and I/O plate 210 communicate, and microcontroller and I/O plate 210 control mobile platform 211 and servo, actuator and executor
212, and connect electromechanical and optical sensor 213.When each subsystem is simultaneously in use, robot 1 can be into facility
Client provides customer service and otherwise executes various functionality described herein.
Fig. 3 be according to the first illustrative embodiments of the disclosure for the robot device 1 using Fig. 1 in facility
The schematic diagram of the robot system 10 of customer service is provided.Relative to Fig. 1 and Fig. 3, robot 1 can be with robot system
10 other component is communicated to promote comprehensive function.The wireless telecom equipment 13 of robot 1 can by computer network 3 with
Calculating center 4 communicates.Network 3 may include any amount of network addressable unit, such as first network 101, server
100, the second network 102, local radio communication equipment 103 (it can be such as local wireless router) or another machine
People.When system 10 includes more than one robot 1, robot 1 can communicate with one another to share its position or at another
Its position, or exchange other information are shown on the instrument board of robot.Computer network 101 and 102, which can be, to be set
Any wired or wireless computer network communicated between standby, including local area network (LAN), wide area network (WAN), internet,
Wireless LAN, Wi-Fi, mobile device network, IEEE 802.11, GSM, GPRS, UMTS, 3G and 4G communication protocol or this field
Any other network arrangement known to those of ordinary skill and/or agreement.
Calculating center 4 can be any cloud computing center, data center, supercomputer and other powerful calculating
Resource.Calculating center 4 may include promotion database 140, and it includes such as promotional video, image, product description, promotion are excellent
Elder generation, promotional offer, the various sales promotion information and product material for promoting history, Brand Relationship and other information.May be used also at calculating center 4
To include the digital inventory database 141 of facility, digital inventory database 141 may include the three-dimensional scanning about product
Structural information, and calculate center 4 and can share the inventory database 141 with robot computer 14, such as showing
Consumer.The inventory database 141 can be integrated into the map of facility by calculating center 4, and therefore can be with so high
Precision identify the position of product in map, the different product on same shelf is fully distinguished for inventory
Control purpose.Calculating center 4 can also include customer database 142, it includes the anonymous information of client, such as purchasing history,
Information of member card, robot interactive history and customer action pattern analysis etc..
Robot 1 can be communicated by computer network 3 and operator's network system 5 with station 2 is supported.Support station 2
Can be used for that operator 80 and the client interacted with robot 1 70 is allowed to communicate.Support station may include video camera 121,
Support station computer 114, touch-screen display 120, microphone 116, loudspeaker 118 and such as keyboard, mouse or control stick
Input equipment 117.Station 2 is supported to be normally located remotely from the position of robot 1, such as in remote client's service for facility
The heart.Although illustrating only a robot 1 and support station 2, those of ordinary skill in the related art it will be readily understood that
System 10 may include multiple robots 1 and/or multiple supports station 2.
Robot 1 (for example, by network 3) can be sent to by the video image for supporting the video camera 121 at station 2 to shoot,
Wherein they may be displayed on touch-screen display 50.Audio signal can also be by supporting station microphone 116 to pick up and sending
To robot 1, they can be broadcasted by loudspeaker 18 there.Similarly, vision signal and audio signal can be by robots 1
It picks up and is sent to support station 2.Therefore, the operator 80 at support station 2 can check the image of distant place robot environment simultaneously
Hear the audio of distant place robot environment.
Therefore, can execute between the operator 80 at support station 2 and the client 70 communicated with robot 1 has audio
The bidirectional video communications of connection.For example, the video camera 39 of robot 1 can capture the video counts of the people in camera coverage
According to, and microphone 16 can capture the audio-frequency information from people.The information is sent to support station 2, it can be exported there
To display 120 and/or loudspeaker 118.Meanwhile carrying out the video information of operator 80 at self-supporting station 2 and audio-frequency information can
To be captured respectively by video camera 121 and microphone 116, and it is sent to robot 1, it can be output to display 50 there
And/or loudspeaker 18.The present invention can use with platform-independent two-way video chat protocol, do not need public ip address, no
Special tunnel (tunneling) or firewall configuration are needed, does not need remote information technical assistance, and can use simultaneously
Wi-Fi is connected with 4G.
Operator 80 can support station 2 to issue order to robot 1 to navigate to be located remotely from and support station 2 by operation
Some position.It can be realized with touch-screen display 120 or the input equipment 117 communicated with support station computer 114
The order of robot 1.Order can be received and be explained by computer 114, and for example pass through network 101, server 100, net
Then network 103, wireless telecom equipment 103 are sent to robot 1, robot computer 14 to mobile platform 11.Robot 1 will be rung
It should be mobile as needed in the received order of institute.
System 10 can operate together with System Back-end software 131, and System Back-end software 131 can be any commercially available
Installation and operation in computer operating system (including Windows, Mac OS, Linux, Android, iOS etc.).Facility managers
Or agent 81 can on facility computer and mobile device (such as smart phone and tablet computer) on install and use
Software 131.The software can permit the sequence and priority of the task of manager or the definition of agent 81 robot 1, and will be all
Such as promotional video, image, product description show and colour upload to the promotion database 140 at calculating center 4.Software 131 can
To automatically process the information inputted by manager or agent 81, and it is converted into the format that robot 1 is understood that.One
In a example, software 131 can be operated or be connected thereto by calculating center 4, and including being connected to digital inventory database 141
Digital inventory's interface, and can be used in facility execute stock control task, as Fig. 9-22 is further described.
Operation example:
Robot 1 for example can be explored automatically using its sensor total field of view, top 3D sensor and lower part 3D sensor
Facility.It can use the viewdata captured by 3D sensor and can be by such as GPS, IPS, INS or other such technologies
The corresponding location information that robot location's detector determines generates the map of facility.The map of facility can store in machine
In people's computer 14 or computer center 4.Top 3D sensor can be used to scan by client's bring sample in robot 1,
And found in digital inventory with the matched structural information of the structural information of sample, to position the product in facility.So
Afterwards, robot 1 can by touch panel device 19 or back displays device 20 to client show about product information (such as
Its position in facility, price, sales promotion information etc.) or by guidance client to facility in product position come to client provide
Further service.
Robot 1 for example can carry out small movement by using its mobile platform 11 or its robot head 23 to attract
The attention of client.Robot 1 can also be by showing static graphics or dynamic on touch panel device 19 or back displays device 20
Figure (it may include head portrait figure) is come the attention that attracts clients.Robot 1 can generate audio by using loudspeaker 18
Signal is come the attention that further attracts clients.Robot 1 can be by monitoring the AC response rate of client, end rotation, face
Expression mode or otherwise sensing various parameters further detect client 70 and determine to its interest " the psychological shape of client
State ", as previously described.When interest is lower, robot will skip this client, and navigate back to its home position in facility.
The home position of robot 1 can be the position or any in facility of facility managers or the predetermined charging station of agent
Other positions.In the way for returning to its home position, robot 1 can select may have more clients based on the data of capture
The route of density, and other clients for needing to service are found using upper part 3D sensor.Robot 1 can pass through analysis
Facial expression, walking mode, end rotation, visual direction of client etc. determine client that needs service.If robot 1 is looked for
To the client that may be wanted help, it will inquire whether client wants help close to client and using voice and graphical information.
Once interacting between robot 1 and client, robot 1 also can detecte whether client keeps contacting with it.Work as machine
People 1 has found client when being no longer participate in interactive, it returns to the task in its home position by current task and restoring is terminated.
Calculating center 4 and computer 14 can each self-operating the speech engine software of multilingual packet can be used.As
As those skilled in the relevant arts will readily appreciate that, speech engine software can be or including any of speech engine,
Speech-to-text or other such technologies.Speech engine software enables robot 1 with multilingual and client in nature
It is contacted in voice dialogue.Robot 1 will receive spoken message by microphone 16 in voice dialogue.At known information
Reason technology, spoken message will be handled by computer 14 and calculating center 4, to extract key message and to understand the need of client
It wants, such as purchase interest, product information, position enquiring, counsel requests, and executes other customer service action.
Client 70 can exchange with robot 1 about his/her demand by voice or by operation touch-screen equipment 19,
And if the product needed in facility without client, robot 1 can provide certain substitution solutions for client.Example
Such as, if client wants to buy cereal in retail shop, robot 1 can be by accessing the product being stored in calculating center 4
Inventory to provide total Options and the milk promotion of bread basket on touch panel device 19, because the people of purchase cereal is also very
Milk may be bought.Meanwhile robot 1 provides client 70 being directed to cereal fraction, such as by saying that " I can guide you to 10
Cereal fraction in number channel? " if client 70 wants receiving service, he/her be can be confirmed (for example, by saying " yes,
Please "), that position that then robot 1 will guide client 70 to arrive shop.If the desired cereal of client 70 has been sold out in shop
Product, then robot 1 can be provided in line and order cereal and be conveyed to the address of client (it, which can store, is for example calculating
In center 4).
Robot 1 can provide information associated with customer demand and suggestion on touch panel device 19, and may be used also
Client 70 is directed to the part that can be able to satisfy the facility of needs to provide.If client 70 receives guidance, robot 1 will
The path for connecting current location to required part is found on map indoors.Required part from current location to facility can have
Multiple Path Options.Therefore, robot 1 can use path optimization's algorithm to determine which in the multiple Path Options of selection
It is a.For example, robot can based on the considerations of for example show promotion in product and effectively arrive at the destination select most preferably
Path.Robot 1 " can know " which product has high promotion priority by accessing promotion database 140.
Robot 1 is intended to work together with the client on side and other people with friendly and intuitive way.Into one in Fig. 4
Step description, when the navigation of robot 1 is by very dynamic environment present in facility indoors or other business locations,
Dynamically it can talk and exchange with people.For example, robot 1 can roam around facility attempts to inquire whether random client needs
Any help is wanted, rather than it is motionless and wait interaction to be parked in home position.It will not cross-question client, and only some when
Between inquire client, in order to avoid frustrating.When client is close, it can detect the face of client to know when on the scene they are.
Robot 1 can be by acoustic expression oneself, can also be by acting and the color of mood lamp express oneself, and mood lamp can be with
Including the various light emitting diodes (LEDs) being located in robot 1.Robot 1 will be relatively slow mobile in busy/crowded part
And by its LED color is changed to indicate the various states locating for it, for example, it is crowded, lose, help someone, contact, normally patrol
It patrols or other states.Other than the communication of audible and LED based, it can be used only when meeting new client in robot 1
Pedestal come execute wobbling action with obtain they attention and/or attempt with they contact with.
When navigation by inside plant or guide client pass through inside plant when, robot 1 can express its state and its
To the consciousness of environment, to seem related to client.For this purpose, when robot 1 is waiting people just on the way, or pass through
When close people, robot will say the phrase of some courtesy, such as " sorry (Excuse me) ", when the biography of robot
Sensor due to many people around without allow its very freely navigate when, robot by the client for reminding it guiding it
Feel crowded, and when robot 1 is immovable, it can apologize sincerely, and say that it is seized.When robot 1 is in mesh
Cursor position is nearby and when its route of barrier obstruction, and robot 1 will notify client, operator or other people articles just preceding
Side, rather than change the very long path around route to retail space.
Fig. 4 is the active path rule used according to the robot device 1 of Fig. 1 of the first illustrative embodiments of the disclosure
The schematic diagram drawn.When facility is most of idle, finding the path in facility for client usually may be simple mapping process.
However, many facilities may move in building there are many object, this create robots 1 must be in the dynamic wherein operated
Environment.For example, these facilities may include shop, market, shopping center, grocery store, market, theme park, warehouse, airport,
Bus or railway station, or need any other public or private facility existing for robot or place.Robot 1 is necessary
It can be operated in the dynamic environment for the facility that it is used.For example, have in business environment indoors many mobile objects (from people to
Shopping cart arrives mobile goods plate), robot must navigate through wherein.In these dynamic environment, robot 1 be can use
The active path planning of moving obstacle is considered when design path.
Consider the example that client walks in 1 front of mobile robot.As shown in figure 4, robot 1 can be along typical path
164 (for example, moving predetermined path in detection barrier 166) 160 are moved along a first direction towards destination 162
It is dynamic.The typical path 164 can be made of typical path planning algorithm, which simply designs towards barrier 166
The path on right side, because barrier 166 is located at the left side of robot 1 in planning.However, when barrier 166 starts along direction
168 it is mobile when, it is mobile towards the right side of robot 1 that robot 1 must take into account barrier 166, and be therefore moved to by with
The position of the interception of typical path 164 simultaneously causes robot 1 and barrier 166 to collide.
Collision between robot 1 and moving obstacle 166 in order to prevent, robot 1 can create the second path or master
Dynamic path 170 is navigated in a manner of depending on the movement of barrier 166 around barrier 166.For example, the detection of robot 1 can
To include the moving obstacle 166 of any object (for example, people, cart etc.), and track the movement of moving obstacle 166.Base
In the movement of moving obstacle 166, robot 1 determines that navigation is navigated by moving obstacle 166 or around moving obstacle 166
Optimal path.In as shown in Figure 4, active path 170 can surround the rear portion of moving obstacle 166.Robot can lead to
It crosses initiatively braking robot 1 and carrys out quick response emergent barrier in its visual field.The active brake may include
Brief speed command is provided on the direction opposite with the movement of robot.
It should be noted that robot 1 can be used radar, physical shock sensor and 3d depth transducer combination or
Other sensors disclosed herein, to provide additional detectability for the barrier for being more difficult to detection.These barriers
May be included in the children on or below the travelling route of robot 1 and small barrier, main sensors may to this view and
Lose.Collision detection and obstacle avoidance can usually be helped by being used together these sensors, to ensure that robot 1 is young in facility
Carefully and successfully navigate.
Fig. 5 be according to the first illustrative embodiments of the disclosure in facility article and client interact
The schematic diagram of the robot device 1 of Fig. 1.Robot 1 can may include display structure 300 in the facility wherein operated, can be with
It is any structure for showing product or service in facility.Common display structure may include shelf, desk, exhibition booth or
Exhibition position.Display structure 300 may include any amount of article 301 on it, and can also include one or more use
In the wireless transmitter 302 of automatic identification and data capture.
Relative to Fig. 1-2 and Fig. 5, as the guidance of robot 1 or adjoint client 70 and mobile close display structure 300,
Structure 300 and article 301 thereon, and overleaf display can be identified by one or more of its 3D sensor
Sales promotion information 310 relevant to article 301 is shown on 20.Sales promotion information 310 on back displays device 20 may include article 301
One or more images 311, the text 313 about article 301, the promotion title 312 about article 301 and about structure
300 essential information 314.Essential information 314 may include structure feature, product category and subclass, position etc..For example, such as
Fruit structure 300 is shelf or desk in retail shop, and when robot 1 guides client to pass through channel, display 20 can edge
Channel show the advertisement of shelf or the product on desk.When showing the advertisement of product, robot 1 can be by promotion state note
It records in the promotion database for calculating center 4, which, which can monitor and analyze, provides for each brand in inventory
How many promotion.Calculating center 4 is recorded in the quantity that robot 1 can also be directed to the client of some display structure 300
In promotion database.
Robot 1 can be used as the personal shopper of client 70, and personalized service and suggestion are provided to client 70.
The ability that robot 1 provides this personalized service can provide branch by the artificial engine software run on calculating center 4
It holds.For example, robot 1 can observe the hair color of client, eye when client 70 is look for the special article of such as clothes
Eyeball color, facial contours, height, body shape etc. simultaneously propose fashion suggestion.It may include clothes, accessories or other such objects
Associated fashion consultation information between product and the physics or other features of people can store in calculating center 4 and by robot
1 access, to provide personalized fashion suggestion.For product that is too big or being easy to steal, robot 1 can be mentioned to client 70
View receives product in cash desk.Client 70 can select to receive product by voice informing robot 1, or can click on
Reception button on touch panel device.Once client 70 confirms that he/her wants to receive product in cash desk, robot 1 can be beaten
A receipt is printed off so that client 70 takes cash desk to.
When client 70 checks display structure 300, robot 1 can track the eyes of client by eye tracking apparatus 29
Its 3D sensor shooting image is moved through to monitor the behavior of client.Robot 1 can transmit data to calculating center
4, calculating center 4 can analyze data and extract information about client 70, for example, client check product, client is to production
The emotion of product, client spend in time on product etc..
When robot 1 can operate in facility, it can be located at a position by its 3D sensor whithin a period of time
Place shoots the picture of client to monitor that client flows.For example, robot 1 can be arrived down in retail shop with 6 points (6PM) in the afternoon
Client's photo in channel is come into and is walked out in shooting between 7 points of noon (7PM), and sends calculating center 4 for these photos, in calculating
The heart 4 can analyze picture and calculate that how many people during this period of time passes through this channel.Calculating center 4 can further be divided
Client's flow data is analysed and integrated, and the mobile figure of the client being formed in entire facility, may include as client spends big portion
The information of most-often used any paths in part and client between timesharing.
Robot 1 can pass through its sensor (such as microphone, 3D sensor, touch screen, biometric sensor etc.)
The feature of client is captured, and sends calculating center 4 for client characteristics data.Calculating center 4 will handle this feature data and look for
To particular customer 70 certain modes (such as speech pattern, facial characteristics, gesture mode, bio-identification mode etc.) and deposited
It stores up in the customer database for calculating center 4.When client 70 returns in facility, robot 1 can for example pass through access and store
Client 70 is identified as returning to client by the information customer characteristic in customer database.When robot 1 passes through above-mentioned processing
Recognize when returning to client, can current purchase interest based on the client and in the past shopping history in 19 He of touch panel device
Advertisement is shown on back displays device 20.The purchase that robot can return to client by accesses customer database 142 " knowing " is emerging
Interest and shopping history.
Robot 1 can monitor any failure of the equipment in facility and reporting device or need to clean or safeguard.For example,
Robot 1 can monitor refrigerator and household freezer in retail shop, and if the equipment there are failures, or if there is opening
Door, or if there is leakage or other cleanings need, then robot 1 to System Back-end send report.Robot 1 can be into
One step monitors the price of the product in facility, and the price in the display price of such as fruit product and digital inventory mismatches, then
Report will be sent to System Back-end.
Client 70 can be by requesting the mankind to represent or operate with 1 intercommunication of robot or by operation touch-screen equipment 19
Person 80 logs on to robot 1.Access operator network system 5 is requested operator network system to be connected to by robot 1
One of available operator.Search engine 150 can be used to search all available operators in operator network system 5, and
And all available operators are dialed in simultaneously using dialer software 151.The operator of first response will be connected to robot 1,
And the operator 80 connected can be used support station 2 and log on to robot 1.For example, if client 70 wants and graphic artist
Talk, he/her can tell the graphic artist on robot " I wants to talk with graphic artist " or point touching screen equipment 19
Button, then robot 1 by access operator network system 5 and one of is connected to the available graphic artist on network.Then connect
The computer or smart phone login robot 1 supported on station can be used in the graphic artist connect.
It, can be by showing structure using the shooting of its 3D sensor when robot 1 operates near structure 300
The image of product carrys out the cleaning of the inventory in the display structure 300 of auxiliary equipment on 300.For example, when robot 1 is in retail shop
When moving along channel, the image of the product on the shelf along the channel can be shot.Robot 1 can also be shot in basket
The image of son, shopping cart and the product in client's hand.Robot 1 can send all these images to calculating center 4, and
Calculating center 4 can handle these pictures, identify and distinguish between product, calculate the product quantity left in display structure 300, report
The dislocation product that be placed in structure 300 is accused, and updates all these information in inventory database.When there are misplaced
When product, or when the quantity of some product is lower than preset warning threshold in inventory database, robot 1 can be to after system
Software is held to send alert message, so that facility managers or agent 81 can be arranged to replace misplaced product
And/or product is reloaded in structure 300.
In another embodiment, the method for view-based access control model can be used to determine which in display structure 300 in robot 1
A little articles are out of stock and which article is potential low stock.The process can permit on the estimation shelf of robot 1 or other libraries
The quantity of product in depositing, such as by the product bought for customer displayed on estimation shelf and it is positioned for supply future
The product (case and wait to be placed to the product on shelf) of inventory (inventor).In the method for this view-based access control model, machine
Camera system on people 1 is optimized to shoot image under correct focusing and illumination level.By collecting FRID label information
And/or it can be with the number of further expansion view-based access control model if 3D sensor (if what such information of acting in the middle part of commercial facility)
According to reach even higher accuracy.Can be run in the heart 4 and robot computer 14 in a computer advanced analysis with
Data are checked and analyzed to track.In addition, the method for view-based access control model can predict when inventory is likely to occur shortage of goods in the future, provide
The instruction of the Best Times of new inventory is potentially ordered in advance.
The method of view-based access control model may include any amount of step and mistake for assessing the inventory in display structure 300
Journey.For example, during one day special time period, such as in facility unappropriated early morning, robot 1 can be activated and be passed through
Facility.When robot 1 is by each display structure 300, robot 1 uses its camera system and its any 2D or 3D sensor
High-definition picture is shot for each display structure 300, note is intended to the position XY and its direction (angle on the map of facility generation
Degree).High level computer vision algorithm can be used and the heart 4 or handle image in robot computer 14 in a computer, identify
It shows each product in structure 300 and identifies which article is out of stock.Stock information is transferred to inventory database 141.
The inventory management system of photo or a part of method are captured in the channel as robot 1, it may be predetermined that
The position in the shooting of robot 1 passageway.These pre-determining positions are properly termed as " way point ".By predefining in specific way point
The position at place, it can be ensured that there are enough overlappings between captured adjacent picture, with successfully capture all products and/or
The image of all shelf spaces.If not having enough overlappings between picture, robot 1 may return to the position and retry
Way point.If there is mistake (for example, if captured image goes wrong or robot is obstructed and nothing in initial photo
Method captures way point), then it is possible that such case.
In addition, the intelligence of computer vision, machine learning and mechanical robot, Turkey (turk) can be used in robot 1
Combination is to capture any image with processing screened.System can in captured image positioning product, and by shelf
Image is associated with the dummy model of the shelf in shop.This dummy model of shelf is properly termed as " main shelf ", and its
It can have the complete inventory of product.By using this main shelf technology, if vision used by for some reason is calculated
Method can not identify the label of certain products, then system can use historical product space (such as by main shelf identify that
Sample) can heuristically identify product.
Robot 1 can execute other inventory's incomes to the facility operated in it.For example, when robot 1 captures shelf
On product image when, can identify misplaced article.When this occurs, robot 1 can not only assess object
What product are, additionally it is possible to calculate the quantity of misplaced article.The information can be used for assessing the physical holding of stock quantity of product.The letter
Breath can be also used for sending notice or message to inventory personnel, and misplaced article is placed on to the correct position in facility.
Robot 1 can also be by information and PERCOM peripheral communication that it is collected in facility.For example, robot 1 can be to article
Specific company or supplier send notice or message.This may the generation when robot 1 finds out of stock or low stock article,
It can be used for monitoring the other information about product, such as optimal use date.If the scanning of robot 1 is possessed by particular brand
Shelf, then system can send to the supplier of brand and notify.The supplier of brand is able to access that system (at least has limited
Access right), to check the current state of the shelf of the brand and determine that next product is delivered to the Best Times of specific position.
Supplier can also provide the feedback in relation to being supplied to the accuracy of its information to system.If shelf are by possessing facility or shop
Company possessed, then system can to the employee of the said firm provide display shelf current state instrument board, such as relative to
Instrument board in software 131 described in Fig. 9-22.
Inventory database 141 can also obtain facility data from the inventory system of facility.It then can be by robot 1
Scan data is associated with the inventory system of facility, is identified as people and pays attention to the difference prepared.It is then possible to be the work in facility
Make personnel agent 81 can be used software 131 check it should be noted that facility in item lists and be not required to it is noted that
Item lists, and each situation is handled in daily work.Software 131 can also allow facility managers check employee work
Work and any operation that they may need to do, such as order additional product inventory.Software 131 can also allow enterprise/total
Portion staff supervises the situation of facility, and checks the analysis of inventory.Which article software 131 can be it is further proposed that or predicts
In the near future may be out of stock, the manager and associate of facility can solve these and suggest or predict.
Various technology and methods can be used to improve the accuracy of the image capture capabilities of robot 1.For example, indoors
In retail space, image capture demand can change, such as visual field and picture size according to several key factors.These factors
It can be determined by the physical size of the robot 1 and the identified object of needs that are located in channel.It therefore, can be by image weight
Folded requirement is drawn to determine placement of the video camera on 1 platform of robot so that there are enough overlappings with compensating due to avoidance
The variation of the robot path risen.The image of overlapping provides safety margin, can be to avoid by lacking number in the set of capture
Strong point and the problem of cause.
Another technology that robot 1 uses is to adjust camera parameters according to the environment of facility.For example, using from navigation
Information derived from data, robot 1 have the ability to change the focal length for capturing image automatically.The ability can be by knowing channel width
It is determined with positioning of the robot 1 in shop.Auto-focusing can be enabled in highly dynamic environment, wherein general image
Clarity is priority.In another embodiment, the camera system of robot 1 can be automatically configured to suitable in facility
Illumination is properly focused on the high-definition image during inventory assessment on the product in display structure 300.Camera shooting
This adjustment of machine parameter includes making to fill up illumination adjustment image sensor sensitivity automatically, this can be there are shade or low light
Retail space region in provide clearly and consistent imaging.Illumination is filled up by the integral illumination in specific image capture region
It determines.Illumination can be in intensity and proliferation it is adjustable, to better adapt to environmental demand.
In addition, illumination is filled up in addition also allows higher shutter speed, the speed of platform is increased, to reduce
Stock control process completes the required time.For this purpose, the speed of shooting image has directly required shutter speed and illumination
It influences.Higher platform speed will reduce in data group the time needed for image capture, but need additional illumination to keep scheming
Image quality amount and consistency.Lower platform speed will increase the image capture time and reduce required additional illumination amount.Robot 1
It can be according to status requirement adjust automatically platform speed.
Fig. 6 is the offer customer service in facility for Fig. 1 according to the first illustrative embodiments of the disclosure
The block diagram 400 for the vision algorithm that robot device uses.It should be noted that the description of any process or frame in flow chart are interpreted as
Representation module, section, code section include the steps instructed for realizing the one or more of specific logical function in this process
Suddenly, and as disclosed technique field the skilled person will understand that, the realization of substitution include within the scope of this disclosure, wherein
Function can be executed from shown or discussion sequence, including substantially simultaneously or in reverse order, be depended on involved
Function.
The image of camera system and sensor shooting can be used to identify in facility in the vision algorithm that robot uses
Show the product in structure.In an embodiment of vision algorithm, navigates robot autonomously and channel and scan product
It shows (frame 401).Navigation channel and the display for scanning product may include generating street view, 3D data or the aphorama in channel
Scheme (frame 402).Robot Scanning bar code and product, and according to any article (frame 403) out of stock of the article identification being not present.Base
In the data capture from robot, robot updates interior data library or about the external database (frame for calculating environment
404).Then, facility staff personnel can check article on inventory's instrument board on the computing device, which carries
In robot or outside robot (frame 405).Then, facility staff personnel can practically handle display difference article and
Corresponding movement (box 406) is marked on the instrument board of robot.
Vision algorithm is carrying out tag extraction, the barcode detection of product and the extraction of product, is determining whether article is out of stock
Or in stock and provide indoor environment streetscape in terms of can be particularly useful.Relative to tag extraction, robot be can use
Color threshold and contour detecting determine the position of the Product labelling comprising label information.Then the label of extraction is used for bar shaped
Code detection.Barcode detection can use the gradient magnitude of image (label) both horizontally and vertically, can be used
Scharr operator determines.It can determine the region with high-level gradient and low vertical gradient.It can be smooth from gradient image
High-frequency noise.Blurred picture may need to carry out threshold process, and accord on threshold binary image using morphological operations.Use profile
The barcode size or text field of Detection and Extraction from label allows to identify Item Information, price, the position of article and for search graph
The window size of article as in.
In order to determine which article is in stock or (showing the availability of the article in structure) out of stock, form can be transported
Operator is applied to structural context.In general, the channel of retail shop is divided into three kinds of different classifications: hook (pegs), shelf and support
Disk.Such as consider hook article, if an article shortage of goods, can detecte the circle in the background of channel.In predetermined window size
Circle density aid in determining whether which article is out of stock.In order to reduce region of search, window size can be encoded in channel
On bar code in.It is possible, firstly, to extract the region comprising bar coded sticker using different morphological operations symbols, object is provided
The descriptions of product, position (channel and article), price and window size.Around label, described window then can be determined
Circle density in size.If circle density is higher, it can determine that the article is out of stock.
It should be noted that procedures described herein can be used together with the multiple images of compiling together, so-called " figure
As splicing ".Image mosaic may be implemented to consider the region of the image close to image boundary.It should be noted that relative to room
Interior streetscape, after scanning and processing, the image and information of the availability about article can be available.It is spelled using image
It connects, the streetscape view of indoor version can also be generated from data, such as continuous image mosaic is together to form in facility
The consecutive image in channel.
Robot 1 can have the other function that can be used alone or be used in combination with robot capability as described herein
And nonproductive task.For example, robot 1 can in facility conveying products.Robot system may be coupled to the online of retailer
Order system, and robot 1 can have cludy, compartment or other regions in its main body or in its main body, can use
In article is moved to dedicated location be used for online order deliver or picking.Therefore, when on-line purchase product, robot energy
It is enough to fetch product from facility and bring it to customer service platform so as to client's picking.Robot 1 can also receive to come from user
E-payment, such as by receiving credit card or the e-payment of other forms (such as using cellular phone).Vehicle electronics
Payment system may be coupled to the payment system of commercial facility, and the transaction and payment that occur with robot 1 can recorde
In the payment system that facility uses.When robot 1 can fetch on-line purchase and receive the product of e-payment, on-line purchase
The customer of product directly can fetch his or her product from robot 1, and wherein then robot 1 will for customer payment robot 1
Product is released to customer.
Fig. 7 is the machine in the facility using the robot device of Fig. 1 according to the first illustrative embodiments of the disclosure
The schematic diagram of device people's system.As shown in fig. 7, robot 1 can be connect with Docking station 110.Docking station may include shell structure
111, shell structure 111 has physical alignment guiding piece 112, facilitates the charging end on the motion platform 11 by robot 1
Mouth 113a is properly aligned with the compatibility charhing unit 113b on Docking station 110.Docking station 110 may include charging equipment and group
Part enables charge to be transmitted to the charging port 113a of robot 1 by the electric contact on charhing unit 113b, so that machine
People 1 is re-energised.It should be noted that the cooperation between charging port 113a and charhing unit 113b may include a variety of different
Design and configuration, it is all these all to think within the scope of this disclosure.In addition, although charging station 110 is mainly used for robot 1
Charging, it is noted that charging station 110 can be also used for energy otherwise as the charging of robot 1, and can
For being connected to robot 1 for other function (such as communication or maintenance).
It should be noted that robot 1 can be combined with Docking station 110 executes various self-charging functions.In an example
In, robot 1 can be programmed for based on timetable or as needed determining sleep pattern and automatically wake up mode.At this
In example, robot 1 can detect when that battery capacity is low.If it is, its own can automatically be transported to pair by it
It goes to the station to meet somebody.Then its own can be docked to Docking station 110 by it, once and its successfully complete, robot 1 can start to fill
Electricity.Then itself is placed in sleep pattern by it, wherein other than certain components (such as router, PC and power panel), machine
Most of components in people 1 all power off.This can help robot to be constantly connected to internet.When robot 1 completes to charge
Or when planned Bluetooth wakeup, it can voluntarily open and restart its software.Then its from Docking station be detached from, and as usual after
It is continuous to carry out customer service/inventory scan.
Fig. 7 also shows the additional mechanical feature that can be used together with robot 1.For example, can have can for robot 1
Installation part 115 is stretched, various devices, such as video camera, light source or other devices can be carried.Extensible installation part 115 can
To be telescopic device, or another device with the mechanical arm that can extend far from robot 1.When Camera Positioning is extensible
When on installation part 115, image can be captured from high vantage point.When high-resolution camera is mounted on extensible bracket
When on 115, these images can be high-definition picture.In conjunction with the movement of robot 1, can hang down on extensible bracket 115
Straight ground mobile camera and can laterally mobile camera around the facility in it using robot 1.Such as glisten
The light source of lamp etc can also be mounted on extensible bracket 115 and can extend vertically.By allowing robot 1 in shop
In shelf, capture image on display or other objects or light source be provided as needed, the use of extensible bracket 115 can
To provide advantage.
Fig. 8 is showing according to the mobile platform 11 of the robot device 1 of Fig. 1 of the first illustrative embodiments of the disclosure
It is intended to.Specifically, Fig. 8 shows 1 wheel suspension component of robot, can be used together with the motion platform 11 of robot 1.
Wheel suspension component may include the wheel and engine pack 11a for being attached to solid mounting plate 11c, including electric drive is started
Machine 11b, wheel 30 and structural frame members.Mounting plate 11c can be connected to wheel and electricity by one or more vertical axis 11d
Motivation component 11a, vertical axis 11d allow mounting plate 11c vertically to move relative to wheel and electric motor assembly 11a.One
Or multiple compressed spring 11e can be used for providing downward power on mounting plate 11c, to allow wheel 30 to keep constant with ground
Tractive force.
Fig. 9-22 is to be operated together according to the first illustrative embodiments of the disclosure and the software 131 of Fig. 3 to set
Apply the image of interior digital inventory's administration interface 500 that intelligent Service is provided.Digital inventory's administration interface 500 can have can be
The instrument board shown on graphic user interface, the instrument board can be used for completing various stock control responsibilities.User name can be used
Digital inventory's administration interface 500 is protected with password login screen, so that only approved individual is able to access that the number in system
According to.Once user's Internet access, he or she can check the data that robot 1 is collected, to execute usually by the people in shop
Inventory's task that work is handled in person.
As shown in figure 9, digital inventory's administration interface 500 may include map overview screen 502 comprising facility 504
Describe the graphical representation or map in channel or similar characteristics and provides a user the information centre 506 of inventory data.In information
The heart 506 can provide number or text data, identify the number of the inventory variance in entire facility or in the particular department of facility
Amount.Digital inventory's administration interface 500 can also include the different displays or configuration for navigating to digital inventory's administration interface 500
Various select buttons 505.The map of facility 504 may include identification frame 508 to indicate the position of inventory variance or at which
In may need the concern of people.For example, frame 508 can indicate the position on the special modality 509 of facility, wherein digital inventory
There are inventory variances between record according to the physical holding of stock of the determining product of the product inventory in library 141 (Fig. 3) and robot 1.
User can choose the specific part or frame of the map of facility 504, the inventory variance in the part to identify facility into one
Walk information.The process can permit the specific data that user individually fetches each frame 508.
Figure 10 shows the shelf page 510 of digital inventory's administration interface 500, can be according to the choosing of the frame 508 in Fig. 9
It selects to be shown to user.The shelf page 510 may include identify about selected frame 508 text information (including department mark
Knowledge, the position on passageway and shelf location).The shelf page 510 can also include the specific products in facility representated by frame 508
(such as title and product code of product) and appointed product whether there is in the identification information on shelf.It can also include producing
The image 512 of the shelf space for the facility that product should be located at.Image can be the photograph shot during stock processing by robot 1
Piece, and can be superimposed with figure 514 specifically to identify product should be located at where.As shown in Figure 10, figure 514 can be
Superposition shape (rectangle shown in such as) on photo should be located at the where on shelf with identification product out of stock.
If the user desired that obtaining the more information about Related product, then he or she can select from the shelf page 510
It selects the title of product and retrieves the detailed article page 516, as shown in figure 11.The detailed article page 516 may include the shelf page
The information (including having the image or photo 512 of the figure 514 for identifying (or low stock) product out of stock) for including on 510, may be used also
To include further information (SKU, existing product quantity, order product quantity including product, last order day about product
Phase, estimated order arrival date, last order receive quantity, product last sales date, minimum of date, last order of receiving are ordered
Odd number amount, show quantity, product whether be located at that multiple positions in facility, whether warehouse out of stock, whether product out-of-date and
Whether buyer has blocked styate) and other relevant informations.The information can be provided with standardized playlists or form,
In for each standard be supplied to the values of fixed output quota product.The detailed article page 516 can have select button 518, can be used for
Additional information or editor's information are retrieved, or adds the annotation about product.
As shown, the detailed product that interface allows user to access any given product in facility with inventory variance is believed
Breath.Using the information, then facility staff person can physically go to product that should be located at the position in facility, and use operation
The correct information update number inventory administration interface 500 of the portable computing device of software 131.For example, user is able to use
Select button 518 come solve the problems, such as, update article condition or addition about article annotation to be for future reference.
In order to check the summary of inventory variance, digital inventory's administration interface 500 can have Report Page 520, such as Figure 12
It is shown.Report Page 520 may include the conversion summary with the details about all inventory variances.This may include
It closes the project to be processed of inventory variance, solved the information of project and/or total project, these information may be by department or other points
Class provides.User can choose particular department (department 1A, 1B, 1C as shown in the figure) (such as " tool department "), should with retrieval
The summary data of project in department.The information may include being appointed using what the select button 518 of the detailed article page 516 inputted
What information or data, and about other data of the specific products in the department, as illustrated in figs. 13-14.The information can be with
Including select button 521, for executing various movements or acquisition status information.These select buttons 521 may include " protecting later
Deposit " button, " still opening " button, " adjustment database " button, " solution " button, " zero manually " button, " known theft "
Button, " replacement manually " button etc..
As shown in figure 13, other data 522 about product can also be provided, including product number, the title of product, production
The position of product and any record about Product Status from employee.In addition, as shown in figure 14, data can be with different
Form tissue, such as by listing all additional datas 522 about multiple products on the same page, wherein user can benefit
Shown data are further refined with select button 521.
Software 131 can also include that trend and data assay surface 530 as shown in figure 15 lead facility or department
The manager led is particularly useful.Trend and data assay surface 530 may include the homepage 532 with various menus, including
Global menu 534 and calendar select menu 536.Global menu 534 may include the product about adjustment on sale, inventory
Or the information of the percentage of dollar or the product of outmoded project.Calendar selection menu 536 can permit user and select specific time
Section, such as predetermined time frame or specific date are (for example, several days, a few weeks, some months or previous time period are (for example, " preceding several
Month ")).Calendar selection menu 536 can also have calendar combobox 537, and user is allowed to be manually entered or select specific time
Section.User can manipulate calendar selection menu 536 or calendar combobox 537 to retrieve the product data of any given time period.
Homepage 532 can also include the graphical representation 538 of data, such as produce relative to the inventory in each department of given time period
The percentage on sale of product.Graphical representation 538 may include the data line on chart, these data line keyings are shown to chart key
Data line correspond to specific products, department or the another aspect of inventory.Produce sector and its corresponding can also be provided for user
The list of product, to check inventory's percentage of the product of each department.
If user goes for more details about product in any particular department or department, he or she can
To extend any department's list, as shown in figure 16.Extension information can show classification or subclass 539 in particular department, together
When show the percentage of the specific set product of inventory in facility, so that user can extend or fold menu to retrieve required letter
Breath.Any classification or subclass 539 can be further expanded to check the information about specific products, it as shown in figure 17, can
To include product number and/or title, product size or about other features of product.If user asks for more information, he
Or she can select product on interface, and the detailed article page 516 of the product can be provided to them, such as Figure 18 institute
Show.Again, which may include the position relative to the number of SKU described in Figure 11, name of product, price, in market on shelf
It sets and additional stock information.
As shown in figure 19, user can carry out the existing tune of product inventory with usage trend and data assay surface 530
It is whole.In this feature, the various pages with label 542 can be provided a user, can choose label 542 to show in facility
Classification or subclass 539 list, such as pass through produce sector etc..User can extend any classification 539 to check such
More detailed product menu in not, as shown in figure 20.User can further expand menu to check in classification or subclass
Specific products 543.As shown in figure 20, the detailed data 544 about existing adjustment can be provided a user, be included in trend and
The quantity of the product reflected in data assay surface 530, the quantity of the product of adjusted inventory and it is responsible for any action
Employee or individual.User is capable of the movement that approval data reflects and approval button 546 or user can be by using other
Action Button 548 specifies specific products to need further movement.
As shown in figure 21, trend and data assay surface 530 can also be used to provide the existing adjustment about value of the dollar
Data.Adjustment dollar interface 540 may include the graphical display 552 about the dollar adjustment in a period of time.It is also possible to wrap
Containing the product listed by department, classification or subclass.User can expanded list to check any of specific products and the product
Value of the dollar adjustment.
Figure 22 depicts outmoded article interface 550, can be used for providing the data about the specific products in facility, the spy
Fixed output quota product are unsold or are not in great demand and therefore, it is considered that are outmoded.The product listed on the outmoded article page may include and
It sells or otherwise handles to increase the product of sales volume.For example, if when article does not have inventory to be more than 2 weeks or another section
Between, since customer does not buy it, product may be listed on outmoded article interface 550.Outmoded article interface 550 can be mentioned to user
Detailed bill and the product for each outmoded product related data (such as its date in facility, SKU number, on
Secondary order and the date received, the date of last time sale and Product Status).The data with graphically 554 and can mention
It is shown for the table of the product data about outmoded article.For example, table can provide the date having been sold out corresponding to product,
Date, the date that last time orders, the date of last time sale and the state that the identification information of SKU or article, last time receive
Title data 556 and other information.The product information 558 of each corresponding title can be listed in below title data 556.
Although Fig. 9-22 shows the certain interfaces and function of software 131, it is noted that software 131 can be based on
The data that facility inner machine people 1 collects execute many other functions, including the traditionally only function by manually performing.
It should be emphasized that the above embodiment of the disclosure, especially any " preferably " embodiment is only possible shows
Example embodiment, proposes just for the sake of the principle of the disclosure is expressly understood.Do not depart from the disclosure spirit and
In the case where principle, many change and modification can be carried out to the above embodiment of the disclosure.For example, robot can assess
The reliability of its sensing data, including navigation and vision and its explanation to the data.When robot determines that it is uncertain
When situation, for example, it can modify its behavior to keep its own safety and its surrounding based on the value in covariance matrix
Safety.It can also determine to ask for help when robot is not known.For example, being if robot not can determine that special article
No shortage of goods, the then photo that robot can send or store the place are interpreted for people.Similarly, if robot is mobile in shop
When be stuck and (physically block or only lose), then robot can require people by remotely take over so that its unclamp simultaneously
Return to its road.
All such modifications and variations are intended to be included in the scope of the present disclosure and are protected by the appended claims.
Claims (15)
1. a kind of for providing the robot of intelligent Service in commercial facility, comprising:
Mobile platform；
Upper sensor is used to detect the object in the top visual field of robot；
Lower sensor is used to detect the object in the lower part visual field of robot；
Display；With
Robot computer is communicated with mobile platform, upper sensor and lower sensor, and computer has processor and meter
Calculation machine readable memory, wherein robot computer is configured to make an inventory the product in commercial facility.
2. robot according to claim 1, wherein the robot computer is further configured to from remote location
It supports the operator at station to receive order, and is operated based on the received order of institute.
3. robot according to claim 2, wherein the robot computer is additionally configured to make an inventory in the following manner
Product in commercial facility:
Capture the reported visual sensation of the product in commercial facility；And
The reported visual sensation of product is handled to determine the product quantity in commercial facility；And
Determining product quantity is compared with inventory record.
4. robot according to any one of claim 1-3 further includes the calculating communicated with the robot computer
Center, wherein the calculating center has the addressable digital inventory database of the robot computer.
5. robot described in any one of -4 according to claim 1, wherein the robot computer is additionally configured to using depositing
Map and the upper sensor and the lower sensor at least one of of the storage in robot computer are in business
Navigation in facility.
6. robot according to any one of claims 1-5 further includes the product being configured in capture commercial facility
The product scanning means of reported visual sensation, wherein the product scanning means further includes at least one of the following: video camera, 2D
Video camera, 3D video camera, laser range finder, scanner, structured light scanner, ultrasound scanner, RADAR scanner, LIDAR are swept
Retouch instrument and RFID scanner.
7. a kind of for providing the robot system of intelligent inventory service in commercial facility, comprising:
Robot, upper sensor with the object in mobile platform, the top visual field for detecting robot, for examining
It the lower sensor of object surveyed in the lower part visual field of robot, display and is passed with mobile platform, upper sensor and lower part
The robot computer of sensor communication, computer have processor and computer-readable memory；With
Support station, station is supported with station display is supported and the support with supporting station display to communicate far from robot in position
It stands computer, supports station computer that there is processor and computer-readable memory；
Wherein the robot is configured to visually capture the data of item for sale in commercial facility, and will capture data extremely
Few a part communication is stood to the support.
8. system according to claim 7, wherein support station computer further includes being configured to and the processor one
The programmable code executed is acted, wherein the programmable code has digital inventory's administration interface, number inventory's management field
Face has the instrument board shown on the GUI for supporting station computer.
9. system according to claim 8, wherein the digital library deposit administration interface further include commercial facility graphically
Figure and information centre part, wherein the position of inventory variance is indicated on graphical map identification graphical map, wherein selecting position
The lteral data of the inventory variance of the position will be retrieved by setting.
10. the system according to any one of claim 8-9, wherein it further includes shelf page that the digital library, which deposits administration interface,
Face, wherein the shelf page shows the photo of the product shot in commercial facility by the robot, wherein the photo
Further include the figure of overlapping, indicates the position for lacking product in photo.
11. the system according to any one of claim 7-10, wherein support station computer further include be configured to
The programmable code that the processor executes together, wherein the programmable code has trend and data assay surface, tool
There is the instrument board shown on the GUI for supporting station computer, wherein business can be checked on the GUI for supporting station computer
The history inventory data of facility.
12. system according to any one of claims 7-11, wherein the robot further includes being configured to capture business
The product scanning means of the reported visual sensation of product in facility, wherein the product scanning means further includes at least one in following
A: video camera, 2D video camera, 3D video camera, laser range finder, scanner, structured light scanner, ultrasound scanner, RADAR are swept
Retouch instrument, LIDAR scanner and RFID scanner.
13. a kind of made an inventory the method for commercial facility with robot, which comprises
Robot is provided in commercial facility, wherein the robot has mobile platform, for detecting the robot
The upper sensor of object in the portion visual field, the lower sensor of object in the lower part visual field for detecting robot, display
Device, be configured to capture commercial facility in product reported visual sensation product scanning means and with mobile platform, upper sensor,
The robot computer of lower sensor and the communication of product scanning means；
Use the reported visual sensation of the product in product scanning means capture commercial facility；And
The reported visual sensation of product is handled to determine the inventory data of product in commercial facility.
14. further including according to the method for claim 13, at least one of the following:
By the way that the product quantity determined to be compared with the digital inventory database in the calculating center far from robot, handle
The reported visual sensation of product, to determine the inventory data of product in commercial facility；And
By using video camera, 2D video camera, 3D video camera, laser range finder, scanner, structured light scanner, ultrasonic scanning
At least one of instrument, RADAR scanner, LIDAR scanner and RFID scanner scan the product in commercial facility, utilize production
Product scanning means captures the reported visual sensation of the product in commercial facility, and wherein robot moves in commercial facility.
15. according to the method for claim 14, wherein corresponding in robot records commercial facility captures each product
Reported visual sensation position position.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US15/369,812 | 2016-12-05 | ||
| US15/369,812 US10311400B2 (en) | 2014-10-24 | 2016-12-05 | Intelligent service robot and related systems and methods |
| PCT/US2017/064753 WO2018106719A1 (en) | 2016-12-05 | 2017-12-05 | Intelligent service robot and related systems and methods |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110352116A true CN110352116A (en) | 2019-10-18 |
Family
ID=62492141
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201780075073.9A Pending CN110352116A (en) | 2016-12-05 | 2017-12-05 | Intellect service robot and related systems and methods |
Country Status (4)
| Country | Link |
|---|---|
| EP (1) | EP3549045A4 (en) |
| JP (1) | JP2020502649A (en) |
| CN (1) | CN110352116A (en) |
| WO (1) | WO2018106719A1 (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114115218A (en) * | 2020-08-26 | 2022-03-01 | 丰田自动车株式会社 | Autonomous mobile robot control system, method, storage medium, and control device |
| CN114227683A (en) * | 2021-12-23 | 2022-03-25 | 江苏木盟智能科技有限公司 | Robot control method, system, terminal device and storage medium |
| CN114949876A (en) * | 2022-05-16 | 2022-08-30 | 浙江师范大学 | Expansion and contraction intermittent conversion type child cognitive training toy robot |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP7379814B2 (en) | 2018-11-21 | 2023-11-15 | 富士フイルムビジネスイノベーション株式会社 | Autonomous mobile devices and programs |
| US10913605B2 (en) * | 2019-01-22 | 2021-02-09 | Everseen Limited | Warehouse management system |
| US11293152B2 (en) | 2019-04-09 | 2022-04-05 | Abb Schweiz Ag | Robotic restocking and safety systems for automated retail store environments |
| CN109910032B (en) * | 2019-04-30 | 2024-06-11 | 四川智慧式停车服务有限公司 | Robot, management system and business model |
| RU195456U1 (en) * | 2019-09-09 | 2020-01-29 | Общество с ограниченной ответственностью "Цифровые технологии ритейла" | Mobile robotic device |
| GB201914058D0 (en) * | 2019-09-30 | 2019-11-13 | Botsandus Ltd | A robot fleet and control system therefor |
| CN114012746B (en) * | 2021-10-28 | 2023-07-14 | 深圳市普渡科技有限公司 | Robot, information playing method, control device and medium |
| JP7297224B1 (en) | 2022-10-14 | 2023-06-26 | トリノ・ガーデン株式会社 | KPI measurement system, KPI measurement method |
| WO2024106033A1 (en) * | 2022-11-15 | 2024-05-23 | パナソニックＩｐマネジメント株式会社 | Control method, robot, and program |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7693757B2 (en) * | 2006-09-21 | 2010-04-06 | International Business Machines Corporation | System and method for performing inventory using a mobile inventory robot |
| WO2013138193A2 (en) * | 2012-03-12 | 2013-09-19 | Bar Code Specialties, Inc. (Dba Bcs Solutions) | Rail-mounted robotic inventory system |
| US20130343640A1 (en) * | 2012-06-21 | 2013-12-26 | Rethink Robotics, Inc. | Vision-guided robots and methods of training them |
| US9205886B1 (en) * | 2011-05-06 | 2015-12-08 | Google Inc. | Systems and methods for inventorying objects |
| US20160114488A1 (en) * | 2014-10-24 | 2016-04-28 | Fellow Robots, Inc. | Customer service robot and related systems and methods |
| CN106002916A (en) * | 2016-06-08 | 2016-10-12 | 国网天津市电力公司 | Warehousing robot capable of automatically making tour for inspecting, cleaning and checking inventories |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9015072B2 (en) * | 2013-08-19 | 2015-04-21 | Xerox Corporation | Method and apparatus for automated inventory management using depth sensing |
| US9821464B2 (en) * | 2015-05-04 | 2017-11-21 | Bby Solutions, Inc. | Random-access robotic inventory dispensary: operation prioritization |
-
2017
- 2017-12-05 EP EP17878094.6A patent/EP3549045A4/en not_active Withdrawn
- 2017-12-05 CN CN201780075073.9A patent/CN110352116A/en active Pending
- 2017-12-05 WO PCT/US2017/064753 patent/WO2018106719A1/en unknown
- 2017-12-05 JP JP2019530044A patent/JP2020502649A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7693757B2 (en) * | 2006-09-21 | 2010-04-06 | International Business Machines Corporation | System and method for performing inventory using a mobile inventory robot |
| US9205886B1 (en) * | 2011-05-06 | 2015-12-08 | Google Inc. | Systems and methods for inventorying objects |
| WO2013138193A2 (en) * | 2012-03-12 | 2013-09-19 | Bar Code Specialties, Inc. (Dba Bcs Solutions) | Rail-mounted robotic inventory system |
| US20130343640A1 (en) * | 2012-06-21 | 2013-12-26 | Rethink Robotics, Inc. | Vision-guided robots and methods of training them |
| US20160114488A1 (en) * | 2014-10-24 | 2016-04-28 | Fellow Robots, Inc. | Customer service robot and related systems and methods |
| CN106002916A (en) * | 2016-06-08 | 2016-10-12 | 国网天津市电力公司 | Warehousing robot capable of automatically making tour for inspecting, cleaning and checking inventories |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114115218A (en) * | 2020-08-26 | 2022-03-01 | 丰田自动车株式会社 | Autonomous mobile robot control system, method, storage medium, and control device |
| CN114115218B (en) * | 2020-08-26 | 2024-03-26 | 丰田自动车株式会社 | Autonomous mobile robot control system, autonomous mobile robot control method, storage medium and autonomous mobile robot control device |
| CN114227683A (en) * | 2021-12-23 | 2022-03-25 | 江苏木盟智能科技有限公司 | Robot control method, system, terminal device and storage medium |
| CN114227683B (en) * | 2021-12-23 | 2024-02-09 | 江苏木盟智能科技有限公司 | Robot control method, system, terminal device and storage medium |
| CN114949876A (en) * | 2022-05-16 | 2022-08-30 | 浙江师范大学 | Expansion and contraction intermittent conversion type child cognitive training toy robot |
Also Published As
| Publication number | Publication date |
|---|---|
| EP3549045A1 (en) | 2019-10-09 |
| JP2020502649A (en) | 2020-01-23 |
| EP3549045A4 (en) | 2020-04-22 |
| WO2018106719A1 (en) | 2018-06-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110352116A (en) | Intellect service robot and related systems and methods | |
| US10311400B2 (en) | Intelligent service robot and related systems and methods | |
| JP6502491B2 (en) | Customer service robot and related system and method | |
| US6584375B2 (en) | System for a retail environment | |
| US7206753B2 (en) | Methods for facilitating a retail environment | |
| US11687865B2 (en) | Detecting changes of items hanging on peg-hooks | |
| US11914785B1 (en) | Contactless user interface | |
| US11341355B2 (en) | Robot and method of controlling the same | |
| US10867280B1 (en) | Interaction system using a wearable device | |
| CN109998429A (en) | Mobile clean robot artificial intelligence for context aware | |
| CN107331051A (en) | Unattended portable store system and retail method | |
| US20090006286A1 (en) | Method and apparatus for implementing digital video modeling to identify unexpected behavior | |
| CN107864679A (en) | System and method for commercialization electronical display | |
| US11448508B2 (en) | Systems and methods for autonomous generation of maps | |
| CN113126629B (en) | Method for robot to actively search target and intelligent robot | |
| JP2003099629A (en) | Remote selling system and method for merchandise | |
| US20200250736A1 (en) | Systems, method and apparatus for frictionless shopping | |
| WO2016051183A1 (en) | System and method for monitoring display unit compliance | |
| CN114897455A (en) | Unmanned warehouse management system and method based on display enhancement technology | |
| JP6809267B2 (en) | Information processing equipment, information processing systems and programs | |
| JP7467400B2 (en) | MOBILE STORE MANAGEMENT SYSTEM, MOBILE STORE MANAGEMENT DEVICE, MOBILE STORE MANAGEMENT METHOD, AND COMPUTER PROGRAM | |
| US20240144354A1 (en) | Dynamic store feedback systems for directing users | |
| JP2006344226A (en) | Information presentation system of visual field agreement type, and portable information terminal for use in it |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20191018 |
|
| WD01 | Invention patent application deemed withdrawn after publication |