CN106292657A - Mobile robot and patrol path setting method thereof - Google Patents

Mobile robot and patrol path setting method thereof Download PDF

Info

Publication number
CN106292657A
CN106292657A CN201610596000.1A CN201610596000A CN106292657A CN 106292657 A CN106292657 A CN 106292657A CN 201610596000 A CN201610596000 A CN 201610596000A CN 106292657 A CN106292657 A CN 106292657A
Authority
CN
China
Prior art keywords
user
patrol
order
mobile robot
patrol path
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201610596000.1A
Other languages
Chinese (zh)
Other versions
CN106292657B (en
Inventor
陈本东
张芊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Horizon Robotics Technology Research and Development Co Ltd
Original Assignee
Beijing Horizon Robotics Technology Research and Development Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Horizon Robotics Technology Research and Development Co Ltd filed Critical Beijing Horizon Robotics Technology Research and Development Co Ltd
Publication of CN106292657A publication Critical patent/CN106292657A/en
Application granted granted Critical
Publication of CN106292657B publication Critical patent/CN106292657B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles

Landscapes

  • Engineering & Computer Science (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Manipulator (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Abstract

The present invention relates to mobile robot and patrol path setting method thereof.A kind of patrol path setting method for mobile robot comprises the steps that the first order in response to user, enters patrol path and arranges pattern;Lock described user, and follow the tracks of the movement of described user;While following the tracks of the movement of described user, determine mobile route;And mobile route determined by inciting somebody to action is stored as going on patrol path.The method of the present invention is simple to operation, and need not extras, and the scope of application is more extensive.

Description

Mobile robot and patrol path setting method thereof
Technical field
Present invention relates in general to human-computer interaction technique field, more specifically it relates to a kind of mobile robot and setting The method putting the patrol path of this mobile robot.
Background technology
Intelligent robot relates to the crossing domain of subjects, such as, include artificial intelligence, radio communication, quasiconductor, meter Calculation machine, mechanical engineering, electrical engineering etc..Along with the sustainable development of Science and Technology, develop and be manufactured that many kinds The intelligent robot of class, serves the every field of human being's production and life, such as, has sweeping robot, nurse robot, removes Fortune robot, security patrol robot etc..These robots can in working region independently or according to set path Mobile, to realize its designed function.
In the prior art, typically app program is run for moving by portable equipment such as mobile phone, tablet PC etc. Mobile robot sets mobile route, also referred to as goes on patrol path.As a example by mobile phone, its can pass through such as bluetooth, WLAN, The Internet or even communication network set up communication connection with mobile robot.By running app program, mobile phone can show The working environment map of storage on mobile robot, and the patrol road of mobile robot can be edited on this map Footpath.After the editor completing patrol path, this patrol path can be saved in depositing included by mobile robot by app program In reservoir, perform for mobile robot.
Summary of the invention
But, there is many defects in above-mentioned traditional patrol path setting method.First, be appropriate only for can for this traditional method Mobile robot has been provided with the situation of the map of its working environment, because mobile phone app application program needs at this map On the basis of edit the patrol path of mobile robot.Secondly, this traditional method needs to use mobile phone or tablet PC so Extras, and require that such as mobile phone has bigger display screen, add system cost.Furthermore, patrol for setting Patrolling the app application program in path and relate generally to the operation of complexity, it is for being unfamiliar with the people of computer operation, such as older Speech, is difficult.
In order at least solve one of the problems referred to above, present applicant proposes the patrol for mobile robot of a kind of novelty The path plan of establishment, its movement following the tracks of human user by making mobile robot comes conveniently and efficiently for mobile robot Patrol path is set.
An aspect according to the application, it is provided that a kind of patrol path setting method for mobile robot, it can Including: in response to first order of user, enter patrol path and pattern is set;Lock described user, and follow the tracks of described user Movement;While following the tracks of the movement of described user, determine mobile route;And mobile route determined by inciting somebody to action is stored as patrolling Patrol path.
In certain embodiments, described patrol path setting method may also include that the second order in response to described user, Or it is back to initial point in response to following the tracks of described user, completes described patrol path.
In certain embodiments, when described first order is local trigger command or telecommunication order, locking is described User can include locking described user based on image recognition.In certain embodiments, described use is locked based on image recognition Family comprises the steps that and utilizes imageing sensor capturing ambient image;Identify the one or more potential use in the ambient image captured Family;When identifying a potential user, lock one potential user;And when identifying multiple potential user, then Lock that potential user immediate.In certain embodiments, when identifying multiple potential user, then determine whether institute Show the type of the first order, if described first order is local trigger command, then lock in the plurality of potential user A close user, if described first order is telecommunication order, then locks in the plurality of potential user at first That user of movement, if there being multiple user moving, then locks that user moved immediate.
In certain embodiments, when described first order is voice command, locks described user and can include based on image Identify and the combination of sound localization locks described user.In certain embodiments, based on image recognition and the group of sound localization The described user of incompatible locking comprises the steps that and determines Sounnd source direction based on described first order as voice command;By image sensing Device rotates to described Sounnd source direction;Utilize described imageing sensor capturing ambient image on described Sounnd source direction;Identify also And lock the user on described Sounnd source direction in the ambient image captured.In certain embodiments, based on image recognition Combination with sound localization locks described user and comprises the steps that step a, determines based on described first order as voice command Sounnd source direction;Step b, rotates imageing sensor to described Sounnd source direction;Step c, utilizes described imageing sensor in institute State capturing ambient image on Sounnd source direction;Step d, based on user described in the ambient image identification captured;Step e, if base Successfully identify described user in the ambient image captured, then lock described user;Step f, if based on the environment captured Image fails and identifies described user, then determine whether to advance along described Sounnd source direction according to preset strategy;Step g, if Determine according to preset strategy and advance along described Sounnd source direction, then drive described mobile robot to advance along described Sounnd source direction, It is then return to step c;And step h, if determining according to preset strategy and not advancing along described Sounnd source direction, the most do not drive institute State mobile robot to advance along described Sounnd source direction, and also return to step c.
In certain embodiments, the movement following the tracks of described user comprises the steps that the profile information obtaining described user;And with The profile of user described in track moves.In certain embodiments, described profile information can include whole human body contour outline, head One or more in other local configurations of profile or human body.
In certain embodiments, the movement following the tracks of described user may also include that in response to the 3rd life from described user Order, performs corresponding mobile.Described 3rd order can be voice command or telecommunication order.
In certain embodiments, determine that mobile route comprises the steps that each path determining described mobile robot process Point is relative to the relative coordinate of previous path point.
In certain embodiments, will determined by mobile route be stored as go on patrol path comprise the steps that described removable machine All path point of device people's process are stored as going on patrol path;Or by all path point of described mobile robot process , by described user the 4th order indicated by one or more path point be stored as go on patrol path.Described 4th order can To be voice command, local one of trigger command and telecommunication order.
In certain embodiments, described mobile robot has had the map of working environment.The patrol road stored Footpath is in the path in the map of described working environment.
In certain embodiments, described mobile robot not yet has the map of working environment.Now, described patrol road Footpath method to set up may also include that while following the tracks of the movement of described user, runs the SLAM algorithm ground with construction work environment Figure.The patrol path stored is in the path in the map of constructed working environment.
In certain embodiments, described patrol path setting method may also include that the 5th order in response to described user, The patrol time in described patrol path and one or more in patrol frequency are set.Described 5th order can be voice life Make, local one of trigger command and telecommunication order.In certain embodiments, arrange described patrol path the patrol time and Patrol frequency in one or more can described mobile robot follow the tracks of described user complete described patrol path time Carry out.After completing the setting of patrol time and patrol frequency, described mobile robot can exit described patrol road Line arranges pattern.
According to further aspect of the application, it is provided that a kind of mobile robot, it comprises the steps that for making described removable The driver part that robot moves;For capturing at least one imageing sensor of image;For capturing at least one of sound Microphone;And at least one processor communicated with described imageing sensor and described microphone, described processor is configured to Run the computer instruction being stored in computer-readable medium to perform method discussed above.
In certain embodiments, at least one microphone described can include microphone array.
Another aspect according to the application, it is provided that a kind of computer program, it can include computer program instructions, Described computer program instructions makes described processor perform side discussed above when being run by one or more processors Method.
Another aspect according to the application, it is provided that a kind of computer-readable medium, it can store computer program and refer to Order, described computer program instructions can make the execution of described processor be discussed above when being run by one or more processors Method.
Compared with prior art, the scheme that the application proposes need not extra equipment, can arrange mobile robot Patrol path.And, the scheme of the application has evaded the app application program operation of complexity, simple and convenient, even if thus The people being unfamiliar with computer operation also is able to the scheme easily with the application to arrange the patrol path of mobile robot. Additionally, the scheme of the application can be applicable to the situation that mobile robot not yet has the map of working environment, therefore should Broader by scope and scene.
Accompanying drawing explanation
By combining accompanying drawing, the exemplary embodiment of the application is described in more detail, the application above-mentioned and its His purpose, feature and advantage will be apparent from.Accompanying drawing is used for providing the further reason of the exemplary embodiment to the application Solve, and constitute the part of description, be used for explaining the application together with the exemplary embodiment of the application, and do not constitute right The restriction of the application.In the accompanying drawings, same or analogous reference typically represents same or analogous parts or step.
Fig. 1 is to illustrate the patrol path setting method for mobile robot according to the present invention one exemplary embodiment Flow chart.
Fig. 2 is the flow process illustrating the method locking user based on image recognition according to the present invention one exemplary embodiment Figure.
Fig. 3 A is to illustrate that the combination based on image recognition and sound localization according to the present invention one exemplary embodiment is locked Determine the flow chart of the method for user.
Fig. 3 B is to illustrate that the combination based on image recognition and sound localization according to another exemplary embodiment of the present invention comes The flow chart of the method for locking user.
Fig. 4 A is to illustrate the example that the mobile robot according to the present invention one exemplary embodiment is followed the tracks of when user moves The schematic diagram of scene.
Fig. 4 B is to illustrate that the mobile robot according to another exemplary embodiment of the present invention follows the tracks of showing when user moves The schematic diagram of example scene.
Fig. 5 is the block diagram of the structure illustrating the mobile robot according to another exemplary embodiment of the present invention.
Detailed description of the invention
Below, example embodiment according to the application will be described in detail by referring to the drawings.Obviously, described embodiment is only It is only a part of embodiment rather than whole embodiments of the application of the application, it should be appreciated that the application is not by described herein The restriction of example embodiment.
Summary of the invention
During as it was previously stated, prior art arranges patrol path for mobile robot, need to use such as mobile phone, flat board The portable mobile equipment of computer etc runs app application program to be carried out, and app application program needs to read removable machine The environmental map of the upper storage of device people, draws or edits the patrol path for mobile robot based on this map.This biography There is many defects in the patrol path setting method of system, such as, needs extra equipment, and operation is complicated, and it also requires pre-prophet Know environmental map.In the case of such as dynamic environment, on mobile robot storage environmental map may with reality Environment is not consistent, causes the patrol path arranged in app application program can not go on patrol.
The present invention solves the problems referred to above by simple technical scheme.One basic conception of the present invention is, Yong Huke To lead mobile robot to go on patrol once along patrol path, mobile robot can be remembered to go on patrol path, and may be used later To perform patrol by this path.This scheme is referred to as " teaching-reappear (teach-playback) " pattern.So, originally Invention avoids mobile phone and the needs of app application program, it is not necessary to complicated interface and function to app application program are grasped Make, and have only to lead mobile robot to walk once along patrol path, simple and quick.The solution of the present invention is not only suitable for In mobile robot it is known that the situation of environmental map, and apply also for mobile robot and be still unaware of environment ground The situation of figure.In the case of the latter, such as, mobile robot can patrol to arrange along the traveling of patrol path following user While patrolling path, set up environmental map by instant location and map structuring (SLAM) algorithm.Patrol due to the present invention Path setting method is to complete the setting to patrol path along Actual path while going on patrol, and which ensure that set patrol Concordance between path and up-to-date true environment, thus avoid the patrol route because such as mobile phone app application program is arranged And contradiction between true dynamic environment and cause going on patrol the disabled problem in path.
After the ultimate principle describing the application, carry out specifically to introduce the various non-limit of the application below with reference to the accompanying drawings Property embodiment processed.
Exemplary embodiment
Fig. 1 is to illustrate the patrol path setting method for mobile robot according to the present invention one exemplary embodiment Flow chart.As it is shown in figure 1, the patrol path setting method 100 for mobile robot can start from step 110, can move Mobile robot receives and comes from first order of user and enter patrol path and arrange pattern.First order can be any shape The order of formula.Such as, in certain embodiments, the first order can be local trigger command, and such as user is by by removable The order that robot triggers from physical button with it or touch screen;In certain embodiments, the first order can also be Telecommunication order, such as user use and are exclusively used in the infrared signal order that the remote controller of mobile robot sends, or use In the order using the mobile phone of oneself to send, mobile phone can pass through WLAN, the Internet or communication network etc. and removable machine Device people sets up communication connection, and runs specific app application program to control mobile robot.It should be understood that here When with mobile phone app application program controlling mobile robot to arrange patrol path, can not require that app application program reads The environmental map of storage in mobile robot.Preferably, in certain embodiments, the first order can be from the language of user Sound order.Such as, user can send voice command to mobile robot and " arrange patrol path ", on mobile robot Microphone unit such as microphone array pickup user speech and extract key word by speech recognition algorithm " patrol is set Path ", thus enter patrol path in response to this order and pattern is set.Enter patrol path set in correct response the first order After putting pattern, mobile robot can also be provided out certain output and successfully enter this pattern to represent, such as may be used To utilize it to have the luminous component light to emission particular color or passage of scintillation light to have indicated entry into this pattern, or can be in order to Send the sound of " come into patrol path arrange pattern " with speaker to represent and be successfully entered this pattern.
After entering patrol path arrange pattern, mobile robot can step 120 judge its received the The form of one order, it is voice command, or non-voice order, the most aforementioned local trigger command or telecommunication order. Form based on the first order, mobile robot can perform different processing modes.Such as, when first order be voice life When making, mobile robot then combination based on image recognition and sound localization can identify and lock user, such as step Shown in 140;When the first order is not voice command, mobile robot then can be based only upon image recognition and identifies and lock Determine user, as illustrated in step 130.Identify and lock the step of user, such as step 130 and step 140, will enter again below Row detailed description.
After step 130 or step 140 successfully lock user, mobile robot then can follow the tracks of user and with Family and move, as shown in step 150.User can be with mobile robot in the work environment along the patrol road wanted to set up Advance in footpath.Mobile robot is except utilizing track algorithm to follow in addition to user moves, it is also possible to receive from the second of user Order moves.Such as, in some application-specific scene, user may desire to mobile robot patrol human user not Can arrive or be not suitable for the region arrived, now user can move to this region with the second order Lai Shi robot, and from Oneself need not move to this region.It can therefore be understood that, the second order preferably voice command or telecommunication order.
While moving with user, mobile robot can perform location algorithm to determine its mobile route.Can move Mobile robot can use existing multiple localization method, such as Relative localization method, such as dead reckoning (Dead Reckoning) etc., absolute fix method, such as GPS location, beacon location, map match location and probabilistic localization etc., and bag Include relative localization and absolute fix combination positioning mode.Mobile robot can select according to different application scenarios Different localization methods, such as Indoor Robot can use Relative localization method, and outdoor robot can use absolute fix Method or integrated positioning etc..
As a example by the dead reckoning being suitable in the Relative localization method of Indoor Robot, mobile robot is known, and it is initial Position, is set to initial point.Robot follow user move time, it utilizes such as displacement measurement parts, such as speedometer And accelerometer, course transmitter, such as angular rate gyroscope, magnetic compass and differential speedometer etc., determine each of which path point Displacement vector relative to a upper path point.This it is also referred to as relative coordinate relative to the displacement vector of a upper path point, And the relative coordinate of all path point is added up, i.e. can get the position relative to initial point, the current location of mobile robot Moving vector, it is also referred to as absolute coordinate.In certain embodiments, mobile robot can use dead reckoning to determine it The relative coordinate of each path point on mobile route.
It is understood that dead reckoning is a cumulative process, measurement error and calculate error can over time and by Gradually accumulate, cause positioning precision the most constantly to decline.Therefore, in certain embodiments, it is also possible to by dead reckoning and other Sensor information combines and positions.Such as, in certain embodiments, have currently when the most stored in mobile robot During the map of working environment, mobile robot can also utilize such as laser sensor, sonar sensor etc. to determine and can move Mobile robot is relative to the distance of other map feature, for correcting the coordinate of its current location.
It will of course be appreciated that the present invention also may be used on the feelings that mobile robot not yet has the map of working environment Condition.In this case, mobile robot can followed while movement, utilizes such as instant location and map structuring (SLAM) algorithm carrys out the map of construction work environment, determines its mobile route in this map simultaneously.Now, removable machine People can use many algorithms to improve positioning precision, such as, correct the physical arrangement of robot with UMBmark and be used to The systematic error that property sensor accuracy factor causes, with Gyrodometry algorithm correct deflection error that ground injustice causes, Propose sensor fuse information with Scheding et al. and correct wheel slip impact on positioning etc..These algorithms are all It known in the art, and be the most no longer described in greater detail.
Although being presented above more relative localization algorithm embodiment, however, it is understood that when mobile robot is in During suitable application scenarios, also can use absolute fix or integrated positioning.Equally, these location algorithms are also known in the art , the most no longer it is described in greater detail.
As it has been described above, mobile robot determines its mobile route (in other words, each on mobile route in step 160 The coordinate of individual path point).When mobile robot followed by user return to initial point time, mobile robot may determine that Complete whole mobile route, thus this mobile route is stored as going on patrol path, as indicated at step 170.Implement at other In example, it is possible to so that mobile robot is when the 3rd order receiving user, just determines and have been completed whole mobile road Footpath, and perform to go on patrol path storing step 170.3rd order can be any type of order, such as aforementioned voice order, Local trigger command or telecommunication order etc..Such as, mobile robot is receiving the voice command of user " patrol knot Bundle " time, just determine and complete whole patrol path.This is conducive to such as patrol path not to be closed loop, or through initial point shape Also to continue to go on patrol the situation in other paths after becoming closed loop.
Although not shown in the flow chart of Fig. 1, but in certain embodiments, determine complete whole patrol path it After, and before storage patrol path (step 170), it is also possible to the patrol parameter in this patrol path is set, such as go on patrol the time, Patrol frequency etc..Such as, mobile robot " can please arrange the patrol time " etc. with play cuing voice and point out user setup The patrol time, user then sends voice command such as " 13: 30 ", " to some next day 7 " etc. at 22 and arranges the patrol time.Can Mobile robot such as " can please arrange patrol frequency " with play cuing voice etc. and point out user setup to go on patrol frequency, can move Mobile robot can also be play some options such as " option A, every day " further and supply user to select.User can set with voice Put patrol frequency, or can select from the option provided.Certainly, user can also be in its other party of other times Formula arranges patrol parameter.For example, it is possible to the most just arrange patrol parameter when entering patrol path and arranging pattern, it is possible to use Touch display screen etc. on mobile robot arranges patrol parameter.Patrol with other in the set patrol parameter such as time Patrol path inconsistent time, it is also possible to prompting user, allow user select preferentially to perform which patrol, or abandon which patrol.? After completing the setting of various patrol parameter, the mobile route with these parameters can be stored as patrolling by mobile robot Patrol path.
Finally, after completing patrol the arranging of path, mobile robot can exit patrol path in step 180 and set Put pattern, and can inform that user exits patrol path and arranges pattern by such as prompt tone or light signal.
Above with reference to as described in step 130 and 140, mobile robot can utilize image based on the type of the first order Identify or the combination of image recognition and sound localization locks user.Fig. 2 shows according to the present invention one exemplary embodiment The flow chart of the method locking user based on image recognition, it can be used for the step 130 of such as Fig. 1.As in figure 2 it is shown, base Lock the method 200 of user in image recognition and may begin at the step 210 of first capturing ambient image.Should be understood that in the present invention In mention capture image time, not only contain capture single-frame images situation, but also contain capture image sequence situation, i.e. connect Capture multiple image continuously or with predetermined time interval, and utilize the multiple image with predetermined time interval pre-to realize Phase purpose.This understanding is also applied for the situation of such as picking up surrounding sounds.In step 210, mobile robot can enter Row rotates to capture the image of 360 degree of surroundings.
It follows that in step 220, mobile robot can utilize image recognition to calculate based on the ambient image captured Method identifies potential user.Such as, mobile robot can utilize face recognition algorithms to identify potential user, can also profit Potential user is identified, such as human body contour outline recognizer, characteristics of human body position outline identification algorithm such as head with other algorithms Contouring recognizer, trunk outline identification algorithm etc. identify potential user.
Then, recognition result can be checked in step 230, identify a potential user or multiple potential to determine User.In ideal conditions, when only identifying a potential user, it can be determined that this potential user seeks to lead and can move Mobile robot carries out going on patrol the user that path is arranged, and the most directly locks this user, as illustrated in step 240.On the other hand, when When step 220 identifies two or more potential user, then also need to identify from these multiple potential users further Go out that user that have issued the first order to mobile robot.Although the most not shown, but at some In embodiment, can directly lock that user immediate in these multiple potential users.In further embodiments, it is possible to To proceed to step 250 to determine whether that the type of the first order is local command or remote command.When determining the first life Order is local trigger command, such as during the order triggered by mobile robot physical button itself or touch screen, and can To judge to send that user of the first order just on mobile robot side, therefore mobile robot locks immediate That user, as shown in step 260.When determining that the first order is telecommunication order, such as, sent by mobile phone or remote controller Order time, then can lock that potential user moved at first in step 270, as it can be assumed that this user touch Send out after the patrol path of mobile robot arranges pattern, mobile robot can be led at once to go to go on patrol.One The most in particular cases, multiple potential user may be had simultaneously all to move, as default policy, now can lock immediate That user moved.By these steps, complete the process utilizing image recognition to lock user.
Fig. 3 A shows that the combination based on image recognition and sound localization according to the present invention one exemplary embodiment is locked Determine the flow chart of the method for user, Fig. 3 B show according to another exemplary embodiment of the present invention based on image recognition and sound source The combination of location locks the flow chart of the method for user, and the method shown in Fig. 3 A and Fig. 3 B can be used for the step of such as Fig. 1 140。
With reference first to Fig. 3 A, combination based on image recognition and sound localization locks the method 300 of user and can start from Step 310, determines Sounnd source direction based on the first order as voice command.Mobile robot can utilize such as Mike Wind array carries out sound localization step 310.In short, due to each mike in microphone array arrangement position not With, the first voice command that user sends, i.e. sound wave, the phase place arriving each mike in atmosphere after transmission is the most different.Root According to each mike to phase differential during same sound collection, same sound can be calculated and arrive the time of every pair of mike Difference, then this sound source be at this to mike present position as focus, the time of advent difference corresponding to transfer voice distance For on the hyperboloid of parameter.Use multipair mike to obtain multiple time difference, the most just obtain multiple hyperboloid, then sound source position Put and be in these bi-curved joinings, here it is the ultimate principle of sound localization.In the step 310, it may not be necessary to Determine the accurate location (that is, direction+distance) of sound source, and have only to determine Sounnd source direction.
Then in step 320, the imageing sensor on mobile robot can be rotated to sound determined by step 310 On direction, source, and capture the image on Sounnd source direction in a step 330.In step 340, utilize on the Sounnd source direction captured Image, identify and lock user.The step of capture image and identification and locking user can be generally such as above reference Described in Fig. 2, repeat no more here.
In the method shown in Fig. 3 A, owing to have employed sound localization to determine the direction of user, therefore, it is possible to significantly carry The accuracy of high user locking, and save locking time.For example, it is possible to Direct Recognition and the use that is locked on Sounnd source direction Family, without the user being concerned about in the other direction.Should be understood that in the method shown in Fig. 3 A, it is also possible to utilize aforementioned first Subsequent voice order after order constantly corrects Sounnd source direction, to improve the accuracy of user's locking further.
In some cases, even if determining correct audio direction, it is also possible to can not identify and lock user.Fig. 3 B Shown method 300' to some extent solves this problem.Step 310,320 and 330 in method 300' of Fig. 3 B are permissible Identical with the method 300 of Fig. 3 A, omit the repeated description to it the most here.
With reference to Fig. 3 B, after step 330 captures image, proceed to step 350, it is determined whether successfully identify user. If successfully identifying user, then proceeding to step 360, locking this user;Otherwise proceed to step 370, based on predetermined policy Judge whether to move to Sounnd source direction.Described predetermined policy such as includes whether cannot continue already at ad-hoc location Advance (the most having arrived wall limit etc.), or interrupt because receiving new voice command currently processed etc..If it is true in step 370 Surely can continue to move to Sounnd source direction, then move a distance, then repeated execution of steps 330 in step 380 to Sounnd source direction Image capture operation.Due to from sound source position closer to, so capture image after successfully identify user probability increase.The opposing party Face, if determining in step 370 and can not continuing to move to Sounnd source direction, then this process also returns to step 330, multiimage Capture and the process identified, to attempt locking user.
Although described above is combination based on image recognition and sound localization to identify and lock the embodiment of user, but To the invention is not restricted to these specific embodiments, but can also use other embodiments, such as the applicant formerly in Those disclosed in state application for a patent for invention 201610341566.X, the full content of the document is incorporated in by quoting entirety This.
In embodiment described above, image recognition and speech recognition technology are used.Should be understood that the present invention not only may be used To utilize existing image and speech recognition technology, but also the Classical correlation technology that can utilize or develop future.Also Should be understood that image recognition described here not only includes the identification of visual image, but also such as infrared image can be included, swash The identification technology of light image, ultrasonography etc..
Fig. 4 A shows that the mobile robot according to the present invention one exemplary embodiment follows the tracks of example when user moves Scene.Such as, mobile robot 10 can be arranged in working environment 400, can have and some can not arrive in working environment 400 The position reached, such as 402,404 etc..Docking station 410 for mobile robot 10 is arranged on certain position in working environment 400 Place.When not working, mobile robot 10 can rest in and be charged at docking station 410.Mobile robot 10 has had The map of whole working environment 400, and the position of docking station 410 can be set to initial point.
By performing previously described patrol path setting method, mobile robot 10 can be followed user 20 and be moved, Thus generate mobile route 420.Although in more previously described embodiments, whole path 420 can be saved as patrol Path, although in some embodiments it may also by some path point on path 420, such as path point 422,424,426 and 428 save as patrol path.Such as, when mobile robot 10 arrives a certain path point such as path point 422, user 20 can With send order " arrange patrol point " using by this path point as the position that must go on patrol.In certain embodiments, use Family 20 even can also arrange the time that should go on patrol this position.Similarly, user 20 can set when leading robot 10 to move Put multiple such patrol point, such as, put 424,426 and 428 etc..When completing whole path, mobile robot 10 is permissible These sequences put are saved as patrol path.When performing patrol afterwards, mobile robot 10 is according to this path, predetermined Time arrive predetermined patrol point, and the path between going on patrol a little independently can be determined by mobile robot 10.
The exemplary scene of Fig. 4 B is similar with Fig. 4 A, except mobile robot 10 does not has the map of working environment 400' in advance Outside.Therefore, in the case of Fig. 4 B, while mobile robot 10 can move following user 20, it is instant fixed to perform Position and map structuring (SLAM) generate the map of working environment.As shown in Figure 4 B, mobile robot 10 has generated edge A part of map in path 420.When completing to go on patrol path, mobile robot 10 can first preserve this schemes partially, The map of whole working environment is suitably completed opportunity again after Deng.
Described above is the embodiment of the certain operations relevant to the patrol path setting method for mobile robot, The example of mobile robot is described referring to Fig. 5.Fig. 5 is to illustrate moving according to another exemplary embodiment of the present invention The block diagram of the structure of mobile robot.As it is shown in figure 5, mobile robot 10 can include imageing sensor 11, microphone 12, storage Device 13, output block 14, driver part 15 and processor 17, these parts are connected to each other by bus system 16.
Imageing sensor 11 can be the most one or more photographic head, and it can be monocular cam, binocular camera Or more purpose photographic head.However, it is understood that the invention is not restricted to visual light imaging, but, imageing sensor 11 can also Being such as infrared imaging device, ultrasonic imaging device, laser imaging device etc., so these devices can capture and can be used for The image that family identifies.Additionally, imageing sensor 11 can be also used for monitoring working environment during patrol.
Microphone 12 can be such as mike, preferably microphone array, and it is used for receiving voice command, carries out sound source Location etc..
Memorizer 13 can store the data for processor 17 or programmed instruction.Such as, processor 17 can be in order to By the data gathered from imageing sensor 11 and microphone 12, and utilize the data of storage in memorizer 13, perform memorizer In 13, the programmed instruction of storage is to implement those described above method and steps.The result that processor 17 performs can store In memorizer 13, or exported by output block 14.Output block 14 can be various types of output block, such as, raise Sound device, printer, display, bulb etc..Bus system 16 ensure that data can be at these including driver part 15 Transmit between parts.
Driver part 15 can be used for driving mobile robot 10 to move.Such as, driver part 15 can include motor and By motor-driven wheel or crawler belt etc..The operation of motor can be controlled by processor 17.
Processor 17 can include one or more processor, or can include the multiple process encapsulated in a processor Device core.In addition to conventional CPU, processor 17 can also be so-called coprocessor, such as, be used for processing figure Picture or the graphics coprocessor of video signal, for processing the audio frequency coprocessor etc. of audio signal.Processor 17 may also comprise It is exclusively used in certain parts of process or the so-called controller etc. of certain signal.
Although it is not shown, mobile robot 10 also can include many miscellaneous parts, the most such as gyroscope, laser The sensor of device, sonar etc, perform specific function such as clean, carry, the executor of the function such as detection, and for outward Boundary sets up the communication component etc. that radio communication connects.Depend on that mobile robot 10 is designed to the field of application, removable Robot 10 can have various different parts.
Application Scenarios-Example
The mobile robot 10 of the present invention can be applicable to various scene.Such as, robot 10 can be used for sweeping the floor of family Robot, for utilizing said method to arrange the sweep-out pattern of sweeping robot.Robot 10 can also be such as work Transfer robot in factory, goods can be transported at another according to the route specified at one by it in the time specified.Machine Device people 10 can also be the security patrol robot for family, community, factory etc., and it can be arranged at night along fixing Route goes on patrol, to guarantee the safety of family, community and factory.Such robot can perform the work of the most traditional night watchman With.Robot 10 can also is that the nurse robot for such as family, nursing house, hospital etc., and it can be according to projected route Patrol is to guarantee that such as old man, sufferer etc. are all in normal state.Such as, lie in old man or patient when robot 10 " seeing " On the ground, or when " listening " arrives old man or the calling of patient or call for help, can give notice to relevant people or mechanism in time or alert Accuse
Should be understood that mobile robot 10 described above and correlation technique thereof can also be applied to make robot 10 Other aspects of many moved along regulation path.
Illustrative computer program product and computer-readable recording medium
In addition to said method and equipment, embodiments herein can also is that computer program, and it includes meter Calculation machine programmed instruction, described computer program instructions makes described processor perform this specification description when being run by processor According to the various embodiment of the application patrol path setting method in each step.
Described computer program can be write for holding with the combination in any of one or more programming languages The program code of row the embodiment of the present application operation, described programming language includes object oriented program language, such as Java, C++ etc., also include the process type programming language of routine, such as " C " language or similar programming language.Journey Sequence code can perform the most on the user computing device, perform the most on a user device, independent soft as one Part bag performs, part part on the user computing device performs or on a remote computing completely in remote computing device Or perform on server.
Additionally, embodiments herein can also is that computer-readable recording medium, on it, storage has computer program to refer to Order, described computer program instructions make when being run by processor described processor perform this specification description according to this Shen Please various embodiments patrol path setting method in each step.
Described computer-readable recording medium can use the combination in any of one or more computer-readable recording medium.Computer-readable recording medium can To be readable signal medium or readable storage medium storing program for executing.Readable storage medium storing program for executing such as can include but not limited to electricity, magnetic, optical, electrical The system of magnetic, infrared ray or quasiconductor, device or device, or above combination.Readable storage medium storing program for executing is more specifically Example (non exhaustive list) including: has the electrical connection of one or more wire, portable disc, hard disk, random access memory Device (RAM), read only memory (ROM), erasable type programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc Read only memory (CD-ROM), light storage device, magnetic memory device or the combination of above-mentioned any appropriate.
The ultimate principle of the application is described above in association with specific embodiment, however, it is desirable to it is noted that in this application The advantage mentioned, advantage, effect etc. are only exemplary rather than limiting, it is impossible to think that these advantages, advantage, effect etc. are the application Each embodiment is prerequisite.It addition, detail disclosed above is merely to the effect of example and the work that readily appreciates With, and unrestricted, above-mentioned details is not limiting as the application for above-mentioned concrete details must be used to realize.
The device that relates in the application, device, equipment, the block diagram of system are only used as exemplary example and are not intended to Require or hint must be attached according to the mode shown in block diagram, arrange, configure.As it would be recognized by those skilled in the art that , can connect by any-mode, arrange, configure these devices, device, equipment, system.Such as " include ", " comprising ", " tool Have " etc. word be open vocabulary, refer to " including but not limited to ", and use can be exchanged with it.Vocabulary used herein above "or" and " with " refer to vocabulary "and/or", and use can be exchanged with it, unless it is not such that context is explicitly indicated.Here made Vocabulary " such as " refer to phrase " such as, but not limited to ", and use can be exchanged with it.
It may also be noted that in the equipment and method of the application, each parts or each step are can to decompose and/or weight Combination nova.These decompose and/or reconfigure the equivalents that should be regarded as the application.
Although each step described the most in a certain order in method, however, it is understood that these steps also may be used To carry out in a different order, or multiple step can be carried out simultaneously.Or in certain embodiments, some step is permissible Carry out with being continued for.The method of the present invention contains all these different execution sequence.
The above description of the aspect disclosed in offer is so that any person skilled in the art can make or use this Application.Various amendments in terms of these are readily apparent to those skilled in the art, and define at this General Principle can apply to other aspects without deviating from scope of the present application.Therefore, the application is not intended to be limited to Aspect shown in this, but according to the widest range consistent with principle disclosed herein and novel feature.
In order to purpose of illustration and description has been presented for above description.Additionally, this description is not intended to the reality of the application Execute example and be restricted to form disclosed herein.Although already discussed above multiple exemplary aspect and embodiment, but this area skill Art personnel are it will be recognized that its some modification, revise, change, add and sub-portfolio.

Claims (19)

1. for a patrol path setting method for mobile robot, including:
In response to first order of user, enter patrol path and pattern is set;
Lock described user, and follow the tracks of the movement of described user;
While following the tracks of the movement of described user, determine mobile route;And
Mobile route determined by Jiang is stored as going on patrol path.
Go on patrol path setting method the most as claimed in claim 1, also include:
In response to second order of described user, or it is back to initial point in response to following the tracks of described user, completes described patrol road Footpath.
Go on patrol path setting method the most as claimed in claim 1, wherein, when described first order is local trigger command or remote During journey communications command, lock described user and include locking described user based on image recognition, and
Wherein, when described first order is voice command, locks described user and include based on image recognition and sound localization Combination locks described user.
Go on patrol path setting method the most as claimed in claim 3, wherein, lock described user based on image recognition and include:
Utilize imageing sensor capturing ambient image;
Identify the one or more potential users in the ambient image captured;
When identifying a potential user, lock one potential user;And
When identifying multiple potential user, lock that potential user immediate.
Go on patrol path setting method the most as claimed in claim 4, wherein, when identifying multiple potential user, the most further The type of the first order shown in judging, if described first order is local trigger command, then locks the plurality of potential user In an immediate user, if described first order be telecommunication order, then lock in the plurality of potential user That user moved at first, if there being multiple user all moving, then lock that user moved immediate.
Going on patrol path setting method the most as claimed in claim 3, wherein, combination based on image recognition and sound localization is locked Fixed described user includes:
Sounnd source direction is determined based on described first order as voice command;
Imageing sensor is rotated to described Sounnd source direction;
Utilize described imageing sensor capturing ambient image on described Sounnd source direction;
Identify and lock the user on described Sounnd source direction in captured ambient image.
Going on patrol path setting method the most as claimed in claim 3, wherein, combination based on image recognition and sound localization is locked Fixed described user includes:
Step a, determines Sounnd source direction based on described first order as voice command;
Step b, rotates imageing sensor to described Sounnd source direction;
Step c, utilizes described imageing sensor capturing ambient image on described Sounnd source direction;
Step d, based on user described in the ambient image identification captured;
Step e, successfully identifies described user if based on the ambient image captured, then locks described user;
Step f, fails if based on the ambient image captured and identifies described user, then determine whether according to preset strategy Advance along described Sounnd source direction;
Step g, if determining according to preset strategy and advancing along described Sounnd source direction, then drives described mobile robot along described Sounnd source direction advances, and is then return to step c;And
Step h, if determining according to preset strategy and not advancing along described Sounnd source direction, does not the most drive described mobile robot edge Described Sounnd source direction advances, and also returns to step c.
Going on patrol path setting method the most as claimed in claim 1, wherein, the movement following the tracks of described user includes:
Obtain the profile information of described user;And
The profile following the tracks of described user moves,
Wherein, described profile information includes other local configurations of whole human body contour outline, contouring head, trunk profile or human body In one or more.
Going on patrol path setting method the most as claimed in claim 8, wherein, the movement following the tracks of described user also includes:
In response to the 3rd order from described user, perform corresponding mobile, wherein said 3rd order be voice command or Telecommunication order.
Go on patrol path setting method the most as claimed in claim 1, wherein it is determined that mobile route includes:
Determine each path point relative coordinate relative to previous path point of described mobile robot process.
11. go on patrol path setting method as claimed in claim 10, and wherein, mobile route determined by general is stored as going on patrol road Footpath includes:
It is stored as all path point of described mobile robot process going on patrol path;Or
By in all path point of described mobile robot process, by described user the 4th order indicated by one or Multiple path point are stored as going on patrol path, and wherein, described 4th order is voice command, local trigger command and telecommunication life One of order.
12. go on patrol path setting method as claimed in claim 1, and wherein, described mobile robot has working environment Map, and
Wherein, the patrol path stored is in the path in the map of described working environment.
13. go on patrol path setting method as claimed in claim 1, also include:
While following the tracks of the movement of described user, run the SLAM algorithm map with construction work environment,
Wherein, the patrol path stored is in the path in the map of constructed working environment.
14. go on patrol path setting method as claimed in claim 1, also include:
In response to the 5th order of described user, patrol time and or many in patrol frequency in described patrol path are set Individual, wherein, described 5th order is voice command, local one of trigger command and telecommunication order.
15. go on patrol path setting method as claimed in claim 14, wherein, arrange the patrol time in described patrol path and patrol The one or more tracking when described user completes described patrol path at described mobile robot patrolled in frequency is carried out.
16. 1 kinds of mobile robots, including:
For the driver part making described mobile robot move;
For capturing at least one imageing sensor of image;
For capturing at least one microphone of sound;And
At least one processor communicated with described imageing sensor and described microphone, described processor is configured to run storage Computer instruction in computer-readable medium is to perform the method described in any one in claim 1-15.
17. mobile robots as claimed in claim 16, wherein, at least one microphone described includes microphone array.
18. 1 kinds of computer programs, including computer program instructions, described computer program instructions is one or more Processor makes described processor perform the method described in any one in claim 1-15 when running.
19. 1 kinds of computer-readable mediums, wherein storage has computer program instructions, and described computer program instructions is by one Or make described processor perform the method described in any one in claim 1-15 during multiple processor operation.
CN201610596000.1A 2016-07-22 2016-07-26 Mobile robot and patrol path setting method thereof Active CN106292657B (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610587469 2016-07-22
CN2016105874699 2016-07-22

Publications (2)

Publication Number Publication Date
CN106292657A true CN106292657A (en) 2017-01-04
CN106292657B CN106292657B (en) 2020-05-01

Family

ID=57652757

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610596000.1A Active CN106292657B (en) 2016-07-22 2016-07-26 Mobile robot and patrol path setting method thereof

Country Status (1)

Country Link
CN (1) CN106292657B (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106781332A (en) * 2017-02-14 2017-05-31 上海斐讯数据通信技术有限公司 The method and system of alarm are realized by sweeping robot
CN107180277A (en) * 2017-05-24 2017-09-19 江西理工大学 Using the unmanned plane polling path planing method of the reverse harmony search of elite
CN107315414A (en) * 2017-07-14 2017-11-03 灵动科技(北京)有限公司 A kind of method, device and the robot of control machine people walking
CN107633270A (en) * 2017-09-29 2018-01-26 上海与德通讯技术有限公司 Intelligent identification Method, robot and computer-readable recording medium
CN109048899A (en) * 2018-08-15 2018-12-21 深圳市烽焌信息科技有限公司 A kind of patrol robot and storage medium
CN109062212A (en) * 2018-08-15 2018-12-21 深圳市烽焌信息科技有限公司 A kind of robot and storage medium for patrol
CN109085833A (en) * 2018-08-15 2018-12-25 深圳市烽焌信息科技有限公司 A kind of patrol robot and storage medium
CN109159131A (en) * 2018-09-25 2019-01-08 浙江智玲机器人科技有限公司 Unmanned meal delivery robot
CN109784821A (en) * 2019-04-04 2019-05-21 白冰 A kind of garden servicer and garden service system
CN110941265A (en) * 2019-11-05 2020-03-31 盟广信息技术有限公司 Map entry method and device, computer equipment and storage medium
CN111436861A (en) * 2018-12-27 2020-07-24 北京奇虎科技有限公司 Block edge closing processing method, electronic equipment and readable storage medium
CN112017661A (en) * 2019-05-31 2020-12-01 江苏美的清洁电器股份有限公司 Voice control system and method of sweeping robot and sweeping robot
CN113787517A (en) * 2021-08-20 2021-12-14 科沃斯机器人股份有限公司 Self-moving robot control method, device, equipment and readable storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040158358A1 (en) * 2003-02-06 2004-08-12 Matsushita Electric Industrial Co., Ltd. Method of teaching traveling path to robot and robot having function of learning traveling path
CN104635737A (en) * 2015-01-22 2015-05-20 马鞍山纽泽科技服务有限公司 Garden tour blind-guide-handcart intelligent system
CN104932501A (en) * 2015-05-29 2015-09-23 深圳市傲通环球空气过滤器有限公司 Movement device with path self-learning function
CN105159302A (en) * 2015-09-23 2015-12-16 许继集团有限公司 Automatic guided vehicle (AGV) and automatic navigation method thereof

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040158358A1 (en) * 2003-02-06 2004-08-12 Matsushita Electric Industrial Co., Ltd. Method of teaching traveling path to robot and robot having function of learning traveling path
CN104635737A (en) * 2015-01-22 2015-05-20 马鞍山纽泽科技服务有限公司 Garden tour blind-guide-handcart intelligent system
CN104932501A (en) * 2015-05-29 2015-09-23 深圳市傲通环球空气过滤器有限公司 Movement device with path self-learning function
CN105159302A (en) * 2015-09-23 2015-12-16 许继集团有限公司 Automatic guided vehicle (AGV) and automatic navigation method thereof

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106781332A (en) * 2017-02-14 2017-05-31 上海斐讯数据通信技术有限公司 The method and system of alarm are realized by sweeping robot
CN107180277A (en) * 2017-05-24 2017-09-19 江西理工大学 Using the unmanned plane polling path planing method of the reverse harmony search of elite
CN107180277B (en) * 2017-05-24 2020-03-27 江西理工大学 Unmanned aerial vehicle inspection path planning method applying elite reverse harmony search
CN107315414B (en) * 2017-07-14 2021-04-27 灵动科技(北京)有限公司 Method and device for controlling robot to walk and robot
CN107315414A (en) * 2017-07-14 2017-11-03 灵动科技(北京)有限公司 A kind of method, device and the robot of control machine people walking
CN107633270A (en) * 2017-09-29 2018-01-26 上海与德通讯技术有限公司 Intelligent identification Method, robot and computer-readable recording medium
CN109048899A (en) * 2018-08-15 2018-12-21 深圳市烽焌信息科技有限公司 A kind of patrol robot and storage medium
CN109062212A (en) * 2018-08-15 2018-12-21 深圳市烽焌信息科技有限公司 A kind of robot and storage medium for patrol
CN109085833A (en) * 2018-08-15 2018-12-25 深圳市烽焌信息科技有限公司 A kind of patrol robot and storage medium
CN109159131A (en) * 2018-09-25 2019-01-08 浙江智玲机器人科技有限公司 Unmanned meal delivery robot
CN111436861B (en) * 2018-12-27 2023-02-17 北京奇虎科技有限公司 Block edge closing processing method, electronic equipment and readable storage medium
CN111436861A (en) * 2018-12-27 2020-07-24 北京奇虎科技有限公司 Block edge closing processing method, electronic equipment and readable storage medium
CN109784821A (en) * 2019-04-04 2019-05-21 白冰 A kind of garden servicer and garden service system
CN112017661A (en) * 2019-05-31 2020-12-01 江苏美的清洁电器股份有限公司 Voice control system and method of sweeping robot and sweeping robot
CN110941265A (en) * 2019-11-05 2020-03-31 盟广信息技术有限公司 Map entry method and device, computer equipment and storage medium
CN113787517A (en) * 2021-08-20 2021-12-14 科沃斯机器人股份有限公司 Self-moving robot control method, device, equipment and readable storage medium
CN113787517B (en) * 2021-08-20 2023-09-22 科沃斯机器人股份有限公司 Self-moving robot control method, device, equipment and readable storage medium

Also Published As

Publication number Publication date
CN106292657B (en) 2020-05-01

Similar Documents

Publication Publication Date Title
CN106292657A (en) Mobile robot and patrol path setting method thereof
Li et al. Vision-based mobile indoor assistive navigation aid for blind people
JP4460528B2 (en) IDENTIFICATION OBJECT IDENTIFICATION DEVICE AND ROBOT HAVING THE SAME
US9517559B2 (en) Robot control system, robot control method and output control method
CN104783736B (en) Robot cleaner and people's treatment method using the robot cleaner
JP3945279B2 (en) Obstacle recognition apparatus, obstacle recognition method, obstacle recognition program, and mobile robot apparatus
CN100369487C (en) Object detection device, object detection server, and object detection method
US7873448B2 (en) Robot navigation system avoiding obstacles and setting areas as movable according to circular distance from points on surface of obstacles
CN110275540A (en) Semantic navigation method and its system for sweeping robot
JP2009136967A (en) Communication robot
JP5411789B2 (en) Communication robot
WO2021077941A1 (en) Method and device for robot positioning, smart robot, and storage medium
CN112161618B (en) Storage robot positioning and map construction method, robot and storage medium
CN107566743A (en) The video frequency monitoring method of mobile robot
He et al. Wearable ego-motion tracking for blind navigation in indoor environments
US20200376676A1 (en) Method of localization using multi sensor and robot implementing same
CN106548231A (en) Mobile controller, mobile robot and the method for moving to optimal interaction point
CN110146098A (en) A kind of robot map enlargement method, device, control equipment and storage medium
US10062302B2 (en) Vision-assist systems for orientation and mobility training
WO2019216016A1 (en) Information processing device, information processing method, and program
US11785430B2 (en) System and method for real-time indoor navigation
CN108733059A (en) A kind of guide method and robot
WO2020114214A1 (en) Blind guiding method and apparatus, storage medium and electronic device
Nair et al. ASSIST: Personalized indoor navigation via multimodal sensors and high-level semantic information
US20230050825A1 (en) Hands-Free Crowd Sourced Indoor Navigation System and Method for Guiding Blind and Visually Impaired Persons

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant