WO2020102946A1 - 构建地图、定位、导航、控制方法及系统、移动机器人 - Google Patents

构建地图、定位、导航、控制方法及系统、移动机器人

Info

Publication number
WO2020102946A1
WO2020102946A1 PCT/CN2018/116251 CN2018116251W WO2020102946A1 WO 2020102946 A1 WO2020102946 A1 WO 2020102946A1 CN 2018116251 W CN2018116251 W CN 2018116251W WO 2020102946 A1 WO2020102946 A1 WO 2020102946A1
Authority
WO
WIPO (PCT)
Prior art keywords
mobile robot
information
map
image
projection
Prior art date
Application number
PCT/CN2018/116251
Other languages
English (en)
French (fr)
Inventor
崔彧玮
李重兴
Original Assignee
珊口（深圳）智能科技有限公司
珊口（上海）智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 珊口（深圳）智能科技有限公司, 珊口（上海）智能科技有限公司 filed Critical 珊口（深圳）智能科技有限公司
Priority to PCT/CN2018/116251 priority Critical patent/WO2020102946A1/zh
Priority to CN201880002428.6A priority patent/CN109643127B/zh
Priority to US16/532,552 priority patent/US11320834B2/en
Publication of WO2020102946A1 publication Critical patent/WO2020102946A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • G05D1/0253—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0268—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means
    • G05D1/0274—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means using mapping information stored in a memory device
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • G05D1/0251—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L11/00—Machines for cleaning floors, carpets, furniture, walls, or wall coverings
    • A47L11/40—Parts or details of machines not provided for in groups A47L11/02 - A47L11/38, or not restricted to one of these groups, e.g. handles, arrangements of switches, skirts, buffers, levers
    • A47L11/4011—Regulation of the cleaning machine by electric means; Control systems and remote control systems therefor
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L11/00—Machines for cleaning floors, carpets, furniture, walls, or wall coverings
    • A47L11/40—Parts or details of machines not provided for in groups A47L11/02 - A47L11/38, or not restricted to one of these groups, e.g. handles, arrangements of switches, skirts, buffers, levers
    • A47L11/4061—Steering means; Means for avoiding obstacles; Details related to the place where the driver is accommodated
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L9/00—Details or accessories of suction cleaners, e.g. mechanical means for controlling the suction or for effecting pulsating action; Storing devices specially adapted to suction cleaners or parts thereof; Carrying-vehicles specially adapted for suction cleaners
    • A47L9/28—Installation of the electric equipment, e.g. adaptation or attachment to the suction cleaner; Controlling suction cleaners by electric means
    • A47L9/2805—Parameters or conditions being sensed
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L9/00—Details or accessories of suction cleaners, e.g. mechanical means for controlling the suction or for effecting pulsating action; Storing devices specially adapted to suction cleaners or parts thereof; Carrying-vehicles specially adapted for suction cleaners
    • A47L9/28—Installation of the electric equipment, e.g. adaptation or attachment to the suction cleaner; Controlling suction cleaners by electric means
    • A47L9/2805—Parameters or conditions being sensed
    • A47L9/281—Parameters or conditions being sensed the amount or condition of incoming dirt or dust
    • A47L9/2815—Parameters or conditions being sensed the amount or condition of incoming dirt or dust using optical detectors
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L9/00—Details or accessories of suction cleaners, e.g. mechanical means for controlling the suction or for effecting pulsating action; Storing devices specially adapted to suction cleaners or parts thereof; Carrying-vehicles specially adapted for suction cleaners
    • A47L9/28—Installation of the electric equipment, e.g. adaptation or attachment to the suction cleaner; Controlling suction cleaners by electric means
    • A47L9/2836—Installation of the electric equipment, e.g. adaptation or attachment to the suction cleaner; Controlling suction cleaners by electric means characterised by the parts which are controlled
    • A47L9/2852—Elements for displacement of the vacuum cleaner or the accessories therefor, e.g. wheels, casters or nozzles
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0219—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory ensuring the processing of the whole working surface
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0221—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving a learning process
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0223—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving speed control of the vehicle
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0268—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means
    • G05D1/027—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means comprising intertial navigation means, e.g. azimuth detector
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
    • G05D1/0278—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle using satellite positioning signals, e.g. GPS
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
    • G05D1/0285—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle using signals transmitted via a public communication network, e.g. GSM network
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L2201/00—Robotic cleaning machines, i.e. with automatic control of the travelling movement or the cleaning operation
    • A47L2201/04—Automatic control of the travelling movement; Automatic obstacle detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Definitions

  • the present application relates to the technical field of navigation control of autonomous robots, in particular to a method, system, mobile robot for building maps, positioning, navigation, and control.
  • a mobile robot is a machine device that automatically performs work. It can not only accept human command, but also run pre-programmed programs, or act according to the principles and programs formulated with artificial intelligence technology. This type of mobile robot can be used indoors or outdoors, can be used in industry or home, can be used to replace security patrols, replace people to clean the ground, can also be used for family companionship, auxiliary office, etc.
  • the mobile robot can build the map data of the site where the robot is located, on the other hand, it can also provide route planning and route planning based on the constructed map data. Adjustment and navigation services, which make mobile robots move more efficiently.
  • mobile robots determine the current position in combination with the position information provided by the mobile sensors, and help locate the current position by identifying the image features contained in the single image taken by the camera device , So that the mobile robot can match the image features captured at the current position with the stored positions of the matched image features on the map, thereby achieving rapid positioning.
  • similar positioning methods cannot accurately locate the position of the mobile robot on the map.
  • the purpose of the present application is to provide a method, system, and mobile robot for constructing maps, positioning, navigation, and control to solve the problem that the physical objects cannot be accurately positioned in the prior art.
  • the first aspect of the present application provides a method for constructing a map for a mobile robot including a camera device, wherein the camera device captures a solid object within the field of view at the location of the mobile robot And projected onto the traveling plane of the mobile robot to obtain a projected image
  • the method of constructing a map includes: acquiring the first projected image and the second projection corresponding to the mobile robot traveling to the first position and the second position, respectively Image; based on the angle information of the same physical object identified from the first and second projected images relative to the mobile robot at the first and second positions, and the first and second positions , Determine the projection position information of the physical object in the traveling plane; mark the determined projection position information of the physical object into a preset map, so as to use the map to generate a navigation route.
  • the method further includes the step of identifying the target area corresponding to the same physical object in the first projected image and the second projected image based on the preset type of the physical object information, and The step of marking the identified entity object information in the map.
  • the step of identifying the target area corresponding to the same physical object in the first projected image and the second projected image based on the preset type of the physical object information includes at least one of the following Species: Identify the target area corresponding to the same physical object in the first projected image and the second projected image based on the preset feature information of multiple physical objects; use the preset image recognition algorithm to project the first projected image and the second projected image
  • the target area corresponding to the same entity object in the image is mapped with preset multiple entity object information.
  • the method further includes at least one step of: based on the first target area identified in the first projected image, and using image tracking technology in the second projected image Identify the corresponding second target area to obtain the target area corresponding to the same physical object in the two projection images; match the target area in the first projection image and the second projection image with the common target characteristics to obtain the two projection images Corresponds to the target area of the same entity object.
  • the first aspect of the present application further includes acquiring posture information of the mobile device at corresponding first and second positions, and obtaining the mobile robot from the first position to the second position Relative posture change information of the position; correspondingly, based on the angle information of the same physical object identified from the first and second projected images with respect to the mobile robot at the first and second positions, respectively
  • the step of determining the projection position information of the physical object in the traveling plane includes: offsetting the identified same physical object relative to the mobile robot in the first position based on the relative posture change information An angle deviation in the angle information of each of the position and the second position; and determining the projection position information of the solid object in the travel plane based on the offset angle information and the first position and the second position.
  • the step of determining the projection position information of the solid object in the travel plane includes: based on the angle information, the first position, and the second position, constructing Geometric relationship in the traveling plane; using the constructed geometric relationship to determine the projection position information of the solid object in the traveling plane.
  • the following step is further performed: according to the movement information of the mobile robot, measuring from the first position Relative position information between the current position of the mobile robot; and determining the second position based on the measured relative position information and acquiring the corresponding second projection image.
  • the method further includes the following steps: during the movement of the mobile robot, for the same identified physical object, the foregoing acquiring two projection images and determining are performed multiple times.
  • the second aspect of the present application provides a navigation method for a mobile robot including a camera device, including: acquiring a map constructed using any of the above-described method of constructing a map; using The map sets a navigation route for the mobile robot to perform movement control according to the navigation route.
  • the map includes entity object information and its location information; the step of setting a navigation route using the map includes at least one of the following: based on instruction information including entity object information , Setting a navigation route for navigating to the physical object information; based on the current position information of the mobile robot and the physical object information in the map, setting a navigation route for the mobile robot to move from the current position and pass at least one physical object.
  • a third aspect of the present application provides a positioning method for a mobile robot including a camera device, wherein the camera device captures and projects a solid object within the field of view at the location of the mobile robot and projects To the traveling plane of the mobile robot to obtain a projection image, the mobile robot pre-stores a map marked with the projection position information of the physical object, the positioning method includes: traveling to the first position and the mobile robot respectively The first projection image and the second projection image are correspondingly acquired at the second location; wherein, both the first projection image and the second projection image include a target area projected by the same physical object marked on the map; based on The angle information of the same physical object identified in the first projection image and the second projection image relative to the mobile robot at the first position and the second position, and the first position and the second position determine the Positioning information of the mobile robot on the traveling plane to generate a navigation route based on the positioning information.
  • it further includes the step of identifying the target area corresponding to the same physical object in the first projected image and the second projected image based on the preset type of the physical object information, and The step of determining the location information of the entity object information in the map based on the identified type of the entity object information.
  • the step of identifying the target area corresponding to the same physical object in the first projected image and the second projected image based on the preset type of entity object information includes at least one of the following Species: Identify the target area corresponding to the same physical object in the first projected image and the second projected image based on the preset feature information of multiple physical objects; use the preset image recognition algorithm to project the first projected image and the second projected image
  • the target area corresponding to the same entity object in the image is mapped with preset multiple entity object information.
  • it further includes at least one of the following steps: based on the first target area identified in the first projected image, and using image tracking technology in the second projected image Identify the corresponding second target area to obtain the target area corresponding to the same physical object in the two projection images; match the target area in the first projection image and the second projection image with the common target characteristics to obtain the two projection images Corresponds to the target area of the same entity object.
  • it further includes acquiring posture information of the mobile device at the corresponding first and second positions, and obtaining the mobile robot from the first position to the second position Relative posture change information of the position; correspondingly, based on the angle information of the same physical object identified from the first and second projected images with respect to the mobile robot at the first and second positions, respectively
  • the step of determining the positioning information of the mobile robot on the traveling plane includes: offsetting the identified same physical object relative to the mobile robot at the first position and based on the relative posture change information An angle deviation in the angle information of each of the second positions; and determining positioning information of the mobile robot in the travel plane based on the offset angle information and the first and second positions.
  • the step of determining the positioning information of the mobile robot in the travel plane includes: based on the angle information, the first position and the second position The relative position information between them is used to construct the geometric relationship in the traveling plane; the constructed geometric relationship is used to determine the positioning information of the mobile robot in the traveling plane.
  • the following step is further performed: according to the movement information of the mobile robot, measuring from the first position Relative position information between the current position of the mobile robot; and determining a second position based on the relative position information and acquiring a corresponding second projection image.
  • the positioning method further includes the steps of: for the identified same physical object, and the first position and the first projected image taken, the mobile robot Capture corresponding second projection images at multiple second positions, and perform the aforementioned determination of the mobile robot at the first position according to the first projection images and second projection images respectively captured at the first position and multiple second positions
  • the step of positioning information of the position to obtain at least two corresponding positioning positions of the mobile robot obtained by performing the above steps each time; based on the obtained at least two corresponding first positions of the mobile robot Positioning information of the position, performing error correction processing on the positioning position of the mobile robot at the first position, so as to determine the positioning position after the error correction as the positioning position of the mobile robot at the first position; based on the determined
  • the positioning position of the mobile robot at the first position determines the current positioning information of the mobile robot.
  • the fourth aspect of the present application provides a navigation method for a mobile robot including a camera device, including: acquiring a map constructed using any of the above-described map construction methods; and Determine the current positioning information of the mobile robot by using the positioning method as described above; use the map and the positioning information to set a navigation route for the mobile robot to perform movement control according to the navigation route.
  • the map includes entity object information and its location information; and the step of setting a navigation route using the map and positioning information includes at least one of the following: based on including entity object information Instruction information, setting a navigation route to navigate from the positioning information to the physical object information; based on the positioning information of the mobile robot and the physical object information in the map, setting the mobile robot to move through at least one of the self-location information The navigation route of the entity object.
  • a fifth aspect of the present application provides a mobile robot control method for a mobile robot including a camera device and a mobile device, including: acquiring a navigation route as described above, and following The navigation route controls the mobile device to adjust the position and posture of the mobile robot to autonomously move along the navigation route.
  • a sixth aspect of the present application provides a system for building a map for a mobile robot, including: a camera device, which is provided in the mobile robot and used to capture a field of view at the location of the mobile robot The solid objects within the range are projected onto the traveling plane of the mobile robot to obtain a projected image; a processing device, connected to the camera device, is used to run at least one program to execute any of the above-mentioned map building methods.
  • the main optical axis of the camera device is perpendicular to the traveling plane of the mobile robot.
  • a seventh aspect of the present application provides a navigation system for a mobile robot, including: a camera device, which is provided in the mobile robot and used to capture the field of view at the location of the mobile robot A physical object is projected onto the traveling plane of the mobile robot to obtain a projected image; a processing device, connected to the camera device, is used to run at least one program to perform any of the navigation methods described above.
  • the main optical axis of the camera device is perpendicular to the traveling plane of the mobile robot.
  • an eighth aspect of the present application provides a mobile robot, including: a camera device, provided in the mobile robot, for capturing and projecting solid objects within the field of view at the location of the mobile robot A traveling plane of the mobile robot to obtain a projected image; a mobile device provided on the mobile robot for controlledly adjusting the position and posture of the mobile robot; a first processing device connected to the camera device For running at least one program to perform any of the navigation methods described above; a second processing device, connected to the first processing device and the mobile device, for running at least one program to be based on the first processing The navigation route provided by the device controls the mobile device to execute the control method as described above.
  • the main optical axis of the camera device is perpendicular to the traveling plane of the mobile robot.
  • a ninth aspect of the present application provides a data processing device for a mobile robot including a camera device, wherein the camera device captures a solid object within the field of view at the location of the mobile robot and Projected onto the traveling plane of the mobile robot to obtain a projected image, characterized by comprising: a data interface for connecting the camera device; a storage unit for storing at least one program; a processing unit and the storage unit Connected to a data interface for acquiring images captured by the camera device via the data interface; and for executing the at least one program to perform any method for constructing a map as described above, or performing any method as described above The aforementioned navigation method, or the execution of the control method described above.
  • a tenth aspect of the present application provides a computer-readable storage medium that stores at least one program that, when called, executes the construction of a map as described above Method, or execute the navigation method as described above, or execute the control method as described above.
  • the method, system, and mobile robot for constructing maps, positioning, navigation, and control of the present application acquire corresponding first projections about the traveling plane of the mobile robot at the mobile robot's travel to the first position and the second position, respectively An image and a second projected image; based on the respective angular information of the same physical object identified from the first projected image and the second projected image relative to the mobile robot at the first position and the second position, and the first For the position and the second position, construct a map or locate the mobile robot, and plan a navigation route according to the constructed map and the result of the positioning, and control the mobile robot to move along the navigation route.
  • This application uses two images at two locations to more accurately determine the location information of the physical object, improve the accuracy of the constructed map, and plan the navigation route based on the more accurate positioning information of the mobile robot to accurately control the movement of the mobile robot , Improve the accuracy of mobile robot operation and improve human-computer interaction.
  • FIG. 1 shows a schematic diagram of a mobile robot in a corresponding physical space with a solid object a when it takes a projection image containing a solid object a.
  • FIG. 2 shows a schematic flowchart of a method for constructing a map of the present application in a specific embodiment.
  • FIG. 3 is a schematic diagram showing the principle of determining the position information of the physical object corresponding to the common target area based on the angle information and the first position and the second position.
  • FIG. 4 is a schematic diagram showing the principle of determining the position information of the physical object corresponding to the common target area based on the angle information and the first position and the second position.
  • FIG. 5 shows a schematic flowchart of the navigation method of the present application in a specific embodiment.
  • FIG. 6 shows a schematic flowchart of a navigation method of the present application in a specific embodiment.
  • FIG. 7 shows a schematic flowchart of a positioning method of the present application in a specific embodiment.
  • FIG. 8 shows a schematic flowchart of a method for controlling the mobile robot of the present application in a specific embodiment.
  • FIG. 9 shows a schematic diagram of the composition of the system for constructing a map of the present application in a specific embodiment.
  • FIG. 10 shows a schematic diagram of the composition of the navigation system of the present application in a specific embodiment.
  • FIG. 11 shows a schematic diagram of the composition of the mobile robot of the present application in a specific embodiment.
  • FIG. 12 shows a schematic diagram of the composition of the data processing device of the present application in a specific embodiment.
  • A, B or C or "A, B and / or C” means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C” .
  • the exception to this definition only occurs when a combination of elements, functions, steps, or operations are inherently mutually exclusive in certain ways.
  • Mobile robots perform mobile operations based on navigation control technology.
  • VSLAM Visual Simultaneous Localization and Mapping, real-time visual positioning and map construction
  • the mobile robot constructs a map through the visual information provided by the visual sensor and the mobile information provided by the mobile sensor, and provides the mobile robot with navigation capabilities according to the constructed map, so that the mobile robot can move autonomously.
  • the vision sensor includes a camera device.
  • the movement sensor include a speed sensor, an odometer sensor, a distance sensor, a cliff sensor, and the like.
  • the mobile robot moves on the traveling plane, and acquires and stores a projection image about the traveling plane in advance.
  • the camera device takes a solid object within the field of view at the location of the mobile robot and projects it onto the traveling plane of the mobile robot to obtain a projected image.
  • the physical objects include, for example, televisions, air conditioners, chairs, shoes, and leather balls.
  • the mobile robot determines the current position in combination with the position information provided by the mobile sensor, and helps locate the current position by identifying the image features contained in the single image taken by the camera device, so that the mobile robot can locate it
  • the image features captured at the current position correspond to the stored positions of the matched image features in the map, thereby achieving rapid positioning.
  • similar positioning methods cannot accurately locate the position of the mobile robot on the map.
  • the present application provides a method for constructing a map, which is applied to a mobile robot, by which the entity object taken in the field of view of the camera device can be accurately mapped to the map used by the mobile robot, Furthermore, a map is generated for the mobile robot to set the navigation route according to the located solid object.
  • the physical object includes any physical entity that can be photographed in the physical space moved by the mobile robot, examples but not limited to: balls, shoes, walls, flower pots, coats, roofs, lights, trees, tables, chairs , Refrigerator, TV, sofa, socks, tiled objects, cups, etc.
  • the tiled objects include but are not limited to floor mats, floor tile maps laid on the floor, and tapestries and paintings hanging on the wall.
  • the camera device includes but is not limited to any one of a fisheye camera module and a wide-angle (or non-wide-angle) camera module.
  • the mobile robots include, but are not limited to: family companion mobile robots, cleaning robots, patrol mobile robots, glass-wiping robots, and the like.
  • the mobile robot includes at least one camera device.
  • the camera device takes a solid object within the field of view at the location of the mobile robot and projects it onto the traveling plane of the mobile robot to obtain a projected image.
  • the mobile robot includes a camera device, which is disposed on the top, shoulder, or back of the mobile robot, and the main optical axis is perpendicular to the traveling plane of the mobile robot.
  • the mobile robot includes a plurality of camera devices, and the main optical axis of one camera device is perpendicular to the traveling plane of the mobile robot.
  • FIG. 1 shows a schematic diagram of a mobile robot and a physical object a in a corresponding physical space when they capture a projection image containing the physical object a.
  • the captured physical object a projects the position D1 in the projected image M1 and the same physical object a projects the position D2 in the travel plane M2, where the positions D1 and D2 move relative
  • the position D of the robot has the characteristics of the same angle.
  • FIG. 2 shows a schematic flowchart of a method for constructing a map of the present application in a specific embodiment.
  • the method of building a map can be executed by a processing device included in a mobile robot.
  • the processing device is an electronic device capable of performing numerical operations, logical operations, and data analysis, including but not limited to: CPU, GPU, FPGA, etc., and an intermediate data used for temporarily storing the intermediate data generated during the operation Volatile memory, etc.
  • step S210 the first projected image and the second projected image are correspondingly acquired where the mobile robot travels to the first position and the second position, respectively.
  • the two projected images selected by the processing device should contain the same physical object captured.
  • the processing device acquires the corresponding first projection image and second projection image at the first position and the second position within a moving distance smaller than the diameter according to the diameter of the field of view of the camera device.
  • the processing device may pre-set the time interval at which the camera captures the image to acquire the still images captured at different times through the camera, and the corresponding first and second positions when the still image is captured;
  • the time interval should be at least less than the length of time it takes the mobile robot to move the diameter of a field of view.
  • the camera device can shoot video. Since the video is composed of image frames, during the movement of the mobile robot, the processing device continuously or discontinuously collects the image frames in the acquired video, and then the processing device selects two of the images as Two projected images, and obtaining the corresponding first position and second position, wherein the two selected images have partially overlapping fields of view.
  • the camera device causes the mobile robot to take a projection image of a solid object in its field of view projected onto the traveling plane of the mobile robot at a preset time period, and then the processing device acquires the preset image via the camera device Projection images taken at different moments taken in a time period, and two of the projection images are selected as the first projection image and the second projection image, and the position corresponding to the first projection image is the first position, the second The position corresponding to the projected image is the second position.
  • the time period can be represented by a time unit, or the time period is represented by the number of intervals of image frames.
  • the mobile robot communicates with an intelligent terminal, and the intelligent terminal can modify the time period through a specific APP.
  • the modification interface of the time period is displayed on the touch screen of the smart terminal, and the modification of the time period is completed by touching the modification interface; or directly to the
  • the mobile robot sends a time period modification instruction to modify the time period.
  • the time period modification instruction is, for example, a speech including the modification instruction, and the speech is, for example, "period modification to three seconds".
  • the speech is "the image frame interval is modified to five pictures”.
  • the processing device determines that the basis for selecting the first projection image and the second projection image at the first position and the second position is that the two projection images contain a common target area.
  • the processing device performs step S210 at the same time, or without necessarily performing the sequence, at least one of the following steps S211 and S212.
  • step S211 based on the first target area identified in the first projected image, and using image tracking technology to identify the corresponding second target area in the second projected image, to obtain the correspondence between the two projected images The target area of the same entity object.
  • the second projected image and its corresponding second position are acquired.
  • the processing device uses the image features in the first projection image obtained by the image processing algorithm and determines the first target area based on the image features, and uses image tracking technology to track at least one projection captured by the camera after the first projection image Image, and after continuous tracking to determine that the image feature still exists in the second projection image captured at the second location, determine to acquire the first projection image and the second projection image captured at the first location and the second location, And determine that the tracked image features are the same physical object.
  • the image features include preset graphic features corresponding to the type of physical object, or image features obtained through an image processing algorithm.
  • the image processing algorithm includes but is not limited to at least one of the following: grayscale processing, sharpening processing, contour extraction, angle extraction, line extraction, using image processing algorithms obtained through machine learning.
  • the image processing algorithms obtained through machine learning include but are not limited to: neural network algorithms, clustering algorithms, etc.
  • step S212 the target areas in the first projection image and the second projection image having common target characteristics are matched to obtain target areas corresponding to the same physical object in the two projection images.
  • the processing device obtains two projected images at the first position and the second position respectively, and recognizes the image features separately, and matches the image features in the two projected images, thereby matching the image features in the target area As a target area with a common target feature, and determine that the target area with a common target feature corresponds to the same entity object.
  • the processing device takes the two projection images acquired at the first position and the second position as the first projection image and the second projection image, respectively, and determines the corresponding first position and second position.
  • the processing device maps the image features identified in the first projected image to the corresponding image area in the second projected image by means of the direction changed by the mobile robot moving from the first position to the second position, And find matching image features in the vicinity of the image area, so that the target area where the matched image features are located is the target area with common target characteristics, and it is determined that the target area with common target characteristics corresponds to the same entity object.
  • the processing device will determine the first projected image and the second projected image, as well as the corresponding first and second positions.
  • the image features representing the common target area include at least one of the following: feature points, feature lines, feature colors, image features on key areas within the corresponding common target area marked in the image based on the object recognition model, and so on.
  • the characteristic points include corner points, end points, and inflection points corresponding to the physical objects. Determine the relative spatial position of the local position of the object corresponding to each feature point on the solid object and the mobile robot.
  • the characteristic line is, for example, a contour line identifying the solid object. Even if the mobile robot compares the first projected image and the second projected image according to the characteristic information of the preset multiple physical objects, and the first projected image and the second projected image The areas in which both match the image features of a solid object are the common target area.
  • Examples of the object recognition model include an object recognition algorithm model constructed based on a neural network, a clustering algorithm model for identifying objects, and the like.
  • the object recognition model is used to identify the common target area in the two projected images, the same color rectangular frame (or rounded rectangular frame, circle, etc.) can be used to represent the identified common target in the two projected images.
  • the area, the edge area and the center area of the corresponding two rectangular frames are the key areas, and the image features in the corresponding areas are extracted.
  • the processing device can determine the type of the entity object.
  • the processing device may not necessarily be able to determine that the physical object corresponding to the common target area matched in the first projection image and the second projection image is a table.
  • the method for building a map further includes: identifying a target area corresponding to the same physical object in the first projected image and the second projected image based on the preset type of the physical object information.
  • the preset type of entity object information refers to the category to which the identified entity object belongs. For example, dining tables and writing desks are classified as types of tables, and for example, computer chairs and benches are classified as types of chairs.
  • the types of the entity target information are pre-screened based on the environment in which the mobile robot moves and stored in the storage device of the mobile robot.
  • the type of the entity target information is characterized by the feature information of the entity target that may be ingested and placed in the environment or the image feature of the entity target in the image.
  • the type of the entity target information may be characterized by the image feature of the object, which can identify the entity object in the image.
  • the processing device identifies target regions corresponding to the same physical object in the first projected image and the second projected image based on preset feature information of multiple physical objects. For example, the target regions corresponding to the same solid object in the first projected image and the second projected image are traversed through preset image features of various classes to determine the category to which the identified solid object belongs.
  • the kind of entity target information may be characterized by object classification.
  • the processing device uses a preset image recognition algorithm to map the target area corresponding to the same entity object in the first projected image and the second projected image with preset multiple kinds of entity object information.
  • the program stored in the storage device includes the network structure and connection method of the neural network model.
  • the neural network model may be a convolutional neural network, and the network structure includes an input layer, at least one hidden layer, and at least one output layer.
  • the input layer is used to receive the captured image or the pre-processed image;
  • the hidden layer includes a convolution layer and an activation function layer, and may even include a normalization layer, a pooling layer, and a fusion layer At least one of; etc .;
  • the output layer is used to output an image labeled with an object type label.
  • the connection method is determined according to the connection relationship of each layer in the neural network model. For example, the connection relationship between the front and back layers set based on data transmission, the connection relationship with the data of the front layer based on the size of the convolution kernel in each hidden layer, and the full connection.
  • the neural network model classifies each object identified from the image.
  • the processing device in order to determine the first position and the second position, the processing device also performs steps S213 and S214 during the execution of step S210.
  • step S213 based on the movement information of the mobile robot, relative position information from the first position to the current position of the mobile robot is measured.
  • step S214 the second position is determined based on the measured relative position information and the corresponding second projection image is acquired.
  • the position measuring device of the mobile robot is used, and the processing device monitors the mobile robot from the first The relative displacement and relative angle between a position and the current position of the mobile robot, and when it is determined that the relative displacement and / or relative angle exceeds a preset measurement threshold, determine the second position and obtain the second position corresponding to the second position 2. Project the image.
  • the measurement threshold may be a fixed value or a relative value.
  • the measurement threshold is 5 cm.
  • the measurement threshold is n times the number of roller turns of the mobile robot, n> 1.
  • the mobile robot's position measuring device is used to obtain the mobile robot's movement information.
  • the position measuring device includes, but is not limited to, a displacement sensor, a distance measuring sensor, an angle sensor, a binocular camera device, a speed sensor, etc., which are all provided in the mobile robot.
  • the processing device may monitor only one of the position measuring devices to monitor the relative displacement and relative deflection angle of the mobile robot from the first position to the current position of the mobile robot.
  • the position measuring device is a counting sensor provided on the motor of the mobile robot, and counts by the number of turns of the motor to obtain the relative displacement of the mobile robot from the first position to the second position, and uses the motor Obtain posture information etc. from the angle of operation.
  • the position measuring device measures the traveling speed of the mobile robot, and calculates the relative displacement from the first position to the second position using the traveling speed and the travel time. Or, combining the position data provided by various sensors, camera devices, etc. to monitor the relative displacement and relative angle of the mobile robot from the first position to the current position of the mobile robot. For example, using preset weights and combining multiple sets of distance measurement data of a mobile robot relative to a fixed target acquired by a binocular camera device, and data measured by motor operation to determine the current position of the mobile robot from the first position to the mobile robot Relative displacement and relative angle between positions.
  • the position measuring device is a GPS (Global Positioning System), which monitors the vector displacement and vector deviation angle of the mobile robot from the first position to the current position of the mobile robot according to the GPS ; And when it is determined that the phase vector displacement and / or vector deviation angle exceeds the preset measurement threshold, determine the second position and obtain a second projection image corresponding to the second position, and obtain the first position and the second position Relative location information between.
  • GPS Global Positioning System
  • the position measuring device is a device based on measuring wireless signals, for example, the position measuring device is a Bluetooth (or WiFi) positioning device; the position measuring device is based on the first and second positions Measure the power of the received wireless positioning signal to determine the relative position of each position relative to the preset wireless positioning signal transmitting device; use the position measuring device to monitor the mobile robot from the first position to the current position of the mobile robot Relative displacement and relative declination angle; and when it is determined that the relative displacement and / or relative declination angle exceeds a preset measurement threshold, determine a second position and obtain a second projection image corresponding to the second position, and obtain the The relative position information between the first position and the second position.
  • the position measuring device is a Bluetooth (or WiFi) positioning device
  • the position measuring device is based on the first and second positions Measure the power of the received wireless positioning signal to determine the relative position of each position relative to the preset wireless positioning signal transmitting device
  • use the position measuring device to monitor the mobile robot from the first position to the current position of the mobile robot Re
  • the mapping relationship between the unit grid length and the actual displacement is predetermined, and the mobile robot is monitored according to the movement information obtained during the movement of the mobile robot.
  • step S220 based on the respective angular information of the same physical object identified from the first and second projected images with respect to the mobile robot at the first position and the second position, and the first position and The second position determines the projection position information of the physical object in the traveling plane.
  • the processing device determines the angle between the target area of the corresponding solid object and the coordinate reference line through the projected image, it means that it can be determined that the solid object is located in a direction offset from the traveling direction of the mobile robot by the angle.
  • the target area in the projected image of the same position of the mobile robot at different positions and the locations of the mobile robot are mapped to the coordinate system, which can be based on the
  • the obtained angles and positions are used to construct a geometric relationship, and the projected position information of the solid object in the traveling plane is determined using the constructed geometric relationship.
  • the geometric relationship includes but is not limited to: triangle relationship, quadrilateral relationship, etc.
  • the coordinate system is used to help understanding. During actual map construction, the coordinate system is not necessarily drawn or displayed.
  • the step S220 includes: constructing a geometric relationship in the traveling plane based on the angle information, the first position and the second position; and determining the entity using the constructed geometric relationship Projection position information of the object in the traveling plane.
  • FIG. 3 is a schematic diagram illustrating the principle of determining the position information of the physical object based on the angle information and the first position and the second position.
  • the processing device acquires the first projected image shown in FIG. 3 at the first position P1.
  • the second position P2 acquires a second projection image as shown in FIG.
  • connection P1P2 is formed between the first position P1 and the second position P2; and, the processing device further determines that the same physical object is located at Target areas O1 and O2 in the first projected image and the second projected image; for the same physical object, it should be located on the line P1O1 between the target area O1 and the first position P1, and located on the target
  • the connection line P2O2 between the area O2 and the second position P2 in other words, the physical object is located at the position O where the line segments P1O1 and P2O2 intersect, that is, the position O is the projection position where the physical object is projected onto the traveling plane.
  • the processing device determines the point according to the angle ⁇ 1 between the connection P1O1 and the connection P1P2, the angle ⁇ 1 between the connection P2O2 and the connection P1P2, and the distance between the connection P1P2 The position of the point O in the triangle formed by P1, P2, and O, and then the projection position information of the solid object in the traveling plane is obtained.
  • the acquired first position and second position include displacement information and posture information.
  • relative posture change information can be obtained according to the posture information of the first position and the second position.
  • the relative posture change information includes an angle at which the mobile robot is rotated at a second position relative to the mobile robot at a first position in a traveling plane.
  • the step S120 includes: offsetting the angle deviation of the identified angle information of the same physical object relative to the mobile robot at the first position and the second position based on the relative posture change information; and based on the offset The angle information and the first position and the second position determine the projection position information of the solid object in the traveling plane.
  • FIG. 4 is a schematic diagram illustrating the principle of determining the position information of the physical object corresponding to the common target area based on the angle information and the first position and the second position.
  • the mobile robot moves in the direction of travel in the positive direction of the Y axis at the first position P1 'and moves in the positive direction of the Y' axis to the left of the positive direction of the Y axis at the second position P2 ', thereby
  • the processing device may determine that the relative posture change information of the mobile robot between the first position and the second position is the included angle ⁇ .
  • the included angle ⁇ can be provided by an angle sensor in the position measuring device of the mobile robot.
  • the processing device determines that the physical object corresponding to the target area is located at the line P1'O1 'according to the position of the target area in the coordinate system in the first projection image taken at the first position P1'
  • its angle ⁇ 2 with the positive direction of the Y-axis according to the position of the target area in the coordinate system of the second projection image taken at the second position P2 ′, determine the corresponding area of the target area
  • the physical object is located on the line where the line P2'O2 'is located, and its angle ⁇ 2 with the positive direction of the Y' axis is due to the angle ⁇ between the positive direction of the Y 'axis and the positive direction of the Y axis.
  • the processing device determines the connection line between the physical object and the second position, it is necessary to offset the angle deviation caused by the included angle ⁇ , that is, the supplementary angle obtained by subtracting the included angle ⁇ from the included angle ⁇ 2 is the included angle ⁇ 2 ′. That is, the projection position information of the solid object can be determined according to the distance between the connection line P1'P2 ', the triangular relationship constructed by the included angle ⁇ 2 and the included angle ⁇ 2'.
  • the height of the solid object in the vertical direction of the traveling plane is not considered.
  • the height measurement device of the mobile robot can measure the height information of the physical object when locating the position of the physical object. It should be regarded as an embodiment expanded based on the map construction scheme provided by the present application. It will not be detailed here.
  • step S230 the determined projection position information of the solid object is marked on a preset map.
  • the processing device obtains the position information of each mapped onto the map according to the first position, the second position and the obtained projection position information of the solid object in the traveling plane, that is, the physical object
  • the location information of the projection is marked on the map.
  • the processing device may mark the image feature of the physical object at the corresponding position of the map, so that when the mobile robot again captures the image feature matching this image feature In the target area, use the above method to determine the position of the mobile robot on the map.
  • the identified entity target information is marked on the map according to the identified entity object information.
  • a text description, image identification, or number corresponding to each of the entity target information may be marked on the map.
  • the text description may be a name description of the type of each entity target information, such as a table, chair, Name description of objects such as flower pots, TVs and refrigerators.
  • the name corresponding to the table is described as "table”
  • the name corresponding to the TV is described as "TV”.
  • the image identifier may be an icon of an actual image corresponding to the type of target information of each entity.
  • the number may be a digital label arranged in advance corresponding to each of the entity target information. For example, "001" stands for refrigerator, "002" stands for chair, "003" stands for table, etc.
  • the projected position information of the marked solid object is continuously corrected, or marked on the map after the correction. For this reason, during the movement of the mobile robot, for any of the identified physical objects, any of the examples described in the foregoing steps S210-S220 is performed multiple times to obtain corresponding corresponding entities by performing the above steps each time At least two projected position information of the object. Due to the material of the traveling plane of the mobile robot, the first position information and the second position information obtained during the movement of the mobile robot cannot be accurately determined. For example, when the mobile robot moves on the carpet, its actual moving distance is less than the distance measured by the position measuring device. As another example, when the mobile robot moves on the floor, its actual moving distance is close to the distance measured by the position measuring device.
  • the method for constructing a map provided by the present application can use the projection position of at least two triangles composed of multiple positions such as a first position, a second position, and a third position of the mobile robot during movement to the same physical object
  • the information is determined, and error correction is performed using the following steps, that is, a step of performing error correction processing on the projection position of the physical object based on the obtained at least two projection position information corresponding to the same physical object.
  • the error correction method includes but is not limited to: 1) Performing an error correction process on each obtained projection position information in a weighted average manner to mark the error-corrected projection position information on a preset map. For example, the weighted average of the horizontal and vertical coordinates in the obtained projection position information of the same physical object is obtained, and the error-corrected projection position information is obtained and marked on the map. 2) Iteratively process each obtained projection position information in a numerical optimization manner to obtain estimated projection position information after iterative processing, and mark the estimated projection position information on a preset map.
  • the initial value is used for the second time to determine the projection position information corresponding to the entity object, and the estimated projection position information is obtained through an iterative loop of preset conditions, and the estimated projection position information and the corresponding entity
  • the information such as the image characteristics and type of the object is marked on the map; wherein, the initial value may be set randomly, or obtained according to the methods provided in the foregoing steps S210-S220.
  • the Nth time to obtain the projection position information according to the foregoing steps S210-S220 includes using at least one position information and corresponding angle information that the mobile robot has measured, and the newly measured position information and corresponding angle information , To determine the Nth projection position information; or use all the newly measured two position information and the corresponding angle information to determine the Nth projection position information; or two position information measured from any two different And the corresponding angle information to determine the Nth projection position information; where N> 1.
  • the multiple projection position information corresponding to the same entity object is collected in the manner shown in steps S210-S220, and each projection position information is substituted into a numerical optimization algorithm including a gradient to filter or Estimate the projected position information with the smallest error, and mark the obtained projected position information with the smallest error and the image features and types of the corresponding solid objects on the map.
  • the method for constructing a map of the present application captures a projection image of a solid object projected on the traveling plane of the mobile robot according to the camera device taking the projection of a solid object in the field of view of the location of the mobile robot to obtain the mobile robot in A first projected image at the position and a second projected image at the second position; and based on the angle information in the traveling plane of the common target area identified from the first projected image and the second projected image, And the first position and the second position, to determine the position information of the physical object corresponding to the common target area in the physical space traveled by the mobile robot; to obtain more accurate position information of the physical object, and to The determined position information of the entity object is marked into a preset map, thereby improving the accuracy of the map.
  • FIG. 5 is a schematic flowchart of a navigation method of the present application in a specific embodiment.
  • the navigation method is applied to the mobile robot including a camera device.
  • the camera device takes a solid object in the field of view at the location of the mobile robot and projects it onto the traveling plane of the mobile robot to obtain a projected image.
  • the navigation The method is mainly performed by mobile robots.
  • a map constructed by a method of constructing a map is acquired; the method of constructing a map includes: correspondingly acquiring a first projection image and a second projection where the mobile robot travels to a first position and a second position, respectively Image; determine the common target area based on the angle information in the traveling plane of the common target area identified from the first and second projected images, and the first and second positions The position information of the corresponding physical object in the physical space traveled by the mobile robot; mark the determined position information of the physical object on a preset map.
  • the method of constructing the map please refer to FIG. 2 and related description about the method shown in FIG. 2.
  • step S520 a navigation route is set using the map for the mobile robot to perform movement control according to the navigation route.
  • a navigation route suitable for the purpose of movement can be set using the map.
  • the navigation route it sets needs to pass through each actual object.
  • the navigation route set by the robot may use the actual object marked as the destination address.
  • a navigation route corresponding to the cleaning mode is set.
  • the map contains entity object information and location information of entity objects; the step S520 further includes a step S521.
  • the step S521 includes: setting a navigation route for navigating to the entity object information based on the instruction information containing the entity object information;
  • the entity object information is, for example, a name description for each kind of entity object information , For example, including the description of the names of objects such as tables, chairs, flower pots, TVs and refrigerators.
  • the way to obtain the instruction information containing the entity object information includes but is not limited to: voice mode, text mode, etc.
  • the instruction may further include an execution instruction of the mobile robot.
  • the instructions also include cleaning instructions, inspection instructions, remote control instructions, and so on.
  • the step S521 may include: acquiring a voice message, and identifying an instruction containing entity object information from the voice message.
  • the mobile robot can directly receive the user's voice information and recognize the instruction of the entity object information included in the information. For example, the user can directly speak “table” to the mobile robot, and after receiving the instruction, the mobile robot moves to the table to perform the corresponding processing set in advance. And the navigation route for the mobile robot to move from the current position to the table can be planned according to the information of the solid objects passing on the route. The mobile robot can move through the flowerpot, TV, sofa, etc. from the current position to the navigation route of the table.
  • the cleaning robot after presetting the cleaning robot receiving the user's instruction containing the physical object information, the navigation route is planned according to the constructed map, so that the mobile robot moves to the position corresponding to the physical object information for cleaning .
  • the cleaning robot forms a navigation route formed by the plant based on the flowerpot, TV and sofa according to the constructed map, and the cleaning The robot moves to the table and performs the cleaning operation after passing the navigation route formed by the flower pot, TV, and sofa.
  • the voice information is not limited to short instructions that only represent physical object information, but may also be long instructions that include physical object information. For example, if the user voices "go to the desk", the mobile robot can recognize the voice information. Entity object information "table" instruction, and then follow-up operations.
  • the step S521 further includes: obtaining an instruction containing entity object information from a terminal device.
  • the terminal device is wirelessly connected with the mobile robot.
  • a user inputs an instruction containing information of a physical object via a terminal device in text.
  • the user inputs "table” in text through the mobile phone APP.
  • an instruction for inputting physical object information via a terminal device via voice For example, the user inputs “table” in a voice manner through the mobile phone APP.
  • the voice information input by the user is not limited to short instructions representing only the physical object information, but may also be long instructions including the physical object information, for example, if the user's voice "go to the desk", the terminal device translates it into text and extracts it Among them, keywords such as table match the translated text to the corresponding instructions and send them to the mobile robot.
  • the terminal device can be connected to the mobile robot in a wireless manner such as wifi connection, near field communication, or Bluetooth pairing to transmit the command received by the terminal device to the mobile robot for subsequent operations.
  • the terminal device is, for example, a smart phone, a tablet computer, a wearable device, or other smart devices with smart processing functions.
  • step S520 further includes step S522.
  • the step S522 includes: setting a navigation route for the mobile robot to move from the current position and pass at least one physical object based on the current position information of the mobile robot and the physical object information in the map.
  • the mobile robot generates navigation from the current position information to the target position information based on the current position of the mobile robot on the map, the target position where the entity object information is located, and the position information or preset path of other objects on the map route.
  • the navigation route may be the shortest route from the current position of the mobile robot to the destination position.
  • the navigation route may be a route from the current position of the mobile robot to the destination position that does not overlap with the previous travel path of the mobile robot. Taking the mobile robot as a cleaning robot as an example, the current route is set based on the uncleaned area and the cleaned area. There is no need to repeat the navigation route for cleaning the cleaned area from the position to the destination position, thereby improving the cleaning efficiency.
  • the mobile robot is a cleaning robot; the step S520 further includes: setting a navigation route traversing the physical object in the sweeping area based on the physical objects marked in the map and their location information.
  • the cleaning robot designs a navigation route for traversing the cleaning area based on the predetermined cleaning area, where the cleaning robot determines that it is convenient to clean according to the corresponding label information according to the marking information of the solid objects in the map within the cleaning area Navigation route.
  • the cleaning area includes but is not limited to at least one of the following: a cleaning area divided according to a preset number of grids, a cleaning area divided according to rooms, and the like. For example, in a sweeping area in the acquired map, the table and its location information are marked. Therefore, when designing the navigation route, the navigation route including the rotation around the table legs is designed.
  • the navigation method of the present application determines the relative spatial position between the identified entity object and the mobile robot by identifying the entity object corresponding to the pre-marked entity object information and based on the image captured by the camera device, as described in accordance with FIG. 2
  • the map constructed by the method marks the entity object information to generate the map marked with the entity object information, so that in subsequent movement control, the user instruction can be identified based on the entity object information marked on the map
  • the destination location, and then moving to the location improves the accuracy of the mobile robot's navigation route and improves human-computer interaction.
  • FIG. 6 shows a schematic flowchart of a navigation method of the present application in a specific embodiment.
  • the navigation method is applied to the mobile robot including a camera device.
  • the camera device takes a solid object in the field of view at the location of the mobile robot and projects it onto the traveling plane of the mobile robot to obtain a projected image.
  • the navigation The method is mainly performed by mobile robots.
  • step S610 a map constructed by a method of building a map is acquired; and a positioning method is used to determine the current positioning information of the mobile robot.
  • the method of constructing the map includes: acquiring the first projection image and the second projection image corresponding to the mobile robot traveling to the first position and the second position respectively; based on the first projection image and the second projection image The angle information of the common target area identified in the image in the traveling plane, and the first position and the second position determine the physical object corresponding to the common target area in the physical travel of the mobile robot Position information in space; mark the determined position information of the solid object on a preset map.
  • FIG. 2 For the method of constructing the map, please refer to FIG. 2 and related description about the method shown in FIG. 2.
  • FIG. 7 shows a schematic flowchart of a positioning method of the present application in a specific embodiment.
  • the camera device takes a solid object in the field of view at the location of the mobile robot and projects it onto the traveling plane of the mobile robot to obtain a projected image
  • the main optical axis of the camera device is perpendicular to the traveling plane
  • the positioning The method may be performed by a processing device included in the mobile robot.
  • the processing device is an electronic device capable of performing numerical operations, logical operations, and data analysis, including but not limited to: CPU, GPU, FPGA, etc., and an intermediate data used for temporarily storing the intermediate data generated during the operation Volatile memory, etc.
  • the positioning method includes:
  • Step S710 Pre-generate the map constructed by the method of constructing the map; wherein, for the method of constructing the map, refer to FIG. 2 and the related description about the method shown in FIG. 2.
  • Step S720 Acquire the first projected image and the second projected image correspondingly with the camera device when the mobile robot travels to the first position and the second position; wherein, in the first projected image and the second projected image Each contains the target area projected by the same physical object marked on the map;
  • the processing device acquires the corresponding first projection image and second projection image at the first position and the second position within a moving distance smaller than the diameter according to the diameter of the field of view of the camera device.
  • the processing device may pre-set the time interval at which the camera captures the image to acquire the still images captured at different times through the camera, and the corresponding first and second positions when the still image is captured;
  • the time interval should be at least less than the length of time it takes the mobile robot to move the diameter of a field of view.
  • the camera device can shoot video. Since the video is composed of image frames, during the movement of the mobile robot, the processing device continuously or discontinuously collects the image frames in the acquired video, and then the processing device selects two of the images as Two projected images, and obtaining the corresponding first position and second position, wherein the two selected images have partially overlapping fields of view.
  • the camera device causes the mobile robot to take a projection image of a solid object in its field of view projected onto the traveling plane of the mobile robot at a preset time period, and then the processing device acquires the preset image via the camera device Projection images taken at different moments taken in a time period, and two of the projection images are selected as the first projection image and the second projection image, and the position corresponding to the first projection image is the first position, the second The position corresponding to the projected image is the second position.
  • the time period can be represented by a time unit, or the time period is represented by the number of intervals of image frames.
  • the mobile robot communicates with an intelligent terminal, and the intelligent terminal can modify the time period through a specific APP.
  • the modification interface of the time period is displayed on the touch screen of the smart terminal, and the modification of the time period is completed by touching the modification interface; or directly to the
  • the mobile robot sends a time period modification instruction to modify the time period.
  • the time period modification instruction is, for example, a speech including the modification instruction, and the speech is, for example, "period modification to three seconds".
  • the speech is "the image frame interval is modified to five pictures”.
  • the processing device determines that the basis for selecting the first projection image and the second projection image at the first position and the second position is that the two projection images contain a common target area.
  • the processing device simultaneously performs step S720, or does not necessarily perform sequentially, and also performs at least one of the following steps S721 and S722.
  • step S721 based on the first target area identified in the first projected image, and using image tracking technology to identify the corresponding second target area in the second projected image, to obtain the correspondence between the two projected images The target area of the same entity object.
  • the second projected image and its corresponding second position are acquired.
  • the processing device uses the image features in the first projection image obtained by the image processing algorithm and determines the first target area based on the image features, and uses image tracking technology to track at least one projection captured by the camera after the first projection image Image, and after continuous tracking to determine that the image feature still exists in the second projection image captured at the second location, determine to acquire the first projection image and the second projection image captured at the first location and the second location, And determine that the tracked image features are the same physical object.
  • the image features include preset graphic features corresponding to the type of physical object, or image features obtained through an image processing algorithm.
  • the image processing algorithm includes but is not limited to at least one of the following: grayscale processing, sharpening processing, contour extraction, angle extraction, line extraction, using image processing algorithms obtained through machine learning.
  • the image processing algorithms obtained through machine learning include but are not limited to: neural network algorithms, clustering algorithms, etc.
  • step S722 the target areas in the first projection image and the second projection image having common target characteristics are matched to obtain target areas corresponding to the same physical object in the two projection images.
  • the processing device obtains two projected images at the first position and the second position respectively, and recognizes the image features separately, and matches the image features in the two projected images, thereby matching the image features in the target area As a target area with a common target feature, and determine that the target area with a common target feature corresponds to the same entity object.
  • the processing device takes the two projection images acquired at the first position and the second position as the first projection image and the second projection image, respectively, and determines the corresponding first position and second position.
  • the processing device maps the image features identified in the first projected image to the corresponding image area in the second projected image by means of the direction changed by the mobile robot moving from the first position to the second position, And find matching image features in the vicinity of the image area, so that the target area where the matched image features are located is the target area with the common target characteristics, and it is determined that the target area with the common target characteristics corresponds to the same entity object.
  • the processing device will determine the first projected image and the second projected image, as well as the corresponding first and second positions.
  • the image features representing the common target area include at least one of the following: feature points, feature lines, feature colors, image features on key areas in the corresponding common target area marked in the image based on the object recognition model, and the like.
  • the characteristic points include corner points, end points, and inflection points corresponding to the physical objects. Determine the relative spatial position of the local position of the object corresponding to each feature point on the solid object and the mobile robot.
  • the characteristic line is, for example, a contour line identifying the solid object. Even if the mobile robot compares the first projected image and the second projected image according to the characteristic information of the preset multiple physical objects, and the first projected image and the second projected image The areas in which both match the image features of a solid object are the common target area.
  • Examples of the object recognition model include an object recognition algorithm model constructed based on a neural network, a clustering algorithm model for identifying objects, and the like.
  • the object recognition model is used to identify the common target area in the two projected images, the same color rectangular frame (or rounded rectangular frame, circle, etc.) can be used to represent the identified common target in the two projected images.
  • the area, the edge area and the center area of the corresponding two rectangular frames are the key areas, and the image features in the corresponding areas are extracted.
  • the processing device can determine the type of the entity object.
  • the processing device may not necessarily be able to determine that the physical object corresponding to the common target area matched in the first projection image and the second projection image is a table.
  • the positioning method further includes: identifying a target area corresponding to the same physical object in the first projected image and the second projected image based on the preset type of the physical object information.
  • the preset type of entity object information refers to the category to which the identified entity object belongs. For example, dining tables and writing desks are classified as types of tables, and for example, computer chairs and benches are classified as types of chairs.
  • the types of the entity target information are pre-screened based on the environment in which the mobile robot moves and stored in the storage device of the mobile robot.
  • the type of the entity target information is characterized by the feature information of the entity target that may be ingested and placed in the environment or the image feature of the entity target in the image.
  • the type of the entity target information may be characterized by the image feature of the object, which can identify the entity object in the image.
  • the processing device identifies target regions corresponding to the same physical object in the first projected image and the second projected image based on preset feature information of multiple physical objects. For example, the target regions corresponding to the same solid object in the first projected image and the second projected image are traversed through preset image features of various classes to determine the category to which the identified solid object belongs.
  • the kind of entity target information may be characterized by object classification.
  • the processing device uses a preset image recognition algorithm to map the target area corresponding to the same entity object in the first projected image and the second projected image with preset multiple kinds of entity object information.
  • the program stored in the storage device includes the network structure and connection method of the neural network model.
  • the neural network model may be a convolutional neural network, and the network structure includes an input layer, at least one hidden layer, and at least one output layer.
  • the input layer is used to receive the captured image or the pre-processed image;
  • the hidden layer includes a convolution layer and an activation function layer, and may even include a normalization layer, a pooling layer, and a fusion layer At least one of; etc .;
  • the output layer is used to output an image labeled with an object type label.
  • the connection method is determined according to the connection relationship of each layer in the neural network model. For example, the connection relationship between the front and back layers set based on data transmission, the connection relationship with the data of the front layer based on the size of the convolution kernel in each hidden layer, and the full connection.
  • the neural network model classifies each object identified from the image.
  • the processing device in order to determine the first position and the second position, the processing device also performs steps S731 and S732 during the execution of step S730.
  • step S731 based on the movement information of the mobile robot, relative position information from the first position to the current position of the mobile robot is measured.
  • step S732 the second position is determined based on the measured relative position information and the corresponding second projection image is acquired.
  • the position measuring device of the mobile robot is used, and the processing device monitors the mobile robot from the first The relative displacement and relative angle between a position and the current position of the mobile robot, and when it is determined that the relative displacement and / or relative angle exceeds a preset measurement threshold, determine the second position and obtain the second position corresponding to the second position 2. Project the image.
  • the measurement threshold may be a fixed value or a relative value.
  • the measurement threshold is 5 cm.
  • the measurement threshold is n times the number of roller turns of the mobile robot, n> 1.
  • the mobile robot's position measuring device is used to obtain the mobile robot's movement information.
  • the position measuring device includes, but is not limited to, a displacement sensor, a distance measuring sensor, an angle sensor, a binocular camera device, a speed sensor, etc., which are all provided in the mobile robot.
  • the processing device may monitor only one of the position measuring devices to monitor the relative displacement and relative deflection angle of the mobile robot from the first position to the current position of the mobile robot.
  • the position measuring device is a counting sensor provided on the motor of the mobile robot, and counts by the number of turns of the motor to obtain the relative displacement of the mobile robot from the first position to the second position, and uses the motor Obtain posture information etc. from the angle of operation.
  • the position measuring device measures the traveling speed of the mobile robot, and calculates the relative displacement from the first position to the second position using the traveling speed and the travel time. Or, combining the position data provided by various sensors, camera devices, etc. to monitor the relative displacement and relative angle of the mobile robot from the first position to the current position of the mobile robot. For example, using preset weights and combining multiple sets of distance measurement data of a mobile robot relative to a fixed target acquired by a binocular camera device, and data measured by motor operation to determine the current position of the mobile robot from the first position to the mobile robot Relative displacement and relative angle between positions.
  • the position measuring device is a GPS (Global Positioning System), which monitors the vector displacement and vector deviation angle of the mobile robot from the first position to the current position of the mobile robot according to the GPS ; And when it is determined that the phase vector displacement and / or vector deviation angle exceeds the preset measurement threshold, determine the second position and obtain a second projection image corresponding to the second position, and obtain the first position and the second position Relative location information between.
  • GPS Global Positioning System
  • the position measuring device is a device based on measuring wireless signals, for example, the position measuring device is a Bluetooth (or WiFi) positioning device; the position measuring device is based on the first and second positions Measure the power of the received wireless positioning signal to determine the relative position of each position relative to the preset wireless positioning signal transmitting device; use the position measuring device to monitor the mobile robot from the first position to the current position of the mobile robot Relative displacement and relative declination angle; and when it is determined that the relative displacement and / or relative declination angle exceeds a preset measurement threshold, determine a second position and obtain a second projection image corresponding to the second position, and obtain the The relative position information between the first position and the second position.
  • the position measuring device is a Bluetooth (or WiFi) positioning device
  • the position measuring device is based on the first and second positions Measure the power of the received wireless positioning signal to determine the relative position of each position relative to the preset wireless positioning signal transmitting device
  • use the position measuring device to monitor the mobile robot from the first position to the current position of the mobile robot Re
  • the mapping relationship between the unit grid length and the actual displacement is predetermined, and the mobile robot is monitored according to the movement information obtained during the movement of the mobile robot.
  • step S730 After determining the relative position information between the first position and the second position, and the angle information of the mobile robot relative to the identified solid object at the first position and the second position, respectively, step S730 is executed.
  • Step S730 Based on the angle information of the same physical object identified from the first and second projected images with respect to the mobile robot at the first and second positions, and the first and second positions Position, determining positioning information of the mobile robot on the traveling plane, so as to generate a navigation route based on the positioning information.
  • the mobile robot can map the recognized image features or entity object information types with the corresponding information marked in the map.
  • the mapping relationship is one or more.
  • the mapping relationship may be only one, and the position of the identified physical object in the physical space can be obtained by the position in the map .
  • the type of the identified image feature or entity object information is not unique in the map
  • the positional relationship, and the relative positional relationship of the mobile robot's movement obtain a mapping relationship that matches multiple features or types from the map, and then obtain the physical location of each marked feature or type in the traveling plane of the mobile robot from the map.
  • the position of the solid object captured by the camera device in the projection image to represent the position of the solid object projected on the traveling plane of the mobile robot, and use the solid object in the projection image
  • the angle in the position relative to the moving direction of the mobile robot represents the angle of the position of the traveling plane of the physical object projected onto the moving robot relative to the moving direction of the mobile robot.
  • the plane of the acquired projection image is parallel to the traveling plane of the mobile robot. Therefore, in the coordinate system constructed based on the traveling plane of the mobile robot (or based on the plane of the projection image), the same entity
  • the deflection angle of the object in the projected image between the target area and the coordinate reference line corresponds to the deflection angle of the position of the entity object projected on the traveling plane relative to the coordinate reference line.
  • the coordinate reference line is a straight line in the plane where the coordinate system is located.
  • the coordinate reference line is a straight line where the advancing direction of the mobile robot is located.
  • the target area in the projected image of the same position of the mobile robot at different positions and the locations of the mobile robot are mapped to the coordinate system, which can be based on the
  • the obtained angles and positions are used to construct a geometric relationship, and the constructed geometric relationship is used to determine the positioning information of the mobile robot in the traveling plane.
  • the geometric relationship includes but is not limited to: triangle relationship, quadrilateral relationship, etc.
  • the coordinate system is used to help understanding. During actual map construction, the coordinate system is not necessarily drawn or displayed.
  • the step S730 further includes: constructing a geometric relationship in the traveling plane based on the angle information, the first position and the second position; and determining the geometric relationship using the constructed geometric relationship Positioning information of the mobile robot in the travel plane.
  • FIG. 3 is a schematic diagram illustrating the principle of determining the position information of the solid object based on the angle information and the first position and the second position.
  • the processing device acquires the first projection image shown in FIG. 3 at the first position P1
  • the processing device acquires the second projection image shown in FIG.
  • the device may obtain the connection P1P2 formed between the first position P1 and the second position P2 according to the aforementioned measurement methods; and, the processing device also determines that the same physical object is in the first projected image and the Target areas O1 and O2 in the second projected image; for the same physical object, it should be located on the connection line P1O1 between the target area O1 and the first position P1, and between the target area O2 and the second position P2 On the connecting line P2O2, in other words, the physical object is located at the position O where the line segments P1O1 and P2O2 intersect, that is, the position O is the projection position where the physical object is projected onto the traveling plane.
  • the processing device is based on the angle ⁇ 1 between the connection P1O1 and the connection P1P2, the angle ⁇ 1 between the connection P2O2 and the connection P1P2, the distance between the connection P1P2, and according to the map
  • the position of the point O in the travel plane obtained in step B is the physical position of the mobile robot in the travel plane at the points P1 and P2 of the triangle formed by the points P1, P2, and O.
  • the acquired first position and second position include displacement information and posture information.
  • relative posture change information can be obtained according to the posture information of the first position and the second position.
  • the relative posture change information includes an angle at which the mobile robot is rotated at a second position relative to the mobile robot at a first position in a traveling plane.
  • the step S730 further includes: offsetting the angle deviation of the identified angle information of the same physical object relative to the mobile robot at the first position and the second position based on the relative posture change information; And the first position and the second position determine the positioning information of the mobile robot in the traveling plane.
  • FIG. 4 is a schematic diagram illustrating the principle of determining the position information of the physical object corresponding to the common target area based on the angle information and the first position and the second position.
  • the mobile robot moves in the direction of travel in the positive direction of the Y axis at the first position P1 'and moves in the positive direction of the Y' axis to the left of the positive direction of the Y axis at the second position P2 ', thereby
  • the processing device may determine that the relative posture change information of the mobile robot between the first position and the second position is the included angle ⁇ .
  • the included angle ⁇ can be provided by an angle sensor in the position measuring device of the mobile robot.
  • the processing device determines that the physical object corresponding to the target area is located at the line P1'O1 'according to the position of the target area in the coordinate system in the first projection image taken at the first position P1'
  • its angle ⁇ 2 with the positive direction of the Y-axis according to the position of the target area in the coordinate system of the second projection image taken at the second position P2 ′, determine the corresponding area of the target area
  • the physical object is located on the line where the line P2'O2 'is located, and its angle ⁇ 2 with the positive direction of the Y' axis is due to the angle ⁇ between the positive direction of the Y 'axis and the positive direction of the Y axis.
  • the processing device determines the connection line between the physical object and the second position, it is necessary to offset the angle deviation caused by the included angle ⁇ , that is, the supplementary angle obtained by subtracting the included angle ⁇ from the included angle ⁇ 2 is the included angle ⁇ 2 ′. That is, according to the distance between the connection line P1'P2 ', the triangular relationship constructed by the included angle ⁇ 2 and the included angle ⁇ 2', determine the location information.
  • the height of the solid object in the vertical direction of the traveling plane is not considered.
  • the height measurement device of the mobile robot can measure the height information of the physical object when locating the position of the physical object. It should be regarded as an embodiment expanded based on the map construction scheme provided by the present application. It will not be detailed here.
  • the determined positioning information of the mobile robot is continuously corrected, or determined after multiple positioning corrections.
  • the positioning method further includes: for the identified same physical object, and the first location and the captured first projection image, the mobile robot is allowed to capture corresponding second projection images at a plurality of second locations And perform the aforementioned steps of determining the positioning information of the mobile robot at the first position according to the first projected image and the second projected image respectively captured by the first position and the plurality of second positions, to obtain At least two positioning information corresponding to the first position of the mobile robot obtained by performing the above steps.
  • this step can use the relative position information between the first position and each second position, and the angle of the physical object identified in the first projected image relative to the first position
  • the information, and the angle information of the physical object identified in each second projected image relative to the second position, and the position information of the physical object marked in the map respectively determine the positioning information of each first position.
  • the relative position information between the first position and the second position acquired during the mobile robot's movement cannot be accurately determined. For example, when the mobile robot moves on the carpet, its actual moving distance is less than the distance measured by the position measuring device.
  • the positioning information of the first position calculated by the relative position information, the angle information, and the position information of the physical object marked on the map will be different.
  • the method for constructing a map provided by the present application may be composed of at least two triangles composed of a first position of the mobile robot during movement, a plurality of second positions, and position information of the physical objects marked on the map, etc.
  • the positioning information of the first position is determined, and error correction is performed using the following steps, that is, positioning the mobile robot at the first position based on the obtained positioning information of at least two of the mobile robot corresponding to the first position
  • the position is subjected to error correction processing so as to determine the positioning position after the error correction as the positioning position when the mobile robot is at the first position.
  • the error correction methods include but are not limited to: 1) Perform an error correction process on the obtained positioning information using a weighted average method to use the error-corrected positioning information as the positioning of the mobile robot at the first position information. For example, the obtained projection position information of the same physical object is weighted and averaged on the horizontal and vertical coordinates to obtain error corrected positioning information as positioning information of the mobile robot at the first position. 2) Iteratively process the positioning information of the first position obtained by numerical optimization to obtain the estimated positioning information corresponding to the first position after iterative processing, and use the estimated positioning information as the mobile robot at the first position Location information.
  • the moving direction of the mobile robot is controlled and the second time Performing a positioning operation on the first position, obtaining an estimated positioning information corresponding to the first position through an iterative loop of preset conditions, and using the estimated positioning information corresponding to the first position as the positioning information of the first position;
  • the initial value may be set randomly, or obtained according to the methods provided in the foregoing steps S710-S720.
  • the current positioning information of the mobile robot is determined.
  • the relative position information of the mobile robot's position measuring device is used to move from the first position to the current position, and the current positioning information of the mobile robot is determined according to the obtained positioning information of the first position.
  • the measurement error of the mobile robot's position measuring device is determined, and according to the measurement The error cancels the error of the relative position information measured by the mobile robot from the first position to the current position, and determines its positioning information at the current position.
  • the positioning method of the present application captures the first projected image at the first position and the second projected image at the second position of the mobile robot according to the camera device; and based on the first projected image and the second projected image
  • the angle information of the identified common target area in the traveling plane, and the first and second positions determine the positioning information of the mobile robot in the traveling plane; to obtain more accurate information about movement Robot positioning information.
  • the present application After determining the current position of the mobile robot, the present application also provides a navigation method, which includes step S610 and step S620.
  • step S610 a map constructed using any method for constructing a map, and positioning information of the mobile robot obtained using the foregoing steps S710-S730 are acquired.
  • step S620 a navigation route is set using the previously constructed map and the obtained positioning information, so that the mobile robot can perform movement control according to the navigation route.
  • the navigation method is mainly executed by the processing device of the mobile robot, and the processing device provides the designed navigation route to the lower computer for the lower computer to control the mobile device such as the motor in the mobile robot to move.
  • the lower computer are MCU, CPU and so on.
  • a navigation route adapted to the movement destination can be set using the map.
  • the navigation route it sets needs to pass through each actual object.
  • the navigation route set by the robot may use the actual object marked as the destination address.
  • a navigation route corresponding to the cleaning mode is set.
  • the map contains entity object information and location information of entity objects; the step S620 further includes a step S621.
  • the step S621 includes: setting a navigation route for navigating to the entity object information based on the instruction information containing the entity object information; in this embodiment, the entity object information is, for example, a name description for each kind of entity object information , For example, including the description of the names of objects such as tables, chairs, flower pots, TVs and refrigerators.
  • the way to obtain the instruction information containing the entity object information includes but is not limited to: voice mode, text mode, etc.
  • the instruction may further include an execution instruction of the mobile robot.
  • the instructions also include cleaning instructions, inspection instructions, remote control instructions, and so on.
  • the step S621 may include: acquiring a voice message, and identifying an instruction containing entity object information from the voice message.
  • the mobile robot can directly receive the user's voice information and recognize the instruction of the entity object information included in the information. For example, the user can directly speak “table” to the mobile robot, and after receiving the instruction, the mobile robot moves to the table to perform the corresponding processing set in advance. And the navigation route for the mobile robot to move from the current position to the table can be planned according to the information of the solid objects passing on the route. The mobile robot can move through the flowerpot, TV, sofa, etc. from the current position to the navigation route of the table.
  • the cleaning robot after presetting the cleaning robot receiving the user's instruction containing the physical object information, the navigation route is planned according to the constructed map, so that the mobile robot moves to the position corresponding to the physical object information for cleaning .
  • the cleaning robot forms a navigation route formed by the plant based on the flowerpot, TV and sofa according to the constructed map, and the cleaning The robot moves to the table and performs the cleaning operation after passing the navigation route formed by the flower pot, TV, and sofa.
  • the voice information is not limited to short instructions representing only physical object information, but may also be long instructions including physical object information, for example, if the user voices "go to the desk", the mobile robot may recognize Entity object information "table” instruction, and then follow-up operations.
  • the step S621 further includes: obtaining an instruction containing entity object information from a terminal device.
  • the terminal device is wirelessly connected with the mobile robot.
  • a user inputs an instruction containing information of a physical object via a terminal device in text.
  • the user inputs "table” in text through the mobile phone APP.
  • an instruction for inputting physical object information via a terminal device via voice For example, the user inputs “table” in a voice manner through the mobile phone APP.
  • the voice information input by the user is not limited to short instructions representing only the physical object information, but may also be long instructions including the physical object information, for example, if the user's voice "go to the desk", the terminal device translates it into text and extracts it Among them, keywords such as table match the translated text to the corresponding instructions and send them to the mobile robot.
  • the terminal device can be connected to the mobile robot in a wireless manner such as wifi connection, near field communication, or Bluetooth pairing to transmit the command received by the terminal device to the mobile robot for subsequent operations.
  • the terminal device is, for example, a smart phone, a tablet computer, a wearable device, or other smart devices with smart processing functions.
  • step S620 further includes step S622.
  • the step S622 includes: setting a navigation route for the mobile robot to move through at least one physical object based on the positioning information of the mobile robot and the physical object information in the map.
  • the mobile robot generates navigation from the current position information to the target position information based on the current position of the mobile robot on the map, the target position where the entity object information is located, and the position information or preset path of other objects on the map route.
  • the navigation route may be the shortest route from the current position of the mobile robot to the destination position.
  • the navigation route may be a route from the current position of the mobile robot to the destination position that does not overlap with the previous travel path of the mobile robot. Taking the mobile robot as a cleaning robot as an example, the current route is set based on the uncleaned area and the cleaned area. There is no need to repeat the navigation route for cleaning the cleaned area from the position to the destination position, thereby improving the cleaning efficiency.
  • the mobile robot is a cleaning robot; the step S620 further includes: setting a navigation route traversing the physical objects in the sweeping area based on the physical objects marked in the map and their location information.
  • the cleaning robot designs a navigation route for traversing the cleaning area based on the predetermined cleaning area, where the cleaning robot determines that it is convenient to clean according to the corresponding label information according to the marking information of the solid objects in the map within the cleaning area Navigation route.
  • the cleaning area includes but is not limited to at least one of the following: a cleaning area divided according to a preset number of grids, a cleaning area divided according to rooms, and the like. For example, in a sweeping area in the acquired map, the table and its location information are marked. Therefore, when designing the navigation route, the navigation route including the rotation around the table legs is designed.
  • the navigation method of the present application determines the relative spatial position between the identified entity object and the mobile robot by identifying the entity object that matches the pre-marked entity object information and based on the image captured by the camera device, and is described in accordance with FIG. 2
  • the map constructed by the method marks the entity object information to generate the map marked with the entity object information. According to the positioning method shown in FIG. 7, more accurate positioning information about the mobile robot is obtained. And the use of the map and positioning information to set the navigation route improves the accuracy of the mobile robot's navigation route and improves human-computer interaction.
  • FIG. 8 shows a schematic flowchart of a method for controlling a mobile robot of the present application in a specific embodiment.
  • the control method of the mobile robot is applied to a mobile robot including a camera device and a mobile device, and the control method is mainly performed by a control system of the mobile robot, the control system includes controlling the movement of the mobile robot according to the acquired navigation route
  • the controller of the device includes controlling the movement of the mobile robot according to the acquired navigation route
  • the controller of the device includes a roller and a driver of the roller, and the driver is a motor and the like.
  • the controller is exemplified by MCU.
  • the temporary memory includes but is not limited to registers, volatile memory, and the like.
  • step S810 a navigation route is obtained; the navigation route is a navigation route formed according to the navigation method shown in FIG. 5 or FIG. 6 and related descriptions.
  • step S820 the mobile device is controlled according to the navigation route to adjust the position and posture of the mobile robot to autonomously move along the navigation route.
  • the position information and posture information of the mobile robot are acquired in real time through the mobile sensor of the mobile robot, and the mobile device is controlled according to the navigation route so that the mobile robot can respond accordingly The attitude moves on the navigation route.
  • the destination position indicated by the physical object information on the map marked with the physical object information, the current position and current pose of the mobile robot on the map obtained by the mobile sensor, and the Other information, such as position information of other objects or a preset path controls the mobile robot to move from the corresponding current position to the corresponding destination position.
  • the mobile robot can be controlled to move from the current position to the destination according to the movable route marked on the map, and when the mobile robot needs to turn during the autonomous movement according to the movable route, the mobile robot is allowed to perform Rotate to adjust the posture of the mobile robot and travel along the next path in the adjusted posture.
  • the mobile robot may also send a moving route from the corresponding current location to the corresponding destination location to the terminal device communicating with the mobile robot to display on the map displayed by the terminal device Movement route, current location information, destination location information, or any combination thereof, so as to facilitate the user to view.
  • the current position of the robot can be displayed on the map displayed by the terminal device in real time, and when the destination position is the position of the table, the text description, display and display of the "table" at the destination position on the map The icon corresponding to the image of the table or the preset number corresponding to the table is displayed, and the moving route from the current position of the mobile robot to the table is displayed on the map.
  • the terminal device modifies the movement route or the destination location information by modifying the navigation route of the displayed map, and updates the modification operation to the mobile robot to cause the movement The robot moves according to the modified movement route or destination location information.
  • the modification operation may be a touch operation on a touch screen displaying the movement route and destination location information, or it may be to cause the terminal device to receive voice information including a modification instruction.
  • the current mobile robot is entering the table via the TV according to the moving route, and the user finds that there is an untouchable obstacle between the table and the TV, and then changes the destination location information to the TV to stop the mobile robot At the TV position, to avoid collision with the obstacle.
  • the modification instruction is voice information, here is, for example: "the modification destination is TV”.
  • a modification instruction can also be directly sent to the mobile robot by voice.
  • a voice of "modifying the destination location as a TV” is sent to the mobile robot, and the destination location of modifying the navigation route of the mobile robot is a TV.
  • the control method of the mobile robot of the present application forms a navigation route about the constructed map according to the method shown in FIG. 5 or FIG. 6, and the entity object information is marked in the constructed map to generate the marked entity object
  • the map of the information, and in the movement control of the mobile robot, the destination position in the user instruction can be identified based on the information of the physical objects marked on the map to form the navigation route, and then the mobile robot is controlled according to the The navigation route moves to the destination location, which improves the accuracy of the mobile robot's navigation route and improves human-computer interaction.
  • FIG. 9 shows a schematic diagram of the composition of the system for constructing a map of the present application in a specific embodiment.
  • a system for building a map is used for the mobile robot.
  • the system for building a map includes a camera device 910 and a processing device 920.
  • the camera device 910 is provided in the mobile robot, and is used to capture a solid object within the field of view at the location of the mobile robot and project it onto the traveling plane of the mobile robot to obtain a projected image; the camera device 910 includes but It is not limited to any of fisheye camera modules and wide-angle (or non-wide-angle) camera modules.
  • the mobile robots include, but are not limited to: family companion mobile robots, cleaning robots, patrol mobile robots, glass-wiping robots, and the like.
  • the mobile robot includes at least one camera device 910.
  • the camera device 910 takes a solid object in the field of view at the location of the mobile robot and projects it onto the traveling plane of the mobile robot to obtain a projected image.
  • the mobile robot includes a camera device, which is disposed on the top, shoulder, or back of the mobile robot, and the main optical axis is perpendicular to the traveling plane of the mobile robot.
  • the mobile robot includes a plurality of camera devices, and the main optical axis of one camera device 910 is perpendicular to the traveling plane of the mobile robot.
  • the projection image formed by the projection of the image captured by the camera set in the above manner on the travel plane of the mobile robot is equivalent to the vertical projection of the image captured by the camera on the travel plane, based on the movement
  • the first angle of the connection of the common target area with respect to the connection of the mobile robot from the first position to the second position is based on the acquisition of the mobile camera at the second position with respect to the A second projected image of the traveling plane projection, identifying the common target area in the second projected image, and acquiring a connection line between the mobile robot and the common target area at the second position regarding the movement
  • the second angle at which the robot travels from the first position to the second position, and according to the distance between the first position and the second position and the first angle and the second angle Acquire the position of the solid object corresponding to the common target area projected into the traveling plane.
  • the projection position of the solid object projected into the traveling plane is The angle of the line of the mobile robot at the first position with respect to the line of the mobile robot traveling from the first position to the second position is the same as the first angle, the physical object The angle between the projection position projected into the traveling plane and the connection line of the mobile robot at the second position with respect to the connection line of the mobile robot traveling from the first position to the second position is The second angle is the same.
  • the processing device 920 is an electronic device capable of performing numerical operations, logical operations, and data analysis, including but not limited to: CPU, GPU, FPGA, etc., and volatile intermediate data generated during temporary storage operations sexual memory, etc.
  • the processing device 920 is connected to the camera device 910 and is used to run at least one program to execute a method of building a map. For the method of constructing the map, refer to FIG. 2 and related descriptions regarding FIG. 2, and details are not described here.
  • FIG. 10 shows a schematic diagram of the composition of the navigation system of the present application in a specific embodiment.
  • a navigation system is used for the mobile robot.
  • the navigation system includes a camera device 1010 and a processing device 1020.
  • the camera device 1010 is provided in the mobile robot, and is used to capture a solid object in the field of view at the location of the mobile robot and project it onto the traveling plane of the mobile robot to obtain a projected image; the camera device 1010 includes but It is not limited to any of fisheye camera modules and wide-angle (or non-wide-angle) camera modules.
  • the mobile robots include, but are not limited to: family companion mobile robots, cleaning robots, patrol mobile robots, glass-wiping robots, and the like.
  • the mobile robot includes at least one camera device 1010.
  • the camera device 1010 takes a solid object within the field of view at the location of the mobile robot and projects it onto the traveling plane of the mobile robot to obtain a projected image.
  • the mobile robot includes a camera device 1010, which is disposed on the top, shoulder, or back of the mobile robot, and the main optical axis is perpendicular to the traveling plane of the mobile robot.
  • the mobile robot includes a plurality of camera devices, and the main optical axis of one camera device 1010 is perpendicular to the traveling plane of the mobile robot.
  • the processing device 1020 is an electronic device capable of performing numerical operations, logical operations, and data analysis, including but not limited to: CPU, GPU, FPGA, etc., and volatile intermediate data generated during temporary storage operations Sexual memory, etc.
  • the processing device 1020 is connected to the camera device 1010 for running at least one program to execute the navigation method.
  • the navigation method refer to FIG. 5 and related descriptions about FIG. 5, or for the navigation method, refer to FIG. 6 and related descriptions about FIG. 6, and details are not described herein.
  • FIG. 11 shows a schematic diagram of the composition of the mobile robot of the present application in a specific embodiment.
  • the mobile robot includes a camera device 1110, a mobile device 1120, a first processing device 1130, and a second processing device 1140.
  • the camera device 1110 is provided in the mobile robot.
  • the camera device 1110 takes a solid object in the field of view at the location of the mobile robot and projects it onto the traveling plane of the mobile robot to obtain a projected image; Including but not limited to any one of a fisheye camera module and a wide-angle (or non-wide-angle) camera module.
  • the mobile robots include, but are not limited to: family companion mobile robots, cleaning robots, patrol mobile robots, glass-wiping robots, and the like.
  • the mobile robot includes at least one camera device 1010.
  • the camera device 1010 takes a solid object within the field of view at the location of the mobile robot and projects it onto the traveling plane of the mobile robot to obtain a projected image.
  • the mobile robot includes a camera device 1010, which is disposed on the top, shoulder, or back of the mobile robot, and the main optical axis is perpendicular to the traveling plane of the mobile robot.
  • the mobile robot includes a plurality of camera devices, and the main optical axis of one camera device 1010 is perpendicular to the traveling plane of the mobile robot.
  • the mobile device 1120 is provided in the mobile robot for controlledly adjusting the position and posture of the mobile robot; in some embodiments, the mobile device 1120 includes a roller and a roller driver, and the driver is Motor etc.
  • the first processing device 1130 is connected to the camera device 1110 for running at least one program to execute a navigation method; for the navigation method, refer to FIG. 5 and related descriptions of FIG. 5 or refer to FIG. 6 and related descriptions of FIG. 6 The description will not be repeated here.
  • the second processing device 1140 connected to the first processing device 1130 and the mobile device 1120, is used to run at least one program to control the mobile device 1120 based on the navigation route provided by the first processing device 1130 Implement control methods.
  • control method refer to FIG. 8 and related descriptions regarding FIG. 8, and details are not described here.
  • FIG. 12 shows a schematic diagram of the composition of the data processing device of the present application in a specific embodiment.
  • the data processing device is used for a mobile robot including a camera device, wherein the camera device picks up a solid object within the field of view at the location of the mobile robot and projects it onto the traveling plane of the mobile robot to obtain a projected image, the data processing
  • the device includes a data interface 1210, a storage unit 1220, and a processing unit 1230.
  • the data interface 1210 is used to connect the camera device; the camera device takes a solid object in the field of view at the location of the mobile robot and projects it onto the traveling plane of the mobile robot to obtain a projected image.
  • the mobile robot includes a camera device, which is disposed on the top, shoulder, or back of the mobile robot, and the main optical axis is perpendicular to the traveling plane of the mobile robot.
  • the mobile robot includes a plurality of camera devices, and the main optical axis of one camera device is perpendicular to the traveling plane of the mobile robot.
  • the storage unit 1220 is configured to store at least one program
  • the processing unit 1230 is connected to the storage unit 1220 and the data interface 1210 for acquiring the image captured by the camera device via the data interface 1210; and for executing the at least one program to execute building a map Method, or execute a navigation method, or execute a control method.
  • a map Method or execute a navigation method, or execute a control method.
  • a computer-readable storage medium stores at least one program, and the at least one program executes a method of building a map when called, or Perform a navigation method or a control method.
  • the method of constructing the map refer to FIG. 2 and related descriptions of FIG. 2
  • the navigation method refer to FIG. 5 and related descriptions of FIG. 5 or refer to FIG. 6 and related descriptions of FIG. 6, and for the control method, refer to FIG. 8 and related descriptions regarding FIG. 8 are not repeated here.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • General Physics & Mathematics (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Automation & Control Theory (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Electromagnetism (AREA)
  • Mechanical Engineering (AREA)
  • Theoretical Computer Science (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
  • Manipulator (AREA)
  • Navigation (AREA)

Abstract

提供一种构建地图、定位、导航、控制方法及系统、移动机器人，该构建地图的方法包括：分别在移动机器人行进至第一位置和第二位置处获取第一投影图像和第二投影图像（S210）；基于从第一投影图像和第二投影图像中所识别出的同一实体对象相对于移动机器人在第一位置及第二位置各自的角度信息、以及第一位置和第二位置，确定实体对象在行进平面中的投影位置信息（S220）；将所确定的实体对象的投影位置信息标记到预设的地图中（S230）。由此通过两个位置处的两幅图像较为精准确定实体对象的位置，提高构建的地图的精确性。

Description

构建地图、定位、导航、控制方法及系统、移动机器人 技术领域
本申请涉及自主机器人的导航控制技术领域，特别是涉及一种构建地图、定位、导航、控制方法及系统、移动机器人。
背景技术
移动机器人是自动执行工作的机器装置。它既可以接受人类指挥，又可以运行预先编排的程序，也可以根据以人工智能技术制定的原则纲领行动。这类移动机器人可用在室内或室外，可用于工业或家庭，可用于取代保安巡视、取代人们清洁地面，还可用于家庭陪伴、辅助办公等。
移动机器人基于视觉传感器所提供的视觉信息并结合其他移动传感器所提供的移动数据，一方面能够构建机器人所在场地的地图数据，另一方面，还可基于所构建的地图数据提供路线规划、路线规划调整及导航服务，这使得移动机器人的移动效率更高。然而，在实际应用中，在现有的实际应用中，移动机器人结合移动传感器所提供的位置信息确定当前位置，以及通过识别摄像装置所摄取的单幅图像所包含的图像特征来帮助定位当前位置，以便于移动机器人将其在当前位置所拍摄的图像特征，与所存储的相匹配图像特征在地图中的位置进行对应，由此实现快速定位。然而类似的定位方法，并不能准确定位移动机器人在地图中的位置。
发明内容
鉴于以上所述现有技术的缺点，本申请的目的在于提供一种构建地图、定位、导航、控制方法及系统、移动机器人，用于解决现有技术中不能对实体对象进行精确定位的问题。
为实现上述目的及其他相关目的，本申请的第一方面提供一种构建地图的方法，用于包含摄像装置的移动机器人，其中，所述摄像装置在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像，所述构建地图的方法包括：分别在所述移动机器人行进至第一位置和第二位置处对应获取第一投影图像和第二投影图像；基于从所述第一投影图像和第二投影图像中所识别出的同一实体对象相对于移动机器人在第一位置及第二位置各自的角度信息、以及所述第一位置和第二位置，确定所述实体对象在所述行进平面中的投影位置信息；将所确定的实体对象的投影位置信息标记到预设的地图中，以便利用所述地图生成导航路线。
在本申请的第一方面的某些实施方式中，还包括基于预设的实体对象信息的种类，识别所述第一投影图像和第二投影图像中对应同一实体对象的目标区域的步骤，以及将所识别出的实体对象信息标记在所述地图中的步骤。
在本申请的第一方面的某些实施方式中，所述基于预设的实体对象信息的种类，识别第一投影图像和第二投影图像中对应同一实体对象的目标区域的步骤包括以下至少一种：基于预设置的多种实体对象的特征信息，识别第一投影图像和第二投影图像中对应同一实体对象的目标区域；利用预设的图像识别算法，将第一投影图像和第二投影图像中对应同一实体对象的目标区域与预设的多种实体对象信息进行映射处理。
在本申请的第一方面的某些实施方式中，还包括以下至少一步骤：基于在所述第一投影图像中识别的第一目标区域，并利用图像追踪技术在所述第二投影图像中识别对应的第二目标区域，以得到两幅投影图像中对应同一实体对象的目标区域；匹配所述第一投影图像和第二投影图像中具有共同目标特征的目标区域，以得到两幅投影图像中对应同一实体对象的目标区域。
在本申请的第一方面的某些实施方式中，还包括在对应所述第一位置和第二位置处获取所述移动装置的姿态信息，并得到所述移动机器人自第一位置至第二位置的相对姿态变化信息；对应地，基于从所述第一投影图像和第二投影图像中所识别出的同一实体对象相对于移动机器人在第一位置及第二位置各自的角度信息、以及所述第一位置和第二位置，确定所述实体对象在所述行进平面中的投影位置信息的步骤包括：基于所述相对姿态变化信息抵消所识别出的同一实体对象相对于移动机器人在第一位置及第二位置各自的角度信息中的角度偏差；以及基于抵消后的角度信息和所述第一位置和第二位置确定所述实体对象在所述行进平面中的投影位置信息。
在本申请的第一方面的某些实施方式中，所述确定实体对象在所述行进平面中的投影位置信息的步骤包括：基于所述角度信息、所述第一位置和第二位置，构建在所述行进平面内的几何关系；利用所构建的几何关系确定所述实体对象在所述行进平面中的投影位置信息。
在本申请的第一方面的某些实施方式中，在所述移动机器人从第一位置移动至第二位置期间还执行以下步骤：根据所述移动机器人的移动信息，测量从所述第一位置至移动机器人当前位置之间的相对位置信息；以及基于所测量的相对位置信息确定第二位置并获取对应的第二投影图像。
在本申请的第一方面的某些实施方式中，所述方法还包括以下步骤：在所述移动机器人移动期间，对于所识别出的同一实体对象，多次执行前述获取两幅投影图像以及确定所述实体对象的投影位置信息的步骤，以获得藉由每次执行上述步骤而得到的同一实体对象的至少 两个投影位置信息；以及基于所得到的对应同一实体对象的至少两个投影位置信息，对所述实体对象的投影位置进行误差修正处理，以便将误差修正后的投影位置信息标记到预设地图中。
为实现上述目的及其他相关目的，本申请的第二方面提供一种导航方法，用于包含摄像装置的移动机器人，包括：获取利用如上任一所述的构建地图的方法所构建的地图；利用所述地图设置导航路线，以供所述移动机器人按照所述导航路线进行移动控制。
在本申请的第二方面的某些实施方式中，所述地图中包含实体对象信息及其位置信息；所述利用地图设置导航路线的步骤包括以下至少一种：基于包含实体对象信息的指令信息，设置导航至所述实体对象信息的导航路线；基于所述移动机器人的当前位置信息以及地图中的实体对象信息，设置所述移动机器人自当前位置移动并途径至少一个实体对象的导航路线。
在本申请的第二方面的某些实施方式中，所述移动机器人为清洁机器人；所述利用地图设置导航路线的步骤包括：基于所述地图中所标记的实体对象及其位置信息，设置遍历清扫区域中途径所述实体对象的导航路线。
为实现上述目的及其他相关目的，本申请的第三方面提供一种定位方法，用于包含摄像装置的移动机器人，其中，所述摄像装置在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像，所述移动机器人中预存储标记有实体对象的投影位置信息的地图，所述定位方法包括：分别在所述移动机器人行进至第一位置和第二位置处对应获取第一投影图像和第二投影图像；其中，所述第一投影图像和第二投影图像中均包含所述地图中所标记的同一实体对象所投影的目标区域；基于从所述第一投影图像和第二投影图像中所识别出的同一实体对象相对于移动机器人在第一位置及第二位置各自的角度信息、以及所述第一位置和第二位置，确定所述移动机器人在所述行进平面的定位信息，以便基于所述定位信息生成导航路线。
在本申请的第三方面的某些实施方式中，还包括基于预设的实体对象信息的种类，识别所述第一投影图像和第二投影图像中对应同一实体对象的目标区域的步骤，以及基于所识别出的实体对象信息的种类确定实体对象信息在所述地图中位置信息的步骤。
在本申请的第三方面的某些实施方式中，所述基于预设的实体对象信息的种类，识别第一投影图像和第二投影图像中对应同一实体对象的目标区域的步骤包括以下至少一种：基于预设置的多种实体对象的特征信息，识别第一投影图像和第二投影图像中对应同一实体对象的目标区域；利用预设的图像识别算法，将第一投影图像和第二投影图像中对应同一实体对象的目标区域与预设的多种实体对象信息进行映射处理。
在本申请的第三方面的某些实施方式中，还包括以下至少一步骤：基于在所述第一投影 图像中识别的第一目标区域，并利用图像追踪技术在所述第二投影图像中识别对应的第二目标区域，以得到两幅投影图像中对应同一实体对象的目标区域；匹配所述第一投影图像和第二投影图像中具有共同目标特征的目标区域，以得到两幅投影图像中对应同一实体对象的目标区域。
在本申请的第三方面的某些实施方式中，还包括在对应所述第一位置和第二位置处获取所述移动装置的姿态信息，并得到所述移动机器人自第一位置至第二位置的相对姿态变化信息；对应地，基于从所述第一投影图像和第二投影图像中所识别出的同一实体对象相对于移动机器人在第一位置及第二位置各自的角度信息、以及所述第一位置和第二位置，确定所述移动机器人在所述行进平面的定位信息的步骤包括：基于所述相对姿态变化信息抵消所识别出的同一实体对象相对于移动机器人在第一位置及第二位置各自的角度信息中的角度偏差；以及基于抵消后的角度信息和所述第一位置和第二位置确定所述移动机器人在所述行进平面中的定位信息。
在本申请的第三方面的某些实施方式中，所述确定所述移动机器人在所述行进平面中的定位信息的步骤包括：基于所述角度信息、所述第一位置和第二位置之间相对位置信息，构建在所述行进平面内的几何关系；利用所构建的几何关系确定所述移动机器人在所述行进平面中的定位信息。
在本申请的第三方面的某些实施方式中，在所述移动机器人从第一位置移动至第二位置期间还执行以下步骤：根据所述移动机器人的移动信息，测量从所述第一位置到移动机器人当前位置之间的相对位置信息；以及基于所述相对位置信息确定第二位置并获取对应的第二投影图像。
在本申请的第三方面的某些实施方式中，所述定位方法还包括以下步骤：对于所识别出的同一实体对象，和所述第一位置及所拍摄的第一投影图像，令移动机器人在多个第二位置拍摄对应的第二投影图像，并根据第一位置和多个第二位置各自拍摄的第一投影图像和第二投影图像分别执行前述确定所述移动机器人在所述第一位置的定位信息的步骤，以获得藉由每次执行上述步骤而得到的所述移动机器人的至少两个对应第一位置的定位信息；基于所得到的所述移动机器人的至少两个对应第一位置的定位信息，对所述移动机器人在第一位置时的定位位置进行误差修正处理，以便将误差修正后的定位位置确定为所述移动机器人在第一位置时的定位位置；基于所确定的所述移动机器人在第一位置时的定位位置，确定移动机器人当前的定位信息。
为实现上述目的及其他相关目的，本申请的第四方面提供一种导航方法，用于包含摄像 装置的移动机器人，包括：获取利用如上任一所述的构建地图的方法所构建的地图；以及利用如上任一所述的定位方法确定所述移动机器人当前的定位信息；利用所述地图和定位信息设置导航路线，以供所述移动机器人按照所述导航路线进行移动控制。
在本申请的第四方面的某些实施方式中，所述地图中包含实体对象信息及其位置信息；所述利用地图和定位信息设置导航路线的步骤包括以下至少一种：基于包含实体对象信息的指令信息，设置从所述定位信息导航至所述实体对象信息的导航路线；基于所述移动机器人的定位信息以及地图中的实体对象信息，设置所述移动机器人自定位信息移动并途径至少一个实体对象的导航路线。
在本申请的第四方面的某些实施方式中，所述移动机器人为清洁机器人；所述利用地图设置导航路线的步骤包括：基于所述地图中所标记的实体对象及其位置信息，设置遍历清扫区域中途径所述实体对象的导航路线。
为实现上述目的及其他相关目的，本申请的第五方面提供一种移动机器人的控制方法，用于包含摄像装置和移动装置的移动机器人，包括：获取如上任一所述的导航路线，并按照所述导航路线控制所述移动装置以调整所述移动机器人的位置和姿态，以沿所述导航路线进行自主移动。
为实现上述目的及其他相关目的，本申请的第六方面提供一种构建地图的系统，用于移动机器人，包括：摄像装置，设置于所述移动机器人，用于在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像；处理装置，与所述摄像装置相连，用于运行至少一程序，以执行如上任一所述的构建地图方法。
在本申请的第六方面的某些实施方式中，所述摄像装置的主光轴垂直于所述移动机器人的行进平面。
为实现上述目的及其他相关目的，本申请的第七方面提供一种导航系统，用于移动机器人，包括：摄像装置，设置于所述移动机器人，用于在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像；处理装置，与所述摄像装置相连，用于运行至少一程序，以执行如上任一所述的导航方法。
在本申请的第七方面的某些实施方式中，所述摄像装置的主光轴垂直于所述移动机器人的行进平面。
为实现上述目的及其他相关目的，本申请的第八方面提供一种移动机器人，包括：摄像装置，设置于所述移动机器人，用于在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像；移动装置，设置于所述移动机器人，用于受控地调整所述移动机器人的位置和姿态；第一处理装置，连接所述所述摄像装置，用于运 行至少一程序，以执行如上任一所述的导航方法；第二处理装置，连接于所述第一处理装置和移动装置，用于运行至少一程序，以基于所述第一处理装置所提供的导航路线，控制所述移动装置执行如上所述的控制方法。
在本申请的第八方面的某些实施方式中，所述摄像装置的主光轴垂直于所述移动机器人的行进平面。
为实现上述目的及其他相关目的，本申请的第九方面提供一种数据处理装置，用于包含摄像装置的移动机器人，其中，所述摄像装置在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像，其特征在于，包括：数据接口，用于连接所述摄像装置；存储单元，用于存储至少一程序；处理单元，与所述存储单元和数据接口相连，用于藉由所述数据接口获取所述摄像装置拍摄的图像；以及用于执行所述至少一程序以执行如上任一所述的构建地图的方法、或执行如上任一所述的导航方法、或者执行如上所述的控制方法。
为实现上述目的及其他相关目的，本申请的第十方面提供一种计算机可读的存储介质，存储至少一种程序，所述至少一种程序在被调用时执行如上任一所述的构建地图的方法、或执行如上任一所述的导航方法、或者执行如上所述的控制方法。
如上所述，本申请的构建地图、定位、导航、控制方法及系统、移动机器人，分别在所述移动机器人行进至第一位置和第二位置处对应获取关于移动机器人的行进平面的第一投影图像和第二投影图像；基于从所述第一投影图像和第二投影图像中所识别出的同一实体对象相对于移动机器人在第一位置及第二位置各自的角度信息、以及所述第一位置和第二位置，构建地图或对移动机器人进行定位，且根据构建的地图和定位的结果规划导航路线，控制该移动机器人沿所述导航路线移动。本申请通过两个位置处的两幅图像较为精准确定实体对象的位置信息，提高构建的地图的精确性，且根据较为精确的移动机器人的定位信息规划导航路线，以精准的控制移动机器人的移动，提高移动机器人运行的精度以及提高人机交互性。
附图说明
图1显示为移动机器人拍摄包含实体对象a的投影图像时，其与实体对象a在相应物理空间的示意图。
图2显示为本申请的构建地图的方法在一具体实施例中的流程示意图。
图3显示为基于角度信息以及第一位置和第二位置确定共同目标区域对应的实体对象的位置信息的原理示意图。
图4显示为基于角度信息以及第一位置和第二位置确定共同目标区域对应的实体对象的 位置信息的原理示意图。
图5显示为本申请的导航方法在一具体实施例中的流程示意图。
图6显示为本申请的导航方法在一具体实施例中的流程示意图。
图7显示为本申请的定位方法在一具体实施例中的流程示意图。
图8显示为本申请的移动机器人的控制方法在一具体实施例中的流程示意图。
图9显示为本申请的构建地图的系统在一具体实施例中的组成示意图。
图10显示为本申请的导航系统在一具体实施例中的组成示意图。
图11显示为本申请的移动机器人在一具体实施例中的组成示意图。
图12显示为本申请的数据处理装置在一具体实施例中的组成示意图。
具体实施方式
以下由特定的具体实施例说明本申请的实施方式，熟悉此技术的人士可由本说明书所揭露的内容轻易地了解本申请的其他优点及功效。
再者，如同在本文中所使用的，单数形式“一”、“一个”和“该”旨在也包括复数形式，除非上下文中有相反的指示。应当进一步理解，术语“包含”、“包括”表明存在所述的特征、步骤、操作、元件、组件、项目、种类、和/或组，但不排除一个或多个其他特征、步骤、操作、元件、组件、项目、种类、和/或组的存在、出现或添加。此处使用的术语“或”和“和/或”被解释为包括性的，或意味着任一个或任何组合。因此，“A、B或C”或者“A、B和/或C”意味着“以下任一个：A；B；C；A和B；A和C；B和C；A、B和C”。仅当元件、功能、步骤或操作的组合在某些方式下内在地互相排斥时，才会出现该定义的例外。
移动机器人基于导航控制技术执行移动操作。其中，受移动机器人所应用的场景影响，当移动机器人处于未知环境的未知位置时，利用VSLAM(Visual Simultaneous Localization and Mapping，基于视觉的即时定位与地图构建)技术可以帮助移动机器人构建地图并执行导航操作。具体地，移动机器人通过视觉传感器所提供的视觉信息以及移动传感器所提供的移动信息来构建地图，并根据所构建的地图为移动机器人提供导航能力，使得移动机器人能自主移动。其中，所述视觉传感器举例包括摄像装置。所述移动传感器举例包括速度传感器、里程计传感器、距离传感器、悬崖传感器等。所述移动机器人在行进平面进行移动，预先获取并存储关于所述行进平面的投影图像。所述摄像装置在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像。所述实体对象例如包含：电视机、空调、椅子、鞋子、皮球等。在现有的实际应用中，移动机器人结合移动传感器所提供的位置信息确定当前位置，以及通过识别摄像装置所摄取的单幅图像所包含的图像特征来 帮助定位当前位置，以便于移动机器人将其在当前位置所拍摄的图像特征，与所存储的相匹配图像特征在地图中的位置进行对应，由此实现快速定位。然而类似的定位方法，并不能准确定位移动机器人在地图中的位置。
为此，本申请提供一种构建地图的方法，应用于移动机器人中，通过该构建地图的方法可以精确地将在摄像装置视场范围内摄取的实体对象映射到移动机器人所使用的地图中，进而根据所定位的实体对象，生成以供移动机器人设置导航路线的地图。其中，所述实体对象包含移动机器人所移动的物理空间中任何可被拍摄的物理实体，其举例但不限于：球、鞋、墙壁、花盆、衣帽、屋顶、灯、树、桌子、椅子、冰箱、电视、沙发、袜、平铺物体、杯子等。其中，平铺物体包括但不限于平铺在地板上的地垫、地砖贴图，以及挂在墙壁上的挂毯、挂画等。所述摄像装置包括但不限于鱼眼摄像模块、广角(或非广角)摄像模块中的任一种。所述移动机器人包括但不限于：家庭陪伴式移动机器人、清洁机器人、巡逻式移动机器人、擦玻璃的机器人等。
在此，所述移动机器人包括至少一个摄像装置。所述摄像装置在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像。例如，移动机器人包含一个摄像装置，其设置于所述移动机器人顶部、肩部或背部，且主光轴垂直于所述移动机器人的行进平面。又如，移动机器人包含多个摄像装置，其中一个摄像装置的主光轴垂直于所述移动机器人的行进平面。
由于，所述摄像装置的主光轴垂直于行进平面，因此，摄像装置所摄取的二维图像所在平面与所述行进平面具有平行关系。请参阅图1，其显示为移动机器人拍摄包含实体对象a的投影图像时，其与实体对象a在相应物理空间的示意图。在摄像装置拍摄一幅投影图像时，所拍摄到的实体对象a投影到该投影图像M1中的位置D1与同一实体对象a投影到行进平面M2中的位置D2，其中位置D1和D2相对于移动机器人的位置D具有相同角度的特点。以此类推，我们用摄像装置所摄取的实体对象在投影图像中的位置来表示该实体对象投影至所述移动机器人的行进平面的位置，且利用所述实体对象在投影图像中的位置相对于所述移动机器人移动方向的角度来表征该实体对象投影至所述移动机器人的行进平面的位置相对于所述移动机器人移动方向的角度。
利用上述特点并请参阅图2，图2显示为本申请的构建地图的方法在一具体实施例中的流程示意图。其中，所述构建地图的方法可由移动机器人包括的处理装置来执行。其中，所述处理装置为一种能够进行数值运算、逻辑运算及数据分析的电子设备，其包括但不限于：CPU、GPU、FPGA等，以及用于暂存运算期间所产生的中间数据的易失性存储器等。
在步骤S210中，分别在所述移动机器人行进至第一位置和第二位置处对应获取第一投影 图像和第二投影图像。其中，处理装置所选取的两张投影图像应包含所拍摄到的同一实体对象。
在一些实施方式中，处理装置依据摄像装置的视场范围的直径，在小于所述直径的移动距离内的第一位置和第二位置获取对应的第一投影图像及第二投影图像。
例如，在移动机器人移动期间，处理装置可以预先设定摄像装置拍摄图像的时间间隔获取经摄像装置拍摄的不同时刻下的静态图像，以及拍摄静态图像时所对应的第一位置和第二位置；其中，所述时间间隔应至少小于移动机器人移动一个视场范围的直径所花费的时长。又如，摄像装置可以拍摄视频，由于视频是由图像帧构成的，在移动机器人移动期间，处理装置连续或不连续地采集所获取的视频中的图像帧，然后处理装置选用其中两帧图像作为两幅投影图像，并获取所对应的第一位置和第二位置，其中选取的两帧图像具有部分重叠的视场。
又如，所述摄像装置以预设的时间周期令所述移动机器人摄取其视场范围内的实体对象投影至所述移动机器人的行进平面的投影图像，然后处理装置获取经摄像装置以预设时间周期摄取的不同时刻下的投影图像，且选取其中两张投影图像作为第一投影图像和第二投影图像，且该第一投影图像所对应的位置为所述第一位置，所述第二投影图像所对应的位置为所述第二位置。其中，所述时间周期可由时间单位表示，或者所述时间周期由图像帧的间隔数量来表示。
再如，所述移动机器人与智能终端通信，所述智能终端可以通过特定的APP对所述时间周期进行修改。例如在打开所述APP后，在所述智能终端的触摸屏上显示所述时间周期的修改界面，通过对所述修改界面的触摸操作，完成对所述时间周期的修改；又或者直接向所述移动机器人发送时间周期修改指令以对所述时间周期进行修改，所述时间周期修改指令，例如为包括修改指令的语音，所述语音例如为“周期修改为三秒”。又如，所述语音为“图像帧间隔修改为五幅”。
在又一些实施方式中，处理装置确定选取第一位置和第二位置处的第一投影图像和第二投影图像的依据是两幅投影图像中包含共同目标区域。在此，所述处理装置在执行步骤S210的同时，或者无必然执行顺序地，还执行以下步骤S211和S212中的至少一种。
在步骤S211中，基于在所述第一投影图像中识别的第一目标区域，并利用图像追踪技术在所述第二投影图像中识别对应的第二目标区域，以得到两幅投影图像中对应同一实体对象的目标区域。由此获取第二投影图像及其所对应的第二位置。
在此，处理装置利用图像处理算法得到的第一投影图像中的图像特征，并基于图像特征确定第一目标区域，利用图像追踪技术追踪自第一投影图像之后摄像装置所摄取的至少一幅 投影图像，并在经持续追踪而确定图像特征在第二位置所拍摄的第二投影图像中仍然存在，则确定获取第一位置和第二位置各自所拍摄的第一投影图像和第二投影图像，以及确定所追踪的图像特征在为同一实体对象。其中，所述图像特征包括预设的对应实体对象种类的图形特征，或者经图像处理算法而得到的图像特征。其中，所述图像处理算法包括但不限于以下至少一种：灰度处理、锐化处理、轮廓提取、角提取、线提取，利用经机器学习而得到的图像处理算法。利用经机器学习而得到的图像处理算法包括但不限于：神经网络算法、聚类算法等。
在步骤S212中，匹配所述第一投影图像和第二投影图像中具有共同目标特征的目标区域，以得到两幅投影图像中对应同一实体对象的目标区域。
在一些示例中，处理装置分别在第一位置和第二位置获取两幅投影图像，并分别识别图像特征，并匹配两幅投影图像中的图像特征，由此将相匹配的图像特征所在目标区域作为具有共同目标特征的目标区域，并确定具有共同目标特征的目标区域对应同一实体对象。由此，处理装置将分别在第一位置和第二位置所获取的两幅投影图像作为第一投影图像和第二投影图像，并确定对应的第一位置和第二位置。
在另一些示例中，处理装置借助于移动机器人从第一位置移动至第二位置所改变的方向，将在第一投影图像中所识别出的图像特征映射到第二投影图像中对应图像区域，并在该图像区域附近查找相配的图像特征，由此将相匹配的图像特征所在目标区域作为具有共同目标特征的目标区域，并确定具有共同目标特征的目标区域对应同一实体对象。由此，处理装置将确定所述第一投影图像和第二投影图像，以及确定对应的第一位置和第二位置。
其中，表示共同目标区域的图像特征包括以下至少一种：特征点、特征线、特征颜色、基于物体识别模型而在图像中所标记的对应共同目标区域内的关键区域上的图像特征等。其中，所述特征点包括与所述实体对象对应的角点、端点、拐点等。确定实体对象上各特征点所对应的物体局部位置分别与移动机器人的相对空间位置。所述特征线例如为标识所述实体对象的轮廓线。即令所述移动机器人将所述第一投影图像和所述第二投影图像根据所述预设的多种实体对象的特征信息进行比对，且所述第一投影图像和所述第二投影图像中均与一实体对象的图像特征匹配的区域为所述共同目标区域。所述物体识别模型举例包括基于神经网络而构建的物体识别算法模型、用于识别物体的聚类算法模型等。当利用物体识别模型而识别两幅投影图像中的共同目标区域时，可在两幅投影图像中使用相同颜色的矩形框(或圆角矩形框、圆形等)来表示所识别出的共同目标区域，所对应的两个矩形框的边沿区域、中心区域等即为所述关键区域，并提取相应区域内的图像特征。
需要说明的是，上述任一示例所得到的实体对象并非一定能够被处理装置所识别，即处 理装置能够确定实体对象所属种类。例如，处理装置并非一定能够确定在第一投影图像和第二投影图像中相匹配的共同目标区域所对应的实体对象为桌子。
在一些实际应用中，所述构建地图的方法还包括：基于预设的实体对象信息的种类，识别第一投影图像和第二投影图像中对应同一实体对象的目标区域。其中，预设的实体对象信息的种类是指所识别的实体对象所属的种类。例如，饭桌、写字台均归类于桌子种类，又如，电脑椅和板凳均归类于椅子种类等。其中，所述实体目标信息的种类是基于移动机器人所移动的环境情况预先筛选并存储在移动机器人的存储装置中的。所述实体目标信息的种类由可能被摄取的、且摆放在所述环境中的实体目标的特征信息或实体目标在图像中的图像特征来表征。
在一些示例中，所述实体目标信息的种类可以由物体的图像特征来表征，所述图像特征能够标识图像中的实体对象。所述处理装置基于预设置的多种实体对象的特征信息，识别第一投影图像和第二投影图像中对应同一实体对象的目标区域。例如，利用预设的各种类的图像特征中遍历在第一投影图像和第二投影图像中对应同一实体对象的目标区域，以确定所识别的实体对象归属的种类。
在又一些示例中，所述实体目标信息的种类可以由物体分类来表征。所述处理装置利用预设的图像识别算法，将第一投影图像和第二投影图像中对应同一实体对象的目标区域与预设的多种实体对象信息进行映射处理。例如，存储装置所存储的程序包含神经网络模型的网络结构及连接方式。在某些实施例中，所述神经网络模型可以为卷积神经网络，所述网络结构包括输入层、至少一层隐藏层和至少一层输出层。其中，所述输入层用于接收所拍摄的图像或者经预处理后的图像；所述隐藏层包含卷积层和激活函数层，甚至还可以包含归一化层、池化层、融合层中的至少一种等；所述输出层用于输出标记有物体种类标签的图像。所述连接方式根据各层在神经网络模型中的连接关系而确定。例如，基于数据传输而设置的前后层连接关系，基于各隐藏层中卷积核尺寸而设置与前层数据的连接关系，以及全连接等。所述神经网络模型从图像中识别出的各物体分类。
另外，为了确定第一位置和第二位置，所述处理装置还在执行步骤S210期间执行步骤S213和S214。
在步骤S213中，根据所述移动机器人的移动信息，测量从所述第一位置至移动机器人当前位置之间的相对位置信息。
在步骤S214中，基于所测量的相对位置信息确定第二位置并获取对应的第二投影图像。
在此，受移动机器人的行进平面的材质影响，为提高第一位置和第二位置的测量准确性，在移动机器人移动期间，利用移动机器人的位置测量装置，处理装置监测移动机器人从所述 第一位置到移动机器人当前位置之间的相对位移及相对偏角，并当判断所述相对位移和/或相对偏角超出预设测量阈值时，确定第二位置并获取对应该第二位置的第二投影图像。其中，所述测量阈值可以一固定值，或相对值。例如，所述测量阈值为5cm。又如，所述测量阈值为移动机器人的滚轮圈数的n倍，n>1。
在一些实施例中，利用所述移动机器人的位置测量装置获取所述移动机器人的移动信息。所述位置测量装置包括但不限于均设置在移动机器人的位移传感器、测距传感器、角度传感器、双目摄像装置、速度传感器等。在此，处理装置可仅监测位置测量装置中的一种来监测移动机器人从所述第一位置到移动机器人当前位置之间的相对位移及相对偏角。例如，所述位置测量装置为设置于所述移动机器人的马达的计数传感器，利用马达运转的圈数进行计数以获得所述移动机器人自第一位置移动至第二位置的相对位移，以及利用马达运转的角度获取姿态信息等。又如，所述位置测量装置测量移动机器人的行进速度，并利用行进速度及所行进的时长计算自第一位置移动至第二位置的相对位移。或者，结合多种传感器和摄像装置等所提供的位置数据来监测移动机器人从所述第一位置到移动机器人当前位置之间的相对位移及相对偏角。例如，利用预设权重并结合双目摄像装置所获取的移动机器人相对于固定目标的多组测距数据，以及马达运转所测量的数据，来确定移动机器人从所述第一位置到移动机器人当前位置之间的相对位移及相对偏角。
在另一些实施例中，所述位置测量装置为GPS(Global Positioning System，全球定位系统)，根据该GPS监测移动机器人从所述第一位置到移动机器人当前位置之间的矢量位移及矢量偏角；并当判断所述相矢量位移和/或矢量偏角超出预设测量阈值时，确定第二位置并获取对应该第二位置的第二投影图像，以及获取所述第一位置和第二位置之间的相对位置信息。在又一些实施例中，所述位置测量装置为基于测量无线信号而定位的装置，例如，所述位置测量装置为蓝牙(或WiFi)定位装置；位置测量装置根据在第一位置和第二位置各自对所接收的无线定位信号的功率进行测量，来确定各位置相对于预设无线定位信号发射装置的相对位置；利用该位置测量装置监测移动机器人从所述第一位置到移动机器人当前位置之间的相对位移及相对偏角；并当判断所述相对位移和/或相对偏角超出预设测量阈值时，确定第二位置并获取对应该第二位置的第二投影图像，以及获取所述第一位置和第二位置之间的相对位置信息。
在另一些实施例中，以所述地图为一种栅格地图为例，预先确定单位栅格长度与实际位移之间的映射关系，按照移动机器人在移动期间所得到的移动信息，监测移动机器人从所述第一位置到移动机器人当前位置之间的栅格数及栅格偏角；并当判断所述栅格数和/或栅格偏角超出预设测量阈值时，确定第二位置并获取对应该第二位置的第二投影图像，以及获取所 述第一位置和第二位置之间的相对位置信息。
在步骤S220中，基于从所述第一投影图像和第二投影图像中所识别出的同一实体对象相对于移动机器人在第一位置及第二位置各自的角度信息、以及所述第一位置和第二位置，确定所述实体对象在所述行进平面中的投影位置信息。
参见图1可知，由于所获取的投影图像所在平面是与移动机器人的行进平面平行的，因此，在基于移动机器人的行进平面(或者基于投影图像所在平面)而构建的坐标系中，同一实体对象在投影图像中目标区域与坐标基准线之间的偏角，相当于该实体对象投影在所述行进平面的位置相对于所述坐标基准线的偏角。其中，所述坐标基准线为所述坐标系所在平面内的直线。例如，所述坐标基准线为移动机器人的前进方向所在直线。换言之，当处理装置通过投影图像确定对应实体对象的目标区域与坐标基准线之间的角度时，意味着可以确定该实体对象位于与移动机器人行进方向偏移所述角度的方向上。
因此，当移动机器人在行进平面移动期间，将所述移动机器人所在不同位置以及在各位置处所拍摄到的同一实体对象在投影图像中的目标区域，均映射到所述坐标系中，可基于所得到的各角度和各位置而构建几何关系，利用所构建的几何关系确定所述实体对象在所述行进平面中的投影位置信息。其中，所述几何关系包括但不限于：三角形关系、四边形关系等。
需要说明的是所述坐标系用于帮助理解，在实际地图构建期间，并非一定绘制或显示所述坐标系。
在一些实施方式中，所述步骤S220包括：基于所述角度信息、所述第一位置和第二位置，构建在所述行进平面内的几何关系；以及利用所构建的几何关系确定所述实体对象在所述行进平面中的投影位置信息。
请参阅图3，图3显示为基于角度信息以及第一位置和第二位置确定实体对象位置信息的原理示意图。以移动机器人沿直线从第一位置P1移动至第二位置P2为例，利用上述坐标系，处理装置在所述第一位置P1获取如图3所示的第一投影图像，所述处理装置在所述第二位置P2获取如图3所示的第二投影图像，所述第一位置P1和所述第二位置P2间形成连线P1P2；以及，所述处理装置还确定同一实体对象分别在第一投影图像和所述第二投影图像中的目标区域O1和O2；对于同一实体对象来说，其应位于所述目标区域O1与第一位置P1的连线P1O1上，以及位于所述目标区域O2与第二位置P2的连线P2O2上，换言之，实体对象位于线段P1O1和P2O2相交的位置O，即该位置O为该实体对象投影到行进平面的投影位置。其中，处理装置根据所述连线P1O1与所述连线P1P2之间的夹角α1、连线P2O2与所述连线P1P2之间的夹角β1、以及连线P1P2之间的距离，确定点P1、P2、O所构成的三角形中点O的位置，进而得到所述实体对象在所述行进平面中的投影位置信息。
以移动机器人从第一位置移动至第二位置时姿态发生变化为例，所获取的第一位置和第二位置中包含位移信息和姿态信息。其中，根据第一位置和第二位置的姿态信息，可得到相对姿态变化信息。其中，所述相对姿态变化信息包括在行进平面内所述移动机器人在第二位置相对于所述移动机器人在第一位置所转动的角度。对应地，所述步骤S120包括：基于所述相对姿态变化信息抵消所识别出的同一实体对象相对于移动机器人在第一位置及第二位置各自的角度信息中的角度偏差；以及基于抵消后的角度信息和所述第一位置和第二位置确定所述实体对象在所述行进平面中的投影位置信息。
参阅图4所示，图4显示为基于角度信息以及第一位置和第二位置确定共同目标区域对应的实体对象的位置信息的原理示意图。所述移动机器人在第一位置P1’处向Y轴的正方向的行进方向移动，以及在第二位置P2’处沿向Y轴正方向偏左γ角度Y’轴的正方向移动，由此，处理装置可确定移动机器人在第一位置和第二位置之间的相对姿态变化信息为夹角γ。其中，夹角γ可由移动机器人的位置测量装置中的角度传感器提供。
仍参见图4，处理装置根据在所述第一位置P1’处摄取的第一投影图像中目标区域在坐标系中的位置，确定该目标区域所对应的实体对象位于连线P1’O1’所在直线上，其与所述Y轴的正方向的夹角α2，根据在所述第二位置P2’处摄取的第二投影图像中目标区域在坐标系中的位置，确定该目标区域所对应的实体对象位于连线P2’O2’所在直线上，其与所述Y’轴的正方向的夹角β2，由于Y’轴的正方向与Y轴的正方向之间夹角γ，因此，所述处理装置在确定实体对象相距第二位置的连线时，需抵消夹角γ所产生的角度偏差，即将所述夹角β2减去所述夹角γ后的补角为夹角β2’。即可根据所述连线P1’P2’的距离、所述夹角α2和所述夹角β2’所构建的三角关系，确定所述实体对象的投影位置信息。
需要说明的是，上述任一示例中均未考虑实体对象在所述行进平面的垂直方向上的高度。在实际应用中，利用移动机器人的高度测量装置，可在定位实体对象的位置时测量其高度信息。其应视为基于本申请所提供的地图构建方案而拓展的一种实施例。在此不予详述。
在步骤S230中，将所确定的实体对象的投影位置信息标记到预设的地图中。在此，处理装置根据地图与物理空间的比例关系，按照第一位置、第二位置和所得到的实体对象在行进平面中的投影位置信息，得到各自映射到地图中的位置信息，即将实体对象的投影位置信息标记到地图中。其中，在一些实施方式中，为表示所标记位置的实体对象，所述处理装置可将实体对象的图像特征标记在地图的对应位置处，以便当移动机器人再次拍摄到与此图像特征相匹配的目标区域时，利用上述方式，确定移动机器人在地图中的位置。
在另一些实施方式中，根据所识别出的实体对象信息，将所识别出的实体目标信息标记在所述地图中。例如可以将对应各所述实体目标信息的文字描述、图像标识、或者编号标记 于所述地图中，所述文字描述可以为对各实体目标信息的种类的名称描述，例如包括对桌子、椅子、花盆、电视以及冰箱等物体的名称描述。例如对应所述桌子的名称描述为“桌子”，对应所述电视的名称描述为“电视”。所述图像标识可以为对应各实体目标信息的种类的实际形象的图标。所述编号可以为预先对应各所述实体目标信息进行编排的数字标号。例如“001”代表冰箱，“002”代表椅子，“003”代表桌子等。
在又一些实施方式中，所标记的实体对象的投影位置信息是被不断纠正的，或经纠正后被标记到地图中的。为此，在所述移动机器人移动期间，对于所识别出的同一实体对象，多次执行前述步骤S210-S220中所述的任一示例，以获得藉由每次执行上述步骤而得到对应同一实体对象的至少两个投影位置信息。受移动机器人所行进平面的材料影响，移动机器人移动期间历次获取的第一位置信息、第二位置信息无法精准确定。例如，当移动机器人在地毯上移动时，其真实移动的距离小于其通过位置测量装置所测量的距离。又如，当移动机器人在地板上移动时，其真实移动的距离接近其通过位置测量装置所测量的距离。因此，在多次执行步骤S210-S220期间，藉由第一位置信息、第二位置信息、角度信息等而计算得到的同一实体对象的投影位置信息将会有差异。因此，本申请所提供的构建地图的方法可藉由移动机器人在移动期间的第一位置、第二位置、第三位置等多个位置所构成的至少两个三角形对同一个实体对象的投影位置信息进行确定，并利用以下步骤进行误差修正，即基于所得到的对应同一实体对象的至少两个投影位置信息，对所述实体对象的投影位置进行误差修正处理的步骤。
在此，所述误差修正的方式包括但不限于：1)采用加权平均的方式对所得到的各投影位置信息进行误差修正处理，以将误差修正后的投影位置信息标记到预设地图中。例如，将所得到的同一实体对象的投影位置信息中横纵坐标进行加权平均，得到误差修正后的投影位置信息并标记到地图中。2)采用数值优化的方式将所得到的各投影位置信息进行迭代处理，以得到经迭代处理后估计的投影位置信息，并将所估计的投影位置信息标记到预设地图中。以一种实时标记实体对象的位置的方式举例，预设所识别出的实体对象的投影位置信息的初始值基础上，利用第一次得到的对应该实体对象的投影位置信息与所述初始值所构成的梯度，进行第二次对应该实体对象的投影位置信息的确定，通过预设条件的迭代循环，得到所估计的投影位置信息，，并将所估计的投影位置信息及所对应的实体对象的图像特征、类型等信息标记在地图中；其中，所述初始值可以随机设置，或者依据前述步骤S210-S220所提供的方式得到。在此，第N次依据前述步骤S210-S220得到投影位置信息的方式包括利用移动机器人已经测得的至少一个位置信息及所对应的角度信息、及新测得的位置信息及所对应的角度信息，来确定第N次投影位置信息；或者利用全部新测得的两个位置信息及所对应的角度信 息，来确定第N次投影位置信息；或者从任意不同两次测得的两个位置信息及所对应的角度信息，来确定第N次投影位置信息；其中，N>1。以一种非实时标记实体对象位置的方式举例，利用步骤S210-S220所示方式收集对应同一实体对象的多个投影位置信息，并将各投影位置信息代入包含梯度的数值优化算法，以筛选或估计出误差最小的投影位置信息，并将所得到的误差最小的投影位置信息及所对应的实体对象的图像特征、类型等信息标记在地图中。
本申请的构建地图的方法根据所述摄像装置摄取所述移动机器人所在位置的视场范围内的实体对象投影至所述移动机器人的行进平面的投影图像，以分别获得所述移动机器人在第一位置处的第一投影图像和第二位置处的第二投影图像；且基于自所述第一投影图像和第二投影图像中所识别出的共同目标区域在所述行进平面中的角度信息、以及所述第一位置和第二位置，确定所述共同目标区域所对应的实体对象在所述移动机器人所行进的物理空间中的位置信息；以得到较为精确的实体对象的位置信息，且将所确定的实体对象的位置信息标记到预设的地图中，从而提高该地图的精确性。
请参阅图5，图5显示为本申请的导航方法在一具体实施例中的流程示意图。所述导航方法应用于所述包含摄像装置的移动机器人，所述摄像装置在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像，所述导航方法主要由移动机器人来执行。
在步骤S510中，获取利用构建地图的方法所构建的地图；所述构建地图的方法包括：分别在所述移动机器人行进至第一位置和第二位置处对应获取第一投影图像和第二投影图像；基于自所述第一投影图像和第二投影图像中所识别出的共同目标区域在所述行进平面中的角度信息、以及所述第一位置和第二位置，确定所述共同目标区域所对应的实体对象在所述移动机器人所行进的物理空间中的位置信息；将所确定的实体对象的位置信息标记到预设的地图中。所述构建地图的方法可参阅图2及关于图2所示方法的相关描述。
在步骤S520中，利用所述地图设置导航路线，以供所述移动机器人按照所述导航路线进行移动控制。
在此，根据移动机器人的实际应用，利用所述地图可设置适应于移动目的的导航路线。例如，对于安防机器人来说，在检查地图中所标记的实际对象期间，其设置的导航路线需途径每个实际对象。又如，对于家庭陪伴机器人来说，通过远程操控而执行电器控制期间，其设置的导航路线可以所标记的实际对象为目的地址。再如，对于清洁机器人来说，根据在预设的清扫区域内的实际对象的位置，设置对应于清洁模式的导航路线。
在一些具体实施例中，所述地图中包含实体对象的实体对象信息及其位置信息；所述步骤S520还包括步骤S521。
所述步骤S521包括：基于包含实体对象信息的指令信息，设置导航至所述实体对象信息的导航路线；于本实施例中，所述实体对象信息例如为对各实体对象信息的种类的名称描述，例如包括对桌子、椅子、花盆、电视以及冰箱等物体的名称描述。
其中，获取包含实体对象信息的指令信息的方式包括但不限于：语音方式、文本方式等。在此，根据用户对移动机器人的操作目的，所述指令中还可以包含移动机器人的执行指令。例如，所述指令还包括清扫指令、巡视指令、远程操控指令等。
在一种实施例中，所述步骤S521可以包括：获取一语音信息，并从所述语音信息中识别包含实体对象信息的指令。在一示例中，移动机器人可以直接接收用户的语音信息并识别所述信息中包括的实体对象信息的指令。例如，用户可以直接向移动机器人语音“桌子”，移动机器人在接收到所述指令后移动至桌子以进行预先设定的相应处理。且可以根据路线上经过的实体对象信息规划所述移动机器人从当前的位置移动至桌子的导航路线。所述移动机器人从当前位置移动到所述桌子的导航路线上可经过花盆、电视、沙发等。以清洁机器人为例，预先设定清洁机器人在接收到用户包含实体对象信息的指令后，根据所构建的地图规划导航路线，以令所述移动机器人移动至该实体对象信息对应的位置以进行清扫，则在用户向清洁机器人语音“桌子”的情况下，清洁机器人在接收到该语音指令后，根据所构建的地图，形成令所根据花盆、电视和沙发形成的导航路线，且所述清洁机器人在经过根据所述花盆、电视和沙发形成的导航路线后移动至桌子处并执行清扫操作。另外，所述语音信息并不限于仅表示实体对象信息的短指令，还可以是包括实体对象信息的长指令，例如，用户语音“去桌子处”，则移动机器人可以识别语音信息中所包括的实体对象信息“桌子”指令，然后进行后续操作。
在另一种实施例中，所述步骤S521还包括：自一终端设备获取包含实体对象信息的指令。其中，所述终端设备与移动机器人无线连接。在一示例中，用户经由终端设备以文本方式输入包含实体对象信息的指令。例如，用户通过手机APP以文本方式输入“桌子”。在另一示例中，用于经由终端设备以语音方式输入包含实体对象信息的指令。例如，用户通过手机APP以语音方式输入“桌子”。另外，用户输入的语音信息并不限于仅表示实体对象信息的短指令，还可以是包括实体对象信息的长指令，例如，用户语音“去桌子处”，则终端设备将其翻译成文本并提取其中桌子等关键词，将所翻译的文本匹配到相应的指令上发送给移动机器人。在此，终端设备可以与移动机器人以wifi连接、近场通信或蓝牙配对等无线方式连接，以将终端设备接收的指令传送给移动机器人进行后续操作。所述终端设备例如为智能手机、平板电脑、可穿戴设备或其他具有智能处理功能的智能设备。
在另一些具体实施例中，所述步骤S520还包括步骤S522。所述步骤S522包括：基于所述移动机器人的当前位置信息以及地图中的实体对象信息，设置所述移动机器人自当前位置 移动并途径至少一个实体对象的导航路线。在某些实施例中，移动机器人根据地图上移动机器人的当前位置、实体对象信息所在的目的位置、以及地图上其他物体的位置信息或预设路径等生成自当前位置信息至目的位置信息的导航路线。例如，所述导航路线可以是自移动机器人的当前位置至目的位置的最短路线。又如，所述导航路线可以是自移动机器人的当前位置至目的位置的与移动机器人先前行进路径不重复的路线，以移动机器人为清洁机器人为例，根据未清洁区域和已清洁区域设置自当前位置至目的位置的无需重复清洁已清洁区域的导航路线，由此提高清洁效率。
在一些应用中，所述移动机器人为清洁机器人；所述步骤S520还包括：基于所述地图中所标记的实体对象及其位置信息，设置遍历清扫区域中途径所述实体对象的导航路线。在一些示例中，清洁机器人基于预先确定的清扫区域设计遍历该清扫区域的导航路线，其中，根据地图中位于所述清扫区域内的实体对象的标记信息，清洁机器人根据相应的标记信息确定便于清洁的导航路线。其中，所述清扫区域包括但不限于以下至少一种：按照预设栅格数量划分的清扫区域、以及按照房间分割的清扫区域等。例如，在所获取的地图中的一清扫区域内，标记有桌子及其位置信息，故，在设计导航路线时设计包含围绕桌腿旋转的导航路线。
本申请的导航方法，通过识别与预先标记的实体对象信息相符的实体对象并基于摄像装置所摄取的图像确定所识别的实体对象与移动机器人之间的相对空间位置，以在依据图2所述方法所构建的地图中将所述实体对象信息进行标记，以生成标记有所述实体对象信息的所述地图，以便在后续移动控制中可以基于地图上标记的实体对象信息来识别用户指令中的目的位置，进而移动至该位置处，提高了移动机器人的导航路线的精确性，且提高人机交互性。
参阅图6，图6显示为本申请的导航方法在一具体实施例中的流程示意图。所述导航方法应用于所述包含摄像装置的移动机器人，所述摄像装置在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像，所述导航方法主要由移动机器人来执行。
在步骤S610中，获取利用构建地图的方法所构建的地图；且利用定位方法确定所述移动机器人当前的定位信息。
其中，所述构建地图的方法包括：分别在所述移动机器人行进至第一位置和第二位置处对应获取第一投影图像和第二投影图像；基于自所述第一投影图像和第二投影图像中所识别出的共同目标区域在所述行进平面中的角度信息、以及所述第一位置和第二位置，确定所述共同目标区域所对应的实体对象在所述移动机器人所行进的物理空间中的位置信息；将所确定的实体对象的位置信息标记到预设的地图中。所述构建地图的方法可参阅图2及关于图2所示方法的相关描述。
本申请还提供一种定位方法。在一些应用中，当移动机器人被放置在未知位置时，一种利用移动机器人中的摄像装置，以及移动一段距离所得到的相对位置信息，来进行自我定位的方式，可参见图7所示，图7显示为本申请的定位方法在一具体实施例中的流程示意图。其中，所述摄像装置在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像，所述摄像装置的主光轴垂直于行进平面，所述定位方法可由移动机器人包括的处理装置来执行。其中，所述处理装置为一种能够进行数值运算、逻辑运算及数据分析的电子设备，其包括但不限于：CPU、GPU、FPGA等，以及用于暂存运算期间所产生的中间数据的易失性存储器等。所述定位方法包括：
步骤S710：预生成利用构建地图的方法所构建的地图；其中，所述构建地图的方法可参阅图2及关于图2所示方法的相关描述。
步骤S720：分别在所述移动机器人行进至第一位置和第二位置处利用所述摄像装置对应获取第一投影图像和第二投影图像；其中，所述第一投影图像和第二投影图像中均包含所述地图中所标记的同一实体对象所投影的目标区域；
在一些实施方式中，处理装置依据摄像装置的视场范围的直径，在小于所述直径的移动距离内的第一位置和第二位置获取对应的第一投影图像及第二投影图像。
例如，在移动机器人移动期间，处理装置可以预先设定摄像装置拍摄图像的时间间隔获取经摄像装置拍摄的不同时刻下的静态图像，以及拍摄静态图像时所对应的第一位置和第二位置；其中，所述时间间隔应至少小于移动机器人移动一个视场范围的直径所花费的时长。又如，摄像装置可以拍摄视频，由于视频是由图像帧构成的，在移动机器人移动期间，处理装置连续或不连续地采集所获取的视频中的图像帧，然后处理装置选用其中两帧图像作为两幅投影图像，并获取所对应的第一位置和第二位置，其中选取的两帧图像具有部分重叠的视场。
又如，所述摄像装置以预设的时间周期令所述移动机器人摄取其视场范围内的实体对象投影至所述移动机器人的行进平面的投影图像，然后处理装置获取经摄像装置以预设时间周期摄取的不同时刻下的投影图像，且选取其中两张投影图像作为第一投影图像和第二投影图像，且该第一投影图像所对应的位置为所述第一位置，所述第二投影图像所对应的位置为所述第二位置。其中，所述时间周期可由时间单位表示，或者所述时间周期由图像帧的间隔数量来表示。
再如，所述移动机器人与智能终端通信，所述智能终端可以通过特定的APP对所述时间周期进行修改。例如在打开所述APP后，在所述智能终端的触摸屏上显示所述时间周期的修改界面，通过对所述修改界面的触摸操作，完成对所述时间周期的修改；又或者直接向所述 移动机器人发送时间周期修改指令以对所述时间周期进行修改，所述时间周期修改指令，例如为包括修改指令的语音，所述语音例如为“周期修改为三秒”。又如，所述语音为“图像帧间隔修改为五幅”。
在又一些实施方式中，处理装置确定选取第一位置和第二位置处的第一投影图像和第二投影图像的依据是两幅投影图像中包含共同目标区域。在此，所述处理装置在执行步骤S720的同时，或者无必然执行顺序地，还执行以下步骤S721和S722中的至少一种。
在步骤S721中，基于在所述第一投影图像中识别的第一目标区域，并利用图像追踪技术在所述第二投影图像中识别对应的第二目标区域，以得到两幅投影图像中对应同一实体对象的目标区域。由此获取第二投影图像及其所对应的第二位置。
在此，处理装置利用图像处理算法得到的第一投影图像中的图像特征，并基于图像特征确定第一目标区域，利用图像追踪技术追踪自第一投影图像之后摄像装置所摄取的至少一幅投影图像，并在经持续追踪而确定图像特征在第二位置所拍摄的第二投影图像中仍然存在，则确定获取第一位置和第二位置各自所拍摄的第一投影图像和第二投影图像，以及确定所追踪的图像特征在为同一实体对象。其中，所述图像特征包括预设的对应实体对象种类的图形特征，或者经图像处理算法而得到的图像特征。其中，所述图像处理算法包括但不限于以下至少一种：灰度处理、锐化处理、轮廓提取、角提取、线提取，利用经机器学习而得到的图像处理算法。利用经机器学习而得到的图像处理算法包括但不限于：神经网络算法、聚类算法等。
在步骤S722中，匹配所述第一投影图像和第二投影图像中具有共同目标特征的目标区域，以得到两幅投影图像中对应同一实体对象的目标区域。
在一些示例中，处理装置分别在第一位置和第二位置获取两幅投影图像，并分别识别图像特征，并匹配两幅投影图像中的图像特征，由此将相匹配的图像特征所在目标区域作为具有共同目标特征的目标区域，并确定具有共同目标特征的目标区域对应同一实体对象。由此，处理装置将分别在第一位置和第二位置所获取的两幅投影图像作为第一投影图像和第二投影图像，并确定对应的第一位置和第二位置。
在另一些示例中，处理装置借助于移动机器人从第一位置移动至第二位置所改变的方向，将在第一投影图像中所识别出的图像特征映射到第二投影图像中对应图像区域，并在该图像区域附近查找相配的图像特征，由此将相匹配的图像特征所在目标区域作为具有共同目标特征的目标区域，并确定具有共同目标特征的目标区域对应同一实体对象。由此，处理装置将确定所述第一投影图像和第二投影图像，以及确定对应的第一位置和第二位置。
其中，表示共同目标区域的图像特征包括以下至少一种：特征点、特征线、特征颜色、 基于物体识别模型而在图像中所标记的对应共同目标区域内的关键区域上的图像特征等。其中，所述特征点包括与所述实体对象对应的角点、端点、拐点等。确定实体对象上各特征点所对应的物体局部位置分别与移动机器人的相对空间位置。所述特征线例如为标识所述实体对象的轮廓线。即令所述移动机器人将所述第一投影图像和所述第二投影图像根据所述预设的多种实体对象的特征信息进行比对，且所述第一投影图像和所述第二投影图像中均与一实体对象的图像特征匹配的区域为所述共同目标区域。所述物体识别模型举例包括基于神经网络而构建的物体识别算法模型、用于识别物体的聚类算法模型等。当利用物体识别模型而识别两幅投影图像中的共同目标区域时，可在两幅投影图像中使用相同颜色的矩形框(或圆角矩形框、圆形等)来表示所识别出的共同目标区域，所对应的两个矩形框的边沿区域、中心区域等即为所述关键区域，并提取相应区域内的图像特征。
需要说明的是，上述任一示例所得到的实体对象并非一定能够被处理装置所识别，即处理装置能够确定实体对象所属种类。例如，处理装置并非一定能够确定在第一投影图像和第二投影图像中相匹配的共同目标区域所对应的实体对象为桌子。
在一些实际应用中，所述定位方法还包括：基于预设的实体对象信息的种类，识别第一投影图像和第二投影图像中对应同一实体对象的目标区域。其中，预设的实体对象信息的种类是指所识别的实体对象所属的种类。例如，饭桌、写字台均归类于桌子种类，又如，电脑椅和板凳均归类于椅子种类等。其中，所述实体目标信息的种类是基于移动机器人所移动的环境情况预先筛选并存储在移动机器人的存储装置中的。所述实体目标信息的种类由可能被摄取的、且摆放在所述环境中的实体目标的特征信息或实体目标在图像中的图像特征来表征。
在一些示例中，所述实体目标信息的种类可以由物体的图像特征来表征，所述图像特征能够标识图像中的实体对象。所述处理装置基于预设置的多种实体对象的特征信息，识别第一投影图像和第二投影图像中对应同一实体对象的目标区域。例如，利用预设的各种类的图像特征中遍历在第一投影图像和第二投影图像中对应同一实体对象的目标区域，以确定所识别的实体对象归属的种类。
在又一些示例中，所述实体目标信息的种类可以由物体分类来表征。所述处理装置利用预设的图像识别算法，将第一投影图像和第二投影图像中对应同一实体对象的目标区域与预设的多种实体对象信息进行映射处理。例如，存储装置所存储的程序包含神经网络模型的网络结构及连接方式。在某些实施例中，所述神经网络模型可以为卷积神经网络，所述网络结构包括输入层、至少一层隐藏层和至少一层输出层。其中，所述输入层用于接收所拍摄的图像或者经预处理后的图像；所述隐藏层包含卷积层和激活函数层，甚至还可以包含归一化层、池化层、融合层中的至少一种等；所述输出层用于输出标记有物体种类标签的图像。所述连 接方式根据各层在神经网络模型中的连接关系而确定。例如，基于数据传输而设置的前后层连接关系，基于各隐藏层中卷积核尺寸而设置与前层数据的连接关系，以及全连接等。所述神经网络模型从图像中识别出的各物体分类。
另外，为了确定第一位置和第二位置，所述处理装置还在执行步骤S730期间执行步骤S731和步骤S732。
在步骤S731中，根据所述移动机器人的移动信息，测量从所述第一位置至移动机器人当前位置之间的相对位置信息。
在步骤S732中，基于所测量的相对位置信息确定第二位置并获取对应的第二投影图像。
在此，受移动机器人的行进平面的材质影响，为提高第一位置和第二位置的测量准确性，在移动机器人移动期间，利用移动机器人的位置测量装置，处理装置监测移动机器人从所述第一位置到移动机器人当前位置之间的相对位移及相对偏角，并当判断所述相对位移和/或相对偏角超出预设测量阈值时，确定第二位置并获取对应该第二位置的第二投影图像。其中，所述测量阈值可以一固定值，或相对值。例如，所述测量阈值为5cm。又如，所述测量阈值为移动机器人的滚轮圈数的n倍，n>1。
在一些实施例中，利用所述移动机器人的位置测量装置获取所述移动机器人的移动信息。所述位置测量装置包括但不限于均设置在移动机器人的位移传感器、测距传感器、角度传感器、双目摄像装置、速度传感器等。在此，处理装置可仅监测位置测量装置中的一种来监测移动机器人从所述第一位置到移动机器人当前位置之间的相对位移及相对偏角。例如，所述位置测量装置为设置于所述移动机器人的马达的计数传感器，利用马达运转的圈数进行计数以获得所述移动机器人自第一位置移动至第二位置的相对位移，以及利用马达运转的角度获取姿态信息等。又如，所述位置测量装置测量移动机器人的行进速度，并利用行进速度及所行进的时长计算自第一位置移动至第二位置的相对位移。或者，结合多种传感器和摄像装置等所提供的位置数据来监测移动机器人从所述第一位置到移动机器人当前位置之间的相对位移及相对偏角。例如，利用预设权重并结合双目摄像装置所获取的移动机器人相对于固定目标的多组测距数据，以及马达运转所测量的数据，来确定移动机器人从所述第一位置到移动机器人当前位置之间的相对位移及相对偏角。
在另一些实施例中，所述位置测量装置为GPS(Global Positioning System，全球定位系统)，根据该GPS监测移动机器人从所述第一位置到移动机器人当前位置之间的矢量位移及矢量偏角；并当判断所述相矢量位移和/或矢量偏角超出预设测量阈值时，确定第二位置并获取对应该第二位置的第二投影图像，以及获取所述第一位置和第二位置之间的相对位置信息。在又一些实施例中，所述位置测量装置为基于测量无线信号而定位的装置，例如，所述位置 测量装置为蓝牙(或WiFi)定位装置；位置测量装置根据在第一位置和第二位置各自对所接收的无线定位信号的功率进行测量，来确定各位置相对于预设无线定位信号发射装置的相对位置；利用该位置测量装置监测移动机器人从所述第一位置到移动机器人当前位置之间的相对位移及相对偏角；并当判断所述相对位移和/或相对偏角超出预设测量阈值时，确定第二位置并获取对应该第二位置的第二投影图像，以及获取所述第一位置和第二位置之间的相对位置信息。
在另一些实施例中，以所述地图为一种栅格地图为例，预先确定单位栅格长度与实际位移之间的映射关系，按照移动机器人在移动期间所得到的移动信息，监测移动机器人从所述第一位置到移动机器人当前位置之间的栅格数及栅格偏角；并当判断所述栅格数和/或栅格偏角超出预设测量阈值时，确定第二位置并获取对应该第二位置的第二投影图像，以及获取所述第一位置和第二位置之间的相对位置信息。
在确定了第一位置和第二位置之间的相对位置信息，以及移动机器人分别在第一位置和第二位置相对于所识别出的实体对象的角度信息后，执行步骤S730。
步骤S730：基于从所述第一投影图像和第二投影图像中所识别出的同一实体对象相对于移动机器人在第一位置及第二位置各自的角度信息、以及所述第一位置和第二位置，确定所述移动机器人在所述行进平面的定位信息，以便基于所述定位信息生成导航路线。
在此，根据所述地图中预先标记的图像特征或实体对象信息的类型，移动机器人能够通过将所识别出的图像特征或实体对象信息的类型与地图中所标记的对应信息进行映射。其中，为提高准确定位，所述映射关系为一个或多个。例如当所识别出的图像特征或实体对象信息的类型在地图中为唯一的，则所述映射关系可只为一个，并借由地图中的位置得到所识别出的实体对象在物理空间中的位置。又如，当所识别出的图像特征或实体对象信息的类型在地图中不是唯一的，则可在至少两幅投影图像中识别多个特征或类型，并根据多个图像特征或类型之间的相对位置关系，以及移动机器人移动的相对位置关系，从地图中得到符合多个特征或类型的映射关系，进而从地图中得到所标记的各特征或类型在移动机器人的行进平面中的物理位置。
参阅关于图1的描述可知，我们可以用摄像装置所摄取的实体对象在投影图像中的位置来表示该实体对象投影至所述移动机器人的行进平面的位置，且利用所述实体对象在投影图像中的位置相对于所述移动机器人移动方向的角度来表征该实体对象投影至所述移动机器人的行进平面的位置相对于所述移动机器人移动方向的角度。
再参见图1可知，由于所获取的投影图像所在平面是与移动机器人的行进平面平行的，因此，在基于移动机器人的行进平面(或者基于投影图像所在平面)而构建的坐标系中，同 一实体对象在投影图像中目标区域与坐标基准线之间的偏角，相当于该实体对象投影在所述行进平面的位置相对于所述坐标基准线的偏角。其中，所述坐标基准线为所述坐标系所在平面内的直线。例如，所述坐标基准线为移动机器人的前进方向所在直线。换言之，当处理装置通过投影图像确定对应实体对象的目标区域与坐标基准线之间的角度时，意味着可以确定该实体对象位于与移动机器人行进方向偏移所述角度的方向上。
因此，当移动机器人在行进平面移动期间，将所述移动机器人所在不同位置以及在各位置处所拍摄到的同一实体对象在投影图像中的目标区域，均映射到所述坐标系中，可基于所得到的各角度和各位置而构建几何关系，利用所构建的几何关系确定所述移动机器人在所述行进平面中的定位信息。其中，所述几何关系包括但不限于：三角形关系、四边形关系等。
需要说明的是所述坐标系用于帮助理解，在实际地图构建期间，并非一定绘制或显示所述坐标系。
在一些实施方式中，所述步骤S730还包括：基于所述角度信息、所述第一位置和第二位置，构建在所述行进平面内的几何关系；以及利用所构建的几何关系确定所述移动机器人在所述行进平面中的定位信息。
以移动机器人沿直线从第一位置P1移动至第二位置P2为例，请参阅图3，图3显示为基于角度信息以及第一位置和第二位置确定实体对象位置信息的原理示意图。利用上述坐标系，处理装置在所述第一位置P1获取如图3所示的第一投影图像，所述处理装置在所述第二位置P2获取如图3所示的第二投影图像，处理装置可依据前述提及的各测量方式得到所述第一位置P1和所述第二位置P2间形成连线P1P2；以及，所述处理装置还确定同一实体对象分别在第一投影图像和所述第二投影图像中的目标区域O1和O2；对于同一实体对象来说，其应位于所述目标区域O1与第一位置P1的连线P1O1上，以及位于所述目标区域O2与第二位置P2的连线P2O2上，换言之，实体对象位于线段P1O1和P2O2相交的位置O，即该位置O为该实体对象投影到行进平面的投影位置。其中，处理装置根据所述连线P1O1与所述连线P1P2之间的夹角α1、连线P2O2与所述连线P1P2之间的夹角β1、连线P1P2之间的距离、以及根据地图中所得到的点O在行进平面中的位置，得到所述移动机器人在点P1、P2、O所构成的三角形中点P1和P2在行进平面中的物理位置。
以移动机器人从第一位置移动至第二位置时姿态发生变化为例，所获取的第一位置和第二位置中包含位移信息和姿态信息。其中，根据第一位置和第二位置的姿态信息，可得到相对姿态变化信息。其中，所述相对姿态变化信息包括在行进平面内所述移动机器人在第二位置相对于所述移动机器人在第一位置所转动的角度。对应地，所述步骤S730还包括：基于所述相对姿态变化信息抵消所识别出的同一实体对象相对于移动机器人在第一位置及第二位置 各自的角度信息中的角度偏差；以及基于抵消后的角度信息和所述第一位置和第二位置确定所述移动机器人在所述行进平面中的定位信息。
参阅图4所示，图4显示为基于角度信息以及第一位置和第二位置确定共同目标区域对应的实体对象的位置信息的原理示意图。所述移动机器人在第一位置P1’处向Y轴的正方向的行进方向移动，以及在第二位置P2’处沿向Y轴正方向偏左γ角度Y’轴的正方向移动，由此，处理装置可确定移动机器人在第一位置和第二位置之间的相对姿态变化信息为夹角γ。其中，夹角γ可由移动机器人的位置测量装置中的角度传感器提供。
仍参见图4，处理装置根据在所述第一位置P1’处摄取的第一投影图像中目标区域在坐标系中的位置，确定该目标区域所对应的实体对象位于连线P1’O1’所在直线上，其与所述Y轴的正方向的夹角α2，根据在所述第二位置P2’处摄取的第二投影图像中目标区域在坐标系中的位置，确定该目标区域所对应的实体对象位于连线P2’O2’所在直线上，其与所述Y’轴的正方向的夹角β2，由于Y’轴的正方向与Y轴的正方向之间夹角γ，因此，所述处理装置在确定实体对象相距第二位置的连线时，需抵消夹角γ所产生的角度偏差，即将所述夹角β2减去所述夹角γ后的补角为夹角β2’。即可根据所述连线P1’P2’的距离、所述夹角α2和所述夹角β2’所构建的三角关系，确定所述移动机器人在拍摄第一投影图像和第二投影图像时的位置信息。
需要说明的是，上述任一示例中均未考虑实体对象在所述行进平面的垂直方向上的高度。在实际应用中，利用移动机器人的高度测量装置，可在定位实体对象的位置时测量其高度信息。其应视为基于本申请所提供的地图构建方案而拓展的一种实施例。在此不予详述。
在又一些实施方式中，所确定的移动机器人的定位信息是被不断纠正的，或经多次定位纠正后被确定下来的。为此，所述定位方法还包括：对于所识别出的同一实体对象，和所述第一位置及所拍摄的第一投影图像，令移动机器人在多个第二位置拍摄对应的第二投影图像，并根据第一位置和多个第二位置各自拍摄的第一投影图像和第二投影图像分别执行前述确定所述移动机器人在所述第一位置的定位信息的步骤，以获得藉由每次执行上述步骤而得到的所述移动机器人的至少两个对应第一位置的定位信息。在此，结合前述步骤S710-S720可知，本步骤可藉由第一位置与每个第二位置之间的相对位置信息、第一投影图像中所识别出的实体对象相对于第一位置的角度信息、以及每个第二投影图像中所识别出的实体对象相对于第二位置的角度信息、以及地图中所标记的该实体对象的位置信息，分别确定各次第一位置的定位信息。受移动机器人所行进平面的材料影响，移动机器人移动期间历次获取的第一位置与第二位置之间的相对位置信息无法精准确定。例如，当移动机器人在地毯上移动时，其真实移动的距离小于其通过位置测量装置所测量的距离。又如，当移动机器人在地板上移动时，其真实移动的距离接近其通过位置测量装置所测量的距离。因此，在多次执行步骤S710-S720 期间，藉由相对位置信息、各角度信息和实体对象在地图中标记的位置信息等而计算得到的第一位置的定位信息将会有差异。
因此，本申请所提供的构建地图的方法可藉由移动机器人在移动期间的第一位置、多个第二位置、及实体对象在地图中标记的位置信息等所构成的至少两个三角形，对第一位置的定位信息进行确定，并利用以下步骤进行误差修正，即基于所得到的所述移动机器人的至少两个对应第一位置的定位信息，对所述移动机器人在第一位置时的定位位置进行误差修正处理，以便将误差修正后的定位位置确定为所述移动机器人在第一位置时的定位位置。
在此，所述误差修正的方式包括但不限于：1)采用加权平均的方式对所得到的各定位信息进行误差修正处理，以将误差修正后的定位信息作为移动机器人在第一位置的定位信息。例如，将所得到的同一实体对象的投影位置信息中横纵坐标进行加权平均，得到误差修正后的定位信息作为移动机器人在第一位置的定位信息。2)采用数值优化的方式所得到的第一位置的定位信息进行迭代处理，以得到经迭代处理后估计的对应第一位置的定位信息，并将所估计的定位信息作为移动机器人在第一位置的定位信息。例如，在预设对应第一位置的定位信息的初始值基础上，利用第一次得到的对应第一位置的定位信息与所述初始值所构成的梯度，控制移动机器人移动方向并第二次执行对第一位置的定位操作，通过预设条件的迭代循环，得到所估计的对应第一位置的定位信息，并将所估计的对应第一位置的定位信息作为第一位置的定位信息；其中，所述初始值可以随机设置，或者依据前述步骤S710-S720所提供的方式得到。
在得到减少误差后的第一位置的定位信息后，再结合移动机器人从该第一位置移动至当前位置的相对位置信息，确定移动机器人当前的定位信息。
例如，利用移动机器人的位置测量装置从该第一位置移动至当前位置的相对位置信息，并根据所得到的第一位置的定位信息确定移动机器人当前的定位信息。又如，基于经误差修正后所确定第一位置所对应的定位位置、与未经误差修正的第一位置所对应的各定位信息，确定移动机器人的位置测量装置的测量误差，并根据该测量误差抵消移动机器人从该第一位置移动至当前位置所测量出相对位置信息的误差，以及确定其在当前位置的定位信息。
本申请的定位方法根据所述摄像装置摄取在移动机器人处于第一位置处的第一投影图像和第二位置处的第二投影图像；且基于自所述第一投影图像和第二投影图像中所识别出的共同目标区域在所述行进平面中的角度信息、以及所述第一位置和第二位置，确定所述移动机器人在所述行进平面中的定位信息；以得到较为精确的关于移动机器人的定位信息。
在确定了移动机器人当前的位置后，本申请还提供一种导航方法，即包含步骤S610和步骤S620。
在步骤S610中，获取利用任一构建地图的方法而构建的地图，以及获取利用前述步骤S710-S730所得到的移动机器人的定位信息。
在步骤S620中，利用前述构建的地图和所得到的定位信息设置导航路线，以供所述移动机器人按照所述导航路线进行移动控制。在此，所述导航方法主要由移动机器人的处理装置来执行，并由处理装置将所设计出的导航路线提供给下位机，以供该下位机控制移动机器人中的马达等移动装置进行移动。其中，所述下位机举例为MCU、CPU等。
在此，根据移动机器人的实际应用，利用所述地图可设置适应于移动目的地的导航路线。例如，对于安防机器人来说，在检查地图中所标记的实际对象期间，其设置的导航路线需途径每个实际对象。又如，对于家庭陪伴机器人来说，通过远程操控而执行电器控制期间，其设置的导航路线可以所标记的实际对象为目的地址。再如，对于清洁机器人来说，根据在预设的清扫区域内的实际对象的位置，设置对应于清洁模式的导航路线。
在一些具体实施例中，所述地图中包含实体对象的实体对象信息及其位置信息；所述步骤S620还包括步骤S621。
所述步骤S621包括：基于包含实体对象信息的指令信息，设置导航至所述实体对象信息的导航路线；于本实施例中，所述实体对象信息例如为对各实体对象信息的种类的名称描述，例如包括对桌子、椅子、花盆、电视以及冰箱等物体的名称描述。
其中，获取包含实体对象信息的指令信息的方式包括但不限于：语音方式、文本方式等。在此，根据用户对移动机器人的操作目的，所述指令中还可以包含移动机器人的执行指令。例如，所述指令还包括清扫指令、巡视指令、远程操控指令等。
在一种实施例中，所述步骤S621可以包括：获取一语音信息，并从所述语音信息中识别包含实体对象信息的指令。在一示例中，移动机器人可以直接接收用户的语音信息并识别所述信息中包括的实体对象信息的指令。例如，用户可以直接向移动机器人语音“桌子”，移动机器人在接收到所述指令后移动至桌子以进行预先设定的相应处理。且可以根据路线上经过的实体对象信息规划所述移动机器人从当前的位置移动至桌子的导航路线。所述移动机器人从当前位置移动到所述桌子的导航路线上可经过花盆、电视、沙发等。以清洁机器人为例，预先设定清洁机器人在接收到用户包含实体对象信息的指令后，根据所构建的地图规划导航路线，以令所述移动机器人移动至该实体对象信息对应的位置以进行清扫，则在用户向清洁机器人语音“桌子”的情况下，清洁机器人在接收到该语音指令后，根据所构建的地图，形成令所根据花盆、电视和沙发形成的导航路线，且所述清洁机器人在经过根据所述花盆、电视和沙发形成的导航路线后移动至桌子处并执行清扫操作。另外，所述语音信息并不限于仅表示实体对象信息的短指令，还可以是包括实体对象信息的长指令，例如，用户语音“去桌子处”， 则移动机器人可以识别语音信息中所包括的实体对象信息“桌子”指令，然后进行后续操作。
在另一种实施例中，所述步骤S621还包括：自一终端设备获取包含实体对象信息的指令。其中，所述终端设备与移动机器人无线连接。在一示例中，用户经由终端设备以文本方式输入包含实体对象信息的指令。例如，用户通过手机APP以文本方式输入“桌子”。在另一示例中，用于经由终端设备以语音方式输入包含实体对象信息的指令。例如，用户通过手机APP以语音方式输入“桌子”。另外，用户输入的语音信息并不限于仅表示实体对象信息的短指令，还可以是包括实体对象信息的长指令，例如，用户语音“去桌子处”，则终端设备将其翻译成文本并提取其中桌子等关键词，将所翻译的文本匹配到相应的指令上发送给移动机器人。在此，终端设备可以与移动机器人以wifi连接、近场通信或蓝牙配对等无线方式连接，以将终端设备接收的指令传送给移动机器人进行后续操作。所述终端设备例如为智能手机、平板电脑、可穿戴设备或其他具有智能处理功能的智能设备。
在另一些具体实施例中，所述步骤S620还包括步骤S622。所述步骤S622包括：基于所述移动机器人的定位信息以及地图中的实体对象信息，设置所述移动机器人自定位信息移动并途径至少一个实体对象的导航路线。在某些实施例中，移动机器人根据地图上移动机器人的当前位置、实体对象信息所在的目的位置、以及地图上其他物体的位置信息或预设路径等生成自当前位置信息至目的位置信息的导航路线。例如，所述导航路线可以是自移动机器人的当前位置至目的位置的最短路线。又如，所述导航路线可以是自移动机器人的当前位置至目的位置的与移动机器人先前行进路径不重复的路线，以移动机器人为清洁机器人为例，根据未清洁区域和已清洁区域设置自当前位置至目的位置的无需重复清洁已清洁区域的导航路线，由此提高清洁效率。
在一些应用中，所述移动机器人为清洁机器人；所述步骤S620还包括：基于所述地图中所标记的实体对象及其位置信息，设置遍历清扫区域中途径所述实体对象的导航路线。在一些示例中，清洁机器人基于预先确定的清扫区域设计遍历该清扫区域的导航路线，其中，根据地图中位于所述清扫区域内的实体对象的标记信息，清洁机器人根据相应的标记信息确定便于清洁的导航路线。其中，所述清扫区域包括但不限于以下至少一种：按照预设栅格数量划分的清扫区域、以及按照房间分割的清扫区域等。例如，在所获取的地图中的一清扫区域内，标记有桌子及其位置信息，故，在设计导航路线时设计包含围绕桌腿旋转的导航路线。
本申请的导航方法，通过识别与预先标记的实体对象信息相符的实体对象并基于摄像装置所摄取的图像确定所识别的实体对象与移动机器人之间的相对空间位置，且在依据图2所述方法所构建的地图中将所述实体对象信息进行标记，以生成标记有所述实体对象信息的所述地图。且依据图7所示的定位方法得到较为精确的关于移动机器人的定位信息。且利用所 述地图和定位信息设置导航路线，提高了移动机器人的导航路线的精确性，且提高人机交互性。
参阅图8，图8显示为本申请的移动机器人的控制方法在一具体实施例中的流程示意图。所述移动机器人的控制方法应用于包含摄像装置和移动装置的移动机器人中，所述控制方法主要由移动机器人的控制系统来执行，所述控制系统包括按照所获取的导航路线控制移动机器人的移动装置的控制器，和用于暂存控制期间所产生的中间数据的暂存器。其中，所述移动装置举例包括滚轮和滚轮的驱动器，其中驱动器举例为马达等。所述控制器举例为MCU等。所述暂存器包括但不限于寄存器、易失性存储器等。
在步骤S810中，获取导航路线；所述导航路线为根据图5或图6所示的导航方法及其相关描述所形成的导航路线。
在步骤S820中，按照所述导航路线控制所述移动装置以调整所述移动机器人的位置和姿态，以沿所述导航路线进行自主移动。在此，通过所述移动机器人所具有的移动传感器实时获取所述移动机器人的位置信息和姿态信息，且根据所述导航路线，对所述移动装置进行控制，以令所述移动机器人以相应的姿态在所述导航路线上进行移动。
在某些实施例中，可以根据利用标记有实体对象信息的地图上实体对象信息所指示的目的位置、依据移动传感器获取的移动机器人在所述地图上的当前位置和当前姿态、以及地图上的其他信息如其他物体位置信息或预设路径等控制移动机器人自对应的当前位置移动至对应的目的位置。例如，可以按照地图中所标记的可移动路线控制移动机器人从当前位置移动至目的位置，且移动机器人在根据所述可移动路线进行自主移动的过程中需要转弯时，则令所述移动机器人进行旋转，以调整所述移动机器人的姿态，并以调整后的姿态沿接下来的路径行进。
此外，在某些实施例中，移动机器人还可以将自对应的当前位置移动至对应的目的位置的移动路线发送至与所述移动机器人通信的终端设备，以在终端设备所显示的地图上显示移动路线、当前位置信息、目的位置信息、或其任意组合，从而便于用户查看。例如，在某些情况下，可以在终端设备所显示的地图上实时显示机器人当前位置，且在目的位置为桌子所在位置时，在地图上的目的位置处显示“桌子”的文字描述、显示与桌子的形象对应的图标或者显示预设的与所述桌子对应的编号等，同时在地图上显示从移动机器人当前位置到桌子的移动路线。在另一些实施例中，所述终端设备通过对显示的所述地图的导航路线的修改操作，修改移动路线或目的位置信息，且将修改操作更新至所述移动机器人中，以令所述移动机器人根据修改后的移动路线或目的位置信息进行移动。所述修改的操作可以为通过对显示所述移动路线和目的位置信息的触摸屏的触摸操作，也可以为令所述终端设备接收包括修改指令 的语音信息。例如，当前移动机器人正根据移动路线经过电视形进至桌子处，用户发现在桌子和电视之间存在不可触碰的障碍物，则更改所述目的位置信息为电视，以令所述移动机器人停止在所述电视位置处，以避免与所述障碍物发生碰撞。当所述修改指令为语音信息时，此处例如为：“修改目的位置为电视”。在另一些具体实施例中，还可直接通过语音向所述移动机器人发送修改指令。例如向所述移动机器人发送“修改目的位置为电视”的语音，修改所述移动机器人的导航路线的目的位置为电视。
本申请的移动机器人的控制方法，依据图5或图6所示的方法形成关于所构建的地图的导航路线，构建的地图中将所述实体对象信息进行标记，以生成标记有所述实体对象信息的所述地图，且在所述移动机器人的移动控制中可以基于地图上标记的实体对象信息来识别用户指令中的目的位置，以形成所述导航路线，进而控制所述移动机器人依据所述导航路线移动至目的位置处，提高了移动机器人的导航路线的精确性，且提高人机交互性。
参阅图9，图9显示为本申请的构建地图的系统在一具体实施例中的组成示意图。构建地图的系统，用于所述移动机器人，所述构建地图的系统包括摄像装置910和处理装置920。
所述摄像装置910，设置于所述移动机器人，用于在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像；所述摄像装置910包括但不限于鱼眼摄像模块、广角(或非广角)摄像模块中的任一种。所述移动机器人包括但不限于：家庭陪伴式移动机器人、清洁机器人、巡逻式移动机器人、擦玻璃的机器人等。
在此，所述移动机器人包括至少一个摄像装置910。所述摄像装置910在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像。例如，移动机器人包含一个摄像装置，其设置于所述移动机器人顶部、肩部或背部，且主光轴垂直于所述移动机器人的行进平面。又如，移动机器人包含多个摄像装置，其中一个摄像装置910的主光轴垂直于所述移动机器人的行进平面。且以上述方式设置的摄像装置所拍摄的图像在所述移动机器人的行进平面的投影形成的投影图像，相当于所述摄像装置所拍摄的图像在所述行进平面的垂直投影，基于所述移动摄像机在所述第一位置获取的关于所述行进平面投影的第一投影图像，在所述第一投影图像中识别所述共同目标区域，且获取所述移动机器人在所述第一位置处与所述共同目标区域的连线关于所述移动机器人从所述第一位置行进至所述第二位置的连线的第一角度，基于所述移动摄像机在所述第二位置获取的关于所述行进平面投影的第二投影图像，在所述第二投影图像中识别所述共同目标区域，且获取所述移动机器人在所述第二位置处与所述共同目标区域的连线关于所述移动机器人从所述第一位置行进至所述第二位置的连线的第二角度，且根据所述第一位置和所述第二位置间的距离以及所述第一角度和所述第二角度获取所述共同目标区域对应的实体对象投影到所述行进平面中 的位置。且由于所述摄像头设置于所述移动机器人的顶部，且所述摄像装置的主光轴垂直于所述移动机器人的所述行进平面，所述实体对象投影到所述行进平面中的投影位置与所述移动机器人在所述第一位置处的连线关于所述移动机器人从所述第一位置行进至所述第二位置的连线的夹角与所述第一角度相同，所述实体对象投影到所述行进平面中的投影位置与所述移动机器人在所述第二位置处的连线关于所述移动机器人从所述第一位置行进至所述第二位置的连线的夹角与所述第二角度相同。再参阅图1及关于图1的相关描述，我们用摄像装置910所摄取的实体对象在投影图像中的位置来表示该实体对象投影至所述移动机器人的行进平面的位置，且利用所述实体对象在投影图像中的位置相对于所述移动机器人移动方向的角度来表征该实体对象投影至所述移动机器人的行进平面的位置相对于所述移动机器人移动方向的角度。
所述处理装置920为一种能够进行数值运算、逻辑运算及数据分析的电子设备，其包括但不限于：CPU、GPU、FPGA等，以及用于暂存运算期间所产生的中间数据的易失性存储器等。所述处理装置920与所述摄像装置910相连，用于运行至少一程序，以执行构建地图的方法。所述构建地图的方法参阅图2及关于图2的相关描述，在此不加赘述。
参阅图10，图10显示为本申请的导航系统在一具体实施例中的组成示意图。导航系统用于所述移动机器人，所述导航系统包括摄像装置1010和处理装置1020。
所述摄像装置1010，设置于所述移动机器人，用于在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像；所述摄像装置1010包括但不限于鱼眼摄像模块、广角(或非广角)摄像模块中的任一种。所述移动机器人包括但不限于：家庭陪伴式移动机器人、清洁机器人、巡逻式移动机器人、擦玻璃的机器人等。
在此，所述移动机器人包括至少一个摄像装置1010。所述摄像装置1010在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像。例如，移动机器人包含一个摄像装置1010，其设置于所述移动机器人顶部、肩部或背部，且主光轴垂直于所述移动机器人的行进平面。又如，移动机器人包含多个摄像装置，其中一个摄像装置1010的主光轴垂直于所述移动机器人的行进平面。且参阅图1及关于图1的相关描述，我们用摄像装置1010所摄取的实体对象在投影图像中的位置来表示该实体对象投影至所述移动机器人的行进平面的位置，且利用所述实体对象在投影图像中的位置相对于所述移动机器人移动方向的角度来表征该实体对象投影至所述移动机器人的行进平面的位置相对于所述移动机器人移动方向的角度。
所述处理装置1020为一种能够进行数值运算、逻辑运算及数据分析的电子设备，其包括但不限于：CPU、GPU、FPGA等，以及用于暂存运算期间所产生的中间数据的易失性存储器 等。所述处理装置1020与所述摄像装置1010相连，用于运行至少一程序，以执行导航方法。所述导航方法参阅图5及关于图5的相关描述，或所述导航方法参阅图6及关于图6的相关描述，在此不加赘述。
参阅图11，图11显示为本申请的移动机器人在一具体实施例中的组成示意图。
所述移动机器人包括摄像装置1110、移动装置1120、第一处理装置1130以及第二处理装置1140。
所述摄像装置1110设置于所述移动机器人，所述摄像装置1110在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像；所述摄像装置1010包括但不限于鱼眼摄像模块、广角(或非广角)摄像模块中的任一种。所述移动机器人包括但不限于：家庭陪伴式移动机器人、清洁机器人、巡逻式移动机器人、擦玻璃的机器人等。
在此，所述移动机器人包括至少一个摄像装置1010。所述摄像装置1010在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像。例如，移动机器人包含一个摄像装置1010，其设置于所述移动机器人顶部、肩部或背部，且主光轴垂直于所述移动机器人的行进平面。又如，移动机器人包含多个摄像装置，其中一个摄像装置1010的主光轴垂直于所述移动机器人的行进平面。且参阅图1及关于图1的相关描述，我们用摄像装置1010所摄取的实体对象在投影图像中的位置来表示该实体对象投影至所述移动机器人的行进平面的位置，且利用所述实体对象在投影图像中的位置相对于所述移动机器人移动方向的角度来表征该实体对象投影至所述移动机器人的行进平面的位置相对于所述移动机器人移动方向的角度。
所述移动装置1120设置于所述移动机器人，用于受控地调整所述移动机器人的位置和姿态；在一些实施例中，所述移动装置1120举例包括滚轮和滚轮的驱动器，其中驱动器举例为马达等。
所述第一处理装置1130连接所述摄像装置1110，用于运行至少一程序，以执行导航方法；所述导航方法参阅图5及关于图5的相关描述或参阅图6及关于图6的相关描述，在此不加赘述。
所述第二处理装置1140，连接于所述第一处理装置1130和移动装置1120，用于运行至少一程序，以基于所述第一处理装置1130所提供的导航路线，控制所述移动装置1120执行控制方法。所述控制方法参阅图8及关于图8的相关描述，在此不加赘述。
参阅图12，图12显示为本申请的数据处理装置在一具体实施例中的组成示意图。
数据处理装置用于包含摄像装置的移动机器人，其中，所述摄像装置在移动机器人所在 位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像，所述数据处理装置包括数据接口1210、存储单元1220以及处理单元1230。
所述数据接口1210，用于连接所述摄像装置；所述摄像装置在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像。例如，移动机器人包含一个摄像装置，其设置于所述移动机器人顶部、肩部或背部，且主光轴垂直于所述移动机器人的行进平面。又如，移动机器人包含多个摄像装置，其中一个摄像装置的主光轴垂直于所述移动机器人的行进平面。且参阅图1及关于图1的相关描述，我们用摄像装置所摄取的实体对象在投影图像中的位置来表示该实体对象投影至所述移动机器人的行进平面的位置，且利用所述实体对象在投影图像中的位置相对于所述移动机器人移动方向的角度来表征该实体对象投影至所述移动机器人的行进平面的位置相对于所述移动机器人移动方向的角度。
所述存储单元1220，用于存储至少一程序；
所述处理单元1230，与所述存储单元1220和数据接口1210相连，用于藉由所述数据接口1210获取所述摄像装置拍摄的图像；以及用于执行所述至少一程序以执行构建地图的方法、或执行导航方法、或者执行控制方法。其中，所述构建地图的方法参阅图2及关于图2的相关描述，所述导航方法参阅图5及关于图5的相关描述或参阅图6及关于图6的相关描述，所述控制方法参阅图8及关于图8的相关描述，在此不加赘述。
本申请另一实施例中，还公开一种计算机可读的存储介质，所述计算机可读的存储介质存储至少一种程序，所述至少一种程序在被调用时执行构建地图的方法、或执行导航方法或者执行控制方法。其中，所述构建地图的方法参阅图2及关于图2的相关描述，所述导航方法参阅图5及关于图5的相关描述或参阅图6及关于图6的相关描述，所述控制方法参阅图8及关于图8的相关描述，在此不加赘述。
上述实施例仅例示性说明本申请的原理及其功效，而非用于限制本申请。任何熟悉此技术的人士皆可在不违背本申请的精神及范畴下，对上述实施例进行修饰或改变。因此，举凡所属技术领域中具有通常知识者在未脱离本申请所揭示的精神与技术思想下所完成的一切等效修饰或改变，仍应由本申请的权利要求所涵盖。

Claims (32)

  1. 一种构建地图的方法，用于包含摄像装置的移动机器人，其中，所述摄像装置在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像，其特征在于，所述构建地图的方法包括：
    分别在所述移动机器人行进至第一位置和第二位置处对应获取第一投影图像和第二投影图像；
    基于从所述第一投影图像和第二投影图像中所识别出的同一实体对象相对于移动机器人在第一位置及第二位置各自的角度信息、以及所述第一位置和第二位置，确定所述实体对象在所述行进平面中的投影位置信息；
    将所确定的实体对象的投影位置信息标记到预设的地图中，以便利用所述地图生成导航路线。
  2. 根据权利要求1所述的构建地图的方法，其特征在于，还包括基于预设的实体对象信息的种类，识别所述第一投影图像和第二投影图像中对应同一实体对象的目标区域的步骤，以及将所识别出的实体对象信息标记在所述地图中的步骤。
  3. 根据权利要求2所述的构建地图的方法，其特征在于，所述基于预设的实体对象信息的种类，识别第一投影图像和第二投影图像中对应同一实体对象的目标区域的步骤包括以下至少一种：
    基于预设置的多种实体对象的特征信息，识别第一投影图像和第二投影图像中对应同一实体对象的目标区域；
    利用预设的图像识别算法，将第一投影图像和第二投影图像中对应同一实体对象的目标区域与预设的多种实体对象信息进行映射处理。
  4. 根据权利要求1或2所述的构建地图的方法，其特征在于，还包括以下至少一步骤：
    基于在所述第一投影图像中识别的第一目标区域，并利用图像追踪技术在所述第二投影图像中识别对应的第二目标区域，以得到两幅投影图像中对应同一实体对象的目标区域；
    匹配所述第一投影图像和第二投影图像中具有共同目标特征的目标区域，以得到两幅投影图像中对应同一实体对象的目标区域。
  5. 根据权利要求1或2所述的构建地图的方法，其特征在于，还包括在对应所述第一位置和第二位置处获取所述移动装置的姿态信息，并得到所述移动机器人自第一位置至第二位置 的相对姿态变化信息；
    对应地，基于从所述第一投影图像和第二投影图像中所识别出的同一实体对象相对于移动机器人在第一位置及第二位置各自的角度信息、以及所述第一位置和第二位置，确定所述实体对象在所述行进平面中的投影位置信息的步骤包括：
    基于所述相对姿态变化信息抵消所识别出的同一实体对象相对于移动机器人在第一位置及第二位置各自的角度信息中的角度偏差；以及
    基于抵消后的角度信息和所述第一位置和第二位置确定所述实体对象在所述行进平面中的投影位置信息。
  6. 根据权利要求1或2所述的构建地图的方法，其特征在于，所述确定实体对象在所述行进平面中的投影位置信息的步骤包括：
    基于所述角度信息、所述第一位置和第二位置，构建在所述行进平面内的几何关系；
    利用所构建的几何关系确定所述实体对象在所述行进平面中的投影位置信息。
  7. 根据权利要求1或2所述的构建地图的方法，其特征在于，在所述移动机器人从第一位置移动至第二位置期间还执行以下步骤：
    根据所述移动机器人的移动信息，测量从所述第一位置至移动机器人当前位置之间的相对位置信息；
    基于所测量的相对位置信息确定第二位置并获取对应的第二投影图像。
  8. 根据权利要求1所述的构建地图的方法，其特征在于，所述方法还包括：
    在所述移动机器人移动期间，对于所识别出的同一实体对象，多次执行前述获取两幅投影图像以及确定所述实体对象的投影位置信息的步骤，以获得藉由每次执行上述步骤而得到的同一实体对象的至少两个投影位置信息；
    基于所得到的对应同一实体对象的至少两个投影位置信息，对所述实体对象的投影位置进行误差修正处理，以便将误差修正后的投影位置信息标记到预设地图中。
  9. 一种导航方法，用于包含摄像装置的移动机器人，其特征在于，所述导航方法包括：
    获取利用如权利要求1-8中任一所述的构建地图的方法所构建的地图；
    利用所述地图设置导航路线，以供所述移动机器人按照所述导航路线进行移动控制。
  10. 根据权利要求9所述的导航方法，其特征在于，所述地图中包含实体对象信息及其位置信息；所述利用地图设置导航路线的步骤包括以下至少一种：
    基于包含实体对象信息的指令信息，设置导航至所述实体对象信息的导航路线；
    基于所述移动机器人的当前位置信息以及地图中的实体对象信息，设置所述移动机器人自当前位置移动并途径至少一个实体对象的导航路线。
  11. 根据权利要求9或10所述的导航方法，其特征在于，所述移动机器人为清洁机器人；所述利用地图设置导航路线的步骤包括：基于所述地图中所标记的实体对象及其位置信息，设置遍历清扫区域中途径所述实体对象的导航路线。
  12. 一种定位方法，用于包含摄像装置的移动机器人，其中，所述摄像装置在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像，其特征在于，所述移动机器人中预存储标记有实体对象的投影位置信息的地图，所述定位方法包括：
    分别在所述移动机器人行进至第一位置和第二位置处对应获取第一投影图像和第二投影图像；其中，所述第一投影图像和第二投影图像中均包含所述地图中所标记的同一实体对象所投影的目标区域；
    基于从所述第一投影图像和第二投影图像中所识别出的同一实体对象相对于移动机器人在第一位置及第二位置各自的角度信息、以及所述第一位置和第二位置，确定所述移动机器人在所述行进平面的定位信息，以便基于所述定位信息生成导航路线。
  13. 根据权利要求12所述的定位方法，其特征在于，还包括基于预设的实体对象信息的种类，识别所述第一投影图像和第二投影图像中对应同一实体对象的目标区域的步骤，以及基于所识别出的实体对象信息的种类确定实体对象信息在所述地图中位置信息的步骤。
  14. 根据权利要求13所述的定位方法，其特征在于，所述基于预设的实体对象信息的种类，识别第一投影图像和第二投影图像中对应同一实体对象的目标区域的步骤包括以下至少一种：
    基于预设置的多种实体对象的特征信息，识别第一投影图像和第二投影图像中对应同一实体对象的目标区域；
    利用预设的图像识别算法，将第一投影图像和第二投影图像中对应同一实体对象的目 标区域与预设的多种实体对象信息进行映射处理。
  15. 根据权利要求12或13所述的定位方法，其特征在于，还包括以下至少一步骤：
    基于在所述第一投影图像中识别的第一目标区域，并利用图像追踪技术在所述第二投影图像中识别对应的第二目标区域，以得到两幅投影图像中对应同一实体对象的目标区域；
    匹配所述第一投影图像和第二投影图像中具有共同目标特征的目标区域，以得到两幅投影图像中对应同一实体对象的目标区域。
  16. 根据权利要求12或13所述的定位方法，其特征在于，还包括在对应所述第一位置和第二位置处获取所述移动装置的姿态信息，并得到所述移动机器人自第一位置至第二位置的相对姿态变化信息；
    对应地，基于从所述第一投影图像和第二投影图像中所识别出的同一实体对象相对于移动机器人在第一位置及第二位置各自的角度信息、以及所述第一位置和第二位置，确定所述移动机器人在所述行进平面的定位信息的步骤包括：
    基于所述相对姿态变化信息抵消所识别出的同一实体对象相对于移动机器人在第一位置及第二位置各自的角度信息中的角度偏差；以及
    基于抵消后的角度信息和所述第一位置和第二位置确定所述移动机器人在所述行进平面中的定位信息。
  17. 根据权利要求12或13所述的定位方法，其特征在于，所述确定所述移动机器人在所述行进平面中的定位信息的步骤包括：
    基于所述角度信息、所述第一位置和第二位置之间相对位置信息，构建在所述行进平面内的几何关系；
    利用所构建的几何关系确定所述移动机器人在所述行进平面中的定位信息。
  18. 根据权利要求12或13所述的定位方法，其特征在于，在所述移动机器人从第一位置移动至第二位置期间还执行以下步骤：
    根据所述移动机器人的移动信息，测量从所述第一位置到移动机器人当前位置之间的相对位置信息；
    基于所述相对位置信息确定第二位置并获取对应的第二投影图像。
  19. 根据权利要求12所述的定位方法，其特征在于，所述方法还包括：
    对于所识别出的同一实体对象，和所述第一位置及所拍摄的第一投影图像，令移动机器人在多个第二位置拍摄对应的第二投影图像，并根据第一位置和多个第二位置各自拍摄的第一投影图像和第二投影图像分别执行前述确定所述移动机器人在所述第一位置的定位信息的步骤，以获得藉由每次执行上述步骤而得到的所述移动机器人的至少两个对应第一位置的定位信息；
    基于所得到的所述移动机器人的至少两个对应第一位置的定位信息，对所述移动机器人在第一位置时的定位位置进行误差修正处理，以便将误差修正后的定位位置确定为所述移动机器人在第一位置时的定位位置；
    基于所确定的所述移动机器人在第一位置时的定位位置，确定移动机器人当前的定位信息。
  20. 一种导航方法，用于包含摄像装置的移动机器人，其特征在于，所述导航方法包括：
    获取标记有实体对象的投影位置信息的地图；以及利用如权利要求12-19中任一所述的定位方法确定所述移动机器人当前的定位信息；
    利用所述地图和定位信息设置导航路线，以供所述移动机器人按照所述导航路线进行移动控制。
  21. 根据权利要求20所述的导航方法，其特征在于，所述地图是利用如权利要求1-8中任一所述的构建地图的方法所构建的地图。
  22. 根据权利要求20所述的导航方法，其特征在于，所述地图中包含实体对象信息及其位置信息；所述利用地图和定位信息设置导航路线的步骤包括以下至少一种：
    基于包含实体对象信息的指令信息，设置从所述定位信息导航至所述实体对象信息的导航路线；
    基于所述移动机器人的定位信息以及地图中的实体对象信息，设置所述移动机器人自定位信息移动并途径至少一个实体对象的导航路线。
  23. 根据权利要求20或21所述的导航方法，其特征在于，所述移动机器人为清洁机器人；所述利用地图设置导航路线的步骤包括：基于所述地图中所标记的实体对象及其位置信息，设置遍历清扫区域中途径所述实体对象的导航路线。
  24. 一种移动机器人的控制方法，用于包含摄像装置和移动装置的移动机器人，其特征在于，所述控制方法包括：
    获取如权利要求9-11、或者20-23中任一所述的导航路线，并按照所述导航路线控制所述移动装置以调整所述移动机器人的位置和姿态，以沿所述导航路线进行自主移动。
  25. 一种构建地图的系统，用于移动机器人，其特征在于，包括：
    摄像装置，设置于所述移动机器人，用于在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像；
    处理装置，与所述摄像装置相连，用于运行至少一程序，以执行如权利要求1-8中任一所述的构建地图方法。
  26. 根据权利要求25所述的构建地图的系统，其特征在于，所述摄像装置的主光轴垂直于所述移动机器人的行进平面。
  27. 一种导航系统，用于移动机器人，其特征在于，所述导航系统包括：
    摄像装置，设置于所述移动机器人，用于在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像；
    处理装置，与所述摄像装置相连，用于运行至少一程序，以执行如权利要求9-11、或者20-23中任一所述的导航方法。
  28. 根据权利要求27所述的导航系统，其特征在于，所述摄像装置的主光轴垂直于所述移动机器人的行进平面。
  29. 一种移动机器人，其特征在于，包括：
    摄像装置，设置于所述移动机器人，用于在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像；
    移动装置，设置于所述移动机器人，用于受控地调整所述移动机器人的位置和姿态；
    第一处理装置，连接所述所述摄像装置，用于运行至少一程序，以执行如权利要求9-11、或者20-23中任一所述的导航方法；
    第二处理装置，连接于所述第一处理装置和移动装置，用于运行至少一程序，以基于 所述第一处理装置所提供的导航路线，控制所述移动装置调整所述移动机器人的位置和姿态，以沿所述导航路线进行自主移动。
  30. 根据权利要求29所述的移动机器人，其特征在于，所述摄像装置的主光轴垂直于所述移动机器人的行进平面。
  31. 一种数据处理装置，用于包含摄像装置的移动机器人，其中，所述摄像装置在移动机器人所在位置摄取视场范围内实体对象并投影至所述移动机器人的行进平面，以得到投影图像，其特征在于，所述数据处理装置包括：
    数据接口，用于连接所述摄像装置；
    存储单元，用于存储至少一程序；
    处理单元，与所述存储单元和数据接口相连，用于藉由所述数据接口获取所述摄像装置拍摄的图像；以及用于执行所述至少一程序以执行如权利要求1-8中任一所述的构建地图的方法、或执行如权利要求9-11、或者20-23中任一所述的导航方法、或执行权利要求12-19中任一所述的定位方法、或者执行如权利要求23所述的控制方法。
  32. 一种计算机可读的存储介质，其特征在于，存储至少一种程序，所述至少一种程序在被调用时执行如权利要求1-8中任一所述的构建地图的方法、或执行如权利要求9-11、或者19-22中任一所述的导航方法、或执行权利要求12-19中任一所述的定位方法、或者执行如权利要求23所述的控制方法。
PCT/CN2018/116251 2018-11-19 2018-11-19 构建地图、定位、导航、控制方法及系统、移动机器人 WO2020102946A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
PCT/CN2018/116251 WO2020102946A1 (zh) 2018-11-19 2018-11-19 构建地图、定位、导航、控制方法及系统、移动机器人
CN201880002428.6A CN109643127B (zh) 2018-11-19 2018-11-19 构建地图、定位、导航、控制方法及系统、移动机器人
US16/532,552 US11320834B2 (en) 2018-11-19 2019-08-06 Methods and systems for mapping, localization, navigation and control and mobile robot

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/116251 WO2020102946A1 (zh) 2018-11-19 2018-11-19 构建地图、定位、导航、控制方法及系统、移动机器人

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/532,552 Continuation US11320834B2 (en) 2018-11-19 2019-08-06 Methods and systems for mapping, localization, navigation and control and mobile robot

Publications (1)

Publication Number Publication Date
WO2020102946A1 true WO2020102946A1 (zh) 2020-05-28

Family

ID=66060113

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/116251 WO2020102946A1 (zh) 2018-11-19 2018-11-19 构建地图、定位、导航、控制方法及系统、移动机器人

Country Status (3)

Country Link
US (1) US11320834B2 (zh)
CN (1) CN109643127B (zh)
WO (1) WO2020102946A1 (zh)

Families Citing this family (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11687092B2 (en) 2018-04-23 2023-06-27 Sharkninja Operating Llc Techniques for bounding cleaning operations of a robotic surface cleaning device within a region of interest
WO2020146352A1 (en) * 2019-01-08 2020-07-16 R-Go Robotics Ltd. System and method for monitoring movements
US20220155448A1 (en) * 2019-05-16 2022-05-19 Mitsubishi Electric Corporation Information processing device, information processing method, and storage medium
CN110231042A (zh) * 2019-05-28 2019-09-13 上海车轮互联网服务有限公司 基于定位的线路采集方法及装置
CN110207537A (zh) * 2019-06-19 2019-09-06 赵天昊 基于计算机视觉技术的火控装置及其自动瞄准方法
CN110471409B (zh) * 2019-07-11 2022-12-02 深圳市优必选科技股份有限公司 机器人巡检方法、装置、计算机可读存储介质及机器人
CN112650207B (zh) * 2019-10-11 2024-06-14 杭州萤石软件有限公司 机器人的定位校正方法、设备和存储介质
CN111028658A (zh) * 2019-11-18 2020-04-17 杭州晶一智能科技有限公司 一种基于gps和在线地图的割草机器人电子地图构建方法
CN114846424A (zh) * 2020-01-03 2022-08-02 苏州宝时得电动工具有限公司 自移动设备及其自动移动和工作的方法、及存储介质
CN113156928A (zh) * 2020-01-03 2021-07-23 苏州宝时得电动工具有限公司 自移动设备、终端、服务器自动更新数据模型的方法
EP3929690A1 (en) * 2020-06-22 2021-12-29 Carnegie Robotics, LLC A method and a system for analyzing a scene, room or venueby determining angles from imaging elements to visible navigation elements
WO2022027611A1 (zh) * 2020-08-07 2022-02-10 苏州珊口智能科技有限公司 移动机器人的定位方法、构建地图的方法及移动机器人
CN111932623A (zh) * 2020-08-11 2020-11-13 北京洛必德科技有限公司 一种基于移动机器人的人脸数据自动采集标注方法、系统及其电子设备
EP3958086A1 (en) * 2020-08-19 2022-02-23 Carnegie Robotics, LLC A method and a system of improving a map for a robot
CN114119360A (zh) * 2020-08-26 2022-03-01 广东博智林机器人有限公司 装饰材料铺贴方法和装置
CN112284402B (zh) * 2020-10-15 2021-12-07 广州小鹏自动驾驶科技有限公司 一种车辆定位的方法和装置
CN114449647B (zh) * 2020-10-30 2024-02-09 华为终端有限公司 定位方法及装置
CN112558611B (zh) * 2020-12-15 2022-07-08 深圳市云视机器人有限公司 一种路径规划方法、装置、计算机设备及存储介质
CN113657164B (zh) * 2021-07-15 2024-07-02 美智纵横科技有限责任公司 标定目标对象的方法、装置、清扫设备和存储介质
CN113907645A (zh) * 2021-09-23 2022-01-11 追觅创新科技(苏州)有限公司 移动机器人的定位方法及装置、存储介质及电子装置
CN113654747B (zh) * 2021-09-26 2024-04-16 珠海格力智能装备有限公司 机器人的关节刚度的检测方法、装置和机器人
CN114297761A (zh) * 2021-12-31 2022-04-08 广东博智林机器人有限公司 一种铺贴机器人的作业路径生成方法及系统
CN114371710B (zh) * 2022-01-07 2024-04-30 牧原肉食品有限公司 基于反光柱的移动机器人的导航方法、设备及可读存储介质
CN117035745B (zh) * 2023-10-07 2023-12-19 上海华建电力设备股份有限公司 一种变电站线路保护系统及方法
CN117475438B (zh) * 2023-10-23 2024-05-24 北京点聚信息技术有限公司 基于ocr技术的扫描文件矢量化转换方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110164832A1 (en) * 2010-01-04 2011-07-07 Samsung Electronics Co., Ltd. Image-based localization feature point registration apparatus, method and computer-readable medium
CN103903253A (zh) * 2012-12-28 2014-07-02 联想(北京)有限公司 一种可移动终端定位方法及系统
CN107449427A (zh) * 2017-07-27 2017-12-08 京东方科技集团股份有限公司 一种生成导航地图的方法及设备
CN107907131A (zh) * 2017-11-10 2018-04-13 珊口（上海）智能科技有限公司 定位系统、方法及所适用的机器人
CN107928566A (zh) * 2017-12-01 2018-04-20 深圳市沃特沃德股份有限公司 视觉扫地机器人及障碍物检测方法

Family Cites Families (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100601960B1 (ko) * 2004-08-05 2006-07-14 삼성전자주식회사 로봇의 위치 추적 및 지도 작성 방법
KR100654676B1 (ko) * 2005-03-07 2006-12-08 삼성광주전자 주식회사 로봇청소기
KR100735565B1 (ko) * 2006-05-17 2007-07-04 삼성전자주식회사 구조광을 이용한 물체 검출 방법 및 이를 이용한 로봇
CA3161006A1 (en) * 2011-05-06 2012-11-15 Hadal, Inc. Systems and methods for synthetic aperture sonar
CN103901884B (zh) * 2012-12-25 2017-09-29 联想(北京)有限公司 信息处理方法和信息处理设备
US9646384B2 (en) * 2013-09-11 2017-05-09 Google Technology Holdings LLC 3D feature descriptors with camera pose information
US10012504B2 (en) * 2014-06-19 2018-07-03 Regents Of The University Of Minnesota Efficient vision-aided inertial navigation using a rolling-shutter camera with inaccurate timestamps
WO2016163492A1 (ja) * 2015-04-10 2016-10-13 日本電気株式会社 位置特定装置、位置特定方法及びプログラム
CN107850902B (zh) * 2015-07-08 2022-04-08 深圳市大疆创新科技有限公司 可移动物体上的相机配置
GB2541884A (en) * 2015-08-28 2017-03-08 Imp College Of Science Tech And Medicine Mapping a space using a multi-directional camera
CN108139758A (zh) * 2015-10-09 2018-06-08 深圳市大疆创新科技有限公司 基于显著性特征的载运工具定位
EP3428885A4 (en) * 2016-03-09 2019-08-14 Guangzhou Airob Robot Technology Co., Ltd. CARD CONSTRUCTION METHOD, AND CORRECTION METHOD AND APPARATUS
US10241514B2 (en) * 2016-05-11 2019-03-26 Brain Corporation Systems and methods for initializing a robot to autonomously travel a trained route
JP2018028489A (ja) * 2016-08-18 2018-02-22 トヨタ自動車株式会社 位置推定装置、位置推定方法
CN106920279B (zh) * 2017-03-07 2018-06-19 百度在线网络技术（北京）有限公司 三维地图构建方法和装置
CN106910217A (zh) * 2017-03-17 2017-06-30 驭势科技（北京）有限公司 视觉地图建立方法、计算装置、计算机存储介质和智能车辆
CN106931961B (zh) * 2017-03-20 2020-06-23 成都通甲优博科技有限责任公司 一种自动导航方法及装置
US10436590B2 (en) * 2017-11-10 2019-10-08 Ankobot (Shanghai) Smart Technologies Co., Ltd. Localization system and method, and robot using the same
CN107680135B (zh) * 2017-11-16 2019-07-23 珊口（上海）智能科技有限公司 定位方法、系统及所适用的机器人
CN107943037A (zh) * 2017-11-28 2018-04-20 上海与德科技有限公司 一种基于智能扫地机器人的控制方法及系统
CN108125622A (zh) * 2017-12-15 2018-06-08 珊口（上海）智能科技有限公司 控制方法、系统及所适用的清洁机器人
CN108132666B (zh) * 2017-12-15 2019-04-05 珊口（上海）智能科技有限公司 控制方法、系统及所适用的移动机器人
CN108256430B (zh) * 2017-12-20 2021-01-29 北京理工大学 障碍物信息获取方法、装置及机器人
US11614746B2 (en) * 2018-01-05 2023-03-28 Irobot Corporation Mobile cleaning robot teaming and persistent mapping

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110164832A1 (en) * 2010-01-04 2011-07-07 Samsung Electronics Co., Ltd. Image-based localization feature point registration apparatus, method and computer-readable medium
CN103903253A (zh) * 2012-12-28 2014-07-02 联想(北京)有限公司 一种可移动终端定位方法及系统
CN107449427A (zh) * 2017-07-27 2017-12-08 京东方科技集团股份有限公司 一种生成导航地图的方法及设备
CN107907131A (zh) * 2017-11-10 2018-04-13 珊口（上海）智能科技有限公司 定位系统、方法及所适用的机器人
CN107928566A (zh) * 2017-12-01 2018-04-20 深圳市沃特沃德股份有限公司 视觉扫地机器人及障碍物检测方法

Also Published As

Publication number Publication date
US20200159246A1 (en) 2020-05-21
CN109643127A (zh) 2019-04-16
CN109643127B (zh) 2022-05-03
US11320834B2 (en) 2022-05-03

Similar Documents

Publication Publication Date Title
WO2020102946A1 (zh) 构建地图、定位、导航、控制方法及系统、移动机器人
US10518414B1 (en) Navigation method, navigation system, movement control system and mobile robot
CN108247647B (zh) 一种清洁机器人
WO2020140271A1 (zh) 移动机器人的控制方法、装置、移动机器人及存储介质
WO2019232803A1 (zh) 移动控制方法、移动机器人及计算机存储介质
WO2020113452A1 (zh) 移动目标的监控方法、装置、监控系统及移动机器人
WO2020223974A1 (zh) 更新地图的方法及移动机器人
WO2019090833A1 (zh) 定位系统、方法及所适用的机器人
US20200306989A1 (en) Magnetometer for robot navigation
WO2019095681A1 (zh) 定位方法、系统及所适用的机器人
CN113116224B (zh) 机器人及其控制方法
WO2021146862A1 (zh) 移动设备的室内定位方法、移动设备及控制系统
CN106813672B (zh) 移动机器人的导航方法及移动机器人
CN103389699A (zh) 基于分布式智能监测控制节点的机器人监控及自主移动系统的运行方法
US20190184569A1 (en) Robot based on artificial intelligence, and control method thereof
WO2021159603A1 (zh) 无人机室内导航方法、装置、设备和存储介质
CN105973231A (zh) 一种导航方法和装置
WO2019232804A1 (zh) 软件更新方法、系统、移动机器人及服务器
JP2019041261A (ja) 画像処理システムおよび画像処理システムの設定方法
WO2019001237A1 (zh) 一种移动电子设备以及该移动电子设备中的方法
WO2018228254A1 (zh) 一种移动电子设备以及该移动电子设备中的方法
CN110260866A (zh) 一种基于视觉传感器的机器人定位与避障方法
US20200125597A1 (en) Controlling method for artificial intelligence moving robot
TWI771960B (zh) 一種用於智慧無人載具系統的室內定位及物件搜尋方法
CN114995459A (zh) 机器人的控制方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18940682

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 28/09/2021)

122 Ep: pct application non-entry in european phase

Ref document number: 18940682

Country of ref document: EP

Kind code of ref document: A1