CN115565057B - Map generation method, map generation device, foot robot and storage medium - Google Patents

Map generation method, map generation device, foot robot and storage medium Download PDF

Info

Publication number
CN115565057B
CN115565057B CN202110753733.2A CN202110753733A CN115565057B CN 115565057 B CN115565057 B CN 115565057B CN 202110753733 A CN202110753733 A CN 202110753733A CN 115565057 B CN115565057 B CN 115565057B
Authority
CN
China
Prior art keywords
image
target
scene
images
map
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110753733.2A
Other languages
Chinese (zh)
Other versions
CN115565057A (en
Inventor
豆子飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Xiaomi Mobile Software Co Ltd
Original Assignee
Beijing Xiaomi Mobile Software Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Xiaomi Mobile Software Co Ltd filed Critical Beijing Xiaomi Mobile Software Co Ltd
Priority to CN202110753733.2A priority Critical patent/CN115565057B/en
Publication of CN115565057A publication Critical patent/CN115565057A/en
Application granted granted Critical
Publication of CN115565057B publication Critical patent/CN115565057B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B29/00—Maps; Plans; Charts; Diagrams, e.g. route diagram
    • G09B29/003—Maps
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Physics (AREA)
  • Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • Manipulator (AREA)

Abstract

The application provides a map generation method, a map generation device, a foot robot and a storage medium, wherein the map generation method is used for the foot robot and comprises the steps of obtaining a current scene image; identifying a following object from the current scene image, and determining a first local image corresponding to the following object; in the process of controlling the foot robot to follow the following movement of the following object, updating the current scene image at least once to obtain at least one new scene image, wherein the new scene image comprises the following components: a first partial image corresponding to the following object; and generating a target map from the plurality of scene images and the plurality of first partial images. The map generation method and the map generation device can effectively reduce labor cost and learning cost of map generation, improve map generation efficiency, improve map generation accuracy and improve map generation effect.

Description

Map generation method, map generation device, foot robot and storage medium
Technical Field
The present application relates to the field of robots, and in particular, to a map generating method, a map generating device, a foot robot, and a storage medium.
Background
With the rapid development of science and technology, more and more scenes are beginning to use robots instead of manual operations, and thus robots having various functions, such as wheeled robots or foot robots, capable of providing various business services to users, have been developed.
In the related art, a method for generating a map by using a robot generally includes automatic mapping, i.e., a region is freely explored by the robot, so as to autonomously build the map; the manual drawing is built, and an operation user remotely controls the robot through a mobile terminal, a control handle and the like, so that the robot is manually controlled to move, and an image is built in the moving process of the robot.
Under these modes, some scene areas may be missed, not completely adopted to assist in drawing construction, and relatively high learning cost is required, so that relatively high labor cost and learning cost are required, map generation efficiency is low, map generation is not accurate enough, and map generation effect is affected.
Disclosure of Invention
The present application aims to solve at least one of the technical problems in the related art to some extent.
Therefore, the application aims to provide a map generation method, a map generation device, a foot robot and a storage medium, which can effectively reduce the labor cost and the learning cost of map generation, improve the map generation efficiency, improve the map generation accuracy and improve the map generation effect.
In order to achieve the above object, a map generating method according to an embodiment of a first aspect of the present application is used for a foot robot, and includes: acquiring a current scene image; identifying a following object from the current scene image, and determining a first local image corresponding to the following object; in the process of controlling the foot robot to follow the movement based on the following object, updating the current scene image at least once to obtain at least one new scene image, wherein the new scene image comprises the following components: a first partial image corresponding to the following object; and generating a target map from a plurality of the scene images and a plurality of the first partial images.
In some embodiments of the present disclosure, the identifying a follower object from among the current scene images includes:
Identifying at least one detection image from among the current scene images, wherein the detection image is an image comprising candidate objects;
Determining image features corresponding to images of the candidate object;
And determining a target image feature from at least one image feature, and taking a candidate object to which the target image feature belongs as the following object.
In some embodiments of the disclosure, the generating a target map from a plurality of the scene images and a plurality of the first partial images includes:
identifying a second partial image from among the scene images, the second partial image corresponding to the candidate object;
performing target processing on the scene image according to the second local image to obtain a target scene image;
A target map is generated from the plurality of target scene images and the plurality of first partial images.
In some embodiments of the disclosure, the performing target processing on the scene image according to the second local image to obtain a target scene image includes:
determining a target background image from the scene image;
And replacing a second local image in the scene image with the target background image to obtain the target scene image.
In some embodiments of the present disclosure, the foot robot configures an image capturing device, wherein the generating a target map from a plurality of the target scene images and a plurality of the first partial images based on the image capturing device comprises:
generating a plurality of local depth images corresponding to the first local images respectively;
Determining a plurality of position data corresponding to the foot robot, wherein the foot robot controls the camera device to capture a corresponding scene image based on the position data;
The target map is generated from a plurality of the target scene images, the plurality of local depth images, and the plurality of position data.
In some embodiments of the present disclosure, the generating the target map from a plurality of the target scene images, the plurality of local depth images, and the plurality of position data includes:
generating a point cloud sub-map corresponding to the position data according to each target scene image, the local depth image corresponding to each target scene image and the position data corresponding to each target scene image;
Performing splicing and fusion processing on the plurality of point cloud sub-maps to obtain a point cloud map;
and processing the point cloud map into a grid map, and taking the grid map as the target map.
In some embodiments of the present disclosure, the following object is a user in a moving state in the scene, or a movable object in the scene whose motion state is controlled.
According to the map generation method provided by the embodiment of the first aspect of the application, a current scene image is acquired, a following object is identified from the current scene image, a first local image corresponding to the following object is determined, and in the process of controlling the foot robot to follow movement based on the following object, the current scene image is updated at least once to obtain at least one new scene image, wherein the new scene image comprises: the first partial images corresponding to the following objects and the target map is generated according to the scene images and the first partial images, and the map is automatically generated in the autonomous following process of the foot robot, so that manual operation of a user is not needed, the following path planning by the following objects can be supported, the labor cost and the learning cost of map generation can be effectively reduced, the map generation efficiency is improved, the map generation accuracy is improved, and the map generation effect is improved.
In order to achieve the above object, a map generating apparatus according to a second aspect of the present application is a map generating apparatus for a foot robot, comprising: the acquisition module is used for acquiring the current scene image; the identification module is used for identifying a following object from the current scene image and determining a first local image corresponding to the following object; the updating module is used for updating the current scene image at least once in the process of controlling the foot robot to follow the movement based on the following object so as to obtain at least one new scene image, and the new scene image comprises the following components: a first partial image corresponding to the following object; and a generation module for generating a target map according to a plurality of the scene images and a plurality of the first partial images.
In some embodiments of the disclosure, the identification module is specifically configured to:
Identifying at least one detection image from among the current scene images, wherein the detection image is an image comprising candidate objects;
Determining image features corresponding to images of the candidate object;
And determining a target image feature from at least one image feature, and taking a candidate object to which the target image feature belongs as the following object.
In some embodiments of the present disclosure, the generating module includes:
an identification sub-module, configured to identify a second partial image from among the scene images, where the second partial image corresponds to the candidate object;
The processing sub-module is used for carrying out target processing on the scene image according to the second local image so as to obtain a target scene image;
and the generation sub-module is used for generating a target map according to the plurality of target scene images and the plurality of first local images.
In some embodiments of the disclosure, the processing submodule is specifically configured to:
determining a target background image from the scene image;
And replacing a second local image in the scene image with the target background image to obtain the target scene image.
In some embodiments of the disclosure, the foot robot configures an image capturing device, wherein the generating sub-module is specifically configured to:
generating a plurality of local depth images corresponding to the first local images respectively;
Determining a plurality of position data corresponding to the foot robot, wherein the foot robot controls the camera device to capture a corresponding scene image based on the position data;
The target map is generated from a plurality of the target scene images, the plurality of local depth images, and the plurality of position data.
In some embodiments of the disclosure, the generating submodule is specifically configured to:
generating a point cloud sub-map corresponding to the position data according to each target scene image, the local depth image corresponding to each target scene image and the position data corresponding to each target scene image;
Performing splicing and fusion processing on the plurality of point cloud sub-maps to obtain a point cloud map;
and processing the point cloud map into a grid map, and taking the grid map as the target map.
In some embodiments of the present disclosure, the following object is a user in a moving state in the scene, or a movable object in the scene whose motion state is controlled.
According to the map generation device provided by the embodiment of the second aspect of the application, a current scene image is acquired, a following object is identified from the current scene image, a first local image corresponding to the following object is determined, and in the process of controlling the foot robot to follow movement based on the following object, the current scene image is updated at least once to obtain at least one new scene image, wherein the new scene image comprises: the first partial images corresponding to the following objects and the target map is generated according to the scene images and the first partial images, and the map is automatically generated in the autonomous following process of the foot robot, so that manual operation of a user is not needed, the following path planning by the following objects can be supported, the labor cost and the learning cost of map generation can be effectively reduced, the map generation efficiency is improved, the map generation accuracy is improved, and the map generation effect is improved.
To achieve the above object, a foot robot according to an embodiment of a third aspect of the present application includes: the map generation system comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor realizes the map generation method according to the embodiment of the first aspect of the application when executing the program.
According to the foot robot provided by the embodiment of the third aspect of the application, a current scene image is acquired, a following object is identified from the current scene image, a first local image corresponding to the following object is determined, and in the process of controlling the foot robot to follow movement based on the following object, the current scene image is updated at least once to obtain at least one new scene image, wherein the new scene image comprises: the first partial images corresponding to the following objects and the target map is generated according to the scene images and the first partial images, and the map is automatically generated in the autonomous following process of the foot robot, so that manual operation of a user is not needed, the following path planning by the following objects can be supported, the labor cost and the learning cost of map generation can be effectively reduced, the map generation efficiency is improved, the map generation accuracy is improved, and the map generation effect is improved.
To achieve the above object, a fourth aspect of the present application provides a computer-readable storage medium, which when executed by a processor, implements the map generation method of the present application.
Additional aspects and advantages of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the application.
Drawings
The foregoing and/or additional aspects and advantages of the application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
FIG. 1 is a flow chart of a map generating method according to an embodiment of the present application;
FIG. 2 is a flow chart of a map generating method according to another embodiment of the present application;
Fig. 3 is a schematic structural diagram of a map generating apparatus according to an embodiment of the present application;
Fig. 4 is a schematic structural diagram of a map generating apparatus according to another embodiment of the present application;
fig. 5 is a schematic structural diagram of a foot robot according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the application. On the contrary, the embodiments of the application include all alternatives, modifications and equivalents as may be included within the spirit and scope of the appended claims.
Fig. 1 is a flowchart of a map generating method according to an embodiment of the present application.
The present embodiment is exemplified by the map generation method being configured in the map generation apparatus.
The map generating method in the present embodiment may be configured in a map generating apparatus that may be provided in a foot robot.
The execution body of the present embodiment may be, for example, a central processing unit (Central Processing Unit, CPU) in the legged robot in terms of hardware, or a related background service in the legged robot in terms of software, without limitation.
As shown in fig. 1, the map generating method includes:
S101: a current scene image is acquired.
The embodiment of the application can be particularly applied to an application scene of generating a two-dimensional or three-dimensional map of the scene by using the foot robot, and is not limited.
In some embodiments, the camera may be configured for the foot robot to capture a scene image based on the camera, wherein assuming the foot robot is placed in an application scene for which a map is to be generated, the foot robot may be controlled to turn on the camera, and a frame of the scene image based on the camera is captured at a current point in time, the scene image may be referred to as a current scene image.
Wherein the current scene image may be used to assist the foot robot in identifying the following object.
S102: a follower object is identified from among the current scene images and a first partial image corresponding to the follower object is determined.
After capturing the current scene image, the image capturing device may transmit the scene image to the image recognition module built in the foot robot, and the processor controls the image recognition module to recognize the current scene image, so as to recognize the following object, and then, may determine the first partial image corresponding to the following object.
The first partial image may be a partial image matching the contour region of the following object, among the scene images, and may be referred to as a first partial image.
For example, an image in which a scene image includes a user (the user appears in the image in the form of a portrait of the user) and the portrait has a portrait contour area corresponding to the portrait, and a local area matching the portrait contour area may be referred to as a first local image.
For another example, if the scene image includes a movable ball (the movable ball is displayed in the image in the form of a ball image), the image of the ball has a block of image contour region, and the image of the portion matching the image contour region is referred to as a first partial image.
The following object is identified from the current scene image, and a first partial image corresponding to the following object is determined, and the first partial image can be used for assisting in generating a scene-related map, see the following embodiment.
In some embodiments, the follower object is a user in a scene in a moving state, or a movable object in the scene in a controlled state of motion.
That is, in the embodiment of the present application, the foot robot is supported to follow the user, in which the user can move in the scene to plan the following path, the foot robot captures the scene image, and identifies the user in the moving state from the scene image to trigger the following, so that the moving following is performed based on the following path planned by the user in the moving state, and the map is generated autonomously in the process of moving following.
In the embodiment of the application, the foot robot can be supported to follow the movable object preset by the user, such as a movable small ball, the movable small ball can be controlled to move, under the scene, the movable small ball can be controlled to move in the scene based on the pre-planned following path, the foot robot captures a scene image, and the movable small ball in a moving state is identified from the scene image so as to trigger the following, thereby carrying out the moving following based on the following path of the movable small ball, and generating a map autonomously in the moving following process.
Therefore, in the embodiment of the application, the foot robot is supported to follow the user, and the foot robot is also supported to follow the movable object preset by the user, so that the implementation flexibility of the map generation method can be effectively improved, and the use scene of the foot robot is effectively expanded.
S103: in the process of controlling the foot robot to follow the following movement of the following object, updating the current scene image at least once to obtain at least one new scene image, wherein the new scene image comprises the following components: a first partial image corresponding to a following object.
After the following object is identified from the current scene image and the first partial image corresponding to the following object is determined, the foot robot can be controlled to follow the following object, and it can be understood that, due to the autonomous movement of the following object, in order to avoid the following loss and ensure the following accuracy, the current scene image is updated at least once in the process of controlling the foot robot to follow the following object to move so as to obtain at least one new scene image, so that after each capturing of the new scene image, the new scene object can be adopted to dynamically update the current scene image, and the following object is dynamically identified from the updated scene image so as to assist autonomous following.
The embodiment of the application provides a technical scheme of autonomous image building in an autonomous following process, so that in the embodiment of the application, after at least one new scene image is captured, each scene image can be correspondingly identified to identify information in the scene images of the following object, and a first local image corresponding to the following object in each scene image is determined.
That is, for convenience of explanation, in the embodiment of the present application, local images corresponding to the following object in the scene images captured each time may be referred to as first local images, and since the number of the scene images is plural, each scene image may have a corresponding first local image.
S104: a target map is generated from the plurality of scene images and the plurality of first partial images.
In the process of controlling the foot robot to follow the movement based on the following objects, capturing a plurality of scene images, processing the plurality of scene images to identify a plurality of following objects respectively corresponding to the plurality of scene images, performing image identification on the plurality of scene images, determining a plurality of first partial images respectively corresponding to the plurality of following objects, and generating a target map according to the plurality of scene images and the plurality of first partial images.
Because the scene images which are not interfered by movement are usually adopted when the target map is generated, in the embodiment of the application, each scene image can be processed by adopting a plurality of image processing methods to remove the movement interference factors, the first local image can be deleted for each scene image, so that a plurality of target scene images which are not interfered by movement are obtained, and then the target map can be generated by adopting a map building method of a three-dimensional dense point cloud map according to the plurality of target scene images.
In this embodiment, by acquiring a current scene image, identifying a following object from the current scene image, determining a first local image corresponding to the following object, and updating the current scene image at least once in a process of controlling the foot robot to follow movement based on the following object, so as to obtain at least one new scene image, where the new scene image includes: the first partial images corresponding to the following objects and the target map is generated according to the scene images and the first partial images, and the map is automatically generated in the autonomous following process of the foot robot, so that manual operation of a user is not needed, the following path planning by the following objects can be supported, the labor cost and the learning cost of map generation can be effectively reduced, the map generation efficiency is improved, the map generation accuracy is improved, and the map generation effect is improved.
Fig. 2 is a flowchart of a map generating method according to another embodiment of the present application.
As shown in fig. 2, the map generation method includes:
s201: a current scene image is acquired.
The illustration of S201 may be specifically referred to the above embodiments, and will not be described herein.
S202: at least one detected image is identified from among the current scene images, the detected image being an image comprising a candidate object.
The detection image may be an image of a local area in the scene image, where the local area may include at least an area of a candidate object, and the candidate object may be a plurality of user objects in the scene image, and if the plurality of user objects are identified from the scene image, an accurate following object may be identified from the plurality of user objects, and the detection image may be used to assist in determining the following object from the scene image, and an area occupied by the detection image corresponding to the following object in the scene image may be greater than or equal to an area occupied by the first local image in the scene image.
After the current scene image is acquired, at least one detection image can be identified from the current scene image, and the detection image is an image including a candidate object and then a subsequent step is triggered.
S203: image features corresponding to images of the candidate object are determined.
In some embodiments, the image features corresponding to the detected image (such as brightness, content, chromaticity, etc. of the image) may be directly analyzed, and the image features corresponding to the detected image are taken as the image features corresponding to the candidate objects included in the detected image, which is not limited.
S204: and determining the target image characteristic from at least one image characteristic, and taking the candidate object to which the target image characteristic belongs as a following object.
After at least one detection image is identified from the current scene image and the image characteristics corresponding to the candidate objects in each detection image are determined, the target image characteristics can be determined from the at least one image characteristics, and the candidate objects to which the target image characteristics belong are taken as following objects.
For example, the image features corresponding to each candidate object may be sent to the mobile terminal, and the operation user selects the target image feature from the plurality of image features, or the plurality of image features may also be directly matched with the preset reference image feature of the following object, and the successfully matched image feature is used as the target image feature, which is not limited.
According to the method, the at least one detection image is obtained from the current scene image in a recognized mode, the detection image is the image comprising the candidate object, the image characteristics corresponding to the image of the candidate object are determined, the target image characteristics are determined from the at least one image characteristics, the candidate object to which the target image characteristics belong is used as the following object, the following object is accurately positioned from the scene image, the following efficiency and the following accuracy are guaranteed, and the image characteristics corresponding to the image of the candidate object are obtained through analysis, so that the scene image can be effectively assisted in subsequent processing.
S205: a first partial image corresponding to the following object is determined.
S206: in the process of controlling the foot robot to follow the following movement of the following object, updating the current scene image at least once to obtain at least one new scene image, wherein the new scene image comprises the following components: a first partial image corresponding to a following object.
The examples of S205-S206 can be specifically referred to the above embodiments, and are not described herein.
S207: a second partial image is identified from among the scene images, the second partial image corresponding to the candidate object.
That is, the above-mentioned image of the part matching the contour region of the following object, which is identified from among the scene images, may be referred to as a first partial image, and correspondingly, the image of the part matching the contour region of the candidate object, which is identified from among the scene images, may be referred to as a second partial image, and since both the following object and the candidate object may be in a moving state, in order to secure the map generation effect, the present embodiment supports combining the following object and the candidate object to correspond to the partial image from among the scene images, so as to process the scene images.
In this embodiment, after capturing the plurality of scene images, the image capturing device may transmit the plurality of scene images to the image recognition module built in the legged robot, and the processor controls the image recognition module to recognize the plurality of scene images respectively, so as to recognize candidate objects included in each scene image, and then, may determine a second partial image corresponding to the following object in the scene images.
It will be appreciated that the number of scene images may be plural, and that one or more candidates may be included in each scene image, and that accordingly, one or more second partial images may be included in each scene image, with or without overlapping regions between the different second partial images, without limitation.
S208: and performing target processing on the scene image according to the second local image to obtain a target scene image.
Optionally, in some embodiments, to effectively remove the interference caused by the candidate object in the moving state on the scene image, the target background image may be determined from the scene image, and the second local image in the scene image is replaced by the target background image, so as to obtain the target scene image.
That is, the embodiment of the application supports the manner of deleting the second partial image related to the candidate object from the scene image, and the deleting may be to determine the target background image, where the target background image may be obtained by matting from the scene image, and the matching between the target background image and the surrounding scene of the second partial image may be higher, so that after the second partial image in the scene image is replaced by the target background image, the boundary between the target background image and the second partial image may be ensured to be naturally connected, and the capturing quality of the scene image is ensured, so as to ensure the quality of the map generated subsequently.
Of course, in other embodiments, any other possible manner may be used to perform target processing on the scene image according to the second local image, so as to obtain the target scene image, so that the target scene image can be effectively prevented from being interfered by the moving factor, which is not limited.
S209: a target map is generated from the plurality of target scene images and the plurality of first partial images.
After the target processing is performed on the scene image according to the second partial image to obtain the target scene image, the target map can be directly generated according to the plurality of target scene images and the plurality of first partial images, and the target scene image is the scene image excluding the moving interference factors, so that the generated target map has better image quality.
Alternatively, when generating the target map from the plurality of target scene images and the plurality of first partial images, it may be to generate a plurality of partial depth images corresponding to the plurality of first partial images, respectively, and determine a plurality of position data corresponding to the legged robot, wherein the legged robot controls the image capturing device to capture the corresponding scene image based on the position data, and generates the target map from the plurality of target scene images, the plurality of partial depth images, and the plurality of position data.
Since the first partial image is an image of a part which is recognized to coincide with the contour region of the following object from among the scene images, the above-described generation of a plurality of partial depth images corresponding to the plurality of first partial images, respectively, that is, the generation of a depth image corresponding to the following object (this depth image corresponding to the following object may be referred to as a partial depth image).
For example, a binocular parallax image pickup device may be disposed at the front side of the foot robot so that a binocular parallax image corresponding to the following object is captured based on the binocular parallax image pickup device, and then a depth map calculation is performed on the binocular parallax image to obtain a partial depth image which can be used to describe depth information of a contour region of the following object with respect to an image pickup device disposed on the foot robot, or a partial depth image may be acquired in any other possible manner, such as a structured light manner, a Time of flight (TOF) manner, without limitation.
In addition, the above-mentioned scene image, specifically, the foot robot, is based on certain position data, and controls the camera device arranged on the foot robot to capture the scene image, so that when the foot robot follows and moves to different positions, and captures a plurality of scene images, corresponding scene images generally correspond to different position data, so that in the embodiment of the application, after the acquired plurality of local depth images, a plurality of position data corresponding to the foot robot can be determined, wherein the foot robot controls the camera device to capture the corresponding scene image based on the position data, and then the position data, the local depth images and the corresponding target scene images can be used for assisting in generating the target map, thereby realizing that related data for generating the target map is captured autonomously in the process of controlling the foot robot to follow and move based on the following object.
The embodiment of the application can support real-time processing of each acquired target scene image, the local depth image and the position data corresponding to the target scene image, or can support processing of the acquired data after the follow-up operation is finished so as to assist in image construction, and is not limited to the processing.
In some embodiments, the target map is generated according to the plurality of target scene images, the plurality of local depth images and the plurality of position data, which may be that a point cloud sub-map corresponding to the position data is generated according to each target scene image, the corresponding local depth image and the corresponding position data, and the plurality of point cloud sub-maps are spliced and fused to obtain a point cloud map, and the point cloud map is processed into a grid map, and the grid map is used as the target map, so that autonomous following control of the legged robot and a map building method of a three-dimensional dense point cloud map are effectively fused, the generating efficiency of the target map is improved, two-dimensional or three-dimensional map building of the surrounding environment is completed while the legged robot follows the moving object, and in the map building process, the operator can be free to guide the legged robot to a scene area of a desired map.
In this embodiment, by acquiring a current scene image, identifying a following object from the current scene image, determining a first local image corresponding to the following object, and updating the current scene image at least once in a process of controlling the foot robot to follow movement based on the following object, so as to obtain at least one new scene image, where the new scene image includes: the first partial images corresponding to the following objects and the target map is generated according to the scene images and the first partial images, and the map is automatically generated in the autonomous following process of the foot robot, so that manual operation of a user is not needed, the following path planning by the following objects can be supported, the labor cost and the learning cost of map generation can be effectively reduced, the map generation efficiency is improved, the map generation accuracy is improved, and the map generation effect is improved. The following object is accurately positioned from the scene images by identifying at least one detection image from the current scene images, wherein the detection image is an image comprising candidate objects, determining image characteristics corresponding to the images of the candidate objects, determining target image characteristics from at least one image characteristics and taking the candidate objects to which the target image characteristics belong as following objects, so that the following efficiency and the following accuracy are ensured, and the image characteristics corresponding to the images of the candidate objects are obtained through analysis, so that the scene images can be effectively assisted in subsequent processing. After the target background image is adopted to replace a second local image in the scene image, the natural connection of the boundaries of the target background image and the second local image can be ensured, and the capturing quality of the scene image is ensured, so that the quality of a map generated subsequently is ensured. The method realizes that related data for generating a target map is autonomously captured in the process of controlling the foot robot to follow the movement based on the following object.
Fig. 3 is a schematic structural diagram of a map generating apparatus according to an embodiment of the present application.
As shown in fig. 3, the map generating apparatus may be used for a foot robot, and the apparatus 30 includes:
An acquiring module 301, configured to acquire a current scene image;
an identifying module 302, configured to identify a following object from among current scene images, and determine a first local image corresponding to the following object;
an updating module 303, configured to update, during the process of controlling the foot robot to follow the movement based on the following object, the current scene image at least once to obtain at least one new scene image, where the new scene image includes: a first partial image corresponding to the following object; and
The generating module 304 is configured to generate a target map according to the plurality of scene images and the plurality of first partial images.
In some embodiments of the present application, the identification module 302 is specifically configured to:
identifying at least one detection image from the current scene image, wherein the detection image is an image comprising candidate objects;
determining image features corresponding to the images of the candidate objects;
and determining the target image characteristic from at least one image characteristic, and taking the candidate object to which the target image characteristic belongs as a following object.
In some embodiments of the present application, as shown in fig. 4, fig. 4 is a schematic structural diagram of a map generating apparatus according to another embodiment of the present application, and a generating module 304 includes:
an identifying sub-module 3041, configured to identify a second partial image from among the scene images, where the second partial image corresponds to the candidate object;
A processing sub-module 3042, configured to perform target processing on the scene image according to the second local image, so as to obtain a target scene image;
a generation sub-module 3043 for generating a target map from the plurality of target scene images and the plurality of first partial images.
In some embodiments of the present application, the processing sub-module 3042 is specifically configured to:
determining a target background image from the scene image;
and replacing a second local image in the scene image with the target background image to obtain a target scene image.
In some embodiments of the application, the foot robot configures an imaging device, wherein the generating sub-module 3043 is specifically configured to:
Generating a plurality of local depth images corresponding to the plurality of first local images respectively;
determining a plurality of position data corresponding to the foot robot, wherein the foot robot controls the camera device to capture a corresponding scene image based on the position data;
A target map is generated from the plurality of target scene images, the plurality of local depth images, and the plurality of position data.
In some embodiments of the present application, the generating submodule 3043 is specifically configured to:
Generating a point cloud sub-map corresponding to the position data according to each target scene image, the local depth image corresponding to each target scene image and the position data corresponding to each target scene image;
performing splicing and fusion processing on the plurality of point cloud sub-maps to obtain a point cloud map;
The point cloud map is processed as a grid map, and the grid map is taken as a target map.
In some embodiments of the application, the follower object is a user in a scene in a moving state, or a movable object in a scene in a controlled state of motion.
It should be noted that the foregoing explanation of the map generating method embodiment is also applicable to the map generating apparatus of this embodiment, and will not be repeated here.
In this embodiment, by acquiring a current scene image, identifying a following object from the current scene image, determining a first local image corresponding to the following object, and updating the current scene image at least once in a process of controlling the foot robot to follow movement based on the following object, so as to obtain at least one new scene image, where the new scene image includes: the first partial images corresponding to the following objects and the target map is generated according to the scene images and the first partial images, and the map is automatically generated in the autonomous following process of the foot robot, so that manual operation of a user is not needed, the following path planning by the following objects can be supported, the labor cost and the learning cost of map generation can be effectively reduced, the map generation efficiency is improved, the map generation accuracy is improved, and the map generation effect is improved.
Fig. 5 is a schematic structural diagram of a foot robot according to an embodiment of the present application.
The foot robot includes:
memory 501, processor 502, and a computer program stored on memory 501 and executable on processor 502.
The processor 502 implements the map generation method provided in the above embodiment when executing a program.
In one possible implementation, the foot robot further comprises:
A communication interface 503 for communication between the memory 501 and the processor 502.
Memory 501 for storing a computer program executable on processor 502.
The memory 501 may include high-speed RAM memory and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
A processor 502 for implementing the map generation method of the above embodiment when executing a program.
If the memory 501, the processor 502, and the communication interface 503 are implemented independently, the communication interface 503, the memory 501, and the processor 502 may be connected to each other via a bus and perform communication with each other. The bus may be an industry standard architecture (Industry Standard Architecture, abbreviated ISA) bus, an external device interconnect (PERIPHERAL COMPONENT, abbreviated PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated EISA) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, only one thick line is shown in fig. 5, but not only one bus or one type of bus.
Alternatively, in a specific implementation, if the memory 501, the processor 502, and the communication interface 503 are integrated on a chip, the memory 501, the processor 502, and the communication interface 503 may perform communication with each other through internal interfaces.
The processor 502 may be a central processing unit (Central Processing Unit, abbreviated as CPU), or an Application SPECIFIC INTEGRATED Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the application.
The present embodiment also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the map generation method as above.
In order to implement the above embodiments, the present application also proposes a computer program product which, when executed by a processor, performs the map generation method shown in the above embodiments.
It should be noted that in the description of the present application, the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Furthermore, in the description of the present application, unless otherwise indicated, the meaning of "a plurality" is two or more.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and further implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or a portion of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, and where the program, when executed, includes one or a combination of the steps of the method embodiments.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product.
The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, or the like.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the present application have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the application, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the application.

Claims (12)

1. A map generation method for a foot robot, the method comprising:
Acquiring a current scene image;
identifying a following object from the current scene image, and determining a first local image corresponding to the following object;
in the process of controlling the foot robot to follow the movement based on the following object, updating the current scene image at least once to obtain at least one new scene image, wherein the new scene image comprises the following components: a first partial image corresponding to the following object; and
Generating a target map according to a plurality of the scene images and a plurality of the first partial images;
The generating a target map from the plurality of the scene images and the plurality of the first partial images includes:
Identifying a second partial image from among the scene images, the second partial image corresponding to a candidate object;
performing target processing on the scene image according to the second local image to obtain a target scene image;
Generating a target map according to a plurality of target scene images and a plurality of first partial images;
the performing target processing on the scene image according to the second local image to obtain a target scene image, including:
determining a target background image from the scene image;
And replacing a second local image in the scene image with the target background image to obtain the target scene image.
2. The method of claim 1, wherein the identifying a follower object from among the current scene images comprises:
Identifying at least one detection image from among the current scene images, wherein the detection image is an image comprising candidate objects;
Determining image features corresponding to images of the candidate object;
And determining a target image feature from at least one image feature, and taking a candidate object to which the target image feature belongs as the following object.
3. The method of claim 2, wherein the foot robot configures a camera, wherein the generating a target map from the plurality of target scene images and the plurality of first partial images based on the camera capturing the scene images comprises:
generating a plurality of local depth images corresponding to the first local images respectively;
Determining a plurality of position data corresponding to the foot robot, wherein the foot robot controls the camera device to capture a corresponding scene image based on the position data;
The target map is generated from a plurality of the target scene images, the plurality of local depth images, and the plurality of position data.
4. The method of claim 3, wherein the generating the target map from a plurality of the target scene images, the plurality of local depth images, and the plurality of location data comprises:
generating a point cloud sub-map corresponding to the position data according to each target scene image, the local depth image corresponding to each target scene image and the position data corresponding to each target scene image;
Performing splicing and fusion processing on the plurality of point cloud sub-maps to obtain a point cloud map;
and processing the point cloud map into a grid map, and taking the grid map as the target map.
5. The method of any of claims 1-4, wherein the following object is a user in a moving state in the scene or a movable object in the scene with a controlled motion state.
6. A map generation apparatus for a foot-type robot, the apparatus comprising:
the acquisition module is used for acquiring the current scene image;
The identification module is used for identifying a following object from the current scene image and determining a first local image corresponding to the following object;
the updating module is used for updating the current scene image at least once in the process of controlling the foot robot to follow the movement based on the following object so as to obtain at least one new scene image, and the new scene image comprises the following components: a first partial image corresponding to the following object; and
A generation module for generating a target map according to a plurality of the scene images and a plurality of the first partial images;
The generating module comprises:
An identification sub-module, configured to identify a second local image from among the scene images, where the second local image corresponds to a candidate object;
The processing sub-module is used for carrying out target processing on the scene image according to the second local image so as to obtain a target scene image;
a generation sub-module for generating a target map according to a plurality of the target scene images and a plurality of the first partial images;
the processing submodule is specifically configured to:
determining a target background image from the scene image;
And replacing a second local image in the scene image with the target background image to obtain the target scene image.
7. The apparatus of claim 6, wherein the identification module is specifically configured to:
Identifying at least one detection image from among the current scene images, wherein the detection image is an image comprising candidate objects;
Determining image features corresponding to images of the candidate object;
And determining a target image feature from at least one image feature, and taking a candidate object to which the target image feature belongs as the following object.
8. The apparatus of claim 7, wherein the foot robot configures an imaging device, wherein the generating sub-module is specifically configured to, based on the imaging device capturing the scene image:
generating a plurality of local depth images corresponding to the first local images respectively;
Determining a plurality of position data corresponding to the foot robot, wherein the foot robot controls the camera device to capture a corresponding scene image based on the position data;
The target map is generated from a plurality of the target scene images, the plurality of local depth images, and the plurality of position data.
9. The apparatus of claim 8, wherein the generating sub-module is specifically configured to:
generating a point cloud sub-map corresponding to the position data according to each target scene image, the local depth image corresponding to each target scene image and the position data corresponding to each target scene image;
Performing splicing and fusion processing on the plurality of point cloud sub-maps to obtain a point cloud map;
and processing the point cloud map into a grid map, and taking the grid map as the target map.
10. The apparatus of any of claims 6-9, wherein the following object is a user in a moving state in the scene or a movable object in the scene with a controlled motion state.
11. A foot robot, comprising:
memory, a processor and a computer program stored on the memory and executable on the processor, which processor implements the map generation method according to any of claims 1-5 when executing the program.
12. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the map generation method as claimed in any one of claims 1-5.
CN202110753733.2A 2021-07-02 2021-07-02 Map generation method, map generation device, foot robot and storage medium Active CN115565057B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110753733.2A CN115565057B (en) 2021-07-02 2021-07-02 Map generation method, map generation device, foot robot and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110753733.2A CN115565057B (en) 2021-07-02 2021-07-02 Map generation method, map generation device, foot robot and storage medium

Publications (2)

Publication Number Publication Date
CN115565057A CN115565057A (en) 2023-01-03
CN115565057B true CN115565057B (en) 2024-05-24

Family

ID=84736568

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110753733.2A Active CN115565057B (en) 2021-07-02 2021-07-02 Map generation method, map generation device, foot robot and storage medium

Country Status (1)

Country Link
CN (1) CN115565057B (en)

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101556154A (en) * 2008-10-13 2009-10-14 美新半导体（无锡）有限公司 Positioning and path map generation system and data acquisition analysis method thereof
CN101619984A (en) * 2009-07-28 2010-01-06 重庆邮电大学 Mobile robot visual navigation method based on colorful road signs
CN102087530A (en) * 2010-12-07 2011-06-08 东南大学 Vision navigation method of mobile robot based on hand-drawing map and path
CN102288176A (en) * 2011-07-07 2011-12-21 中国矿业大学(北京) Coal mine disaster relief robot navigation system based on information integration and method
CN103345258A (en) * 2013-06-16 2013-10-09 西安科技大学 Target tracking method and system of football robot
CN104036524A (en) * 2014-06-18 2014-09-10 哈尔滨工程大学 Fast target tracking method with improved SIFT algorithm
WO2017088720A1 (en) * 2015-11-26 2017-06-01 纳恩博（北京）科技有限公司 Method and device for planning optimal following path and computer storage medium
CN110298269A (en) * 2019-06-13 2019-10-01 北京百度网讯科技有限公司 Scene image localization method, device, equipment and readable storage medium storing program for executing
CN110861082A (en) * 2019-10-14 2020-03-06 北京云迹科技有限公司 Auxiliary mapping method and device, mapping robot and storage medium
CN111060924A (en) * 2019-12-02 2020-04-24 北京交通大学 SLAM and target tracking method
CN111835941A (en) * 2019-04-18 2020-10-27 北京小米移动软件有限公司 Image generation method and device, electronic equipment and computer readable storage medium
CN112396675A (en) * 2019-08-12 2021-02-23 北京小米移动软件有限公司 Image processing method, device and storage medium
CN112884835A (en) * 2020-09-17 2021-06-01 中国人民解放军陆军工程大学 Visual SLAM method for target detection based on deep learning

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11462054B2 (en) * 2019-10-21 2022-10-04 Analog Devices International Unlimited Company Radar-based indoor localization and tracking system

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101556154A (en) * 2008-10-13 2009-10-14 美新半导体（无锡）有限公司 Positioning and path map generation system and data acquisition analysis method thereof
CN101619984A (en) * 2009-07-28 2010-01-06 重庆邮电大学 Mobile robot visual navigation method based on colorful road signs
CN102087530A (en) * 2010-12-07 2011-06-08 东南大学 Vision navigation method of mobile robot based on hand-drawing map and path
CN102288176A (en) * 2011-07-07 2011-12-21 中国矿业大学(北京) Coal mine disaster relief robot navigation system based on information integration and method
CN103345258A (en) * 2013-06-16 2013-10-09 西安科技大学 Target tracking method and system of football robot
CN104036524A (en) * 2014-06-18 2014-09-10 哈尔滨工程大学 Fast target tracking method with improved SIFT algorithm
WO2017088720A1 (en) * 2015-11-26 2017-06-01 纳恩博（北京）科技有限公司 Method and device for planning optimal following path and computer storage medium
CN111835941A (en) * 2019-04-18 2020-10-27 北京小米移动软件有限公司 Image generation method and device, electronic equipment and computer readable storage medium
CN110298269A (en) * 2019-06-13 2019-10-01 北京百度网讯科技有限公司 Scene image localization method, device, equipment and readable storage medium storing program for executing
CN112396675A (en) * 2019-08-12 2021-02-23 北京小米移动软件有限公司 Image processing method, device and storage medium
CN110861082A (en) * 2019-10-14 2020-03-06 北京云迹科技有限公司 Auxiliary mapping method and device, mapping robot and storage medium
CN111060924A (en) * 2019-12-02 2020-04-24 北京交通大学 SLAM and target tracking method
CN112884835A (en) * 2020-09-17 2021-06-01 中国人民解放军陆军工程大学 Visual SLAM method for target detection based on deep learning

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Feature Detection Performance Based Benchmarking of Motion Deblurring Methods: Applications to Vision for Legged Robots;Gokhan Koray Gultekin;《Image and Vision Computing》;论文全文 *
基于分数阶的电动轮足式机器人腿部阻抗控制研究;赵江波;《北京理工大学学报》;论文全文 *

Also Published As

Publication number Publication date
CN115565057A (en) 2023-01-03

Similar Documents

Publication Publication Date Title
EP3171334B1 (en) Pose estimation apparatus and vacuum cleaner system
CN111442722A (en) Positioning method, positioning device, storage medium and electronic equipment
US20140334713A1 (en) Method and apparatus for constructing map for mobile robot
CN108062763B (en) Target tracking method and device and storage medium
CN111923011B (en) Live working execution method and device and live working system
CN109965781B (en) Control method, device and system for collaborative work of sweeping robot
CN110245567B (en) Obstacle avoidance method and device, storage medium and electronic equipment
CN112286185B (en) Sweeping robot, three-dimensional map building method and system thereof and computer readable storage medium
CN113902721A (en) Workpiece position adjusting method, control processing device and adjusting system
CN115336250A (en) Photographing instruction method, photographing instruction device, and photographing device
CN110348351B (en) Image semantic segmentation method, terminal and readable storage medium
CN115471731A (en) Image processing method, image processing apparatus, storage medium, and device
CN115565057B (en) Map generation method, map generation device, foot robot and storage medium
CN112352417A (en) Focusing method of shooting device, system and storage medium
CN115220375A (en) Robot control method, robot control device, storage medium, and electronic apparatus
US9030501B2 (en) Methods and systems for modifying a display of a field of view of a robotic device to include zoomed-in and zoomed-out views
CN111935389B (en) Shot object switching method and device, shooting equipment and readable storage medium
CN115703234B (en) Robot control method, device, robot and storage medium
CN113450414A (en) Camera calibration method, device, system and storage medium
CN112907654B (en) Method and device for optimizing external parameters of multi-camera, electronic equipment and storage medium
CN110444102B (en) Map construction method and device and unmanned equipment
CN114610035A (en) Pile returning method and device and mowing robot
CN115480631A (en) Gesture recognition method, terminal control method, device, terminal and storage medium
KR102592450B1 (en) Apparatus for improving performance of vision recognition algorithm for autonomous driving and control method thereof
CN111738906A (en) Indoor road network generation method and device, storage medium and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant