CN116820075A - Navigation method, navigation device, robot and storage medium - Google Patents
Navigation method, navigation device, robot and storage medium Download PDFInfo
- Publication number
- CN116820075A CN116820075A CN202211060512.8A CN202211060512A CN116820075A CN 116820075 A CN116820075 A CN 116820075A CN 202211060512 A CN202211060512 A CN 202211060512A CN 116820075 A CN116820075 A CN 116820075A
- Authority
- CN
- China
- Prior art keywords
- robot
- pose
- camera
- determining
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000003860 storage Methods 0.000 title claims abstract description 10
- 230000000007 visual effect Effects 0.000 claims abstract description 16
- 230000033001 locomotion Effects 0.000 claims description 53
- 238000004590 computer program Methods 0.000 claims description 16
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 6
- 238000013507 mapping Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000005457 optimization Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 241000255969 Pieris brassicae Species 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000005111 flow chemistry technique Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Landscapes
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The application is applicable to the technical field of visual navigation, and provides a navigation method, a navigation device, a robot and a storage medium, wherein the method is applied to the robot, and the robot is provided with a camera; determining the current pose of a camera when shooting the first image; determining a reference frame matched with the first image from prestored key frames, wherein the key frames are second images shot by a camera of the robot moving on a planned path, and the texture richness of the second images is larger than a preset value; and controlling the robot to move according to the current pose and the corresponding target pose when the camera shoots the reference frame, so that the camera moves to the target pose. The application is beneficial to the fact that abundant textures exist in the visual field of the robot all the time, so that the robot can automatically cruise on a given path accurately.
Description
Technical Field
The application belongs to the technical field of robot vision, and particularly relates to a navigation method, a navigation device, a robot and a storage medium.
Background
Visual autonomous navigation of a mobile robot is an important research field in robotics, and one of important tasks is to enable the mobile robot to autonomously navigate on a given path; in this task, the farther the mobile robot deviates from a given path, the higher the risk; reducing the deviation of the mobile robot from a given path depends on the presence of rich textures in the environment in which the mobile robot is located.
However, for some environments with weak textures, because the field of view of a camera mounted on the mobile robot is smaller, the observation range is limited, and the fact that abundant textures exist in the field of view of the camera all the time cannot be ensured, so that the mobile robot is easy to be abnormal or terminated when autonomously navigated on a given path, and the safety of the mobile robot in the navigation process is lower.
Disclosure of Invention
In view of the above, the embodiments of the present application provide a navigation method, a navigation device, a robot, and a storage medium, so as to solve the problem that in the prior art, it is not guaranteed that abundant textures exist in a camera view of the robot all the time.
In a first aspect, an embodiment of the present application provides a navigation method applied to a robot, where a camera is installed on the robot, the method including:
acquiring a first image shot by the camera;
determining a current pose of the camera when shooting the first image;
determining a reference frame matched with the first image from prestored key frames, wherein the key frames are second images shot by the camera when the robot moves on a planned path, and the texture richness of the second images is larger than a preset value;
and controlling the robot to move according to the current pose and the target pose corresponding to the camera shooting the reference frame, so that the camera moves to the target pose.
According to the navigation method provided by the embodiment of the application, the terminal equipment firstly acquires the shooting image of the space, then determines the reference frame matched with the shooting image from the prestored key frame, so that a strong association relation is initially established between the robot and the given path, and then takes the pose of the camera when the camera shoots the reference frame as the target pose of the camera, the robot is controlled to move, the camera is moved to the target pose, so that the fact that abundant textures exist in the visual field of the camera all the time is ensured, and the robot can be ensured to accurately realize autonomous navigation on the planned path under the weak texture scene.
In a second aspect, an embodiment of the present application provides a navigation device, the device including:
the image acquisition module is used for acquiring a first image shot by the camera;
the pose determining module is used for determining the current pose of the camera according to the first image;
the reference frame determining module is used for determining a reference frame matched with the first image from prestored key frames, wherein the key frames are second images shot by the camera when the robot moves on a planned path, and the texture richness of the second images is larger than a preset value;
and the motion control module is used for controlling the robot to move according to the current pose and the target pose corresponding to the camera shooting the reference frame, so that the camera moves to the target pose.
In a third aspect, an embodiment of the present application provides a robot comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the method according to the first aspect as described above when the computer program is executed.
In a fourth aspect, embodiments of the present application provide a computer readable storage medium storing a computer program which, when executed by a processor, implements the steps of the method of the first aspect described above.
It will be appreciated that the advantages of the second to fourth aspects may be found in the relevant description of the first aspect and are not repeated here.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings used in the description of the embodiments or the prior art will be briefly described below.
FIG. 1 is a schematic diagram of a given path provided by an embodiment of the present application;
FIG. 2 is a flow chart of a navigation method according to an embodiment of the present application;
fig. 3 is a flowchart of a navigation method step S200 according to an embodiment of the present application;
fig. 4 is a flowchart of a navigation method step S210 according to an embodiment of the present application;
FIG. 5 is a flowchart of a navigation method step S310 according to an embodiment of the present application;
FIG. 6 is a flowchart of a navigation method step S410 according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a navigation device according to an embodiment of the present application;
fig. 8 is a schematic structural view of a robot according to an embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
In the description of the present specification and the appended claims, the terms "first," "second," "third," and the like are used merely to distinguish between descriptions and are not to be construed as indicating or implying relative importance.
Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.
In order to illustrate the technical scheme of the application, the following description is made by specific examples.
In this embodiment, the execution subject of the navigation method is a terminal device. It will be appreciated that the terminal device may be a robot, or a mobile phone, a tablet computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (personal digital assistant, PDA) or the like, which are installed on the robot, and the embodiment of the present application does not limit the specific type of the terminal device.
The navigation method provided by the embodiment of the application is described below by taking the terminal device as an example of a robot.
Generally, in the daily life scene of people, there is almost no scene without texture at all, such as a corridor with large white walls on both sides, and the ground or the ceiling has rich texture. Therefore, in a weak texture scene, there is also a region of rich texture.
Without loss of generality, the preset space where the planned path is located may be mapped before the robot performs the navigation method. To increase the flexibility of the camera, a robot arm is mounted on the robot, which may be a six-degree-of-freedom arm, such as the arm model mechmarm, and the camera is mounted on the side of the arm remote from the robot. Referring to fig. 1, fig. 1 is a top view, so the mechanical arm is not shown; in fig. 1, the included angle area of two adjacent non-arrowed dashed lines represents the field of view of the camera, the dashed box represents the space in which the robot is located, and the box with letter a, the box with letter B, and the box with letter C each represent a partially textured region in space, with arrowed dashed lines representing a given path. Illustratively, when the robot moves on the planned path, the camera is in an open state and images can be acquired at a speed of 30 frames per second, and at the same time, an operator can manually control the robot and the camera through a remote controller to aim the camera at a region with rich texture in the space, so that the camera can always aim at the region with rich texture when moving on the planned path. In practical applications, the speed at which the camera captures images may be increased appropriately as desired, such as 50 frames per second. It should be noted that the navigation path in fig. 1 is merely an example and is a straight line, and the specific shape is not limited, and in other possible implementations, the motion path may be any shape.
After the robot obtains the multi-frame photographed images, in order to reduce the situation that a large number of photographed images occupy memory and reduce redundancy of information, the robot can select key and representative photographed images, namely key frames, from the multi-frame photographed images.
Specifically, to select the key frame, the terminal device may first determine a feature point of the third image captured by the camera, where the feature point represents a pixel point in the captured image that is significantly different from surrounding pixel points, and the feature point is also called a key point or a corner point, for example, a frame angle of the window frame may be taken as the feature point. Wherein feature points are obtained from the third image of each frame by existing feature point detection techniques such as Scale-invariant feature transform (SIFT), acceleration robust features (Speeded Up Robust Features, SURF) and orientation fast rotation (Oriented Fast and Rotated Brief, ORB); in practical application, the robot can select a corresponding feature point detection technology according to specific requirements.
Then, the robot may preset an information threshold, and the robot may determine whether the third image has enough information according to the information threshold, for example, when the number of feature points included in the third image in a certain frame is greater than or equal to the information threshold, it may indicate that the third image has enough new messages, and the robot may determine that the captured image is a key image, that is, a key frame. In some possible implementations, for any two adjacent frames of third images, determining a co-view landmark in the two adjacent frames of third images, the co-view landmark representing a commonly photographed landmark in the two frames of third images; if the co-view landmark of the third image of the subsequent frame is less than 90%, indicating that the third image of the subsequent frame generates 10% new information with respect to the third image of the previous frame, the robot can determine the third image of the subsequent frame as a key frame. In practical application, the specific value of the information threshold and the specific value of the co-view road mark ratio can be selected according to specific requirements, and whether the photographed image of each frame has enough information or not can be judged according to the information threshold by adopting the prior art, so that the description is omitted.
After the robot determines the key frames, descriptors can be respectively established for the characteristic points of all the key frames, and because the descriptors have higher uniqueness, the influence of external factors such as illumination change, visual angle change and the like on the characteristic points can be reduced, so that the robustness of the data can be improved.
After the robot establishes descriptors for the feature points of all the key frames, the robot can respectively input each key frame and the corresponding descriptors into a preset visual word bag model, and establishes a visual feature word bag library, so that the reference frames can be conveniently and efficiently found in the subsequent navigation process. The visual feature word bag library stores classification information of each image feature. It should be noted that, the prior art may be used for both establishing the feature descriptors of each feature point and establishing the visual feature word bag library through the visual word bag model, so that description is omitted.
After the key frame is determined, the robot can perform sparse three-dimensional reconstruction on the preset space according to the key frame based on synchronous positioning and mapping (Simultaneous Localization And Mapping, SLAM) technology to obtain a sparse map, and the sparse map is used as a map of the preset space in the subsequent navigation process. For example, the position of the planned path can be determined according to the sparse map, and the pose of the robot at each moment is the pose in the sparse map. Meanwhile, compared with a map obtained by mapping each frame of image, the sparse map is built, so that the calculated amount in the mapping process can be reduced, and the calculation efficiency is improved. And determining the pose of the camera corresponding to the key frame through the coordinates of the feature points in the key frame, the preset internal reference matrix of the camera and the positions of the feature points in the space. Meanwhile, according to the corresponding relation between the pose of the camera and the pose of the mechanical arm and the pose of the robot, the pose of the mechanical arm corresponding to the key frame and the pose of the robot are determined, wherein the pose of the robot can be the pose of the center of the robot.
In an embodiment, the robot determines the pose corresponding to the keyframe by a nonlinear optimization method. Specifically, when different key frames shoot the same road sign, the same feature points exist. And determining the associated key frames according to the number of the same characteristic points in the different key frames. In the associated key frames, for any two key frames, there is a coordinate error when feature points in one key frame are mapped into the other key frame. Each key frame comprises a plurality of feature points, when mapping from one key frame to another key frame, the accumulated errors corresponding to all the feature points are determined, the camera pose and map point information with the smallest accumulated errors are determined, the camera pose with the smallest accumulated errors is used as the pose corresponding to the key frame, and the corresponding map point information is used as the map point in the sparse map.
The robot can firstly determine the predicted pose, and takes the predicted pose as an initial value of a nonlinear optimization method to obtain the pose corresponding to the key frame, so that the calculation efficiency of determining the pose corresponding to the key frame can be improved. When the robot is started, the robot can firstly determine the repositioning pose, then take the repositioning pose as the pose at the current moment, and then determine the predicted pose at the current moment according to the pose at the last moment and the uniform motion model of the camera. The repositioning pose can be a preset pose or can be determined according to an image shot by the robot in real time.
The navigation method provided by the embodiment of the application is described below, and can be applied to a robot, a mechanical arm is arranged on the robot, and a camera is arranged on one side of the mechanical arm far away from the robot. Referring to fig. 2, fig. 2 shows a flow chart of a navigation method, which is described in detail as follows:
in S100, a first image captured by a camera is acquired.
Specifically, the first image is an image photographed in real time by a camera during movement of the robot.
In S200, a current pose of the camera when capturing the first image is determined.
Specifically, in order to facilitate the robot to realize accurate autonomous navigation on the given path, the terminal device may determine the current pose of the camera according to the first image, thereby ensuring that the robot does not deviate from the given path.
In some possible implementations, to enable the terminal device to determine the current pose of the camera, as shown in fig. 3, step S200 includes, but is not limited to, the following steps:
s210, determining the pose of the camera at the last moment.
Specifically, the pose at the previous time is the actual pose at the previous time of the camera, and can be determined according to the control amount for driving the camera.
S220, determining the current pose according to the pose at the previous moment and the uniform motion model of the camera.
Specifically, the terminal device may determine the predicted pose at the current moment according to the pose at the previous moment and the uniform motion model of the camera, and by combining with a nonlinear optimization technology.
In some possible implementations, as shown in fig. 4, when the robot is just started, the robot cannot determine the current pose from the pose at the previous moment, so the repositioning pose can be determined by the following method, and the repositioning pose is taken as the current pose.
S211, determining a first bag-of-words representation form corresponding to the initial image according to the initial image shot by the camera and a preset visual characteristic bag-of-words library.
In particular, the terminal device may capture, from an initial image captured by the camera, i.e. key frames captured when creating the navigation path,determining a first word bag representation form corresponding to the initial image according to a preset visual characteristic word bag library; first bag of words representation such as Image = { (W) 1 ,1),(W 2 ,2),…,(W K K) }, wherein W 1 To W K Representing the clustered descriptors in the initial image,ω 1 to omega K The weight corresponding to the descriptor is represented, and the weight calculation formula may be:
ω K ＝TF k ×IDF k ，
in the formula, TF k Representing the frequency of occurrence of the belonging classification in each frame of key image,representing the frequency of the belonging classification in the key image;
an inverse text frequency index representing the kth belonging class, k being a positive integer, ++>Representing the proportion of the number of features of the belonging classification in the bag of words.
S212, determining a target key frame matched with the initial image from the key frames according to the first word bag representation form and the second word bag representation form corresponding to the key frames.
Specifically, the method for determining the second bag-of-words representation is the same as the method for determining the first bag-of-words representation. The robot may compare each descriptor of the first bag-of-word representation and the second bag-of-word representation with a corresponding weight to determine a target keyframe from the keyframes that matches the initial image.
S213, determining repositioning pose according to the camera pose corresponding to the target key frame.
Specifically, the terminal device may use the camera pose corresponding to the target key frame as a repositioning pose, that is, the current pose of the robot when the robot is started. In S300, a reference frame matching the first image is determined from among pre-stored key frames.
Specifically, the key frame is a second image captured by a camera of the robot moving on the planned path, and it is emphasized that the second image is determined from a third image according to the walking direction and the euclidean distance, and the texture richness of the third image is greater than a preset value. The image with the rich texture degree larger than the preset value is an image capable of realizing robot navigation. The terminal device can determine a reference frame matched with the first image from a plurality of prestored key frames according to the descriptors and the weights of the feature points, so that the first image is matched with the reference frame.
In some possible implementations, to further reduce the deviation of the robot from a given path, as shown in fig. 5, the steps at step S300 include, but are not limited to, the following steps:
and S310, determining the position and the length of the sliding window according to the motion state and the current pose of the robot.
Specifically, the motion state of the robot includes an initial state, a halfway state and an end state, the robot can determine the sliding window position and the sliding window length of the matched sliding window according to the motion state and the current pose of the robot, the sliding window can select a designated key frame as a processing object, and the number of shot images as the processing object is limited, so that the search range is reduced, and the calculation speed is improved.
And S320, determining candidate reference frames from the key frames according to the sliding window positions and the sliding window lengths.
Specifically, the key frame selected according to the sliding window position and the sliding window length is the candidate reference frame.
And S330, determining the reference frame from the candidate reference frames according to the movement direction of the robot and the Euclidean distance between the first image and the candidate reference frames.
For example, when the robot moves forward, the reference frame whose photographed content is the area in front of the robot may be selected from the candidate reference frames, and when the robot moves backward, the reference frame whose photographed content is the area in back of the robot may be selected from the candidate reference frames. And then respectively calculating Euclidean distances between the first image and a plurality of candidate reference frames, and finally selecting a shooting image with the minimum Euclidean distance as the reference frame, wherein the Euclidean distances between images of different frames can be calculated by adopting the prior art, so that the description is omitted.
In S400, according to the current pose and the target pose corresponding to the camera shooting the reference frame, the robot is controlled to move, so that the camera moves to the target pose.
Specifically, the camera moves to a target pose, which is beneficial for the autonomous navigation of the robot on a given path.
In some possible implementations, to achieve the camera moving to the target pose, as shown in fig. 6, steps S400 include, but are not limited to, the following:
s410, determining a motion vector of the camera according to the current pose and the target pose.
For example, the camera pose corresponding to the reference frame is the target pose, and the motion scalar of the camera in the X-axis, the motion scalar in the YaW axis and the motion scalar in the Z-axis can be calculated according to the current pose and the target pose.
S420, determining the motion vector of the robot according to the pose of the mechanical arm corresponding to the current pose, the pose of the robot corresponding to the current pose, the pose of the mechanical arm corresponding to the target pose and the pose of the robot corresponding to the target pose.
Specifically, the motion vector of the robot refers to the motion vector of the whole robot, the motion vector of the robot can be determined according to the pose of the mechanical arm corresponding to the current pose and the pose of the mechanical arm corresponding to the target pose, the motion vector of the center of the robot is determined according to the pose of the robot corresponding to the current pose and the pose of the robot corresponding to the target pose, and the motion vector of the center of the robot, namely the motion vector of the chassis of the robot, is determined through the motion vector of the mechanical arm and the motion vector of the center of the robot.
S430, controlling the robot to move according to the motion vector of the camera and the motion vector of the robot.
Specifically, the terminal device can cooperatively control the machine to move to the target pose according to the motion vector of the camera and the motion vector of the robot, so that the pose when the camera, the mechanical arm and the robot move on the planning path in the process of drawing the robot is repeated. For example, the terminal device may input the motion vector of the camera and the motion vector of the center of the robot into the corresponding controllers, respectively, to obtain the control amounts of the camera and the robot. The specific control quantity of the mechanical arm can be obtained by calculation in the prior art, so that the detailed description is omitted; the controller can be a PID controller, and the corresponding calculation formula of the control quantity is as follows:
wherein U (t) represents a motion vector; k (K) p Representing a preset proportion; err (t) represents an error value; k (K) I Representing an integral; k (K) D Representing a preset differential coefficient.
The implementation principle of the navigation method in the embodiment of the application is as follows: if a camera fixed on the robot chassis is used, the construction of the image fails when a scene with weak texture is encountered, unless the motion path of the robot chassis is selected intentionally by a person, but the motion path is not free and does not necessarily exist.
In the application, a camera is arranged on a mechanical arm of a robot, when the robot moves on a planned path, an operator manually controls the camera through a remote controller, so that the camera aims at a region with rich textures in a space and shoots, a key frame is determined, and the pose of the camera and the pose of the robot when the camera shoots the key frame and a sparse map are stored; when the robot is required to autonomously navigate on a planned path, the robot firstly acquires a first image of a space where the robot is located, calculates the pose of the first image through a PNP technology, then determines a reference frame matched with the first image from prestored key frames, takes the pose of the camera when the camera shoots the reference frame as a target pose of the camera, controls the robot to move, enables the camera to move to the target pose, enables the camera to reproduce the pose when shooting abundant textures, and further ensures that the camera always has abundant textures in the visual field, and the robot chassis can move randomly due to the fact that the mechanical arm can ensure the abundance degree of the textures in the visual field, so that the motion path of the robot chassis is smoother, and the robot is favorable for realizing autonomous navigation on a given path accurately even under a weak texture scene.
It should be noted that, the sequence number of each step in the above embodiment does not mean the sequence of execution sequence, and the execution sequence of each process should be determined by its function and internal logic, and should not limit the implementation process of the embodiment of the present application in any way.
Embodiments of the present application also provide a navigation device, only a portion relevant to the present application is shown for convenience of description, as shown in fig. 7, the device 700 includes:
an image acquisition module 710, configured to acquire a first image captured by a camera;
a pose determining module 720, configured to determine a current pose of the camera when the camera captures the first image;
the reference frame determining module 730 is configured to determine a reference frame that matches the first image from prestored key frames, where the key frames are second images captured by a camera when the robot moves on the planned path, and the texture richness of the second images is greater than a preset value;
the motion control module 740 is configured to control the robot to move according to the current pose and the target pose corresponding to the camera shooting the reference frame, so as to move the camera to the target pose.
Optionally, the reference frame determining module 730 includes:
and the sliding window determining submodule is used for determining the sliding window position and the sliding window length according to the motion state and the current pose of the robot.
And the candidate reference frame determining submodule is used for determining candidate reference frames from the key frames according to the sliding window positions and the sliding window lengths.
The reference frame determining submodule is used for determining the reference frame from the candidate reference frames according to the motion direction of the robot and the Euclidean distance between the first image and the candidate reference frames.
Optionally, the motion control module 740 includes:
the first motion vector determining sub-module is used for determining the motion vector of the camera according to the current pose and the target pose.
The second motion vector determining sub-module is used for determining the motion vector of the robot according to the pose of the mechanical arm corresponding to the current pose, the pose of the robot corresponding to the current pose, the pose of the mechanical arm corresponding to the target pose and the pose of the robot corresponding to the target pose.
And the motion control sub-module is used for controlling the robot to move according to the motion vector of the camera and the motion vector of the robot.
Optionally, the pose determining module 720 includes:
and the pose determining submodule is used for determining the pose of the camera at the last moment.
And the current pose determining submodule is used for determining the current pose according to the pose at the last moment and the uniform motion model of the camera.
Optionally, the apparatus 700 further includes:
the first word bag representation form determining unit is used for determining a first word bag representation form corresponding to the initial image according to the initial image shot by the camera and a preset visual characteristic word bag library.
And the target key frame determining unit is used for determining a target key frame matched with the initial image from the key frames according to the first bag-of-word representation form and the second bag-of-word representation form corresponding to the key frames.
The repositioning pose determining unit is used for determining repositioning pose according to the camera pose corresponding to the target key frame, and the repositioning pose is used as the current pose of the robot when the robot is started.
Optionally, the apparatus 700 further includes:
the sparse map construction module is used for carrying out sparse three-dimensional reconstruction on a preset space according to the key frames to obtain a sparse map, the planned path is located in the preset space, and the current pose is the pose in the sparse map.
Optionally, the apparatus 700 further includes:
and the third image acquisition module is used for acquiring a third image shot by the camera.
And the descriptor determining module is used for determining the number of the characteristic points in the third image and the preset parallax.
And the key frame determining module is used for determining the key frame from the third image according to the number and the preset parallax.
It should be noted that, because the content of information interaction and execution process between the modules and the embodiment of the method of the present application are based on the same concept, specific functions and technical effects thereof may be referred to in the method embodiment section, and details thereof are not repeated herein.
The embodiment of the application also provides a robot, on which a six-degree-of-freedom mechanical arm is mounted, a mobile robot chassis is mounted at the bottom of the robot, and the development framework of the robot can be a robot operating system (Robot Operating System, ROS), as shown in fig. 8, the robot 801 of this embodiment comprises: a processor 802, a memory 803, and a computer program 804 stored in the memory 803 and executable on the processor 802. The processor 802, when executing the computer program 804, implements the steps in the above-described flow processing method embodiment, such as steps S100 to S400 shown in fig. 1; alternatively, processor 802, when executing computer program 804, performs the functions of the modules in the apparatus described above, such as the functions of modules 701 through 704 shown in fig. 7.
The robot 801 includes, but is not limited to, a processor 802, a memory 803. It will be appreciated by those skilled in the art that fig. 8 is merely an example of a robot 801 and is not meant to be limiting of the robot 801, and may include more or fewer components than shown, or may combine certain components, or different components, e.g., the robot 801 may also include input and output devices, network access devices, buses, etc.
The processor 802 may be a central processing unit (Central Processing Unit, CPU), such as a processor of the type Intel Core i7, but also other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.; a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 803 may be an internal storage unit of the robot 801, such as a hard disk or a memory of the robot 801, or the memory 803 may be an external storage device of the robot 801, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like provided on the robot 801; further, the memory 803 may also include both an internal memory unit and an external memory device of the robot 801, the memory 803 may also store a computer program 804 and other programs and data required for the robot 801, and the memory 803 may also be used to temporarily store data that has been output or is to be output.
An embodiment of the present application also provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of the various method embodiments described above. Wherein the computer program comprises computer program code, the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc.; the computer readable medium may include: any entity or device capable of carrying computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth.
The above embodiments are not intended to limit the scope of the present application, so: all equivalent changes in the method, principle and structure of the present application should be covered by the protection scope of the present application.
Claims (10)
1. A navigation method applied to a robot having a camera mounted thereon, the method comprising:
acquiring a first image shot by the camera;
determining a current pose of the camera when shooting the first image;
determining a reference frame matched with the first image from prestored key frames, wherein the key frames are second images shot by the camera when the robot moves on a planned path, and the texture richness of the second images is larger than a preset value;
and controlling the robot to move according to the current pose and the target pose corresponding to the camera shooting the reference frame, so that the camera moves to the target pose.
2. The method of claim 1, wherein the determining a reference frame from pre-stored key frames that matches the first image comprises:
determining a sliding window position and a sliding window length according to the motion state of the robot and the current pose;
determining candidate reference frames from the key frames according to the sliding window positions and the sliding window lengths;
and determining the reference frame from the candidate reference frames according to the movement direction of the robot and the Euclidean distance between the first image and the candidate reference frames.
3. The method of claim 1, wherein the robot comprises a robotic arm, the camera being mounted on the robotic arm; the controlling the robot to move according to the current pose and the target pose corresponding to the camera shooting the reference frame comprises the following steps:
determining a motion vector of the camera according to the current pose and the target pose;
determining a motion vector of the robot according to the pose of the mechanical arm corresponding to the current pose, the pose of the robot corresponding to the current pose, the pose of the mechanical arm corresponding to the target pose and the pose of the robot corresponding to the target pose;
and controlling the robot to move according to the motion vector of the camera and the motion vector of the robot.
4. The method of claim 1, wherein the determining the current pose of the camera when the first image was taken comprises:
determining a pose of the camera at a previous moment;
and determining the current pose according to the pose at the last moment and the uniform motion model of the camera.
5. The method of claim 1, wherein prior to the determining the current pose of the camera when capturing the first image, the method further comprises:
determining a first bag-of-words representation form corresponding to the initial image according to the initial image shot by the camera and a preset visual characteristic bag-of-words library;
determining a target key frame matched with the initial image from the key frames according to the first word bag representation form and the second word bag representation form corresponding to the key frames;
and determining a repositioning pose according to the camera pose corresponding to the target key frame, wherein the repositioning pose is used as the current pose of the robot when the robot is started.
6. The method of claim 1, wherein prior to the acquiring the first image taken by the camera, the method further comprises:
and carrying out sparse three-dimensional reconstruction on a preset space according to the key frame to obtain a sparse map, wherein the planned path is positioned in the preset space, and the current pose is the pose in the sparse map.
7. The method of claim 6, wherein prior to the sparse three-dimensional reconstruction of the preset space from the keyframes, the method further comprises:
acquiring a third image shot by the camera;
determining the number of feature points in the third image and preset parallax;
and determining the key frame from the third image according to the number and the preset parallax.
8. A navigation device, the device comprising:
the image acquisition module is used for acquiring a first image shot by the camera;
the pose determining module is used for determining the current pose of the camera according to the first image;
the reference frame determining module is used for determining a reference frame matched with the first image from prestored key frames, wherein the key frames are second images shot by the camera when the robot moves on a planned path, and the texture richness of the second images is larger than a preset value;
and the motion control module is used for controlling the robot to move according to the current pose and the target pose corresponding to the camera shooting the reference frame, so that the camera moves to the target pose.
9. A robot comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211060512.8A CN116820075A (en) | 2022-08-31 | 2022-08-31 | Navigation method, navigation device, robot and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211060512.8A CN116820075A (en) | 2022-08-31 | 2022-08-31 | Navigation method, navigation device, robot and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116820075A true CN116820075A (en) | 2023-09-29 |
Family
ID=88120880
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211060512.8A Pending CN116820075A (en) | 2022-08-31 | 2022-08-31 | Navigation method, navigation device, robot and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116820075A (en) |
-
2022
- 2022-08-31 CN CN202211060512.8A patent/CN116820075A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107990899B (en) | Positioning method and system based on SLAM | |
| Wang et al. | Development of UAV-based target tracking and recognition systems | |
| Liu et al. | RDS-SLAM: Real-time dynamic SLAM using semantic segmentation methods | |
| US10953545B2 (en) | System and method for autonomous navigation using visual sparse map | |
| CN111325796B (en) | Method and apparatus for determining pose of vision equipment | |
| CN109727288B (en) | System and method for monocular simultaneous localization and mapping | |
| CN109166149B (en) | Positioning and three-dimensional line frame structure reconstruction method and system integrating binocular camera and IMU | |
| CN110568447B (en) | Visual positioning method, device and computer readable medium | |
| CN112179330B (en) | Pose determination method and device of mobile equipment | |
| CN110111388B (en) | Three-dimensional object pose parameter estimation method and visual equipment | |
| Tan et al. | Robust monocular SLAM in dynamic environments | |
| CN110363817B (en) | Target pose estimation method, electronic device, and medium | |
| CN110648397A (en) | Scene map generation method and device, storage medium and electronic equipment | |
| CN111754579A (en) | Method and device for determining external parameters of multi-view camera | |
| CN112967340A (en) | Simultaneous positioning and map construction method and device, electronic equipment and storage medium | |
| Jiang et al. | A 500-fps pan-tilt tracking system with deep-learning-based object detection | |
| CN110348351B (en) | Image semantic segmentation method, terminal and readable storage medium | |
| CN113592706B (en) | Method and device for adjusting homography matrix parameters | |
| CN112258647B (en) | Map reconstruction method and device, computer readable medium and electronic equipment | |
| CN114859370A (en) | Positioning method and apparatus, computer apparatus, and computer-readable storage medium | |
| CN116734834A (en) | Positioning and mapping method and device applied to dynamic scene and intelligent equipment | |
| CN111079535A (en) | Human skeleton action recognition method and device and terminal | |
| CN112085842A (en) | Depth value determination method and device, electronic equipment and storage medium | |
| CN116820075A (en) | Navigation method, navigation device, robot and storage medium | |
| Huang et al. | Image-based localization for indoor environment using mobile phone |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |