CN117570969A - Mower visual positioning method, device, equipment and storage medium - Google Patents
Mower visual positioning method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN117570969A CN117570969A CN202410060045.1A CN202410060045A CN117570969A CN 117570969 A CN117570969 A CN 117570969A CN 202410060045 A CN202410060045 A CN 202410060045A CN 117570969 A CN117570969 A CN 117570969A
- Authority
- CN
- China
- Prior art keywords
- mower
- pose
- positioning
- lawn
- visual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000000007 visual effect Effects 0.000 title claims abstract description 149
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000003860 storage Methods 0.000 title claims abstract description 12
- 239000003550 marker Substances 0.000 claims abstract description 100
- 238000001514 detection method Methods 0.000 claims abstract description 33
- 238000005457 optimization Methods 0.000 claims abstract description 16
- 230000004438 eyesight Effects 0.000 claims abstract description 8
- 239000011159 matrix material Substances 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 13
- 230000009466 transformation Effects 0.000 claims description 11
- 238000010586 diagram Methods 0.000 claims description 8
- 230000004807 localization Effects 0.000 claims 1
- 230000036544 posture Effects 0.000 description 13
- 230000008569 process Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000000605 extraction Methods 0.000 description 3
- 238000009966 trimming Methods 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 230000001131 transforming effect Effects 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/005—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 with correlation of navigation data from several sources, e.g. map or contour matching
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01D—HARVESTING; MOWING
- A01D34/00—Mowers; Mowing apparatus of harvesters
- A01D34/006—Control or measuring arrangements
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/04—Interpretation of pictures
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
- G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
- G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
- G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
- G01C21/1656—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments with passive imaging devices, e.g. cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Environmental Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Harvester Elements (AREA)
Abstract
The invention relates to the technical field of mowers, and discloses a method, a device, equipment and a storage medium for positioning the vision of a mowing machine, wherein the method comprises the following steps: visual detection is carried out on the marker positioning pile of the lawn where the mower is positioned, and the absolute pose of the marker is obtained; when the mower is in a mowing task, determining the absolute pose of the mower corresponding to the mower through the absolute pose of the marker based on the marker positioning pile; and performing global optimization on the absolute pose of the mower and the visual inertial pose of the camera to obtain the important visual positioning pose. According to the invention, the absolute pose of the lawn mark plate positioning pile is used for determining the absolute pose of the lawn mower, and finally the absolute pose of the lawn mower and the visual inertial pose of the camera of the lawn mower are globally optimized, so that the condition that the lawn mower is easy to accumulate and drift in positioning in an environment shielded by RTK signals is avoided, the traditional visual slam positioning is further optimized, the important sense positioning pose with higher precision is obtained, and the problem of accumulated drift is effectively solved.
Description
Technical Field
The invention relates to the technical field of mowers, in particular to a method, a device, equipment and a storage medium for positioning a mower in a visual mode.
Background
Along with the development of scientific technology, the mower robot can automatically mow according to a planned path under the condition of no participation of personnel, is widely applied to the scenes of maintenance of family courtyard lawns, trimming of large grasslands and the like, and provides great convenience for the production and life of people. The main positioning scheme of the current mower is RTK positioning or visual slam positioning. RTK positioning utilizes differential data between a reference station and a mower robot to measure, so that accurate positioning of the position of the mower robot is realized, and the precision of the RTK positioning can reach a level of a few centimeters. The visual slam positioning adopts a traditional positioning mode based on visual perception, namely, the pose of the mower robot is calculated by collecting the surrounding environment images of the mower robot and extracting the characteristics in the natural environment.
The former has higher absolute positioning accuracy when the RTK signal is better, but has the defects that the RTK signal is easy to be blocked and the positioning is inaccurate when the signal is unstable; while the latter can adapt to the environment of RTK signal shielding, the latter has the disadvantage that the positioning is easy to accumulate drift.
The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art.
Disclosure of Invention
The invention mainly aims to provide a visual positioning method, device, equipment and storage medium for a mower, and aims to solve the technical problems of inaccurate positioning and easy accumulated drift when signals are unstable in traditional RTK positioning or visual slam positioning.
To achieve the above object, the present invention provides a visual positioning method for a mower, the method comprising the steps of:
visual detection is carried out on a marker positioning pile of a lawn where a mower is positioned, and the absolute position and the pose of a marker in the lawn are obtained;
when the mower is in a mowing task, determining the absolute pose of the mower corresponding to the mower through the absolute pose of the marker plate based on the marker plate positioning pile;
and performing global optimization on the absolute pose of the mower and the visual inertial pose of a camera of the mower to obtain the important pose.
Optionally, the visual detection of the sign board positioning pile of the lawn where the mower is located, to obtain the absolute pose of the sign board positioning pile in the lawn, includes:
detecting the environment of a lawn where a mower is positioned, and obtaining an environment map corresponding to the lawn;
Visual detection is carried out on the signpost positioning pile of the lawn, so that the relative pose of the signpost between the signpost positioning pile and the mower is obtained, and the signpost positioning pile is pre-installed in the preset range of the lawn;
and calculating the relative pose of the signpost based on the environment map to obtain the absolute pose of the signpost positioning pile in the lawn.
Optionally, before the visual detection is performed on the sign board positioning pile of the lawn to obtain the sign board relative pose between the sign board positioning pile and the mower, the method further includes:
detecting a signpost of the lawn to obtain a rectangular frame area of the signpost;
extracting features of the interested region of the rectangular frame region to obtain matching feature points of the signboards;
judging whether the matching characteristic points are matched with a preset positioning pile template or not;
if yes, judging that the signpost matched with the preset locating pile template is a signpost locating pile.
Optionally, the visual detection of the sign positioning pile of the lawn, to obtain a sign relative pose between the sign positioning pile and the mower, includes:
Visual detection is carried out on the signpost positioning pile of the lawn, and a current picture frame of the signpost positioning pile is obtained;
performing matrix transformation on the matching feature points corresponding to the current image frame and the preset positioning pile template to obtain a homography matrix between the current image frame and the preset positioning pile template;
carrying out homography transformation on a template two-dimensional point preset on the preset positioning pile template according to the homography matrix to obtain a picture frame two-dimensional point corresponding to the current picture frame;
and carrying out data processing on the three-dimensional distance between the two-dimensional point of the image frame and the two-dimensional point of the template through a PNP algorithm to obtain the relative pose of the marker between the marker positioning pile and the mower.
Optionally, when the mower is in a mowing task, determining, based on the sign positioning pile, an absolute pose of the mower corresponding to the mower through the absolute pose of the sign, including:
judging whether the signpost positioning pile is in the visual range of the mower when the mower is in a mowing task;
if yes, based on the signpost positioning pile, obtaining the relative pose of the mower between the mower and the signpost positioning pile;
And resolving the relative pose of the mower and the absolute pose of the signpost to obtain the absolute pose of the mower in the lawn.
Optionally, after determining whether the sign positioning pile is within the visual range of the mower when the mower is in a mowing task, the method further includes:
when the signpost positioning pile is not in the visual range, starting a camera and an IMU sensor of the mower;
acquiring a camera visual inertial pose of the mower in the lawn based on the camera and an IMU sensor;
and controlling the mower to execute a mowing task according to a preset planning path according to the visual inertial pose of the camera.
Optionally, the global optimization of the absolute pose of the mower and the visual inertial pose of the camera of the mower to obtain the important pose comprises:
acquiring a camera visual inertial pose of the mower through a camera and an IMU sensor;
taking the visual inertial pose of the camera as a relative constraint condition and taking the absolute pose of the mower as an absolute constraint condition;
constructing a current pose diagram of the mower by taking the relative constraint condition and the absolute constraint condition as targets;
And optimizing the current pose graph to obtain the corresponding important and perceived positioning pose of the mower in the current pose graph.
In addition, in order to achieve the above object, the present invention also proposes a visual positioning device for a mower, the device comprising:
the sign board detection module is used for carrying out visual detection on a sign board positioning pile of a lawn where the mower is positioned, and obtaining the absolute pose of the sign board positioning pile in the lawn;
the pose determining module is used for determining the absolute pose of the mower corresponding to the mower through the absolute pose of the signpost based on the signpost positioning pile when the mower is in a mowing task;
and the visual positioning module is used for performing global optimization on the absolute pose of the mower and the visual inertial pose of the camera of the mower to obtain the important visual positioning pose.
In addition, to achieve the above object, the present invention also proposes a visual positioning device for a mower, the device comprising: a memory, a processor, and a mower visual positioning program stored on the memory and executable on the processor, the mower visual positioning program configured to implement the steps of the mower visual positioning method as described above.
In addition, in order to achieve the above object, the present invention also proposes a storage medium having stored thereon a mower visual positioning program which, when executed by a processor, implements the steps of the mower visual positioning method as described above.
The method comprises the steps of obtaining the absolute position and posture of a signboard positioning pile in a lawn where a mower is located by performing visual detection on the signboard positioning pile of the lawn; then, when the mower is in a mowing task, determining the absolute pose of the mower corresponding to the mower through the absolute pose of the marker plate based on the marker plate positioning pile; and finally, performing global optimization on the absolute pose of the mower and the visual inertial pose of a camera of the mower to obtain the important pose. According to the invention, the absolute pose of the lawn mark plate positioning pile is used for determining the absolute pose of the lawn mower, and finally the absolute pose of the lawn mower and the visual inertial pose of the camera of the lawn mower are globally optimized, so that the condition that the lawn mower is easy to accumulate and drift in positioning in an environment shielded by RTK signals is avoided, the traditional visual slam positioning is further optimized, the important sense positioning pose with higher precision is obtained, and the problem of accumulated drift is effectively solved.
Drawings
FIG. 1 is a schematic diagram of a mower visual positioning device of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flow chart of a first embodiment of a visual positioning method of a mower according to the present invention;
FIG. 3 is a flow chart of a second embodiment of a visual positioning method of a mower according to the present invention;
FIG. 4 is a flow chart of a third embodiment of a visual positioning method of a mower according to the present invention;
fig. 5 is a block diagram of a first embodiment of a visual positioning device for a mower according to the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a mower visual positioning device in a hardware running environment according to an embodiment of the present invention.
As shown in fig. 1, the mower visual positioning device may comprise: a processor 1001, such as a central processing unit (Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, a memory 1005. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a Wireless interface (e.g., a Wireless-Fidelity (Wi-Fi) interface). The Memory 1005 may be a high-speed random access Memory (Random Access Memory, RAM) or a stable nonvolatile Memory (NVM), such as a disk Memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
It will be appreciated by those skilled in the art that the structure shown in fig. 1 is not limiting of the mower visual positioning device and may include more or fewer components than illustrated, or certain components may be combined, or a different arrangement of components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and a mower visual positioning program may be included in the memory 1005 as one type of storage medium.
In the mower visual positioning device shown in fig. 1, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the mower visual positioning device of the present invention may be provided in the mower visual positioning device, and the mower visual positioning device invokes the mower visual positioning program stored in the memory 1005 through the processor 1001, and executes the mower visual positioning method provided by the embodiment of the present invention.
The embodiment of the invention provides a visual positioning method for a mower, and referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of the visual positioning method for the mower.
In this embodiment, the mower visual positioning method includes the following steps:
Step S10: and visually detecting the marker positioning pile of the lawn where the mower is positioned to obtain the absolute pose of the marker positioning pile in the lawn.
It should be noted that, the execution body of the method of the present embodiment may be an electronic device having a visual detection function and a visual positioning function, for example, a mowing robot control computer, or may be another electronic device capable of implementing the same or similar functions, for example, a mowing machine visual positioning device, which is not limited in this embodiment. The visual positioning device of the mower can be arranged in the mower and used for controlling the mower to perform visual detection and visual positioning. Here, the present embodiment and the following embodiments will be specifically described with reference to the above-described mower vision positioning apparatus (positioning apparatus for short).
It is understood that a mower is a robot device capable of automatically trimming a lawn, and a camera, an IMU module, and other built-in sensor devices can be mounted inside the mower, and external environment detection can be performed through the camera and the IMU module, and the mower can autonomously move in a predetermined area to cut the lawn by using a razor or a blade.
It should be appreciated that the sign posts are devices provided on the lawn for marking, locating and indicating, and that the sign posts may be customized for particular markings, such as particular rectangular boxes and textures, as the present embodiment is not limited in this regard. The positioning device can detect the rectangular frame of the marker positioning piles in real time so as to ensure that the marker positioning piles can be shot at intervals when the mower works, and the mower can be positioned.
It should be understood that the absolute position and attitude of the sign post refers to the precise position and attitude information of the sign post in the three-dimensional space of the lawn, describing the precise position (translation) and orientation (rotation) of the sign post relative to the global reference coordinate system of the lawn. By analyzing characteristic points or patterns on the marker positioning pile and combining data acquired by a camera or a sensor of the mower, the absolute pose position and the absolute pose of the marker positioning pile in the lawn can be calculated, so that the marker positioning pile in the lawn is positioned.
In a specific implementation, the positioning equipment can carry out visual detection on the signpost positioning pile of the lawn through the camera carried by the mower and the IMU module, and can calculate the absolute pose of the signpost positioning pile in the lawn by analyzing characteristic points or patterns on the signpost positioning pile and combining data acquired by the camera of the mower and the IMU module, thereby realizing the positioning of the signpost positioning pile in the lawn.
Step S20: and when the mower is in a mowing task, determining the absolute pose of the mower corresponding to the mower through the absolute pose of the marker plate based on the marker plate positioning pile.
The mowing task means that the mower is placed in an area where a lawn needs to be mowed, and the mower is automatically mowed. In mowing tasks, the mower typically operates in a predetermined pattern, such as traveling in a fixed path, mowing at a set height, automatically bypassing when encountering an obstacle, and so forth. Surrounding environment can be sensed through a laser sensor, an ultrasonic sensor, a camera and the like, so that collision obstacles or injury to personnel can be avoided, and safety of a mowing process is ensured. The mowing task is finished by the mower, so that the labor and time cost for manually trimming the lawn can be saved, and the lawn can be efficiently trimmed.
It is understood that absolute pose of a mower refers to the precise position and pose information of the mower in the three-dimensional space of the lawn, describing the precise position (translation) and orientation (rotation) of the mower relative to the global reference frame of the lawn. By analyzing characteristic points or patterns of the mower relative to different angles on the signpost positioning pile and combining data acquired by a camera or a sensor of the mower, the absolute pose position and the absolute pose of the mower in the lawn can be calculated, so that the mower is positioned in the lawn.
The mower can be positioned and navigated through the absolute pose of the mower, so that the mower can accurately move and operate and cover the whole lawn area. The method can also be used for map construction and region division, so that the mower can work according to a preset region, the mowing task of the mower can be effectively and safely finished in the working process, and the efficiency and quality of lawn maintenance are improved.
In a specific implementation, the positioning device can detect the positioning pile of the sign board in real time when the mower is in a mowing task, calculate the pose of the sign board relative to the camera when the positioning pile of the sign board is detected, and calculate the absolute pose of the mower in the lawn based on the absolute pose of the sign board, thereby realizing the positioning of the mower in the lawn.
Step S30: and performing global optimization on the absolute pose of the mower and the visual inertial pose of a camera of the mower to obtain the important pose.
It should be noted that the camera visual inertial pose (Camera Visual Inertial Odometry, VIO) is to estimate the motion and position of the camera in the three-dimensional space of the lawn by fusing data of the camera and inertial sensors (such as gyroscopes and accelerometers). The VIO pose combines the advantages of a camera vision sensor and an inertial sensor, and the accurate position and pose information of the camera in the lawn space is calculated through a nonlinear optimization algorithm by utilizing image features observed by the camera of the mower and accelerometer and gyroscope data measured by the inertial sensor. Through the camera visual inertial pose, the approximate estimated pose of the mower in the lawn can be obtained, so that the approximate motion tracking of the mower is realized.
Because the VIO pose has accumulated error, the error of the VIO pose running for a long time is gradually increased and possibly exceeds the available range of the mower; at the moment, the absolute pose of the mower in the lawn, which is calculated through the signpost, is required to be repositioned to eliminate the accumulated error, so that the important sense positioning pose is obtained.
In a specific implementation, considering that the accumulated error exists in the VIO pose, the error of the VIO pose which runs for a long time gradually increases, and the available range of the mower may be exceeded. The positioning equipment can globally optimize the absolute pose of the mower and the visual inertial pose of the camera of the mower at the moment to obtain the important pose with higher precision, thereby solving the problem of accumulated drift of the mower during positioning.
The positioning equipment of the embodiment can carry out visual detection on the signpost positioning pile of the lawn through the camera carried by the mower and the IMU module, and can calculate the absolute pose of the signpost positioning pile in the lawn by analyzing the characteristic points or patterns on the signpost positioning pile and combining the data acquired by the camera of the mower and the IMU module, thereby realizing the positioning of the signpost positioning pile in the lawn. And then, when the mower is in a mowing task, the marker positioning pile can be detected in real time, when the marker positioning pile is detected, the pose of the marker relative to the camera is calculated, and the absolute pose of the mower in the lawn is calculated based on the absolute pose of the marker, so that the mower is positioned in the lawn. Considering that the VIO pose has accumulated error, the error of the VIO pose which runs for a long time is gradually increased and possibly exceeds the available range of the mower. And finally, the absolute pose of the mower and the visual inertial pose of a camera of the mower can be globally optimized to obtain the important pose with higher precision, so that the problem of accumulated drift of the mower during positioning is solved. According to the method, the absolute pose of the lawn marker positioning pile is used for determining the absolute pose of the lawn mower, and finally the absolute pose of the lawn mower and the visual inertial pose of the camera of the lawn mower are subjected to global optimization, so that the situation that the lawn mower is easy to accumulate and drift in positioning in an environment shielded by RTK signals is avoided, the traditional visual slam positioning is further optimized, the important sense positioning pose with higher precision is obtained, and the problem of accumulated drift is effectively solved.
Referring to fig. 3, fig. 3 is a schematic flow chart of a second embodiment of a visual positioning method for a mower according to the present invention.
Based on the first embodiment, in this embodiment, the step S10 includes:
step S11: and detecting the environment of the lawn where the mower is positioned, and obtaining an environment map corresponding to the lawn.
The environment map is a map model for modeling and representing the spatial and geographical information of the lawn where the mower is located. The mower presents information such as objects, structures, features and the like in a lawn environment in a visual manner by collecting and integrating sensor data, geographic information and other related data. By establishing an environment map, the functions of precise positioning, path planning and the like of the mower are realized.
Step S12: and visually detecting the signpost positioning pile of the lawn to obtain the relative pose of the signpost between the signpost positioning pile and the mower, wherein the signpost positioning pile is pre-installed in the preset range of the lawn.
The relative pose of the marker plate refers to the motion pose and the position relation of the marker plate positioning pile relative to the mower, and the relative position and the rotation relation between a camera of the mower and the marker plate positioning pile in a three-dimensional space are described.
When the camera of the mower captures the image of the marker positioning pile, the relative pose between the camera and the marker can be estimated through image processing and computer vision technology. By accurately estimating the relative pose of the camera and the marker positioning pile, accurate pose tracking can be realized.
It will be appreciated that the preset range is a range in which the marker post is installed in the lawn, and the mower can photograph from time to time when working, for example, the center position or the peripheral edge position of the lawn, and the preset range may be determined according to specific requirements and design requirements, which is not limited in this embodiment.
The positioning device can detect the rectangular frame of the marker positioning piles in real time so as to ensure that the marker positioning piles can be shot at intervals when the mower works, and the mower can be positioned.
Step S13: and calculating the relative pose of the signpost based on the environment map to obtain the absolute pose of the signpost positioning pile in the lawn.
For ease of understanding, the explanation of the settlement of the absolute pose of the sign is given as an example, but the present embodiment is not limited thereto. In the process of the lawn mower edge extension map building, the positioning equipment calculates the absolute pose T_w_c of a key frame in a map in real time through the camera and the sensor data, meanwhile, the positioning equipment detects the marker positioning pile and calculates the pose T_c_m of the marker positioning pile relative to the camera, and then the absolute pose T_w_m=T_w_c in the map of the marker positioning pile is represented by T_c_m.
Furthermore, the pose of the marker positioning pile relative to the camera can be stored in the positioning equipment, and the pose is extended for one circle to perform loop optimization, so that the observed landmarks or repeated features are observed again to perform consistency correction and optimization of the map. And finally, calculating the absolute pose of the marker positioning pile in the map based on the optimized pose of the key frame and the pose of the marker relative to the camera.
In the concrete implementation, after the positioning pile of the signpost is installed, the positioning equipment can start the mower to run along the edge of the lawn for one circle, and an environment map is established through a camera of the mower and the IMU module. And detecting the marker positioning piles, calculating the relative positions and postures of the marker positioning piles relative to the marker of the camera when the marker positioning piles are detected, and calculating the absolute positions and postures of the marker of each marker positioning pile in the map based on the key frame postures of the camera. Therefore, the precise positioning of the mower is realized by building an environment map and a signpost positioning pile.
Further, before step S12, the method further includes: detecting a signpost of the lawn to obtain a rectangular frame area of the signpost; extracting features of the interested region of the rectangular frame region to obtain matching feature points of the signboards; judging whether the matching characteristic points are matched with a preset positioning pile template or not; if yes, judging that the signpost matched with the preset locating pile template is a signpost locating pile.
Considering that the lawn is provided with the preset marker positioning piles, the lawn can also comprise public welfare markers, road directing markers and other markers, so that the lawn markers need to be identified first.
The rectangular frame area is a rectangular boundary frame drawn when positioning and identifying the signboard. When using image processing and computer vision algorithms to detect and identify a sign, a particular algorithm may first be used to find a rectangular box area in the image. Through the rectangular frame area, the position and the boundary of the signpost can be accurately determined by an auxiliary recognition algorithm.
It will be appreciated that the region of interest (Region of Interest, ROI) is a portion of the image area that is centrally processed and analyzed for a particular object or region of interest in the image processing of rectangular box areas. The region of interest may be a rectangular, circular, or polygonal shaped region, as this embodiment is not limited in this regard. By defining the region of interest, the identification of the marker by the positioning device can be concentrated in the key region, so that the calculation amount is reduced and the processing efficiency is improved.
It should be understood that the matching feature points perform feature extraction on the region of interest, and the obtained feature points, such as corner points, edges, spots, and the like, may be used as feature points. Then, matching algorithms (such as orb feature point matching algorithm, sift feature point matching algorithm, surf feature point matching algorithm and other feature point matching algorithms) are utilized to compare the similarity between the description matching feature points and find out the best matching pair.
The preset spud template is a template for determining the signboard spud. The marker positioning pile can be rapidly determined by matching the matching characteristic points of the marker with the preset positioning pile template.
In a specific implementation, considering that the preset signpost positioning piles exist on the lawn, the lawn may further comprise public welfare signposts, road directing signposts and other signposts, so that the signposts of the lawn need to be identified first. Because the customized signpost positioning pile is provided with the specific rectangular frames and textures, the positioning equipment can detect the rectangular frame area of the signpost in real time, and the rectangular frames which do not meet the conditions are filtered through the length-width ratio of the rectangular frame area. And then extracting the region of interest of the rectangular frame region passing through the screening, and carrying out feature extraction on the region of interest to obtain the matched feature points of the signpost. And finally, performing feature matching with a preset positioning pile template, wherein if the feature matching is successful, the successful detection of the positioning pile of the sign board is indicated. Through the region of interest, the identification of the positioning equipment to the signboards can be concentrated in the key region, so that the calculated amount is reduced and the processing efficiency is improved.
Further, in the present embodiment, step S12 includes: visual detection is carried out on the signpost positioning pile of the lawn, and a current picture frame of the signpost positioning pile is obtained; performing matrix transformation on the matching feature points corresponding to the current image frame and the preset positioning pile template to obtain a homography matrix between the current image frame and the preset positioning pile template; carrying out homography transformation on a template two-dimensional point preset on the preset positioning pile template according to the homography matrix to obtain a picture frame two-dimensional point corresponding to the current picture frame; and carrying out data processing on the three-dimensional distance between the two-dimensional point of the image frame and the two-dimensional point of the template through a PNP algorithm to obtain the relative pose of the marker between the marker positioning pile and the mower.
It should be noted that, the homography matrix, also called homography transformation matrix, is used to project the plane described by the current frame onto the plane described by the preset spud template. The alignment between the current image frame and the preset positioning pile template can be realized through the homography matrix.
It can be appreciated that a PNP algorithm (perselect-n-Point) is a method for calculating the pose of a camera, which can be used to calculate the rotational and translational poses of a sign board from known two-dimensional points of a frame and the template. In short, the position and orientation of the sign in the environment map can be inferred from the position of the point on the image.
In practical implementation, when the marker positioning pile is successfully detected, the positioning equipment can perform visual detection on the marker positioning pile of the lawn to obtain the current picture frame of the marker positioning pile. And then calculating a homography matrix of the current image frame and a preset positioning pile template through the matching characteristic points, and transforming four preset two-dimensional points on the preset positioning pile template through homography transformation to obtain four two-dimensional points corresponding to the current image frame. And calculating the sign board relative pose of the sign board positioning pile relative to the camera through a PNP algorithm by using the three-dimensional distances of the corresponding four points on the preset positioning pile template which are measured in advance. Therefore, alignment between the current image frame and a preset positioning pile template is realized through the homography matrix, and the relative position and pose of the marker plate positioning pile relative to the camera are rapidly determined.
After the positioning pile of the signboard is installed, the positioning equipment can start the mower to drive along the edge of the lawn for one circle, and an environment map is built through a camera of the mower and the IMU module. And detecting the marker positioning piles, calculating the relative positions and postures of the marker positioning piles relative to the marker of the camera when the marker positioning piles are detected, and calculating the absolute positions and postures of the marker of each marker positioning pile in the map based on the key frame postures of the camera. Therefore, the precise positioning of the mower is realized by building an environment map and a signpost positioning pile. Further, considering that the lawn is not only provided with the preset marker positioning piles, the lawn may also comprise public welfare markers, road directing markers and other markers, so that the lawn markers need to be identified first. Because the customized signpost positioning pile is provided with the specific rectangular frames and textures, the positioning equipment can detect the rectangular frame area of the signpost in real time, and the rectangular frames which do not meet the conditions are filtered through the length-width ratio of the rectangular frame area. And then extracting the region of interest of the rectangular frame region passing through the screening, and carrying out feature extraction on the region of interest to obtain the matched feature points of the signpost. And finally, performing feature matching with a preset positioning pile template, wherein if the feature matching is successful, the successful detection of the positioning pile of the sign board is indicated. Through the region of interest, the identification of the positioning equipment to the signboards can be concentrated in the key region, so that the calculated amount is reduced and the processing efficiency is improved. Furthermore, when the marker positioning pile is successfully detected, the positioning equipment can perform visual detection on the marker positioning pile of the lawn to obtain the current picture frame of the marker positioning pile. And then calculating a homography matrix of the current image frame and a preset positioning pile template through the matching characteristic points, and transforming four preset two-dimensional points on the preset positioning pile template through homography transformation to obtain four two-dimensional points corresponding to the current image frame. And calculating the sign board relative pose of the sign board positioning pile relative to the camera through a PNP algorithm by using the three-dimensional distances of the corresponding four points on the preset positioning pile template which are measured in advance. Therefore, alignment between the current image frame and a preset positioning pile template is realized through the homography matrix, and the relative position and pose of the marker plate positioning pile relative to the camera are rapidly determined.
Referring to fig. 4, fig. 4 is a schematic flow chart of a third embodiment of a visual positioning method for a mower according to the present invention.
Based on the above embodiments, in this embodiment, the step S20 includes:
step S21: when the mower is in a mowing task, judging whether the signpost positioning pile is in the visual range of the mower.
It should be noted that the visual range is the range in which the mower observes and perceives the surrounding environment, and may be implemented by installing a camera or other sensor technology. In general, the field of view of a mower should cover the area in front of and to the side of the mower in order to timely perceive obstacles, borderlines and other important features. This ensures that the mower will avoid collisions during movement and will perform mowing in the correct area.
Step S22: if so, based on the sign positioning pile, the relative pose of the mower between the mower and the sign positioning pile is obtained.
The relative pose of the mower refers to the movement pose and the position relation of the mower relative to the signpost positioning pile, and the relative position and the rotation relation between a camera of the mower and the signpost positioning pile in a three-dimensional space are described.
When the camera of the mower captures the image of the marker positioning pile, the relative pose between the camera and the marker can be estimated through image processing and computer vision technology. By accurately estimating the relative pose of the camera and the marker positioning pile, accurate pose tracking can be realized.
Step S23: and resolving the relative pose of the mower and the absolute pose of the signpost to obtain the absolute pose of the mower in the lawn.
In a specific implementation, when the mower is in a mowing task, the positioning device can firstly judge whether the signpost positioning pile is in the visual range of the mower. When the marker positioning pile is detected in real time, determining the relative pose of the mower between the mower and the marker positioning pile based on the marker positioning pile. And finally, calculating the absolute pose of the mower in the map according to the calculated relative pose of the camera and the mower relative pose of the marker positioning pile and the absolute pose of the marker in the map. Therefore, accurate posture tracking is realized by accurately estimating the relative postures of the camera of the mower and the marker positioning pile.
Further, after step S21, the method in this embodiment further includes: when the signpost positioning pile is not in the visual range, starting a camera and an IMU sensor of the mower; acquiring a camera visual inertial pose of the mower in the lawn based on the camera and an IMU sensor; and controlling the mower to execute a mowing task according to a preset planning path according to the visual inertial pose of the camera.
The IMU sensor is an inertial measurement unit sensor, and may be used to measure and detect acceleration and angular velocity of an object. The attitude, the position and the motion state of the object on the lawn can be estimated through the IMU sensor, so that the attitude tracking of the object on the lawn is realized.
It is understood that the preset planned path is a path preset for mowing operation when the mower performs mowing operation. By executing the mowing task according to the preset planned path, the mower can efficiently cover vegetation within a specified range, maintain the uniformity and effect of mowing, and accurately stop or return to the base, thereby improving the automation degree and accuracy of mowing operation.
In a specific implementation, when a sign spud is not detected, a positioning device may turn on a camera and IMU sensor of the mower. And outputting the camera visual inertial pose of the mower in the lawn through a camera and an IMU sensor. And finally, controlling the mower to execute a mowing task according to a preset planning path according to the visual inertial pose of the camera. When the signpost positioning pile is not detected, the posture, the position and the motion state of the object on the lawn are estimated through the IMU sensor, so that positioning is realized.
Further, in the present embodiment, step S30 includes: acquiring a camera visual inertial pose of the mower through a camera and an IMU sensor; taking the visual inertial pose of the camera as a relative constraint condition and taking the absolute pose of the mower as an absolute constraint condition; constructing a current pose diagram of the mower by taking the relative constraint condition and the absolute constraint condition as targets; and optimizing the current pose graph to obtain the corresponding important and perceived positioning pose of the mower in the current pose graph.
The relative constraint condition is a condition for optimizing the camera visual inertial pose as the pose. The absolute constraint condition is to take the absolute pose of the mower as a pose optimization condition.
In the specific implementation, the visual inertial pose of the camera, which is output by the positioning equipment through the camera and the IMU sensor, is taken as relative constraint, the absolute pose of the mower, which is calculated through the signpost, is taken as absolute constraint, and the current pose graph is constructed together for optimization, so that the high-precision and high-importance visual positioning pose is obtained. Therefore, absolute positioning based on the positioning piles of the signboards can be realized, and the problem of accumulated drift is effectively solved.
In this embodiment, when the mower is in a mowing task, the positioning device may first determine whether the sign positioning pile is within a visual range of the mower. When the marker positioning pile is detected in real time, determining the relative pose of the mower between the mower and the marker positioning pile based on the marker positioning pile. And finally, calculating the absolute pose of the mower in the map according to the calculated relative pose of the camera and the mower relative pose of the marker positioning pile and the absolute pose of the marker in the map. Therefore, accurate posture tracking is realized by accurately estimating the relative postures of the camera of the mower and the marker positioning pile. Further, when a sign spud is not detected, the positioning apparatus may turn on the camera and IMU sensor of the mower. And outputting the camera visual inertial pose of the mower in the lawn through a camera and an IMU sensor. And finally, controlling the mower to execute a mowing task according to a preset planning path according to the visual inertial pose of the camera. When the signpost positioning pile is not detected, the posture, the position and the motion state of the object on the lawn are estimated through the IMU sensor, so that positioning is realized. Furthermore, the visual inertial pose of the camera, which is output by the positioning equipment through the camera and the IMU sensor, is taken as relative constraint, the absolute pose of the mower, which is calculated through the signpost, is taken as absolute constraint, and the current pose graph is constructed together for optimization, so that the important visual positioning pose with higher precision is obtained. Therefore, absolute positioning based on the positioning piles of the signboards can be realized, and the problem of accumulated drift is effectively solved.
In addition, the embodiment of the invention also provides a storage medium, wherein the storage medium is stored with a mower visual positioning program, and the mower visual positioning program realizes the steps of the mower visual positioning method when being executed by a processor.
Referring to fig. 5, fig. 5 is a block diagram of a first embodiment of a visual positioning device for a mower according to the present invention.
As shown in fig. 5, a visual positioning device for a mower according to an embodiment of the present invention includes:
the sign board detection module 501 is used for visually detecting a sign board positioning pile of a lawn where a mower is positioned to obtain an absolute sign board pose of the sign board positioning pile in the lawn;
the pose determining module 502 is configured to determine, based on the sign positioning pile, an absolute pose of the mower corresponding to the mower according to the absolute pose of the sign when the mower is in a mowing task;
the visual positioning module 503 is configured to globally optimize the absolute pose of the mower and the visual inertial pose of the camera of the mower, so as to obtain a weighted sense positioning pose.
The positioning equipment of the embodiment can carry out visual detection on the signpost positioning pile of the lawn through the camera carried by the mower and the IMU module, and can calculate the absolute pose of the signpost positioning pile in the lawn by analyzing the characteristic points or patterns on the signpost positioning pile and combining the data acquired by the camera of the mower and the IMU module, thereby realizing the positioning of the signpost positioning pile in the lawn. And then, when the mower is in a mowing task, the marker positioning pile can be detected in real time, when the marker positioning pile is detected, the pose of the marker relative to the camera is calculated, and the absolute pose of the mower in the lawn is calculated based on the absolute pose of the marker, so that the mower is positioned in the lawn. Considering that the VIO pose has accumulated error, the error of the VIO pose which runs for a long time is gradually increased and possibly exceeds the available range of the mower. And finally, the absolute pose of the mower and the visual inertial pose of a camera of the mower can be globally optimized to obtain the important pose with higher precision, so that the problem of accumulated drift of the mower during positioning is solved. According to the method, the absolute pose of the lawn marker positioning pile is used for determining the absolute pose of the lawn mower, and finally the absolute pose of the lawn mower and the visual inertial pose of the camera of the lawn mower are subjected to global optimization, so that the situation that the lawn mower is easy to accumulate and drift in positioning in an environment shielded by RTK signals is avoided, the traditional visual slam positioning is further optimized, the important sense positioning pose with higher precision is obtained, and the problem of accumulated drift is effectively solved.
Based on the first embodiment of the visual positioning device for the mower, a second embodiment of the visual positioning device for the mower is provided.
In this embodiment, the signpost detection module 501 is further configured to perform environmental detection on a lawn where a mower is located, so as to obtain an environmental map corresponding to the lawn; visual detection is carried out on the signpost positioning pile of the lawn, so that the relative pose of the signpost between the signpost positioning pile and the mower is obtained, and the signpost positioning pile is pre-installed in the preset range of the lawn; and calculating the relative pose of the signpost based on the environment map to obtain the absolute pose of the signpost positioning pile in the lawn.
Further, the sign detection module 501 is further configured to detect a sign of the lawn, to obtain a rectangular frame area of the sign; extracting features of the interested region of the rectangular frame region to obtain matching feature points of the signboards; judging whether the matching characteristic points are matched with a preset positioning pile template or not; if yes, judging that the signpost matched with the preset locating pile template is a signpost locating pile.
Further, the signpost detection module 501 is further configured to perform visual detection on the signpost positioning pile of the lawn, to obtain a current frame of the signpost positioning pile; performing matrix transformation on the matching feature points corresponding to the current image frame and the preset positioning pile template to obtain a homography matrix between the current image frame and the preset positioning pile template; carrying out homography transformation on a template two-dimensional point preset on the preset positioning pile template according to the homography matrix to obtain a picture frame two-dimensional point corresponding to the current picture frame; and carrying out data processing on the three-dimensional distance between the two-dimensional point of the image frame and the two-dimensional point of the template through a PNP algorithm to obtain the relative pose of the marker between the marker positioning pile and the mower.
Further, the pose determining module 502 is further configured to determine, when the mower is in a mowing task, whether the sign positioning pile is within a visual range of the mower; if yes, based on the signpost positioning pile, obtaining the relative pose of the mower between the mower and the signpost positioning pile; and resolving the relative pose of the mower and the absolute pose of the signpost to obtain the absolute pose of the mower in the lawn.
Further, the visual positioning module 503 is further configured to turn on a camera and an IMU sensor of the mower when the sign positioning pile is not in the visual range; acquiring a camera visual inertial pose of the mower in the lawn based on the camera and an IMU sensor; and controlling the mower to execute a mowing task according to a preset planning path according to the visual inertial pose of the camera.
Further, the visual positioning module 503 is further configured to obtain a camera visual inertial pose of the mower through a camera and an IMU sensor; taking the visual inertial pose of the camera as a relative constraint condition and taking the absolute pose of the mower as an absolute constraint condition; constructing a current pose diagram of the mower by taking the relative constraint condition and the absolute constraint condition as targets; and optimizing the current pose graph to obtain the corresponding important and perceived positioning pose of the mower in the current pose graph.
Other embodiments or specific implementations of the visual positioning device for a mower of the present invention may refer to the above method embodiments, and will not be described herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. read-only memory/random-access memory, magnetic disk, optical disk), comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (10)
1. A method of visual positioning of a mower, the method comprising:
visual detection is carried out on a marker positioning pile of a lawn where a mower is positioned, and the absolute position and the pose of a marker in the lawn are obtained;
when the mower is in a mowing task, determining the absolute pose of the mower corresponding to the mower through the absolute pose of the marker plate based on the marker plate positioning pile;
and performing global optimization on the absolute pose of the mower and the visual inertial pose of a camera of the mower to obtain the important pose.
2. The method for visually positioning a lawn mower according to claim 1, wherein the visually detecting the positioning pile of the sign board of the lawn in which the lawn mower is located to obtain the absolute pose of the sign board of the positioning pile of the sign board in the lawn comprises:
detecting the environment of a lawn where a mower is positioned, and obtaining an environment map corresponding to the lawn;
visual detection is carried out on the signpost positioning pile of the lawn, so that the relative pose of the signpost between the signpost positioning pile and the mower is obtained, and the signpost positioning pile is pre-installed in the preset range of the lawn;
And calculating the relative pose of the signpost based on the environment map to obtain the absolute pose of the signpost positioning pile in the lawn.
3. The method for visually positioning a mower of claim 2, wherein the step of visually inspecting the marker post of the lawn further comprises, before the step of obtaining the relative position of the marker post between the marker post and the mower:
detecting a signpost of the lawn to obtain a rectangular frame area of the signpost;
extracting features of the interested region of the rectangular frame region to obtain matching feature points of the signboards;
judging whether the matching characteristic points are matched with a preset positioning pile template or not;
if yes, judging that the signpost matched with the preset locating pile template is a signpost locating pile.
4. A method of visual positioning of a lawn mower as claimed in claim 3, wherein said visually inspecting said sign posts of said lawn to obtain a sign relative pose between said sign posts and said lawn mower comprises:
visual detection is carried out on the signpost positioning pile of the lawn, and a current picture frame of the signpost positioning pile is obtained;
Performing matrix transformation on the matching feature points corresponding to the current image frame and the preset positioning pile template to obtain a homography matrix between the current image frame and the preset positioning pile template;
carrying out homography transformation on a template two-dimensional point preset on the preset positioning pile template according to the homography matrix to obtain a picture frame two-dimensional point corresponding to the current picture frame;
and carrying out data processing on the three-dimensional distance between the two-dimensional point of the image frame and the two-dimensional point of the template through a PNP algorithm to obtain the relative pose of the marker between the marker positioning pile and the mower.
5. The method for visually positioning a mower according to claim 1, wherein determining an absolute pose of the mower corresponding to the mower based on the marker positioning pile when the mower is in a mowing task, comprises:
judging whether the signpost positioning pile is in the visual range of the mower when the mower is in a mowing task;
if yes, based on the signpost positioning pile, obtaining the relative pose of the mower between the mower and the signpost positioning pile;
and resolving the relative pose of the mower and the absolute pose of the signpost to obtain the absolute pose of the mower in the lawn.
6. The method for visual positioning of a lawnmower of claim 5, wherein after determining whether the sign spud is within the visual range of the lawnmower while the lawnmower is in a mowing mission, further comprising:
when the signpost positioning pile is not in the visual range, starting a camera and an IMU sensor of the mower;
acquiring a camera visual inertial pose of the mower in the lawn based on the camera and an IMU sensor;
and controlling the mower to execute a mowing task according to a preset planning path according to the visual inertial pose of the camera.
7. The method of claim 1-6, wherein globally optimizing the absolute pose of the mower with the inertial pose of the camera vision of the mower to obtain a weighted sense localization pose comprises:
acquiring a camera visual inertial pose of the mower through a camera and an IMU sensor;
taking the visual inertial pose of the camera as a relative constraint condition and taking the absolute pose of the mower as an absolute constraint condition;
constructing a current pose diagram of the mower by taking the relative constraint condition and the absolute constraint condition as targets;
And optimizing the current pose graph to obtain the corresponding important and perceived positioning pose of the mower in the current pose graph.
8. A visual positioning device for a mower, the device comprising:
the sign board detection module is used for carrying out visual detection on a sign board positioning pile of a lawn where the mower is positioned, and obtaining the absolute pose of the sign board positioning pile in the lawn;
the pose determining module is used for determining the absolute pose of the mower corresponding to the mower through the absolute pose of the signpost based on the signpost positioning pile when the mower is in a mowing task;
and the visual positioning module is used for performing global optimization on the absolute pose of the mower and the visual inertial pose of the camera of the mower to obtain the important visual positioning pose.
9. A mower vision positioning apparatus, the apparatus comprising: a memory, a processor, and a mower visual positioning program stored on the memory and executable on the processor, the mower visual positioning program configured to implement the steps of the mower visual positioning method of any one of claims 1 to 7.
10. A storage medium having stored thereon a mower visual positioning program which when executed by a processor performs the steps of the mower visual positioning method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410060045.1A CN117570969A (en) | 2024-01-16 | 2024-01-16 | Mower visual positioning method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410060045.1A CN117570969A (en) | 2024-01-16 | 2024-01-16 | Mower visual positioning method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117570969A true CN117570969A (en) | 2024-02-20 |
Family
ID=89862880
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410060045.1A Pending CN117570969A (en) | 2024-01-16 | 2024-01-16 | Mower visual positioning method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117570969A (en) |
-
2024
- 2024-01-16 CN CN202410060045.1A patent/CN117570969A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112584697B (en) | Autonomous machine navigation and training using vision system | |
| JP7082545B2 (en) | Information processing methods, information processing equipment and programs | |
| EP3234718B1 (en) | Robotic vehicle learning site boundary | |
| US9639960B1 (en) | Systems and methods for UAV property assessment, data capture and reporting | |
| CN109324337B (en) | Unmanned aerial vehicle route generation and positioning method and device and unmanned aerial vehicle | |
| JP2022106924A (en) | Device and method for autonomous self-position estimation | |
| US20200159227A1 (en) | Method of navigating a vehicle and system thereof | |
| EP3051493B1 (en) | Survey data processing device, survey data processing method, and program therefor | |
| EP3842751B1 (en) | System and method of generating high-definition map based on camera | |
| CN104714547A (en) | Autonomous gardening vehicle with camera | |
| CN114322980A (en) | Method for obtaining position coordinates and drawing electronic map, computer-readable storage medium, and autonomous operating apparatus | |
| CN112686951A (en) | Method, device, terminal and storage medium for determining robot position | |
| US20210240195A1 (en) | Systems and methods for utilizing images to determine the position and orientation of a vehicle | |
| CN115454077A (en) | Automatic lawn mower, control method thereof, and computer-readable storage medium | |
| US20220076446A1 (en) | Camera orientation estimation | |
| JP5375249B2 (en) | Moving path planning device, moving body control device, and moving body | |
| CA3215318A1 (en) | Systems and methods for assisting a crane operator | |
| EP3769598B1 (en) | Placement position notification system | |
| CN117570969A (en) | Mower visual positioning method, device, equipment and storage medium | |
| EP3816753B1 (en) | Method for learning at least one section of a boundary edge of a surface for a surface treatment system, method for operating a surface processing system, learning system and green area processing system | |
| CN109901589B (en) | Mobile robot control method and device | |
| CN112685527A (en) | Method, device and electronic system for establishing map | |
| US20240045439A1 (en) | Task execution system, task execution method, and task execution program | |
| EP3637056B1 (en) | Method and system for generating navigation data for a geographical location | |
| US20230320263A1 (en) | Method for determining information, remote terminal, and mower |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |