CN110060202B - Monocular SLAM algorithm initialization method and system - Google Patents
Monocular SLAM algorithm initialization method and system Download PDFInfo
- Publication number
- CN110060202B CN110060202B CN201910319610.0A CN201910319610A CN110060202B CN 110060202 B CN110060202 B CN 110060202B CN 201910319610 A CN201910319610 A CN 201910319610A CN 110060202 B CN110060202 B CN 110060202B
- Authority
- CN
- China
- Prior art keywords
- pixel points
- image
- slam algorithm
- monocular
- overlapping area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004422 calculation algorithm Methods 0.000 title claims abstract description 130
- 238000011423 initialization method Methods 0.000 title claims abstract description 9
- 238000000034 method Methods 0.000 claims abstract description 35
- 230000008859 change Effects 0.000 claims description 19
- 238000006243 chemical reaction Methods 0.000 claims description 14
- 238000006073 displacement reaction Methods 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 4
- 238000013519 translation Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 abstract description 5
- 230000008569 process Effects 0.000 description 14
- 230000008901 benefit Effects 0.000 description 7
- 230000000007 visual effect Effects 0.000 description 6
- 230000003068 static effect Effects 0.000 description 5
- 238000012937 correction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000004888 barrier function Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 235000015243 ice cream Nutrition 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- G06T3/047—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
Abstract
The invention provides an initialization method and a system of a monocular SLAM algorithm, wherein the method comprises the following steps: and selecting pixel points in an image overlapping area from the acquired image, converting the selected pixel points into pixel points corresponding to a binocular SLAM algorithm, calculating three-dimensional coordinates of the converted pixel points in the image overlapping area based on the binocular SLAM algorithm, converting the three-dimensional coordinates of the pixel points into coordinate information of a camera coordinate system, and initializing the monocular SLAM algorithm according to the coordinate information of the camera coordinate system. Therefore, the embodiment of the invention can assist the monocular SLAM algorithm to realize initialization according to the binocular SLAM algorithm, so that the monocular system corresponding to the monocular SLAM algorithm can realize initialization more quickly and better, and the stability and the calculation precision of the monocular SLAM algorithm are improved.
Description
Technical Field
The invention relates to the technical field of automobiles, in particular to an initialization method and system of a monocular SLAM algorithm.
Background
In the existing solutions of Auto Park Assist Systems (APAs), 4 low-cost fisheye cameras are generally provided to implement an all-vehicle Monitoring system (AVM) function or achieve a visual display effect related to the APA. In the prior art, in the process of finding a parking space by using an APA, the parking space and static obstacles such as an ice cream cone, a wheel gear, a rod, a net rail and the like are generally required to be detected, and even some unusual obstacles such as a bicycle, a chair and the like are required to be detected. The method has the advantages that the static obstacles are detected by adopting the current common machine learning method, the types of the obstacles and a large amount of training data need to be known in advance, and the universality and the flexibility for finding the parking spaces are poor.
In addition, at present, the latest research focus is to use point cloud to realize static obstacle detection, but a better Global Camera (Global Shutter Camera) or a binocular Camera is generally required, and the existing fish-eye cameras arranged in a large number on the automobile cannot be well realized. The existing visual SLAM (Simultaneous Localization And mapping) point cloud generating method based on the monocular fisheye camera mainly has the problems of poor stability, low precision, difficulty in obtaining the real scale of a static obstacle And the like. The poor stability is mainly expressed in the aspects of frequent initialization failure in the point cloud generation process, large deviation of point cloud generation, large noise, large scale deviation of point cloud which can be generated, and the like. The precision problem and the real scale problem are mainly shown in other aspects that the point cloud generated by the monocular SLAM algorithm has no real scale, the scale has larger randomness, and the point cloud changes with the scene, the key frame interval and the like.
Disclosure of Invention
In view of the above, the present invention has been made to provide an initialization method and system of a monocular SLAM algorithm that overcomes or at least partially solves the above-mentioned problems.
According to an aspect of the present invention, there is provided a method for initializing a monocular SLAM algorithm, including:
selecting pixel points in an image overlapping region from the collected images, wherein the image overlapping region is a content overlapping region of two images collected through different angles;
converting the selected pixel points into pixel points corresponding to a binocular SLAM algorithm, and calculating three-dimensional coordinates of the converted pixel points in an image overlapping area based on the binocular SLAM algorithm;
converting the three-dimensional coordinates of the pixel points into coordinate information of a camera coordinate system;
and initializing the monocular SLAM algorithm according to the coordinate information of the camera coordinate system.
Optionally, selecting a pixel point in an image overlapping region from the collected image includes:
acquiring a collected image, and inputting the acquired image into a preset camera model;
and after the image is processed by the camera model, selecting pixel points in an image overlapping area from the processed image according to a monocular SLAM algorithm.
Optionally, converting the selected pixel point into a pixel point corresponding to a binocular SLAM algorithm, including:
and converting the selected pixel points into pixel points corresponding to a binocular SLAM algorithm based on an image coordinate system.
Optionally, calculating the three-dimensional coordinates of the converted pixel points in the image overlapping area based on the binocular SLAM algorithm includes:
acquiring a base line distance between cameras corresponding to pre-measured acquired images;
and calculating the three-dimensional coordinates of the converted pixel points in the image overlapping area based on the binocular SLAM algorithm and according to the baseline distance between the cameras.
Optionally, after initializing the monocular SLAM algorithm according to the coordinate information of the camera coordinate system, the method further includes:
selecting pixel points in an image overlapping area from the collected images according to an initialized monocular SLAM algorithm, wherein the image overlapping area is a content overlapping area of two images collected from different angles;
converting the selected pixel points into pixel points corresponding to a binocular SLAM algorithm, and calculating three-dimensional coordinates of the converted pixel points in an image overlapping area based on the binocular SLAM algorithm;
converting the three-dimensional coordinates of the pixel points into coordinate information of a camera coordinate system, and calculating the pose change of a camera for collecting images from an angle based on the coordinate information of the camera coordinate system by using the initialized monocular SLAM algorithm;
calculating three-dimensional coordinates of pixel points in an image non-overlapping area according to the pose change, wherein the image non-overlapping area is an area where the contents of two images acquired through different angles do not overlap;
and generating a point cloud according to the three-dimensional coordinates of the pixel points in the non-overlapping area and the overlapping area of the image.
Optionally, calculating three-dimensional coordinates of pixel points in the non-overlapping region of the image according to the pose change includes:
calculating the displacement distance between any two different moments of a camera which collects images from an angle through an initialized monocular SLAM algorithm according to the pose change;
and calculating the three-dimensional coordinates of the pixel points in the non-overlapping area of the image according to the displacement distance by using the initialized monocular SLAM algorithm.
Optionally, the pose change of the camera comprises: a rotation matrix and/or a translation matrix of the camera.
According to another aspect of the present invention, there is also provided an initialization system of monocular SLAM algorithm, comprising a monocular subsystem, a binocular subsystem, and a transformation subsystem, wherein,
the monocular subsystem selects pixel points in an image overlapping area from the collected images, wherein the image overlapping area is a content overlapping area of two images collected through different angles;
the conversion subsystem converts the selected pixel points into pixel points which are suitable for the binocular subsystem and correspond to a binocular SLAM algorithm;
the binocular subsystem calculates the three-dimensional coordinates of the converted pixel points in the image overlapping area based on the binocular SLAM algorithm;
the conversion subsystem is used for converting the three-dimensional coordinates of the pixel points into coordinate information of a camera coordinate system suitable for the monocular subsystem;
and the monocular subsystem initializes the monocular SLAM algorithm according to the coordinate information of the camera coordinate system.
According to yet another aspect of the present invention, there is also provided a computer storage medium having stored thereon computer program code which, when run on a computing device, causes the computing device to execute the method of initializing the monocular SLAM algorithm of any of the embodiments above.
In the embodiment of the invention, after the pixel points in the image overlapping area are selected from the collected image and are converted into the pixel points corresponding to the binocular SLAM algorithm, the three-dimensional coordinates of the converted pixel points in the image overlapping area are calculated based on the binocular SLAM algorithm, the three-dimensional coordinates of the pixel points are converted into the coordinate information of a camera coordinate system, and then the monocular SLAM algorithm can be initialized according to the coordinate information of the camera coordinate system. Therefore, the embodiment of the invention can calculate and obtain the more accurate and stable coordinate information of the pixel points in the image overlapping area according to the binocular SLAM algorithm, so that the coordinate information of the pixel points is utilized to assist the monocular SLAM algorithm to realize initialization, a monocular system corresponding to the monocular SLAM algorithm can be initialized more quickly and better, and the stability and the calculation precision of the monocular SLAM algorithm are improved.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments thereof, taken in conjunction with the accompanying drawings.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 shows a flow diagram of a method for initializing a monocular SLAM algorithm according to one embodiment of the present invention;
FIG. 2 shows a schematic view of a capture field of view of a fisheye camera on a vehicle, in accordance with one embodiment of the invention;
fig. 3 is a schematic structural diagram illustrating an initialization system of the monocular SLAM algorithm according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
In order to solve the above technical problem, an embodiment of the present invention provides an initialization method for a monocular SLAM algorithm. Fig. 1 is a flowchart illustrating an initialization method of a monocular SLAM algorithm according to one embodiment of the present invention. Referring to fig. 1, the method includes at least steps S102 to S108.
Step S102, selecting pixel points in an image overlapping area from the collected images, wherein the image overlapping area is a content overlapping area of two images collected through different angles.
In the step, the pixel points in the image overlapping area are actually activated points, the uniform point selection is preferably performed when the pixel points are selected in the image overlapping area, and the density of the selected points is slightly larger, so that the requirement of initializing the monocular SLAM algorithm is met.
And step S104, converting the selected pixel points into pixel points corresponding to a binocular SLAM algorithm, and calculating the three-dimensional coordinates of the converted pixel points in the image overlapping area based on the binocular SLAM algorithm.
In this step, the three-dimensional coordinates of the converted pixel points refer to three-dimensional coordinate information based on a world coordinate system.
And step S106, converting the three-dimensional coordinates of the pixel points into coordinate information of a camera coordinate system.
And step S108, initializing the monocular SLAM algorithm according to the coordinate information of the camera coordinate system.
In the embodiment of the invention, after the pixel points in the image overlapping area are selected from the collected image and are converted into the pixel points corresponding to the binocular SLAM algorithm, the three-dimensional coordinates of the converted pixel points in the image overlapping area are calculated based on the binocular SLAM algorithm, the three-dimensional coordinates of the pixel points are converted into the coordinate information of a camera coordinate system, and then the monocular SLAM algorithm can be initialized according to the coordinate information of the camera coordinate system. Therefore, the embodiment of the invention calculates and obtains the more accurate and stable coordinate information of the pixel points in the image overlapping area through the binocular SLAM algorithm, so that the coordinate information of the pixel points is utilized to assist the monocular SLAM algorithm to realize initialization, a monocular system corresponding to the monocular SLAM algorithm can be initialized more quickly and better, and the stability and the calculation precision of the monocular SLAM algorithm are improved.
Because the prior vision SLAM algorithm needs the image-based correction graph, the field angle FOV of the correction graph is small, and the overlapping area of two adjacent cameras is not large enough. And the problems of stability and scale precision encountered by a monocular algorithm can be well solved by adopting a binocular vision SLAM algorithm. Therefore, in the embodiment of the invention, at least two cameras are used as two eyes, so that a sufficient field angle overlapping area is provided for the binocular vision SLAM algorithm, and the at least two cameras can be respectively positioned at any positions of different angles on the vehicle.
Referring to step S102 above, in an embodiment of the present invention, the image overlapping area may be a content overlapping area in two images respectively captured by any two cameras installed at different positions of the vehicle. When selecting the pixel points in the image overlapping region from the collected images, the pixel points of any object in the image overlapping region can be selected from one or more frames of images collected by one of the cameras during initial work. Of course, the device for acquiring an image in the embodiment of the present invention is not limited to a camera, and may also be other image acquisition devices, which is not specifically limited in the embodiment of the present invention.
In order to take the low cost and the large visual angle of the camera into account, the cameras arranged at different positions and angles of the vehicle can adopt fisheye cameras which can acquire images and have a large visual angle range.
For example, referring to fig. 2, the acquisition field of view of the fisheye camera mounted on the automobile is represented by a semicircle, and the automobile of the embodiment shown in fig. 2 has 4 fisheye cameras mounted therein, which are respectively located in the front, rear, left, and right directions of the automobile body. Assuming that two adjacent fisheye cameras comprise a front fisheye camera and a side fisheye camera, when a plurality of pixel points of any object are selected from images corresponding to the side fisheye cameras, uniform point selection needs to be performed in an image overlapping area and an image non-overlapping area respectively, and the density of the selected points in the image overlapping area is slightly larger.
Referring to step S102, in an embodiment of the present invention, in the process of selecting the pixel points in the image overlapping area from the acquired image, in order to directly use the image acquired by the camera, after the image acquired by the camera is acquired, the acquired image may be input into a preset camera model, the camera model processes the image, and then the pixel points in the image overlapping area are selected from the processed image according to the monocular SLAM algorithm.
In the embodiment of the invention, the monocular pinhole Camera Model and the fisheye Camera Model (omnidirective Camera Model) are respectively used for processing the images acquired by the fisheye Camera, and the coverage range of the activated pixels in the images processed by the fisheye Camera Model is larger and the number of the activated pixels is larger by comparing the effective pixel points in the images processed by the monocular pinhole Camera Model and the fisheye Camera Model. Therefore, the fish-eye camera model is preferably adopted in the embodiment of the invention, so that distortion correction of the fish-eye image acquired by the fish-eye camera is not required (namely, the fish-eye image is not required to be converted into a plane image), the content of the fish-eye image can be directly applied to the visual SLAM algorithm, and the consistency and quality of the final point cloud are improved. Moreover, the fisheye camera model is used to increase the field angle and improve the matching precision.
If the fisheye camera model is selected as the preset camera model, in the step S102, in order to directly use the image acquired by the fisheye camera, the image acquired by the fisheye camera is prevented from being corrected to a non-fisheye image, so as to prevent the field of view of the image from being reduced. According to the embodiment of the invention, two images respectively acquired by two fisheye cameras can be acquired firstly, then the two images are respectively input into the fisheye camera model, then after the images are processed by the fisheye camera model, the image corresponding to any fisheye camera is selected from the processed images, and a plurality of pixel points of any object are selected from the selected processed images.
In general, two adjacent fisheye cameras mounted on a vehicle are far away from each other and oriented differently, and the angles at which images are acquired are different. For example, for a front fisheye camera and a side fisheye camera adjacent to each other in a car, the two cameras are far away from each other and are respectively directed to the front side and the left side/right side. When selecting an image corresponding to any fisheye camera, it is necessary to determine which fisheye camera is more appropriate to select the image corresponding to the fisheye camera according to an actual application scene.
For example, in an auto park scene, a driver cannot visually see a scene to the side of the vehicle because the driver can visually see the scene in front of the vehicle. Therefore, the visual field of the side fisheye camera can be selected as a target area, namely, an image corresponding to the side fisheye camera is selected, so that a driver can conveniently detect the static barriers at the side of the vehicle when looking for a parking space.
Referring to step S104, in an embodiment of the present invention, in the process of converting the pixel point located in the image overlapping area in the selected pixel point into the pixel point corresponding to the binocular SLAM algorithm, the pixel point may be converted by combining the image coordinate system, and a specific conversion process may be implemented in the prior art, which is not described herein again.
In another embodiment of the present invention, in the process of calculating the three-dimensional coordinates of the converted pixel points in the image overlapping area based on the binocular SLAM algorithm, the pre-measured baseline distance between the cameras corresponding to the acquired images may be obtained, and then the binocular SLAM algorithm is used to calculate the three-dimensional coordinates of the converted pixel points in the image overlapping area according to the baseline distance between the cameras. The three-dimensional coordinate information in this embodiment further includes depth information of the pixel points.
In this embodiment, if the fisheye cameras installed in front of and on the side of the vehicle are used to form the binocular, the base line distance between the fisheye cameras in front of and on the side of the vehicle may be measured in advance, and then the binocular SLAM algorithm is used to calculate the three-dimensional coordinates of the converted pixel points in the image overlapping area according to the base line distance between the two fisheye cameras.
After initializing the monocular SLAM algorithm, the binocular SLAM algorithm can be used for assisting the initialized monocular SLAM algorithm to acquire mature points in the acquired image more quickly and accurately in real time, and point clouds of the object are generated according to three-dimensional coordinates of pixel points in a non-overlapping area and an overlapping area of the image. Therefore, the embodiment of the invention also provides a binocular point cloud generating method which mainly comprises the following steps 1 to 5.
Step 1, selecting pixel points in an image overlapping area from an acquired image according to an initialized monocular SLAM algorithm.
In this step, the image overlap region is a content overlap region of two images acquired through different angles. In addition, because the camera installed on the vehicle can collect images in real time, the method can select pixel points in the image overlapping area from the images collected in real time.
In the step, a plurality of pixel points selected from the acquired image refer to mature points, and are stable pixel points which are further screened after the pixel points are primarily selected from the image, and the mature points can be subsequently calculated to obtain three-dimensional coordinate information.
And 2, converting the selected pixel points into pixel points corresponding to a binocular SLAM algorithm, and calculating the three-dimensional coordinates of the converted pixel points in the image overlapping area based on the binocular SLAM algorithm.
In this step, when the selected pixel point is converted into a pixel point corresponding to the binocular SLAM algorithm, the selected pixel point may also be converted into a pixel point corresponding to the binocular SLAM algorithm based on the image coordinate system.
And 3, converting the three-dimensional coordinates of the pixel points into coordinate information of a camera coordinate system, and calculating the pose change of the camera for acquiring images from an angle based on the coordinate information of the camera coordinate system by using the initialized monocular SLAM algorithm.
And 4, calculating the three-dimensional coordinates of pixel points in the image non-overlapping area according to the pose change, wherein the image non-overlapping area is an area where the contents of two images acquired through different angles are not overlapped.
And 5, generating point clouds according to the three-dimensional coordinates of the pixel points in the non-overlapping area and the overlapping area of the image.
According to the embodiment of the invention, the cameras arranged at different positions and angles of the vehicle form a binocular system, after the binocular SLAM algorithm in the binocular system is used for calculating the three-dimensional coordinates (namely the coordinates of a world coordinate system) of the pixel points in the image overlapping area, the three-dimensional coordinates of the pixel points in the image overlapping area can be converted into the coordinate information of the camera coordinate system of one camera according to the conversion mode of the coordinate value marks between different coordinate systems, and then the three-dimensional coordinates of the pixel points in the image non-overlapping area can be calculated according to the monocular SLAM algorithm and based on the coordinate information of the camera coordinate system obtained through conversion, therefore, the scheme of the embodiment can carry out scale calculation according to the pixel point information in the image overlapping area of the collected image, and the pixel point information of the image non-. Because the binocular SLAM algorithm has the advantages of no need of initialization, capability of acquiring more accurate scale information of an object, better stability and the like, the binocular SLAM algorithm can be used for assisting the monocular SLAM algorithm to acquire mature points in an acquired image more quickly and accurately, and therefore the stability of point cloud generation and the accuracy of acquiring the real scale of the object are improved.
Referring to the step 1, in an embodiment of the present invention, the process of selecting the pixel points in the image overlapping area may refer to the above embodiment, after the image acquired by the camera is acquired, the acquired image is input into the preset camera model, the camera model processes the image, and then the pixel points in the image overlapping area are selected from the processed image according to the initialized monocular SLAM algorithm.
Referring to step 4 above, in an embodiment of the present invention, a specific process of calculating three-dimensional coordinates of pixel points in a non-overlapping area of an image according to a pose change of a camera is to first calculate a displacement distance of the camera at any two different times by using an initialized monocular SLAM algorithm and according to the pose change of the camera. And then, calculating the three-dimensional coordinates of the pixel points in the non-overlapping region of the image according to the displacement distance by using the initialized monocular SLAM algorithm.
In this embodiment, the camera with the changed pose refers to a camera corresponding to one image of the selected pixel point. Therefore, the camera corresponding to the displacement distance calculated between any two different moments is the camera corresponding to the image of the selected pixel point. For example, two cameras at any angle are respectively installed in front of and at the side of the vehicle, and in the above embodiment, pixel points of any object in the image overlapping area are selected from the images collected by the cameras at the side.
In this embodiment, the displacement distance of a camera between any two different time instants can also be understood as the baseline distance of the camera between any two different time instants.
In an embodiment of the invention, the pose change of the camera may include a rotation matrix and/or a translation matrix of the camera.
Based on the same inventive concept, the embodiment of the invention also provides an initialization system of the monocular SLAM algorithm. Fig. 3 is a schematic structural diagram illustrating an initialization system of the monocular SLAM algorithm according to an embodiment of the present invention. Referring to fig. 3, the initialization system 300 for the monocular SLAM algorithm includes a monocular subsystem 320, a transformation subsystem 330, and a binocular subsystem 340.
Now, the functions of the components or devices of the initialization system 300 of the monocular SLAM algorithm and the connection relationship between the components will be described:
the monocular subsystem 320 selects pixel points in an image overlapping region from the acquired images, wherein the image overlapping region is a content overlapping region of two images acquired through different angles;
the conversion subsystem 330 is coupled with the monocular subsystem 320 and converts the selected pixel points into pixel points which are suitable for the binocular subsystem 340 and correspond to the binocular SLAM algorithm;
the binocular subsystem 340 is coupled with the conversion subsystem 330 and used for calculating the three-dimensional coordinates of the converted pixel points in the image overlapping area based on a binocular SLAM algorithm;
a conversion subsystem 330 for converting the three-dimensional coordinates of the pixel points into coordinate information of a camera coordinate system suitable for the monocular subsystem 320;
the monocular subsystem 320 initializes the monocular SLAM algorithm according to the coordinate information of the camera coordinate system.
In an embodiment of the present invention, the monocular subsystem 320 may further obtain a collected image, input the obtained image into a preset camera model, and select a pixel point in an image overlapping region according to a monocular SLAM algorithm from the processed image after the image is processed by the camera model.
In an embodiment of the present invention, when the conversion subsystem 330 converts the selected pixel point into a pixel point corresponding to the binocular SLAM algorithm, the selected pixel point may be converted into a pixel point corresponding to the binocular SLAM algorithm based on the image coordinate system.
In an embodiment of the present invention, the binocular subsystem 340 may first obtain a pre-measured baseline distance between cameras corresponding to the acquired images, and then calculate three-dimensional coordinates of the converted pixel points in the image overlapping area according to the baseline distance between the cameras based on a binocular SLAM algorithm.
In an embodiment of the present invention, after the monocular subsystem 320 initializes the monocular SLAM algorithm according to the coordinate information of the camera coordinate system, the corresponding monocular subsystem 320, the binocular subsystem 340, and the transformation subsystem 330 may further perform a process of generating an object point cloud, which includes the following specific processes:
and the monocular subsystem 320 selects pixel points in the image overlapping region from the acquired image according to the initialized monocular SLAM algorithm.
The image overlapping area is a content overlapping area of two images acquired by cameras at different angles. Moreover, since the camera mounted on the vehicle can capture images in real time, the monocular subsystem 320 may select pixel points within the image overlap region from the images captured in real time.
The conversion subsystem 330 converts the selected pixel points into pixel points which are suitable for the binocular subsystem 340 and correspond to the binocular SLAM algorithm.
And the binocular subsystem 340 calculates the three-dimensional coordinates of the converted pixel points in the image overlapping area based on a binocular SLAM algorithm.
The conversion subsystem 330 converts the three-dimensional coordinates of the pixel points into coordinate information of a camera coordinate system suitable for the monocular subsystem 320.
The monocular subsystem 320 calculates the pose change of the camera acquiring the image from an angle based on the coordinate information of the camera coordinate system by using a monocular SLAM algorithm, and calculates the three-dimensional coordinates of any object pixel point in the non-overlapping area of the image according to the pose change. The image non-overlapping area is a region where the contents of two images acquired through different angles do not overlap.
And the monocular subsystem 320 generates point clouds according to the three-dimensional coordinates of the pixel points in the non-overlapping area and the overlapping area of the images.
In an embodiment of the present invention, when the monocular subsystem 320 calculates the three-dimensional coordinates of the pixel points in the non-overlapping region of the image according to the pose change, the displacement distance between any two different times of the camera that collects the image from an angle may be calculated according to the pose change by the initialized monocular SLAM algorithm. And then, calculating the three-dimensional coordinates of the pixel points in the non-overlapping area of the image according to the displacement distance by using the initialized monocular SLAM algorithm.
In an embodiment of the invention, the pose change of the camera comprises a rotation matrix and/or a translation matrix of the camera.
An embodiment of the present invention further provides a computer storage medium, where a computer program code is stored, and when the computer program code runs on a computing device, the computer storage medium causes the computing device to execute the initialization method of the monocular SLAM algorithm in any of the above embodiments.
According to any one or a combination of the above preferred embodiments, the following advantages can be achieved by the embodiments of the present invention:
in the embodiment of the invention, after the pixel points in the image overlapping area are selected from the collected image and are converted into the pixel points corresponding to the binocular SLAM algorithm, the three-dimensional coordinates of the converted pixel points in the image overlapping area are calculated based on the binocular SLAM algorithm, the three-dimensional coordinates of the pixel points are converted into the coordinate information of a camera coordinate system, and then the monocular SLAM algorithm can be initialized according to the coordinate information of the camera coordinate system. Therefore, the embodiment of the invention can assist the monocular SLAM algorithm to realize initialization according to the binocular SLAM algorithm, so that the monocular system corresponding to the monocular SLAM algorithm can realize initialization more quickly and better, and the stability and the calculation precision of the monocular SLAM algorithm are improved.
It is clear to those skilled in the art that the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and for the sake of brevity, further description is omitted here.
In addition, the functional units in the embodiments of the present invention may be physically independent of each other, two or more functional units may be integrated together, or all the functional units may be integrated in one processing unit. The integrated functional units may be implemented in the form of hardware, or in the form of software or firmware.
Those of ordinary skill in the art will understand that: the integrated functional units, if implemented in software and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computing device (e.g., a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention when the instructions are executed. And the aforementioned storage medium includes: u disk, removable hard disk, Read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disk, and other various media capable of storing program code.
Alternatively, all or part of the steps of implementing the foregoing method embodiments may be implemented by hardware (such as a computing device, e.g., a personal computer, a server, or a network device) associated with program instructions, which may be stored in a computer-readable storage medium, and when the program instructions are executed by a processor of the computing device, the computing device executes all or part of the steps of the method according to the embodiments of the present invention.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments can be modified or some or all of the technical features can be equivalently replaced within the spirit and principle of the present invention; such modifications or substitutions do not depart from the scope of the present invention.
Claims (9)
1. A monocular SLAM algorithm initialization method comprises the following steps:
selecting pixel points in an image overlapping region from the collected images, wherein the image overlapping region is a content overlapping region of two images collected through different angles;
converting the selected pixel points into pixel points corresponding to a binocular SLAM algorithm, and calculating three-dimensional coordinates of the converted pixel points in an image overlapping area based on the binocular SLAM algorithm;
converting the three-dimensional coordinates of the pixel points into coordinate information of a camera coordinate system;
and initializing the monocular SLAM algorithm according to the coordinate information of the camera coordinate system.
2. The method of claim 1, wherein selecting pixel points within an image overlap region from the captured image comprises:
acquiring a collected image, and inputting the acquired image into a preset camera model;
and after the image is processed by the camera model, selecting pixel points in an image overlapping area from the processed image according to a monocular SLAM algorithm.
3. The method of claim 1 or 2, wherein converting the selected pixel points to pixel points corresponding to a binocular SLAM algorithm comprises:
and converting the selected pixel points into pixel points corresponding to a binocular SLAM algorithm based on an image coordinate system.
4. The method of claim 1 or 2, wherein calculating three-dimensional coordinates of the transformed pixel points in the image overlap region based on the binocular SLAM algorithm comprises:
acquiring a base line distance between cameras corresponding to pre-measured acquired images;
and calculating the three-dimensional coordinates of the converted pixel points in the image overlapping area based on the binocular SLAM algorithm and according to the baseline distance between the cameras.
5. The method of claim 1, wherein after initializing the monocular SLAM algorithm according to the coordinate information of the camera coordinate system, further comprising:
selecting pixel points in an image overlapping area from the collected images according to an initialized monocular SLAM algorithm, wherein the image overlapping area is a content overlapping area of two images collected from different angles;
converting the selected pixel points into pixel points corresponding to a binocular SLAM algorithm, and calculating three-dimensional coordinates of the converted pixel points in an image overlapping area based on the binocular SLAM algorithm;
converting the three-dimensional coordinates of the pixel points into coordinate information of a camera coordinate system, and calculating the pose change of a camera for collecting images from an angle based on the coordinate information of the camera coordinate system by using the initialized monocular SLAM algorithm;
calculating three-dimensional coordinates of pixel points in an image non-overlapping area according to the pose change, wherein the image non-overlapping area is an area where the contents of two images acquired through different angles do not overlap;
and generating a point cloud according to the three-dimensional coordinates of the pixel points in the non-overlapping area and the overlapping area of the image.
6. The method of claim 5, wherein calculating three-dimensional coordinates of pixel points within a non-overlapping region of the image from the pose changes comprises:
calculating the displacement distance between any two different moments of a camera which collects images from an angle through an initialized monocular SLAM algorithm according to the pose change;
and calculating the three-dimensional coordinates of the pixel points in the non-overlapping area of the image according to the displacement distance by using the initialized monocular SLAM algorithm.
7. The method of claim 5 or 6,
the pose change of the camera comprises: a rotation matrix and/or a translation matrix of the camera.
8. An initialization system of monocular SLAM algorithm comprises a monocular subsystem, a binocular subsystem and a conversion subsystem, wherein,
the monocular subsystem selects pixel points in an image overlapping area from the collected images, wherein the image overlapping area is a content overlapping area of two images collected through different angles;
the conversion subsystem converts the selected pixel points into pixel points which are suitable for the binocular subsystem and correspond to a binocular SLAM algorithm;
the binocular subsystem calculates the three-dimensional coordinates of the converted pixel points in the image overlapping area based on the binocular SLAM algorithm;
the conversion subsystem is used for converting the three-dimensional coordinates of the pixel points into coordinate information of a camera coordinate system suitable for the monocular subsystem;
and the monocular subsystem initializes the monocular SLAM algorithm according to the coordinate information of the camera coordinate system.
9. A computer storage medium having computer program code stored thereon which, when run on a computing device, causes the computing device to perform the method of initializing the monocular SLAM algorithm of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910319610.0A CN110060202B (en) | 2019-04-19 | 2019-04-19 | Monocular SLAM algorithm initialization method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910319610.0A CN110060202B (en) | 2019-04-19 | 2019-04-19 | Monocular SLAM algorithm initialization method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110060202A CN110060202A (en) | 2019-07-26 |
| CN110060202B true CN110060202B (en) | 2021-06-08 |
Family
ID=67319832
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910319610.0A Active CN110060202B (en) | 2019-04-19 | 2019-04-19 | Monocular SLAM algorithm initialization method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110060202B (en) |
Families Citing this family (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10678244B2 (en) | 2017-03-23 | 2020-06-09 | Tesla, Inc. | Data synthesis for autonomous control systems |
| US11409692B2 (en) | 2017-07-24 | 2022-08-09 | Tesla, Inc. | Vector computational unit |
| US10671349B2 (en) | 2017-07-24 | 2020-06-02 | Tesla, Inc. | Accelerated mathematical engine |
| US11157441B2 (en) | 2017-07-24 | 2021-10-26 | Tesla, Inc. | Computational array microprocessor system using non-consecutive data formatting |
| US11893393B2 (en) | 2017-07-24 | 2024-02-06 | Tesla, Inc. | Computational array microprocessor system with hardware arbiter managing memory requests |
| US11561791B2 (en) | 2018-02-01 | 2023-01-24 | Tesla, Inc. | Vector computational unit receiving data elements in parallel from a last row of a computational array |
| US11215999B2 (en) | 2018-06-20 | 2022-01-04 | Tesla, Inc. | Data pipeline and deep learning system for autonomous driving |
| US11361457B2 (en) | 2018-07-20 | 2022-06-14 | Tesla, Inc. | Annotation cross-labeling for autonomous control systems |
| US11636333B2 (en) | 2018-07-26 | 2023-04-25 | Tesla, Inc. | Optimizing neural network structures for embedded systems |
| US11562231B2 (en) | 2018-09-03 | 2023-01-24 | Tesla, Inc. | Neural networks for embedded devices |
| SG11202103493QA (en) | 2018-10-11 | 2021-05-28 | Tesla Inc | Systems and methods for training machine models with augmented data |
| US11196678B2 (en) | 2018-10-25 | 2021-12-07 | Tesla, Inc. | QOS manager for system on a chip communications |
| US11816585B2 (en) | 2018-12-03 | 2023-11-14 | Tesla, Inc. | Machine learning models operating at different frequencies for autonomous vehicles |
| US11537811B2 (en) | 2018-12-04 | 2022-12-27 | Tesla, Inc. | Enhanced object detection for autonomous vehicles based on field view |
| US11610117B2 (en) | 2018-12-27 | 2023-03-21 | Tesla, Inc. | System and method for adapting a neural network model on a hardware platform |
| US10997461B2 (en) | 2019-02-01 | 2021-05-04 | Tesla, Inc. | Generating ground truth for machine learning from time series elements |
| US11567514B2 (en) | 2019-02-11 | 2023-01-31 | Tesla, Inc. | Autonomous and user controlled vehicle summon to a target |
| US10956755B2 (en) | 2019-02-19 | 2021-03-23 | Tesla, Inc. | Estimating object properties using visual image data |
| CN113661513A (en) * | 2019-12-27 | 2021-11-16 | 深圳市大疆创新科技有限公司 | Image processing method, image processing device, image processing system and storage medium |
| CN111311684B (en) * | 2020-04-01 | 2021-02-05 | 亮风台（上海）信息科技有限公司 | Method and equipment for initializing SLAM |
| CN112348868A (en) * | 2020-11-06 | 2021-02-09 | 养哇(南京)科技有限公司 | Method and system for recovering monocular SLAM scale through detection and calibration |
| CN112380963A (en) * | 2020-11-11 | 2021-02-19 | 东软睿驰汽车技术（沈阳）有限公司 | Depth information determination method and device based on panoramic all-round looking system |
| CN112067007B (en) * | 2020-11-12 | 2021-01-29 | 湖北亿咖通科技有限公司 | Map generation method, computer storage medium, and electronic device |
| CN113763252B (en) * | 2021-09-16 | 2022-12-09 | 中国电子科技集团公司第五十四研究所 | Geodetic coordinate system and SLAM coordinate system conversion method for unmanned aerial vehicle |
| CN117434570B (en) * | 2023-12-20 | 2024-02-27 | 绘见科技(深圳)有限公司 | Visual measurement method, measurement device and storage medium for coordinates |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016021252A1 (en) * | 2014-08-05 | 2016-02-11 | ソニー株式会社 | Information processing device, information processing method, and image display system |
| CN106570913A (en) * | 2016-11-04 | 2017-04-19 | 上海玄彩美科网络科技有限公司 | Feature based monocular SLAM (Simultaneous Localization and Mapping) quick initialization method |
| CN106595639A (en) * | 2016-12-27 | 2017-04-26 | 纳恩博（北京）科技有限公司 | Positioning system and positioning method and device thereof and robot |
| CN107657640A (en) * | 2017-09-30 | 2018-02-02 | 南京大典科技有限公司 | Intelligent patrol inspection management method based on ORB SLAM |
| CN109493378A (en) * | 2018-10-29 | 2019-03-19 | 宁波研新工业科技有限公司 | A kind of measuring for verticality method combined based on monocular vision with binocular vision |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8930023B2 (en) * | 2009-11-06 | 2015-01-06 | Irobot Corporation | Localization by learning of wave-signal distributions |
-
2019
- 2019-04-19 CN CN201910319610.0A patent/CN110060202B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016021252A1 (en) * | 2014-08-05 | 2016-02-11 | ソニー株式会社 | Information processing device, information processing method, and image display system |
| CN106570913A (en) * | 2016-11-04 | 2017-04-19 | 上海玄彩美科网络科技有限公司 | Feature based monocular SLAM (Simultaneous Localization and Mapping) quick initialization method |
| CN106595639A (en) * | 2016-12-27 | 2017-04-26 | 纳恩博（北京）科技有限公司 | Positioning system and positioning method and device thereof and robot |
| CN107657640A (en) * | 2017-09-30 | 2018-02-02 | 南京大典科技有限公司 | Intelligent patrol inspection management method based on ORB SLAM |
| CN109493378A (en) * | 2018-10-29 | 2019-03-19 | 宁波研新工业科技有限公司 | A kind of measuring for verticality method combined based on monocular vision with binocular vision |
Non-Patent Citations (2)
| Title |
|---|
| Fast initialization for feature-based monocular slam;A. Shaobo 等;《2017 IEEE International Conference on Image Processing (ICIP)》;20171231;第2119-2123页 * |
| 室内环境下结合里程计的双目视觉SLAM研究;王晓华 等;《西安理工大学学报》;20090430;第25卷(第4期);第466-471页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110060202A (en) | 2019-07-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110060202B (en) | Monocular SLAM algorithm initialization method and system | |
| KR102275310B1 (en) | Mtehod of detecting obstacle around vehicle | |
| JP7444605B2 (en) | How to calculate the location of the tow hitch | |
| KR101842811B1 (en) | Driver assistance system for displaying surroundings of a vehicle | |
| JP6675448B2 (en) | Vehicle position detecting method and device | |
| JP5739584B2 (en) | 3D image synthesizing apparatus and method for visualizing vehicle periphery | |
| CN103778649A (en) | Imaging surface modeling for camera modeling and virtual view synthesis | |
| DK2239706T3 (en) | A method for real-time camera and obtaining visual information of three-dimensional scenes | |
| KR20090103165A (en) | Monocular Motion Stereo-Based Free Parking Space Detection Apparatus and Method | |
| KR20150008056A (en) | Single-camera distance estimation | |
| CN107590444B (en) | Method and device for detecting static obstacle and storage medium | |
| CN110084851B (en) | Binocular point cloud generation method and system | |
| CN103411536A (en) | Auxiliary driving obstacle detection method based on binocular stereoscopic vision | |
| JP7107931B2 (en) | Method and apparatus for estimating range of moving objects | |
| US9892519B2 (en) | Method for detecting an object in an environmental region of a motor vehicle, driver assistance system and motor vehicle | |
| US20160288711A1 (en) | Distance and direction estimation of a target point from a vehicle using monocular video camera | |
| GB2574697A (en) | Method, system and device of obtaining 3D-information of objects | |
| JP2023505891A (en) | Methods for measuring environmental topography | |
| EP2432660B1 (en) | Method and device for extending a visibility area | |
| CN107209930A (en) | Look around image stability method and device | |
| JP4696925B2 (en) | Image processing device | |
| KR101071061B1 (en) | Apparatus and method for driving assistance using feature of vehicle, and microprocessor and recording medium used thereto | |
| CN112270311B (en) | Near-target rapid detection method and system based on vehicle-mounted looking-around back projection | |
| GB2540527A (en) | Perspective shifting parking camera system | |
| JP2021086258A (en) | Attitude estimation apparatus and attitude estimation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220323 Address after: 430051 No. b1336, chuanggu startup area, taizihu cultural Digital Creative Industry Park, No. 18, Shenlong Avenue, Wuhan Economic and Technological Development Zone, Wuhan, Hubei Province Patentee after: Yikatong (Hubei) Technology Co.,Ltd. Address before: No.c101, chuanggu start up area, taizihu cultural Digital Industrial Park, No.18 Shenlong Avenue, Wuhan Economic Development Zone, Hubei Province Patentee before: HUBEI ECARX TECHNOLOGY Co.,Ltd. |
|
| TR01 | Transfer of patent right |