CN113465573A - Monocular distance measuring method and device and intelligent device - Google Patents
Monocular distance measuring method and device and intelligent device Download PDFInfo
- Publication number
- CN113465573A CN113465573A CN202110738325.XA CN202110738325A CN113465573A CN 113465573 A CN113465573 A CN 113465573A CN 202110738325 A CN202110738325 A CN 202110738325A CN 113465573 A CN113465573 A CN 113465573A
- Authority
- CN
- China
- Prior art keywords
- coordinate
- pixel
- reference point
- inverse perspective
- monocular camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C3/00—Measuring distances in line of sight; Optical rangefinders
- G01C3/26—Measuring distances in line of sight; Optical rangefinders using a parallactic triangle with fixed angles and a base of variable length, at, near, or formed by the object
- G01C3/28—Measuring distances in line of sight; Optical rangefinders using a parallactic triangle with fixed angles and a base of variable length, at, near, or formed by the object with provision for reduction of the distance into the horizontal plane
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/04—Interpretation of pictures
- G01C11/06—Interpretation of pictures by comparison of two or more pictures of the same area
- G01C11/08—Interpretation of pictures by comparison of two or more pictures of the same area the pictures not being supported in the same relative position as when they were taken
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/04—Interpretation of pictures
- G01C11/30—Interpretation of pictures by triangulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Multimedia (AREA)
- Electromagnetism (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The embodiment of the application provides a monocular distance measuring method, a monocular distance measuring device and an intelligent device, wherein the method comprises the following steps: shooting a plan view through a monocular camera; determining a homography matrix according to the vertex angle pixel coordinates of the first region image; determining a reference point from the bottom edge of the rectangular area, and acquiring the distance between the reference point and the monocular camera; determining the corresponding reference point pixel coordinate of the reference point in the first area image, and acquiring the inverse perspective reference point coordinate corresponding to the reference point pixel coordinate; carrying out target object detection on other flat views shot by the monocular camera to obtain a target object boundary frame, carrying out inverse perspective transformation on any pixel point coordinate on the bottom edge of the target object boundary frame according to the homography matrix, and obtaining a target inverse perspective coordinate corresponding to any pixel point coordinate on the bottom edge; and determining the distance between the target object and the monocular camera according to the acquired related parameters. Therefore, the distance of the target object can be measured through the monocular camera, the distance measuring condition is reduced, and the distance measuring efficiency is improved.
Description
Technical Field
The present disclosure relates to image processing technologies, and in particular, to a monocular distance measuring method, device and intelligent device.
Background
When a monocular camera is used for taking a picture, the monocular camera is actually used for taking a projection of a scene left on an imaging plane of the monocular camera, and the picture reflects a three-dimensional world in a two-dimensional form. Obviously, the depth information of the shooting scene is lost in the shooting process. In a monocular camera, the distance of an object in the scene from the monocular camera cannot be calculated from a single picture.
With the continuous development of artificial intelligence in recent years, the automatic driving technology based on artificial intelligence also receives wide attention, and distance measurement algorithms of monocular cameras based on depth learning are also proposed in succession, but most of the algorithms calculate the distance of an object through a neural network by knowing the imaging size and the posture of the object in a camera, the distance of the object under each posture needs to be acquired by the algorithm for an irregular object, the requirement on data is very strict, and the problem that the distance measurement condition is difficult to meet in the distance measurement technology of the existing monocular camera is caused.
Disclosure of Invention
In order to solve the technical problem, the embodiment of the application provides a monocular distance measuring method, a monocular distance measuring device and an intelligent device.
In a first aspect, an embodiment of the present application provides a monocular distance measuring method, where the method includes:
shooting a plane view through a monocular camera, wherein a ground visible area of the monocular camera comprises a rectangular area, and the plane view comprises a first area image corresponding to the rectangular area;
determining a homography matrix for inverse perspective transformation of the flat view into a top view according to the coordinates of the vertex pixels of the first region image;
determining a reference point from the bottom edge of the rectangular area, and acquiring the distance between the reference point and the monocular camera;
determining the corresponding reference point pixel coordinate of the reference point in the first area image, and acquiring the corresponding inverse perspective reference point coordinate of the reference point pixel coordinate;
carrying out target object detection on other flat views shot by the monocular camera to obtain a target object boundary frame, carrying out inverse perspective transformation on any pixel point coordinate on the bottom edge of the target object boundary frame according to the homography matrix, and obtaining a target inverse perspective coordinate corresponding to any pixel point coordinate on the bottom edge;
and determining the distance between the target object and the monocular camera according to the inverse perspective reference point coordinate, the target inverse perspective coordinate, the corresponding relation between the physical scale and the pixel scale and the distance between the reference point and the monocular camera.
Optionally, the determining a homography matrix for inverse perspective transforming the flat view into a top view according to the coordinates of the vertex pixels of the first region image includes:
acquiring vertex angle pixel coordinates of the first region image and corresponding inverse perspective vertex angle pixel coordinates;
and determining the homography matrix according to the vertex angle pixel coordinate of the first region image and the inverse perspective vertex angle pixel coordinate.
Optionally, the obtaining of the inverse perspective reference point coordinate corresponding to the reference point pixel coordinate includes:
and setting an inverse perspective coordinate of the pixel coordinate of the reference point according to the size of the top view.
Optionally, the obtaining of the vertex angle pixel coordinate of the first region image and the inverse perspective vertex angle pixel coordinate corresponding to the vertex angle pixel coordinate includes:
correcting the planar view to obtain a corrected planar view, wherein the corrected planar view comprises a second area image, and the vertex angle pixel coordinate of the second area image is used as the vertex angle pixel coordinate of the first area image;
and determining the pixel coordinate of the inverse perspective vertex angle according to the inverse perspective coordinate of the pixel coordinate of the reference point, the corresponding relation between the physical scale and the pixel scale and the side length of the rectangular area.
Optionally, the method further includes:
shooting a checkerboard image through the monocular camera, and acquiring internal parameters and distortion parameters of the monocular camera according to the checkerboard image;
the correcting the plan view to obtain a corrected plan view includes:
and correcting the plan view according to the internal parameters and the distortion parameters to obtain the corrected plan view.
Optionally, the inverse perspective coordinates of the reference point pixel coordinates include a first direction reference point pixel coordinate and a second direction reference point pixel coordinate, and the second direction is perpendicular to the first direction;
determining the inverse perspective vertex angle pixel coordinate according to the inverse perspective coordinate of the reference point pixel coordinate, the corresponding relation between the physical scale and the pixel scale, and the side length of the rectangular area, wherein the determining comprises the following steps:
and determining the first direction coordinate and the second direction coordinate of the inverse perspective vertex angle pixel coordinate according to the first direction reference point pixel coordinate, the second direction reference point pixel coordinate, the side length of the rectangular area, the corresponding relation between the physical scale and the pixel scale and the position relation between the inverse perspective vertex angle pixel coordinate and the inverse perspective reference point coordinate.
Optionally, the determining the distance between the target object and the monocular camera according to the inverse perspective reference point coordinate, the target inverse perspective coordinate, the corresponding relationship between the physical scale and the pixel scale, and the distance between the reference point and the monocular camera includes:
subtracting the pixel coordinate of the first direction reference point from the first direction coordinate of the target inverse perspective coordinate to obtain a first pixel difference value; multiplying the first pixel difference value by the corresponding relation between the physical scale and the pixel scale to obtain a first product; taking the first product as a first directional distance between the target object and the monocular camera; and/or the presence of a gas in the gas,
subtracting the pixel coordinate of the second direction reference point from the second direction coordinate of the target inverse perspective coordinate to obtain a second pixel difference value; multiplying the second pixel difference value by the corresponding relation between the physical scale and the pixel scale to obtain a second product; and adding the second product to the sum of the second direction distance between the reference point and the monocular camera to obtain the second direction distance between the target object and the monocular camera.
In a second aspect, an embodiment of the present application provides a monocular distance measuring device, where the monocular distance measuring device includes:
the device comprises a shooting module, a first image acquisition module, a second image acquisition module and a display module, wherein the shooting module is used for shooting a plane view through a monocular camera, the ground visible area of the monocular camera comprises a rectangular area, and the plane view comprises a first area image corresponding to the rectangular area;
the first determination module is used for determining a homography matrix for converting the flat view into a top view in an inverse perspective mode according to the coordinates of the vertex angle pixels of the first area image;
the acquisition module is used for determining a reference point from the bottom edge of the rectangular area and acquiring the distance between the reference point and the monocular camera;
the first processing module is used for determining the corresponding reference point pixel coordinate of the reference point in the first area image and acquiring the inverse perspective reference point coordinate corresponding to the reference point pixel coordinate;
the second processing module is used for detecting a target object for other flat views shot by the monocular camera to obtain a target object boundary frame, and carrying out inverse perspective transformation on any pixel point coordinate at the bottom edge of the target object boundary frame according to the homography matrix to obtain a target inverse perspective coordinate corresponding to any pixel point coordinate at the bottom edge;
and the second determining module is used for determining the distance between the target object and the monocular camera according to the inverse perspective reference point coordinate, the target inverse perspective coordinate, the corresponding relation between the physical scale and the pixel scale and the distance between the reference point and the monocular camera.
Optionally, the first determining module is further configured to obtain vertex angle pixel coordinates of the first region image and corresponding inverse perspective vertex angle pixel coordinates of the first region image;
and determining the homography matrix according to the vertex angle pixel coordinate of the first region image and the inverse perspective vertex angle pixel coordinate.
Optionally, the first determining module is further configured to set an inverse perspective coordinate of the pixel coordinate of the reference point according to the size of the top view.
Optionally, the first determining module is further configured to correct the planar view to obtain a corrected planar view, where the corrected planar view includes a second region image, and a vertex pixel coordinate of the second region image is used as a vertex pixel coordinate of the first region image;
and determining the pixel coordinate of the inverse perspective vertex angle according to the inverse perspective coordinate of the pixel coordinate of the reference point, the corresponding relation between the physical scale and the pixel scale and the side length of the rectangular area.
Optionally, the monocular distance measuring device further includes:
the correction module is used for shooting a checkerboard image through the monocular camera and acquiring internal parameters and distortion parameters of the monocular camera according to the checkerboard image;
the first determining module is further configured to correct the planar view according to the internal parameters and the distortion parameters to obtain the corrected planar view.
Optionally, the inverse perspective coordinates of the reference point pixel coordinates include a first direction reference point pixel coordinate and a second direction reference point pixel coordinate, and the second direction is perpendicular to the first direction;
the first determining module is further configured to determine a first direction coordinate and a second direction coordinate of the inverse perspective vertex angle pixel coordinate according to the first direction reference point pixel coordinate, the second direction reference point pixel coordinate, the side length of the rectangular region, the corresponding relationship between the physical dimension and the pixel dimension, and the position relationship between the inverse perspective vertex angle pixel coordinate and the inverse perspective reference point coordinate.
Optionally, the second determining module is further configured to subtract the first direction reference point pixel coordinate from the first direction coordinate of the target inverse perspective coordinate to obtain a first pixel difference value; multiplying the first pixel difference value by the corresponding relation between the physical scale and the pixel scale to obtain a first product; taking the first product as a first directional distance between the target object and the monocular camera; and/or the presence of a gas in the gas,
subtracting the pixel coordinate of the second direction reference point from the second direction coordinate of the target inverse perspective coordinate to obtain a second pixel difference value; multiplying the second pixel difference value by the corresponding relation between the physical scale and the pixel scale to obtain a second product; and adding the second product to the sum of the second direction distance between the reference point and the monocular camera to obtain the second direction distance between the target object and the monocular camera.
In a third aspect, an embodiment of the present application provides an intelligent device, which includes a monocular camera, a memory, and a processor, where the memory stores a computer program, and the computer program executes the monocular distance measuring method provided in the first aspect when the processor runs.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program runs on a processor, the computer program performs the monocular ranging method provided in the first aspect.
According to the monocular distance measuring method, the monocular distance measuring device and the intelligent device, the monocular camera is used for shooting the plane view; determining a homography matrix for inverse perspective transformation of the flat view into a top view according to the coordinates of the vertex pixels of the first region image; determining a reference point from the bottom edge of the rectangular area, and acquiring the distance between the reference point and the monocular camera; determining the corresponding reference point pixel coordinate of the reference point in the first area image, and acquiring the corresponding inverse perspective reference point coordinate of the reference point pixel coordinate; carrying out target object detection on other flat views shot by the monocular camera to obtain a target object boundary frame, carrying out inverse perspective transformation on any pixel point coordinate on the bottom edge of the target object boundary frame according to the homography matrix, and obtaining a target inverse perspective coordinate corresponding to any pixel point coordinate on the bottom edge; and determining the distance between the target object and the monocular camera according to the inverse perspective reference point coordinate, the target inverse perspective coordinate, the corresponding relation between the physical scale and the pixel scale and the distance between the reference point and the monocular camera. Therefore, the distance of the target object can be measured through the monocular camera, excessive data requirements do not exist, the distance measurement of irregular target objects can be realized, the distance measurement condition is reduced, and the distance measurement efficiency is improved.
Drawings
In order to more clearly explain the technical solutions of the present application, the drawings needed to be used in the embodiments are briefly introduced below, and it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope of protection of the present application. Like components are numbered similarly in the various figures.
Fig. 1 shows a flow chart of a monocular distance measuring method provided in an embodiment of the present application;
FIG. 2 is a schematic diagram illustrating a plan view provided by an embodiment of the present application;
FIG. 3 illustrates another schematic plan view provided by an embodiment of the present application;
FIG. 4 is a flow chart illustrating a corrected plan view provided by an embodiment of the present application;
FIG. 5 illustrates a schematic diagram of an inverted perspective view provided by embodiments of the present application;
FIG. 6 is a schematic diagram of a physical coordinate system provided by an embodiment of the present application;
fig. 7 shows a schematic structural diagram of a monocular distance measuring device provided in an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments.
The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
Hereinafter, the terms "including", "having", and their derivatives, which may be used in various embodiments of the present application, are intended to indicate only specific features, numbers, steps, operations, elements, components, or combinations of the foregoing, and should not be construed as first excluding the existence of, or adding to, one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing.
Furthermore, the terms "first," "second," "third," and the like are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the present application belong. The terms (such as those defined in commonly used dictionaries) should be interpreted as having a meaning that is consistent with their contextual meaning in the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein in various embodiments.
Example 1
The embodiment of the disclosure provides a monocular distance measuring method.
Specifically, as shown in fig. 1, the monocular distance measuring method includes
Step S101, a flat view is photographed by a monocular camera.
In this embodiment, the ground visible region of the monocular camera includes a rectangular region, and the plan view includes a first region image corresponding to the rectangular region.
It should be added that the floor may be laid with tiles of a rectangular pattern, a plurality of tiles of the rectangular pattern may form a rectangular area, and the corresponding first area image may be determined by performing image detection on the plan view. In addition, in other implementations, when the floor does not have a rectangular pattern such as tiles, a rectangular area may be provided on the floor, the rectangular area being provided in the floor visible area of the monocular camera.
Referring to fig. 2, the monocular camera 201 is disposed on a floor 202, the floor 202 is paved with tiles having a rectangular pattern, a plurality of tiles having a rectangular pattern may form a rectangular area, for example, tiles having 4 rectangular patterns form a rectangular area 203, and by performing image detection on a plan view, a corresponding first area image 203 may be determined.
And step S102, determining a homography matrix for converting the flat view into a top view in an inverse perspective manner according to the pixel coordinates of the top corner of the first region image.
Referring to fig. 2, a pixel coordinate system, i.e., the pixel coordinate of the pixel point A, B, C, D, may be established by using the top left corner of fig. 2 as the origin, the left vertical side as the y-axis, and the top as the x-axis.
In this embodiment, the homography matrix may be calculated from the coordinates of the corner pixels of the first region image and the inverse perspective coordinates of the corner pixels of the first region image in the later stage in advance.
Step S103, determining a reference point from the bottom edge of the rectangular area, and acquiring the distance between the reference point and the monocular camera.
In this embodiment, the reference point may be any point on the bottom side of the rectangular region. For example, the midpoint of the bottom side of the rectangular region may be used as a reference point.
In this embodiment, the distance between the reference point and the monocular camera is obtained by measuring the distance between the reference point and the monocular camera by a user, or performing image analysis processing according to the number of tiles of the rectangular pattern spaced between the reference point and the monocular camera, which is not limited herein.
Step S104, determining the corresponding reference point pixel coordinate of the reference point in the first area image, and acquiring the inverse perspective reference point coordinate corresponding to the reference point pixel coordinate.
In this embodiment, the rectangular area is located in the ground visible area of the monocular camera, and the first area image in the flat view includes all the sides of the rectangular area. When the center point of the bottom side of the rectangular region is taken as the reference point, as shown in fig. 2, the pixel coordinate of the reference point corresponding to the reference point in the first region image 203 is a pixel point E.
And step S105, carrying out target object detection on other flat views shot by the monocular camera to obtain a target object boundary frame, carrying out inverse perspective transformation on any pixel point coordinate on the bottom edge of the target object boundary frame according to the homography matrix, and obtaining a target inverse perspective coordinate corresponding to any pixel point coordinate on the bottom edge.
In this embodiment, the monocular camera takes other plan views while keeping its height and grip angle unchanged. Target detection fee algorithms such as Yolo V5, Fast-RCNN and the like can be selected for target object detection. The main purpose of target object detection is to outline the outline of an object to be detected by using a rectangular frame. Therefore, the bottom edge of the target object framed by the Bounding Box of the target detection is located on the calibrated two-dimensional plane.
Referring to FIG. 3, the first plane view 300 includes a target object 302 and a target object bounding box 301, and the target object bounding box 301 includes a bottom edge 3013, a top edge 3011, a left vertical edge 3012, and a right vertical edge 3013. And performing inverse perspective transformation on the coordinates of any pixel point on the bottom edge 3013 according to the homography matrix to obtain the target inverse perspective coordinates corresponding to the coordinates of any pixel point on the bottom edge 3013. For example, the coordinates of the midpoint in the bottom line 3013 are subjected to inverse perspective transformation according to the homography matrix, and the target inverse perspective coordinates corresponding to the coordinates of the midpoint in the bottom line 3013 are obtained.
In this embodiment, inverse perspective transformation is performed on the pixel point coordinates of the bottom edge midpoint of the target object boundary frame according to the homography matrix, so as to obtain target inverse perspective coordinates corresponding to the pixel point coordinates of the bottom edge midpoint.
In this embodiment, according to formula 1, inverse perspective transformation is performed on the pixel point coordinates of the bottom side midpoint of the target object boundary frame, so as to obtain target inverse perspective coordinates corresponding to the pixel point coordinates of the bottom side midpoint.
Equation 1:
wherein, (u, v) represents the pixel point coordinate of the bottom edge midpoint of the target object boundary frame, and (x, y) is the target inverse perspective coordinate corresponding to the pixel point coordinate of the bottom edge midpoint. Z denotes a depth value.
And step S106, determining the distance between the target object and the monocular camera according to the inverse perspective reference point coordinate, the target inverse perspective coordinate, the corresponding relation between the physical scale and the pixel scale and the distance between the reference point and the monocular camera.
In this embodiment, in order to correspond the pixel coordinates to the real physical coordinates, a corresponding relationship between the pixel scale and the physical scale is preset, that is, each pixel corresponds to the length of the real world. For example, 1 pixel corresponds to s meters of the real world.
Optionally, step S102 includes:
acquiring vertex angle pixel coordinates of the first region image and corresponding inverse perspective vertex angle pixel coordinates;
and determining the homography matrix according to the vertex angle pixel coordinate of the first region image and the inverse perspective vertex angle pixel coordinate.
In this embodiment, the coordinates of the corner pixels of the first region image are p1(u1，v1)，p2(u2，v2)，p3(u3，v3)，p4(u4，v4) The corresponding inverse perspective vertex angle pixel coordinates are q respectively1(x1，y1)，q2(x2，y2)，q3(x3，y3)，q4(x4，y4). According to p1(u1，v1)，p2(u2，v2)，p3(u3，v3)，p4(u4，v4)，q1(x1，y1)，q2(x2，y2)，q3(x3，y3)，q4(x4，y4) A homography matrix is calculated.
Optionally, in step S104, the acquiring an inverse perspective reference point coordinate corresponding to the reference point pixel coordinate includes:
and setting an inverse perspective coordinate of the pixel coordinate of the reference point according to the size of the top view.
In this embodiment, the size of the top view obtained by performing inverse perspective transformation on the plan view can be set. Half of the width of the top view may be set as a first direction pixel coordinate of an inverse perspective coordinate of the reference point pixel coordinate, and three quarters of the height of the top view may be set as a second direction pixel coordinate of the inverse perspective coordinate of the reference point pixel coordinate. In other embodiments, the height of the top view may also be set to a second direction pixel coordinate of an inverse perspective coordinate of the reference point pixel coordinate. And are not intended to be limiting herein.
Optionally, the obtaining of the vertex angle pixel coordinate of the first region image and the inverse perspective vertex angle pixel coordinate corresponding to the vertex angle pixel coordinate includes:
correcting the planar view to obtain a corrected planar view, wherein the corrected planar view comprises a second area image, and the vertex angle pixel coordinate of the second area image is used as the vertex angle pixel coordinate of the first area image;
and determining the pixel coordinate of the inverse perspective vertex angle according to the inverse perspective coordinate of the pixel coordinate of the reference point, the corresponding relation between the physical scale and the pixel scale and the side length of the rectangular area.
Referring to fig. 4, the vertex coordinates of the second area image 401 in the corrected flat view are the pixel coordinates of the pixel point F, G, H, I, and the edge of the second area image 401 is relatively straight compared to the first area image 203 before correction, thereby reducing the distortion effect.
Optionally, the method further includes:
shooting a checkerboard image through the monocular camera, and acquiring internal parameters and distortion parameters of the monocular camera according to the checkerboard image;
the correcting the plan view to obtain a corrected plan view includes:
and correcting the plan view according to the internal parameters and the distortion parameters to obtain the corrected plan view.
In this embodiment, in order to determine the relationship between the three-dimensional geometric position of a point on the surface of an object in space and the corresponding point in the image, a geometric model of the image of the camera must be established, and these geometric model parameters are the camera parameters. Under most conditions, the parameters can be obtained only through experiments and calculation, and the process of solving the internal parameter, the external parameter and the distortion parameter is called as camera calibration. The transformation from the image coordinate system to the camera coordinate system yields equation 1, where the 3 × 4 matrix on the right side of the equation is referred to as the camera's internal reference matrix, where fxAnd fyFocal lengths, u, of the x-axis and y-axis, respectively0And v0Is the optical center coordinate of the camera, and D is the physical scale parameter.
Equation 2:
in this embodiment, besides calibrating the internal parameters of the monocular camera, the distortion parameters of the monocular camera need to be calibrated. Distortion simply means that a straight line projected onto a picture cannot be maintained as a straight line due to a monocular camera lens, resulting in optical distortion. The camera distortion is mainly divided into two types, radial distortion and tangential distortion.
In this embodiment, the parameters of the monocular camera and the distortion parameters are obtained by taking a checkerboard image with the monocular camera. And carrying out distortion correction on the flat view shot by the monocular camera according to the calibrated camera internal parameters and the distortion parameters.
Optionally, the inverse perspective coordinates of the reference point pixel coordinates include a first direction reference point pixel coordinate and a second direction reference point pixel coordinate, and the second direction is perpendicular to the first direction;
determining the inverse perspective vertex angle pixel coordinate according to the inverse perspective coordinate of the reference point pixel coordinate, the corresponding relation between the physical scale and the pixel scale, and the side length of the rectangular area, wherein the determining comprises the following steps:
and determining the first direction coordinate and the second direction coordinate of the inverse perspective vertex angle pixel coordinate according to the first direction reference point pixel coordinate, the second direction reference point pixel coordinate, the side length of the rectangular area, the corresponding relation between the physical scale and the pixel scale and the position relation between the inverse perspective vertex angle pixel coordinate and the inverse perspective reference point coordinate.
In this embodiment, if the reference point is the bottom midpoint of the rectangular area, half of the width of the top view is set as the first direction pixel coordinate of the inverse perspective coordinate of the reference point pixel coordinate, three quarters of the height of the top view is set as the second direction pixel coordinate of the inverse perspective coordinate of the reference point pixel coordinate, and the inverse perspective coordinate of the reference point pixel coordinate is set to (x, y). Because the reference point is the middle point of the bottom edge of the rectangular area, the pixel coordinate of the inverse perspective vertex angle can be deduced according to the position relation between the pixel coordinate of the inverse perspective vertex angle and the coordinate of the inverse perspective reference point.
Referring to fig. 5, if the reference point is the middle point of the bottom side of the rectangular region, the inverse perspective coordinate of the pixel coordinate of the reference point is the coordinate of the pixel point N in fig. 5, and is set to (x, y), the inverse perspective vertex angle pixel coordinate corresponds to the coordinate of the pixel point J, K, L, M, if the length of the bottom side of the rectangular region is W meters, the length of the vertical side perpendicular to the bottom side is L meters, and each pixel point corresponds to s meters of the physical world, it can be obtained that the bottom side is W/s pixels in the inverse perspective transformation diagram, and the vertical side is L/s pixels in the inverse perspective transformation diagram. Coordinates of pixel point J, K, L, M, q1(x-W/2s，y-L/s)，q2(x+W/2s，y-L/s)，q3(x-W/2s，y)，q4(x+W/2s，y)。
It is further added that the homography matrix of the inverse perspective transformation is set according to equation 3:
equation 3:
from p1Inverse perspective conversion to q1Taking the correspondence between p1 and q1 as an example, first, p is defined as1And q is1Coordinate transformation to its secondary coordinate (u)1，v11) and (x)1，y11), both can be given formula 4 by homography:
expanding the equation yields equation 5-equation 7:
equation 5: h is1u1+h2v1+g3＝x1；
Equation 6: h is4u1+h5v1+h6＝y1；
Equation 7: h is7u1+h8v1+h9＝1；
The homography matrix has the advantage of no dimension deformation, so that h can be ordered9When h is equal to 17u1+h8v1When the value is 0, the formula 8-the formula 9 are substituted;
equation 8: h is1u1+h2v1+h3-h7u1x1-h8v1y1＝x1；
Equation 9: h is4u1+h5v1+h6-h7u1x1-h8v1y1＝y1；
With the correspondence of p2 and q2, equations 10-11 can be derived:
equation 10: h is1u2+h2u2+h3-h7u2x2-h8v2y2＝x2；
Equation 11: h is4u2+h5v2+h6-h7u2x2-h8v2y2＝y2；
From the correspondence of p2 and q2, equations 12-13 can be derived:
equation 12: h is1u3+h2v3+h3-h7u3x3-h8v3y3＝x3；
Equation 13: h is4u3+h5v3+h6-h7u3x3-h8v3y3＝y3；
From the correspondence of p3 and q3, equations 14-15 can be derived:
equation 14: h is1u4+h2v4+h3-h7u4x4-h8v4y4＝x4；
Equation 15: h is4u4+h5v3+h6-h7u44-h8v4y4＝y4；
p1(u1，v1)，p2(u2，v2)，p3(u3，v3)，p4(u4，v4) Can be directly read from a flat view, and the corresponding inverse perspective vertex angle pixel coordinate q1(x1，y1)，q2(x2，y2)，q3(x3，y3)，q4(x4，y4) By q1(x-W/2s，y-L/s)，q2(x+W/2s，y-L/s)，q3(x-W/2s，y)，q4And (x + W/2s, y) replacing, substituting into a formula 8-a formula 15, and calculating to obtain a homography matrix H.
Optionally, step S106 includes:
subtracting the pixel coordinate of the first direction reference point from the first direction coordinate of the target inverse perspective coordinate to obtain a first pixel difference value; multiplying the first pixel difference value by the corresponding relation between the physical scale and the pixel scale to obtain a first product; taking the first product as a first directional distance between the target object and the monocular camera; and/or the presence of a gas in the gas,
subtracting the pixel coordinate of the second direction reference point from the second direction coordinate of the target inverse perspective coordinate to obtain a second pixel difference value; multiplying the second pixel difference value by the corresponding relation between the physical scale and the pixel scale to obtain a second product; and adding the second product to the sum of the second direction distance between the reference point and the monocular camera to obtain the second direction distance between the target object and the monocular camera.
In this embodiment, in order to calculate the physical coordinates of the road pixel points in the other plan views, the origin of the physical coordinate system needs to be determined. As shown in fig. 6, the projected point of the monocular camera on the road surface is set as the origin of the physical coordinate system, the road surface is set as a 2D plane, the line of sight parallel to the camera is set as the positive y-axis direction, and the right side perpendicular to the y-axis is set as the positive x-axis direction. Measuring the distance between the origin of the coordinate system and the reference point of the rectangular area, specifically, in this embodiment, the reference point of the rectangular area is the bottom side middle point, and measuring the y-direction distance L between the origin of the coordinate system and the bottom side middle point1And (4) rice. Since the image coordinates of the middle point of the bottom side of the rectangle are (x, y), it can be seen that the coordinates of the target inverse perspective coordinates (a, b) of any pixel point at the bottom side of the target object bounding box in the other flat view in the physical coordinate system are ((a-x) s, (b-y) s + L)1). Wherein (a-x) s and (b-y) s + L1The real physical distances between the target object and the monocular camera in the x-axis direction and the y-axis direction are respectively represented, and the unit is meter.
Note that the target inverse perspective coordinate (a) of the coordinate of the midpoint of the bottom side of the target object bounding box in the other plan views is1，b1) The coordinates in the physical coordinate system are ((a)1-x)s，(b1-y)s+L1)。Wherein (a)1-x) s and (b)1-y)s+L11And respectively representing the real physical distances of the middle points of the bottom edges of the target object bounding boxes in the x-axis direction and the y-axis direction, wherein the unit is meter.
According to the monocular distance measuring method provided by the embodiment, a monocular camera is used for shooting a plane view; determining a homography matrix for inverse perspective transformation of the flat view into a top view according to the coordinates of the vertex pixels of the first region image; determining a reference point from the bottom edge of the rectangular area, and acquiring the distance between the reference point and the monocular camera; determining the corresponding reference point pixel coordinate of the reference point in the first area image, and acquiring the corresponding inverse perspective reference point coordinate of the reference point pixel coordinate; carrying out target object detection on other flat views shot by the monocular camera to obtain a target object boundary frame, carrying out inverse perspective transformation on any pixel point coordinate on the bottom edge of the target object boundary frame according to the homography matrix, and obtaining a target inverse perspective coordinate corresponding to any pixel point coordinate on the bottom edge; and determining the distance between the target object and the monocular camera according to the inverse perspective reference point coordinate, the target inverse perspective coordinate, the corresponding relation between the physical scale and the pixel scale and the distance between the reference point and the monocular camera. Therefore, the distance of the target object can be measured through the monocular camera, excessive data requirements do not exist, the distance measurement of irregular target objects can be realized, the distance measurement condition is reduced, and the distance measurement efficiency is improved.
Example 2
In addition, the embodiment of the disclosure provides a monocular distance measuring device.
In this embodiment, the monocular distance measuring device may be an intelligent device such as an intelligent vehicle or a robot.
Specifically, as shown in fig. 7, the monocular distance measuring device 700 includes:
a shooting module 701, configured to shoot a plane view through a monocular camera, where a ground visible region of the monocular camera includes a rectangular region, and the plane view includes a first region image corresponding to the rectangular region;
a first determining module 702, configured to determine a homography matrix for inverse perspective transformation of the flat view into a top view according to coordinates of corner pixels of the first region image;
an obtaining module 703, configured to determine a reference point from a bottom edge of the rectangular region, and obtain a distance between the reference point and the monocular camera;
a first processing module 704, configured to determine a reference point pixel coordinate corresponding to the reference point in the first area image, and obtain an inverse perspective reference point coordinate corresponding to the reference point pixel coordinate;
a second processing module 705, configured to perform target object detection on other flat views shot by the monocular camera to obtain a target object boundary box, perform inverse perspective transformation on any pixel coordinate at the bottom edge of the target object boundary box according to the homography matrix, and obtain a target inverse perspective coordinate corresponding to any pixel coordinate at the bottom edge;
a second determining module 706, configured to determine a distance between the target object and the monocular camera according to the inverse perspective reference point coordinate, the target inverse perspective coordinate, a correspondence between a physical scale and a pixel scale, and a distance between the reference point and the monocular camera.
Optionally, the first determining module 702 is further configured to obtain vertex angle pixel coordinates of the first region image and corresponding inverse perspective vertex angle pixel coordinates thereof;
and determining the homography matrix according to the vertex angle pixel coordinate of the first region image and the inverse perspective vertex angle pixel coordinate.
Optionally, the first determining module 702 is further configured to set an inverse perspective coordinate of the pixel coordinate of the reference point according to the size of the top view.
Optionally, the first determining module 702 is further configured to correct the planar view to obtain a corrected planar view, where the corrected planar view includes a second area image, and a vertex pixel coordinate of the second area image is used as a vertex pixel coordinate of the first area image;
and determining the pixel coordinate of the inverse perspective vertex angle according to the inverse perspective coordinate of the pixel coordinate of the reference point, the corresponding relation between the physical scale and the pixel scale and the side length of the rectangular area.
Optionally, the monocular distance measuring device 700 further includes:
the correction module is used for shooting a checkerboard image through the monocular camera and acquiring internal parameters and distortion parameters of the monocular camera according to the checkerboard image;
the first determining module 702 is further configured to correct the planar view according to the internal parameter and the distortion parameter, so as to obtain the corrected planar view.
Optionally, the inverse perspective coordinates of the reference point pixel coordinates include a first direction reference point pixel coordinate and a second direction reference point pixel coordinate, and the second direction is perpendicular to the first direction;
the first determining module 702 is further configured to determine a first direction coordinate and a second direction coordinate of the inverse perspective vertex angle pixel coordinate according to the first direction reference point pixel coordinate, the second direction reference point pixel coordinate, the side length of the rectangular region, the corresponding relationship between the physical dimension and the pixel dimension, and the position relationship between the inverse perspective vertex angle pixel coordinate and the inverse perspective reference point coordinate.
Optionally, the second determining module 706 is further configured to subtract the first direction coordinate of the target inverse perspective coordinate from the first direction reference point pixel coordinate to obtain a first pixel difference value; multiplying the first pixel difference value by the corresponding relation between the physical scale and the pixel scale to obtain a first product; taking the first product as a first directional distance between the target object and the monocular camera; and/or the presence of a gas in the gas,
subtracting the pixel coordinate of the second direction reference point from the second direction coordinate of the target inverse perspective coordinate to obtain a second pixel difference value; multiplying the second pixel difference value by the corresponding relation between the physical scale and the pixel scale to obtain a second product; and adding the second product to the sum of the second direction distance between the reference point and the monocular camera to obtain the second direction distance between the target object and the monocular camera.
The monocular distance measuring device 700 provided in this embodiment may implement the monocular distance measuring method shown in embodiment 1, and is not described herein again to avoid repetition.
The monocular distance measuring device provided by the embodiment shoots a plane view through the monocular camera; determining a homography matrix for inverse perspective transformation of the flat view into a top view according to the coordinates of the vertex pixels of the first region image; determining a reference point from the bottom edge of the rectangular area, and acquiring the distance between the reference point and the monocular camera; determining the corresponding reference point pixel coordinate of the reference point in the first area image, and acquiring the corresponding inverse perspective reference point coordinate of the reference point pixel coordinate; carrying out target object detection on other flat views shot by the monocular camera to obtain a target object boundary frame, carrying out inverse perspective transformation on any pixel point coordinate on the bottom edge of the target object boundary frame according to the homography matrix, and obtaining a target inverse perspective coordinate corresponding to any pixel point coordinate on the bottom edge; and determining the distance between the target object and the monocular camera according to the inverse perspective reference point coordinate, the target inverse perspective coordinate, the corresponding relation between the physical scale and the pixel scale and the distance between the reference point and the monocular camera. Therefore, the distance of the target object can be measured through the monocular camera, excessive data requirements do not exist, the distance measurement of irregular target objects can be realized, the distance measurement condition is reduced, and the distance measurement efficiency is improved.
Example 3
Furthermore, an embodiment of the present disclosure provides an intelligent device, which includes a monocular camera, a memory, and a processor, where the memory stores a computer program, and the computer program executes the monocular distance measuring method provided in the foregoing method embodiment 1 when running on the processor.
In this embodiment, the intelligent device may be an intelligent device such as an intelligent vehicle or a robot.
Wherein the processor is configured to: shooting a plane view through a monocular camera, wherein a ground visible area of the monocular camera comprises a rectangular area, and the plane view comprises a first area image corresponding to the rectangular area;
determining a homography matrix for inverse perspective transformation of the flat view into a top view according to the coordinates of the vertex pixels of the first region image;
determining a reference point from the bottom edge of the rectangular area, and acquiring the distance between the reference point and the monocular camera;
determining the corresponding reference point pixel coordinate of the reference point in the first area image, and acquiring the corresponding inverse perspective reference point coordinate of the reference point pixel coordinate;
carrying out target object detection on other flat views shot by the monocular camera to obtain a target object boundary frame, carrying out inverse perspective transformation on any pixel point coordinate on the bottom edge of the target object boundary frame according to the homography matrix, and obtaining a target inverse perspective coordinate corresponding to any pixel point coordinate on the bottom edge;
and determining the distance between the target object and the monocular camera according to the inverse perspective reference point coordinate, the target inverse perspective coordinate, the corresponding relation between the physical scale and the pixel scale and the distance between the reference point and the monocular camera.
Optionally, the processor is further configured to: acquiring vertex angle pixel coordinates of the first region image and corresponding inverse perspective vertex angle pixel coordinates;
and determining the homography matrix according to the vertex angle pixel coordinate of the first region image and the inverse perspective vertex angle pixel coordinate.
Optionally, the processor is further configured to: and setting an inverse perspective coordinate of the pixel coordinate of the reference point according to the size of the top view.
Optionally, the processor is further configured to: correcting the planar view to obtain a corrected planar view, wherein the corrected planar view comprises a second area image, and the vertex angle pixel coordinate of the second area image is used as the vertex angle pixel coordinate of the first area image;
and determining the pixel coordinate of the inverse perspective vertex angle according to the inverse perspective coordinate of the pixel coordinate of the reference point, the corresponding relation between the physical scale and the pixel scale and the side length of the rectangular area.
Optionally, the processor is further configured to: shooting a checkerboard image through the monocular camera, and acquiring internal parameters and distortion parameters of the monocular camera according to the checkerboard image;
and correcting the plan view according to the internal parameters and the distortion parameters to obtain the corrected plan view.
Optionally, the inverse perspective coordinates of the reference point pixel coordinates include a first direction reference point pixel coordinate and a second direction reference point pixel coordinate, and the second direction is perpendicular to the first direction;
a processor further configured to: and determining the first direction coordinate and the second direction coordinate of the inverse perspective vertex angle pixel coordinate according to the first direction reference point pixel coordinate, the second direction reference point pixel coordinate, the side length of the rectangular area, the corresponding relation between the physical scale and the pixel scale and the position relation between the inverse perspective vertex angle pixel coordinate and the inverse perspective reference point coordinate.
Optionally, the processor is further configured to: subtracting the pixel coordinate of the first direction reference point from the first direction coordinate of the target inverse perspective coordinate to obtain a first pixel difference value; multiplying the first pixel difference value by the corresponding relation between the physical scale and the pixel scale to obtain a first product; taking the first product as a first directional distance between the target object and the monocular camera; and/or the presence of a gas in the gas,
subtracting the pixel coordinate of the second direction reference point from the second direction coordinate of the target inverse perspective coordinate to obtain a second pixel difference value; multiplying the second pixel difference value by the corresponding relation between the physical scale and the pixel scale to obtain a second product; and adding the second product to the sum of the second direction distance between the reference point and the monocular camera to obtain the second direction distance between the target object and the monocular camera.
The intelligent device provided in this embodiment can implement the monocular distance measurement method shown in embodiment 1, and is not described herein again to avoid repetition.
The intelligent device provided by the embodiment shoots a plan view through the monocular camera; determining a homography matrix for inverse perspective transformation of the flat view into a top view according to the coordinates of the vertex pixels of the first region image; determining a reference point from the bottom edge of the rectangular area, and acquiring the distance between the reference point and the monocular camera; determining the corresponding reference point pixel coordinate of the reference point in the first area image, and acquiring the corresponding inverse perspective reference point coordinate of the reference point pixel coordinate; carrying out target object detection on other flat views shot by the monocular camera to obtain a target object boundary frame, carrying out inverse perspective transformation on any pixel point coordinate on the bottom edge of the target object boundary frame according to the homography matrix, and obtaining a target inverse perspective coordinate corresponding to any pixel point coordinate on the bottom edge; and determining the distance between the target object and the monocular camera according to the inverse perspective reference point coordinate, the target inverse perspective coordinate, the corresponding relation between the physical scale and the pixel scale and the distance between the reference point and the monocular camera. Therefore, the distance of the target object can be measured through the monocular camera, excessive data requirements do not exist, the distance measurement of irregular target objects can be realized, the distance measurement condition is reduced, and the distance measurement efficiency is improved.
Example 4
The application also provides a computer readable storage medium on which a computer program is stored, a monocular camera is used for shooting a plane view, the ground visible area of the monocular camera comprises a rectangular area, and the plane view comprises a first area image corresponding to the rectangular area;
determining a homography matrix for inverse perspective transformation of the flat view into a top view according to the coordinates of the vertex pixels of the first region image;
determining a reference point from the bottom edge of the rectangular area, and acquiring the distance between the reference point and the monocular camera;
determining the corresponding reference point pixel coordinate of the reference point in the first area image, and acquiring the corresponding inverse perspective reference point coordinate of the reference point pixel coordinate;
carrying out target object detection on other flat views shot by the monocular camera to obtain a target object boundary frame, carrying out inverse perspective transformation on any pixel point coordinate on the bottom edge of the target object boundary frame according to the homography matrix, and obtaining a target inverse perspective coordinate corresponding to any pixel point coordinate on the bottom edge;
and determining the distance between the target object and the monocular camera according to the inverse perspective reference point coordinate, the target inverse perspective coordinate, the corresponding relation between the physical scale and the pixel scale and the distance between the reference point and the monocular camera.
The computer program when executed by a processor implements the steps of:
acquiring vertex angle pixel coordinates of the first region image and corresponding inverse perspective vertex angle pixel coordinates;
and determining the homography matrix according to the vertex angle pixel coordinate of the first region image and the inverse perspective vertex angle pixel coordinate.
Optionally, the computer program further implements the following steps when executed by the processor:
and setting an inverse perspective coordinate of the pixel coordinate of the reference point according to the size of the top view.
Optionally, the computer program further implements the following steps when executed by the processor:
correcting the planar view to obtain a corrected planar view, wherein the corrected planar view comprises a second area image, and the vertex angle pixel coordinate of the second area image is used as the vertex angle pixel coordinate of the first area image;
and determining the pixel coordinate of the inverse perspective vertex angle according to the inverse perspective coordinate of the pixel coordinate of the reference point, the corresponding relation between the physical scale and the pixel scale and the side length of the rectangular area.
Optionally, the computer program further implements the following steps when executed by the processor:
shooting a checkerboard image through the monocular camera, and acquiring internal parameters and distortion parameters of the monocular camera according to the checkerboard image;
and correcting the plan view according to the internal parameters and the distortion parameters to obtain the corrected plan view.
Optionally, the inverse perspective coordinates of the reference point pixel coordinates include a first direction reference point pixel coordinate and a second direction reference point pixel coordinate, and the second direction is perpendicular to the first direction;
the computer program when executed by a processor further realizes the steps of:
and determining the first direction coordinate and the second direction coordinate of the inverse perspective vertex angle pixel coordinate according to the first direction reference point pixel coordinate, the second direction reference point pixel coordinate, the side length of the rectangular area, the corresponding relation between the physical scale and the pixel scale and the position relation between the inverse perspective vertex angle pixel coordinate and the inverse perspective reference point coordinate.
Optionally, the computer program further implements the following steps when executed by the processor:
subtracting the pixel coordinate of the first direction reference point from the first direction coordinate of the target inverse perspective coordinate to obtain a first pixel difference value; multiplying the first pixel difference value by the corresponding relation between the physical scale and the pixel scale to obtain a first product; taking the first product as a first directional distance between the target object and the monocular camera; and/or the presence of a gas in the gas,
subtracting the pixel coordinate of the second direction reference point from the second direction coordinate of the target inverse perspective coordinate to obtain a second pixel difference value; multiplying the second pixel difference value by the corresponding relation between the physical scale and the pixel scale to obtain a second product; and adding the second product to the sum of the second direction distance between the reference point and the monocular camera to obtain the second direction distance between the target object and the monocular camera.
In this embodiment, the computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.
The computer-readable storage medium provided in this embodiment may implement the monocular distance measuring method shown in embodiment 1, and is not described herein again to avoid repetition.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present application.
While the present embodiments have been described with reference to the accompanying drawings, it is to be understood that the invention is not limited to the precise embodiments described above, which are meant to be illustrative and not restrictive, and that various changes may be made therein by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (10)
1. A monocular distance measuring method, comprising:
shooting a plane view through a monocular camera, wherein a ground visible area of the monocular camera comprises a rectangular area, and the plane view comprises a first area image corresponding to the rectangular area;
determining a homography matrix for inverse perspective transformation of the flat view into a top view according to the coordinates of the vertex pixels of the first region image;
determining a reference point from the bottom edge of the rectangular area, and acquiring the distance between the reference point and the monocular camera;
determining the corresponding reference point pixel coordinate of the reference point in the first area image, and acquiring the corresponding inverse perspective reference point coordinate of the reference point pixel coordinate;
carrying out target object detection on other flat views shot by the monocular camera to obtain a target object boundary frame, carrying out inverse perspective transformation on any pixel point coordinate on the bottom edge of the target object boundary frame according to the homography matrix, and obtaining a target inverse perspective coordinate corresponding to any pixel point coordinate on the bottom edge;
and determining the distance between the target object and the monocular camera according to the inverse perspective reference point coordinate, the target inverse perspective coordinate, the corresponding relation between the physical scale and the pixel scale and the distance between the reference point and the monocular camera.
2. The method of claim 1, wherein determining a homography matrix for inverse perspective transformation of the plan view into a top view from vertex pixel coordinates of the first region image comprises:
acquiring vertex angle pixel coordinates of the first region image and corresponding inverse perspective vertex angle pixel coordinates;
and determining the homography matrix according to the vertex angle pixel coordinate of the first region image and the inverse perspective vertex angle pixel coordinate.
3. The method of claim 2, wherein said obtaining inverted perspective fiducial coordinates corresponding to said fiducial pixel coordinates comprises:
and setting an inverse perspective coordinate of the pixel coordinate of the reference point according to the size of the top view.
4. The method of claim 3, wherein the obtaining of vertex pixel coordinates and corresponding inverse perspective vertex pixel coordinates of the first region image comprises:
correcting the planar view to obtain a corrected planar view, wherein the corrected planar view comprises a second area image, and the vertex angle pixel coordinate of the second area image is used as the vertex angle pixel coordinate of the first area image;
and determining the pixel coordinate of the inverse perspective vertex angle according to the inverse perspective coordinate of the pixel coordinate of the reference point, the corresponding relation between the physical scale and the pixel scale and the side length of the rectangular area.
5. The method of claim 4, further comprising:
shooting a checkerboard image through the monocular camera, and acquiring internal parameters and distortion parameters of the monocular camera according to the checkerboard image;
the correcting the plan view to obtain a corrected plan view includes:
and correcting the plan view according to the internal parameters and the distortion parameters to obtain the corrected plan view.
6. The monocular distance measuring method of claim 4, wherein the inverse perspective coordinates of the reference point pixel coordinates comprise a first direction reference point pixel coordinate and a second direction reference point pixel coordinate, the second direction being perpendicular to the first direction;
determining the inverse perspective vertex angle pixel coordinate according to the inverse perspective coordinate of the reference point pixel coordinate, the corresponding relation between the physical scale and the pixel scale, and the side length of the rectangular area, wherein the determining comprises the following steps:
and determining the first direction coordinate and the second direction coordinate of the inverse perspective vertex angle pixel coordinate according to the first direction reference point pixel coordinate, the second direction reference point pixel coordinate, the side length of the rectangular area, the corresponding relation between the physical scale and the pixel scale and the position relation between the inverse perspective vertex angle pixel coordinate and the inverse perspective reference point coordinate.
7. The monocular distance measuring method of claim 6, wherein the determining the distance between the target object and the monocular camera based on the inverse perspective reference point coordinate, the target inverse perspective coordinate, the correspondence between physical scale and pixel scale, and the distance between the reference point and the monocular camera comprises:
subtracting the pixel coordinate of the first direction reference point from the first direction coordinate of the target inverse perspective coordinate to obtain a first pixel difference value; multiplying the first pixel difference value by the corresponding relation between the physical scale and the pixel scale to obtain a first product; taking the first product as a first directional distance between the target object and the monocular camera; and/or the presence of a gas in the gas,
subtracting the pixel coordinate of the second direction reference point from the second direction coordinate of the target inverse perspective coordinate to obtain a second pixel difference value; multiplying the second pixel difference value by the corresponding relation between the physical scale and the pixel scale to obtain a second product; and adding the second product to the sum of the second direction distance between the reference point and the monocular camera to obtain the second direction distance between the target object and the monocular camera.
8. A monocular distance measuring device, the device comprising:
the device comprises a shooting module, a first image acquisition module, a second image acquisition module and a display module, wherein the shooting module is used for shooting a plane view through a monocular camera, the ground visible area of the monocular camera comprises a rectangular area, and the plane view comprises a first area image corresponding to the rectangular area;
the first determination module is used for determining a homography matrix for converting the flat view into a top view in an inverse perspective mode according to the coordinates of the vertex angle pixels of the first area image;
the acquisition module is used for determining a reference point from the bottom edge of the rectangular area and acquiring the distance between the reference point and the monocular camera;
the first processing module is used for determining the corresponding reference point pixel coordinate of the reference point in the first area image and acquiring the inverse perspective reference point coordinate corresponding to the reference point pixel coordinate;
the second processing module is used for detecting a target object for other flat views shot by the monocular camera to obtain a target object boundary frame, and carrying out inverse perspective transformation on any pixel point coordinate at the bottom edge of the target object boundary frame according to the homography matrix to obtain a target inverse perspective coordinate corresponding to any pixel point coordinate at the bottom edge;
and the second determining module is used for determining the distance between the target object and the monocular camera according to the inverse perspective reference point coordinate, the target inverse perspective coordinate, the corresponding relation between the physical scale and the pixel scale and the distance between the reference point and the monocular camera.
9. An intelligent device comprising a monocular camera, a memory, and a processor, the memory storing a computer program that, when executed by the processor, performs the monocular ranging method of any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that it stores a computer program which, when run on a processor, performs the monocular ranging method of any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110738325.XA CN113465573A (en) | 2021-06-30 | 2021-06-30 | Monocular distance measuring method and device and intelligent device |
| PCT/CN2021/131460 WO2023273108A1 (en) | 2021-06-30 | 2021-11-18 | Monocular distance measurement method and apparatus, and intelligent apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110738325.XA CN113465573A (en) | 2021-06-30 | 2021-06-30 | Monocular distance measuring method and device and intelligent device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113465573A true CN113465573A (en) | 2021-10-01 |
Family
ID=77876665
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110738325.XA Pending CN113465573A (en) | 2021-06-30 | 2021-06-30 | Monocular distance measuring method and device and intelligent device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113465573A (en) |
| WO (1) | WO2023273108A1 (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114049394A (en) * | 2021-11-23 | 2022-02-15 | 智道网联科技(北京)有限公司 | Monocular distance measuring method, device, equipment and storage medium |
| WO2023273108A1 (en) * | 2021-06-30 | 2023-01-05 | 深圳市优必选科技股份有限公司 | Monocular distance measurement method and apparatus, and intelligent apparatus |
| CN116681778A (en) * | 2023-06-06 | 2023-09-01 | 固安信通信号技术股份有限公司 | Distance measurement method based on monocular camera |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104504680A (en) * | 2014-11-23 | 2015-04-08 | 北京联合大学 | Inverse perspective calibration method used for intelligent vehicle |
| WO2019184885A1 (en) * | 2018-03-30 | 2019-10-03 | 杭州海康威视数字技术股份有限公司 | Method, apparatus and electronic device for calibrating extrinsic parameters of camera |
| CN110599548A (en) * | 2019-09-02 | 2019-12-20 | Oppo广东移动通信有限公司 | Camera calibration method and device, camera and computer readable storage medium |
| CN111047536A (en) * | 2019-12-18 | 2020-04-21 | 深圳市汉森软件有限公司 | CCD image correction method, device, equipment and storage medium |
| CN111474519A (en) * | 2020-04-28 | 2020-07-31 | 广东博智林机器人有限公司 | Positioning method, device, equipment and storage medium |
| CN111521117A (en) * | 2019-08-21 | 2020-08-11 | 长城汽车股份有限公司 | Monocular vision distance measuring method, storage medium and monocular camera |
| CN112669382A (en) * | 2020-12-30 | 2021-04-16 | 联想未来通信科技(重庆)有限公司 | Image-based distance determination method and device |
| CN112665577A (en) * | 2020-12-29 | 2021-04-16 | 北京电子工程总体研究所 | Monocular vision target positioning method and system based on inverse perspective transformation matrix |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9802539B2 (en) * | 2015-04-01 | 2017-10-31 | Robert Bosch Gmbh | Distance and direction estimation of a target point from a vehicle using monocular video camera |
| KR102485318B1 (en) * | 2016-05-26 | 2023-01-05 | 현대자동차주식회사 | System and Method for Recognizing Cut-in Vehicle on the basis of Monocular Image |
| CN110542895A (en) * | 2018-05-28 | 2019-12-06 | 北京京东尚科信息技术有限公司 | monocular-based freespace distance measurement method |
| CN109343041B (en) * | 2018-09-11 | 2023-02-14 | 昆山星际舟智能科技有限公司 | Monocular distance measuring method for advanced intelligent auxiliary driving |
| CN110031829B (en) * | 2019-04-18 | 2021-07-09 | 北京联合大学 | Target accurate distance measurement method based on monocular vision |
| CN111380503B (en) * | 2020-05-29 | 2020-09-25 | 电子科技大学 | Monocular camera ranging method adopting laser-assisted calibration |
| CN112927306B (en) * | 2021-02-24 | 2024-01-16 | 深圳市优必选科技股份有限公司 | Calibration method and device of shooting device and terminal equipment |
| CN113465573A (en) * | 2021-06-30 | 2021-10-01 | 深圳市优必选科技股份有限公司 | Monocular distance measuring method and device and intelligent device |
-
2021
- 2021-06-30 CN CN202110738325.XA patent/CN113465573A/en active Pending
- 2021-11-18 WO PCT/CN2021/131460 patent/WO2023273108A1/en unknown
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104504680A (en) * | 2014-11-23 | 2015-04-08 | 北京联合大学 | Inverse perspective calibration method used for intelligent vehicle |
| WO2019184885A1 (en) * | 2018-03-30 | 2019-10-03 | 杭州海康威视数字技术股份有限公司 | Method, apparatus and electronic device for calibrating extrinsic parameters of camera |
| CN111521117A (en) * | 2019-08-21 | 2020-08-11 | 长城汽车股份有限公司 | Monocular vision distance measuring method, storage medium and monocular camera |
| CN110599548A (en) * | 2019-09-02 | 2019-12-20 | Oppo广东移动通信有限公司 | Camera calibration method and device, camera and computer readable storage medium |
| CN111047536A (en) * | 2019-12-18 | 2020-04-21 | 深圳市汉森软件有限公司 | CCD image correction method, device, equipment and storage medium |
| CN111474519A (en) * | 2020-04-28 | 2020-07-31 | 广东博智林机器人有限公司 | Positioning method, device, equipment and storage medium |
| CN112665577A (en) * | 2020-12-29 | 2021-04-16 | 北京电子工程总体研究所 | Monocular vision target positioning method and system based on inverse perspective transformation matrix |
| CN112669382A (en) * | 2020-12-30 | 2021-04-16 | 联想未来通信科技(重庆)有限公司 | Image-based distance determination method and device |
Non-Patent Citations (1)
| Title |
|---|
| 本书编委会: "《2017卫星导航定位与北斗系统应用 深化北斗应用》", 测绘出版社, pages: 134 - 135 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023273108A1 (en) * | 2021-06-30 | 2023-01-05 | 深圳市优必选科技股份有限公司 | Monocular distance measurement method and apparatus, and intelligent apparatus |
| CN114049394A (en) * | 2021-11-23 | 2022-02-15 | 智道网联科技(北京)有限公司 | Monocular distance measuring method, device, equipment and storage medium |
| CN116681778A (en) * | 2023-06-06 | 2023-09-01 | 固安信通信号技术股份有限公司 | Distance measurement method based on monocular camera |
| CN116681778B (en) * | 2023-06-06 | 2024-01-09 | 固安信通信号技术股份有限公司 | Distance measurement method based on monocular camera |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023273108A1 (en) | 2023-01-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113465573A (en) | Monocular distance measuring method and device and intelligent device | |
| KR101249791B1 (en) | Image generating method for around view of vehicle | |
| US7773799B2 (en) | Method for automatic stereo measurement of a point of interest in a scene | |
| EP2745504B1 (en) | Image projector, image processing method, computer program and recording medium | |
| CN111263142B (en) | Method, device, equipment and medium for testing optical anti-shake of camera module | |
| CN110809786A (en) | Calibration device, calibration chart, chart pattern generation device, and calibration method | |
| JP2007036482A (en) | Information projection display and program | |
| US20190082173A1 (en) | Apparatus and method for generating a camera model for an imaging system | |
| CN112270719B (en) | Camera calibration method, device and system | |
| KR20180105875A (en) | Camera calibration method using single image and apparatus therefor | |
| CN111681186A (en) | Image processing method and device, electronic equipment and readable storage medium | |
| CN115457147A (en) | Camera calibration method, electronic device and storage medium | |
| CN114387347B (en) | Method, device, electronic equipment and medium for determining external parameter calibration | |
| CN111383264A (en) | Positioning method, positioning device, terminal and computer storage medium | |
| CN113658279B (en) | Camera internal reference and external reference estimation method, device, computer equipment and storage medium | |
| JP2011155412A (en) | Projection system and distortion correction method in the same | |
| KR102023087B1 (en) | Method for camera calibration | |
| CN111553969B (en) | Texture mapping method, medium, terminal and device based on gradient domain | |
| CN110458951B (en) | Modeling data acquisition method and related device for power grid pole tower | |
| JP2002109518A (en) | Three-dimensional shape restoring method and system therefor | |
| KR101996226B1 (en) | Apparatus for measuring three-dimensional position of subject and method thereof | |
| KR100933304B1 (en) | An object information estimator using the single camera, a method thereof, a multimedia device and a computer device including the estimator, and a computer-readable recording medium storing a program for performing the method. | |
| CN115127481A (en) | Stripe projection 3D measuring method, terminal device and computer readable storage medium | |
| CN111179347B (en) | Positioning method, positioning equipment and storage medium based on regional characteristics | |
| CN110232715B (en) | Method, device and system for self calibration of multi-depth camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |