WO2020038118A1 - 车载摄像头的姿态估计方法、装置和系统及电子设备 - Google Patents

车载摄像头的姿态估计方法、装置和系统及电子设备 Download PDF

Info

Publication number
WO2020038118A1
WO2020038118A1 PCT/CN2019/093911 CN2019093911W WO2020038118A1 WO 2020038118 A1 WO2020038118 A1 WO 2020038118A1 CN 2019093911 W CN2019093911 W CN 2019093911W WO 2020038118 A1 WO2020038118 A1 WO 2020038118A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
vehicle
horizon
camera
lane line
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/093911
Other languages
English (en)
French (fr)
Chinese (zh)
Inventor
戴兴
王哲
石建萍
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Sensetime Technology Development Co Ltd
Original Assignee
Beijing Sensetime Technology Development Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Sensetime Technology Development Co Ltd filed Critical Beijing Sensetime Technology Development Co Ltd
Priority to SG11202000641UA priority Critical patent/SG11202000641UA/en
Priority to JP2020510614A priority patent/JP6995188B2/ja
Priority to US16/748,785 priority patent/US11205284B2/en
Publication of WO2020038118A1 publication Critical patent/WO2020038118A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N17/00—Diagnosis, testing or measuring for television systems or their details
    • H04N17/002—Diagnosis, testing or measuring for television systems or their details for television cameras
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R11/00—Arrangements for holding or mounting articles, not otherwise provided for
    • B60R11/04—Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/40—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the details of the power supply or the coupling to vehicle components
    • B60R2300/402—Image calibration
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/804—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for lane monitoring
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30256—Lane; Road marking

Definitions

  • the present application relates to the field of intelligent driving technologies, and in particular, to a method, a device, and a system for estimating the attitude of a vehicle camera, and an electronic device.
  • Intelligent driving systems such as Advanced Driver Assistance System (ADAS) or unmanned driving systems use various sensors installed on vehicles (automobiles, electric vehicles, trains, etc.) during the driving process of the vehicle.
  • vehicles autonomouss, electric vehicles, trains, etc.
  • the surrounding environment is sensed at any time, thereby assisting the driver in controlling the vehicle and alerting the driver to possible dangers, and improving the driver's safety and comfort during driving.
  • the on-board camera is the main sensor, and the data it collects is particularly important.
  • the on-board camera works in a specific attitude, and the data it collects is data in that particular attitude, and the on-camera camera may vibrate during the driving of the vehicle, and the on-camera camera may be disassembled during the maintenance process of the on-board camera.
  • the attitude of the car camera changes.
  • the embodiments of the present application provide a technical solution for attitude estimation of a vehicle camera and its application.
  • an embodiment of the present application provides a pose estimation method for a vehicle camera, including:
  • an embodiment of the present application further provides a posture estimation device for a vehicle camera, including:
  • a lane line detection module configured to perform lane line detection on a road surface based on a video stream of a vehicle traveling road surface collected by a vehicle camera;
  • a horizon information acquisition module configured to acquire horizon information of a road surface of the vehicle according to a detection result of a lane line
  • An attitude information acquisition module is configured to obtain attitude information of the vehicle-mounted camera according to the horizon information.
  • an electronic device including:
  • a memory configured to store program instructions
  • the processor is configured to call and execute the program instructions in the memory, and execute the method steps in any feasible implementation manner of the first aspect.
  • an embodiment of the present application provides a readable storage medium, where the computer program is stored in the readable storage medium, and the computer program is configured to execute the method in any one of the feasible implementation manners in the first aspect. step.
  • an embodiment of the present application provides a pose estimation system for a vehicle-mounted camera, which is applied to a vehicle, and includes a camera installed on the vehicle and a feasible implementation according to any one of the above-mentioned second aspects in communication with the camera.
  • a pose estimation device for an in-vehicle camera in the system is provided.
  • an embodiment of the present application further provides a computer program product including computer instructions.
  • the computer instructions When the computer instructions are run in a processor of a device, the method in any one of the feasible implementation manners of the first aspect is implemented. step.
  • the pose estimation method and device of an on-board camera include: performing lane line detection of a road surface based on a video stream of the vehicle's running road surface collected by the on-board camera; obtaining horizon information of the road surface of the vehicle according to the detection result of the lane line; The horizon information obtains the attitude information of the vehicle camera.
  • the attitude estimation method of the vehicle camera provided in the embodiment of the present application, the position of the vehicle camera does not need to be fixed, and the attitude of the camera can be obtained in real time, thereby improving the accuracy of the attitude estimation of the vehicle camera.
  • FIG. 1 is a schematic flowchart of a pose estimation method of a vehicle camera provided in Embodiment 1 of the present application; FIG.
  • FIG. 2 is an example of an application scenario applicable to the embodiment of the present application
  • FIG. 3 is a first schematic image of an image captured by a vehicle camera in an embodiment of the present application
  • FIG. 4 is a second schematic diagram of an image captured by a vehicle camera in an embodiment of the present application.
  • FIG. 5 is a schematic flowchart of an attitude estimation method of a vehicle camera provided in Embodiment 2 of the present application;
  • FIG. 6 is a schematic flowchart of an attitude estimation method of a vehicle camera provided in Embodiment 3 of the present application.
  • FIG. 7 is a schematic diagram of a principle of pitch angle estimation of a vehicle camera provided in an embodiment of the present application.
  • FIG. 8 is a schematic diagram illustrating a principle of horizontal deflection angle estimation of a vehicle camera provided in an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of an attitude estimation apparatus for a vehicle camera provided in an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a posture information acquisition module according to an embodiment of the present application.
  • FIG. 11 is a physical block diagram of an electronic device according to an embodiment of the present application.
  • FIG. 12 is a schematic structural diagram of a pose estimation system of a vehicle camera provided in an embodiment of the present application.
  • the subject of the pose estimation method of the vehicle camera in the embodiment of the present application may be an electronic device such as a terminal device or a server, where the terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, or a cordless device.
  • UE user equipment
  • PDA Personal Digital Processing
  • handheld devices computing devices
  • on-board cameras other in-vehicle devices other than cameras
  • wearable devices may be implemented by a processor invoking computer-readable instructions stored in a memory.
  • FIG. 1 is a schematic flowchart of a pose estimation method of a vehicle camera provided in Embodiment 1 of the present application.
  • lane line detection is performed on an image captured by an on-board camera, and lane line information in the image is determined according to the lane line detection result. Then, the attitude of the camera is obtained based on the lane line information in each image, and the calculation amount is small and reduced. To the position requirements of the camera.
  • the attitude estimation method of a vehicle camera includes the following steps:
  • S101 Perform lane line detection on the road surface based on a video stream of a vehicle running road surface collected by an on-board camera.
  • FIG. 2 is an example of an application scenario applicable to the embodiment of the present application.
  • an on-vehicle camera provided on the front of the vehicle captures the road surface, and the video stream of the road surface of the vehicle collected by the on-board camera is captured in the video stream Include at least one frame of image.
  • the vehicle-mounted camera is installed at any position on the front windshield of the vehicle in the embodiments of the present application.
  • the road surface in each embodiment of the present application is a structured road.
  • Structured roads generally refer to well-structured highways such as highways and urban trunk roads. Such roads have road markings such as lane lines, the background environment of the road is relatively simple, and the geometric characteristics of the road are also obvious.
  • Unstructured roads generally refer to less-structured roads such as urban non-arterial roads and rural streets. Such roads have no lane lines and / or clear road boundaries, and are affected by shadows and water marks. Areas and off-road areas are difficult to distinguish.
  • the in-vehicle camera attitude after the in-vehicle camera attitude is estimated, it can be used in various application scenarios such as in-vehicle device positioning, navigation, and road scene restoration.
  • the process of determining that the vehicle is traveling on a structured road may specifically include:
  • the lane line detection of the road surface is performed based on the video stream of the vehicle traveling road surface collected by the on-board camera; when an image includes at least two lane lines in the video stream, it is determined that the vehicle is traveling on a structured road.
  • the lane line detection of the road surface is performed on the image obtained by the camera.
  • the vehicle may be determined.
  • Driving on a structured road it may be detected that a certain frame image includes at least two lane lines adjacent to the vehicle, and it is determined that the vehicle is traveling on a structured road.
  • the lane line detection result may include lane line information.
  • the above lane line information may be information of two lane lines of a left and right chord of a vehicle or information of any two lane lines on a road surface.
  • the two lane lines may be either straight lane lines or curved lane lines, which are not limited in the embodiment of the present disclosure.
  • the above lane line information may be represented by a lane line function, and the acquisition process of the lane line information will be described in the following embodiments.
  • the horizon information may be a horizon function in an image captured by a vehicle camera.
  • a feasible method for obtaining horizon information includes:
  • FIG. 3 is a first schematic diagram of an image captured by a vehicle camera in an embodiment of the present application.
  • FIG. 4 is a second schematic diagram of an image captured by a vehicle camera in an embodiment of the present application.
  • the image can be detected by algorithms such as feature extraction to obtain lane lines in the image.
  • the lane lines are generally straight lines as shown in FIG. 3 and curves as shown in FIG. 4.
  • a lane line function in the image can be fitted.
  • acquiring a lane line function corresponding to at least two lane lines in the image specifically includes:
  • image segmentation algorithms can be used to detect lanes in the image, and according to the detection results in the image Marked lane line pixels that belong to the lane line.
  • convolutional neural network algorithms can be used to detect lanes in the image, and according to the detection results in the image Marked lane line pixels that belong to the lane line.
  • lane line functions of all lane lines in the image can be obtained.
  • the lane line function is usually a linear function.
  • the lane lines intersect at the horizon, so the visual intersection of each lane line function falls on the horizon.
  • the coordinates of the visual intersection in the image can be obtained according to the lane line functions.
  • the lane line is a curve
  • the point of the lane line function within the coordinate range of the image pixels is used as the visual intersection of the lane line function.
  • the horizon function can be obtained according to the visual intersections in each frame of the image obtained.
  • the attitude estimation of the vehicle camera further includes:
  • the route information may be a route function.
  • Vanishing point The vanishing point of the on-board camera refers to the vanishing point of the road surface of the route captured by the on-board camera as the perspective of the on-board camera moves. According to the perspective principle, since the vehicle driving route is perpendicular to the horizon and the vanishing point of the on-board camera in the image is located on the vehicle driving route, the route function of the vehicle can be obtained according to the obtained horizon function and image vanishing point.
  • the attitude information of the vehicle-mounted camera includes at least one of the following: a rotation angle, a pitch angle, and a horizontal deflection angle of the vehicle-mounted camera.
  • the attitude of the on-board camera changes, the horizon and the route in the images captured by the on-board camera are different. Therefore, the attitude of the on-board camera may be obtained according to the obtained horizon information and route information.
  • the rotation angle of the vehicle camera can be determined according to the slope information of the horizon. Obtain the horizontal deflection angle of the vehicle camera according to the route information. Obtain the pitch angle of the on-board camera according to the horizon information.
  • the pose estimation method of a vehicle camera includes: detecting a lane line of a road surface based on a video stream of a vehicle traveling road surface collected by the vehicle camera; obtaining horizon information of the vehicle road surface according to a detection result of the lane line; Get the pose information of the car camera.
  • the position of the vehicle camera does not need to be fixed, and the attitude of the camera can be obtained in real time, thereby improving the accuracy of the attitude estimation of the vehicle camera.
  • the solutions involved in the above embodiments can be applied to intelligent driving scenarios, such as in assisted driving or automatic driving scenarios, and the safety of assisted driving or automatic driving can be improved by acquiring the accurate attitude of the on-board camera in real time.
  • the vehicle positioning, navigation, and scene restoration may be further performed according to the posture information of the vehicle camera.
  • FIG. 5 is a schematic flowchart of an attitude estimation method of a vehicle camera provided in Embodiment 2 of the present application.
  • the process of acquiring the horizon information according to the visual intersection of the lane lines is explained in detail.
  • obtaining horizon information based on the visual intersection of lane lines includes:
  • the pixel coordinates of the visual intersection points in each frame image are counted, and the intersection point probability map and the intersection point probability map are obtained according to the pixel coordinates of the visual intersection points in the N frame images.
  • the value of each pixel in N is the number of images where the visual intersection in the N frames of images is located at that pixel. Wherein, N is a positive integer greater than 1.
  • intersection probability map After obtaining the intersection probability map, a density-based clustering algorithm is used to remove outliers, and a visual intersection point belonging to the horizon is determined in the intersection probability map.
  • a horizon function can be constructed according to the coordinates of each visual intersection, and the horizon function is a linear function.
  • this embodiment is based on the perspective principle. Considering that the visual intersections of the lane lines are located on the horizon, the horizon information can be determined according to the visual intersections of the lane lines in each image, which simplifies the way to obtain the horizon information and reduces the vehicle camera. The calculation amount of the pose estimation.
  • the method for obtaining vanishing points may be exemplified as follows: average processing is performed on the obtained visual intersection points belonging to the horizon to obtain vanishing points of the vehicle camera.
  • the horizontal and vertical coordinates of each visual intersection can be averaged to determine the horizontal and vertical coordinates of the vanishing point. Get Vanishing Point.
  • an exemplary manner for obtaining the vanishing point may also be shown in FIG. 6.
  • the process for acquiring the vanishing point includes:
  • the value of each pixel in the probability image indicates the probability that each pixel belongs to the lane line.
  • a probability image of the straight lane lines can be obtained according to statistics.
  • the value of each pixel in the probability image indicates the probability that each pixel belongs to the lane line.
  • the value of a pixel point in the probability image may further indicate the number of times that the pixel point belongs to the lane line in the N frames of images.
  • a pixel point belonging to the lane line is determined in the probability image, and a function of the lane line can be fitted according to the pixel point belonging to the lane line.
  • a visual intersection of a function of each lane line is obtained as a vanishing point.
  • this embodiment is based on the perspective principle. Considering that the visual intersection of the lane lines is the vanishing point, the probability image is obtained based on the statistics of multiple frames of images, and then the lane line function is determined to obtain the vanishing point according to the visual intersection of the lane lines. The method of obtaining vanishing points is simplified, and the calculation amount of the pose estimation of the vehicle camera is reduced.
  • obtaining the elevation angle of the vehicle camera according to the horizon function specifically includes:
  • the pitch angle of the in-vehicle camera is obtained according to the distance from a pixel point in the image mapped to the horizon function and the focal length of the in-vehicle camera based on the main optical axis of the in-vehicle camera.
  • the distance D1 from the pixel point in the image mapped by the main optical axis of the vehicle camera to the horizon function is obtained, and the pitch angle of the vehicle camera is obtained according to arctan (D1 / (f * PM));
  • f is the focal length of the vehicle camera
  • PM is the internal parameter of the image collected by the vehicle camera
  • the unit of the internal parameter is pixel / mm, which indicates the number of pixels that can be imaged per millimeter on the imaging element.
  • FIG. 7 is a schematic diagram of a pitch angle estimation of a vehicle camera provided in an embodiment of the present application.
  • FIG. 7 is a side view of a vehicle traveling on a road surface.
  • BE is the road surface on which the vehicle is traveling
  • BD is perpendicular to the ground
  • AD is parallel to the ground
  • the dotted line where AD is located represents a ground parallel line at the same height as the camera.
  • MA is f, which is the focal length of the vehicle camera.
  • MN is the size of the imaging element in the camera.
  • the pixel point in the image mapped by the main optical axis of the vehicle camera is P
  • PQ is the distance D1 from the pixel point mapped in the image of the main optical axis of the vehicle camera to the horizon function.
  • the unit of D1 is pixel
  • MN PQ / PM.
  • obtaining the horizontal deflection angle of the vehicle camera according to the route information includes:
  • the horizontal deflection angle of the on-vehicle camera is obtained according to the distance from the pixel point mapped to the route function and the focal length of the on-vehicle camera on the main optical axis of the on-vehicle camera.
  • the distance D2 from the pixel point in the image mapped by the main optical axis of the vehicle camera to the route function is obtained, and the horizontal deflection angle of the vehicle camera is obtained according to arctan (D2 / (f * PM)).
  • FIG. 8 is a schematic diagram of a method for estimating a horizontal deflection angle of a vehicle camera provided in an embodiment of the present application.
  • FIG. 8 is a top view of a vehicle traveling on a road surface.
  • ⁇ is the horizontal deflection angle of the on-board camera
  • GAC is the heading of the vehicle
  • tan ⁇ GH / f.
  • GH is the size of the imaging element in the camera
  • GH CD / PM.
  • the point in the image where the main optical axis of the vehicle camera is mapped in the image is point D
  • CD is the distance D2 from the point in the image where the main optical axis of the vehicle camera is mapped in the image.
  • f is the focal length of the vehicle camera.
  • acquiring attitude information of the vehicle camera according to the horizon information includes:
  • the rotation angle of the vehicle camera is determined according to the slope information of the horizon.
  • the horizon function is a linear function in the image.
  • the horizon function is a horizontal straight line.
  • the slope of the horizon function can indicate the rotation angle of the on-board camera.
  • FIG. 9 is a schematic structural diagram of a pose estimation device for a vehicle camera provided in an embodiment of the application. As shown in FIG. 9, the pose estimation device of the vehicle camera includes:
  • the lane line detection module 901 is configured to perform lane line detection on a road surface based on a video stream of a vehicle traveling road surface collected by a vehicle camera;
  • the horizon information acquisition module 902 is configured to acquire horizon information of the road surface of the vehicle according to the detection result of the lane line;
  • An attitude information acquisition module 903 is configured to acquire attitude information of the on-vehicle camera according to the horizon information.
  • the pose estimation method and device of an on-board camera include: performing lane line detection of a road surface based on a video stream of the vehicle's running road surface collected by the on-board camera; obtaining horizon information of the road surface of the vehicle according to the detection result of the lane line; The horizon information obtains the attitude information of the vehicle camera.
  • the attitude estimation method of the vehicle camera provided in the embodiment of the present application, the position of the vehicle camera does not need to be fixed, and the position of the camera is not fixed. The attitude can be acquired in real time, thus improving the accuracy of the attitude estimation of the vehicle camera
  • the road surface is a structured road
  • / or the on-vehicle camera is installed at any position on a front windshield of the vehicle.
  • the road surface in each embodiment of the present application is a structured road.
  • Structured roads generally refer to well-structured highways such as highways and urban trunk roads. Such roads have road markings such as lane lines, the background environment of the road is relatively simple, and the geometric characteristics of the road are also obvious.
  • Unstructured roads generally refer to less-structured roads such as urban non-arterial roads and rural streets. Such roads have no lane lines and / or clear road boundaries, and are affected by shadows and water marks. Areas and off-road areas are difficult to distinguish.
  • the attitude information of the vehicle-mounted camera includes a rotation angle of the vehicle-mounted camera.
  • the attitude information acquisition module 903 may further include:
  • the rotation angle obtaining unit 1001 is configured to determine a rotation angle of the vehicle-mounted camera according to the slope information of the horizon.
  • the attitude information of the vehicle-mounted camera further includes a horizontal deflection angle of the vehicle-mounted camera.
  • the attitude estimation device of the in-vehicle camera further includes: a route information acquisition module 904, configured to acquire route information of the vehicle according to the horizon information.
  • the posture information acquisition module 903 includes:
  • a horizontal deflection angle acquiring unit 1002 is configured to acquire a horizontal deflection angle of the vehicle-mounted camera according to the route information.
  • the horizontal deflection angle obtaining unit 1002 is specifically configured to obtain the horizontal deflection angle of the on-vehicle camera according to the route information and the focal length of the on-vehicle camera.
  • the attitude information of the vehicle camera includes a pitch angle of the vehicle camera.
  • the posture information acquisition module 903 includes:
  • a pitch angle acquiring unit 1003 is configured to acquire a pitch angle of the vehicle camera according to the horizon information and a focal length of the vehicle camera.
  • the horizon information acquisition module 902 includes:
  • the lane line information acquiring unit 9021 is configured to fit a lane line according to a detection result of the lane line to obtain lane line information of at least two lane lines;
  • the intersection acquisition unit 9022 is configured to obtain a visual intersection of the lane lines according to the lane line information of the at least two lane lines;
  • the horizon information acquisition unit 9023 is configured to acquire horizon information according to a visual intersection of the lane lines.
  • the lane line information obtaining unit 9021 is specifically configured to obtain lane line pixel points belonging to the lane line according to a detection result of the lane line;
  • the lane line is fitted according to the lane line pixel points to obtain lane line information of at least two lane lines.
  • the horizon information obtaining unit 9023 is specifically configured to:
  • the lane lines intersect at the horizon, so the visual intersection of each lane line function falls on the horizon.
  • the coordinates of the visual intersection in the image can be obtained according to the lane line functions.
  • the lane line is a curve
  • the point of the lane line function within the coordinate range of the image pixels is used as the visual intersection of the lane line function.
  • the attitude estimation device of the in-vehicle camera further includes: a vanishing point acquisition module 905, configured to, according to at least two lanes in each frame of the multi-frame images included in the video stream, Visual intersections of lines to obtain visual intersections belonging to the horizon;
  • the attitude estimation device of the in-vehicle camera further includes: a vanishing point acquisition module 905, configured to acquire a probability image of a lane line according to at least two frames of the video stream, the probability The value of each pixel in the image indicates the probability that each pixel belongs to the lane line;
  • the attitude estimation device of the vehicle camera further includes:
  • a camera calibration module 906 is configured to calibrate the on-vehicle camera according to the attitude information.
  • the attitude estimation device of the vehicle camera further includes:
  • the vehicle positioning module 907 is configured to determine positioning information of the vehicle according to the posture information.
  • FIG. 11 is a physical block diagram of an electronic device according to an embodiment of the present application. As shown in FIG. 11, the electronic device includes:
  • the processor 1102 is configured to call and execute a program instruction in the memory, and execute the method steps described in the foregoing method embodiments.
  • FIG. 12 is a schematic structural diagram of an attitude estimation system of a vehicle camera provided in an embodiment of the present application. The system is applied to a vehicle. As shown in FIG. ⁇ pose estimation device 1202.
  • An embodiment of the present application further provides a readable storage medium.
  • a computer program is stored in the readable storage medium, and the computer program is configured to execute the method steps described in the foregoing method embodiments.
  • An embodiment of the present application also provides a computer program product, which includes computer instructions, and when the computer instructions are executed in a processor of a device, the method steps described in the foregoing method embodiments are implemented.
  • a person of ordinary skill in the art may understand that all or part of the steps of implementing the foregoing method embodiments may be implemented by a program instructing related hardware.
  • the aforementioned program may be stored in a computer-readable storage medium.
  • the steps including the foregoing method embodiments are executed; and the foregoing storage medium includes: various media that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Mechanical Engineering (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)
  • Studio Devices (AREA)
PCT/CN2019/093911 2018-08-24 2019-06-28 车载摄像头的姿态估计方法、装置和系统及电子设备 Ceased WO2020038118A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
SG11202000641UA SG11202000641UA (en) 2018-08-24 2019-06-28 Vehicle-mounted camera pose estimation method, apparatus, and system, and electronic device
JP2020510614A JP6995188B2 (ja) 2018-08-24 2019-06-28 車載カメラの姿勢推定方法、装置およびシステムならびに電子機器
US16/748,785 US11205284B2 (en) 2018-08-24 2020-01-21 Vehicle-mounted camera pose estimation method, apparatus, and system, and electronic device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810973920.XA CN110858405A (zh) 2018-08-24 2018-08-24 车载摄像头的姿态估计方法、装置和系统及电子设备
CN201810973920.X 2018-08-24

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/748,785 Continuation US11205284B2 (en) 2018-08-24 2020-01-21 Vehicle-mounted camera pose estimation method, apparatus, and system, and electronic device

Publications (1)

Publication Number Publication Date
WO2020038118A1 true WO2020038118A1 (zh) 2020-02-27

Family

ID=69592397

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/093911 Ceased WO2020038118A1 (zh) 2018-08-24 2019-06-28 车载摄像头的姿态估计方法、装置和系统及电子设备

Country Status (5)

Country Link
US (1) US11205284B2 (https=)
JP (1) JP6995188B2 (https=)
CN (1) CN110858405A (https=)
SG (1) SG11202000641UA (https=)
WO (1) WO2020038118A1 (https=)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3893150A1 (en) * 2020-04-09 2021-10-13 Tusimple, Inc. Camera pose estimation techniques
US11573095B2 (en) 2017-08-22 2023-02-07 Tusimple, Inc. Verification module system and method for motion-based lane detection with multiple sensors
US11740093B2 (en) 2018-02-14 2023-08-29 Tusimple, Inc. Lane marking localization and fusion
US11852498B2 (en) 2018-02-14 2023-12-26 Tusimple, Inc. Lane marking localization
US12270661B2 (en) 2018-02-14 2025-04-08 Tusimple, Inc. Lane marking localization and fusion

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12579684B2 (en) * 2020-08-18 2026-03-17 Foresight Automotive Ltd. System and method for pose estimation of sensors using motion and angular shift
CN113490967A (zh) * 2020-09-22 2021-10-08 深圳市锐明技术股份有限公司 一种相机的标定方法、标定装置及电子设备
CN112384952A (zh) * 2020-09-22 2021-02-19 深圳市锐明技术股份有限公司 一种相机的标定方法、标定装置及电子设备
CN112597965B (zh) * 2021-01-05 2024-10-18 株洲中车时代电气股份有限公司 一种驾驶行为识别方法、装置和计算机可读存储介质
CN112907678B (zh) * 2021-01-25 2022-05-13 深圳佑驾创新科技有限公司 车载相机外参姿态动态估计方法、装置、计算机设备
JP7776100B2 (ja) 2021-04-16 2025-11-26 株式会社コムテック 再生制御システム及びコンピュータプログラム
CN115564793B (zh) * 2021-06-30 2024-12-27 惠州市德赛西威智能交通技术研究院有限公司 一种车载前向摄像头的位姿实时计算方法
CN113744330B (zh) * 2021-08-16 2025-04-29 苏州挚途科技有限公司 车载摄像头的俯仰角确定方法、装置及电子设备
US11875580B2 (en) * 2021-10-04 2024-01-16 Motive Technologies, Inc. Camera initialization for lane detection and distance estimation using single-view geometry
CN114494439B (zh) * 2022-01-25 2023-08-15 襄阳达安汽车检测中心有限公司 Hil仿真测试中摄像头位姿校准方法、装置、设备及介质
KR20240103749A (ko) * 2022-12-27 2024-07-04 현대자동차주식회사 차량 및 그 제어방법
US12405116B2 (en) * 2023-03-03 2025-09-02 Telenav, Inc. Navigation system with automatic optical calibration mechanism and method of operation thereof
CN116708989A (zh) * 2023-05-12 2023-09-05 北京鉴智科技有限公司 一种车载摄像头的控制方法、装置、车辆和存储介质
CN117252880B (zh) * 2023-11-17 2024-01-16 宝德华南(深圳)热能系统有限公司 一种新能源汽车的adas综合评测系统
KR20250110561A (ko) * 2024-01-12 2025-07-21 현대자동차주식회사 멀티 카메라 자세 추정 방법 및 그 장치
CN121414856A (zh) * 2025-12-24 2026-01-27 浙江吉利控股集团有限公司 车载相机外参标定方法、电子设备及程序产品

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106875448A (zh) * 2017-02-16 2017-06-20 武汉极目智能技术有限公司 一种车载单目摄像头外部参数自标定方法
CN107133985A (zh) * 2017-04-20 2017-09-05 常州智行科技有限公司 一种基于车道线消逝点的车载摄像机自动标定方法
CN107730551A (zh) * 2017-01-25 2018-02-23 问众智能信息科技（北京）有限公司 车载相机姿态自动估计的方法和装置
CN108367710A (zh) * 2016-06-15 2018-08-03 Jvc 建伍株式会社 姿势变化判定装置、俯瞰影像生成装置、俯瞰影像生成系统、姿势变化判定方法以及程序
CN108450058A (zh) * 2015-12-28 2018-08-24 英特尔公司 实时自动车载相机校准
CN108898638A (zh) * 2018-06-27 2018-11-27 江苏大学 一种车载摄像头在线自动标定方法

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4670528B2 (ja) * 2005-07-29 2011-04-13 日産自動車株式会社 撮像装置のずれ検出方法、撮像装置のずれ補正方法及び撮像装置
US20110115912A1 (en) * 2007-08-31 2011-05-19 Valeo Schalter Und Sensoren Gmbh Method and system for online calibration of a video system
US8487991B2 (en) * 2008-04-24 2013-07-16 GM Global Technology Operations LLC Clear path detection using a vanishing point
JP5065449B2 (ja) * 2009-06-16 2012-10-31 株式会社日本自動車部品総合研究所 消失点推定装置およびプログラム
US9129161B2 (en) * 2013-05-31 2015-09-08 Toyota Jidosha Kabushiki Kaisha Computationally efficient scene classification
JP5906272B2 (ja) * 2014-03-28 2016-04-20 富士重工業株式会社 車両用ステレオ画像処理装置
GB201410612D0 (en) * 2014-06-13 2014-07-30 Tomtom Int Bv Methods and systems for generating route data
CN106204574B (zh) 2016-07-07 2018-12-21 兰州理工大学 基于目标平面运动特征的相机位姿自标定方法
KR102538592B1 (ko) * 2016-08-31 2023-05-31 현대모비스 주식회사 탑-뷰 영상 생성 장치 및 그 방법
CN106679633B (zh) * 2016-12-07 2019-06-04 东华大学 一种车载测距系统及方法
CN107145825A (zh) * 2017-03-31 2017-09-08 纵目科技（上海）股份有限公司 地平面拟合、摄像头标定方法及系统、车载终端
CN108229386B (zh) * 2017-12-29 2021-12-14 百度在线网络技术（北京）有限公司 用于检测车道线的方法、装置和介质
CN108875603B (zh) * 2018-05-31 2021-06-04 上海商汤智能科技有限公司 基于车道线的智能驾驶控制方法和装置、电子设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108450058A (zh) * 2015-12-28 2018-08-24 英特尔公司 实时自动车载相机校准
CN108367710A (zh) * 2016-06-15 2018-08-03 Jvc 建伍株式会社 姿势变化判定装置、俯瞰影像生成装置、俯瞰影像生成系统、姿势变化判定方法以及程序
CN107730551A (zh) * 2017-01-25 2018-02-23 问众智能信息科技（北京）有限公司 车载相机姿态自动估计的方法和装置
CN106875448A (zh) * 2017-02-16 2017-06-20 武汉极目智能技术有限公司 一种车载单目摄像头外部参数自标定方法
CN107133985A (zh) * 2017-04-20 2017-09-05 常州智行科技有限公司 一种基于车道线消逝点的车载摄像机自动标定方法
CN108898638A (zh) * 2018-06-27 2018-11-27 江苏大学 一种车载摄像头在线自动标定方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
MA, LIN ET AL.: "EJ (Dynamic Approach of Camera Auto-Calibration for Vision System on Autonomous Vehicle", JOURNAL OFXI'AN JIAOTONG UNIVERSITY, vol. 39, no. 10, 31 October 2005 (2005-10-31) *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11573095B2 (en) 2017-08-22 2023-02-07 Tusimple, Inc. Verification module system and method for motion-based lane detection with multiple sensors
US11874130B2 (en) 2017-08-22 2024-01-16 Tusimple, Inc. Verification module system and method for motion-based lane detection with multiple sensors
US12276516B2 (en) 2017-08-22 2025-04-15 Tusimple, Inc. Verification module system and method for motion-based lane detection with multiple sensors
US11740093B2 (en) 2018-02-14 2023-08-29 Tusimple, Inc. Lane marking localization and fusion
US11852498B2 (en) 2018-02-14 2023-12-26 Tusimple, Inc. Lane marking localization
US12270661B2 (en) 2018-02-14 2025-04-08 Tusimple, Inc. Lane marking localization and fusion
US12352592B2 (en) 2018-02-14 2025-07-08 Tusimple, Inc. Lane marking localization
EP3893150A1 (en) * 2020-04-09 2021-10-13 Tusimple, Inc. Camera pose estimation techniques
US11810322B2 (en) 2020-04-09 2023-11-07 Tusimple, Inc. Camera pose estimation techniques

Also Published As

Publication number Publication date
US20200160561A1 (en) 2020-05-21
US11205284B2 (en) 2021-12-21
JP6995188B2 (ja) 2022-01-14
SG11202000641UA (en) 2020-03-30
JP2020533667A (ja) 2020-11-19
CN110858405A (zh) 2020-03-03

Similar Documents

Publication Publication Date Title
WO2020038118A1 (zh) 车载摄像头的姿态估计方法、装置和系统及电子设备
CN110567469B (zh) 视觉定位方法、装置、电子设备及系统
CN104854637B (zh) 移动物体位置姿态角推定装置及移动物体位置姿态角推定方法
CN107636679B (zh) 一种障碍物检测方法及装置
KR101143176B1 (ko) 조감도를 이용한 주차구획 인식 방법, 장치 및 그를 이용한주차 보조 시스템
CN112352260B (zh) 车道估计装置、方法以及计算机可读存储介质
JP6626410B2 (ja) 自車位置特定装置、自車位置特定方法
CN104309606B (zh) 基于360度全景的车道偏离预警方法
WO2020103427A1 (zh) 对象检测方法、相关设备及计算机存储介质
CN110942038B (zh) 基于视觉的交通场景识别方法、装置、介质及电子设备
CN106740841A (zh) 基于动态控制的车道线检测方法、装置及车载设备
WO2017189121A1 (en) Method and device for capturing image of traffic sign
CN111213153A (zh) 目标物体运动状态检测方法、设备及存储介质
CN110378836B (zh) 获取对象的3d信息的方法、系统和设备
CN110036429A (zh) 驾驶支援系统及驾驶支援装置
CN114898325B (zh) 一种车辆危险变道检测方法、装置及电子设备
CN113345087B (zh) 基于单目视觉的地表模型拟合方法和装置
JP2003178309A (ja) 移動量推定装置
CN113903103B (zh) 局部图像生成装置、局部图像生成方法以及存储介质
CN111316337A (zh) 车载成像装置的安装参数的确定与驾驶控制方法及设备
CN111738033A (zh) 基于平面分割的车辆行驶信息确定方法及装置、车载终端
CN113902047B (zh) 图像元素匹配方法、装置、设备以及存储介质
US20200193184A1 (en) Image processing device and image processing method
JP6622664B2 (ja) 自車位置特定装置、及び自車位置特定方法
CN112183413B (zh) 一种车位检测方法、装置、存储介质及车辆

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2020510614

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19853133

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 28.05.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19853133

Country of ref document: EP

Kind code of ref document: A1