WO2019047641A1 - 车载摄像头的姿态误差估计方法和装置 - Google Patents

车载摄像头的姿态误差估计方法和装置 Download PDF

Info

Publication number
WO2019047641A1
WO2019047641A1 PCT/CN2018/098621 CN2018098621W WO2019047641A1 WO 2019047641 A1 WO2019047641 A1 WO 2019047641A1 CN 2018098621 W CN2018098621 W CN 2018098621W WO 2019047641 A1 WO2019047641 A1 WO 2019047641A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
preset
relative pose
posture
vehicle camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/098621
Other languages
English (en)
French (fr)
Inventor
郑超
郁浩
唐坤
闫泳杉
张云飞
姜雨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Baidu Online Network Technology Beijing Co Ltd
Original Assignee
Baidu Online Network Technology Beijing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Baidu Online Network Technology Beijing Co Ltd filed Critical Baidu Online Network Technology Beijing Co Ltd
Publication of WO2019047641A1 publication Critical patent/WO2019047641A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Definitions

  • the present invention relates to the field of in-vehicle device technologies, and in particular to the field of image processing technologies for in-vehicle cameras, and more particularly to an attitude error estimation method and apparatus for an in-vehicle camera.
  • the in-vehicle camera is the main sensor, and the data it collects is especially important for automatic driving decisions.
  • the on-board camera works in a specific posture, and the data collected is the data in the specific posture. Since the vehicle camera may vibrate during the running of the vehicle, the process of repairing the vehicle camera may disassemble the vehicle camera, resulting in a change in the attitude of the vehicle camera. The collected data may be deviated, so it is necessary to correct the attitude error of the vehicle camera.
  • An existing camera attitude error estimation method is to calculate the attitude parameter of the camera by imaging an auxiliary marker (for example, a calibration plate including a specific marker point, etc.), or iterating by using an ICP (Iterative Closet Point) The way to estimate the attitude error.
  • the former has a large limitation on the conditions of use. The latter usually only finds the local optimum value, and cannot obtain the global optimal value. The accuracy of the attitude estimation of the vehicle camera needs to be improved.
  • the embodiments of the present application provide an attitude error estimation method and apparatus for an in-vehicle camera.
  • an embodiment of the present application provides a method for estimating an attitude error of an in-vehicle camera, comprising: acquiring a first image of a preset scene acquired by an in-vehicle camera in a preset standard posture; performing an attitude error estimation step, and estimating an attitude error
  • the method includes: obtaining a second image of the preset scene acquired by the on-vehicle camera in the current posture; comparing the first image and the second image by using the relative pose estimation model to obtain a relative posture of the on-vehicle camera and the preset standard posture Pose information, wherein the relative pose estimation model is generated based on deep learning network training.
  • the method further includes the step of training the relative pose estimation model based on the deep learning network, comprising: acquiring a third image of the preset scene acquired by the onboard camera in the plurality of preset test poses, wherein each pre- Setting the relative pose information of the test pose and the preset standard pose has been marked; constructing sample data based on the relative pose information of the first image, the third image, and the marked preset test pose and the preset standard pose; based on the sample data pair
  • the constructed deep learning network is trained to obtain a relative pose estimation model.
  • the attitude error estimating step before the comparing the first image and the second image with the relative pose estimation model, further comprises: extracting feature points of the first image and the second image; determining the first image and the first Whether the two images contain feature points corresponding to the same spatial point; comparing the first image and the second image by using the relative pose estimation model includes: in response to determining that the first image and the second image contain features corresponding to the same spatial point Point, using the relative pose estimation model to align the first image with the second image.
  • the method further includes: determining whether the relative pose information of the current posture of the in-vehicle camera and the preset standard posture satisfies a preset condition; and determining the relative position of the current posture of the in-vehicle camera and the preset standard posture in response to determining The posture information does not satisfy the preset condition, and based on the relative posture information of the current posture of the vehicle camera and the preset standard posture, after correcting the current posture of the vehicle-mounted camera, the attitude error estimating step is performed.
  • the relative pose information includes a pitch angle and a yaw angle of the current pose relative to the preset standard pose.
  • the embodiment of the present application provides an attitude error estimating apparatus for an in-vehicle camera, comprising: an acquiring unit configured to acquire a first image of a preset scene acquired by an in-vehicle camera in a preset standard posture; and an estimating unit, Configuring to perform an attitude error estimation step, the attitude error estimating step includes: acquiring a second image of the preset scene acquired by the on-vehicle camera in the current posture; and comparing the first image and the second image by using the relative pose estimation model The relative pose information of the current posture of the in-vehicle camera and the preset standard posture, wherein the relative pose estimation model is generated based on the deep learning network training.
  • the apparatus further includes a model training unit configured to train the relative pose estimation model based on the deep learning network; the model training unit is specifically configured to: acquire a pre-acquisition acquired by the on-board camera in a plurality of preset test poses a third image of the scene, wherein the relative pose information of each preset test pose and the preset standard pose is marked; based on the relative relationship between the first image, the third image, and the marked preset test pose and the preset standard pose The pose information constructs sample data; the constructed deep learning network is trained based on the sample data to obtain a relative pose estimation model.
  • the model training unit is specifically configured to: acquire a pre-acquisition acquired by the on-board camera in a plurality of preset test poses a third image of the scene, wherein the relative pose information of each preset test pose and the preset standard pose is marked; based on the relative relationship between the first image, the third image, and the marked preset test pose and the preset standard pose
  • the pose information constructs sample data; the constructed deep learning network is trained based
  • the attitude error estimating step before the comparing the first image and the second image with the relative pose estimation model, further comprises: extracting feature points of the first image and the second image; determining the first image and the first Whether the two images contain feature points corresponding to the same spatial point; comparing the first image and the second image by using the relative pose estimation model includes: in response to determining that the first image and the second image contain features corresponding to the same spatial point Point, using the relative pose estimation model to align the first image with the second image.
  • the apparatus further includes: a determining unit configured to determine whether the relative pose information of the current posture of the in-vehicle camera and the preset standard posture satisfies a preset condition; and the correcting unit configured to determine the vehicle in response to determining The relative pose information of the current posture of the camera and the preset standard posture does not satisfy the preset condition, and the posture error is performed after correcting the current posture of the vehicle camera based on the relative posture information of the current posture of the vehicle camera and the preset standard posture. Estimation step.
  • the relative pose information includes a pitch angle and a yaw angle of the current pose relative to the preset standard pose.
  • an embodiment of the present application provides an apparatus, including: one or more processors; a storage device, configured to store one or more programs, when one or more programs are executed by one or more processors, One or more processors are caused to implement the attitude error estimation method of the above-described on-board camera.
  • the method and device for estimating the attitude error of the in-vehicle camera obtains the first image of the preset scene acquired by the in-vehicle camera in the preset standard posture, and then performs the attitude error estimating step, where the posture error estimating step includes: Obtaining a second image of the preset scene acquired by the onboard camera in the current posture; comparing the first image and the second image by using the relative pose estimation model to obtain the relative pose information of the current posture of the onboard camera and the preset standard posture Among them, the relative pose estimation model is based on the deep learning network training generation, which realizes the automation and high-precision estimation of the vehicle camera error.
  • FIG. 1 is an exemplary system architecture diagram to which the present application can be applied;
  • FIG. 2 is a flow chart of one embodiment of an attitude error estimation method of an in-vehicle camera according to the present application
  • FIG. 3 is a flow chart of another embodiment of an attitude error estimation method of an in-vehicle camera according to the present application.
  • FIG. 4 is a flow chart of still another embodiment of an attitude error estimating method of an in-vehicle camera according to the present application.
  • FIG. 5 is a schematic structural diagram of an embodiment of an attitude error estimating apparatus of an in-vehicle camera of the present application.
  • FIG. 6 is a schematic structural diagram of a computer system suitable for implementing a terminal device or a server of an embodiment of the present application.
  • FIG. 1 shows an exemplary system architecture 100 of an embodiment of an attitude error estimation method of an in-vehicle camera of the present application or an attitude error estimating apparatus of an in-vehicle camera.
  • the system architecture 100 may include an unmanned vehicle 101, an in-vehicle camera 102 and an in-vehicle control unit 103 mounted on the unmanned vehicle 101, and a server 104.
  • the in-vehicle control unit 103 can be connected to the server 104 over a network, which can include various types of connections, such as wired, wireless communication links, fiber optic cables, and the like.
  • the in-vehicle camera 102 can collect image data of an unmanned vehicle travel path or a docking environment.
  • the in-vehicle control unit 103 may be an ECU (Electronic Control Unit) for analyzing and controlling the operating state of each component of the vehicle.
  • the in-vehicle control unit 103 can acquire image data collected by the in-vehicle camera 102, respond according to the image data, and can also control storage and transmission of the image data.
  • the server 104 can establish a connection with the in-vehicle control unit 103 via a network, and the in-vehicle control unit 103 can transmit image data and other sensor data to the server 104.
  • the server 104 may perform processing such as feature extraction, matching, classification, and the like on the image data and other sensor data, and feed back the processing result to the in-vehicle control unit 103.
  • the in-vehicle control unit 103 can respond according to the received processing result.
  • the in-vehicle camera 102 In order to accurately match the data in the database (e.g., to accurately match the data in the map data and the road image database), the in-vehicle camera 102 needs to operate in a standard pose, i.e., the in-vehicle camera 102 needs to be fixed at a particular relative position to the driverless vehicle 101.
  • the location of the relationship ( Figure 1 position A).
  • the attitude error of the in-vehicle camera 102 can be estimated using the image acquired by the in-vehicle camera 102, and the posture of the in-vehicle camera 102 can be adjusted based on the estimation result.
  • attitude error estimation method of the in-vehicle camera may be performed by the in-vehicle control unit 103 or the server 104. Accordingly, the attitude error estimation device of the in-vehicle camera may be disposed on the in-vehicle control unit 103 or the server 104. in.
  • FIG. 1 the number of unmanned vehicles, on-board cameras, on-board control units, and servers in FIG. 1 is merely illustrative. Depending on the needs of the implementation, there can be any number of unmanned vehicles, on-board cameras, on-board control units and servers.
  • attitude error estimation method of the vehicle camera includes the following steps:
  • Step 201 Acquire a first image of a preset scene collected by the onboard camera in a preset standard posture.
  • the electronic device (for example, the server or the in-vehicle control unit shown in FIG. 1) on which the attitude error estimating method of the in-vehicle camera operates may be connected to the in-vehicle camera through a wired connection or a wireless connection.
  • the electronic device can obtain a first image of the preset scene captured by the on-board camera in a preset standard posture by connecting with the on-board camera.
  • the electronic device can be connected to a storage device for storing image data collected by the in-vehicle camera, and the first image is acquired from the storage device.
  • the preset standard posture may be a standard working posture of a predefined in-vehicle camera, and the image data acquired under the standard working posture may be used as a reference image in a reference image database of the unmanned vehicle.
  • the image captured by the in-vehicle camera in the preset standard attitude can be used to construct a reference image database of the unmanned vehicle.
  • the preset standard attitude can be expressed by the relative position of the vehicle camera relative to the ground and/or a reference point on the vehicle in the standard working posture.
  • the preset standard posture of the vehicle camera can be 1.5 meters perpendicular to the ground, and the optical axis Parallel to the ground and in line with the direction of the front.
  • the preset scene is a preset scene, and may be a scene including preset points, for example, may include a calibration board; or may be a natural scene, such as a road scene.
  • the vehicle camera can acquire an image of the preset scene as the first image in a preset standard posture, and store the first image to a preset storage device.
  • the electronic device on which the attitude error estimating method of the in-vehicle camera operates may extract the first image from the storage device.
  • Step 202 performing an attitude error estimation step.
  • step 202 may specifically include step 2021 and step 2022.
  • step 2021 a second image of the preset scene acquired by the in-vehicle camera in the current posture is acquired.
  • the electronic device may acquire a second image acquired by the on-board camera in the current posture by using the connection with the in-vehicle camera. That is, the electronic device can acquire an image acquired by the same in-vehicle camera in the preset standard posture and in the current posture for the same preset scene.
  • the preset scene may be set as a scene containing the marker object
  • the vehicle camera may be controlled to collect an image of the scene including the marker object as a first image and stored in a standard posture, and control the vehicle camera in the current posture.
  • An image of the scene containing the marker object is acquired as the second image.
  • step 2022 the first pose and the second image are compared using the relative pose estimation model to obtain relative pose information of the current pose of the onboard camera and the preset standard pose.
  • the relative pose estimation model is generated based on deep learning network training.
  • the relative pose estimation model is used to estimate relative pose information between different cameras that capture different images of the input.
  • the input of the relative pose estimation model may be two images, and the output may be relative pose information of the camera that acquires two images.
  • the relative pose information may include a translation parameter and a rotation angle parameter.
  • the relative pose estimation model described above is generated based on deep learning network training, and may be a network model including an input layer, multiple hidden layers, and an output layer.
  • the relative pose estimation model may be a CNN (Convolutional Neural Network) model, an RNN (Recurrent Neural Network) model, or the like.
  • the training data may include a sample set and a test set.
  • the training data may be an existing data that is verified to be accurate for the attitude error estimation result, and the training data may be acquired through the network or in the field.
  • the training data includes a plurality of pairs of input data and output data, and the input data may be images acquired by the in-vehicle camera in different poses, and the output data is corresponding relative pose information.
  • the first image and the second image are images of the same preset scene captured by the same vehicle camera in different postures
  • the first image and the second image include two-dimensional image information of the same spatial point
  • the estimation model may compare the first image and the second image, and use the deep learning network model to analyze the association between the coordinates of the same spatial point in the different image coordinate systems in the first image and the second image through the multi-layer network
  • the structure extracts and classifies the features, and then obtains the conversion parameters between different image coordinate systems, that is, the attitude error of the on-board camera can be obtained.
  • the relative pose estimation model based on deep learning network training, the global optimal value of the pose error can be obtained, which improves the accuracy of pose error estimation.
  • the attitude error of the in-vehicle camera is the difference between the current attitude of the in-vehicle camera and the preset standard attitude, that is, the relative pose information between the attitude of the in-vehicle camera when acquiring the second image and the posture when the first image is acquired. .
  • the relative pose information may include an attitude angle of the current posture relative to the preset standard posture, where the attitude angle may include a pitch angle and a yaw angle.
  • the pitch angle may be a deflection angle of the current attitude of the vehicle camera relative to the ground direction relative to the preset standard attitude
  • the yaw angle may be the current attitude of the vehicle camera in a direction parallel to the ground relative to the preset standard attitude. Deflection angle.
  • the relative pose information may further include a translation parameter of the current posture relative to the preset standard posture, where the translation parameter may indicate a translation amount of the current posture relative to the preset standard posture in a plane parallel to the ground and The amount of translation in a plane perpendicular to the ground.
  • the attitude error estimation method of the in-vehicle camera of the above embodiment of the present application obtains the first image of the preset scene acquired by the onboard camera in the preset standard posture, and then performs an attitude error estimation step, the posture error estimation step includes: acquiring the vehicle a second image of the preset scene captured by the camera in the current posture; comparing the first image and the second image by using the relative pose estimation model to obtain relative pose information of the current posture of the vehicle camera and the preset standard posture, wherein
  • the relative pose estimation model is based on the deep learning network training generation, and the global optimal value of the attitude error estimation can be obtained, which realizes the automatic and high-precision estimation of the attitude error of the vehicle camera.
  • the pose error estimation step 202 may further include: extracting features of the first image and the second image Point; determining whether the first image and the second image contain feature points corresponding to the same spatial point.
  • the electronic device may extract a feature point from the first image and the second image by using a feature point extraction algorithm such as a Scale-invariant feature transform (SITF), and then feature points of the first image. Matching with the feature points of the second image determines whether the first image and the second image contain feature points corresponding to the same spatial point.
  • a feature point extraction algorithm such as a Scale-invariant feature transform (SITF)
  • the above step 2022 may include: in response to determining that the first image and the second image correspond to feature points of the same spatial point, the relative pose estimation model is used to compare the first image and the second image.
  • the first image and the second image include feature points corresponding to the same spatial point, it may be determined that the images of the preset scene in the first image and the second image have more obvious features, and the image information is rich, and may be used for estimation.
  • the attitude error of the car camera may include: in response to determining that the first image and the second image correspond to feature points of the same spatial point, the relative pose estimation model is used to compare the first image and the second image.
  • the feature points corresponding to the same spatial point are not included in the first image and the second image, it may be determined that the image information of the first image and the second image is too small, and is not suitable for performing an attitude error on the in-vehicle camera. Estimate. At this time, the first image and the second image of the other preset scenes collected by the in-vehicle camera in the preset standard posture and the current posture respectively can be obtained for feature point extraction and matching, so that the feature information such as the texture can be selected. The first image and the second image of the preset scene, thereby improving the accuracy of the on-board camera error estimation.
  • FIG. 3 shows a flow chart of another embodiment of an attitude error estimation method for an in-vehicle camera according to the present application.
  • the flow 300 of the attitude error estimation method of the in-vehicle camera of the embodiment may include the following steps:
  • Step 301 Acquire a first image of a preset scene collected by the onboard camera in a preset standard posture.
  • the electronic device (such as the vehicle control unit or the server shown in FIG. 1) on which the attitude error estimation method of the in-vehicle camera operates may be connected to the in-vehicle camera by wire or wirelessly, and the in-vehicle camera is preset.
  • the electronic device may also be connected to a storage device (for example, a vehicle black box) for storing an image captured by the in-vehicle camera, and obtain a first image of the preset scene acquired by the in-vehicle camera in a preset standard posture from the storage device.
  • a storage device for example, a vehicle black box
  • the preset standard posture may be a standard working posture of a preset in-vehicle camera
  • the preset scene is a preset scene, and may be a scene including a preset marker point, for example, may be a scene including a calibration board;
  • Step 302 training a relative pose estimation model based on the deep learning network.
  • the phase pose estimation model can be trained based on the deep learning network by means of supervised learning.
  • Step 302 may specifically include step 3021, step 3022, and step 3023.
  • step 3021 a third image of the preset scene acquired by the onboard camera in a plurality of preset test poses is acquired.
  • the relative pose information of each of the preset test poses and the preset standard poses is marked. That is to say, the relative pose information of each preset test pose and the preset standard pose is known. Specifically, a plurality of sets of relative pose information may be defined, and the pose of the in-vehicle camera in the preset standard posture is adjusted according to the defined relative pose information of each set, and the posture of the in-vehicle camera after the adjustment is the preset test pose. Then, an image of the preset scene is acquired in a preset test posture, and a third image corresponding to each set of relative pose information is obtained.
  • the relative pose information may include a pose angle and a translation parameter of three coordinate axes in the onboard camera coordinate system along the preset standard pose.
  • sample data is constructed based on the first image, the third image, and the relative pose posture information of the marked preset test pose and the preset standard pose.
  • sample data for training a relative pose estimation model can be constructed.
  • Each sample data includes input data and output data of the model.
  • the input data is a first image and a third image
  • the output data is relative posture information of a posture of the in-vehicle camera that captures the third image and a preset standard posture.
  • the first image and the third image are respectively obtained in step 301 and step 3021.
  • the relative pose information of the posture of the in-vehicle camera of the third image and the preset standard posture is marked information, for example, can be artificially defined. Information.
  • the relative pose information of the preset test pose of the on-vehicle camera and the preset standard pose, and the third image acquired by the preset test pose and the first image acquired under the preset standard pose may be used as the first image
  • the vehicle camera can acquire a corresponding third image in a plurality of preset test postures, and multiple sample data corresponding to the plurality of preset test gestures can be obtained.
  • multiple sample data sets can be generated to generate a sample data set.
  • step 3023 the constructed deep learning network is trained based on the sample data to obtain a relative pose estimation model.
  • the structure of the deep learning network can be constructed, and the deep learning network is trained by using the sample data to obtain a relative pose estimation model.
  • the relative pose model may be a model that calculates a relative attitude error between the poses of the cameras that acquire the two images based on the two images input.
  • a deep learning network can be constructed based on existing neural network models such as CNN and RNN. The sample data is input into the constructed deep learning network for training, and the model structure of the deep learning network is optimized to obtain a relative pose estimation model.
  • some sample data may be randomly extracted from the sample data set to generate a training set, and other sample data is used as a test set.
  • the training set is used to train the deep learning network and test the performance of the deep learning network based on the test set. If the test concludes that the performance of the deep learning network does not reach the expected index, the number of sample data in the training set can be increased, the training can be continued, and the deep learning network can be continuously adjusted through the training set and the test set to obtain the final relative pose estimation model. .
  • step 303 an error estimation step is performed.
  • the error estimation step includes step 3031 and step 3032.
  • step 3031 a second image of the preset scene acquired by the in-vehicle camera in the current posture is acquired.
  • the electronic device can acquire a second image acquired by the on-board camera in the current posture for the preset scene.
  • the current posture and the preset standard posture have an attitude error to be estimated.
  • the relative pose attitude is used to compare the first image and the second image with the relative pose estimation model to obtain relative pose information of the current pose of the onboard camera and the preset standard pose.
  • the relative pose estimation model is generated based on deep learning network training, that is, trained by step 302.
  • the first image acquired by the obtained in-vehicle camera in the preset standard posture and the second image acquired in the current posture may be input into the relative pose estimation model to obtain a relative posture between the current posture and the preset standard posture.
  • the information, the relative pose information is an attitude error between the current pose and the preset standard pose, and may include a translation parameter and a rotation angle parameter.
  • the step 301, the step 303, the step 3031, and the step 3032 in the foregoing method are the same as the step 201, the step 202, the step 2021, and the step 2022 in the foregoing embodiment, and the foregoing is directed to the step 201, the step 202, the step 2021, and the step 2022.
  • the description is also applicable to step 301, step 303, step 3031, and step 3032 in this embodiment, and details are not described herein again.
  • the embodiment adds the step of training the relative pose estimation model based on the deep learning network, and uses the in-vehicle camera to collect the preset scene in multiple preset test poses.
  • the image and the relative pose information of the preset test pose and the preset standard pose can construct a large amount of sample data, so that the trained model is more accurate, and the third image in the sample data can also be based on nature.
  • Scene generation reduces the constraints on sample data and extends the flexibility of the relative pose estimation model.
  • FIG. 4 a flow diagram of still another embodiment of an attitude error estimation method for an in-vehicle camera in accordance with the present application is shown.
  • the flow 400 of the attitude error estimation method of the vehicle-mounted camera includes the following steps:
  • Step 401 Acquire a first image of a preset scene collected by the onboard camera in a preset standard posture.
  • the electronic device for example, the server or the in-vehicle control unit shown in FIG. 1 on which the attitude error estimation method of the in-vehicle camera operates may be from a car camera or a storage device by a wired connection or a wireless connection.
  • the storage device of the image data collected by the camera acquires a first image of the preset scene captured by the on-board camera in a preset standard posture.
  • Step 402 performing an attitude error estimation step.
  • the posture error estimating step in step 402 may specifically include step 4021 and step 4022.
  • Step 4021 Acquire a second image of the preset scene collected by the onboard camera in the current posture.
  • the electronic device may acquire a second image of the preset scene acquired by the on-board camera in the current posture of the estimated attitude error by a wired or wireless connection.
  • the first image and the second image are images of the same preset scene captured by the in-vehicle camera in different postures.
  • Step 4022 Compare the first image and the second image by using the relative pose estimation model to obtain relative pose information of the current posture of the in-vehicle camera and the preset standard posture.
  • the first image and the second image may be input into the relative pose estimation model, and the relative pose information of the current state and the preset standard pose is estimated by using the relative pose model, thereby obtaining the pose error of the current pose.
  • the relative pose estimation model is generated based on deep learning network training.
  • Step 401, step 402, step 4021, and step 4022 in the foregoing method flow are the same as step 201, step 202, step 2021, and step 2022 in the foregoing embodiment, respectively, and are directed to step 201, step 202, step 2021, and step 2022.
  • the description is also applicable to step 401, step 402, step 4021, and step 4022 in this embodiment, and details are not described herein again.
  • the attitude error estimation method flow 400 of the in-vehicle camera of the embodiment may further include:
  • Step 403 Determine whether the relative pose information of the current posture of the on-vehicle camera and the preset standard posture satisfies a preset condition.
  • the preset condition may be a condition indicating that the estimated relative pose information accuracy is as expected.
  • the relative pose information may include a translation parameter and a rotation angle parameter
  • the preset condition may be that the translation parameter and the rotation angle parameter are within a preset numerical range, or the translation parameter and the rotation angle parameter satisfy an convergence condition
  • the convergence condition may be, for example, that the difference between the current translation parameter or the rotation angle parameter and the translation parameter or the rotation angle parameter obtained by the previous attitude error estimation step is less than a preset value; or may be, for example, that the current attitude error is less than Threshold value, etc.
  • Step 404 in response to determining that the relative pose information of the current posture of the in-vehicle camera and the preset standard posture does not satisfy the preset condition, based on the relative pose information of the current posture of the in-vehicle camera and the preset standard posture, the current on-vehicle camera The posture is corrected, and then the attitude error estimating step is performed.
  • the current posture of the in-vehicle camera can be corrected based on the relative pose information estimated in step 402.
  • the estimated translation parameters and rotation angle parameters can be transmitted to the servo attitude control mechanism for adjustment of the camera attitude.
  • the posture error of the current posture of the corrected in-vehicle camera can be estimated, and the posture error estimation can be performed based on the corrected in-vehicle camera returning step 402, and the relative position of the corrected current posture and the preset standard posture can be obtained.
  • Position information is the posture of the in-vehicle camera.
  • step 403 After obtaining the relative pose information of the corrected current posture and the preset standard posture, proceeding to step 403 according to the flow of the embodiment, determining whether the relative pose information of the current posture of the on-vehicle camera and the preset standard posture satisfies The condition is set, and when the relative pose information does not satisfy the preset condition, the process returns to step 402.
  • the attitude error estimation process of the in-vehicle camera can be ended when the relative pose information satisfies the preset condition.
  • the preset condition may be that the parameter for characterizing the relative pose information is less than a certain threshold.
  • the attitude error estimation method of the in-vehicle camera of the embodiment can cyclically execute the attitude error estimation step 402, continuously adjusting the current posture of the on-vehicle camera, and making the current posture approach the preset standard posture until the relative position obtained by the attitude error estimation step.
  • the posture information satisfies the preset condition, and the attitude of the vehicle camera can be calibrated when the vehicle camera deviates from the preset standard posture, thereby realizing the precise control of the attitude of the vehicle camera.
  • the present application provides an embodiment of an attitude error estimating apparatus for an in-vehicle camera, and the apparatus embodiment corresponds to the method embodiment shown in FIG.
  • the device can be specifically applied to various electronic devices.
  • the attitude error estimating apparatus 500 of the in-vehicle camera of the present embodiment includes an acquiring unit 501 and an estimating unit 502.
  • the obtaining unit 501 can be configured to acquire a first image of a preset scene collected by the onboard camera in a preset standard posture.
  • the estimating unit 502 may be configured to perform an attitude error estimating step, the step of estimating the second image of the preset scene acquired by the in-vehicle camera in the current posture; and comparing the first image and the second by using the relative pose estimation model
  • the image obtains relative pose information of the current posture of the on-vehicle camera and the preset standard posture, wherein the relative pose estimation model is generated based on the deep learning network training.
  • the obtaining unit 501 can be connected to the in-vehicle camera through a wired connection manner or a wireless connection manner, and obtain a first image of the preset scene captured by the on-vehicle camera in a preset standard posture by connecting with the in-vehicle camera.
  • the first image may be acquired by a connection with a storage device for storing image data acquired by the in-vehicle camera.
  • the preset standard posture is a standard working posture that can be a predefined in-vehicle camera.
  • the estimating unit 502 can be configured to estimate an attitude error between the current posture of the onboard camera and the preset standard posture.
  • the estimating unit 502 can be configured to perform the error estimating step described above.
  • the acquired first image and the second image may be compared with a relative pose estimation model based on the deep learning network training, and the preset standard posture of the vehicle camera when acquiring the first image is obtained.
  • the relative pose information of the current pose when the second image is acquired.
  • the relative pose information may include a pitch angle and a yaw angle of the current pose relative to the preset standard pose.
  • the attitude error may include a pitch angle and a yaw angle of the current pose relative to the preset standard pose.
  • the apparatus 500 described above may further include a model training unit configured to train the relative pose estimation model based on the deep learning network.
  • the model training unit is configured to: acquire a third image of the preset scene collected by the vehicle camera in a plurality of preset test postures, wherein the relative pose information of each preset test gesture and the preset standard posture is marked; The sample data is constructed based on the relative pose information of the first image, the third image, and the marked preset test pose and the preset standard pose; and the sample data is input into the constructed deep learning network for training to obtain a relative pose estimation model.
  • the attitude error estimating step performed by the estimating unit may further include: extracting feature points of the first image and the second image; Whether the first image and the second image contain feature points corresponding to the same spatial point; comparing the first image and the second image by using the relative pose estimation model includes: in response to determining that the first image and the second image comprise corresponding to The feature points of the same spatial point are compared with the first image and the second image by using the relative pose estimation model. In this way, it is possible to avoid the lack of effective image features in the first image and the second image, so that the attitude error estimation cannot achieve high precision.
  • the apparatus 500 described above may further include a determining unit and a correcting unit.
  • the determining unit may be configured to determine whether the relative pose information of the current posture of the in-vehicle camera and the preset standard posture satisfies a preset condition; and the correcting unit is configured to determine a relative position of the current posture of the in-vehicle camera and the preset standard posture in response to determining The posture information does not satisfy the preset condition, and based on the relative posture information of the current posture of the vehicle camera and the preset standard posture, after correcting the current posture of the vehicle-mounted camera, the attitude error estimating step is performed. In this way, the posture of the on-vehicle camera can be constantly corrected to approach the preset standard posture, and the precise control of the posture of the on-vehicle camera can be realized.
  • the attitude error estimating apparatus 500 of the in-vehicle camera of the above embodiment of the present application acquires the first image of the preset scene acquired by the in-vehicle camera in the preset standard posture by the acquiring unit, and then performs the following error estimating step by using the estimating unit: acquiring the in-vehicle camera a second image of the preset scene acquired in the current posture, and then using the relative pose estimation model generated based on the deep learning network training to compare the first image and the second image to obtain the current posture and the preset standard posture of the vehicle camera
  • the relative pose information realizes the automation and high-precision estimation of the attitude error of the vehicle camera.
  • apparatus 500 may correspond to various steps in the methods described with reference to Figures 2, 3, and 4. Thus, the operations and features described above for the method are equally applicable to the apparatus 500 and the units contained therein, and are not described herein again.
  • FIG. 6 a block diagram of a computer system 600 suitable for use in implementing a terminal device or server of an embodiment of the present application is shown.
  • the terminal device or server shown in FIG. 6 is merely an example, and should not impose any limitation on the functions and scope of use of the embodiments of the present application.
  • computer system 600 includes a central processing unit (CPU) 601 that can be loaded into a program in random access memory (RAM) 603 according to a program stored in read only memory (ROM) 602 or from storage portion 608. And perform various appropriate actions and processes.
  • RAM random access memory
  • ROM read only memory
  • RAM random access memory
  • various programs and data required for the operation of the system 600 are also stored.
  • the CPU 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604.
  • An input/output (I/O) interface 605 is also coupled to bus 604.
  • the following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a storage portion 608 including a hard disk or the like. And a communication portion 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the Internet.
  • Driver 610 is also coupled to I/O interface 605 as needed.
  • a removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory or the like, is mounted on the drive 610 as needed so that a computer program read therefrom is installed into the storage portion 608 as needed.
  • an embodiment of the present disclosure includes a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for executing the method illustrated in the flowchart.
  • the computer program can be downloaded and installed from the network via communication portion 609, and/or installed from removable media 611.
  • the central processing unit (CPU) 601 the above-described functions defined in the method of the present application are performed.
  • the computer readable medium described herein may be a computer readable signal medium or a computer readable storage medium or any combination of the two.
  • the computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.
  • a computer readable storage medium may be any tangible medium that can contain or store a program, which can be used by or in connection with an instruction execution system, apparatus or device.
  • a computer readable signal medium may include a data signal that is propagated in the baseband or as part of a carrier, carrying computer readable program code. Such propagated data signals can take a variety of forms including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the computer readable signal medium can also be any computer readable medium other than a computer readable storage medium, which can transmit, propagate, or transport a program for use by or in connection with the instruction execution system, apparatus, or device.
  • Program code embodied on a computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
  • each block of the flowchart or block diagram can represent a module, a program segment, or a portion of code that includes one or more of the logic functions for implementing the specified.
  • Executable instructions can also occur in a different order than that illustrated in the drawings. For example, two successively represented blocks may in fact be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts can be implemented in a dedicated hardware-based system that performs the specified function or operation. Or it can be implemented by a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments of the present application may be implemented by software or by hardware.
  • the described unit may also be provided in the processor, for example, as a processor comprising an acquisition unit and an estimation unit.
  • the name of the unit does not constitute a limitation on the unit itself in some cases.
  • the acquiring unit may also be described as “acquiring the first image of the preset scene acquired by the in-vehicle camera in the preset standard posture. unit”.
  • the present application also provides a computer readable medium, which may be included in the apparatus described in the above embodiments, or may be separately present and not incorporated into the apparatus.
  • the computer readable medium carries one or more programs, when the one or more programs are executed by the device, causing the device to: acquire a first image of a preset scene acquired by the onboard camera in a preset standard posture;
  • An attitude error estimating step comprising: acquiring a second image of the preset scene acquired by the onboard camera in a current posture; comparing the first image with the relative pose estimation model And the second image is obtained, and the relative pose information of the current posture of the in-vehicle camera and the preset standard posture is obtained, wherein the relative pose estimation model is generated based on the deep learning network training.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本申请公开了车载摄像头的姿态误差估计方法和装置。车载摄像头的姿态误差估计方法的一具体实施方式包括：获取车载摄像头在预设标准姿态下采集的预设场景的第一图像；执行姿态误差估算步骤，姿态误差估算步骤包括：获取车载摄像头在当前姿态下采集的预设场景的第二图像；利用相对位姿估计模型比对第一图像和第二图像，得出车载摄像头的当前姿态与预设标准姿态的相对位姿信息，其中，相对位姿估计模型基于深度学习网络训练生成。该方法可以实现车载摄像头误差的自动化、高精度估计。

Description

车载摄像头的姿态误差估计方法和装置
本专利申请要求于2017年9月5日提交的、申请号为201710790085.1、申请人为百度在线网络技术(北京)有限公司、发明名称为“车载摄像头的姿态误差估计方法和装置”的中国专利申请的优先权，该申请的全文以引用的方式并入本申请中
技术领域
本申请涉及车载设备技术领域，具体涉及车载摄像头的图像处理技术领域，尤其涉及车载摄像头的姿态误差估计方法和装置。
背景技术
在自动驾驶技术领域，车载摄像头作为主要的传感器，其所采集的数据对自动驾驶决策尤为重要。通常车载摄像头以特定的姿态工作，其所采集的数据均为该特定姿态下的数据。由于车辆行驶过程中车载摄像头可能发生震动，车载摄像头的维修等过程会对车载摄像头进行拆装，导致车载摄像头的姿态发生变化，则采集的数据可能发生偏差，因此需要校正车载摄像头的姿态误差。
现有的一种摄像头姿态误差估算方法为通过对辅助的标记物(例如包含特定标记点的标定板等)成像来计算摄像头的姿态参数，或者采用ICP(Iterative Closet Point，迭代最近点)等迭代的方式来估算姿态误差。前者对使用条件具有较大的限制，后者通常只能找到局部最优值，无法得出全局最优值，车载摄像头姿态误差估计的准确度有待提升。
发明内容
为了解决上述背景技术部分提到的一个或多个技术问题，本申请实施例提供了车载摄像头的姿态误差估计方法和装置。
第一方面，本申请实施例提供了一种车载摄像头的姿态误差估计方法，包括：获取车载摄像头在预设标准姿态下采集的预设场景的第一图像；执行姿态误差估算步骤，姿态误差估算步骤包括：获取车载摄像头在当前姿态下采集的预设场景的第二图像；利用相对位姿估计模型比对第一图像和第二图像，得出车载摄像头的当前姿态与预设标准姿态的相对位姿信息，其中，相对位姿估计模型基于深度学习网络训练生成。
在一些实施例中，上述方法还包括基于深度学习网络训练相对位姿估计模型的步骤，包括：获取车载摄像头在多个预设测试姿态下采集的预设场景的第三图像，其中，各预设测试姿态与预设标准姿态的相对位姿信息已标记；基于第一图像、第三图像和已标记的预设测试姿态与预设标准姿态的相对位姿信息构建样本数据；基于样本数据对构建的深度学习网络进行训练，得到相对位姿估计模型。
在一些实施例中，在利用相对位姿估计模型比对第一图像和第二图像之前，上述姿态误差估算步骤还包括：提取第一图像和第二图像的特征点；判断第一图像和第二图像是否包含对应于同一空间点的特征点；利用相对位姿估计模型比对第一图像和第二图像，包括：响应于判断出第一图像和第二图像包含对应于同一空间点的特征点，利用相对位姿估计模型比对第一图像和第二图像。
在一些实施例中，上述方法还包括：判断车载摄像头的当前姿态与预设标准姿态的相对位姿信息是否满足预设条件；响应于判断出车载摄像头的当前姿态与预设标准姿态的相对位姿信息不满足预设条件，基于车载摄像头的当前姿态与预设标准姿态的相对位姿信息，对车载摄像头的当前姿态进行校正之后，执行姿态误差估算步骤。
在一些实施例中，上述相对位姿信息包括当前姿态相对于预设标准姿态的俯仰角和偏航角。
第二方面，本申请实施例提供了一种车载摄像头的姿态误差估计装置，包括：获取单元，配置用于获取车载摄像头在预设标准姿态下采集的预设场景的第一图像；估算单元，配置用于执行姿态误差估算步骤，姿态误差估算步骤包括：获取车载摄像头在当前姿态下采集的 预设场景的第二图像；利用相对位姿估计模型比对第一图像和第二图像，得出车载摄像头的当前姿态与预设标准姿态的相对位姿信息，其中，相对位姿估计模型基于深度学习网络训练生成。
在一些实施例中，上述装置还包括模型训练单元，配置用于基于深度学习网络训练相对位姿估计模型；模型训练单元具体配置用于：获取车载摄像头在多个预设测试姿态下采集的预设场景的第三图像，其中，各预设测试姿态与预设标准姿态的相对位姿信息已标记；基于第一图像、第三图像和已标记的预设测试姿态与预设标准姿态的相对位姿信息构建样本数据；基于样本数据对构建的深度学习网络进行训练，得到相对位姿估计模型。
在一些实施例中，在利用相对位姿估计模型比对第一图像和第二图像之前，上述姿态误差估算步骤还包括：提取第一图像和第二图像的特征点；判断第一图像和第二图像是否包含对应于同一空间点的特征点；利用相对位姿估计模型比对第一图像和第二图像，包括：响应于判断出第一图像和第二图像包含对应于同一空间点的特征点，利用相对位姿估计模型比对第一图像和第二图像。
在一些实施例中，上述装置还包括：判断单元，配置用于判断车载摄像头的当前姿态与预设标准姿态的相对位姿信息是否满足预设条件；校正单元，配置用于响应于判断出车载摄像头的当前姿态与预设标准姿态的相对位姿信息不满足预设条件，基于车载摄像头的当前姿态与预设标准姿态的相对位姿信息，对车载摄像头的当前姿态进行校正之后，执行姿态误差估算步骤。
在一些实施例中，上述相对位姿信息包括当前姿态相对于预设标准姿态的俯仰角和偏航角。
第三方面，本申请实施例提供了一种设备，包括：一个或多个处理器；存储装置，用于存储一个或多个程序，当一个或多个程序被一个或多个处理器执行，使得一个或多个处理器实现上述车载摄像头的姿态误差估计方法。
本申请实施例提供的车载摄像头的姿态误差估计方法和装置，通过获取车载摄像头在预设标准姿态下采集的预设场景的第一图像，而 后执行姿态误差估算步骤，该姿态误差估算步骤包括：获取车载摄像头在当前姿态下采集的预设场景的第二图像；利用相对位姿估计模型比对第一图像和第二图像，得出车载摄像头的当前姿态与预设标准姿态的相对位姿信息，其中，相对位姿估计模型基于深度学习网络训练生成，实现了车载摄像头误差的自动化、高精度估计。
附图说明
通过阅读参照以下附图所作的对非限制性实施例详细描述，本申请的其它特征、目的和优点将会变得更明显：
图1是本申请可以应用于其中的示例性系统架构图；
图2是根据本申请的车载摄像头的姿态误差估计方法的一个实施例的流程图；
图3是根据本申请的车载摄像头的姿态误差估计方法的另一个实施例的流程图；
图4是根据本申请的车载摄像头的姿态误差估计方法的又一个实施例的流程图；
图5是本申请的车载摄像头的姿态误差估计装置的一个实施例的结构示意图；
图6是适于用来实现本申请实施例的终端设备或服务器的计算机系统的结构示意图。
具体实施方式
下面结合附图和实施例对本申请作进一步的详细说明。可以理解的是，此处所描述的具体实施例仅仅用于解释相关发明，而非对该发明的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与有关发明相关的部分。
需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。下面将参考附图并结合实施例来详细说明本申请。
图1示出了可以应用本申请的车载摄像头的姿态误差估计方法或 车载摄像头的姿态误差估计装置的实施例的示例性系统架构100。
如图1所示，系统架构100可以包括无人驾驶车辆101，安装在无人驾驶车辆101上的车载摄像头102和车载控制单元103、以及服务器104。车载控制单元103可以通过网络与服务器104连接，该网络可以包括各种连接类型，例如有线、无线通信链路或者光纤电缆等等。
车载摄像头102可以采集无人驾驶车辆行驶路径或停靠环境的图像数据。车载控制单元103可以为ECU(Electronic Control Unit，电子控制单元)，用于对车辆的各部件的工作状态进行分析和控制。车载控制单元103可以获取车载摄像头102采集的图像数据，根据图像数据进行响应，还可以控制图像数据的存储和传输。
服务器104可以通过网络与车载控制单元103建立连接，车载控制单元103可以将图像数据和其他传感器数据发送至服务器104。服务器104可以对图像数据和其他传感器数据进行诸如特征提取、匹配、分类等处理，将处理结果反馈至车载控制单元103。车载控制单元103可以根据接收到的处理结果进行响应。
通常为了精确匹配数据库中的数据(例如精确匹配地图数据和道路图像数据库中的数据)，车载摄像头102需要工作在标准姿态下，即车载摄像头102需要固定于与无人驾驶车辆101具有特定相对位置关系的位置(如图1位置A)。当车载摄像102头由于震动、拆装操作等发生位置变化(例如变化至图1位置B)、产生姿态误差时，车载摄像头102采集的图像数据无法与数据库中的数据很好地匹配，这时可以利用车载摄像头102采集的图像对车载摄像头102的姿态误差进行估计，并基于估计结果对车载摄像头102的姿态进行调整。
需要说明的是，本申请实施例所提供的车载摄像头的姿态误差估计方法可以由车载控制单元103或服务器104执行，相应地，车载摄像头的姿态误差估计装置可以设置于车载控制单元103或服务器104中。
应该理解，图1中的无人驾驶车辆、车载摄像头、车载控制单元和服务器的数目仅仅是示意性的。根据实现需要，可以具有任意数目 的无人驾驶车辆、车载摄像头、车载控制单元和服务器。
继续参考图2，示出了根据本申请的车载摄像头的姿态误差估计方法的一个实施例的流程200。该车载摄像头的姿态误差估计方法，包括以下步骤：
步骤201，获取车载摄像头在预设标准姿态下采集的预设场景的第一图像。
在本实施例中，车载摄像头的姿态误差估计方法运行于其上的电子设备(例如图1所示的服务器或车载控制单元)可以通过有线连接方式或者无线连接方式与车载摄像头连接。该电子设备可以通过与车载摄像头的连接获取车载摄像头在预设标准姿态下采集的预设场景的第一图像。或者该电子设备可以与用于存储车载摄像头采集的图像数据的存储设备连接，从存储设备获取上述第一图像。
在这里，预设标准姿态可以为预先定义的车载摄像头的标准工作姿态，该标准工作姿态下采集的图像数据可以作为无人驾驶车辆的参考图像数据库中的参考图像。或者说车载摄像头在预设标准姿态下采集的图像可以用于构建无人驾驶车辆的参考图像数据库。上述预设标准姿态可以用标准工作姿态下车载摄像头相对于地面和/或车辆上某一参照点的相对位置来表示，例如车载摄像头的预设标准姿态可以为与地面垂直距离1.5米，光轴与地面平行且与车头方向一致。
上述预设场景是预先设定的场景，可以为包含预设标记点的场景，例如可以包括标定板；也可以为自然场景，例如道路场景。车载摄像头可以在预设标准姿态下采集该预设场景的图像作为上述第一图像，并将第一图像存储至预设的存储设备。在估计车载摄像头的姿态误差时，上述车载摄像头的姿态误差估计方法运行于其上的电子设备可以从该存储设备中提取出该第一图像。
步骤202，执行姿态误差估算步骤。
在本实施例中，步骤202具体可以包括步骤2021和步骤2022。
在步骤2021中，获取车载摄像头在当前姿态下采集的预设场景的第二图像。
在本实施例中，上述电子设备可以通过与车载摄像头的连接获取 车载摄像头在当前姿态下对上述预设场景采集的第二图像。也即上述电子设备可以获取同一车载摄像头在预设标准姿态下和当前姿态下对相同的预设场景采集的图像。举例来说，可以设定预设场景为包含标志物体的场景，可以控制车载摄像头在标准姿态下采集包含该包含标志物体的场景的图像作为第一图像并进行存储，控制车载摄像头在当前姿态下采集包含标志物体的场景的图像作为第二图像。
在步骤2022中，利用相对位姿估计模型比对第一图像和第二图像，得出车载摄像头的当前姿态与预设标准姿态的相对位姿信息。
在本实施例中，相对位姿估计模型基于深度学习网络训练生成。相对位姿估计模型用于估计采集输入的不同图像的不同摄像头之间的相对位姿信息。该相对位姿估计模型的输入可以为两幅图像，输出可以为采集两幅图像的摄像头的相对位姿信息。在这里，相对位姿信息可以包括平移参数和旋转角参数。
上述相对位姿估计模型为基于深度学习网络训练生成的，可以是包括输入层、多个隐层和输出层的网络模型。可选地，相对位姿估计模型可以为CNN(Convolutional Neural Network，卷积神经网络)模型、RNN(Recurrent neural Network、循环神经网络)模型等。训练数据可以包括样本集和测试集，训练数据可以为已有的被验证为姿态误差估计结果准确的数据，训练数据可以通过网络获取，也可以实地采集。训练数据包括多对输入数据和输出数据，输入数据可以为车载摄像头在不同位姿下采集的图像，输出数据为对应的相对位姿信息。
由于上述第一图像和第二图像是同一车载摄像头在不同姿态下采集的同一预设场景的图像，则第一图像和第二图像中包含了相同空间点的二维图像信息，上述相对位姿估计模型可以对第一图像和第二图像进行比对，利用深度学习网络模型分析第一图像和第二图像中同一空间点在不同的图像坐标系中的坐标之间的关联，通过多层网络结构进行特征提取、分类，进而得出不同图像坐标系之间的转换参数，即可以得出车载摄像头的姿态误差。通过采用基于深度学习网络训练的相对位姿估计模型对相对位姿信息进行估计，可以得到姿态误差的全局最优值，从而提升了姿态误差估算精度。
车载摄像头的姿态误差即为车载摄像头的当前姿态与预设标准姿态之间的姿态差异，也就是车载摄像头在采集第二图像时的姿态与采集第一图像时的姿态之间的相对位姿信息。
可选地，相对位姿信息可以包括上述当前姿态相对于上述预设标准姿态的姿态角，这里的姿态角可以包括俯仰角和偏航角。其中俯仰角可以为车载摄像头的当前姿态相对于预设标准姿态下在垂直于地面方向上的偏转角度，偏航角可以为车载摄像头的当前姿态相对于预设标准姿态下在平行于地面方向上的偏转角度。
进一步可选地，上述相对位姿信息还可以包括当前姿态相对于预设标准姿态的平移参数，这里的平移参数可以表示当前姿态相对于预设标准姿态在与地面平行的平面内的平移量以及在垂直于地面的平面内的平移量。
本申请上述实施例的车载摄像头的姿态误差估计方法，通过获取车载摄像头在预设标准姿态下采集的预设场景的第一图像，而后执行姿态误差估算步骤，该姿态误差估算步骤包括：获取车载摄像头在当前姿态下采集的预设场景的第二图像；利用相对位姿估计模型比对第一图像和第二图像，得出车载摄像头的当前姿态与预设标准姿态的相对位姿信息，其中，相对位姿估计模型基于深度学习网络训练生成，能够得到姿态误差估计的全局最优值，实现了车载摄像头姿态误差的自动化、高精度估计。
在本实施例的一些可选的实现方式中，在利用相对位姿估计模型比对第一图像和第二图像之前，姿态误差估算步骤202还可以包括：提取第一图像和第二图像的特征点；判断第一图像和第二图像是否包含对应于同一空间点的特征点。具体地，上述电子设备可以采用诸如SITF(Scale-invariant feature transform，尺度不变特征变换)等特征点提取算法从第一图像和第二图像分别提取出特征点，然后对第一图像的特征点和第二图像的特征点进行匹配，判断第一图像和第二图像是否包含对应于同一空间点的特征点。则上述步骤2022可以包括：响应于判断出第一图像和第二图像对应于同一空间点的特征点，利用相对位姿估计模型比对第一图像和第二图像。在第一图像和第二图像包含 对应于同一空间点的特征点时，可以确定第一图像和第二图像中上述预设场景的像具有较明显的特征，图像信息较丰富，可以用于估算车载摄像头的姿态误差。
进一步地，若第一图像和第二图像中不包含对应于同一空间点的特征点，则可以确定第一图像和第二图像的图像信息过少，不适于用来对车载摄像头的姿态误差进行估算。这时，可以获取车载摄像头分别在预设标准姿态下和当前姿态下采集的其他预设场景的第一图像和第二图像进行特征点提取和匹配，这样，可以选出纹理等特征信息较丰富的预设场景的第一图像和第二图像，进而提升车载摄像头误差估计的精准度。
请参考图3，其示出了根据本申请的车载摄像头的姿态误差估计方法的另一个实施例的流程图。如图3所示，本实施例的车载摄像头的姿态误差估计方法的流程300，可以包括以下步骤：
步骤301，获取车载摄像头在预设标准姿态下采集的预设场景的第一图像。
在本实施例中，车载摄像头的姿态误差估计方法运行于其上的电子设备(如图1所示车载控制单元或服务器)可以通过有线或无线的方式与车载摄像头连接，获取车载摄像头在预设标准姿态下采集的预设场景的第一图像。上述电子设备也可以与用于存储车载摄像头采集的图像的存储设备(例如车辆黑匣子)连接，并从该存储设备获取车载摄像头在预设标准姿态下采集的预设场景的第一图像。在这里，预设标准姿态可以为预先定义的车载摄像头的标准工作姿态，预设场景是预先设定的场景，可以为包含预设标记点的场景，例如可以为包括标定板的场景；也可以为自然场景，例如道路场景。
步骤302，基于深度学习网络训练相对位姿估计模型。
在本实施例中，可以采用监督学习的方式，基于深度学习网络训练相位位姿估计模型。步骤302具体可以包括步骤3021、步骤3022和步骤3023。
在步骤3021中，获取车载摄像头在多个预设测试姿态下采集的预设场景的第三图像。
在本实施例中，上述各预设测试姿态与预设标准姿态的相对位姿信息已标记。也就是说，各预设测试姿态与预设标准姿态的相对位姿信息是已知的。具体地，可以定义多组相对位姿信息，按照定义好的每组相对位姿信息对预设标准姿态下的车载摄像头进行位姿调整，调整之后的车载摄像头的姿态即为预设测试姿态。然后在预设测试姿态下采集预设场景的图像，获得与定义的各组相对位姿信息对应的第三图像。可选地，相对位姿信息可以包括姿态角和沿预设标准姿态的车载摄像头坐标系中三个坐标轴的平移参数。
在步骤3022中，基于第一图像、第三图像和已标记的预设测试姿态与预设标准姿态的相对位姿信息构建样本数据。
在本实施例中，可以构建用于训练相对位姿估计模型的样本数据。每条样本数据包括该模型的输入数据和输出数据。输入数据为第一图像和第三图像，输出数据为采集第三图像的车载摄像头的姿态与预设标准姿态的相对位姿信息。在这里，第一图像和第三图像分别在步骤301和步骤3021中获得，采集第三图像的车载摄像头的姿态与预设标准姿态的相对位姿信息为已标记的信息，例如可以为人为定义的信息。
在本实施例中，可以将车载摄像头的预设测试姿态与预设标准姿态的相对位姿信息与该预设测试姿态采集到的第三图像和在预设标准姿态下采集的第一图像作为该预设测试姿态对应的样本数据。车载摄像头可以在多个预设测试姿态下采集对应的第三图像，则可以得到多个预设测试姿态对应的多条样本数据。在一些可选的实现方式中，可以将多条样本数据集合生成样本数据集。
在步骤3023中，基于样本数据对构建的深度学习网络进行训练，得到相对位姿估计模型。
在本实施例中，可以构建深度学习网络的结构，并利用样本数据来训练深度学习网络，得到相对位姿估计模型。相对位姿模型可以是基于输入的两幅图像计算出采集两幅图像的摄像头的姿态之间的相对姿态误差的模型。具体可以基于CNN、RNN等已有的神经网络模型构建深度学习网络，将样本数据输入构建的深度学习网络进行训练，优化深度学习网络的模型结构，得到相对位姿估计模型。
在本实施例的一些可选的实现方式中，可以从样本数据集中随机抽取一些样本数据生成训练集，另一些样本数据作为测试集。利用训练集来训练深度学习网络，并基于测试集来测试深度学习网络的性能。如果测试得出深度学习网络的性能未达到预期的指标，则可以增加训练集中样本数据的数量，继续训练，通过训练集和测试集不断对深度学习网络进行调整，得到最终的相对位姿估计模型。
步骤303，执行误差估计步骤。
之后，可以执行误差估计步骤。具体地误差估计步骤包括步骤3031和步骤3032。
在步骤3031中，获取车载摄像头在当前姿态下采集的预设场景的第二图像。
上述电子设备可以获取车载摄像头在当前姿态下对预设场景采集的第二图像。这里的当前姿态与预设标准姿态具有待估计的姿态误差。
在步骤3032中，利用相对位姿估计模型比对第一图像和第二图像，得出车载摄像头的当前姿态与预设标准姿态的相对位姿信息。
在这里，相对位姿估计模型是基于深度学习网络训练生成的，即由步骤302训练生成的。可以将获取的车载摄像头在预设标准姿态下采集的第一图像和在当前姿态下采集的第二图像输入该相对位姿估计模型，得出当前姿态与预设标准姿态之间的相对位姿信息，该相对位姿信息即为当前姿态与预设标准姿态之间的姿态误差，可以包括平移参数和旋转角参数。
上述方法流程中的步骤301、步骤303、步骤3031、步骤3032分别与前述实施例中的步骤201、步骤202、步骤2021、步骤2022相同，上文针对步骤201、步骤202、步骤2021、步骤2022的描述也适用于本实施中的步骤301、步骤303、步骤3031、步骤3032，此处不再赘述。
从图3可以看出，与图2所示实施例相比，本实施例增加了基于深度学习网络训练相对位姿估计模型的步骤，利用车载摄像头在多个预设测试姿态下采集预设场景的图像，并标记多个预设测试姿态与预设标准姿态的相对位姿信息，可以构建出大量的样本数据，使训练得 出的模型更准确，且样本数据中第三图像也可以基于自然场景生成，从而降低了对样本数据的限制，能够拓展相对位姿估计模型的灵活性。
继续参考图4，其示出了根据本申请的车载摄像头的姿态误差估计方法的又一个实施例的流程图。如图4所示，该车载摄像头的姿态误差估计方法的流程400，包括以下步骤：
步骤401，获取车载摄像头在预设标准姿态下采集的预设场景的第一图像。
在本实施例中，车载摄像头的姿态误差估计方法运行于其上的电子设备(例如图1所示的服务器或车载控制单元)可以通过有线连接方式或者无线连接方式从车载摄像头或用于存储车载摄像头采集的图像数据的存储设备获取车载摄像头在预设标准姿态下采集的预设场景的第一图像。
步骤402，执行姿态误差估算步骤。
步骤402中姿态误差估算步骤具体可以包括步骤4021和步骤4022。
其中，步骤4021，获取车载摄像头在当前姿态下采集的预设场景的第二图像。
在本实施例中，上述电子设备可以通过有线或无线的连接方式获取车载摄像头在待估计姿态误差的当前姿态下采集的预设场景的第二图像。上述第一图像和第二图像为车载摄像头在不同姿态下采集的同一预设场景的图像。
步骤4022，利用相对位姿估计模型比对第一图像和第二图像，得出车载摄像头的当前姿态与预设标准姿态的相对位姿信息。
之后，可以将第一图像和第二图像输入相对位姿估计模型，利用相对位姿模型估算当前状态与预设标准姿态的相对位姿信息，从而得出当前姿态的姿态误差。在这里，相对位姿估计模型是基于深度学习网络训练生成的。
上述方法流程中的步骤401、步骤402、步骤4021、步骤4022分别与前述实施例中的步骤201、步骤202、步骤2021、步骤2022相同，上文针对步骤201、步骤202、步骤2021、步骤2022的描述也适用于 本实施中的步骤401、步骤402、步骤4021、步骤4022，此处不再赘述。
本实施例的车载摄像头的姿态误差估计方法流程400还可以包括：
步骤403，判断车载摄像头的当前姿态与预设标准姿态的相对位姿信息是否满足预设条件。
在本实施例中，在步骤402估算得出车载摄像头的当前姿态与预设标准姿态的相对位姿信息之后，可以判断估算出的相对位姿信息是否满足预设条件。预设条件可以是表示估算出的相对位姿信息准确度达到预期的条件。
可选地，相对位姿信息可以包括平移参数和旋转角参数，则预设条件可以为平移参数和旋转角参数在预设的数值范围内，或者平移参数和旋转角参数满足收敛条件，进一步地，该收敛条件可以例如为当前平移参数或旋转角参数与上一次姿态误差估算步骤得出的平移参数或旋转角参数之间的差异小于预设的值；又或者可以例如为当前的姿态误差小于阈值等。
步骤404，响应于判断出车载摄像头的当前姿态与预设标准姿态的相对位姿信息不满足预设条件，基于车载摄像头的当前姿态与预设标准姿态的相对位姿信息，对车载摄像头的当前姿态进行校正，之后执行姿态误差估算步骤。
当步骤403的判断结果为“否”时，可以基于步骤402估算出的相对位姿信息对车载摄像头的当前姿态进行校正。例如可以将估算出的平移参数和旋转角参数传输至伺服姿态控制机构进行摄像头姿态的调整。之后，可以对校正后的车载摄像头的当前姿态的姿态误差进行估计，具体可以基于校正后的车载摄像头返回步骤402，进行姿态误差估算，得出校正后的当前姿态与预设标准姿态的相对位姿信息。
在得到校正后的当前姿态与预设标准姿态的相对位姿信息之后，按照本实施例的流程，继续执行步骤403，判断车载摄像头的当前姿态与预设标准姿态的相对位姿信息是否满足预设条件，并在该相对位姿信息不满足预设条件时返回执行步骤402。当相对位姿信息满足预 设条件时可以结束车载摄像头的姿态误差估计流程。在这里，预设条件可以为用于表征相对位姿信息的参数小于一定的阈值。
这样，本实施例的车载摄像头的姿态误差估计方法可以循环执行姿态误差估算步骤402，不断调整车载摄像头的当前姿态，使当前姿态与预设标准姿态逼近，直至姿态误差估算步骤得出的相对位姿信息满足预设的条件，能够在车载摄像头偏离预设标准姿态时将车载摄像头的姿态校准，实现了车载摄像头的姿态的精准控制。
进一步参考图5，作为对上述各图所示方法的实现，本申请提供了一种车载摄像头的姿态误差估计装置的一个实施例，该装置实施例与图2所示的方法实施例相对应，该装置具体可以应用于各种电子设备中。
如图5所示，本实施例的车载摄像头的姿态误差估计装置500包括：获取单元501和估算单元502。其中，获取单元501可以配置用于获取车载摄像头在预设标准姿态下采集的预设场景的第一图像。估算单元502可以配置用于执行姿态误差估算步骤，该误差估算步骤包括：获取车载摄像头在当前姿态下采集的预设场景的第二图像；利用相对位姿估计模型比对第一图像和第二图像，得出车载摄像头的当前姿态与预设标准姿态的相对位姿信息，其中，相对位姿估计模型基于深度学习网络训练生成。
在本实施例中，获取单元501可以通过有线连接方式或者无线连接方式与车载摄像头连接，并通过与车载摄像头的连接获取车载摄像头在预设标准姿态下采集的预设场景的第一图像，也可以通过与用于存储车载摄像头采集的图像数据的存储设备间的连接来获取上述第一图像。在这里，预设标准姿态为可以为预先定义的车载摄像头的标准工作姿态。
估算单元502可以用于对车载摄像头的当前姿态与预设标准姿态之间的姿态误差进行估算。具体地，估算单元502可以配置用于执行上述误差估算步骤。在误差估算步骤中，可以将获取的第一图像和第二图像输入已基于深度学习网络训练的相对位姿估计模型进行比对，得出车载摄像头在采集第一图像时的预设标准姿态和采集第二图像时 的当前姿态的相对位姿信息。
在本实施例的一些可选的实现方式中，相对位姿信息可以包括当前姿态相对于预设标准姿态的俯仰角和偏航角。相应地，姿态误差可以包括当前姿态相对于预设标准姿态的俯仰角和偏航角。
在一些实施例中，上述装置500还可以包括模型训练单元，配置用于基于深度学习网络训练相对位姿估计模型。该模型训练单元具体配置用于：获取车载摄像头在多个预设测试姿态下采集的预设场景的第三图像，其中，各预设测试姿态与预设标准姿态的相对位姿信息已标记；基于第一图像、第三图像和已标记的预设测试姿态与预设标准姿态的相对位姿信息构建样本数据；将样本数据输入构建的深度学习网络进行训练，得到相对位姿估计模型。
在一些实施例中，在利用相对位姿估计模型比对第一图像和第二图像之前，上述估算单元执行的姿态误差估算步骤还可以包括：提取第一图像和第二图像的特征点；判断第一图像和第二图像是否包含对应于同一空间点的特征点；利用相对位姿估计模型比对第一图像和第二图像，包括：响应于判断出第一图像和第二图像包含对应于同一空间点的特征点，利用相对位姿估计模型比对第一图像和第二图像。这样，可以避免第一图像和第二图像中缺乏有效的图像特征而使姿态误差估计无法达到较高的精度。
在一些实施例中，上述装置500还可以包括判断单元和校正单元。判断单元可以配置用于判断车载摄像头的当前姿态与预设标准姿态的相对位姿信息是否满足预设条件；校正单元配置用于响应于判断出车载摄像头的当前姿态与预设标准姿态的相对位姿信息不满足预设条件，基于车载摄像头的当前姿态与预设标准姿态的相对位姿信息，对车载摄像头的当前姿态进行校正之后，执行姿态误差估算步骤。这样，可以对车载摄像头的姿态不断校正使其逼近预设标准姿态，能够实现车载摄像头姿态的精准控制。
本申请上述实施例的车载摄像头的姿态误差估计装置500，通过获取单元获取车载摄像头在预设标准姿态下采集的预设场景的第一图像，随后利用估算单元执行如下误差估算步骤：获取车载摄像头在当 前姿态下采集的预设场景的第二图像，之后利用基于深度学习网络训练生成的相对位姿估计模型比对第一图像和第二图像，得出车载摄像头的当前姿态与预设标准姿态的相对位姿信息，实现了车载摄像头姿态误差的自动化、高精度估计。
应当理解，装置500中记载的诸单元可以与参考图2、图3和图4描述的方法中的各个步骤相对应。由此，上文针对方法描述的操作和特征同样适用于装置500及其中包含的单元，在此不再赘述。
下面参考图6，其示出了适于用来实现本申请实施例的终端设备或服务器的计算机系统600的结构示意图。图6示出的终端设备或服务器仅仅是一个示例，不应对本申请实施例的功能和使用范围带来任何限制。
如图6所示，计算机系统600包括中央处理单元(CPU)601，其可以根据存储在只读存储器(ROM)602中的程序或者从存储部分608加载到随机访问存储器(RAM)603中的程序而执行各种适当的动作和处理。在RAM 603中，还存储有系统600操作所需的各种程序和数据。CPU 601、ROM 602以及RAM 603通过总线604彼此相连。输入/输出(I/O)接口605也连接至总线604。
以下部件连接至I/O接口605：包括键盘、鼠标等的输入部分606；包括诸如阴极射线管(CRT)、液晶显示器(LCD)等以及扬声器等的输出部分607；包括硬盘等的存储部分608；以及包括诸如LAN卡、调制解调器等的网络接口卡的通信部分609。通信部分609经由诸如因特网的网络执行通信处理。驱动器610也根据需要连接至I/O接口605。可拆卸介质611，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器610上，以便于从其上读出的计算机程序根据需要被安装入存储部分608。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信部分609从网络上被下载和安装，和/或从 可拆卸介质611被安装。在该计算机程序被中央处理单元(CPU)601执行时，执行本申请的方法中限定的上述功能。需要说明的是，本申请所述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本申请中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本申请中，计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：无线、电线、光缆、RF等等，或者上述的任意合适的组合。
附图中的流程图和框图，图示了按照本申请各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组 合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本申请实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。所描述的单元也可以设置在处理器中，例如，可以描述为：一种处理器包括获取单元和估算单元。其中，这些单元的名称在某种情况下并不构成对该单元本身的限定，例如，获取单元还可以被描述为“获取车载摄像头在预设标准姿态下采集的预设场景的第一图像的单元”。
作为另一方面，本申请还提供了一种计算机可读介质，该计算机可读介质可以是上述实施例中描述的装置中所包含的；也可以是单独存在，而未装配入该装置中。上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该装置执行时，使得该装置：获取车载摄像头在预设标准姿态下采集的预设场景的第一图像；执行姿态误差估算步骤，所述姿态误差估算步骤包括：获取所述车载摄像头在当前姿态下采集的所述预设场景的第二图像；利用相对位姿估计模型比对所述第一图像和所述第二图像，得出所述车载摄像头的当前姿态与所述预设标准姿态的相对位姿信息，其中，所述相对位姿估计模型基于深度学习网络训练生成。
以上描述仅为本申请的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本申请中所涉及的发明范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述发明构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本申请中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。

Claims (12)

  1. 一种车载摄像头的姿态误差估计方法，其特征在于，所述方法包括：
    获取车载摄像头在预设标准姿态下采集的预设场景的第一图像；
    执行姿态误差估算步骤，所述姿态误差估算步骤包括：
    获取所述车载摄像头在当前姿态下采集的所述预设场景的第二图像；
    利用相对位姿估计模型比对所述第一图像和所述第二图像，得出所述车载摄像头的当前姿态与所述预设标准姿态的相对位姿信息，其中，所述相对位姿估计模型基于深度学习网络训练生成。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括基于深度学习网络训练所述相对位姿估计模型的步骤，包括：
    获取所述车载摄像头在多个预设测试姿态下采集的所述预设场景的第三图像，其中，各所述预设测试姿态与所述预设标准姿态的相对位姿信息已标记；
    基于所述第一图像、所述第三图像和已标记的所述预设测试姿态与所述预设标准姿态的相对位姿信息构建样本数据；
    基于所述样本数据对构建的深度学习网络进行训练，得到所述相对位姿估计模型。
  3. 根据权利要求1所述的方法，其特征在于，在利用相对位姿估计模型比对所述第一图像和所述第二图像之前，所述姿态误差估算步骤还包括：
    提取所述第一图像和所述第二图像的特征点；
    判断所述第一图像和所述第二图像是否包含对应于同一空间点的特征点；
    所述利用相对位姿估计模型比对所述第一图像和所述第二图像，包括：
    响应于判断出所述第一图像和所述第二图像包含对应于同一空间点的特征点，利用相对位姿估计模型比对所述第一图像和所述第二图像。
  4. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    判断所述车载摄像头的当前姿态与所述预设标准姿态的相对位姿信息是否满足预设条件；
    响应于判断出所述车载摄像头的当前姿态与所述预设标准姿态的相对位姿信息不满足预设条件，基于所述车载摄像头的当前姿态与所述预设标准姿态的相对位姿信息，对所述车载摄像头的当前姿态进行校正之后，执行所述姿态误差估算步骤。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，所述相对位姿信息包括所述当前姿态相对于所述预设标准姿态的俯仰角和偏航角。
  6. 一种车载摄像头的姿态误差估计装置，其特征在于，所述装置包括：
    获取单元，配置用于获取车载摄像头在预设标准姿态下采集的预设场景的第一图像；
    估算单元，配置用于执行姿态误差估算步骤，所述姿态误差估算步骤包括：
    获取所述车载摄像头在当前姿态下采集的所述预设场景的第二图像；
    利用相对位姿估计模型比对所述第一图像和所述第二图像，得出所述车载摄像头的当前姿态与所述预设标准姿态的相对位姿信息，其中，所述相对位姿估计模型基于深度学习网络训练生成。
  7. 根据权利要求6所述的装置，其特征在于，所述装置还包括模型训练单元，配置用于基于深度学习网络训练所述相对位姿估计模型；
    所述模型训练单元具体配置用于：
    获取所述车载摄像头在多个预设测试姿态下采集的所述预设场景的第三图像，其中，各所述预设测试姿态与所述预设标准姿态的相对位姿信息已标记；
    基于所述第一图像、所述第三图像和已标记的所述预设测试姿态与所述预设标准姿态的相对位姿信息构建样本数据；
    基于所述样本数据对构建的深度学习网络进行训练，得到所述相对位姿估计模型。
  8. 根据权利要求6所述的装置，其特征在于，在利用相对位姿估计模型比对所述第一图像和所述第二图像之前，所述姿态误差估算步骤还包括：
    提取所述第一图像和所述第二图像的特征点；
    判断所述第一图像和所述第二图像是否包含对应于同一空间点的特征点；
    所述利用相对位姿估计模型比对所述第一图像和所述第二图像，包括：
    响应于判断出所述第一图像和所述第二图像包含对应于同一空间点的特征点，利用相对位姿估计模型比对所述第一图像和所述第二图像。
  9. 根据权利要求6所述的装置，其特征在于，所述装置还包括：
    判断单元，配置用于判断所述车载摄像头的当前姿态与所述预设标准姿态的相对位姿信息是否满足预设条件；
    校正单元，配置用于响应于判断出所述车载摄像头的当前姿态与所述预设标准姿态的相对位姿信息不满足预设条件，基于所述车载摄像头的当前姿态与所述预设标准姿态的相对位姿信息，对所述车载摄像头的当前姿态进行校正之后，执行所述姿态误差估算步骤。
  10. 根据权利要求6-9任一项所述的装置，其特征在于，所述相 对位姿信息包括所述当前姿态相对于所述预设标准姿态的俯仰角和偏航角。
  11. 一种设备，其特征在于，包括：
    一个或多个处理器；
    存储装置，用于存储一个或多个程序，
    当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如权利要求1-5中任一所述的方法。
  12. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，该程序被处理器执行时实现如权利要求1-5中任一所述的方法。
PCT/CN2018/098621 2017-09-05 2018-08-03 车载摄像头的姿态误差估计方法和装置 Ceased WO2019047641A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710790085.1A CN107481292B (zh) 2017-09-05 2017-09-05 车载摄像头的姿态误差估计方法和装置
CN201710790085.1 2017-09-05

Publications (1)

Publication Number Publication Date
WO2019047641A1 true WO2019047641A1 (zh) 2019-03-14

Family

ID=60603809

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/098621 Ceased WO2019047641A1 (zh) 2017-09-05 2018-08-03 车载摄像头的姿态误差估计方法和装置

Country Status (2)

Country Link
CN (1) CN107481292B (zh)
WO (1) WO2019047641A1 (zh)

Families Citing this family (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107481292B (zh) * 2017-09-05 2020-07-28 百度在线网络技术（北京）有限公司 车载摄像头的姿态误差估计方法和装置
US10936886B2 (en) * 2017-12-29 2021-03-02 PlusAI Corp Method and system for stereo based vehicle pose estimation
CN109544630B (zh) * 2018-11-30 2021-02-02 南京人工智能高等研究院有限公司 位姿信息确定方法和装置、视觉点云构建方法和装置
CN111288956B (zh) * 2018-12-07 2022-04-22 顺丰科技有限公司 一种目标姿态确定方法、装置、设备和存储介质
CN109712196B (zh) * 2018-12-17 2021-03-30 北京百度网讯科技有限公司 摄像头标定处理方法、装置、车辆控制设备及存储介质
CN109658323A (zh) * 2018-12-19 2019-04-19 北京旷视科技有限公司 图像获取方法、装置、电子设备和计算机存储介质
CN111563840B (zh) * 2019-01-28 2023-09-05 北京魔门塔科技有限公司 分割模型的训练方法、装置、位姿检测方法及车载终端
CN109849972A (zh) * 2019-02-12 2019-06-07 西安思科赛德电子科技有限公司 智能探测视频采集系统的在线姿态调整系统及其调整方法
US11024054B2 (en) 2019-05-16 2021-06-01 Here Global B.V. Method, apparatus, and system for estimating the quality of camera pose data using ground control points of known quality
CN110132280B (zh) * 2019-05-20 2021-07-13 广州小鹏自动驾驶科技有限公司 室内场景下的车辆定位方法、车辆定位装置和车辆
CN110146869B (zh) * 2019-05-21 2021-08-10 北京百度网讯科技有限公司 确定坐标系转换参数的方法、装置、电子设备和存储介质
CN112183160B (zh) * 2019-07-04 2024-08-13 北京七鑫易维科技有限公司 视线估计方法及装置
CN110796705B (zh) * 2019-10-23 2022-10-11 北京百度网讯科技有限公司 模型误差消除方法、装置、设备及计算机可读存储介质
CN112766023B (zh) * 2019-11-04 2024-01-19 北京地平线机器人技术研发有限公司 目标对象的姿态确定方法、装置、介质及设备
CN112907658B (zh) * 2019-11-19 2025-06-20 华为技术有限公司 视觉定位评估方法和电子设备
CN111462238B (zh) * 2020-04-03 2023-04-07 清华大学 姿态估计优化方法、装置及存储介质
CN112702519B (zh) * 2020-12-24 2022-04-22 联想(北京)有限公司 一种图像处理方法和电子设备
WO2022198442A1 (zh) * 2021-03-23 2022-09-29 深圳市锐明技术股份有限公司 一种货箱监控方法、终端设备及存储介质
CN113538594B (zh) * 2021-06-30 2022-04-22 东风汽车集团股份有限公司 基于方向传感器的车载摄像头标定方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150235378A1 (en) * 2014-02-17 2015-08-20 Samsung Electronics Co., Ltd. Apparatus and method for estimating camera pose
CN106097322A (zh) * 2016-06-03 2016-11-09 江苏大学 一种基于神经网络的视觉系统标定方法
CN106780608A (zh) * 2016-11-23 2017-05-31 北京地平线机器人技术研发有限公司 位姿信息估计方法、装置和可移动设备
CN107481292A (zh) * 2017-09-05 2017-12-15 百度在线网络技术（北京）有限公司 车载摄像头的姿态误差估计方法和装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6058256B2 (ja) * 2011-06-13 2017-01-11 アルパイン株式会社 車載カメラ姿勢検出装置および方法
JP6009894B2 (ja) * 2012-10-02 2016-10-19 株式会社デンソー キャリブレーション方法、及びキャリブレーション装置
CN104482924B (zh) * 2014-12-11 2016-11-09 中国航天空气动力技术研究院 旋成体目标位姿视觉测量方法
CN106599810B (zh) * 2016-12-05 2019-05-14 电子科技大学 一种基于栈式自编码的头部姿态估计方法
CN106981082B (zh) * 2017-03-08 2020-04-17 驭势科技（北京）有限公司 车载摄像头标定方法、装置及车载设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150235378A1 (en) * 2014-02-17 2015-08-20 Samsung Electronics Co., Ltd. Apparatus and method for estimating camera pose
CN106097322A (zh) * 2016-06-03 2016-11-09 江苏大学 一种基于神经网络的视觉系统标定方法
CN106780608A (zh) * 2016-11-23 2017-05-31 北京地平线机器人技术研发有限公司 位姿信息估计方法、装置和可移动设备
CN107481292A (zh) * 2017-09-05 2017-12-15 百度在线网络技术（北京）有限公司 车载摄像头的姿态误差估计方法和装置

Also Published As

Publication number Publication date
CN107481292B (zh) 2020-07-28
CN107481292A (zh) 2017-12-15

Similar Documents

Publication Publication Date Title
CN107481292B (zh) 车载摄像头的姿态误差估计方法和装置
CN109345596B (zh) 多传感器标定方法、装置、计算机设备、介质和车辆
CN109523597B (zh) 相机外参的标定方法和装置
CN109544629B (zh) 摄像头位姿确定方法和装置以及电子设备
EP3627180A1 (en) Sensor calibration method and device, computer device, medium, and vehicle
CN108419446B (zh) 用于激光深度图取样的系统及方法
CN110969055B (zh) 用于车辆定位的方法、装置、设备和计算机可读存储介质
JP2021119507A (ja) 車線の決定方法、車線測位精度の評価方法、車線の決定装置、車線測位精度の評価装置、電子デバイス、コンピュータ可読記憶媒体、及びプログラム
CN110047108B (zh) 无人机位姿确定方法、装置、计算机设备及存储介质
CN114097006A (zh) 交叉模态传感器数据对准
US11170530B2 (en) Calibrating crop row computer vision system
WO2019047651A1 (zh) 驾驶行为预测方法和装置、无人车
US20230141515A1 (en) System and method for stereoscopic image analysis
CN110148177A (zh) 用于确定相机的姿态角的方法、装置、计算设备、计算机可读存储介质以及采集实体
CN110794844A (zh) 自动驾驶方法、装置、电子设备及可读存储介质
CN111563916B (zh) 基于立体视觉的长时无人机跟踪定位方法、系统、装置
CN117687006A (zh) 激光雷达到惯性测量单元的外参标定方法及其装置
CN117870716A (zh) 地图兴趣点的显示方法、装置、电子设备及存储介质
CN112925302A (zh) 机器人位姿控制方法和装置
CN115638788A (zh) 语义矢量地图的构建方法、计算机设备及存储介质
US20250117029A1 (en) Automatic multi-modality sensor calibration with near-infrared images
CN116883504B (zh) 车辆朝向的校准方法、装置、计算机设备及存储介质
CN116823954B (zh) 铰接式车辆的位姿估计方法、装置、车辆及存储介质
CN119169073A (zh) 一种锚孔定位方法及系统
CN117555911A (zh) 训练模型和检测路面要素变更的方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18854884

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 05.08.2020)

122 Ep: pct application non-entry in european phase

Ref document number: 18854884

Country of ref document: EP

Kind code of ref document: A1