CN107730551B - The method and apparatus that in-vehicle camera posture is estimated automatically - Google Patents

The method and apparatus that in-vehicle camera posture is estimated automatically Download PDF

Info

Publication number
CN107730551B
CN107730551B CN201711058488.3A CN201711058488A CN107730551B CN 107730551 B CN107730551 B CN 107730551B CN 201711058488 A CN201711058488 A CN 201711058488A CN 107730551 B CN107730551 B CN 107730551B
Authority
CN
China
Prior art keywords
matrix
vehicle camera
information
road surface
transformation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201711058488.3A
Other languages
Chinese (zh)
Other versions
CN107730551A (en
Inventor
冯煜普
李深
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Volkswagen Mobvoi Beijing Information Technology Co Ltd
Original Assignee
Volkswagen Mobvoi Beijing Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Volkswagen Mobvoi Beijing Information Technology Co Ltd filed Critical Volkswagen Mobvoi Beijing Information Technology Co Ltd
Publication of CN107730551A publication Critical patent/CN107730551A/en
Application granted granted Critical
Publication of CN107730551B publication Critical patent/CN107730551B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose

Abstract

The invention discloses a kind of method and apparatus that in-vehicle camera posture is estimated automatically, are related to automobile technical field, are able to solve the problem of can not estimating in-vehicle camera location information (i.e. posture) automatically in the prior art.Method of the invention, which specifically includes that, obtains the two frame pictures that in-vehicle camera is shot within a preset period of time；According to the travel speed of the internal reference information of the in-vehicle camera, the Feature Points Matching information of the two frames picture and the affiliated vehicle of the in-vehicle camera in the preset time period, the two frames picture opposite actual displacement matrix and practical spin matrix are calculated；It is calculated based on Feature Points Matching information of the preset algorithm to road surface region in the two frames picture, obtains the homography matrix on road surface；The outer ginseng information of the in-vehicle camera is calculated according to the actual displacement matrix, the practical spin matrix and the homography matrix.The present invention is mainly suitable in the scene based on in-vehicle camera posture analysis traffic condition.

Description

The method and apparatus that in-vehicle camera posture is estimated automatically
Technical field
The present invention relates to automobile technical fields, more particularly to a kind of method that in-vehicle camera posture is estimated automatically and dress It sets.
Background technique
With the improvement of people ' s living standards and the accelerating rhythm of life, vehicle is more more and more universal, the pressure of communications and transportation Also therefore unprecedented to increase, the safety problem of car steering is increasingly becoming focus concerned by people.
In order to improve automobile driving safe, a kind of auxiliary driver knows vehicle periphery information so that driver is preparatory The advanced driving assistance system (Advanced Driver AssistantSystem, ADAS) caused danger is avoided to come into being. In-vehicle camera is provided in ADAS, for shooting vehicle periphery information, so as to based on the preassembled position letter of in-vehicle camera Breath (i.e. the posture of in-vehicle camera) picture of shooting is analyzed, obtain with the spacing of front vehicles, whether run-off-road, with And the information such as upcoming traffic security situation.However during driving, the location information of in-vehicle camera is inevitably because of such as vapour Vehicle vibration or user touch the reason of and change, but the prior art can only know in-vehicle camera in advance installation when institute The location information at place, and can not automatically estimate the location information of in-vehicle camera in real time when driving, and then can be because of institute The location information inaccuracy of the in-vehicle camera that uses and lead to the spacing of measurement, the deviation situation that analyzes, analyze The information such as upcoming traffic security situation inaccuracy.Therefore, how to estimate that the location information of in-vehicle camera is urgently to be resolved automatically.
Summary of the invention
In view of this, the method and apparatus that in-vehicle camera posture provided by the invention is estimated automatically, are able to solve existing skill The problem of can not estimating in-vehicle camera location information (i.e. posture) in art automatically.
The purpose of the present invention is what is realized using following technical scheme:
In a first aspect, this method includes following step the present invention provides a kind of method that in-vehicle camera posture is estimated automatically It is rapid:
Obtain two frame pictures in a period of time of interior camera shooting；
Extracting and matching feature points are carried out to two frame pictures, obtain the front and back matched two groups of points of two frames；
According to the internal reference information of camera, the eigenmatrix of two frame pictures is calculated；
The opposite unit displacement transformation and rotation transformation of two frame pictures is obtained according to calculated result；
The automobile real-time speed of acquisition；
It is obtained according to the opposite unit displacement transformation of the automobile real-time speed and the two frames picture and rotation transformation Real displacement transformation and rotation transformation in the time interval of two frame pictures；
The corresponding point matching information of two frame Region Of Interests of front and back is obtained using optical flow algorithm；
Using RANSAC method and then obtain the homography matrix on road surface；
The normal direction on road surface is calculated by the corresponding equation between homography matrix and spin matrix and displacement transformation matrix The coordinate under camera coordinates is measured, to obtain outer ginseng matrix；
According to the outer ginseng matrix adjust automatically.
Second aspect, the present invention provides a kind of method that in-vehicle camera posture is estimated automatically, this method includes following step It is rapid:
Obtain the continuous multiframe picture in a period of time of interior camera shooting；
Operation is filtered to continuous multiframe picture；
According to the internal reference information of camera, the eigenmatrix of multiframe picture is calculated；
The opposite unit displacement transformation and rotation transformation of multiframe picture is obtained according to calculated result；
The automobile real-time speed of acquisition；
It is obtained according to the opposite unit displacement transformation of the automobile real-time speed and the multiframe picture and rotation transformation Real displacement transformation and rotation transformation in the time interval of two frame pictures；
The corresponding point matching information of front and back multiframe Region Of Interest is obtained using optical flow algorithm；
Using RANSAC method and then obtain the homography matrix on road surface；
The normal direction on road surface is calculated by the corresponding equation between homography matrix and spin matrix and displacement transformation matrix The coordinate under camera coordinates is measured, to obtain outer ginseng matrix；
According to the outer ginseng matrix adjust automatically.
The third aspect, the present invention provides a kind of device that in-vehicle camera posture is estimated automatically, which includes following dress It sets:
Module is obtained, the two frame pictures in a period of time for obtaining interior camera shooting；
Characteristic extracting module obtains front and back two frames matched two for carrying out extracting and matching feature points to two frame pictures Group point；
Assertive evidence matrix computing module calculates the eigenmatrix of two frame pictures for the internal reference information according to camera；
First transformation calculations module, for obtaining the opposite unit displacement transformation and rotation of two frame pictures according to calculated result Transformation；
Speed acquiring module, the automobile real-time speed for acquisition；
Second transformation calculations module, for the unit displacement opposite according to the automobile real-time speed and the two frames picture Transformation and rotation transformation obtain real displacement transformation and rotation transformation in the time interval of two frame pictures；
Match information computing module obtains the corresponding point matching letter of two frame Region Of Interests of front and back for application optical flow algorithm Breath；
Homography matrix computing module, for obtaining the homography matrix on road surface in turn using RANSAC method；
Outer ginseng matrix computing module, for passing through the correspondence between homography matrix and spin matrix and displacement transformation matrix Equation calculation goes out coordinate of the normal vector on road surface under camera coordinates, to obtain outer ginseng matrix；
Module is adjusted, for according to the outer ginseng matrix adjust automatically.
Fourth aspect, the present invention provides a kind of methods that in-vehicle camera posture is estimated automatically, which comprises
Obtain the two frame pictures that in-vehicle camera is shot within a preset period of time；
According to the internal reference information of the in-vehicle camera, the Feature Points Matching information of the two frames picture and the vehicle-mounted phase Travel speed of the affiliated vehicle of machine in the preset time period calculates the two frames picture opposite actual displacement matrix and reality Border spin matrix；
It is calculated based on Feature Points Matching information of the preset algorithm to road surface region in the two frames picture, obtains road surface Homography matrix；
It is calculated according to the actual displacement matrix, the practical spin matrix and the homography matrix described vehicle-mounted The outer ginseng information of camera.
5th aspect, the present invention provides a kind of device that in-vehicle camera posture is estimated automatically, described device includes:
Acquiring unit, the two frame pictures shot within a preset period of time for obtaining in-vehicle camera；
First computing unit, for internal reference information, the Feature Points Matching of the two frames picture according to the in-vehicle camera It is opposite to calculate the two frames picture for the travel speed of information and the affiliated vehicle of the in-vehicle camera in the preset time period Actual displacement matrix and practical spin matrix；
Second computing unit, for the Feature Points Matching information based on preset algorithm to road surface region in the two frames picture It is calculated, obtains the homography matrix on road surface；
Third computing unit, for described answering according to the actual displacement matrix, the practical spin matrix and singly square The outer ginseng information of the in-vehicle camera is calculated in battle array.
5th aspect, the present invention provides a kind of storage medium, the storage medium is stored with a plurality of instruction, described instruction Suitable for as processor loads and it is automatic to execute the in-vehicle camera posture as described in first aspect or second aspect or fourth aspect The method of estimation.
6th aspect, the present invention provides a kind of device that in-vehicle camera posture is estimated automatically, described device includes storage Medium and processor；
The processor is adapted for carrying out each instruction；
The storage medium is suitable for storing a plurality of instruction；
Described instruction is suitable for being loaded as the processor and being executed as described in first aspect or second aspect or fourth aspect The method estimated automatically of in-vehicle camera posture.
By above-mentioned technical proposal, the method and apparatus that in-vehicle camera posture provided by the invention is estimated automatically, Neng Goutong Cross the internal reference information of in-vehicle camera, the Feature Points Matching information for the two frame pictures that the in-vehicle camera is shot within a preset period of time with And travel speed of the vehicle in the preset time period, calculate the opposite actual displacement matrix of two frame pictures and practical spin moment Battle array, the characteristic point information for being then based on road surface region in preset algorithm and two frame pictures obtain the homography matrix on road surface, finally The outer ginseng information of in-vehicle camera is estimated according to the automation of actual displacement matrix, practical spin matrix and homography matrix, i.e., it is vehicle-mounted Camera, and then can be based on the location information of correct in-vehicle camera, so that the traffic analyzed with respect to the location information on road surface The accuracy of situation is improved.
The above description is only an overview of the technical scheme of the present invention, in order to better understand the technical means of the present invention, And it can be implemented in accordance with the contents of the specification, and in order to allow above and other objects of the present invention, feature and advantage can It is clearer and more comprehensible, the followings are specific embodiments of the present invention.
Detailed description of the invention
By reading the following detailed description of the preferred embodiment, various other advantages and benefits are common for this field Technical staff will become clear.The drawings are only for the purpose of illustrating a preferred embodiment, and is not considered as to the present invention Limitation.And throughout the drawings, the same reference numbers will be used to refer to the same parts.In the accompanying drawings:
Fig. 1 shows a kind of flow chart of method that in-vehicle camera posture is estimated automatically provided in an embodiment of the present invention；
Fig. 2 shows a kind of composition block diagrams for the device that in-vehicle camera posture is estimated automatically provided in an embodiment of the present invention；
Fig. 3 shows the flow chart for the method that another in-vehicle camera posture provided in an embodiment of the present invention is estimated automatically；
Fig. 4 shows the flow chart for the method that another in-vehicle camera posture provided in an embodiment of the present invention is estimated automatically；
Fig. 5 shows the flow chart for the method that another in-vehicle camera posture provided in an embodiment of the present invention is estimated automatically；
Fig. 6 shows the composition frame for the device that another in-vehicle camera posture provided in an embodiment of the present invention is estimated automatically Figure；
Fig. 7 shows the composition frame for the device that another in-vehicle camera posture provided in an embodiment of the present invention is estimated automatically Figure；
Fig. 8 shows a kind of system architecture diagram that in-vehicle camera posture is estimated automatically provided in an embodiment of the present invention.
Specific embodiment
Exemplary embodiments of the present disclosure are described in more detail below with reference to accompanying drawings.Although showing the disclosure in attached drawing Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here It is limited.On the contrary, these embodiments are provided to facilitate a more thoroughly understanding of the present invention, and can be by the scope of the present disclosure It is fully disclosed to those skilled in the art.
Present invention is further described in detail with reference to the accompanying drawing.
Fig. 1 is the flow diagram for the method that the in-vehicle camera posture of one embodiment of the invention is estimated automatically.
Method includes the following steps:
Step S111 obtains two frame pictures in a period of time of interior camera shooting；
Step S112 carries out extracting and matching feature points to two frame pictures, obtains the front and back matched two groups of points of two frames；
Step S113 calculates the eigenmatrix of two frame pictures according to the internal reference information of camera；
Step S114 obtains the opposite unit displacement transformation and rotation transformation of two frame pictures according to calculated result；
Step S115, the automobile real-time speed of acquisition；
Step S116 is converted according to the opposite unit displacement of the automobile real-time speed and the two frames picture and rotation is become Get real displacement transformation and rotation transformation in the time interval of two frame pictures in return；
Step S117 obtains the corresponding point matching information of two frame Region Of Interests of front and back using optical flow algorithm；
Step S118 using RANSAC method and then obtains the homography matrix on road surface；
Step S119 calculates outlet by the corresponding equation between homography matrix and spin matrix and displacement transformation matrix Coordinate of the normal vector in face under camera coordinates, to obtain outer ginseng matrix；
Step S110, according to the outer ginseng matrix adjust automatically.
Equipment required for the present invention include the outer front of fixation vehicle of front shooting in the car monocular cam or More mesh cameras and therewith by data line be connected the embedded microprocessor for operation be connected with automotive bus system Speed acquiring module.
1) extracting and matching feature points are carried out by the two frame picture of front and back shot to camera, obtains front and back two frames matching Two groups of points.According to the internal reference information of camera, the direct eigenmatrix of two frames can be calculated, and then obtain two frame picture phases Pair unit displacement transformation and rotation transformation.By the automobile real-time speed obtained by speed acquiring module, it is available Real displacement transformation and rotation transformation in the time of two frames.
2) during automobile is advanced, the photo of camera shooting should be largely road surface.It can be with using optical flow algorithm Obtain the corresponding point matching information of two frame Region Of Interests of front and back.Square is singly answered using what RANSAC method can obtain road surface in turn Battle array.
3) road surface can be calculated by the corresponding equation between homography matrix and spin matrix and displacement transformation matrix Coordinate of the normal vector under camera coordinates, to obtain outer ginseng matrix.
4) from the outer ginseng information obtained between single two frame of front and back may due to noise influence and there is deviation, to even Continuous multiframe picture, which is filtered operation, can make the outer ginseng information obtained converge to a stable true scope.
Further, according to still another embodiment of the invention, wherein method includes the following steps:
Step S211 obtains the continuous multiframe picture in a period of time of interior camera shooting；
Step S212 is filtered operation to continuous multiframe picture；
Step S213 calculates the eigenmatrix of multiframe picture according to the internal reference information of camera；
Step S214 obtains the opposite unit displacement transformation and rotation transformation of multiframe picture according to calculated result.
Step S215, the automobile real-time speed of acquisition；
Step S216 is converted according to the opposite unit displacement of the automobile real-time speed and the multiframe picture and rotation is become Get real displacement transformation and rotation transformation in the time interval of two frame pictures in return.
Step S217 obtains the corresponding point matching information of front and back multiframe Region Of Interest using optical flow algorithm；
Step S218 using RANSAC method and then obtains the homography matrix on road surface；
Step S219 calculates outlet by the corresponding equation between homography matrix and spin matrix and displacement transformation matrix Coordinate of the normal vector in face under camera coordinates, to obtain outer ginseng matrix；
Step S220, according to the outer ginseng matrix adjust automatically.
Compared with prior art, the invention has the following advantages that in the prior art, in installation ADAS driving ancillary equipment In the process, user must carry out the multinomial aligning step to camera, to guarantee the accuracy of various functions.And in the mistake of driving Cheng Zhong, the position of camera and angle inevitably change because of the reason of such as thump or user are touched, in turn Lead to the inaccuracy of testing result.Existing technology should obtain with pole geometry from the mode of light stream or image flame detection outer according to list Ginseng.The present invention has broken the above-mentioned inertial thinking of those skilled in the art, and can be realized following effect: obtaining interior camera shooting Two frame pictures in a period of time of head shooting；Extracting and matching feature points are carried out to two frame pictures, obtain front and back two frames matching Two groups of points；According to the internal reference information of camera, the eigenmatrix of two frame pictures is calculated；Two frame figures are obtained according to calculated result Piece opposite unit displacement transformation and rotation transformation.The automobile real-time speed of acquisition；According to the automobile real-time speed and described The opposite unit displacement transformation of two frame pictures and rotation transformation obtain in the time interval of two frame pictures real displacement transformation and Rotation transformation.The corresponding point matching information of two frame Region Of Interests of front and back is obtained using optical flow algorithm；In turn using RANSAC method Obtain the homography matrix on road surface；It is calculated by the corresponding equation between homography matrix and spin matrix and displacement transformation matrix Coordinate of the normal vector on road surface under camera coordinates, to obtain outer ginseng matrix；According to the outer ginseng matrix adjust automatically.It avoids The trouble that user will be calibrated often.
Further, the structural representation for the device that Fig. 2 estimates automatically for the in-vehicle camera posture of one embodiment of the invention Figure.
The device includes following device:
Module 10 is obtained, the two frame pictures in a period of time for obtaining interior camera shooting；
It is matched to obtain two frames of front and back for carrying out extracting and matching feature points to two frame pictures for characteristic extracting module 20 Two groups of points；
Assertive evidence matrix computing module 30 calculates the eigenmatrix of two frame pictures for the internal reference information according to camera；
First transformation calculations module 40, for obtaining the opposite unit displacement transformation and rotation of two frame pictures according to calculated result Transformation is changed.
Speed acquiring module 50, the automobile real-time speed for acquisition；
Second transformation calculations module 60, for the unit position opposite according to the automobile real-time speed and the two frames picture It moves transformation and rotation transformation obtains real displacement transformation and rotation transformation in the time interval of two frame pictures.
Match information computing module 70 obtains the corresponding point matching letter of two frame Region Of Interests of front and back for application optical flow algorithm Breath；
Homography matrix computing module 80, for obtaining the homography matrix on road surface in turn using RANSAC method；
Outer ginseng matrix computing module 90, for passing through pair between homography matrix and spin matrix and displacement transformation matrix Equation calculation is answered to go out coordinate of the normal vector on road surface under camera coordinates, to obtain outer ginseng matrix；
Module 100 is adjusted, for according to the outer ginseng matrix adjust automatically.
Compared with prior art, the invention has the following advantages that in the prior art, in installation ADAS driving ancillary equipment In the process, user must carry out the multinomial aligning step to camera, to guarantee the accuracy of various functions.And in the mistake of driving Cheng Zhong, the position of camera and angle inevitably change because of the reason of such as thump or user are touched, in turn Lead to the inaccuracy of testing result.Existing technology should obtain with pole geometry from the mode of light stream or image flame detection outer according to list Ginseng.The present invention has broken the above-mentioned inertial thinking of those skilled in the art, and can be realized following effect: obtaining interior camera shooting Two frame pictures in a period of time of head shooting；Extracting and matching feature points are carried out to two frame pictures, obtain front and back two frames matching Two groups of points；According to the internal reference information of camera, the eigenmatrix of two frame pictures is calculated；Two frame figures are obtained according to calculated result Piece opposite unit displacement transformation and rotation transformation.The automobile real-time speed of acquisition；According to the automobile real-time speed and described The opposite unit displacement transformation of two frame pictures and rotation transformation obtain in the time interval of two frame pictures real displacement transformation and Rotation transformation.The corresponding point matching information of two frame Region Of Interests of front and back is obtained using optical flow algorithm；In turn using RANSAC method Obtain the homography matrix on road surface；It is calculated by the corresponding equation between homography matrix and spin matrix and displacement transformation matrix Coordinate of the normal vector on road surface under camera coordinates, to obtain outer ginseng matrix；According to the outer ginseng matrix adjust automatically.It avoids The trouble that user will be calibrated often.
Further, the embodiment of the invention provides a kind of methods that in-vehicle camera posture is estimated automatically, as shown in figure 3, The method specifically includes that
101, the two frame pictures that in-vehicle camera is shot within a preset period of time are obtained.
Wherein, the in-vehicle camera in this step is forward sight camera, for acquiring the information of vehicle front, such as be can be solid It is scheduled on the monocular cam or more mesh cameras in the outer front of shooting vehicle in front of car.In order to timely learning in-vehicle camera phase The location information of road pavement, the i.e. outer ginseng information of in-vehicle camera, can periodically obtain in-vehicle camera within a preset period of time Two frame pictures of shooting, then analyze this two frames picture, therefrom obtain the outer ginseng information of in-vehicle camera.Wherein, it presets The state of vehicle driving is more stable in period, when choosing picture, can choose two adjacent within a preset period of time frame figures Piece can also choose two frame pictures within a preset period of time with certain intervals.
102, according to the internal reference information of the in-vehicle camera, the Feature Points Matching information of the two frames picture and the vehicle Travel speed of the affiliated vehicle of camera in the preset time period is carried, the opposite actual displacement matrix of the two frames picture is calculated With practical spin matrix.
Wherein, the internal reference information of in-vehicle camera is used to describe the characteristic information of camera internal, is a kind of from camera coordinates system To the transformation of photo coordinate system.After obtaining two frame pictures, extracting and matching feature points can be carried out to the two frames picture Operation, to obtain Feature Points Matching information, the internal reference information and vehicle for then obtaining in-vehicle camera are in the preset time period Travel speed, and according to internal reference information, Feature Points Matching information and travel speed these three information, calculate in-vehicle camera this two Misalignment and rotational case in frame picture institute interval time.
It should be noted that when the travel speed of the affiliated vehicle of in-vehicle camera within a preset period of time changes, this step Travel speed in rapid can be average overall travel speed of the vehicle in the preset time period.
103, it is calculated, is obtained based on Feature Points Matching information of the preset algorithm to road surface region in the two frames picture The homography matrix on road surface.
In-vehicle camera corresponds to the location information on road surface in order to obtain, it is also necessary to know the road surface feelings in vehicle travel process Condition needs to obtain the homography matrix in two frame pictures about road surface.Specifically, can first obtain from picture about road surface Then point corresponds the point in two frame pictures about road surface, to obtain homography matrix.
104, it is calculated according to the actual displacement matrix, the practical spin matrix and the homography matrix described The outer ginseng information of in-vehicle camera.
Wherein, outer ginseng information is for describing location information of the in-vehicle camera with respect to road surface, is a kind of camera coordinates system to generation The transformation of boundary's coordinate system.Therefore, in order to obtain the outer ginseng information of in-vehicle camera, the actual displacement square of in-vehicle camera can be passed through Corresponding equation between battle array, practical spin matrix and the homography matrix on road surface obtains the normal vector on road surface under camera coordinates system The coordinate and is determined as outer ginseng information with respect to the coordinate of road surface (i.e. world coordinate system) by coordinate, i.e. in-vehicle camera.
The method that in-vehicle camera posture provided in an embodiment of the present invention is estimated automatically can be believed by the internal reference of in-vehicle camera The Feature Points Matching information and vehicle for the two frame pictures that breath, the in-vehicle camera are shot within a preset period of time are in the preset time Travel speed in section, calculates the opposite actual displacement matrix of two frame pictures and practical spin matrix, is then based on pre- imputation In method and two frame pictures road surface region characteristic point information obtain road surface homography matrix, finally according to actual displacement matrix, Practical spin matrix and homography matrix automation estimate the outer ginseng information of in-vehicle camera, i.e. position of the in-vehicle camera with respect to road surface Information, and then can be based on the location information of correct in-vehicle camera, so that the accuracy of the traffic conditions analyzed is mentioned It is high.
Further, according to method shown in Fig. 3, another embodiment of the invention additionally provides a kind of in-vehicle camera appearance The method that state is estimated automatically, as shown in figure 4, this method specifically includes that
201, the two frame pictures that in-vehicle camera is shot within a preset period of time are obtained.
The specific implementation of this step is identical as above-mentioned steps 101, and details are not described herein.
202, according to the internal reference information of the in-vehicle camera, the Feature Points Matching information of the two frames picture and the vehicle Travel speed of the affiliated vehicle of camera in the preset time period is carried, the opposite actual displacement matrix of the two frames picture is calculated With practical spin matrix.
It obtains the opposite actual displacement matrix of two frame pictures and practical spin matrix specific implementation can be first to obtain The unit displacement matrix and practical spin matrix of vehicle obtain actual displacement matrix then in conjunction with Vehicle Speed.Below It is described in detail:
(1) it is calculated according to the Feature Points Matching information of the internal reference information of the in-vehicle camera and the two frames picture The opposite unit displacement matrix of the two frames picture and practical spin matrix.
Specifically, after obtaining two frame pictures to be analyzed, can first to the two frames picture carry out feature point extraction and Matching operation obtains Feature Points Matching information, is then calculated according to the internal reference information and the Feature Points Matching information The eigenmatrix opposite to the two frames picture, finally to the eigenmatrix decomposed to obtain the unit displacement matrix and The practical spin matrix.
Wherein, the characteristic point in two frame pictures may be the same or different, can be depending on specific pictorial information. Eigenmatrix is the matrix for describing two frame pictures feature itself.
(2) travel speed of the vehicle in the preset time period is obtained；According to the travel speed, the unit The actual displacement matrix is calculated in transposed matrix.
After the opposite unit displacement matrix of the travel speed, two frame pictures that obtain vehicle, by unit displacement matrix and vehicle Travel speed be multiplied, actual displacement matrix can be obtained.
203, it is calculated, is obtained based on Feature Points Matching information of the preset algorithm to road surface region in the two frames picture The homography matrix on road surface.
Specifically, can calculate first with optical flow algorithm the road surface region in the two frames picture, obtain described The Feature Points Matching information in road surface region in two frame pictures is then based on consistent (the RANdom Sample of random sampling Consensus, RANSAC) algorithm calculates the Feature Points Matching information in road surface region in the two frames picture, obtain road The homography matrix in face.
204, it is calculated according to the actual displacement matrix, the practical spin matrix and the homography matrix described The outer ginseng information of in-vehicle camera.
The specific implementation of this step is identical as above-mentioned steps 104, and details are not described herein.In addition, about outer ginseng is calculated The simple process figure of information can be as shown in Figure 5.
In practical applications, may it is too dark, rainy etc. weather due to cause in-vehicle camera acquire picture blur, So as to cause the phenomenon of the outer ginseng information inaccuracy of the in-vehicle camera of calculating, and then the lane for leading to the spacing of measurement, analyzing Deviate the information inaccuracy such as situation, the upcoming traffic security situation analyzed.To solve the above-mentioned problems, the embodiment of the present invention mentions A kind of amendment has been supplied to join the method for information, specific step 205-207 as described below outside.
205, it obtains and joins information outside the history for the history picture that at least one shoots before the two frames picture.
Specifically, can since with the Look-ahead when front cross frame picture shooting time nearest front cross frame history picture, It obtains respectively and joins information outside history corresponding to two different frame history pictures, information is joined to current according to history outside so as to subsequent Outer ginseng information be modified.Wherein, it the picture number that is spaced between two frame history pictures and is spaced when between front cross frame picture Picture number it is identical.That is, two frame history pictures are also adjacent when two current frame pictures are adjacent pictures 's；When being spaced N number of picture between two current frame pictures, it is also spaced N number of picture between two frame history pictures, wherein N is positive Integer.
206, join information outside the history based on acquisition to be modified the outer ginseng information obtained based on the two frames picture, obtain To revised outer ginseng information.
The specific implementation of this step can be with are as follows: to joining information outside the history of the acquisition and be based on the two frames picture Obtained outer ginseng information is weighted processing, obtains the revised outer ginseng information.Wherein, the weighted value of all outer ginseng information The sum of be equal to 1.
Illustratively, if this outer ginseng information obtained is outer ginseng information 1, joining information outside history includes outer ginseng information 2, outside Join information 3 and outer ginseng information 4；Then join outside revised outer ginseng information=outer ginseng information 1*30%+ and joins information outside information 2*30%+ Join information 4*20% outside 3*20%+.
207, that the revised outer ginseng information is determined as the in-vehicle camera is required in the preset time period Final outer ginseng information.
Since the outer ginseng information that single obtains may have error, so after will be by the calculated modification of multiple outer ginseng information Outer ginseng information as in-vehicle camera in the preset time period required final outer ginseng information than outer ginseng information that single obtains It is more accurate.
The method that in-vehicle camera posture provided in an embodiment of the present invention is estimated automatically, can not only be in vehicle travel process The automatic outer ginseng information for obtaining in-vehicle camera, additionally it is possible to the outer ginseng information that single obtains be repaired according to information is joined outside history Just, inaccuracy is calculated because of special circumstances with the outer ginseng information for preventing single from obtaining.
Further, according to above method embodiment, another embodiment of the invention additionally provides a kind of in-vehicle camera The device that posture is estimated automatically, as shown in fig. 6, described device specifically includes that acquiring unit 31, the first computing unit 32, second Computing unit 33 and third computing unit 34.Wherein,
Acquiring unit 31, the two frame pictures shot within a preset period of time for obtaining in-vehicle camera；
First computing unit 32, for internal reference information, the characteristic point of the two frames picture according to the in-vehicle camera Travel speed with information and the affiliated vehicle of the in-vehicle camera in the preset time period calculates the two frames picture phase Pair actual displacement matrix and practical spin matrix；
Second computing unit 33, for being believed based on Feature Points Matching of the preset algorithm to road surface region in the two frames picture Breath is calculated, and the homography matrix on road surface is obtained；
Third computing unit 34, for being answered according to the actual displacement matrix, the practical spin matrix and the list The outer ginseng information of the in-vehicle camera is calculated in matrix.
Further, as shown in fig. 7, first computing unit 32 includes:
First computing module 321, for according to the internal reference information of the in-vehicle camera and the feature of the two frames picture The opposite unit displacement matrix of the two frames picture and practical spin matrix is calculated in point match information；
Module 322 is obtained, for obtaining travel speed of the vehicle in the preset time period；
Second computing module 323, for the reality to be calculated according to the travel speed, the unit displacement matrix Transposed matrix.
Further, first computing module 321 is used to carry out extracting and matching feature points behaviour to the two frames picture Make, obtains Feature Points Matching information；Two frame is calculated according to the internal reference information and the Feature Points Matching information The opposite eigenmatrix of picture；The eigenmatrix is decomposed to obtain the unit displacement matrix and the practical spin moment Battle array.
Further, second computing module 323 is used for using optical flow algorithm to the road surface area in the two frames picture Domain is calculated, and the Feature Points Matching information in road surface region in the two frames picture is obtained；Based on the consistent RANSAC of random sampling Algorithm calculates the Feature Points Matching information in road surface region in the two frames picture, obtains the homography matrix on road surface.
Further, the third computing unit 34 is for passing through the actual displacement matrix, the practical spin matrix And the corresponding equation between the homography matrix obtains coordinate of the normal vector on the road surface under the camera coordinates system；It will The coordinate is determined as the outer ginseng information.
Further, the acquiring unit 31, is also used to obtain that at least one shoots before the two frames picture goes through Join information outside the history of history picture；
As shown in fig. 7, described device further include:
Amending unit 35, for joining information outside the history based on acquisition to the outer ginseng information obtained based on the two frames picture It is modified, obtains revised outer ginseng information；
Determination unit 36, for the revised outer ginseng information to be determined as the in-vehicle camera in the preset time Required final outer ginseng information in section.
Further, the amending unit 35 is used to join information outside the history to the acquisition and is based on the two frames picture Obtained outer ginseng information is weighted processing, obtains the revised outer ginseng information.
The device that in-vehicle camera posture provided in an embodiment of the present invention is estimated automatically can be believed by the internal reference of in-vehicle camera The Feature Points Matching information and vehicle for the two frame pictures that breath, the in-vehicle camera are shot within a preset period of time are in the preset time Travel speed in section, calculates the opposite actual displacement matrix of two frame pictures and practical spin matrix, is then based on pre- imputation In method and two frame pictures road surface region characteristic point information obtain road surface homography matrix, finally according to actual displacement matrix, Practical spin matrix and homography matrix automation estimate the outer ginseng information of in-vehicle camera, i.e. position of the in-vehicle camera with respect to road surface Information, and then can be based on the location information of correct in-vehicle camera, so that the accuracy of the traffic conditions analyzed is mentioned It is high.
Further, according to above method embodiment, another embodiment of the invention additionally provides a kind of storage medium, The storage medium is stored with a plurality of instruction, and described instruction is suitable for being loaded by processor and executing in-vehicle camera as described above The method that posture is estimated automatically.
The instruction stored in the storage medium that in-vehicle camera posture provided in an embodiment of the present invention is estimated automatically, can pass through The Feature Points Matching information for the two frame pictures that the internal reference information of in-vehicle camera, the in-vehicle camera are shot within a preset period of time and Travel speed of the vehicle in the preset time period calculates the opposite actual displacement matrix of two frame pictures and practical spin moment Battle array, the characteristic point information for being then based on road surface region in preset algorithm and two frame pictures obtain the homography matrix on road surface, finally The outer ginseng information of in-vehicle camera is estimated according to the automation of actual displacement matrix, practical spin matrix and homography matrix, i.e., it is vehicle-mounted Camera, and then can be based on the location information of correct in-vehicle camera, so that the traffic analyzed with respect to the location information on road surface The accuracy of situation is improved.
Further, according to above method embodiment, another embodiment of the invention additionally provides a kind of in-vehicle camera The device that posture is estimated automatically, described device include storage medium and processor；
The processor is adapted for carrying out each instruction；
The storage medium is suitable for storing a plurality of instruction；
Described instruction is suitable for being loaded by the processor and being executed the side that in-vehicle camera posture as described above is estimated automatically Method.
The device that in-vehicle camera posture provided in an embodiment of the present invention is estimated automatically can be believed by the internal reference of in-vehicle camera The Feature Points Matching information and vehicle for the two frame pictures that breath, the in-vehicle camera are shot within a preset period of time are in the preset time Travel speed in section, calculates the opposite actual displacement matrix of two frame pictures and practical spin matrix, is then based on pre- imputation In method and two frame pictures road surface region characteristic point information obtain road surface homography matrix, finally according to actual displacement matrix, Practical spin matrix and homography matrix automation estimate the outer ginseng information of in-vehicle camera, i.e. position of the in-vehicle camera with respect to road surface Information, and then can be based on the location information of correct in-vehicle camera, so that the accuracy of the traffic conditions analyzed is mentioned It is high.
Further, another embodiment of the invention additionally provides a kind of system that in-vehicle camera posture is estimated automatically, The system includes in-vehicle camera, processor, storage medium and automotive bus system, as shown in figure 8, storage medium is at storage The instruction that device executes is managed, processor is for obtaining the two frame pictures and in-vehicle camera that in-vehicle camera is shot within a preset period of time Internal reference information, automotive bus system detection the travel speed in the preset time period, then according to the internal reference believe The Feature Points Matching information and the travel speed of breath, the two frames picture calculate the opposite actual bit of the two frames picture Move matrix and practical spin matrix, and based on preset algorithm to the Feature Points Matching information in road surface region in the two frames picture into Row calculates, and the homography matrix on road surface is obtained, finally according to the actual displacement matrix, the practical spin matrix and the list Answer matrix that the outer ginseng information of the in-vehicle camera is calculated.
The embodiment of the invention also discloses:
A1, a kind of method that in-vehicle camera posture is estimated automatically, method includes the following steps:
Obtain two frame pictures in a period of time of interior camera shooting；
Extracting and matching feature points are carried out to two frame pictures, obtain the front and back matched two groups of points of two frames；
According to the internal reference information of camera, the eigenmatrix of two frame pictures is calculated；
The opposite unit displacement transformation and rotation transformation of two frame pictures is obtained according to calculated result；
The automobile real-time speed of acquisition；
It is obtained according to the opposite unit displacement transformation of the automobile real-time speed and the two frames picture and rotation transformation Real displacement transformation and rotation transformation in the time interval of two frame pictures.
The corresponding point matching information of two frame Region Of Interests of front and back is obtained using optical flow algorithm；
Using RANSAC method and then obtain the homography matrix on road surface；
The normal direction on road surface is calculated by the corresponding equation between homography matrix and spin matrix and displacement transformation matrix The coordinate under camera coordinates is measured, to obtain outer ginseng matrix.
According to the outer ginseng matrix adjust automatically.
B2, a kind of method that in-vehicle camera posture is estimated automatically, method includes the following steps:
Obtain the continuous multiframe picture in a period of time of interior camera shooting；
Operation is filtered to continuous multiframe picture；
According to the internal reference information of camera, the eigenmatrix of multiframe picture is calculated；
The opposite unit displacement transformation and rotation transformation of multiframe picture is obtained according to calculated result.
The automobile real-time speed of acquisition；
It is obtained according to the opposite unit displacement transformation of the automobile real-time speed and the multiframe picture and rotation transformation Real displacement transformation and rotation transformation in the time interval of two frame pictures；
The corresponding point matching information of front and back multiframe Region Of Interest is obtained using optical flow algorithm；
Using RANSAC method and then obtain the homography matrix on road surface；
The normal direction on road surface is calculated by the corresponding equation between homography matrix and spin matrix and displacement transformation matrix The coordinate under camera coordinates is measured, to obtain outer ginseng matrix；
According to the outer ginseng matrix adjust automatically.
C3, a kind of device that in-vehicle camera posture is estimated automatically, which includes following device:
Module is obtained, the two frame pictures in a period of time for obtaining interior camera shooting；
Characteristic extracting module obtains front and back two frames matched two for carrying out extracting and matching feature points to two frame pictures Group point；
Assertive evidence matrix computing module calculates the eigenmatrix of two frame pictures for the internal reference information according to camera；
First transformation calculations module, for obtaining the opposite unit displacement transformation and rotation of two frame pictures according to calculated result Transformation.
Speed acquiring module, the automobile real-time speed for acquisition；
Second transformation calculations module, for the unit displacement opposite according to the automobile real-time speed and the two frames picture Transformation and rotation transformation obtain real displacement transformation and rotation transformation in the time interval of two frame pictures；
Match information computing module obtains the corresponding point matching letter of two frame Region Of Interests of front and back for application optical flow algorithm Breath；
Homography matrix computing module, for obtaining the homography matrix on road surface in turn using RANSAC method；
Outer ginseng matrix computing module, for passing through the correspondence between homography matrix and spin matrix and displacement transformation matrix Equation calculation goes out coordinate of the normal vector on road surface under camera coordinates, to obtain outer ginseng matrix；
Module is adjusted, for according to the outer ginseng matrix adjust automatically.
D4, a kind of method that in-vehicle camera posture is estimated automatically, which comprises
Obtain the two frame pictures that in-vehicle camera is shot within a preset period of time；
According to the internal reference information of the in-vehicle camera, the Feature Points Matching information of the two frames picture and the vehicle-mounted phase Travel speed of the affiliated vehicle of machine in the preset time period calculates the two frames picture opposite actual displacement matrix and reality Border spin matrix；
It is calculated based on Feature Points Matching information of the preset algorithm to road surface region in the two frames picture, obtains road surface Homography matrix；
It is calculated according to the actual displacement matrix, the practical spin matrix and the homography matrix described vehicle-mounted The outer ginseng information of camera.
D5, the method according to D4, the internal reference information according to the in-vehicle camera, the feature of the two frames picture The travel speed of point match information and the affiliated vehicle of the in-vehicle camera in the preset time period, calculates the two frames figure The opposite actual displacement matrix of piece and practical spin matrix include:
Institute is calculated according to the Feature Points Matching information of the internal reference information of the in-vehicle camera and the two frames picture State the opposite unit displacement matrix of two frame pictures and practical spin matrix；
Obtain travel speed of the vehicle in the preset time period；
The actual displacement matrix is calculated according to the travel speed, the unit displacement matrix.
D6, the method according to D5, the internal reference information according to the in-vehicle camera and the two frames picture The opposite unit displacement matrix of the two frames picture is calculated in Feature Points Matching information and practical spin matrix includes:
Extracting and matching feature points operation is carried out to the two frames picture, obtains Feature Points Matching information；
Opposite intrinsic of the two frames picture is calculated according to the internal reference information and the Feature Points Matching information Matrix；
The eigenmatrix is decomposed to obtain the unit displacement matrix and the practical spin matrix.
D7, the method according to D4, it is described based on preset algorithm to the characteristic point in road surface region in the two frames picture Match information is calculated, and the homography matrix for obtaining road surface includes:
The road surface region in the two frames picture is calculated using optical flow algorithm, obtains road surface in the two frames picture The Feature Points Matching information in region；
Based on the consistent RANSAC algorithm of random sampling to the Feature Points Matching information in road surface region in the two frames picture into Row calculates, and obtains the homography matrix on road surface.
D8, the method according to D4, it is described according to the actual displacement matrix, the practical spin matrix and described The outer ginseng information that the in-vehicle camera is calculated in homography matrix includes:
It is obtained by the corresponding equation between the actual displacement matrix, the practical spin matrix and the homography matrix To the coordinate of the normal vector under the camera coordinates system on the road surface；
The coordinate is determined as the outer ginseng information.
D9, the method according to any one of D4 to D8, the method also includes:
It obtains and joins information outside the history for the history picture that at least one shoots before the two frames picture；
Join information outside history based on acquisition to be modified the outer ginseng information obtained based on the two frames picture, be repaired Outer ginseng information after just；
It is required final in the preset time period that the revised outer ginseng information is determined as the in-vehicle camera Outer ginseng information.
D10, the method according to D9, the history based on acquisition are joined information outside and are obtained to based on the two frames picture Outer ginseng information be modified, obtaining revised outer ginseng information includes:
Processing is weighted to the outer ginseng information joining information outside the history of the acquisition and being obtained based on the two frames picture, Obtain the revised outer ginseng information.
E11, a kind of device that in-vehicle camera posture is estimated automatically, described device include:
Acquiring unit, the two frame pictures shot within a preset period of time for obtaining in-vehicle camera；
First computing unit, for internal reference information, the Feature Points Matching of the two frames picture according to the in-vehicle camera It is opposite to calculate the two frames picture for the travel speed of information and the affiliated vehicle of the in-vehicle camera in the preset time period Actual displacement matrix and practical spin matrix；
Second computing unit, for the Feature Points Matching information based on preset algorithm to road surface region in the two frames picture It is calculated, obtains the homography matrix on road surface；
Third computing unit, for described answering according to the actual displacement matrix, the practical spin matrix and singly square The outer ginseng information of the in-vehicle camera is calculated in battle array.
E12, the device according to E11, first computing unit include:
First computing module, for according to the internal reference information of the in-vehicle camera and the characteristic point of the two frames picture The opposite unit displacement matrix of the two frames picture and practical spin matrix is calculated with information；
Module is obtained, for obtaining travel speed of the vehicle in the preset time period；
Second computing module, for the actual bit to be calculated according to the travel speed, the unit displacement matrix Move matrix.
E13, the device according to E12, first computing module are used to carry out characteristic point to the two frames picture to mention It takes and matching operation, obtains Feature Points Matching information；It is calculated according to the internal reference information and the Feature Points Matching information The eigenmatrix opposite to the two frames picture；Decomposed to obtain the unit displacement matrix and described to the eigenmatrix Practical spin matrix.
E14, the device according to E11, second computing module are used for using optical flow algorithm to the two frames picture In road surface region calculated, obtain the Feature Points Matching information in road surface region in the two frames picture；Based on random sampling Consistent RANSAC algorithm calculates the Feature Points Matching information in road surface region in the two frames picture, and the list for obtaining road surface is answered Matrix.
E15, the device according to E11, the third computing unit is for passing through the actual displacement matrix, the reality Corresponding equation between border spin matrix and the homography matrix obtains the normal vector on the road surface in the camera coordinates system Under coordinate；The coordinate is determined as the outer ginseng information.
E16, the device according to any one of E11 to E15, the acquiring unit, be also used to obtain at least one Join information outside the history of the history picture shot before the two frames picture；
Described device further include:
Amending unit, for join outside the history based on acquisition information to the outer ginseng information obtained based on the two frames picture into Row amendment, obtains revised outer ginseng information；
Determination unit, for the revised outer ginseng information to be determined as the in-vehicle camera in the preset time period Interior required final outer ginseng information.
E17, the device according to E16, the amending unit are used to join information outside the history to the acquisition and be based on The outer ginseng information that the two frames picture obtains is weighted processing, obtains the revised outer ginseng information.
F18, a kind of storage medium, the storage medium are stored with a plurality of instruction, and described instruction is suitable for being added by processor Carry and execute the method that the in-vehicle camera posture as described in any one of A1, B2, D4 to D10 is estimated automatically.
G19, a kind of device that in-vehicle camera posture is estimated automatically, described device includes storage medium and processor；
The processor is adapted for carrying out each instruction；
The storage medium is suitable for storing a plurality of instruction；
Described instruction is suitable for being loaded as the processor and executing the vehicle-mounted phase as described in any one of A1, B2, D4 to D10 The method that machine posture is estimated automatically.
In the above-described embodiments, it all emphasizes particularly on different fields to the description of each embodiment, there is no the portion being described in detail in some embodiment Point, reference can be made to the related descriptions of other embodiments.
It is understood that the correlated characteristic in the above method and device can be referred to mutually.In addition, in above-described embodiment " first ", " second " etc. be and not represent the superiority and inferiority of each embodiment for distinguishing each embodiment.
It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description, The specific work process of device and unit, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
Algorithm and display are not inherently related to any particular computer, virtual system, or other device provided herein. Various general-purpose systems can also be used together with teachings based herein.As described above, it constructs required by this kind of system Structure be obvious.In addition, the present invention is also not directed to any particular programming language.It should be understood that can use various Programming language realizes summary of the invention described herein, and the description done above to language-specific is to disclose this hair Bright preferred forms.
In the instructions provided here, numerous specific details are set forth.It is to be appreciated, however, that implementation of the invention Example can be practiced without these specific details.In some instances, well known method, structure is not been shown in detail And technology, so as not to obscure the understanding of this specification.
Similarly, it should be understood that in order to simplify the disclosure and help to understand one or more of the various inventive aspects, Above in the description of exemplary embodiment of the present invention, each feature of the invention is grouped together into single implementation sometimes In example, figure or descriptions thereof.However, the disclosed method should not be interpreted as reflecting the following intention: i.e. required to protect Shield the present invention claims features more more than feature expressly recited in each claim.More precisely, as following Claims reflect as, inventive aspect is all features less than single embodiment disclosed above.Therefore, Thus the claims for following specific embodiment are expressly incorporated in the specific embodiment, wherein each claim itself All as a separate embodiment of the present invention.
Those skilled in the art will understand that can be carried out adaptively to the module in the equipment in embodiment Change and they are arranged in one or more devices different from this embodiment.It can be the module or list in embodiment Member or component are combined into a module or unit or component, and furthermore they can be divided into multiple submodule or subelement or Sub-component.Other than such feature and/or at least some of process or unit exclude each other, it can use any Combination is to all features disclosed in this specification (including adjoint claim, abstract and attached drawing) and so disclosed All process or units of what method or apparatus are combined.Unless expressly stated otherwise, this specification is (including adjoint power Benefit require, abstract and attached drawing) disclosed in each feature can carry out generation with an alternative feature that provides the same, equivalent, or similar purpose It replaces.
In addition, it will be appreciated by those of skill in the art that although some embodiments described herein include other embodiments In included certain features rather than other feature, but the combination of the feature of different embodiments mean it is of the invention Within the scope of and form different embodiments.For example, in the following claims, embodiment claimed is appointed Meaning one of can in any combination mode come using.
Various component embodiments of the invention can be implemented in hardware, or to run on one or more processors Software module realize, or be implemented in a combination thereof.It will be understood by those of skill in the art that can be used in practice Microprocessor or digital signal processor (DSP) realize what in-vehicle camera posture according to an embodiment of the present invention was estimated automatically The some or all functions of some or all components in method and apparatus.The present invention is also implemented as executing this In described method some or all device or device programs (for example, computer program and computer program Product).It is such to realize that program of the invention can store on a computer-readable medium, it either can have one or more The form of a signal.Such signal can be downloaded from an internet website to obtain, be perhaps provided on the carrier signal or with Any other form provides.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and ability Field technique personnel can be designed alternative embodiment without departing from the scope of the appended claims.In the claims, Any reference symbol between parentheses should not be configured to limitations on claims.Word "comprising" does not exclude the presence of not Element or step listed in the claims.Word "a" or "an" located in front of the element does not exclude the presence of multiple such Element.The present invention can be by means of including the hardware of several different elements and being come by means of properly programmed computer real It is existing.In the unit claims listing several devices, several in these devices can be through the same hardware branch To embody.The use of word first, second, and third does not indicate any sequence.These words can be explained and be run after fame Claim.

Claims (10)

1. a kind of method that in-vehicle camera posture is estimated automatically, which is characterized in that the described method comprises the following steps:
Obtain two frame pictures in a period of time of interior camera shooting；
Extracting and matching feature points are carried out to two frame pictures, obtain the front and back matched two groups of points of two frames；
According to the internal reference information of camera, the eigenmatrix of two frame pictures is calculated；
The opposite unit displacement transformation and opposite rotation transformation of two frame pictures is obtained according to calculated result；
The automobile real-time speed of acquisition；
It is obtained according to the opposite unit displacement transformation of the automobile real-time speed and the two frames picture and opposite rotation transformation Real displacement transformation and true rotation transformation in the time interval of two frame pictures；
The corresponding point matching information of two frame Region Of Interests of front and back is obtained using optical flow algorithm；
Using RANSAC method and then obtain the homography matrix on road surface；
The normal direction on road surface is calculated by the corresponding equation between homography matrix and true rotation transformation and real displacement transformation The coordinate under camera coordinates is measured, to obtain outer ginseng matrix；
According to the outer ginseng matrix adjust automatically.
2. a kind of method that in-vehicle camera posture is estimated automatically, which is characterized in that the described method comprises the following steps:
Obtain the continuous multiframe picture in a period of time of interior camera shooting；
Operation is filtered to continuous multiframe picture；
According to the internal reference information of camera, the eigenmatrix of multiframe picture is calculated；
The opposite unit displacement transformation and opposite rotation transformation of multiframe picture is obtained according to calculated result；
The automobile real-time speed of acquisition；
It is obtained according to the opposite unit displacement transformation of the automobile real-time speed and the multiframe picture and opposite rotation transformation Real displacement transformation and true rotation transformation in the time interval of two frame pictures；
The corresponding point matching information of front and back multiframe Region Of Interest is obtained using optical flow algorithm；
Using RANSAC method and then obtain the homography matrix on road surface；
The normal direction on road surface is calculated by the corresponding equation between homography matrix and true rotation transformation and real displacement transformation The coordinate under camera coordinates is measured, to obtain outer ginseng matrix；
According to the outer ginseng matrix adjust automatically.
3. a kind of device that in-vehicle camera posture is estimated automatically, which is characterized in that the device includes following device:
Module is obtained, the two frame pictures in a period of time for obtaining interior camera shooting；
Characteristic extracting module obtains the front and back matched two groups of points of two frames for carrying out extracting and matching feature points to two frame pictures；
Assertive evidence matrix computing module calculates the eigenmatrix of two frame pictures for the internal reference information according to camera；
First transformation calculations module, for obtaining the opposite unit displacement transformation and opposite rotation of two frame pictures according to calculated result Transformation is changed；
Speed acquiring module, the automobile real-time speed for acquisition；
Second transformation calculations module, for being converted according to the opposite unit displacement of the automobile real-time speed and the two frames picture Real displacement transformation and true rotation transformation in the time interval of two frame pictures are obtained with opposite rotation transformation；
Match information computing module obtains the corresponding point matching information of two frame Region Of Interests of front and back for application optical flow algorithm；
Homography matrix computing module, for obtaining the homography matrix on road surface in turn using RANSAC method；
Outer ginseng matrix computing module, for passing through the correspondence between homography matrix and true rotation transformation and real displacement transformation Equation calculation goes out coordinate of the normal vector on road surface under camera coordinates, to obtain outer ginseng matrix；
Module is adjusted, for according to the outer ginseng matrix adjust automatically.
4. a kind of method that in-vehicle camera posture is estimated automatically, which is characterized in that the described method includes:
Obtain the two frame pictures that in-vehicle camera is shot within a preset period of time；
According to the internal reference information of the in-vehicle camera, the Feature Points Matching information of the two frames picture and in-vehicle camera institute Belong to travel speed of the vehicle in the preset time period, calculates the opposite actual displacement matrix of the two frames picture and practical rotation Torque battle array；
It is calculated based on Feature Points Matching information of the preset algorithm to road surface region in the two frames picture, obtains the list on road surface Answer matrix；
The in-vehicle camera is calculated according to the actual displacement matrix, the practical spin matrix and the homography matrix Outer ginseng information.
5. according to the method described in claim 4, it is characterized in that, the internal reference information according to the in-vehicle camera, described Travel speed of the Feature Points Matching information and the affiliated vehicle of the in-vehicle camera of two frame pictures in the preset time period, It calculates the opposite actual displacement matrix of the two frames picture and practical spin matrix includes:
Described two are calculated according to the Feature Points Matching information of the internal reference information of the in-vehicle camera and the two frames picture The opposite unit displacement matrix of frame picture and practical spin matrix；
Obtain travel speed of the vehicle in the preset time period；
The actual displacement matrix is calculated according to the travel speed, the unit displacement matrix.
6. according to the method described in claim 4, it is characterized in that, the preset algorithm that is based on is to road surface in the two frames picture The Feature Points Matching information in region is calculated, and the homography matrix for obtaining road surface includes:
The road surface region in the two frames picture is calculated using optical flow algorithm, obtains road surface region in the two frames picture Feature Points Matching information；
Based on being carried out by Feature Points Matching information of the consistent RANSAC algorithm of random sampling to road surface region in the two frames picture It calculates, obtains the homography matrix on road surface.
7. according to the method described in claim 4, it is characterized in that, it is described according to the actual displacement matrix, the practical rotation The outer ginseng information that the in-vehicle camera is calculated in torque battle array and the homography matrix includes:
Institute is obtained by the corresponding equation between the actual displacement matrix, the practical spin matrix and the homography matrix State coordinate of the normal vector on road surface under the camera coordinates system；
The coordinate is determined as the outer ginseng information.
8. a kind of device that in-vehicle camera posture is estimated automatically, which is characterized in that described device includes:
Acquiring unit, the two frame pictures shot within a preset period of time for obtaining in-vehicle camera；
First computing unit, for internal reference information, the Feature Points Matching information of the two frames picture according to the in-vehicle camera And travel speed of the affiliated vehicle of in-vehicle camera in the preset time period, calculate the opposite reality of the two frames picture Border transposed matrix and practical spin matrix；
Second computing unit, for being carried out based on Feature Points Matching information of the preset algorithm to road surface region in the two frames picture It calculates, obtains the homography matrix on road surface；
Third computing unit, based on according to the actual displacement matrix, the practical spin matrix and the homography matrix Calculation obtains the outer ginseng information of the in-vehicle camera.
9. a kind of storage medium, which is characterized in that the storage medium is stored with a plurality of instruction, and described instruction is suitable for by handling Device loads and executes the method that the in-vehicle camera posture as described in any one of claim 1,2,4 to 7 is estimated automatically.
10. a kind of device that in-vehicle camera posture is estimated automatically, which is characterized in that described device includes storage medium and processing Device；
The processor is adapted for carrying out each instruction；
The storage medium is suitable for storing a plurality of instruction；
Described instruction is suitable for as the processor loads and executes the vehicle-mounted phase as described in any one of claim 1,2,4 to 7 The method that machine posture is estimated automatically.
CN201711058488.3A 2017-01-25 2017-11-01 The method and apparatus that in-vehicle camera posture is estimated automatically Active CN107730551B (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN2017100630391 2017-01-25
CN201710063039.1A CN107330940A (en) 2017-01-25 2017-01-25 The method and apparatus that in-vehicle camera posture is estimated automatically

Publications (2)

Publication Number Publication Date
CN107730551A CN107730551A (en) 2018-02-23
CN107730551B true CN107730551B (en) 2019-09-17

Family

ID=60193519

Family Applications (2)

Application Number Title Priority Date Filing Date
CN201710063039.1A Pending CN107330940A (en) 2017-01-25 2017-01-25 The method and apparatus that in-vehicle camera posture is estimated automatically
CN201711058488.3A Active CN107730551B (en) 2017-01-25 2017-11-01 The method and apparatus that in-vehicle camera posture is estimated automatically

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN201710063039.1A Pending CN107330940A (en) 2017-01-25 2017-01-25 The method and apparatus that in-vehicle camera posture is estimated automatically

Country Status (1)

Country Link
CN (2) CN107330940A (en)

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107862719B (en) * 2017-11-10 2020-10-27 未来机器人（深圳）有限公司 Method and device for calibrating external parameters of camera, computer equipment and storage medium
EP3531376B1 (en) * 2018-02-21 2020-09-30 Ficosa Adas, S.L.U. Calibrating a camera of a vehicle
CN110197104B (en) * 2018-02-27 2022-03-29 杭州海康威视数字技术股份有限公司 Distance measurement method and device based on vehicle
CN108682038B (en) * 2018-04-27 2021-12-14 腾讯科技（深圳）有限公司 Pose determination method, pose determination device and storage medium
CN110567469B (en) * 2018-06-05 2021-07-20 北京市商汤科技开发有限公司 Visual positioning method and device, electronic equipment and system
CN110858405A (en) 2018-08-24 2020-03-03 北京市商汤科技开发有限公司 Attitude estimation method, device and system of vehicle-mounted camera and electronic equipment
CN109187555A (en) * 2018-09-19 2019-01-11 苏州傲特欣智能科技有限公司 External wall crack detection system and method based on machine vision
CN109242907A (en) * 2018-09-29 2019-01-18 武汉光庭信息技术股份有限公司 A kind of vehicle positioning method and device based on according to ground high speed camera
CN112640417B (en) * 2019-08-09 2021-12-31 华为技术有限公司 Matching relation determining method and related device
CN111260733B (en) * 2020-01-13 2023-03-24 魔视智能科技(上海)有限公司 External parameter estimation method and system of vehicle-mounted all-around multi-camera system
CN111429527B (en) * 2020-03-24 2023-12-01 广东星舆科技有限公司 Automatic external parameter calibration method and system for vehicle-mounted camera
WO2021237574A1 (en) * 2020-05-28 2021-12-02 深圳市大疆创新科技有限公司 Camera parameter determination method and apparatus, and readable storage medium
CN112066988B (en) * 2020-08-17 2022-07-26 联想(北京)有限公司 Positioning method and positioning equipment
CN113639782A (en) * 2021-08-13 2021-11-12 北京地平线信息技术有限公司 External parameter calibration method and device for vehicle-mounted sensor, equipment and medium
CN114663458B (en) * 2022-05-24 2022-10-11 魔门塔(苏州)科技有限公司 Dynamic calibration method, dynamic calibration device, driving state detection method, driving state detection medium and driving state detection equipment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN203142524U (en) * 2013-03-26 2013-08-21 厦门歌乐电子企业有限公司 Road obstacle recognition system
JP5534187B2 (en) * 2010-03-31 2014-06-25 大日本印刷株式会社 Image processing apparatus, image processing method, image processing program, etc.
CN104636724A (en) * 2015-02-02 2015-05-20 华中科技大学 Vehicle-mounted camera rapid pedestrian and vehicle detection method based on goal congruence
CN105142962A (en) * 2013-04-30 2015-12-09 宝马股份公司 Guided vehicle positioning for inductive charging with the assistance of a vehicle camera
CN105849771A (en) * 2013-12-19 2016-08-10 Metaio有限公司 SLAM on a mobile device

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101299237B1 (en) * 2011-11-23 2013-08-22 서울대학교산학협력단 Apparatus and method for detecting object using PTZ camera
CN102800205B (en) * 2012-08-30 2015-06-24 南京大学 Vehicular virtual terminal system based on dynamic map interface

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5534187B2 (en) * 2010-03-31 2014-06-25 大日本印刷株式会社 Image processing apparatus, image processing method, image processing program, etc.
CN203142524U (en) * 2013-03-26 2013-08-21 厦门歌乐电子企业有限公司 Road obstacle recognition system
CN105142962A (en) * 2013-04-30 2015-12-09 宝马股份公司 Guided vehicle positioning for inductive charging with the assistance of a vehicle camera
CN105849771A (en) * 2013-12-19 2016-08-10 Metaio有限公司 SLAM on a mobile device
CN104636724A (en) * 2015-02-02 2015-05-20 华中科技大学 Vehicle-mounted camera rapid pedestrian and vehicle detection method based on goal congruence

Also Published As

Publication number Publication date
CN107330940A (en) 2017-11-07
CN107730551A (en) 2018-02-23

Similar Documents

Publication Publication Date Title
CN107730551B (en) The method and apparatus that in-vehicle camera posture is estimated automatically
Noda et al. Vehicle ego-localization by matching in-vehicle camera images to an aerial image
US9148650B2 (en) Real-time monocular visual odometry
Liu et al. Robust and efficient relative pose with a multi-camera system for autonomous driving in highly dynamic environments
CN110647811A (en) Human face posture detection method and device and computer readable storage medium
CN106203381B (en) Obstacle detection method and device in a kind of driving
CN105844624A (en) Dynamic calibration system, and combined optimization method and combined optimization device in dynamic calibration system
CN110879400A (en) Method, equipment and storage medium for fusion positioning of laser radar and IMU
Phueakjeen et al. A study of the edge detection for road lane
US20050248654A1 (en) Image-based object detection apparatus and method
CN108437893A (en) A kind of method for early warning and device of vehicle lane departure
CN110077416B (en) Decision tree-based driver intention analysis method and system
CN109712196A (en) Camera calibration processing method, device, vehicle control apparatus and storage medium
US20180262739A1 (en) Object detection system
CN112991401B (en) Vehicle running track tracking method and device, electronic equipment and storage medium
JP2020057358A (en) Method and apparatus for acquiring pose information
CN110345936A (en) The track data processing method and its processing system of telecontrol equipment
CN109345591A (en) A kind of vehicle itself attitude detecting method and device
CN110861081B (en) Autonomous positioning method for under-constrained cable parallel robot end effector
KR101272571B1 (en) Simulator for stereo vision system of intelligent vehicle and camera calibration method using the same
WO2016146559A1 (en) Method for determining a position of an object in a three-dimensional world coordinate system, computer program product, camera system and motor vehicle
US20180357792A1 (en) Vision system for a motor vehicle and method of controlling a vision system
CN113971697A (en) Air-ground cooperative vehicle positioning and orienting method
WO2021131364A1 (en) Reflection removal engine generation method, program, reflection removal engine generation device, reflection removal device, abnormality detection method, and component detection method
CN109635658A (en) For the data processing method and device of road test, server

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Address after: Room 3011, Room 3, Building 27, 25 North Third Ring West Road, Haidian District, Beijing

Applicant after: Public Question (Beijing) Information Technology Co., Ltd.

Address before: 100094 Beijing Haidian District Shangzhuang Zhenzhuang Road No. 115 Courtyard Level 522

Applicant before: Intelligent information technology (Beijing) Co., Ltd.

CB02 Change of applicant information
GR01 Patent grant
GR01 Patent grant