CN110533586A - Image split-joint method, device, equipment and system based on vehicle-mounted monocular camera - Google Patents

Image split-joint method, device, equipment and system based on vehicle-mounted monocular camera Download PDF

Info

Publication number
CN110533586A
CN110533586A CN201810501402.8A CN201810501402A CN110533586A CN 110533586 A CN110533586 A CN 110533586A CN 201810501402 A CN201810501402 A CN 201810501402A CN 110533586 A CN110533586 A CN 110533586A
Authority
CN
China
Prior art keywords
frame image
image
current frame
transformation matrix
previous frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810501402.8A
Other languages
Chinese (zh)
Other versions
CN110533586B (en
Inventor
李雪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Priority to CN201810501402.8A priority Critical patent/CN110533586B/en
Publication of CN110533586A publication Critical patent/CN110533586A/en
Application granted granted Critical
Publication of CN110533586B publication Critical patent/CN110533586B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The embodiment of the invention provides a kind of image split-joint method based on vehicle-mounted monocular camera, device, equipment and systems, method includes: to obtain the Orientation differences information of vehicle between the second acquisition moment corresponding with previous frame image at current frame image corresponding first acquisition moment, according to the orientation change information, the first transformation matrix between current frame image and previous frame image is constructed；According to first transformation matrix, current frame image and previous frame image are spliced.As it can be seen that in the present solution, Orientation differences information based on vehicle carries out image mosaic, not by characteristic point on quantity influenced, splicing effect is good.

Description

Image split-joint method, device, equipment and system based on vehicle-mounted monocular camera
Technical field
The present invention relates to auxiliary driving technology fields, more particularly to a kind of image mosaic side based on vehicle-mounted monocular camera Method, device, equipment and system.
Background technique
In existing vehicle, it is usually provided with monocular camera on the outside of vehicle body or in driver's cabin, in vehicle operation, the list Mesh camera carries out Image Acquisition to scenery on the way, and the consecutive frame image collected is spliced, the splicing at big visual angle is obtained The stitching image is showed driver by image, can be played the role of auxiliary and be driven.
The scheme that consecutive frame image is spliced generally includes: in the consecutive frame image, characteristic point pair is extracted, it will be special Sign point, according to registration result, which is spliced to being registrated.In the connection scheme, if extract Number of feature points is less, is spliced according to the registration result of characteristic point pair, and seam is obvious in obtained stitching image, splicing Effect is poor.
Summary of the invention
The embodiment of the present invention is designed to provide a kind of image split-joint method based on vehicle-mounted monocular camera, device, sets Standby and system, to improve splicing effect.
In order to achieve the above objectives, the embodiment of the invention provides a kind of image split-joint method based on vehicle-mounted monocular camera, Include:
Obtain current frame image to be spliced；
Obtain the Orientation differences information of vehicle between the first acquisition moment and the second acquisition moment；Wherein, it described first adopts The collection moment is the current frame image corresponding acquisition moment, and the second acquisition moment is the previous frame of the current frame image The image corresponding acquisition moment；
According to the Orientation differences information, the first transformation between the current frame image and the previous frame image is constructed Matrix；
According to first transformation matrix, the current frame image and the previous frame image are spliced.
Optionally, the Orientation differences information for obtaining vehicle between the first acquisition moment and the second acquisition moment, can be with Include:
According to the motion profile of the trace point in image before the predetermined current frame image, prediction described first Acquire the motion profile of trace point between moment and the second acquisition moment；Wherein, trace point is specific bit in every frame image The pixel set；
According to the obtained motion profile of prediction, calculate vehicle the first acquisition moment and the second acquisition moment it Between relative displacement and relative rotation.
Optionally, described according to the Orientation differences information, construct the current frame image and the previous frame image it Between the first transformation matrix, may include:
According to the relative displacement and the relative rotation, construct between the current frame image and the previous frame image Euler matrivx.
Optionally, it is described obtain first acquisition the moment and second acquisition the moment between vehicle Orientation differences information it Before, can also include:
In the previous frame image of the current frame image and the current frame image, characteristic point pair is extracted；
By the characteristic point to being registrated, according to registration result, the current frame image and the previous frame figure are calculated The second transformation matrix as between；
The deviation between second transformation matrix and predetermined history transformation matrix is calculated, the history converts square Battle array are as follows: the transformation matrix between the previous frame image and the image of previous frame again of the previous frame image；
Judge whether the deviation is greater than preset threshold；
If it does, executing the Orientation differences information for obtaining vehicle between the first acquisition moment and the second acquisition moment The step of；
If it is not greater, the current frame image and the previous frame image are carried out according to second transformation matrix Splicing.
Optionally, in the case where determining that the deviation is not more than the preset threshold, the method also includes:
According to second transformation matrix, by the fortune of the trace point in image before the predetermined current frame image Dynamic rail mark is converted into the coordinate system of the current frame image；Wherein, trace point is the pixel of designated position in every frame image； The motion profile is used to obtain the Orientation differences information of vehicle；
In the coordinate system of the current frame image, the position of the trace point in the current frame image is added to described Motion profile；
In the case where determining that the deviation is greater than the preset threshold, described according to the Orientation differences information, structure It builds after the first transformation matrix between the current frame image and the previous frame image, the method also includes:
According to first transformation matrix, by the fortune of the trace point in image before the predetermined current frame image Dynamic rail mark is converted into the coordinate system of the current frame image；
In the coordinate system of the current frame image, the position of the trace point in the current frame image is added to described Motion profile.
Optionally, described according to first transformation matrix, the current frame image and the previous frame image are carried out Splice, may include:
According to first transformation matrix, by current frame image stitching image corresponding with the previous frame image into Row splicing, obtains the corresponding stitching image of the current frame image；Wherein, the corresponding stitching image of the previous frame image are as follows: What the image before the previous frame image and the previous frame image was spliced.
Optionally, described to obtain current frame image to be spliced, may include:
Obtain the fish eye images of vehicle-mounted monocular camera acquisition；
According to the parameter of the vehicle-mounted monocular camera, the fish eye images are converted into overhead view image, and by the vertical view Image is as the current frame image to be spliced.
In order to achieve the above objectives, the embodiment of the invention also provides a kind of, and the image mosaic based on vehicle-mounted monocular camera fills It sets, comprising:
First obtains module, for obtaining current frame image to be spliced；
Second obtains module, and the Orientation differences for obtaining vehicle between the first acquisition moment and the second acquisition moment are believed Breath；Wherein, the first acquisition moment is the current frame image corresponding acquisition moment, and the second acquisition moment is described The previous frame image of the current frame image corresponding acquisition moment；
Module is constructed, for constructing the current frame image and the previous frame image according to the Orientation differences information Between the first transformation matrix；
First splicing module is used for according to first transformation matrix, by the current frame image and the previous frame figure As being spliced.
Optionally, the second acquisition module specifically can be used for:
According to the motion profile of the trace point in image before the predetermined current frame image, prediction described first Acquire the motion profile of trace point between moment and the second acquisition moment；Wherein, trace point is specific bit in every frame image The pixel set；
According to the obtained motion profile of prediction, calculate vehicle the first acquisition moment and the second acquisition moment it Between relative displacement and relative rotation.
Optionally, the building module, specifically can be used for:
According to the relative displacement and the relative rotation, construct between the current frame image and the previous frame image Euler matrivx.
Optionally, described device can also include:
Extraction module obtains vehicle between the first acquisition moment and the second acquisition moment for obtaining module described second Orientation differences information before, in the previous frame image of the current frame image and the current frame image, extract characteristic point It is right；
First computing module, for the characteristic point to being registrated, according to registration result, to be calculated the present frame figure Picture and the second transformation matrix between the previous frame image；
Second computing module, it is inclined between second transformation matrix and predetermined history transformation matrix for calculating Difference, the history transformation matrix are as follows: the transformation between the previous frame image and the image of previous frame again of the previous frame image Matrix；
Judgment module, for judging whether the deviation is greater than preset threshold；If it does, triggering described second obtains mould Block, if it is not greater, the second splicing module of triggering；
Second splicing module is used for according to second transformation matrix, by the current frame image and the previous frame figure As being spliced.
Optionally, described device can also include:
First conversion module, for determining the case where deviation is not more than the preset threshold in the judgment module Under, according to second transformation matrix, by the movement rail of the trace point in image before the predetermined current frame image Mark is converted into the coordinate system of the current frame image, and triggers adding module；Wherein, trace point is specific bit in every frame image The pixel set；The motion profile is used to obtain the Orientation differences information of vehicle；
Second conversion module is used in the case where the judgment module determines that the deviation is greater than the preset threshold, In the building module according to the Orientation differences information, between the current frame image and the previous frame image is constructed After one transformation matrix, according to first transformation matrix, by image before the predetermined current frame image with The motion profile of track point is converted into the coordinate system of the current frame image, and triggers adding module；
The adding module, in the coordinate system of the current frame image, by the tracking in the current frame image The position of point is added to the motion profile.
Optionally, first splicing module, specifically can be used for:
According to first transformation matrix, by current frame image stitching image corresponding with the previous frame image into Row splicing, obtains the corresponding stitching image of the current frame image；Wherein, the corresponding stitching image of the previous frame image are as follows: What the image before the previous frame image and the previous frame image was spliced.
Optionally, described first module is obtained, specifically can be used for:
Obtain the fish eye images of vehicle-mounted monocular camera acquisition；
According to the parameter of the vehicle-mounted monocular camera, the fish eye images are converted into overhead view image, and by the vertical view Image is as the current frame image to be spliced.
In order to achieve the above objectives, also a kind of electronic equipment of the embodiment of the present invention, including processor, communication interface, memory And communication bus, wherein processor, communication interface, memory complete mutual communication by communication bus；
Memory, for storing computer program；
Processor when for executing the program stored on memory, realizes that any of the above-described kind is based on vehicle-mounted monocular camera Image split-joint method.
In order to achieve the above objectives, also a kind of DAS (Driver Assistant System) of the embodiment of the present invention, comprising: monocular camera and processing are set It is standby；Wherein,
The monocular camera, for collected each current frame image to be sent to the processing equipment；
The processing equipment, for receiving the current frame image；Obtain first acquisition the moment with second acquisition the moment it Between vehicle Orientation differences information；Wherein, the first acquisition moment is the current frame image corresponding acquisition moment, described Second acquisition moment was the previous frame image corresponding acquisition moment of the current frame image；According to the Orientation differences information, Construct the first transformation matrix between the current frame image and the previous frame image；It, will according to first transformation matrix The current frame image is spliced with the previous frame image.
Using illustrated embodiment of the present invention, it is corresponding with previous frame image to obtain the current frame image corresponding first acquisition moment The second acquisition moment between the Orientation differences information of vehicle current frame image and upper one is constructed according to the orientation change information The first transformation matrix between frame image；According to first transformation matrix, current frame image and previous frame image are spliced. As it can be seen that in the present solution, Orientation differences information based on vehicle carries out image mosaic, not by characteristic point on quantity influenced, splice Effect is good.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with It obtains other drawings based on these drawings.
Fig. 1 a is a kind of block schematic illustration provided in an embodiment of the present invention；
Fig. 1 b is another block schematic illustration provided in an embodiment of the present invention；
Fig. 1 c is that the first process of the image split-joint method provided in an embodiment of the present invention based on vehicle-mounted monocular camera is shown It is intended to；
Fig. 2 is vehicle heading change information schematic diagram in the embodiment of the present invention；
Fig. 3 is the displacement curve schematic diagram of each frame image trace point in the embodiment of the present invention；
Fig. 4 is each frame image mosaic schematic diagram in the embodiment of the present invention；
Fig. 5 is that second of process of the image split-joint method provided in an embodiment of the present invention based on vehicle-mounted monocular camera is illustrated Figure；
Fig. 6 is a kind of structural representation of the image splicing device based on vehicle-mounted monocular camera provided in an embodiment of the present invention Figure；
Fig. 7 is the structural schematic diagram of a kind of electronic equipment provided in an embodiment of the present invention；
Fig. 8 is a kind of structural schematic diagram of DAS (Driver Assistant System) provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other Embodiment shall fall within the protection scope of the present invention.
Term of the invention is explained as follows:
Characteristic point refers to that the point of acute variation occurs for gray value of image or the biggish point of curvature is (i.e. on image border The intersection point at two edges).
Perspective transformation matrix refers to the matrix for using picture projection to a new view plane.
Internal reference, refers to parameter relevant to camera self-characteristic, such as the focal length of camera, pixel size, principal point coordinate, abnormal Variable coefficient etc..
Outer ginseng refers to the parameter of camera in world coordinate system, such as position, the direction of rotation of camera etc..
Viewpoint change refers to and shows content according to image, changes viewpoint position and direction.
Overhead view image refers to that the general camera installation ground Shi Huiyu forms certain angle, thus on ground rectangle object Trapezoidal or other shapes can be deformed into.According to parameters such as camera setting angle, lens distortions, convert general camera imaging to From vertical direction downwards from ground visual angle, produced image be overhead view image.If overhead view image have passed through distortion correction And viewpoint change, then body form can be consistent with real-world object shape on ground in overhead view image.
Euler's transformation refers to and belongs to perspective transform one kind, only changes the direction and position of object, do not change shape.
Inventive concept of the invention is as follows:
As shown in Figure 1a, image splicing device provided in an embodiment of the present invention includes two modules: demarcating module and splicing Module.Demarcating module provides look-up table for splicing module, mutually indepedent with splicing module in process.
Look-up table is established in demarcating module in advance, this process generally carries out in parking lot or the shop 4S, and camera is pacified It can only be carried out after dress primary.The process may include:
Camera acquires fish eye images, and demarcating module is corrected the fish eye images using camera internal reference, and according to camera Fish eye images are converted to overhead view image by viewpoint change by outer ginseng, are obtained overhead view image to the look-up table of fish eye images, are somebody's turn to do The mapping relations of pixel in look-up table i.e. fish eye images and overhead view image.
Later, in vehicle travel process, camera acquires fish eye images in real time, and the fish eye images of acquisition are inputted and are spliced The fish eye images are converted to overhead view image, and to consecutive frame using the look-up table established in demarcating module by module, splicing module Overhead view image carries out splicing, exports stitching image.
Specifically, the process that splicing module carries out splicing to consecutive frame overhead view image can be to splice frame by frame.For example, Present frame overhead view image stitching image corresponding with previous frame overhead view image is spliced, the corresponding spelling of previous frame overhead view image In map interlinking picture not only include previous frame overhead view image information, also comprising previous frame overhead view image in addition before image information.This Sample during splicing frame by frame, is continuously increased new content in stitching image, show that content is richer, visual effect is more preferably.
It describes in detail, as shown in Figure 1 b, splicing module may include following submodule: overlooking to splicing module below Image generates submodule, image registration submodule and image co-registration submodule.
Overhead view image is generated the look-up table that submodule is read in demarcating module and is acquired camera in real time using the look-up table Fish eye images be converted to overhead view image, overhead view image is input to image registration submodule.
Image registration submodule obtains perspective transformation matrix by carrying out Characteristic points match in consecutive frame overhead view image. Specifically, this process may include: feature point extraction, feature point description, Feature Points Matching, obtain characteristic point to, calculate it is saturating Depending on transformation matrix.
If the perspective transformation matrix and previous frame overhead view image that are calculated splice the perspective transformation matrix deviation used Less than threshold value, then the perspective transformation matrix being calculated is input to image co-registration submodule by image registration submodule.
If the perspective transformation matrix and previous frame overhead view image that are calculated splice the perspective transformation matrix deviation used Not less than threshold value, then image registration submodule also needs to be modified the perspective transformation matrix being calculated.
Makeover process may include:
It takes one or more pixels in present frame overhead view image as trace point, obtains trace point in stitching image Motion profile.For example, the forming process of the motion profile may include: after getting first frame image, in first frame In the coordinate system of image, the central point of first frame image is appointed as trace point, obtains the coordinate of trace point in first frame image； After getting the second frame image, the coordinate of trace point in the first frame image is converted into the coordinate system of the second frame image, it will The central point of second frame image is also appointed as trace point, obtains the coordinate of trace point in the second frame image, this two frames trace point Coordinate forms one section of motion profile；After getting third frame image, one section of motion profile of above-mentioned formation is converted to third frame In the coordinate system of image, the central point of third frame image is appointed as trace point, obtains the coordinate of trace point in third frame image, The coordinate of trace point in the third frame image is added to one section of motion profile, etc., the treatment process of following frame image It is similar, it repeats no more.Image in this example is overhead view image.
Using the motion profile, relative displacement and the relative rotation of vehicle body are predicted.Relative displacement mentioned here and opposite Corner refers to: in the present frame overhead view image period corresponding with next frame overhead view image, the relative displacement of vehicle body and opposite turning Angle.
Based on the relative displacement and relative rotation, the Euler between present frame overhead view image and next frame overhead view image is calculated Transformation matrix, the Euler's transformation matrix are revised transformation matrix.
Image registration submodule is by the Euler's transformation Input matrix to image co-registration module.
For image co-registration submodule, input may be uncorrected perspective transformation matrix, it is also possible to for amendment Euler's transformation matrix afterwards.Image co-registration submodule by both matrixes consecutive frame overhead view image is spliced in the way of class Seemingly, description is not repartitioned.
As described above, splicing can be for splicing frame by frame: present frame overhead view image is corresponding with previous frame overhead view image Stitching image spliced, in the corresponding stitching image of previous frame overhead view image not only include previous frame overhead view image letter Breath, also comprising previous frame overhead view image in addition before image information.Image registration submodule exports stitching image.
As it can be seen that, if matrix deviation is larger, relative displacement and relative rotation based on vehicle body are to square using above scheme Battle array be modified, using revised matrix carry out image mosaic, not by characteristic point on quantity and quality influenced, splicing effect It is good.
Based on identical inventive concept, the embodiment of the invention provides a kind of image mosaic sides based on vehicle-mounted monocular camera Method, device, equipment and system, this method and device can be applied to the figure being arranged in various electronic equipments, such as mobile phone, vehicle As processing equipment or the in-vehicle camera with image processing function, etc., specifically without limitation.
Fig. 1 c is that the first process of the image split-joint method provided in an embodiment of the present invention based on vehicle-mounted monocular camera is shown It is intended to, comprising:
S101: current frame image to be spliced is obtained.
For example, if executing subject is vehicle-mounted monocular camera, the present frame figure which itself can will acquire As current frame image to be spliced.If executing subject is other electronic equipments communicated to connect with vehicle-mounted monocular camera, Then the current frame image of the available camera acquisition of the electronic equipment is as current frame image to be spliced.
Alternatively, the image that camera can also be acquired is used as original image, general's treated original image is as to be spliced Current frame image.Such as, it is assumed that the original image of vehicle-mounted monocular camera acquisition is fish eye images, then can be according to vehicle-mounted monocular The fish eye images are converted to overhead view image, using the overhead view image as current frame image to be spliced by the parameter of camera.Vehicle The parameter for carrying monocular camera first passes through calibration in advance and obtains.
Alternatively, distortion correction or other optimization processings can also be carried out to original image or above-mentioned overhead view image, Using treated image as current frame image to be spliced.
The embodiment of the present invention can carry out splicing to video flowing, and each frame in video flowing is worked as to be spliced Previous frame.
S102: the Orientation differences information of vehicle between the first acquisition moment and the second acquisition moment is obtained；Wherein, it first adopts Integrate the moment as the current frame image corresponding acquisition moment, the second acquisition moment, which was that the previous frame image of current frame image is corresponding, adopts Collect the moment.
Orientation differences information may include change in location information and direction change information, for example can adopt for vehicle first Collect the relative displacement between moment and the second acquisition moment and relative rotation.As illustrated in fig. 2, it is assumed that vehicle acquires the moment first Position be P0, vehicle is P in the position at the second acquisition moment1, P1Relative to P0Relative displacement in the direction x is △ x, in the side y To relative displacement be △ y, P1Relative to P0Relative rotation in the direction x is b.
For example, it can be obtained by vehicle GPS (Global Positioning System, global positioning system) Location information and steering direction of the vehicle at the first acquisition moment and the second acquisition moment, according to the location information and driving side To relative displacement and relative rotation of the calculating vehicle between the first acquisition moment and the second acquisition moment.
Alternatively, as an implementation, S102 may include: to scheme according to before the predetermined current frame image The motion profile of trace point as in predicts the movement of trace point between the first acquisition moment and the second acquisition moment Track；Wherein, trace point is the pixel of designated position in every frame image；According to the motion profile that prediction obtains, vehicle is calculated Relative displacement and relative rotation between the first acquisition moment and the second acquisition moment.
In the present embodiment, a trace point is specified in every frame image, which can be fixed bit in image The pixel set, for example, can be the movement rail for the motion profile i.e. vehicle that the central point in image, each trace point are formed Mark.Carrying out handling the trace point not included in current frame image in predetermined motion profile to current frame image, still The fortune of the trace point in the distal point and current frame image of the motion profile can be predicted according to the movement tendency of the motion profile Trace point in the previous frame image of dynamic rail mark, the distal point i.e. current frame image, therefore, the motion profile predicted The motion profile of trace point between the first acquisition moment and the second acquisition moment namely predicted.
Predict that obtained motion profile can also be used as fortune of the vehicle between the first acquisition moment and the second acquisition moment Dynamic rail mark, therefore, the motion profile that can be obtained according to prediction calculates relative displacement and the relative rotation of vehicle.
With reference to Fig. 3, (a) in Fig. 3 indicates trace point p in the displacement curve in the direction x, and horizontal axis indicates the frame number of image, The shift value in the longitudinal axis expression direction x；(b) in Fig. 3 indicates trace point p in the displacement curve in the direction y, and horizontal axis indicates image Frame number, the longitudinal axis indicate the shift value in the direction y；(c) in Fig. 3 indicates that the motion profile of trace point p, horizontal axis indicate the direction x Shift value, the longitudinal axis indicate the direction y shift value.Three figures in Fig. 3 are made with the position of image top left corner pixel point For the position of coordinate origin.
Assuming that coordinate value of the trace point in the previous frame image of present frame is (xc, yc), the seat of trace point in the current frame Scale value is (xc+1, yc+1), then relative displacement d of the vehicle in the direction xx=xc+1-xc, relative displacement d of the vehicle in the direction yy= yc+1-yc, relative rotation α=actan (k of vehiclec+1)-actan(kc), wherein kcTrace point is oblique in expression previous frame image Rate, kc+1Indicate the slope of trace point in current frame image.Specifically, can according to the first derivative of motion profile in Fig. 3 (c), K is calculatedcAnd kc+1。dxAlso the △ x, d being equivalent in Fig. 2yAlso the △ y being equivalent in Fig. 2, α are also equivalent in Fig. 2 B.
S103: according to the orientation change information, the first transformation between the current frame image and the previous frame image is constructed Matrix.
As an implementation, the present frame can be constructed according to relative displacement obtained above and relative rotation Euler matrivx between image and the previous frame image.
As described above, it is d that relative displacement of the vehicle between the first acquisition moment and the second acquisition moment, which is calculated,xWith dy, relative rotation α, it is assumed that coordinate value of the vehicle at the second acquisition moment is (x0, y0), seat of the vehicle at the first acquisition moment Scale value is (x1, y1), then it is available:
It can be obtained according to Euler matrivx:
Euler matrivx H are as follows:
S104: according to first transformation matrix, the current frame image and the previous frame image are spliced.
For example, can be by first transformation matrix, it will be where the previous frame image projection to the current frame image Then plane again splices the previous frame image after projection with the current frame image.
As an implementation, can in conjunction with image interpolation algorithm, by the current frame image and the previous frame image into Row splicing.It is appreciated that can be utilized by image interpolation algorithm if there is the pixel not matched that in two images This does not match that the consecutive points pixel value of pixel carries out interpolation processing to the pixel not matched that.
As described above, the embodiment of the present invention can carry out splicing to video flowing, by each frame image in video flowing As current frame image to be spliced.Assuming that current frame image is nth frame image, that is to say, that before S101, obtained The stitching image of N-1 frame image and N-2 frame image therefore can be by nth frame image and the stitching image (the in S104 The stitching image of N-1 frame image and N-2 frame image) spliced.In this case, in splicing, by nth frame image with The N-1 frame image for including in the stitching image is spliced.
As an implementation, S104 may include: according to first transformation matrix, by the current frame image with The corresponding stitching image of the previous frame image is spliced, and the corresponding stitching image of the current frame image is obtained；Wherein, institute State the corresponding stitching image of previous frame image are as follows: the previous frame image is spliced with the image before the previous frame image It obtains.
Continue above-mentioned example, " the corresponding stitching image of previous frame image " can be understood as above-mentioned N-1 frame image and The stitching image of N-2 frame image, " the corresponding stitching image of current frame image " can be understood as above-mentioned nth frame image and N-1 The stitching image of frame image.
In present embodiment, current frame image stitching image corresponding with previous frame image is spliced, previous frame figure As in corresponding stitching image not only include previous frame image information, also comprising previous frame image in addition before image information. As shown in figure 4, f1 indicates that first frame image, f2 indicate that the second frame image, p1 indicate first frame image and the second frame image mosaic Obtained image, f3 indicate that third frame image, p2 indicate the stitching image, etc. of p1 and f3.That is, present embodiment In, it is continuously increased new content during being spliced frame by frame, in stitching image, shows that content is richer, visual effect is more It is good.
In existing scheme, generallys use characteristic point and image mosaic is carried out to the mode of registration, but in some scenes, than On the less highway of scene change, content is similar between consecutive frame image, it is more difficult to extract accurate characteristic point pair.It can To understand, characteristic point is to being belonging respectively to two field pictures, and characteristic point is to the same point corresponded in real space, and scenery becomes Change on less highway, many pixels are similar (pixel on such as highway) in consecutive frame image, are difficult to tell pair The characteristic point pair of the same point therefore the case where characteristic point is to error detection or error hiding should be easy to appear in real space. Alternatively, if Vehicle Speed is very fast, existing overlapping region very little between consecutive frame image, in this case, it is also difficult to Extract the characteristic point pair in consecutive frame image.
In above-mentioned scene, if carrying out image mosaic in such a way that characteristic point is to registration, splicing effect is poor, and utilizes this Inventive embodiments, the Orientation differences information based on vehicle carry out image mosaic, are not influenced by number of feature points and quality, splice Effect is good.
It as an implementation, can be first in the current frame image and the current frame image before S102 In previous frame image, characteristic point pair is extracted；By the characteristic point to being registrated, according to registration result, the present frame is calculated The second transformation matrix between image and the previous frame image；It calculates second transformation matrix and predetermined history becomes Change the deviation between matrix, the history transformation matrix are as follows: the previous frame again of the previous frame image and the previous frame image Transformation matrix between image；Judge whether the deviation is greater than preset threshold；If it does, S102 is executed again, if less In then according to second transformation matrix, the current frame image and the previous frame image are spliced.
In present embodiment, the second transformation matrix will be known as to the transformation matrix that registration result obtains according to characteristic point, it will It is known as the first transformation matrix according to the transformation matrix that the Orientation differences information of vehicle obtains.
In present embodiment, first with characteristic point to the mode of registration, current frame image and previous frame image is calculated Between the second transformation matrix, which can be homography matrix.Assuming that current frame image is nth frame image, Transformation matrix between second transformation matrix i.e. nth frame image and N-1 frame image, predetermined history convert square Transformation matrix of the battle array namely between N-1 frame image and N-2 frame image.If second transformation matrix and history convert square Deviation between battle array is smaller, that is to say, that second transformation matrix meets deflection condition, then can directly utilize second transformation Matrix splices nth frame image and N-1 frame image, does not need to execute subsequent schedule；If the deviation is larger, indicate It carries out that splicing effect is bad in such a way that characteristic point is to registration, the Orientation differences information based on vehicle is needed to carry out image spelling It connects, to improve splicing effect.
History transformation matrix between N-1 frame image and N-2 frame image is the first transformation matrix or the second transformation Matrix.Similar, the second transformation matrix between N-1 frame image and N-2 frame image can be first calculated, if second change The deviation changed between the transformation matrix between matrix and N-2 frame image and N-3 frame is smaller (meeting deflection condition), then N- History transformation matrix between 1 frame image and N-2 frame image is the second transformation matrix, conversely, being then the first transformation matrix.
As described above, a trace point is specified in every frame image, the motion profile i.e. vehicle that each trace point is formed Motion profile, the forming process of the motion profile can be with are as follows: according to the previous frame image of current frame image and current frame image Between transformation matrix, the motion profile of the trace point in image before predetermined current frame image is converted to present frame In the coordinate system of image；In the coordinate system of current frame image, the position of the trace point in current frame image is added to the fortune Dynamic rail mark.
Transformation matrix between the current frame image and the previous frame image is the first transformation matrix or the second transformation square Battle array.It is similar, if the second transformation matrix meets deflection condition, for the second transformation matrix, if being unsatisfactory for deflection condition, For the first transformation matrix.
For example, after getting first frame image, in the coordinate system of first frame image, by the center of first frame image Point is appointed as trace point, obtains the coordinate of trace point in first frame image；After getting the second frame image, by the first frame image The coordinate of middle trace point is converted into the coordinate system of the second frame image, and the central point of the second frame image is appointed as trace point, is obtained The coordinate of the coordinate of trace point into the second frame image, this two frames trace point forms one section of motion profile；Get third frame figure As after, one section of motion profile of above-mentioned formation is converted into the coordinate system of third frame image, by the central point of third frame image It is appointed as trace point, obtains the coordinate of trace point in third frame image, the coordinate of trace point in the third frame image is added to The treatment process of one section of motion profile, etc., following frame image is similar, repeats no more.
Using Fig. 1 c illustrated embodiment of the present invention, current frame image corresponding first acquisition moment and previous frame image are obtained The Orientation differences information of vehicle between the corresponding second acquisition moment, according to the orientation change information, construct current frame image with The first transformation matrix between previous frame image；According to first transformation matrix, current frame image and previous frame image are carried out Splicing.As it can be seen that in the present solution, the Orientation differences information based on vehicle carries out image mosaic, not by the quantity of characteristic point pair and matter Amount influences, and splicing effect is good.
Fig. 5 is that second of process of the image split-joint method provided in an embodiment of the present invention based on vehicle-mounted monocular camera is illustrated Figure, comprising:
S501: current frame image to be spliced is obtained.
For example, if executing subject is vehicle-mounted monocular camera, the present frame figure which itself can will acquire As current frame image to be spliced.If executing subject is other electronic equipments communicated to connect with vehicle-mounted monocular camera, Then the current frame image of the available camera acquisition of the electronic equipment is as current frame image to be spliced.
Alternatively, the image that camera can also be acquired is used as original image, general's treated original image is as to be spliced Current frame image.Such as, it is assumed that the original image of vehicle-mounted monocular camera acquisition is fish eye images, then can be according to vehicle-mounted monocular The fish eye images are converted to overhead view image, as current frame image to be spliced by the parameter of camera.Vehicle-mounted monocular camera Parameter first passes through calibration in advance and obtains.
Alternatively, distortion correction or other optimization processings can also be carried out to original image or above-mentioned overhead view image, Using treated image as current frame image to be spliced.
S502: in this prior in the previous frame image of frame image and the current frame image, characteristic point pair is extracted.
S503: by this feature point to being registrated, according to registration result, the current frame image and the previous frame image are calculated Between the second transformation matrix.Second transformation matrix can be homography matrix.
In the present embodiment, the second transformation matrix will be known as to the transformation matrix that registration result obtains according to characteristic point, by root It is known as the first transformation matrix according to the transformation matrix that the Orientation differences information of vehicle obtains.
S504: the deviation between second transformation matrix and predetermined history transformation matrix, history transformation are calculated Matrix are as follows: the transformation matrix between the previous frame image and the image of previous frame again of the previous frame image.
S505: judge whether the deviation is greater than preset threshold；If it is not greater, S506-S508 is executed, if it does, holding Row S509-S514.
S506: according to second transformation matrix, the current frame image and the previous frame image are spliced.
S507: according to second transformation matrix, by the trace point in image before the predetermined current frame image Motion profile is converted into the coordinate system of the current frame image.
Wherein, trace point is the pixel of designated position in every frame image；The motion profile is used to obtain the orientation of vehicle Change information.
S508: in this prior in the coordinate system of frame image, the position of the trace point in the current frame image is added to this Motion profile.
The execution sequence of S506 and S507-S508 is not construed as limiting, and can be first carried out S506, be executed S507-S508 afterwards；It can also To first carry out S507-S508, S506 is executed afterwards；Or it also may be performed simultaneously S506 and S507-S508.If first carried out S506 executes S507-S508 afterwards, then the coordinate system of the coordinate system of current frame image i.e. stitching image in S507.Namely It says, in the stitching image that current frame image and previous frame image are spliced, on the basis of current frame image, spliced map The coordinate system of picture i.e. the coordinate system of current frame image.
S509: according to the motion profile of the trace point in image before the predetermined current frame image, prediction first Acquire the motion profile of trace point between moment and the second acquisition moment.Wherein, the first acquisition moment is corresponding for current frame image The acquisition moment, second acquisition the moment be current frame image the previous frame image corresponding acquisition moment.
S510: the motion profile obtained according to prediction calculates vehicle at the first acquisition moment and the second acquisition moment Between relative displacement and relative rotation.
S511: it according to the relative displacement and the relative rotation, constructs between the current frame image and the previous frame image First transformation matrix, first transformation matrix can be Euler matrivx.
S512: according to first transformation matrix, the current frame image and the previous frame image are spliced.
S513: according to first transformation matrix, by the trace point in image before the predetermined current frame image Motion profile convert into the coordinate system of the current frame image.
S514: in this prior in the coordinate system of frame image, the position of the trace point in the current frame image is added to this Motion profile.
The execution sequence of S512 and S513-S514 is not construed as limiting, and can be first carried out S512, be executed S513-S14 afterwards；It can also To first carry out S513-S14, S512 is executed afterwards；Or it also may be performed simultaneously S512 and S513-S14.If first carrying out S512, After execute S513-S14, then the coordinate system of the coordinate system of current frame image i.e. stitching image in S513.That is, will work as In the stitching image that prior image frame and previous frame image are spliced, on the basis of current frame image, the seat of stitching image Mark the coordinate system of system i.e. current frame image.
For example, the forming process of the motion profile may include: after getting first frame image, in first frame image Coordinate system in, the central point of first frame image is appointed as trace point, obtains the coordinate of trace point in first frame image；It obtains To after the second frame image, the coordinate of trace point in the first frame image is converted into the coordinate system of the second frame image, by second The central point of frame image is also appointed as trace point, obtains the coordinate of trace point in the second frame image, the coordinate of this two frames trace point Form one section of motion profile；After getting third frame image, one section of motion profile of above-mentioned formation is converted to third frame image Coordinate system in, the central point of third frame image is appointed as trace point, the coordinate of trace point in third frame image is obtained, by this The coordinate of trace point is added to one section of motion profile, etc. in third frame image, and the treatment process of following frame image is similar, It repeats no more.
As an implementation, in S506 and S512, by current frame image stitching image corresponding with previous frame image Spliced, not only includes the information of previous frame image in the corresponding stitching image of previous frame image, also include previous frame image In addition the image information before.As shown in figure 4, f1 indicates that first frame image, f2 indicate that the second frame image, p1 indicate first frame image The image obtained with the second frame image mosaic, f3 indicate that third frame image, p2 indicate the stitching image, etc. of p1 and f3.Also It is to say, in present embodiment, new content is continuously increased during being spliced frame by frame, in stitching image, shows content more Abundant, visual effect is more preferably.
Using embodiment illustrated in fig. 5 of the present invention, current frame image corresponding first acquisition moment and previous frame image are obtained The Orientation differences information of vehicle between the corresponding second acquisition moment, according to the orientation change information, construct current frame image with The first transformation matrix between previous frame image；According to first transformation matrix, current frame image and previous frame image are carried out Splicing.As it can be seen that in the present solution, the Orientation differences information based on vehicle carries out image mosaic, not by number of feature points and quality It influences, splicing effect is good.
Corresponding with above method embodiment, the embodiment of the present invention also provides a kind of image spelling based on vehicle-mounted monocular camera Connection device, as shown in Figure 6, comprising:
First obtains module 601, for obtaining current frame image to be spliced；
Second obtains module 602, for obtaining the Orientation differences of vehicle between the first acquisition moment and the second acquisition moment Information；Wherein, the first acquisition moment is the current frame image corresponding acquisition moment, and the second acquisition moment is institute State the previous frame image corresponding acquisition moment of current frame image；
Module 603 is constructed, for constructing the current frame image and the previous frame figure according to the Orientation differences information The first transformation matrix as between；
First splicing module 604 is used for according to first transformation matrix, by the current frame image and the previous frame Image is spliced.
As an implementation, second module 602 is obtained, specifically can be used for:
According to the motion profile of the trace point in image before the predetermined current frame image, prediction described first Acquire the motion profile of trace point between moment and the second acquisition moment；Wherein, trace point is specific bit in every frame image The pixel set；
According to the obtained motion profile of prediction, calculate vehicle the first acquisition moment and the second acquisition moment it Between relative displacement and relative rotation.
As an implementation, module 603 is constructed, specifically can be used for:
According to the relative displacement and the relative rotation, construct between the current frame image and the previous frame image Euler matrivx.
As an implementation, described device can also include: extraction module, the first computing module, the second calculating mould Block, judgment module and the second splicing module (not shown), wherein
Extraction module obtains vehicle between the first acquisition moment and the second acquisition moment for obtaining module described second Orientation differences information before, in the previous frame image of the current frame image and the current frame image, extract characteristic point It is right；
First computing module, for the characteristic point to being registrated, according to registration result, to be calculated the present frame figure Picture and the second transformation matrix between the previous frame image；
Second computing module, it is inclined between second transformation matrix and predetermined history transformation matrix for calculating Difference, the history transformation matrix are as follows: the transformation between the previous frame image and the image of previous frame again of the previous frame image Matrix；
Judgment module, for judging whether the deviation is greater than preset threshold；If it does, triggering described second obtains mould Block, if it is not greater, the second splicing module of triggering；
Second splicing module is used for according to second transformation matrix, by the current frame image and the previous frame figure As being spliced.
As an implementation, described device can also include: the first conversion module, the second conversion module and addition mould Block (not shown), wherein
First conversion module, for determining the case where deviation is not more than the preset threshold in the judgment module Under, according to second transformation matrix, by the movement rail of the trace point in image before the predetermined current frame image Mark is converted into the coordinate system of the current frame image, and triggers adding module；Wherein, trace point is specific bit in every frame image The pixel set；The motion profile is used to obtain the Orientation differences information of vehicle；
Second conversion module is used in the case where the judgment module determines that the deviation is greater than the preset threshold, In the building module according to the Orientation differences information, between the current frame image and the previous frame image is constructed After one transformation matrix, according to first transformation matrix, by image before the predetermined current frame image with The motion profile of track point is converted into the coordinate system of the current frame image, and triggers adding module；
The adding module, in the coordinate system of the current frame image, by the tracking in the current frame image The position of point is added to the motion profile.
As an implementation, the first splicing module 604, specifically can be used for:
According to first transformation matrix, by current frame image stitching image corresponding with the previous frame image into Row splicing, obtains the corresponding stitching image of the current frame image；Wherein, the corresponding stitching image of the previous frame image are as follows: What the image before the previous frame image and the previous frame image was spliced.
As an implementation, first module 601 is obtained, specifically can be used for:
Obtain the fish eye images of vehicle-mounted monocular camera acquisition；
According to the parameter of the vehicle-mounted monocular camera, the fish eye images are converted into overhead view image, and by the vertical view Image is as the current frame image to be spliced.
Using embodiment illustrated in fig. 6 of the present invention, current frame image corresponding first acquisition moment and previous frame image are obtained The Orientation differences information of vehicle between the corresponding second acquisition moment, according to the orientation change information, construct current frame image with The first transformation matrix between previous frame image；According to first transformation matrix, current frame image and previous frame image are carried out Splicing.As it can be seen that in the present solution, the Orientation differences information based on vehicle carries out image mosaic, not by the quantity of characteristic point pair and matter Amount influences, and splicing effect is good.
The first acquisition module 601 in embodiment illustrated in fig. 6 can be understood as the overhead view image in Fig. 1 b and generate submodule, Second acquisition module 602 and building module 603 can reside in the image registration submodule in Fig. 1 b, and the first splicing module can be with The image co-registration submodule being present in Fig. 1 b.
The image that extraction module, the first computing module, the second computing module and judgment module can reside in Fig. 1 b is matched Quasi- submodule, the second splicing module can reside in the image co-registration submodule in Fig. 1 b.
First conversion module, the second conversion module and adding module can reside in the image registration submodule in Fig. 1 b.
The embodiment of the invention also provides a kind of electronic equipment, as shown in fig. 7, comprises processor 701, communication interface 702, Memory 703 and communication bus 704, wherein processor 701, communication interface 702, memory 703 are complete by communication bus 704 At mutual communication,
Memory 703, for storing computer program；
Processor 701 when for executing the program stored on memory 703, realizes that any of the above-described kind is based on vehicle-mounted list The image split-joint method of mesh camera.
The communication bus that above-mentioned electronic equipment is mentioned can be Peripheral Component Interconnect standard (Peripheral Component Interconnect, PCI) bus or expanding the industrial standard structure (Extended Industry Standard Architecture, EISA) bus etc..The communication bus can be divided into address bus, data/address bus, control bus etc..For just It is only indicated with a thick line in expression, figure, it is not intended that an only bus or a type of bus.
Communication interface is for the communication between above-mentioned electronic equipment and other equipment.
Memory may include random access memory (Random Access Memory, RAM), also may include non-easy The property lost memory (Non-Volatile Memory, NVM), for example, at least a magnetic disk storage.Optionally, memory may be used also To be storage device that at least one is located remotely from aforementioned processor.
Above-mentioned processor can be general processor, including central processing unit (Central Processing Unit, CPU), network processing unit (Network Processor, NP) etc.；It can also be digital signal processor (Digital Signal Processing, DSP), it is specific integrated circuit (Application Specific Integrated Circuit, ASIC), existing It is field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete Door or transistor logic, discrete hardware components.
The embodiment of the invention also provides a kind of computer readable storage medium, the computer readable storage medium memory Computer program is contained, the computer program realizes any of the above-described kind of figure based on vehicle-mounted monocular camera when being executed by processor As joining method.
The embodiment of the invention also provides a kind of DAS (Driver Assistant System)s, as shown in Figure 8, comprising: monocular camera and processing are set It is standby；Wherein,
The monocular camera, for collected each current frame image to be sent to the processing equipment；
The processing equipment, for receiving the current frame image；Obtain first acquisition the moment with second acquisition the moment it Between vehicle Orientation differences information；Wherein, the first acquisition moment is the current frame image corresponding acquisition moment, described Second acquisition moment was the previous frame image corresponding acquisition moment of the current frame image；According to the Orientation differences information, Construct the first transformation matrix between the current frame image and the previous frame image；It, will according to first transformation matrix The current frame image is spliced with the previous frame image.
It may include display screen in the processing equipment, which is used for the image that display splicing obtains.Alternatively, the system In display equipment can be separately provided, which is used for the obtained image of display splicing.Alternatively, the monocular camera, at this It manages equipment and the display equipment may be same equipment.
Processing equipment in the system can execute any of the above-described kind of image split-joint method based on vehicle-mounted monocular camera.
It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that There is also other identical elements in process, method, article or equipment including the element.
Each embodiment in this specification is all made of relevant mode and describes, same and similar portion between each embodiment Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for Fig. 6 institute The image splicing device embodiment based on vehicle-mounted monocular camera shown, electronic equipment embodiment shown in Fig. 7, above-mentioned computer can For reading storage medium embodiment and DAS (Driver Assistant System) embodiment shown in Fig. 8, since it is substantially similar to Fig. 1 a-5 institute The image split-joint method embodiment based on vehicle-mounted monocular camera shown, so being described relatively simple, related place is referring to Fig. 1 a- The part of image split-joint method embodiment shown in 5 based on vehicle-mounted monocular camera illustrates.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the scope of the present invention.It is all Any modification, equivalent replacement, improvement and so within the spirit and principles in the present invention, are all contained in protection scope of the present invention It is interior.

Claims (16)

1. a kind of image split-joint method based on vehicle-mounted monocular camera characterized by comprising
Obtain current frame image to be spliced；
Obtain the Orientation differences information of vehicle between the first acquisition moment and the second acquisition moment；Wherein, when the described first acquisition Carving is the current frame image corresponding acquisition moment, and the second acquisition moment is the previous frame image of the current frame image The corresponding acquisition moment；
According to the Orientation differences information, the first transformation square between the current frame image and the previous frame image is constructed Battle array；
According to first transformation matrix, the current frame image and the previous frame image are spliced.
2. the method according to claim 1, wherein it is described obtain first acquisition the moment with second acquisition the moment it Between vehicle Orientation differences information, comprising:
According to the motion profile of the trace point in image before the predetermined current frame image, first acquisition is predicted The motion profile of trace point between moment and the second acquisition moment；Wherein, trace point is designated position in every frame image Pixel；
According to the motion profile that prediction obtains, vehicle is calculated between the first acquisition moment and the second acquisition moment Relative displacement and relative rotation.
3. according to the method described in claim 2, working as described in building it is characterized in that, described according to the Orientation differences information The first transformation matrix between prior image frame and the previous frame image, comprising:
According to the relative displacement and the relative rotation, the Europe between the current frame image and the previous frame image is constructed Draw matrix.
4. the method according to claim 1, wherein obtaining the first acquisition moment and the second acquisition moment described Between vehicle Orientation differences information before, further includes:
In the previous frame image of the current frame image and the current frame image, characteristic point pair is extracted；
By the characteristic point to being registrated, according to registration result, calculate the current frame image and the previous frame image it Between the second transformation matrix；
Calculate the deviation between second transformation matrix and predetermined history transformation matrix, the history transformation matrix Are as follows: the transformation matrix between the previous frame image and the image of previous frame again of the previous frame image；
Judge whether the deviation is greater than preset threshold；
If it does, executing the step for obtaining the Orientation differences information of vehicle between the first acquisition moment and the second acquisition moment Suddenly；
If it is not greater, the current frame image and the previous frame image are spliced according to second transformation matrix.
5. according to the method described in claim 4, it is characterized in that, determining feelings of the deviation no more than the preset threshold Under condition, the method also includes:
According to second transformation matrix, by the movement rail of the trace point in image before the predetermined current frame image Mark is converted into the coordinate system of the current frame image；Wherein, trace point is the pixel of designated position in every frame image；It is described Motion profile is used to obtain the Orientation differences information of vehicle；
In the coordinate system of the current frame image, the position of the trace point in the current frame image is added to the movement Track；
In the case where determining that the deviation is greater than the preset threshold, described according to the Orientation differences information, institute is constructed After stating the first transformation matrix between current frame image and the previous frame image, the method also includes:
According to first transformation matrix, by the movement rail of the trace point in image before the predetermined current frame image Mark is converted into the coordinate system of the current frame image；
In the coordinate system of the current frame image, the position of the trace point in the current frame image is added to the movement Track.
6., will be described current the method according to claim 1, wherein described according to first transformation matrix Frame image is spliced with the previous frame image, comprising:
According to first transformation matrix, current frame image stitching image corresponding with the previous frame image is spelled It connects, obtains the corresponding stitching image of the current frame image；Wherein, the corresponding stitching image of the previous frame image are as follows: described What the image before previous frame image and the previous frame image was spliced.
7. the method according to claim 1, wherein described obtain current frame image to be spliced, comprising:
Obtain the fish eye images of vehicle-mounted monocular camera acquisition；
According to the parameter of the vehicle-mounted monocular camera, the fish eye images are converted into overhead view image, and by the overhead view image As the current frame image to be spliced.
8. a kind of image splicing device based on vehicle-mounted monocular camera characterized by comprising
First obtains module, for obtaining current frame image to be spliced；
Second obtains module, for obtaining the Orientation differences information of vehicle between the first acquisition moment and the second acquisition moment；Its In, the first acquisition moment is the current frame image corresponding acquisition moment, and the second acquisition moment is described current The previous frame image of the frame image corresponding acquisition moment；
Module is constructed, for constructing between the current frame image and the previous frame image according to the Orientation differences information The first transformation matrix；
First splicing module, for according to first transformation matrix, by the current frame image and the previous frame image into Row splicing.
9. device according to claim 8, which is characterized in that the second acquisition module is specifically used for:
According to the motion profile of the trace point in image before the predetermined current frame image, first acquisition is predicted The motion profile of trace point between moment and the second acquisition moment；Wherein, trace point is designated position in every frame image Pixel；
According to the motion profile that prediction obtains, vehicle is calculated between the first acquisition moment and the second acquisition moment Relative displacement and relative rotation.
10. device according to claim 9, which is characterized in that the building module is specifically used for:
According to the relative displacement and the relative rotation, the Europe between the current frame image and the previous frame image is constructed Draw matrix.
11. device according to claim 8, which is characterized in that described device further include:
Extraction module, for obtaining the side that module obtains vehicle between the first acquisition moment and the second acquisition moment described second Before the change information of position, in the previous frame image of the current frame image and the current frame image, characteristic point pair is extracted；
First computing module, for by the characteristic point to being registrated, according to registration result, calculate the current frame image with The second transformation matrix between the previous frame image；
Second computing module, for calculating the deviation between second transformation matrix and predetermined history transformation matrix, The history transformation matrix are as follows: the transformation square between the previous frame image and the image of previous frame again of the previous frame image Battle array；
Judgment module, for judging whether the deviation is greater than preset threshold；If it does, triggering described second obtains module, If it is not greater, the second splicing module of triggering；
Second splicing module, for according to second transformation matrix, by the current frame image and the previous frame image into Row splicing.
12. device according to claim 11, which is characterized in that described device further include:
First conversion module is used in the case where the judgment module determines that the deviation is not more than the preset threshold, root According to second transformation matrix, the motion profile of the trace point in image before the predetermined current frame image is converted Into the coordinate system of the current frame image, and trigger adding module；Wherein, trace point is the picture of designated position in every frame image Vegetarian refreshments；The motion profile is used to obtain the Orientation differences information of vehicle；
Second conversion module is used in the case where the judgment module determines that the deviation is greater than the preset threshold, in institute Building module is stated according to the Orientation differences information, first constructed between the current frame image and the previous frame image becomes It changes after matrix, according to first transformation matrix, by the trace point in image before the predetermined current frame image Motion profile convert into the coordinate system of the current frame image, and trigger adding module；
The adding module, in the coordinate system of the current frame image, by the trace point in the current frame image Position is added to the motion profile.
13. device according to claim 8, which is characterized in that first splicing module is specifically used for:
According to first transformation matrix, current frame image stitching image corresponding with the previous frame image is spelled It connects, obtains the corresponding stitching image of the current frame image；Wherein, the corresponding stitching image of the previous frame image are as follows: described What the image before previous frame image and the previous frame image was spliced.
14. device according to claim 8, which is characterized in that described first obtains module, is specifically used for:
Obtain the fish eye images of vehicle-mounted monocular camera acquisition；
According to the parameter of the vehicle-mounted monocular camera, the fish eye images are converted into overhead view image, and by the overhead view image As the current frame image to be spliced.
15. a kind of electronic equipment, which is characterized in that including processor, communication interface, memory and communication bus, wherein processing Device, communication interface, memory complete mutual communication by communication bus；
Memory, for storing computer program；
Processor when for executing the program stored on memory, realizes method and step as claimed in claim 1 to 7.
16. a kind of DAS (Driver Assistant System) characterized by comprising monocular camera and processing equipment；Wherein,
The monocular camera, for collected each current frame image to be sent to the processing equipment；
The processing equipment, for receiving the current frame image；Obtain vehicle between the first acquisition moment and the second acquisition moment Orientation differences information；Wherein, it is described first acquisition the moment be the current frame image corresponding acquisition moment, described second Acquire the previous frame image corresponding acquisition moment that the moment is the current frame image；According to the Orientation differences information, building The first transformation matrix between the current frame image and the previous frame image；It, will be described according to first transformation matrix Current frame image is spliced with the previous frame image.
CN201810501402.8A 2018-05-23 2018-05-23 Image stitching method, device, equipment and system based on vehicle-mounted monocular camera Active CN110533586B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810501402.8A CN110533586B (en) 2018-05-23 2018-05-23 Image stitching method, device, equipment and system based on vehicle-mounted monocular camera

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810501402.8A CN110533586B (en) 2018-05-23 2018-05-23 Image stitching method, device, equipment and system based on vehicle-mounted monocular camera

Publications (2)

Publication Number Publication Date
CN110533586A true CN110533586A (en) 2019-12-03
CN110533586B CN110533586B (en) 2023-02-07

Family

ID=68656414

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810501402.8A Active CN110533586B (en) 2018-05-23 2018-05-23 Image stitching method, device, equipment and system based on vehicle-mounted monocular camera

Country Status (1)

Country Link
CN (1) CN110533586B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112648924A (en) * 2020-12-15 2021-04-13 广州小鹏自动驾驶科技有限公司 Suspended object space position determination method based on vehicle-mounted monocular camera equipment

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106817539A (en) * 2016-12-29 2017-06-09 珠海市魅族科技有限公司 The image processing method and system of a kind of vehicle
CN106910217A (en) * 2017-03-17 2017-06-30 驭势科技（北京）有限公司 Vision map method for building up, computing device, computer-readable storage medium and intelligent vehicle
CN107341787A (en) * 2017-07-26 2017-11-10 珠海研果科技有限公司 Method, apparatus, server and the automobile that monocular panorama is parked

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106817539A (en) * 2016-12-29 2017-06-09 珠海市魅族科技有限公司 The image processing method and system of a kind of vehicle
CN106910217A (en) * 2017-03-17 2017-06-30 驭势科技（北京）有限公司 Vision map method for building up, computing device, computer-readable storage medium and intelligent vehicle
CN107341787A (en) * 2017-07-26 2017-11-10 珠海研果科技有限公司 Method, apparatus, server and the automobile that monocular panorama is parked

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
陆天舒等: "基于图像拼接的全景目标检测技术", 《兵工自动化》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112648924A (en) * 2020-12-15 2021-04-13 广州小鹏自动驾驶科技有限公司 Suspended object space position determination method based on vehicle-mounted monocular camera equipment

Also Published As

Publication number Publication date
CN110533586B (en) 2023-02-07

Similar Documents

Publication Publication Date Title
JP7164706B2 (en) Image stitching method and device, in-vehicle image processing device, electronic device, storage medium
GB2593335A (en) Method and apparatus for 3-D auto tagging
EP2081149B1 (en) Weighted average image blending based on relative pixel position
EP2640057A1 (en) Image processing device, image processing method and program
US20070211955A1 (en) Perspective correction panning method for wide-angle image
US20120219190A1 (en) Image processing apparatus, image processing system, and image processing method
CN106997579B (en) Image splicing method and device
CN110288527B (en) Panoramic aerial view generation method of vehicle-mounted panoramic camera
CN106447602A (en) Image mosaic method and device
CN111105347B (en) Method, device and storage medium for generating panoramic image with depth information
CN113029128B (en) Visual navigation method and related device, mobile terminal and storage medium
CN112001912B (en) Target detection method and device, computer system and readable storage medium
JP6570904B2 (en) Correction information output apparatus, image processing apparatus, correction information output method, imaging control system, and moving body control system
US10687044B2 (en) Method and arrangement for calibration of cameras
CN105516579A (en) Image processing method and device and electronic equipment
JP4527638B2 (en) Wide-area surveillance panorama system
CN102081796B (en) Image splicing method and device thereof
CN111738071A (en) Inverse perspective transformation method based on movement change of monocular camera
CN110533586A (en) Image split-joint method, device, equipment and system based on vehicle-mounted monocular camera
EP3701490B1 (en) Method and system of fast image blending for overlapping regions in surround view
CN114821544B (en) Perception information generation method and device, vehicle, electronic equipment and storage medium
US11494934B2 (en) Image processing device, image processing method, and monitoring system
JP2005141655A (en) Three-dimensional modeling apparatus and three-dimensional modeling method
JP2006215657A (en) Method, apparatus, program and program storage medium for detecting motion vector
CN114445455A (en) Blackboard tracking method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant