CN106303283A - A kind of panoramic image synthesis method based on fish-eye camera and system - Google Patents

A kind of panoramic image synthesis method based on fish-eye camera and system Download PDF

Info

Publication number
CN106303283A
CN106303283A CN201610668263.9A CN201610668263A CN106303283A CN 106303283 A CN106303283 A CN 106303283A CN 201610668263 A CN201610668263 A CN 201610668263A CN 106303283 A CN106303283 A CN 106303283A
Authority
CN
China
Prior art keywords
fish
correcting image
eye camera
image
parameter
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201610668263.9A
Other languages
Chinese (zh)
Inventor
火焰
赖长明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
TCL Corp
Original Assignee
TCL Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by TCL Corp filed Critical TCL Corp
Priority to CN201610668263.9A priority Critical patent/CN106303283A/en
Publication of CN106303283A publication Critical patent/CN106303283A/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/90Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/698Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80Camera processing pipelines; Components thereof

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a kind of panoramic image synthesis method based on fish-eye camera and system, method includes: be oppositely arranged by two fish-eye cameras in advance, obtains the first original image of two fish-eye camera sync pulse jamming and the second original image respectively and stores;Respectively the first original image and the second original image are corrected respectively, generate the first correcting image after correcting and the second correcting image;After splicing the first correcting image and the second correcting image and being corrected, generate the 3rd corresponding correcting image and the 4th correcting image;By the overlapping region of the 3rd correcting image and the 4th correcting image according to distance weighted additive fusion, generate the panoramic picture after weighted superposition merges.The present invention realizes the splicing of 360 degree of images, the minimizing of photographic head number by two big Radix Rumicis fish-eye cameras, and the difficulty of demarcating calibration can be greatly lowered, and operand ratio is relatively low, can in embedded device real-time implementation, splicing regions transition is natural.

Description

A kind of panoramic image synthesis method based on fish-eye camera and system
Technical field
The present invention relates to technical field of image processing, particularly relate to a kind of panoramic picture synthesis side based on fish-eye camera Method and system.
Background technology
Now in virtual reality technology, critical component is exactly the synthesis of nature panoramic picture, existing frequently-used undistorted takes the photograph As the visual field of head is usually about 60 degree, in order to synthesize the panoramic picture of 360 degree, multiple photographic head is needed to be combined into shooting head shoot Face matrix shoots, and the most also has the scheme using Radix Rumicis fish-eye camera to carry out pan-shot, and photographic head number can be significantly Degree reduces, and cost decreases.
According to the method for photographic head sphere matrix, generally there is cost high, the problem of the accurate Synchronization Control of multi-cam. In addition to 360 degree of panoramic pictures of synthesis, multiple photographic head need to demarcate calibration, and batch production has many difficulty, spells image The requirement of real time connecing algorithm is the highest, it is impossible to do real-time implementation on the embedded platform of low cost.
Therefore, prior art has yet to be improved and developed.
Summary of the invention
In view of the deficiencies in the prior art, present invention aim at providing a kind of panoramic picture based on fish-eye camera to synthesize Method and system, it is intended to solve multi-cam cost in prior art high, it is impossible to precisely control, multiple photographic head need to demarcate school Accurate, it is impossible to the problem realizing batch production.
Technical scheme is as follows:
A kind of panoramic image synthesis method based on fish-eye camera, wherein, method includes:
A, in advance visual angle is oppositely arranged more than two fish-eye cameras of a predetermined angular, obtains respectively and furnish in opposite directions The first original image of two fish-eye camera sync pulse jamming and the second original image storing;
First original image and the second original image are carried out by the parameter of the fish-eye camera that B, basis are preset respectively respectively Correct, generate the first correcting image after correcting and the second correcting image;
C, splice the first correcting image and the second correcting image, adjust the parameter of fish-eye camera to the first correcting image and After second correcting image is corrected, generate the 3rd corresponding correcting image and the 4th correcting image;
D, by the overlapping region of the 3rd correcting image and the 4th correcting image according to distance weighted additive fusion, generate weighting Panoramic picture after additive fusion.
Described panoramic image synthesis method based on fish-eye camera, wherein, two fish-eye cameras are designated as respectively One fish-eye camera, the second fish-eye camera, described C specifically includes:
C1, obtain the point that in the first correcting image and the second correcting image, pixel is identical and carry out image mosaic;
C2, fix the projective parameter of the second fish-eye camera, adjust the projective parameter of the first fish-eye camera until first Between correcting image and the second correcting image splicing regions, error is minimum;
The projective parameter of the first fish-eye camera after C3, fixed adjustment, adjusts the projective parameter of the second fish-eye camera Until error is minimum between the first correcting image and the second correcting image splicing regions;
C4, according to adjust after projective parameter respectively the first correcting image and the second correcting image are corrected, generate The 3rd corresponding correcting image and the 4th correcting image.
Panoramic image synthesis method based on fish-eye camera described in any one, wherein, the ginseng of described fish-eye camera Number includes focal length, visual angle, the anglec of rotation, center point coordinate, radial distortion parameter and tangential distortion parameter.
Described panoramic image synthesis method based on fish-eye camera, wherein, described C2 specifically includes:
C21, obtain respectively the radial distortion parameter of the first fish-eye camera, tangential distortion parameter and the anglec of rotation, focal length, Center point coordinate;
C22, obtain the radial distortion parameter of the first fish-eye camera, tangential distortion parameter and the anglec of rotation, focal length, center A parameter in point coordinates, fixes other parameters of the first fish-eye camera, fixes the projection ginseng of the second fish-eye camera Number, adjusts the parameter obtained minimum until error between the first correcting image and the second correcting image splicing regions;
C23, it is sequentially adjusted in other parameters of the first fish-eye camera, repeats step C22, until the first fish-eye camera Radial distortion parameter, tangential distortion parameter and the anglec of rotation, focal length, center point coordinate all adjust complete.
Described panoramic image synthesis method based on fish-eye camera, wherein, described D specifically includes:
Pixel in D1, the overlapping region obtained in the 3rd correcting image and the 4th correcting image is corrected the 3rd respectively The first pixel value in image and the second pixel value in the 4th correcting image;
D2, calculate the first pixel value and the second pixel value weighted average after, generate the pixel value of the pixel after superposition;
D3, repetition step D1 and D2, until after the superposition pixel value of all pixels of overlap-add region has all calculated, Generate the anastomosing and splicing result of overlap-add region;
D4, the non-crossover region obtained in the anastomosing and splicing result of overlap-add region and the 3rd correcting image and the 4th correcting image Territory, generates the panoramic picture after weighted superposition merges.
A kind of panoramic picture synthesis system based on fish-eye camera, wherein, system includes:
Pre-set module, in advance, visual angle is oppositely arranged more than two fish-eye cameras of a predetermined angular, point Do not obtain the first original image of two the fish-eye camera sync pulse jamming furnished in opposite directions and the second original image and store;
Correction module, is used for the parameter according to the fish-eye camera preset respectively to the first original image and the second original graph As correcting respectively, generate the first correcting image after correcting and the second correcting image;
Splicing and correction module, for splicing the first correcting image and the second correcting image, adjust the ginseng of fish-eye camera Several first correcting image and the second correcting image are corrected after, generate the 3rd corresponding correcting image and the 4th and correct figure Picture;
Weighted superposition module, for folding the overlapping region of the 3rd correcting image and the 4th correcting image according to distance weighted Add fusion, generate the panoramic picture after weighted superposition merges.
Described panoramic picture synthesis system based on fish-eye camera, wherein, two fish-eye cameras are designated as respectively One fish-eye camera, the second fish-eye camera, described splicing specifically includes with correction module:
Concatenation unit, carries out image spelling for obtaining the point that in the first correcting image and the second correcting image, pixel is identical Connect;
First adjustment unit, for fixing the projective parameter of the second fish-eye camera, adjusts the throwing of the first fish-eye camera Shadow parameter is minimum until error between the first correcting image and the second correcting image splicing regions;
Second adjustment unit, the projective parameter of the first fish-eye camera after fixed adjustment, adjust the second flake and take the photograph As the projective parameter of head is minimum until error between the first correcting image and the second correcting image splicing regions;
Correction unit, for carrying out the first correcting image and the second correcting image respectively according to the projective parameter after adjusting Correction, generates the 3rd corresponding correcting image and the 4th correcting image.
Panoramic picture synthesis system based on fish-eye camera described in any one, wherein, the ginseng of described fish-eye camera Number includes focal length, visual angle, the anglec of rotation, center point coordinate, radial distortion parameter and tangential distortion parameter.
Described panoramic picture synthesis system based on fish-eye camera, wherein, described first adjustment unit specifically includes:
Parameter acquiring unit, for obtain respectively the radial distortion parameter of the first fish-eye camera, tangential distortion parameter and The anglec of rotation, focal length, center point coordinate;
Single parameter adjustment unit, for obtain the radial distortion parameter of the first fish-eye camera, tangential distortion parameter and A parameter in the anglec of rotation, focal length, center point coordinate, fixes other parameters of the first fish-eye camera, fixes the second fish The projective parameter of Eye imaging head, adjusts the parameter obtained until error between the first correcting image and the second correcting image splicing regions Minimum;
Parameter adjustment unit, for being sequentially adjusted in other parameters of the first fish-eye camera, until the first fish-eye camera Radial distortion parameter, tangential distortion parameter and the anglec of rotation, focal length, center point coordinate all adjust complete.
Described panoramic picture synthesis system based on fish-eye camera, wherein, described weighted superposition module is specifically wrapped Include:
Pixel value acquiring unit, for obtaining the pixel in the overlapping region in the 3rd correcting image and the 4th correcting image Point the first pixel value in the 3rd correcting image and the second pixel value in the 4th correcting image respectively;
Pixel value weighted superposition unit, after calculating the first pixel value and the second pixel value weighted average, generates superposition After the superposition pixel value of pixel;
Overlap-add region concatenation unit, for obtaining the superposition pixel value after the pixel of overlap-add region has all calculated, Superposition pixel value according to all pixels generates the anastomosing and splicing result of overlap-add region;
Panoramic picture signal generating unit, rectifys for the anastomosing and splicing result and the 3rd correcting image and the 4th obtaining overlap-add region Non-overlapping region in positive image, generates the panoramic picture after weighted superposition merges.
The invention provides a kind of panoramic image synthesis method based on fish-eye camera and system, the present invention attempts to pass through Two big Radix Rumicis fish-eye cameras realize the splicing of 360 degree of video images, due to the minimizing of photographic head number, can significantly drop The low difficulty demarcating calibration, improves productibility, and stitching algorithm can be applicable to multiple different flake by parameter adjustment and image Head, operand ratio is relatively low, can in embedded device real-time implementation, seamless natural aliasing method can make splicing regions transition from So.
Accompanying drawing explanation
Fig. 1 is the flow process of the preferred embodiment of a kind of based on fish-eye camera the panoramic image synthesis method of the present invention Figure.
Fig. 2 a is the concrete Application Example of a kind of based on fish-eye camera the panoramic image synthesis method of the present invention The fish-eye camera schematic diagram that two are oppositely arranged.
Fig. 2 b is the concrete Application Example of a kind of based on fish-eye camera the panoramic image synthesis method of the present invention The visual angle schematic diagram of two fish-eye cameras in Fig. 2 a.
Fig. 3 a is the concrete Application Example of a kind of based on fish-eye camera the panoramic image synthesis method of the present invention Spherical coordinate fastens a point, and (x, y, z) in the perspective view of X/Y plane.
Fig. 3 b is the concrete Application Example of a kind of based on fish-eye camera the panoramic image synthesis method of the present invention Spherical coordinate fastens a point, and (x, y, z) in the perspective view of X/Y plane.
Fig. 4 is adding of the concrete Application Example of a kind of based on fish-eye camera the panoramic image synthesis method of the present invention Power weight schematic diagram.
Fig. 5 a is the concrete Application Example of a kind of based on fish-eye camera the panoramic image synthesis method of the present invention The first original image that first fish-eye camera gathers.
Fig. 5 b is the concrete Application Example of a kind of based on fish-eye camera the panoramic image synthesis method of the present invention The second original image that second fish-eye camera gathers.
Fig. 6 a is the concrete Application Example of a kind of based on fish-eye camera the panoramic image synthesis method of the present invention The first correcting image after the first original image rectification that first fish-eye camera gathers.
Fig. 6 b is the concrete Application Example of a kind of based on fish-eye camera the panoramic image synthesis method of the present invention The second correcting image after the second original image rectification that second fish-eye camera gathers.
Fig. 7 is the concrete Application Example first of a kind of based on fish-eye camera the panoramic image synthesis method of the present invention Panoramic picture after correcting image and the fusion of the 3rd correcting image.
Fig. 8 is that the function of the preferred embodiment of a kind of based on fish-eye camera the panoramic picture synthesis system of the present invention is former Reason block diagram.
Detailed description of the invention
For making the purpose of the present invention, technical scheme and effect clearer, clear and definite, below to the present invention the most specifically Bright.Should be appreciated that specific embodiment described herein, only in order to explain the present invention, is not intended to limit the present invention.
The invention provides the flow chart of the preferred embodiment of a kind of panoramic image synthesis method based on fish-eye camera, As it is shown in figure 1, wherein, method includes:
Step S100, in advance visual angle is oppositely arranged more than two fish-eye cameras of a predetermined angular, obtains phase respectively To first original image of two fish-eye camera sync pulse jamming of ornaments and the second original image and store;
The parameter of the fish-eye camera that step S200, basis are preset is respectively to the first original image and the second original image Correct respectively, generate the first correcting image after correcting and the second correcting image;
Step S300, splicing the first correcting image and the second correcting image, the parameter adjusting fish-eye camera is rectified first After positive image and the second correcting image are corrected, generate the 3rd corresponding correcting image and the 4th correcting image;
Step S400, by the overlapping region of the 3rd correcting image and the 4th correcting image according to distance weighted additive fusion, Generate the panoramic picture after weighted superposition merges.
When being embodied as, in step S100, input picture derives from two visual angles (Field of View, FOV) and is more than 180 °, the fish-eye camera furnished in opposite directions.In step S200 the parameter of fish-eye camera include focal length, visual angle, the anglec of rotation, in Heart point coordinates, radial distortion parameter and tangential distortion parameter etc..By maximizing splicing position two width image phase in step S300 Closing property or minimize the difference of splicing position two width image, obtains optimization photographic head projective parameter, and this parameter is fixed up, Obtain projected graph by projection formula, the operand of projection is greatly lowered by tabling look-up, adjust photographic head parameter to input Fish eye images is corrected.In step S400 to splicing overlapping part two width images, be weighted aliasing according to relative distance, Make stitching image transition natural.
When being embodied as, Fig. 3 a is that spherical coordinate is fastened a point (x, y, z) in the perspective view of X/Y plane, from spheroid To point, (x, y, vector z) is θ with the angle of Z axis, we term it the angle of incidence of fish-eye camera, (x, y z) exist at center The coordinate of the subpoint on X/Y plane is (x, y, 0), and from ball centre to point, the vector of (x, y, 0) is α with the angle of X-axis, as Fruit assumes that radius of sphericity is unit 1, and we can obtain:
Z=cos θ
X=sin θ * cos α (formula 1)
Y=sin θ * sin α
Fig. 3 b is that spherical coordinate is fastened this point (x, y, z) perspective view in XZ plane, from ball centre to point (x, y, vector z) is θ ^ ' with the angle of Y-axis, and we are called latitude, (x, y, z) seat of the subpoint in XZ plane Be designated as (x, 0, z), from ball centre to point (x, 0, vector z) is α ^' with the angle of Z axis, if it is assumed that radius of sphericity is single Position 1, we can obtain:
Y=cos θ '
X=sin θ ' * sin α ' (formula 2)
Z=sin θ ' * cos α '
By (formula 1) and (formula 2) we can known (x, y, z) X/Y plane projection (x, y, 0) situation Under, it is thus achieved that (x, y, latitude, longitude numerical value z) put:
α '=tan-1(tg θ * cos α) (formula 3)
Because it is desirable that image mosaic is intended in the case of two width fish eye images all project to sphere latitude, longitude, at ball Splicing on face, so, we have only to, to the point on known latitude, longitude sphere, find the point in its corresponding two width fish eye images Position, its pixel is overlapped averagely, it is possible to obtain final anastomosing and splicing image.From (formula 1) (formula 2) (formula 3) we can be in the case of known spatial point longitude and latitude, it is thus achieved that the incident angle θ of fish-eye camera and angle α:
θ=cos-1(sinθ′*cosα′)
α=ctg-1(tg θ ' * sin α ') (formula 4)
According to (formula 4) and (formula 1), it is possible to obtain three dimensions point X/Y plane coordinate (x, y).(formula l) is false If the projection formula that fish-eye camera obtains in the case of not having radial distortion and tangential distortion, and focal length is unit 1, but real Border situation, owing to the visual angle of fish-eye camera is more than 180 °, certainly exists radial distortion, it is also possible to have multiple different types of Fish-eye camera distortion, enumerates several possible distortion as in (table one), and wherein f represents the focal length of camera lens.
Table 1
In view of the radial distortion of fish-eye camera, and assume there is no tangential distortion, according to (formula 4) and (formula 1), can To obtain the three dimensions point coordinate (x ', y ') at X/Y plane:
R=f*fun (θ, angle)
X '=f*fun (θ, angle) * cos α (formula 5)
Y '=f*fun (θ, angle) * sin α
Wherein fun (θ, angle) is θ and the function of fish-eye camera visual angle (FoV) angle, can be by certain in (table 1) Plant analytic formula to represent, it is also possible to by the polynomial repressentation of θ, as shown in (formula 6).It addition, (formula 1) and (formula 5) is all The optical focus assuming known camera lens is (0,0), and in practice, optical focus is certain point collecting image, our mark The coordinate of this central point is (xcenter,ycenter), and can there is tangential distortion and rotation in camera lens, and wherein tangential distortion is permissible It is expressed as:
distort)=ctg-1(tg θ ' * sin α ')+Δ α (formula 7)
Rotation can be expressed as:
Consider above-mentioned various distortion parameter, it is possible to obtain three dimensions point is at the coordinate (x ", y ") of X/Y plane.
In further embodiment, step S400 specifically includes:
Pixel in step S401, the overlapping region obtained in the 3rd correcting image and the 4th correcting image is respectively the The first pixel value in three correcting images and the second pixel value in the 4th correcting image;
Step S402, calculate the first pixel value and the second pixel value weighted average after, generate the folded of the pixel after superposition Add pixel value;
Step S403, repetition step S401 and step S402, until the superposition pixel value of all pixels of overlap-add region After all having calculated, generate the anastomosing and splicing result of overlap-add region;
Step S404, obtain in the anastomosing and splicing result of overlap-add region and the 3rd correcting image and the 4th correcting image not Overlapping region, generates the panoramic picture after weighted superposition merges.
When being embodied as, by (formula 9), we obtain the point of corresponding spatial dimension θ ', longitude α ' from fish eye images Pixel value P:
P (θ ', α ')=fish (x ", y ") (formula 10)
Wherein fish (x ", y ") represents the respective coordinates (x ", y ") pixel value in flake picture.In the present invention, two Visual angle in opposite directions more than the fish-eye camera of 180 ° have an overlapping region (longitude range of such as overlapping region is- 22.5 °~22.5 °), by (formula 10), we can obtain under corresponding Same Latitude, longitude in space o'clock two width fishes The position of point corresponding in eye pattern picture and pixel value P0, P1, the distance of the central point that they crossed according to distance, to two projections Pixel value P0, P1 of point are weighted averagely, it is possible to obtain the stitching image of overlapping region, as shown in (formula 11):
P (θ ', α ')=w0*P0 (θ ', α ')+w1*P1 (θ ', α ') (formula 11)
Formula 11) in w0 and w1 represent weight, relevant with the distance of spatial point distance joint, if cross point Precision is at 15 °, and on that, the weight of half fish eye images is the biggest, and the weight of lower half fish eye images is the least.Miss until final Difference minimum, also obtains the projective parameter that two width images are optimal simultaneously.
Error=∑θ′∑α′(P0(θ′,α′)-P1(θ′,α′))2(formula 12)
(formula 12) participates in the pixel that error summing junction is all overlapping regions, the camera lens of general selected photographic head After, the size of overlapping region is just fixed up.
Point according to juxtaposition region is fan-shaped near near which, determines the weighting weight in (formula 13), such as (Fig. 4) Shown in, the angle of overlapping region is β, and blue dot and fan-shaped top angle are β 0, then upper half fish eye images and lower half fish The weight of eye pattern picture is respectively as follows:
w 0 = β - β 0 β
In conjunction with (formula 12) and the anastomosing and splicing result that is obtained with intersection region according to (formula 11).
In further embodiment, step S300 specifically includes:
Step S301, obtain the point that in the first correcting image and the second correcting image, pixel is identical and carry out image mosaic;
Step S302, fixing the projective parameter of the second fish-eye camera, the projective parameter adjusting the first fish-eye camera is straight Minimum to error between the first correcting image and the second correcting image splicing regions;
The projective parameter of the first fish-eye camera after step S303, fixed adjustment, adjusts the throwing of the second fish-eye camera Shadow parameter is minimum until error between the first correcting image and the second correcting image splicing regions;
Step S304, according to adjust after projective parameter respectively the first correcting image and the second correcting image are carried out school Just, the 3rd corresponding correcting image and the 4th correcting image are generated.
When being embodied as, according to (formula 5), fix some projective parameters, it is thus achieved that two width fish eye images projections initialize ginseng Number.The projective parameter of fixing fish eye images 1, adjusts the projective parameter of fish eye images 2, including focal length, and image center location, radially Distortion, tangential distortion, the anglec of rotation, minimize the error of (formula 12), the projective parameter of fixing fish eye images 2, adjust flake The projective parameter of image 1, including focal length, image center location, radial distortion, tangential distortion, the anglec of rotation etc., minimizes (public Formula 12) error, until total error is less than predictive error.
In further embodiment, step S302 specifically includes:
Step S321, obtain respectively the radial distortion parameter of the first fish-eye camera, tangential distortion parameter and the anglec of rotation, Focal length, center point coordinate;
Step S322, obtain the radial distortion parameter of the first fish-eye camera, tangential distortion parameter and the anglec of rotation, Jiao A parameter in, center point coordinate, fixes other parameters of the first fish-eye camera, fixes the second fish-eye camera Projective parameter, adjusts the parameter obtained minimum until error between the first correcting image and the second correcting image splicing regions;
Step S323, it is sequentially adjusted in other parameters of the first fish-eye camera, repeats step S322, until the first flake is taken the photograph As the radial distortion parameter of head, tangential distortion parameter and the anglec of rotation, focal length, center point coordinate all adjust complete.
When being embodied as, in further embodiment, by the flake of the two of the present invention fish-eye camera synchronous acquisition Image such as Fig. 5 a, shown in 5b, presets some initial parameter values, and such as fixed focal length, making anglec of rotation initial value is 0, tangential distortion It is 0, fixes certain radial distortion parameter, still image central point (x_center, y_center), obtain ball according to (formula 10) Face correcting image;
According to (formula 12) and the image difference that minimizes intersection region, adjusting photographic head parameter, these parameters include Jiao Away from, image center location coordinate (center_x, center_y), radial distortion, tangential distortion, the anglec of rotation etc., these parameters Adjustment can use the method identical with the image difference minimizing intersection region, i.e. adjust a parameter, fix other ginsengs Number, minimizes error, is then sequentially adjusted in each parameter, until convergence;Obtain the sphere as shown in (Fig. 6 a) and (Fig. 6 b) Correcting image;Point according to juxtaposition region is fan-shaped near near which, determines the weighting weight in (formula 13), according to (formula 11) obtains the anastomosing and splicing result of intersection region, as shown in (Fig. 7).
Present invention also offers the function of the preferred embodiment of a kind of panoramic picture synthesis system based on fish-eye camera Theory diagram, as shown in Figure 8, system includes:
Pre-set module 100, in advance, visual angle be oppositely arranged more than two fish-eye cameras of a predetermined angular, Obtain the first original image of two the fish-eye camera sync pulse jamming furnished in opposite directions and the second original image respectively and store;Tool Body is as described in embodiment of the method.
Correction module 200, former to the first original image and second respectively for the parameter according to the fish-eye camera preset Beginning image is corrected respectively, generates the first correcting image after correcting and the second correcting image;Concrete such as embodiment of the method institute State.
Splicing and correction module 300, for splicing the first correcting image and the second correcting image, adjust fish-eye camera After first correcting image and the second correcting image are corrected by parameter, generate the 3rd corresponding correcting image and the 4th and correct Image;Specifically as described in embodiment of the method.
Weighted superposition module 400, for adding the overlapping region of the 3rd correcting image and the 4th correcting image according to distance Power additive fusion, generates the panoramic picture after weighted superposition merges;Specifically as described in embodiment of the method.
Further, described weighted superposition module specifically includes:
Pixel value acquiring unit, for obtaining the pixel in the overlapping region in the 3rd correcting image and the 4th correcting image Point the first pixel value in the 3rd correcting image and the second pixel value in the 4th correcting image respectively;Specifically as real in method Execute described in example.
Pixel value weighted superposition unit, after calculating the first pixel value and the second pixel value weighted average, generates superposition After the superposition pixel value of pixel;Specifically as described in embodiment of the method.
Overlap-add region concatenation unit, for obtaining the superposition pixel value after the pixel of overlap-add region has all calculated, Superposition pixel value according to all pixels generates the anastomosing and splicing result of overlap-add region;Specifically as described in embodiment of the method.
Panoramic picture signal generating unit, rectifys for the anastomosing and splicing result and the 3rd correcting image and the 4th obtaining overlap-add region Non-overlapping region in positive image, generates the panoramic picture after weighted superposition merges;Specifically as described in embodiment of the method.
Further, two fish-eye cameras are designated as the first fish-eye camera, the second fish-eye camera, described splicing respectively Specifically include with correction module:
Concatenation unit, carries out image spelling for obtaining the point that in the first correcting image and the second correcting image, pixel is identical Connect;Specifically as described in embodiment of the method.
First adjustment unit, for fixing the projective parameter of the second fish-eye camera, adjusts the throwing of the first fish-eye camera Shadow parameter is minimum until error between the first correcting image and the second correcting image splicing regions;Specifically as described in embodiment of the method.
Second adjustment unit, the projective parameter of the first fish-eye camera after fixed adjustment, adjust the second flake and take the photograph As the projective parameter of head is minimum until error between the first correcting image and the second correcting image splicing regions;Specifically implement such as method Described in example.
Correction unit, for carrying out the first correcting image and the second correcting image respectively according to the projective parameter after adjusting Correction, generates the 3rd corresponding correcting image and the 4th correcting image;Specifically as described in embodiment of the method.
Further, the parameter of described fish-eye camera includes focal length, visual angle, the anglec of rotation, center point coordinate, radially loses True parameter and tangential distortion parameter;Specifically as described in embodiment of the method.
Further, described first adjustment unit specifically includes:
Parameter acquiring unit, for obtain respectively the radial distortion parameter of the first fish-eye camera, tangential distortion parameter and The anglec of rotation, focal length, center point coordinate;Specifically as described in embodiment of the method.
Single parameter adjustment unit, for obtain the radial distortion parameter of the first fish-eye camera, tangential distortion parameter and A parameter in the anglec of rotation, focal length, center point coordinate, fixes other parameters of the first fish-eye camera, fixes the second fish The projective parameter of Eye imaging head, adjusts the parameter obtained until error between the first correcting image and the second correcting image splicing regions Minimum;Specifically as described in embodiment of the method.
Parameter adjustment unit, for being sequentially adjusted in other parameters of the first fish-eye camera, until the first fish-eye camera Radial distortion parameter, tangential distortion parameter and the anglec of rotation, focal length, center point coordinate all adjust complete;Concrete such as method Described in embodiment.
In sum, the invention provides a kind of panoramic image synthesis method based on fish-eye camera and system, method Including: in advance two fish-eye cameras are oppositely arranged, obtain the first original graph of two fish-eye camera sync pulse jamming respectively Picture and the second original image also store;Respectively the first original image and the second original image are corrected respectively, generate and correct After the first correcting image and the second correcting image;After splicing the first correcting image and the second correcting image and being corrected, raw Become the 3rd corresponding correcting image and the 4th correcting image;By the overlapping region of the 3rd correcting image and the 4th correcting image according to Distance weighted additive fusion, generates the panoramic picture after weighted superposition merges.The present invention passes through two big Radix Rumicis fish-eye cameras Realizing the splicing of 360 degree of images, the minimizing of photographic head number, the difficulty of demarcating calibration can be greatly lowered, operand compares Low, can in embedded device real-time implementation, splicing regions transition is natural.
It should be appreciated that the application of the present invention is not limited to above-mentioned citing, for those of ordinary skills, can To be improved according to the above description or to convert, all these modifications and variations all should belong to the guarantor of claims of the present invention Protect scope.

Claims (10)

1. a panoramic image synthesis method based on fish-eye camera, it is characterised in that described method includes:
A, in advance visual angle is oppositely arranged more than two fish-eye cameras of a predetermined angular, obtains two furnished in opposite directions respectively First original image of fish-eye camera sync pulse jamming and the second original image also store;
First original image and the second original image are corrected by the parameter of the fish-eye camera that B, basis are preset respectively, generate The first correcting image after rectification and the second correcting image;
C, splice the first correcting image and the second correcting image, adjust the parameter of fish-eye camera to the first correcting image and second After correcting image is corrected, generate the 3rd corresponding correcting image and the 4th correcting image;
D, by the overlapping region of the 3rd correcting image and the 4th correcting image according to distance weighted additive fusion, generate weighted superposition Panoramic picture after fusion.
Panoramic image synthesis method based on fish-eye camera the most according to claim 1, it is characterised in that two flakes Photographic head is designated as the first fish-eye camera, the second fish-eye camera respectively, and described C specifically includes:
C1, obtain the point that in the first correcting image and the second correcting image, pixel is identical and carry out image mosaic;
C2, fix the projective parameter of the second fish-eye camera, adjust the projective parameter of the first fish-eye camera until first corrects Between image and the second correcting image splicing regions, error is minimum;
The projective parameter of the first fish-eye camera after C3, fixed adjustment, adjust the second fish-eye camera projective parameter until Between the first correcting image and the second correcting image splicing regions, error is minimum;
C4, according to adjust after projective parameter respectively the first correcting image and the second correcting image are corrected, generate correspondence The 3rd correcting image and the 4th correcting image.
3., according to the panoramic image synthesis method based on fish-eye camera described in any one of claim 1~2, its feature exists In, the parameter of described fish-eye camera includes focal length, visual angle, the anglec of rotation, center point coordinate, radial distortion parameter and tangentially loses True parameter.
Panoramic image synthesis method based on fish-eye camera the most according to claim 3, it is characterised in that described C2 has Body includes:
C21, obtain the radial distortion parameter of the first fish-eye camera, tangential distortion parameter and the anglec of rotation, focal length, center respectively Point coordinates;
C22, obtain the radial distortion parameter of the first fish-eye camera, tangential distortion parameter and the anglec of rotation, focal length, central point sit A parameter in mark, fixes other parameters of the first fish-eye camera, fixes the projective parameter of the second fish-eye camera, adjusts The parameter obtained is minimum until error between the first correcting image and the second correcting image splicing regions;
C23, it is sequentially adjusted in other parameters of the first fish-eye camera, repeats step C22, until the radial direction of the first fish-eye camera Distortion parameter, tangential distortion parameter and the anglec of rotation, focal length, center point coordinate all adjust complete.
Panoramic image synthesis method based on fish-eye camera the most according to claim 4, it is characterised in that described D has Body includes:
Pixel in D1, the overlapping region obtained in the 3rd correcting image and the 4th correcting image is respectively at the 3rd correcting image In the first pixel value and the second pixel value in the 4th correcting image;
D2, calculate the first pixel value and the second pixel value weighted average after, generate the pixel value of the pixel after superposition;
D3, repetition step D1 and D2, until after the superposition pixel value of all pixels of overlap-add region has all calculated, generating The anastomosing and splicing result of overlap-add region;
D4, the non-overlapping region obtained in the anastomosing and splicing result of overlap-add region and the 3rd correcting image and the 4th correcting image, Generate the panoramic picture after weighted superposition merges.
6. a panoramic picture synthesis system based on fish-eye camera, it is characterised in that system includes:
Pre-set module, in advance, visual angle is oppositely arranged more than two fish-eye cameras of a predetermined angular, obtains respectively Take the first original image of two the fish-eye camera sync pulse jamming furnished in opposite directions and the second original image and store;
Correction module, for dividing the first original image and the second original image respectively according to the parameter of the fish-eye camera preset Do not correct, generate the first correcting image after correcting and the second correcting image;
Splicing and correction module, for splicing the first correcting image and the second correcting image, adjust the parameter pair of fish-eye camera After first correcting image and the second correcting image are corrected, generate the 3rd corresponding correcting image and the 4th correcting image;
Weighted superposition module, for melting the overlapping region of the 3rd correcting image and the 4th correcting image according to distance weighted superposition Close, generate the panoramic picture after weighted superposition merges.
Panoramic picture synthesis system based on fish-eye camera the most according to claim 6, it is characterised in that two flakes Photographic head is designated as the first fish-eye camera, the second fish-eye camera respectively, and described splicing specifically includes with correction module:
Concatenation unit, carries out image mosaic for obtaining the point that in the first correcting image and the second correcting image, pixel is identical;
First adjustment unit, for fixing the projective parameter of the second fish-eye camera, adjusts the projection ginseng of the first fish-eye camera Number is minimum until error between the first correcting image and the second correcting image splicing regions;
Second adjustment unit, the projective parameter of the first fish-eye camera after fixed adjustment, adjust the second fish-eye camera Projective parameter until between the first correcting image and the second correcting image splicing regions error minimum;
Correction unit, for carrying out school to the first correcting image and the second correcting image respectively according to the projective parameter after adjusting Just, the 3rd corresponding correcting image and the 4th correcting image are generated.
8., according to the panoramic picture synthesis system based on fish-eye camera described in any one of claim 6~7, its feature exists In, the parameter of described fish-eye camera includes focal length, visual angle, the anglec of rotation, center point coordinate, radial distortion parameter and tangentially loses True parameter.
Panoramic picture synthesis system based on fish-eye camera the most according to claim 8, it is characterised in that described first Adjustment unit specifically includes:
Parameter acquiring unit, for obtaining the radial distortion parameter of the first fish-eye camera, tangential distortion parameter and rotation respectively Angle, focal length, center point coordinate;
Single parameter adjustment unit, for obtaining the radial distortion parameter of the first fish-eye camera, tangential distortion parameter and rotation A parameter in angle, focal length, center point coordinate, fixes other parameters of the first fish-eye camera, fixes the second flake and takes the photograph As the projective parameter of head, adjust the parameter obtained until between the first correcting image and the second correcting image splicing regions error is Little;
Parameter adjustment unit, for being sequentially adjusted in other parameters of the first fish-eye camera, until the footpath of the first fish-eye camera All adjust complete to distortion parameter, tangential distortion parameter and the anglec of rotation, focal length, center point coordinate.
Panoramic picture synthesis system based on fish-eye camera the most according to claim 9, it is characterised in that described in add Power laminating module specifically includes:
Pixel value acquiring unit, divides for obtaining the pixel in the overlapping region in the 3rd correcting image and the 4th correcting image Not the first pixel value in the 3rd correcting image and the second pixel value in the 4th correcting image;
Pixel value weighted superposition unit, after calculating the first pixel value and the second pixel value weighted average, after generating superposition The superposition pixel value of pixel;
Overlap-add region concatenation unit, for obtaining the superposition pixel value after the pixel of overlap-add region has all calculated, according to The superposition pixel value of all pixels generates the anastomosing and splicing result of overlap-add region;Panoramic picture signal generating unit, is used for obtaining folded Add the non-overlapping region in the anastomosing and splicing result in region and the 3rd correcting image and the 4th correcting image, generate weighted superposition and melt Panoramic picture after conjunction.
CN201610668263.9A 2016-08-15 2016-08-15 A kind of panoramic image synthesis method based on fish-eye camera and system Pending CN106303283A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610668263.9A CN106303283A (en) 2016-08-15 2016-08-15 A kind of panoramic image synthesis method based on fish-eye camera and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610668263.9A CN106303283A (en) 2016-08-15 2016-08-15 A kind of panoramic image synthesis method based on fish-eye camera and system

Publications (1)

Publication Number Publication Date
CN106303283A true CN106303283A (en) 2017-01-04

Family

ID=57670931

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610668263.9A Pending CN106303283A (en) 2016-08-15 2016-08-15 A kind of panoramic image synthesis method based on fish-eye camera and system

Country Status (1)

Country Link
CN (1) CN106303283A (en)

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106875339A (en) * 2017-02-22 2017-06-20 长沙全度影像科技有限公司 A kind of fish eye images joining method based on strip scaling board
CN107155017A (en) * 2017-06-30 2017-09-12 上海兆芯集成电路有限公司 Camera bearing calibration and the device using this method
CN107248137A (en) * 2017-04-27 2017-10-13 努比亚技术有限公司 A kind of method and mobile terminal for realizing image procossing
CN107424118A (en) * 2017-03-28 2017-12-01 天津大学 Based on the spherical panorama mosaic method for improving Lens Distortion Correction
CN107705252A (en) * 2017-10-13 2018-02-16 华东理工大学 Splice the method and system of expansion correction suitable for binocular fish eye images
CN107995421A (en) * 2017-11-30 2018-05-04 潍坊歌尔电子有限公司 A kind of panorama camera and its image generating method, system, equipment, storage medium
CN108632504A (en) * 2017-03-15 2018-10-09 致伸科技股份有限公司 multi-lens optical device
CN109166152A (en) * 2018-07-27 2019-01-08 深圳六滴科技有限公司 Bearing calibration, system, computer equipment and the storage medium of panorama camera calibration
CN109727292A (en) * 2018-12-29 2019-05-07 哈尔滨拓博科技有限公司 Based on multi-cam-projector interactive projection system and automation scaling method
WO2019091391A1 (en) * 2017-11-09 2019-05-16 Zhejiang Dahua Technology Co., Ltd. Systems and methods for generating panorama image
CN110019886A (en) * 2017-08-28 2019-07-16 富泰华工业(深圳)有限公司 Full-view image generating means and method
CN110622501A (en) * 2017-03-14 2019-12-27 株式会社尼康 Image processing apparatus and electronic device
CN111062881A (en) * 2019-11-20 2020-04-24 RealMe重庆移动通信有限公司 Image processing method and device, storage medium and electronic equipment
CN111161148A (en) * 2019-12-30 2020-05-15 上海闻泰电子科技有限公司 Panoramic image generation method, device, equipment and storage medium
CN114449172A (en) * 2022-02-14 2022-05-06 维沃移动通信有限公司 Shooting method and device and electronic equipment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101082766A (en) * 2006-06-01 2007-12-05 上海杰图软件技术有限公司 Device and method rapid capturing panoramic view image
CN102222337A (en) * 2011-06-14 2011-10-19 重庆大学 Fisheye image correcting method for image stitching
CN104282010A (en) * 2013-07-06 2015-01-14 南京车联网研究院有限公司 Vehicle multi-fisheye-camera 360-degree overhead-view image splicing curve calibration method
CN104835118A (en) * 2015-06-04 2015-08-12 浙江得图网络有限公司 Method for acquiring panorama image by using two fish-eye camera lenses
CN104851076A (en) * 2015-05-27 2015-08-19 武汉理工大学 Panoramic 360-degree-view parking auxiliary system for commercial vehicle and pick-up head installation method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101082766A (en) * 2006-06-01 2007-12-05 上海杰图软件技术有限公司 Device and method rapid capturing panoramic view image
CN102222337A (en) * 2011-06-14 2011-10-19 重庆大学 Fisheye image correcting method for image stitching
CN104282010A (en) * 2013-07-06 2015-01-14 南京车联网研究院有限公司 Vehicle multi-fisheye-camera 360-degree overhead-view image splicing curve calibration method
CN104851076A (en) * 2015-05-27 2015-08-19 武汉理工大学 Panoramic 360-degree-view parking auxiliary system for commercial vehicle and pick-up head installation method
CN104835118A (en) * 2015-06-04 2015-08-12 浙江得图网络有限公司 Method for acquiring panorama image by using two fish-eye camera lenses

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106875339A (en) * 2017-02-22 2017-06-20 长沙全度影像科技有限公司 A kind of fish eye images joining method based on strip scaling board
CN106875339B (en) * 2017-02-22 2020-03-27 长沙全度影像科技有限公司 Fisheye image splicing method based on strip-shaped calibration plate
CN110622501A (en) * 2017-03-14 2019-12-27 株式会社尼康 Image processing apparatus and electronic device
CN110622501B (en) * 2017-03-14 2021-10-01 株式会社尼康 Image processing apparatus and electronic device
CN108632504A (en) * 2017-03-15 2018-10-09 致伸科技股份有限公司 multi-lens optical device
CN107424118A (en) * 2017-03-28 2017-12-01 天津大学 Based on the spherical panorama mosaic method for improving Lens Distortion Correction
CN107248137B (en) * 2017-04-27 2021-01-15 努比亚技术有限公司 Method for realizing image processing and mobile terminal
CN107248137A (en) * 2017-04-27 2017-10-13 努比亚技术有限公司 A kind of method and mobile terminal for realizing image procossing
CN107155017A (en) * 2017-06-30 2017-09-12 上海兆芯集成电路有限公司 Camera bearing calibration and the device using this method
CN107155017B (en) * 2017-06-30 2021-01-22 上海兆芯集成电路有限公司 Image acquisition method and device
CN110019886A (en) * 2017-08-28 2019-07-16 富泰华工业(深圳)有限公司 Full-view image generating means and method
CN107705252A (en) * 2017-10-13 2018-02-16 华东理工大学 Splice the method and system of expansion correction suitable for binocular fish eye images
CN107705252B (en) * 2017-10-13 2021-04-02 华东理工大学 Method and system suitable for splicing, unfolding and correcting binocular fisheye image
US11551330B2 (en) 2017-11-09 2023-01-10 Zhejiang Dahua Technology Co., Ltd. Systems and methods for generating panorama image
WO2019091391A1 (en) * 2017-11-09 2019-05-16 Zhejiang Dahua Technology Co., Ltd. Systems and methods for generating panorama image
CN107995421A (en) * 2017-11-30 2018-05-04 潍坊歌尔电子有限公司 A kind of panorama camera and its image generating method, system, equipment, storage medium
CN109166152A (en) * 2018-07-27 2019-01-08 深圳六滴科技有限公司 Bearing calibration, system, computer equipment and the storage medium of panorama camera calibration
CN109727292B (en) * 2018-12-29 2020-09-08 哈尔滨拓博科技有限公司 Interactive projection system based on multiple cameras and projector and automatic calibration method
CN109727292A (en) * 2018-12-29 2019-05-07 哈尔滨拓博科技有限公司 Based on multi-cam-projector interactive projection system and automation scaling method
CN111062881A (en) * 2019-11-20 2020-04-24 RealMe重庆移动通信有限公司 Image processing method and device, storage medium and electronic equipment
CN111161148A (en) * 2019-12-30 2020-05-15 上海闻泰电子科技有限公司 Panoramic image generation method, device, equipment and storage medium
CN111161148B (en) * 2019-12-30 2024-01-30 上海闻泰电子科技有限公司 Panoramic image generation method, device, equipment and storage medium
CN114449172A (en) * 2022-02-14 2022-05-06 维沃移动通信有限公司 Shooting method and device and electronic equipment

Similar Documents

Publication Publication Date Title
CN106303283A (en) A kind of panoramic image synthesis method based on fish-eye camera and system
TWI397317B (en) Method for providing output image in either cylindrical mode or perspective mode
EP1310091B1 (en) Fast digital pan tilt zoom video system
ES2326965T3 (en) METHOD TO CAPTURE A PANORAMIC IMAGE THROUGH A RECTANGULAR IMAGE RECEIVER.
US8767037B2 (en) Systems and methods for panoramic imaging
US9686479B2 (en) Method for combining multiple image fields
KR101915729B1 (en) Apparatus and Method for Generating 360 degree omni-directional view
CN111199528A (en) Fisheye image distortion correction method
CN104246795A (en) Method and system for adaptive perspective correction of ultra wide-angle lens images
RU2698479C2 (en) Shooting system, method and applications
CN108364252A (en) A kind of correction of more fish eye lens panorama cameras and scaling method
CN105324791A (en) Coordinate computation device and method, and image processing device and method
CN103295231A (en) Method for geometrically correcting vertically mapped images of fisheye lenses in fisheye image mosaic
CN106204496A (en) A kind of fish eye images processing method and processing device of cradle head camera
US11137582B2 (en) Omnidirectional catadioptric lens with odd aspheric contour or multi-lens
CN108205236A (en) Panoramic camera and its camera lens
CN106530356B (en) A kind of panoramic video camera shooting unit spatial pose scaling method
CN108282606B (en) Panoramic stitching method and equipment
CN104252097B (en) 360-degree panoramic cylindrical object imaging mirror design method, mirror and imaging device
KR101789715B1 (en) A Panoramic camera with improved structure

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20170104

RJ01 Rejection of invention patent application after publication