CN106204427A - A kind of method and system generating panoramic picture based on Bayer image - Google Patents

Abstract

The invention discloses a kind of method and system generating panoramic picture based on Bayer image, be directly based upon Bayer image and realize the generation of panoramic picture, reach keep initial data precision and avoid the middle effect converting link.Its technical scheme is: first gathered multiple raw Bayer image by image capture module, is then passed through Bayer panorama generation module mode based on projection fusion and completes the raw Bayer image conversion to Bayer panoramic picture.

Description

A kind of method and system generating panoramic picture based on Bayer image

Technical field

The present invention relates to Integral Imaging, particularly relate to a kind of Panoramagram generation method based on Bayer image and be System.

Background technology

Bayer format image is raw image format in camera, and each pixel remains photosensitive redness, green or blue Component.Bayer image typically has 12 and 14 storages, remains CMOS or light source that ccd image induction apparatus will capture Signal is converted into the initial data of digital signal.Cmos image sensor obtains the colored letter of image by color filter array Breath, only preserves a kind of color component at each pixel.

The tradition input data of panoramic picture are RGB image, and the panoramic picture of output is the most also RGB data.Work as use When Bayer data do split, need to be converted to high-precision Bayer data the RGB data of 8, on the one hand lose in the transfer Lose pixel precision, on the one hand need extra conversion process.

Summary of the invention

The brief overview of one or more aspect given below is to provide the basic comprehension in terms of these.This general introduction is not The extensive overview of all aspects contemplated, and the key or decisive key element being both not intended to identify all aspects is the most non- Attempt to define the scope in terms of any or all.Its unique purpose is intended to provide the one of one or more aspect in simplified form A little concepts think the sequence of more detailed description given later.

It is an object of the invention to solve the problems referred to above, it is provided that a kind of side generating panoramic picture based on Bayer image Method and system, be directly based upon Bayer image and realize the generation of panoramic picture, reaches keep initial data precision and avoid middle turning Change the effect of link.

The technical scheme is that and present invention is disclosed a kind of method generating panoramic picture based on Bayer image, bag Include:

Gather raw Bayer image；

The mode merged based on projection completes the raw Bayer image conversion to Bayer panoramic picture.

One embodiment of the method generating panoramic picture based on Bayer image according to the present invention, completes Bayer panorama After the conversion of image, also Bayer panoramic picture is processed, be converted to the image pane including RGB or YUV through data Formula.

One embodiment of the method generating panoramic picture based on Bayer image according to the present invention, the mode that projection is merged Specifically include:

Projective parameter is generated based on raw Bayer image；

Look-up table is generated according to projective parameter；

Look-up table is utilized to generate Bayer panoramic picture.

One embodiment of the method generating panoramic picture based on Bayer image according to the present invention, schemes based on original Bayer Specifically include as generating the step of projective parameter:

Raw Bayer image is converted to gray level image；

Gray level image is carried out feature extraction and matching, utilizes match point to optimize projective parameter.

One embodiment of the method generating panoramic picture based on Bayer image according to the present invention, raw according to projective parameter The step becoming look-up table specifically includes:

Create the look-up table mated with the size of the Bayer panoramic picture as target；

It is that look-up table distributes raw Bayer image coordinate mapping relations according to projective parameter；

It is that weight coefficient is merged in look-up table distribution according to the overlapping relation of raw Bayer image with Bayer panoramic picture.

One embodiment of the method generating panoramic picture based on Bayer image according to the present invention, utilizes look-up table to generate The step of Bayer panoramic picture specifically includes:

Create the Bayer panoramic picture as target setting the ratio of width to height；

Data interpolating is carried out, it is thus achieved that the projection result of each pixel of Bayer panoramic picture according to look-up table；

The projection result of each pixel uses fusion weight coefficient be weighted merging, exports Bayer panoramic picture.

Present invention further teaches a kind of system generating panoramic picture based on Bayer image, including:

Image capture module, gathers raw Bayer image；

Bayer panorama generation module, the mode merged based on projection completes raw Bayer image to Bayer panoramic picture Conversion.

One embodiment of the system generating panoramic picture based on Bayer image according to the present invention, system also includes:

Data conversion module, processes Bayer panoramic picture, is converted to the figure including RGB or YUV through data As form.

One embodiment of the system generating panoramic picture based on Bayer image according to the present invention, Bayer panorama generates mould Block includes:

Image Mosaic module, generates projective parameter based on raw Bayer image；

Look-up table generation module, generates look-up table according to projective parameter；

Image projection Fusion Module, utilizes look-up table to generate Bayer panoramic picture.

One embodiment of the system generating panoramic picture based on Bayer image according to the present invention, Image Mosaic module bag Include:

Greyscale image transitions unit, is converted to gray level image by raw Bayer image；

Projective parameter optimizes unit, and gray level image is carried out feature extraction and matching, utilizes match point to optimize projective parameter.

One embodiment of the system generating panoramic picture based on Bayer image according to the present invention, look-up table generation module Including:

Look-up table creating unit, creates the look-up table mated with the size of the Bayer panoramic picture as target；

Coordinate map unit, is that look-up table distributes raw Bayer image coordinate mapping relations according to projective parameter；

Merge weighted coefficient distribution unit, be lookup according to the overlapping relation of raw Bayer image with Bayer panoramic picture Weight coefficient is merged in table distribution.

One embodiment of the system generating panoramic picture based on Bayer image according to the present invention, image projection merges mould Block includes:

Target image creating unit, creates the Bayer panoramic picture as target setting the ratio of width to height；

Projecting cell, carries out data interpolating according to look-up table, it is thus achieved that the projection result of each pixel of Bayer panoramic picture；

Weighted Fusion unit, uses fusion weight coefficient to be weighted merging, output by the projection result of each pixel Bayer panoramic picture.

The present invention contrasts prior art following beneficial effect: it is former that first the present invention is gathered multiple by image capture module Beginning Bayer image, is then passed through the mode that Bayer panorama generation module merges based on projection and completes raw Bayer image and arrive The conversion of Bayer panoramic picture.It is also preferred that the left Bayer panoramic picture can be converted into example as desired by data conversion module Such as picture formats such as RGB or YUV.Comparing with conventional art, be originally inputted and the panorama output of the present invention are Bayer number According to, output panorama remains the data precision of original image.Instant invention overcomes tradition RGB data and generate panorama loss artwork essence Degree and the defect of additional conversion link, directly use the input data that Bayer data merge as projection, according to target Bayer Raw Bayer data is sampled by the component of image sampling position, generates Bayer panoramic picture.This method can keep original The precision of data also reduces unnecessary step of converting.

Accompanying drawing explanation

Fig. 1 shows the flow chart of the preferred embodiment of the method generating panoramic picture based on Bayer image of the present invention.

Fig. 2 to 4 shows the refinement flow chart of the correlation step in the method shown in Fig. 1.

Fig. 5 shows the schematic diagram of the preferred embodiment of the system generating panoramic picture based on Bayer image of the present invention.

Fig. 6 shows the schematic diagram of raw Bayer image (GRBG form).

Fig. 7 shows the schematic diagram of a kind of Bayer interpolation algorithm.

Detailed description of the invention

After reading the detailed description that embodiment of the disclosure in conjunction with the following drawings, it is possible to be more fully understood that the present invention's Features described above and advantage.In the accompanying drawings, each assembly is not necessarily drawn to scale, and has similar correlation properties or feature Assembly be likely to be of same or like reference.

The method generating panoramic picture based on Bayer image

Fig. 1 shows the flow process of the preferred embodiment of the method generating panoramic picture based on Bayer image of the present invention.Please See Fig. 1, the detailed description of the enforcement step of method to the present embodiment is presented herein below.

Step S1: gather raw Bayer image.

Multiple raw Bayer image are as shown in Figure 6.

Step S2: generate projective parameter based on raw Bayer image.

The step for be refined as shown in Figure 2.

Step S21: raw Bayer image is converted to gray level image.

Step S22: gray level image is carried out feature extraction and matching, utilizes match point to optimize projective parameter.

Projective parameter determines panoramic picture to coordinates of original image coordinates mapping relations.

The method of the splicing projection of the present embodiment is merely illustrative, and other method certainly can also be had to realize.

Step S3: generate look-up table according to projective parameter.

The step for be refined as shown in Figure 3.

Step S31: create the look-up table mated with the size of the Bayer panoramic picture as target.

Step S32: foundation projective parameter is that look-up table distributes raw Bayer image coordinate mapping relations.

If panoramic picture coordinate is (pano_x, pano_y), then according to image inverse transformation, it is calculated the original graph of correspondence As coordinate is (img_x, img_y).If original image rounded coordinate is (I, J), bilinear interpolation coefficient d x, dy then has:

$\left\{\begin{array}{c}I=floor(img\_x)\\ J=floor(img\_y)\end{array}\right.$

$\left\{\begin{array}{c}dx=img\_x-I\\ dy=img\_y-J\end{array}\right.$

Certainly, interpolating method can be not limited to bilinear interpolation, it is also possible to realizes with additive method.

Step S33: foundation raw Bayer image is that power is merged in look-up table distribution with the overlapping relation of Bayer panoramic picture Weight coefficient.

Step S4: utilize look-up table to generate Bayer panoramic picture.

The step for be refined as shown in Figure 4.

Step S41: create the Bayer panoramic picture as target setting the ratio of width to height.

Such as, the Bayer panoramic picture as target that the ratio of width to height is 2:1 is created.

Step S42: carry out data interpolating according to look-up table, it is thus achieved that the projection result of each pixel of Bayer panoramic picture.

The data interpolating of Bayer image is that the channel value interpolation according to neighborhood territory pixel goes out the dedicated tunnel at object pixel Value.

Travel through look-up table line by line, obtain coordinates of original image coordinates (I, J), to sphere of movements for the elephants neighborhood picture that (I, J) is the upper left corner Element carries out Bayer data interpolating, and is weighted by bilinear interpolation coefficient (dx, dy), obtains the projection result of each point.Bayer Data interpolating is the passage according to object pixel, neighborhood territory pixel linear interpolation according to same channels on raw Bayer image Generate destination channel value.As it is shown in fig. 7, the point (pano_x, pano_y) on target Bayer panoramic picture is according to Bayer image Form, needs Interpolate Green Channel, and the green channel of sphere of movements for the elephants neighborhood territory pixel is carried out interpolation, with the simple mean value weighting of one is Example, has:

$\left\{\begin{array}{c}{G}_{I,J}=G1\\ G_{I+1,J}=(G1+G2+G3+G4)/4\\ G_{I,J+1}=(G1+G4+G5+G6)/4\\ G_{I+1,J+1}=G4\end{array}\right.$

If Bayer panorama output green channel value is G, bilinear interpolation coefficient is (dx, dy), then:

$\left\{\begin{array}{c}{G}_{up}=(1-dx)G_{I,J}+{\mathrm{dxG}}_{I+1,J}\\ G_{dn}=(1-dx)G_{I,J+1}+{\mathrm{dxG}}_{I+1,J+1}\\ G=(1-dy)G_{up}+{\mathrm{dyG}}_{dn}\end{array}\right.$

Step S43: the projection result of each pixel uses fusion weight coefficient be weighted merging, and Bayer is complete in output Scape image.

It is also preferred that the left after the conversion completing Bayer panoramic picture, also Bayer panoramic picture is processed, turns through data The picture format being changed to including RGB or YUV, the method for conversion is also Bayer data interpolating.

The system of panoramic picture is generated based on Bayer image

Fig. 5 shows the principle of the preferred embodiment of the system generating panoramic picture based on Bayer image of the present invention.Please Seeing Fig. 5, the system of the present embodiment includes image capture module 1, Bayer panorama generation module 2.

Image capture module 1 is used for gathering raw Bayer image.Bayer panorama generation module 2 merges based on projection Mode completes the raw Bayer image conversion to Bayer panoramic picture.

It is also preferred that the left system also includes data conversion module, Bayer panoramic picture is processed, be converted to bag through data Include RGB or YUV at interior picture format.

Bayer panorama generation module 2 farther includes Image Mosaic module 21, look-up table generation module 22 and image and throws Shadow Fusion Module 23.Image Mosaic module 21 is to generate projective parameter based on raw Bayer image, and projective parameter determines panorama Image is to coordinates of original image coordinates mapping relations.Look-up table generation module 22 is based on projective parameter and generates look-up table.Image projection Fusion Module 23 is to utilize look-up table to generate Bayer panoramic picture.

Image Mosaic module 21 includes that greyscale image transitions unit 210, projective parameter optimize unit 211.Gray level image turns Change unit 210 and raw Bayer image is converted to gray level image.Projective parameter optimizes unit 211 and gray level image is carried out feature Extract and coupling, utilize match point to optimize projective parameter.

Look-up table generation module 22 includes that look-up table creating unit 220, coordinate map unit 221 and fusion weight coefficient divide Join unit 222.Look-up table creating unit 220 creates the look-up table mated with the size of the Bayer panoramic picture as target.

Coordinate map unit 221 is that look-up table distributes raw Bayer image coordinate mapping relations according to projective parameter.If it is complete Scape image coordinate is (pano_x, pano_y), then according to image inverse transformation, the coordinates of original image coordinates being calculated correspondence is (img_x,img_y).If original image rounded coordinate is (I, J), bilinear interpolation coefficient d x, dy then has:

$\left\{\begin{array}{c}I=floor(img\_x)\\ J=floor(img\_y)\end{array}\right.$

$\left\{\begin{array}{c}dx=img\_x-I\\ dy=img\_y-J\end{array}\right.$

Certainly, interpolating method can be not limited to bilinear interpolation, it is also possible to realizes with additive method.

Merge the weighted coefficient distribution unit 222 overlapping relation according to raw Bayer image and Bayer panoramic picture for looking into Table distribution is looked for merge weight coefficient.

Image projection Fusion Module 23 includes target image creating unit 230, projecting cell 231 and Weighted Fusion unit 232.Target image creating unit 230 creates the Bayer panoramic picture as target setting depth-width ratio, such as, creates wide high Than the Bayer panoramic picture as target for 2:1.

Projecting cell 231 carries out data interpolating according to look-up table, it is thus achieved that the projection knot of each pixel of Bayer panoramic picture Really.The data interpolating of Bayer image is the dedicated tunnel value that the channel value interpolation according to neighborhood territory pixel goes out at object pixel.

Travel through look-up table line by line, obtain coordinates of original image coordinates (I, J), to sphere of movements for the elephants neighborhood picture that (I, J) is the upper left corner Element carries out Bayer data interpolating, and is weighted by bilinear interpolation coefficient (dx, dy), obtains the projection result of each point.Bayer Data interpolating is the passage according to object pixel, neighborhood territory pixel linear interpolation according to same channels on raw Bayer image Generate destination channel value.As it is shown in fig. 7, the point (pano_x, pano_y) on target Bayer panoramic picture is according to Bayer image Form, needs Interpolate Green Channel, and the green channel of sphere of movements for the elephants neighborhood territory pixel is carried out interpolation, with the simple mean value weighting of one is Example, has:

$\left\{\begin{array}{c}{G}_{I,J}=G1\\ G_{I+1,J}=(G1+G2+G3+G4)/4\\ G_{I,J+1}=(G1+G4+G5+G6)/4\\ G_{I+1,J+1}=G4\end{array}\right.$

If Bayer panorama output green channel value is G, bilinear interpolation coefficient is (dx, dy), then:

$\left\{\begin{array}{c}{G}_{up}=(1-dx)G_{I,J}+{\mathrm{dxG}}_{I+1,J}\\ G_{dn}=(1-dx)G_{I,J+1}+{\mathrm{dxG}}_{I+1,J+1}\\ G=(1-dy)G_{up}+{\mathrm{dyG}}_{dn}\end{array}\right.$

The projection result of each pixel is used fusion weight coefficient to be weighted merging by Weighted Fusion unit 232, output Bayer panoramic picture.

Although illustrate for making explanation simplify said method and be described as a series of actions, it should be understood that and understand, These methods are not limited by the order of action, because according to one or more embodiments, some actions can occur in different order And/or with from depicted and described herein or the most not shown and describe but it will be appreciated by those skilled in the art that other Action occurs concomitantly.

Those skilled in the art will further appreciate that, the various illustrative described in conjunction with the embodiments described herein Logic plate, module, circuit and algorithm steps can be embodied as electronic hardware, computer software or combination of the two.For clearly Chu ground explains orally this interchangeability of hardware and software, various illustrative components, frame, module, circuit and step be above with Its functional form makees what vague generalization described.This type of is functional be implemented as hardware or software depends on specifically applying and Put on the design constraint of total system.Technical staff can realize described by different modes for every kind of application-specific Functional, but such realize decision-making and should not be interpreted to cause departing from the scope of the present invention.

The various illustrative logic plates, module and the circuit that describe in conjunction with presently disclosed embodiment can use general place Reason device, digital signal processor (DSP), special IC (ASIC), field programmable gate array (FPGA) or other can compile Journey logical device, discrete door or transistor logic, discrete nextport hardware component NextPort or its be designed to carry out function described herein Any combination realizes or performs.General processor can be microprocessor, but in alternative, this processor can be to appoint Processor, controller, microcontroller or the state machine what is conventional.Processor is also implemented as the combination of calculating equipment, example One or more microprocessors of cooperating with DSP core with the combination of microprocessor, multi-microprocessor such as DSP or any its He configures this type of.

The method described in conjunction with embodiment disclosed herein or the step of algorithm can be embodied directly in hardware, in by processor Embody in the software module performed or in combination of the two.Software module can reside in RAM memory, flash memory, ROM deposit Reservoir, eprom memory, eeprom memory, depositor, hard disk, removable dish, the appointing of CD-ROM or known in the art What in the storage medium of his form.Exemplary storage medium is coupled to processor so that this processor can be from/to this storage Medium reads and write information.In alternative, storage medium can be integrated into processor.Processor and storage medium can Reside in ASIC.ASIC can reside in user terminal.In alternative, processor and storage medium can be as discrete sets Part is resident in the user terminal.

In one or more exemplary embodiments, described function can be in hardware, software, firmware or its any combination Middle realization.If being embodied as computer program in software, the most each function can be as the instruction of one or more bars or generation Code storage on a computer-readable medium or mat its transmit.Computer-readable medium includes computer-readable storage medium and communicates Both media, it includes any medium facilitating computer program to shift to another ground from a ground.Storage medium can be can quilt Any usable medium that computer accesses.Non-limiting as example, such computer-readable medium can include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage apparatus, maybe can be used to carry or store instruction Or the desirable program code of data structure form and other medium any that can be accessed by a computer.Any connection is also by by rights It is referred to as computer-readable medium.Such as, if software is to use coaxial cable, fiber optic cables, twisted-pair feeder, numeral subscriber's line (DSL) or the most infrared, radio and microwave etc wireless technology from web site, server or other remote source pass Send, then this coaxial cable, fiber optic cables, twisted-pair feeder, DSL or the most infrared, radio and microwave etc is wireless Technology is just included among the definition of medium.Dish (disk) and dish (disc) include compact disc as used herein (CD), laser dish, laser disc, digital versatile dish (DVD), floppy disk and blu-ray disc, its mid-game (disk) often reproduces in the way of magnetic Data, and dish (disc) reproduces data optically with laser.Combinations of the above also should be included in computer-readable medium In the range of.

Thering is provided of this disclosure being previously described is for making any person skilled in the art all can make or use these public affairs Open.Various amendment of this disclosure the most all will be apparent from, and as defined herein general Suitable principle can be applied to other variants spirit or scope without departing from the disclosure.Thus, the disclosure is not intended to be limited Due to example described herein and design, but should be awarded and principle disclosed herein and novel features phase one The widest scope caused.

Claims (12)

1. the method generating panoramic picture based on Bayer image, it is characterised in that including:

Gather raw Bayer image；

The mode merged based on projection completes the raw Bayer image conversion to Bayer panoramic picture.

The method generating panoramic picture based on Bayer image the most according to claim 1, it is characterised in that complete After the conversion of Bayer panoramic picture, also Bayer panoramic picture is processed, be converted to including RGB or YUV through data Picture format.

The method generating panoramic picture based on Bayer image the most according to claim 1, it is characterised in that projection is merged Mode specifically include:

Projective parameter is generated based on raw Bayer image；

Look-up table is generated according to projective parameter；

Look-up table is utilized to generate Bayer panoramic picture.

The method generating panoramic picture based on Bayer image the most according to claim 3, it is characterised in that based on original Bayer image generates the step of projective parameter and specifically includes:

Raw Bayer image is converted to gray level image；

Gray level image is carried out feature extraction and matching, utilizes match point to optimize projective parameter.

The method generating panoramic picture based on Bayer image the most according to claim 3, it is characterised in that according to projection Parameter generates the step of look-up table and specifically includes:

Create the look-up table mated with the size of the Bayer panoramic picture as target；

It is that look-up table distributes raw Bayer image coordinate mapping relations according to projective parameter；

It is that weight coefficient is merged in look-up table distribution according to the overlapping relation of raw Bayer image with Bayer panoramic picture.

The method generating panoramic picture based on Bayer image the most according to claim 3, it is characterised in that utilize and search Table generates the step of Bayer panoramic picture and specifically includes:

Create the Bayer panoramic picture as target setting the ratio of width to height；

Data interpolating is carried out, it is thus achieved that the projection result of each pixel of Bayer panoramic picture according to look-up table；

The projection result of each pixel uses fusion weight coefficient be weighted merging, exports Bayer panoramic picture.

7. the system generating panoramic picture based on Bayer image, it is characterised in that including:

Image capture module, gathers raw Bayer image；

Bayer panorama generation module, the mode merged based on projection completes raw Bayer image turning to Bayer panoramic picture Change.

The system generating panoramic picture based on Bayer image the most according to claim 7, it is characterised in that system is also wrapped Include:

Data conversion module, processes Bayer panoramic picture, is converted to the image pane including RGB or YUV through data Formula.

The system generating panoramic picture based on Bayer image the most according to claim 7, it is characterised in that Bayer panorama Generation module includes:

Image Mosaic module, generates projective parameter based on raw Bayer image；

Look-up table generation module, generates look-up table according to projective parameter；

Image projection Fusion Module, utilizes look-up table to generate Bayer panoramic picture.

The system generating panoramic picture based on Bayer image the most according to claim 9, it is characterised in that Image Mosaic Module includes:

Greyscale image transitions unit, is converted to gray level image by raw Bayer image；

Projective parameter optimizes unit, and gray level image is carried out feature extraction and matching, utilizes match point to optimize projective parameter.

11. systems generating panoramic picture based on Bayer image according to claim 9, it is characterised in that look-up table is raw Module is become to include:

Look-up table creating unit, creates the look-up table mated with the size of the Bayer panoramic picture as target；

Coordinate map unit, is that look-up table distributes raw Bayer image coordinate mapping relations according to projective parameter；

Merging weighted coefficient distribution unit, the overlapping relation according to raw Bayer image with Bayer panoramic picture is that look-up table divides Join fusion weight coefficient.

12. systems generating panoramic picture based on Bayer image according to claim 9, it is characterised in that image projection Fusion Module includes:

Target image creating unit, creates the Bayer panoramic picture as target setting the ratio of width to height；

Projecting cell, carries out data interpolating according to look-up table, it is thus achieved that the projection result of each pixel of Bayer panoramic picture；

Weighted Fusion unit, uses fusion weight coefficient to be weighted merging by the projection result of each pixel, and Bayer is complete in output Scape image.