CN101739710B - Outdoor scene illumination parameter restoration device and working method thereof - Google Patents
Outdoor scene illumination parameter restoration device and working method thereof Download PDFInfo
- Publication number
- CN101739710B CN101739710B CN2009102299869A CN200910229986A CN101739710B CN 101739710 B CN101739710 B CN 101739710B CN 2009102299869 A CN2009102299869 A CN 2009102299869A CN 200910229986 A CN200910229986 A CN 200910229986A CN 101739710 B CN101739710 B CN 101739710B
- Authority
- CN
- China
- Prior art keywords
- image
- skylight
- sunshine
- scene
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
The invention discloses an outdoor scene illumination parameter restoration device and a working method thereof. The device comprises a computer and a video camera. The basic working method comprises the following steps: the image or video stream which is shot by the video camera is input to the computer and processed by the computer, and the restoration result of the illumination parameters of the scene is output. The process is divided into a preprocessing phase and an on-line phase. The device has the working principle that any outdoor scene image can be presented as a linear combination of a sunlight and skylight base image, so that the problem of the restoration of the incident light brightness values of the sunlight and the skylight in the image can be included as a problem of optimizing the real-time solving. The method does not need three-dimensional geometric information and material information of the scene, and can acquire the incident light brightness values of the sunlight and the skylight in the outdoor scene in real time. The method can be directly used in an augmented reality system, and can also be used for outdoor scene video sequence preprocessing, such as image segmentation, pattern recognition and the like.
Description
Technical field
The present invention relates to outdoor scene illumination parameter and recover, relate in particular to a kind of outdoor scene illumination parameter restoration device and method of work thereof.
Background technology
E.Nakamae is in the photo that proposed the dummy object that a computing machine generates is synthesized to real scene in 1986 first [NHIN86].The author sets up the numerical value that system of equations solves the red, green, blue look road of sunshine and skylight respectively by the shadow spots and the non-shadow spots of some diffusion materials in the simple interaction scenarios of user, draw dummy object with these two light sources that recover out again, make the illumination of dummy object and real pictures seem consistent.This method recovers still to have operability for the illumination of a frame or several two field pictures, then can't carry out interactive operation for online video flowing, thereby can't handle fully.After this,, recover fields such as shape and intrinsic image analysis, all have some to obtain the work of illumination parameter in reverse drafting, from light and shade because the appearance of scene is subjected to the influence of incident illumination to a great extent.Below these methods are simply introduced.
1, reverse drafting
In reverse drafting, for fear of the complicacy of directly finding the solution, [LDR2000] adopts the method for hand dipping to obtain position, color and the intensity size of light source in the indoor scene.By means of instrument, this method can obtain illumination parameter comparatively accurately.But apparent, this method is infeasible for obtaining outdoor illumination.In order to obtain the illumination parameter under complex scene and the complex illumination environment, [DYB98] proposes a kind of method based on image.This method is put into omnidirectional's (360 °) incident light that scene obtains place, minute surface ball place to a minute surface ball, by the minute surface racket being taken the photograph the different image of several exposures, can obtain the illumination of high dynamic range of the omnidirectional of scene, on sphere or cube, just can draw this width of cloth image mapped dummy object.The acquisition methods of this photoenvironment based on image has obtained using widely in rendering algorithm, is very easy to obtaining of photoenvironment under the complex environment.But this method also is not suitable for the recovery room photoenvironment of scene outward.1), in the outdoor illumination reason has two:, sun light intensity is very big under the sunny weather, for existing camera, even adopt minimum exposure speed, the reflected light of the sun on the minute surface ball still can supersaturation, is difficult to recover the correct brightness of sunshine, the failure that causes illumination to recover.2), often have cloud to sail in the outdoor scene, perhaps have gentle breeze to whip, the pixel that is difficult to guarantee the multiple pictures taken with different exposures like this is a static or complete matching fully, and therefore the high dynamic range images that obtains is often inaccurate.
2, based on the method for physics
The appearance of an object is the material properties (BRDF) by body surface in the image, how much on object and the common decision of illumination.If the geometry of known scene and material properties, the illumination that then can find the solution scene.Thisly often utilize monochrome information, shade, the key point of image pixel to wait based on the method for solving of physics to ask for illumination parameter.Adopt controllable light source, [NKGR06] resolves into two images with a scene image, is respectively scene image under the direct light irradiation and the scene image under the global illumination.In addition, the illumination parameter of a few thing restoration scenario is also arranged, as [SJ08] in shape from shading.Above-mentioned illumination method for solving based on physical model can be obtained gratifying effect under indoor environment as in the controlled laboratory.But,, therefore be not suitable for the illumination analysis of outdoor scene because these methods mostly need the three-dimensional modeling of scene
3, intrinsic image analysis
In recent years, the intrinsic image analysis receives increasing computer graphical and computer vision research person's concern [TFA05].The intrinsic image decomposition technique resolves into a texture map picture (intrinsic image) and a light image with a natural image, and wherein light image is the product of illumination parameter and scene geometry.The main application of intrinsic image analysis is to extract that the constant amount of illumination is a material properties in the scene, can allow the user that texture map is looked like to do further editing operation etc.Because light image is the product of illumination parameter and scene geometry, therefore this method can not solve illumination parameter by explicitly.In addition, existing in the single image intrinsic image technology robust and accurately not enough of restoration scenario, and computation complexity is higher, can not carry out in real time.
4, the illumination analysis of outdoor scene
In recent years, the illumination of outdoor scene is analyzed and more and more is subject to people's attention, a static scene is passed in time [SMPR07] and the video body taken resolves into 3 width of cloth texture map pictures, two base curves, and a compressible shadow representation, open image in the video body arbitrarily and can come out by above-mentioned key element reconstruct.Owing to be subjected to the influence of assumed condition, think promptly that under sunny weather condition the appearance vector of all pixels is identical under the meaning that has differed from a side-play amount and a scale factor, therefore this method only is suitable for the video decomposition under the sunny weather.Further, [SRM08] proposes color distribution with outdoor light source and is expressed as a two-dimensional linear subspace in the three-dimensional linear space, obtains the colourity of scene light source by methods such as shade extractions.Above-mentioned two kinds of methods are analyzed the outdoor scene image, but do not obtain clear and definite illumination parameter and illumination parameter of no use is drawn dummy object.In addition,, therefore only be applicable to video is carried out aftertreatment, can not online in real time handle video because these two kinds of methods are found the solution on the video body.
The work the most relevant with the present invention is [AJM06].The target of this piece article also is to add dummy object in a real scene, the illumination of change dummy object that can be real-time when the illumination variation of scene.This work is modeled as directional light to sunshine, skylight is modeled as floodlight, by scene being set up the 3D model, can determine the pairing scene point of each pixel on the image, outdoor illumination is modeled as the Phong model, sets up the value that system of equations is found the solution sunshine and skylight.By carrying out a series of pre-service, and adopt method for drafting, make that the embedding to dummy object can reach real-time effect based on image in off-line phase.The shortcoming of this method is the three-dimensional model that needs known scene, and the modeling of outdoor scene is very difficult as previously mentioned, therefore has significant limitation in actual applications.
The main challenge that has two aspects is analyzed in the illumination of outdoor scene: (1) geometric complexity.Compare with indoor scene, the scale of outdoor scene is much bigger.In addition, outdoor scene generally includes the complicated object that is difficult to modeling as trees, landform etc. of some moulding.And existing indoor illumination algorithm generally all requires the three dimensional representation of known scene, and therefore indoor illumination analytical approach also is not suitable for the illumination analysis of outdoor scene.(2) illumination complicacy.Lamp in the outdoor scene (sunshine and skylight) has some special attributes, illumination as outdoor scene is subjected to the influence of cloud layer, weather condition bigger, and can not artificially control, directly be difficult to carry out because scene structure is complicated, more be difficult to requirement of real time according to the accurate anti-illumination parameter of asking of optical illumination model.
Summary of the invention
The objective of the invention is at the deficiencies in the prior art, a kind of outdoor scene illumination parameter restoration device and method of work thereof are provided.This device is simple and easy to do, is made of a computing machine and a video camera.This method can be obtained the illumination parameter of outdoor scene in real time under the prerequisite of geological information that does not need known three-dimensional scenic and material information, the dummy object that makes computing machine generate can seamlessly be fused in the image sequence of real scene of any time.Illumination parameter is meant the brightness value of sunshine and skylight.
A kind of outdoor scene illumination parameter restoration device comprises computing machine and video camera, and computing machine is connected by data line with video camera.
A kind of method of work of outdoor scene illumination parameter restoration device, method is as follows:
1) the video camera photographed scene cloudy image, as skylight base image, and the input computing machine;
2), adopt alternant way to determine the unit of skylight brightness value according to skylight base image;
3) video camera is taken the image sequence of scene under any weather condition, is sent to computing machine in real time, and computing machine is handled present frame;
4) calculate the pairing weighted value of each pixel of present frame by weighting function;
5) calculate the skylight brightness value according to the pairing weighted value of all pixels of present frame;
6) pixel value of current frame image deducts the product of skylight brightness value and skylight base image respective pixel value, obtains the sunshine component value of current frame image respective pixel;
7) do you judge that present frame is first frame of image sequence? be, make then that the sunshine brightness value is 1, the sunshine component value is sunshine base image; Not, then the sunshine component with current frame image obtains the sunshine brightness value divided by sunshine base image, by the sunshine component value of current frame image, and the sunshine base image that obtains upgrading;
8) do you judge that present frame is a last frame? be end-of-job; Not, then next frame is as present frame, and repeating step (3) is to (7).
Specifically introduce five aspects of the present invention:
1) the linear representation model of the image under the outdoor illumination scene
At sunshine is source of parallel light, and skylight is under the hypothesis of equally distributed area source, and (x λ) can be expressed as brightness portion C by solar light irradiation to the luminance brightness L at arbitrary three-dimensional point x place in the outdoor scene
Sun(x, λ) with by the light-struck brightness portion C of sky
Sky(x, linear combination λ).I (x λ) is the pixel value of image pixel x at the λ passage, λ=1,2 wherein, and 3 are respectively three Color Channels of red, green, blue, have:
I(x，λ)＝L
sun(λ)C
sun(x，λ)+L
sky(λ)C
sky(x，λ) (1)
In following formula, C
Sun(x, λ), C
Sky(x λ) represents shade, geometry and the x place at the x place resultant effect about the hiding relation of skylight and sunshine respectively.For given solar azimuth, no matter how sunshine and skylight change, and these two is to determine constantly, therefore is referred to as the basic image of sunshine and skylight respectively.Generally speaking, because the floodlight characteristic of skylight, its basic image all remains unchanged to any solar azimuth.The image of any width of cloth outdoor scene can be expressed as the linear combination of sunshine base image and skylight base image.
2) skylight base image obtains
By the image of scene of video camera shooting, as the skylight base image C of this scene at the cloudy day
Sky(x, λ).
3) find the solution the skylight brightness value of each frame according to the characteristics of sunshine base image
Note has the current frame image I that takes at any time, and (x λ), calculates the weighted value of each pixel of current frame image
Its weighting function is
Ask minimal solution with a that gets parms (λ) to following energy function respectively to different Color Channel λ again:
Wherein Ω is the set of all pixels among the I (x), and L is then arranged
Sky(λ)=a (λ).In the method, be 255 image for gray level, generally get r
1=r
2=18.
4), recover the sunshine brightness value according to the skylight brightness value that recovers
Recovering skylight brightness L
Sky(λ), can from formula (1), solve the sunshine component value L of present frame
Sun(λ) C
Sun(x, λ).Because the sun slowly moves around the equatorial plane, the position of sun of two frame adjacent images differs very little in the online image sequence, therefore thinks that the sunshine base image of adjacent two frames is identical.If present frame is first frame, its sunshine component value L then
Sun(λ) C
Sun(x λ) for its sunshine base image (so relative sunshine brightness is 1), is designated as C
Sun 0(x, λ); Otherwise, use the sunshine base image c of the sunshine component value of present frame divided by first frame
Sun 0(x λ) just can obtain the sunshine brightness value of present frame, and present frame sunshine component L
Sun(λ) C
Sun(x λ) just can obtain the sunshine base image of present frame divided by sunshine brightness, and with it as new sunshine base image.
5) objective function as the formula (2) of the present invention's definition has following three characteristics:
A) use all pixels in the image I (x) to find the solution the skylight brightness value L
Sky(λ), but not use single pixel to find the solution, improved the robustness of algorithm noise.On the other hand, though according to definition C
Sun(x λ) should be nonnegative value, but considers that picture noise is prone in the shadow region, so objective function requirement objective function is minimum under a square of meaning, has further improved the robustness of algorithm to noise.
B) need not I (x) is carried out clear and definite two-value shadow Detection, use the weight of pixel in objective function of the different brightness of a weights functions control.
C) objective function has analytic solution:
The present invention compares with background technology, and its advantage is:
The present invention adopts the linear model method based on image, does not need the three-dimensional geometric information of scene, thereby has avoided all technological challenges of facing when extensive outdoor scene rebuild, makes the illumination restoration methods more practical.This method do not need man-machine interaction to handle in the online stage, thereby the method that guaranteed can be handled video flowing after initial setting is finished.Owing to guaranteed the illumination consistance, dummy object can seamlessly be fused in the background video sequence of real scene.
Description of drawings
Fig. 1 is the process flow diagram of method.
Fig. 2 schematic representation of apparatus.
Embodiment
The invention will be further described below in conjunction with accompanying drawing and example.
Embodiment:
A kind of outdoor scene illumination parameter restoration device as shown in Figure 2, comprises computing machine and video camera, and computing machine is connected by data line with video camera.
A kind of method of work of outdoor scene illumination parameter restoration device, as shown in Figure 1, method is as follows:
1) the video camera photographed scene cloudy image, as skylight base image, and the input computing machine;
2), adopt alternant way to determine the unit of skylight brightness value according to skylight base image;
3) video camera is taken the image sequence of scene under any weather condition, is sent to computing machine in real time, and computing machine is handled present frame;
4) calculate the pairing weighted value of each pixel of present frame by weighting function;
5) calculate the skylight brightness value according to the pairing weighted value of all pixels of present frame;
6) pixel value of current frame image deducts the product of skylight brightness value and skylight base image respective pixel value, obtains the sunshine component value of current frame image respective pixel;
7) do you judge that present frame is first frame of image sequence? be, make then that the sunshine brightness value is 1, the sunshine component value is sunshine base image; Not, then the sunshine component with current frame image obtains the sunshine brightness value divided by sunshine base image, by the sunshine component value of current frame image, and the sunshine base image that obtains upgrading;
8) do you judge that present frame is a last frame? be end-of-job; Not, then next frame is as present frame, and repeating step (3) is to (7).
Claims (1)
1. the method for work of an outdoor scene illumination parameter restoration device is characterized in that, method of work is as follows:
1) the video camera photographed scene cloudy image, as skylight base image, and the input computing machine;
2), adopt alternant way to determine the unit of skylight brightness value according to skylight base image;
3) video camera is taken the image sequence of scene under any weather condition, is sent to computing machine in real time, and computing machine is handled present frame;
4) calculate the pairing weighted value of each pixel of present frame by weighting function;
5) calculate the skylight brightness value according to the pairing weighted value of all pixels of present frame, the computing method of skylight brightness value are as follows:
Note has the current frame image I that takes at any time, and (x λ), calculates the weighted value of each pixel of current frame image
Its weighting function is
C wherein
Sky(x for the skylight base image of input, is 255 image for gray level λ), generally gets r
1=r
2=18; Ask minimal solution with a that gets parms (λ) to following energy function respectively to different Color Channel λ again:
Wherein Ω is the set of all pixels among the I (x), and the skylight brightness value L is then arranged
Sky(λ)=a (λ);
6) pixel value of current frame image deducts the product of skylight brightness value and skylight base image respective pixel value, obtains the sunshine component value of current frame image respective pixel;
7) do you judge that present frame is first frame of image sequence? be, make then that the sunshine brightness value is 1, the sunshine component value is sunshine base image; Not, then the sunshine component value with current frame image obtains the sunshine brightness value divided by sunshine base image, by the sunshine component value of current frame image, and the sunshine base image that obtains upgrading;
8) do you judge that present frame is a last frame? be end-of-job; Not, then next frame is as present frame, and repeating step (3) is to (7).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009102299869A CN101739710B (en) | 2009-11-30 | 2009-11-30 | Outdoor scene illumination parameter restoration device and working method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009102299869A CN101739710B (en) | 2009-11-30 | 2009-11-30 | Outdoor scene illumination parameter restoration device and working method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101739710A CN101739710A (en) | 2010-06-16 |
| CN101739710B true CN101739710B (en) | 2011-08-10 |
Family
ID=42463157
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2009102299869A Expired - Fee Related CN101739710B (en) | 2009-11-30 | 2009-11-30 | Outdoor scene illumination parameter restoration device and working method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101739710B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8872852B2 (en) | 2011-06-30 | 2014-10-28 | International Business Machines Corporation | Positional context determination with multi marker confidence ranking |
| CN106210554B (en) * | 2016-07-29 | 2019-01-25 | 长春理工大学 | A method of synthesis high dynamic range images |
| CN108460841A (en) * | 2018-01-23 | 2018-08-28 | 电子科技大学 | A kind of indoor scene light environment method of estimation based on single image |
| CN110189415B (en) * | 2019-04-30 | 2023-03-14 | 山东科技大学 | Outdoor scene reconstruction method based on delayed video |
-
2009
- 2009-11-30 CN CN2009102299869A patent/CN101739710B/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| CN101739710A (en) | 2010-06-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Rudnev et al. | Nerf for outdoor scene relighting | |
| Karacan et al. | Learning to generate images of outdoor scenes from attributes and semantic layouts | |
| CN110910486B (en) | Indoor scene illumination estimation model, method and device, storage medium and rendering method | |
| Shan et al. | The visual turing test for scene reconstruction | |
| Lalonde et al. | What do the sun and the sky tell us about the camera? | |
| CN102881011B (en) | Region-segmentation-based portrait illumination transfer method | |
| Haefner et al. | Variational uncalibrated photometric stereo under general lighting | |
| Liu et al. | Light source estimation of outdoor scenes for mixed reality | |
| CN101739710B (en) | Outdoor scene illumination parameter restoration device and working method thereof | |
| CN114125310B (en) | Photographing method, terminal device and cloud server | |
| Chen et al. | Lighting virtual objects in a single image via coarse scene understanding | |
| Lalonde et al. | What does the sky tell us about the camera? | |
| CN114241159A (en) | Three-dimensional reconstruction and PBR mapping manufacturing method based on close-range photogrammetry method | |
| CN106097277B (en) | A kind of rope substance point-tracking method that view-based access control model measures | |
| Xing et al. | A practical approach for real-time illumination estimation of outdoor videos | |
| CN114332355A (en) | Weak light multi-view geometric reconstruction method based on deep learning | |
| CN106096085B (en) | A kind of radiant energy dfensity analogy method of tower-type solar thermal power generating system | |
| Xing et al. | On-line illumination estimation of outdoor scenes based on area selection for augmented reality | |
| Xing et al. | Online illumination estimation of outdoor scenes based on videos containing no shadow area | |
| CN112132743B (en) | Video face changing method capable of self-adapting illumination | |
| Lee et al. | Surface IR reflectance estimation and material recognition using ToF camera | |
| Jung et al. | One-day outdoor photometric stereo using skylight estimation | |
| Liu et al. | Real-Time Shadow Detection From Live Outdoor Videos for Augmented Reality | |
| Inose et al. | Refining outdoor photometric stereo based on sky model | |
| Zheng et al. | An extended photometric stereo algorithm for recovering specular object shape and its reflectance properties |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20110810 Termination date: 20141130 |
|
| EXPY | Termination of patent right or utility model |