KR20160088178A - The method and the recording media for original image restoration technology based on scattering noise removal and photon detection for single recorded image - Google Patents

Abstract

The present invention relates to a method for restoring an original image for a single recorded image. The method comprises the steps of: acquiring a single peplogram from light which has strength to which the Beer-Lambert-Bouguer law can be applied and which has been transmitted through a scattering medium; estimating the scattering medium by applying a specific estimation method to the acquired single peplogram; removing the estimated scattering medium from the single peplogram; detecting photons from the peplogram, from which the scattering medium have been removed, by using a specific calculation method, and restoring a 2D original image; overlapping images by using the restored 2D original image, and restoring an image, in which the images have overlapped each other, to a 3D image by using a 3D passive imaging technique. According to the present invention, a technology for removing scattering noise and detecting photons from a single recorded image which has been transmitted through a medium causing severe scattering is employed, thereby providing an original image restoration technology which can provide improved image quality so that an object or a scene can be identified with the unaided eye.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to a method and apparatus for reconstructing an original image through scattered noise removal and photon detection on a single recorded image,

The present invention relates to a method of reconstructing an original image for a single recorded image and a recording medium therefor. More particularly, the present invention relates to a method of estimating a scattering medium by using a statistical inference method and then removing scattering noise, A method for restoring an original image through a method, and a recording medium therefor.

Moving images (images) that pass through a medium that is strongly scattered in natural light conditions are difficult to be visually distinguished. However, in recent years, there has been a growing demand for observing objects or landscapes that have passed through such scattering media in various fields, thus necessitating the above identification technology. For example, it may be useful to identify vehicles ahead of a busy fog area during driving, to find survivors in the smoke at a fire scene, to find fighter pilots hiding in the clouds, It is necessary to secure a field of view such as observing cells through skin tissue. However, when there is a strong scattering medium, it is difficult for a person to identify an object or landscape with the naked eye.

Various methods have been studied to overcome these difficulties. In general research methods, non-invasive image acquisition method using laser and scanning mechanism as active light source, multiple image acquisition method, . However, these methods focused on how to remove the scattering medium and did not consider the method of detecting light information from objects or landscapes, so the results were unsatisfactory. In addition, these methods have a problem that prior information of a scattering medium which changes according to environmental changes is necessarily required, and it is impractical because it is difficult to acquire an image outdoors.

(Prior Patent 1) Korean Patent Publication No. 10-2002-0006781 (Prior Patent 2) Korean Patent Publication No. 10-2011-0103608

Berrolotti, J. et. al. Non-invasive imaging through opaque scattering layers. Nature 491, 232-234 (2012) Nixon, M. et. al. Real-time wavefront shaping through scattering media by all-optical feedback. Nature Photon. 7, 919-924 (2013)

Disclosure of Invention Technical Problem [8] Accordingly, the present invention has been made to solve the above problems, and it is an object of the present invention to provide a method for recovering an original image by detecting a photon after removing scattering noise from a single recorded image passing through a strongly scattering medium, There is a purpose.

According to an aspect of the present invention, there is provided a method for reconstructing an original image, the method comprising the steps of: detecting a light intensity applied by a Beer-Lambert-Bouguer law from a light transmitted through a scattering medium; Obtaining a perpetogram; Estimating a scattering medium by applying a specific estimation method to the acquired single cepstral flow; Removing the estimated scattering medium in the single perphogram; Detecting a photon by a specific calculation method from the perphogram in which the scattering medium is removed, and restoring the two-dimensional original image; Superimposing the images using the restored two-dimensional original image, and restoring the superimposed image into a three-dimensional image using a 3D passive imaging technique; .

The step of estimating the scattering medium by applying the specific estimation method to the obtained single perceptogram may be to estimate the scattering medium based on the maximum likelihood estimation (MLE) method or the Bayesian estimation method.

The step of detecting a photon by a specific calculation method from the perfluorogram from which the scattering medium has been removed and restoring the two-dimensional original image may be performed by using a calculation method based on a Poisson distribution based on a Poisson distribution .

In the step of detecting photons from the perfluorogram from which the scattering medium has been removed, the scattering medium may have a size of the input image ranging between 1% and 10%.

Wherein the step of detecting photons from the peelogram from which the scattering medium is removed measures the number of surviving photons varying from the peakogram, wherein the number of surviving photons is between 1% and 10% of the number of pixels of the input image Value. ≪ / RTI >

In order to restore the original image into a three-dimensional image, natural light may be used.

In order to restore the original image into a three-dimensional image, natural light or infrared light may be used.

In order to recover the original image into a three-dimensional image, visible light and infrared light may be used.

A radar may be used to reconstruct the original image into a three-dimensional image.

According to another embodiment of the present invention, a method for restoring an original image may be implemented as a recording medium on which a program manufactured by any one of the above methods is recorded.

According to the present invention, by using a technique of detecting scattered noise and detecting photons in a single recorded image passed through a strongly scattering medium, it is possible to provide an image quality that can provide an image quality that can visually recognize an object or a landscape There is an effect of providing image restoration technology.

FIG. 1 is a flowchart illustrating a process of restoring an original image by applying perforaphy according to an embodiment of the present invention.
2A is a diagram illustrating application of software in accordance with an embodiment of the present invention.
FIG. 2B is a block diagram illustrating the application of software according to another embodiment of the present invention.
3 is a block diagram of an apparatus for recovering original images by removing scattered noise from a single recorded image according to an embodiment of the present invention.
Figure 4 is a diagram illustrating the overall concept of perphroplography in accordance with an embodiment of the present invention.
5 is a diagram illustrating a process of estimating and removing a scattering medium from a single perfluorogram according to an embodiment of the present invention.
FIG. 6 is a diagram showing a concept of a computer-integrated image reproducing method according to FIG.
FIG. 7 is a diagram showing experimental results of visualized perfluorography according to another embodiment of the present invention. FIG.
8 is a diagram illustrating experimental results applied to various applications according to another embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The present invention proposes a technique of restoring an image of an original object or a landscape by detecting light information of an object or landscape passing through a powerful scattering medium based on statistical optics (that is, a technique of directly detecting a living photon) .

"peplo(veiled)"와 기록한다는 뜻의 그리스어

"grafit(writing)"의 합성어이다. 페플로그래피는 통계 광학에 기초한 하나(단일)의 페플로그램(Peplogram, 가려진 이미지, veiled image)으로부터 생존 광자를 바로 검출하며, 집적 영상(integral imaging, 이미지 축적)방법을 이용하여 3차원적으로 페플로그램을 복원하게 되는 것에 관한 기술을 의미한다. 페플로그래피는 종래와 완벽히 다른 방식으로 산란 매질을 통과한 이미지(동화상)에 대하여 새로운 이미지(동화상)를 제공하기 때문에, 산란 매질과 관련된 동적인 영상 기술(dynamic imaging)분야에 존재하는 다양한 문제를 해결할 수 있다. 또한, 이하에서는 산란 매질 속에 있는 물체나 풍경에 대한 영상을 "Peplogram(페플로그램)"이라고 정의한다.In the following, this technique is defined as "Peplography", which means that the Greek

The word "peplo (veiled)" means Greek

It is a compound word of "grafit (writing)". Perfography detects living photons directly from a single statistical optically based peplogram (veiled image) and uses three-dimensional (3D) imaging techniques Means a technique relating to restoring a perpetogram. Since perforation provides a new image (moving image) for an image (moving image) that has passed through a scattering medium in a completely different manner from the conventional one, various problems that exist in the field of dynamic imaging related to scattering media Can be solved. Hereinafter, an image of an object or a landscape in the scattering medium is defined as "Peplogram ".

Hereinafter, the theory and experimental results supporting the perpetography will be presented with reference to the drawings, and a method for acquiring a three-dimensional image (moving image) through perforation will be described in detail.

FIG. 1 is a flowchart illustrating a process of restoring an original image by applying perforaphy according to an embodiment of the present invention. FIG. 2 (a) is a block diagram illustrating application of software according to an embodiment of the present invention. FIG. 2B is a diagram illustrating the application of software according to another embodiment of the present invention. FIG. 3 is a block diagram of a process of removing scattered noise for a single recorded image and recovering an original image by detecting photons according to an embodiment of the present invention. Referring to FIG. Figure 4 is a diagram illustrating the overall concept of perphroplography in accordance with an embodiment of the present invention.

Referring to FIGS. 1 to 4, a method for restoring an original image using perfluorography is a method in which the intensity of light to which a Beer-Lambert-Bouguer law is applied is changed from light transmitted through a scattering medium to single- (S2) of estimating a scattering medium by applying a specific estimation method to the obtained single perpetogram, removing the estimated scattering medium in the single perpetogram (step S3), detecting the photon by a specific calculation method from the pergologram in which the scattering medium has been removed, restoring the two-dimensional original image (S4), superimposing the image using the restored two- (S5) reconstructing the 3D image into a three-dimensional image using a 3D passive imaging technique; . ≪ / RTI >

Here, the step S2 of estimating the scattering medium by applying a specific estimation method to a single percept may be performed by a maximum likelihood estimation (MLE) method or a Bayesian estimation method.

Also, the step S4 of detecting the photon by a specific calculation method from the perphogram in which the scattering medium is removed and restoring the two-dimensional original image is performed by using the photon coefficient imaging method based on the Poisson distribution Can be carried out by extracting using a calculation method such as

In addition, a radar may be used for restoring the original image into a three-dimensional image, and light by a combination of at least one selected from natural light, visible light, and infrared light may be used.

Details of each step will be described later with reference to the embodiments, drawings and equations.

Referring to FIG. 2A, the method for restoring the original image may be implemented in a form of software (computer program) that is introduced into a specific part and compatible and applicable, Type service or a mobile service.

That is, the software may be embedded in a semiconductor chip and implemented in the form of a semiconductor chip. In addition, it may be formed as an embedded sub-module or module, which is a part of a functional unit in the form of software itself or a semiconductor chip to which the software is applied. In addition, the present invention can be applied to a form of a system (Embedded System) of various devices in the form of software itself, a semiconductor chip or a module into which the software is introduced, and examples thereof include a smart phone, a tablet PC, But are not limited to, work stations, navigation, black boxes, set-top boxes, and the like. Further, the present invention may be applied to a service platform center capable of managing and controlling a system of various devices configured in the form of software itself, a semiconductor chip or module to which the software is applied, and the like.

Referring to FIG. 2B, the software (computer program) embodied in a specific device to be compatible and applicable as a method for restoring the original image is included in an existing software module and is compatible and applicable in various fields Can be applied. That is, it can be applied to reconstruct the image of the original object or landscape by removing scattered noise in various fields.

In the aerospace sector, such as satellite, space shuttle, etc., it is necessary to pass through the earth's atmosphere or in the aeronautical field such as airplane, fighter, helicopter, etc. or through the cloud in the sea area such as fishing boat, ship or cruise ship, or in submarine It is used in the field of traffic such as bicycles, cars and railways, through haze, through mist, through haze, through mist in weather related fields such as fine dust and dust, And can be applied to remove scattered noise in areas such as the health field that may interfere with the field of view. However, in addition to the above-described fields of application, in a case where it is necessary to secure a field of view by image restoration through scattered noise removal, the present invention can be applied to various fields.

3, an apparatus for reconstructing an original image by removing scattering noise from a single recorded image and detecting photons (hereinafter referred to as an 'original image restoring device') includes an image capturing unit 300, And a video display unit.

The image photographing unit 300 photographs a remote object or a landscape. The image capturing unit may include a high magnification telephoto lens and a charge-coupled device (CCD) module for recording image information obtained from the high magnification telephoto lens so as to recognize a distant object or a landscape. Device. ≪ / RTI > Here, the high magnification telephoto lens may include a focus adjustment unit for allowing the user to adjust the focus and a zoom button unit for adjusting the magnification or reduction of a specific portion of the image.

The image correction unit corrects image distortion caused by fog, sea haze, haze, or rainfall on the image captured by the image capturing unit. That is, the image sensing unit may further include a module for sensing an environmental element by receiving the image captured by the image sensing unit and analyzing the sensed environmental factor. The automatic sensing unit may include a fog automatic discriminating unit capable of sensing a fogging element, And a fog component attenuation unit that can enhance the degree of restoration of the image by calculating the fog level according to the fog component sensed by the automatic determination unit.

The automatic fog detecting unit may include an image dividing unit dividing the image captured by the image capturing unit into a plurality of images, and a fog generating unit calculating the fogging factor for each divided image region connected to the image dividing unit.

In addition, the fog component attenuation unit may include a contrast correction unit that corrects a contrast range of the image according to a fog component of the divided image, the contrast component being divided into a plurality of images by the image segmentation unit, An edge processing unit for correcting the image so that an image can be reliably divided to an edge and a color converting unit for converting a color (Red, Green, Blue) component of an image in connection with the edge processing unit so that a color similar to an actual object or a landscape can be expressed And a color information unit.

The image display unit includes a function of displaying the image corrected by the image correction unit to the user. The image display unit may be a head mount type display. In the case of the image display unit formed by the head mount type display, the image display unit is located on the same line as the user's sight line. However, the present invention is not limited to this, and a screen may be provided in real time through wired connection with a wireless communication such as Bluetooth or a smart phone, And can make various changes.

The charging cradle may further include a power supply unit that is charged using a separate charging cradle. The charging cradle may include a battery that can be charged using a wired or wireless charging scheme and rechargeable.

Referring to FIG. 4, the intensity and phase of the light passing through the scattering medium are not fixed but have a random value. In addition, the orange solid line indicates the ray scattered in the perphogram, and the solid red line indicates the survival ray passing through the scattering medium. Thus, after removing the estimated scattering medium from the perphogram, a surviving photon can be obtained. However, at this time, since there is a photon shortage state in which sufficient photons can not be obtained, a new image system capable of immediately sensing the living photons is required. This can be confirmed by FIG. 5 to be described later.

As shown in FIG. 4, it can be assumed that the entire scattered light rays are superimposed on the scattered light rays that are generated by passing through the scattering medium in the small (local) region emerging from the object. That is, it can be assumed that scattered rays for a small region are statistically independent from each other. In other words, using the Central Limit Theorem, assuming that the scattering medium is a superposition of a scattered ray in the local region and a transmitted ballistic ray, the scattering medium is Gaussian (Gaussian) distribution. Therefore, it is possible to estimate the scattering medium using an estimation method such as Maximum Likelihood Estimation (MLE) or Bayesian estimation. When the estimated scattering medium is removed from the perphogram and the remaining light is detected, the original image can be reconstructed. In order to detect the light that is alive, the concept of photon must be used, since the intensity of the living light is so small that it has to be detected by the number of photons. Therefore, in the present invention, photon counting imaging is used to detect photons. A detailed description of the photon counting imaging method will be described later.

Perfrography records images by commonly used natural light sources and image sensors. The recorded image can be identified as a perpetogram. In order to obtain the perpetogram, a scattering medium is estimated and removed by a statistical inference method such as maximum likelihood estimation (MLE). When the scattering medium is removed, the Pepflogram displays a small number of surviving photons. Thereafter, photon counting based on the Poisson distribution is applied to detect the surviving photons, Gram is obtained. Thereafter, as described later, a reconstructed 3D perpetogram with improved image quality can be obtained, which is based on a 3D passive imaging technique using an integrated imaging method.

의 영역은 가우시안 분포를 따르게 된다. 여기서, 단일 페플로그램(

)의 작은 영역이

픽셀의 분해능을 갖는 전하결합소자(CCD, 카메라 등)와 같은 종래의 이미지 기록 장치에 의해 획득될 때, 이에 상응하는 부분은

와 같이 표현되며, 이때

,

,

and

이다.In order to obtain the reconstructed perpetogram, the scattering medium is estimated and removed by statistical inference method. Hereinafter, this will be described in more detail using an equation. Here, the scattering medium is modeled using the Central Limit Theorem because it is a superposition of many small scattering media. In other words,

The region of Gaussian distribution follows. Here, a single perfluorogram (

)

When obtained by a conventional image recording apparatus such as a charge coupled device (CCD, camera, etc.) having resolution of a pixel, a corresponding portion

Is expressed as < RTI ID = 0.0 >

,

,

and

to be.

의 차원을 가진 많은 다른 산란 부분으로 구성된다고 가정할 수 있으며, 또한 산란 매질은 샘플 평균

과 분산

값을 가우시안 분포에 의해 모델링 될 수 있는데, 그 값의 범위는 입력 영상 크기의 1% ~ 10% 사이의 값 중에서 추출될 수 있다. 여기서 i 와 j 는 각각 x와 y방향으로 산란되는 광자 부분의 인덱스(index)이다. 이에, 가우스 변수는 수식 (1)과 같이 표시된다.In other words, a statistical inference method such as the Maximum Likelihood Estimation (MLE) method is applied to infer a medium scattering from a single perpogram for a strongly scattering medium. At this time,

And the scattering medium can be assumed to be composed of the sample average

And dispersion

The value can be modeled by a Gaussian distribution, whose range can be extracted from values between 1% and 10% of the input image size. Where i and j are indices of the photon parts scattered in the x and y directions, respectively. Thus, the Gaussian variable is expressed as Equation (1).

(수식 1)

(Equation 1)

In this case, I denotes the pixel intensity of the perpetogram.

)에 의하며, 최대가능추정(MLE) 방법을 이용하여 확률적으로 실제와 최대한 가까운 값을 갖게 된다.This makes it possible to estimate the scattering medium, which is the unknown value of the sample mean value of the Gaussian distribution

), And has a probability value closest to the actual value using a maximum likelihood estimation (MLE) method.

(샘플 평균값)를 발견하는 것에 의해 산란 매질을 추정할 수 있으며, 이를 위해 수식 (2)부터 수식 (4)에서 보는 바에 따라 최대가능추정(MLE)과 같은 통계적 추측 방법이 이용될 수 있다.More specifically, the unknown parameters in the Gaussian distribution

(2) to (4), statistical estimation methods such as maximum likelihood estimation (MLE) can be used for estimating the scattering medium.

(수식 2)

(Equation 2)

(수식 3)

(Equation 3)

(Equation 4)

)평면상의 표본 평균이다.Here, as shown in equations (3) and (4), the estimated scattering medium is

) Is the sample mean on the plane.

는 수식 (5)와 같이 표현된다.Thereafter, the perfluorogram

Is expressed as Equation (5).

(수식 5)

(Equation 5)

5 is a diagram illustrating a process of estimating and removing a scattering medium from a single perfluorogram according to another embodiment of the present invention. That is, FIG. 5A is a diagram showing an element image without scattering noise according to an embodiment of the present invention. FIG. 5B is a diagram showing an image of a single perfluorogram according to an embodiment of the present invention. FIG. 5C is a diagram showing an image of the scattering medium estimated according to an embodiment of the present invention. FIG. 5D is a view showing an image in which a scattering medium is removed from a single perfluorogram according to an embodiment of the present invention. FIG. FIG. 6 is a diagram showing a concept of a computer-integrated image reproducing method according to FIG.

와 같이 표현된다. 여기서

와

는 각각 산란 전의 빛의 세기와 조명 파장 및 매질에 의존하는 감쇄계수를 의미하며,

는 동질성을 갖는 매질의 두께를 나타낸다. 만약 산란 매질이 동질성이 없다면,

는 공간좌표에 의존한다. 즉, 페플로그래피는 동질성을 갖거나 또는 동질성이 없는 산란 매질 모두에 적용될 수 있다. 최대가능추정(MLE) 방법을 사용하여, 탁한 물에서의 추정된 산란 매개체는 도 5의 c와 같이 나타난다. 이때, 산란 매질의 작은 영역은

이다. 도 5의 d에 도시된 바와 같이 산란 매질을 제거한 페플로그램은 수식 (2)와 수식 (5)에 의해 얻어질 수 있다.Referring to FIG. 5, a model vehicle such as an emblem or a Hummer vehicle model is used for the experiment according to the present invention. For the experiment, about 38 liters of water were mixed with about 14 millimeters of milk, and the intensity of the transmitted light was measured by Beer-Lambert-Bouguer law

. here

Wow

Means the intensity of light before scattering and the attenuation coefficient depending on the wavelength of light and the medium, respectively,

Represents the thickness of the medium having homogeneity. If the scattering medium is not homogeneous,

Depends on the spatial coordinates. That is, perpholraphy can be applied to both homogeneous and homogeneous scattering media. Using the Maximum Likelihood Estimation (MLE) method, the estimated scattering medium in turbid water is shown in Figure 5c. At this time, a small region of the scattering medium

to be. The perphogram obtained by removing the scattering medium as shown in FIG. 5D can be obtained by the equations (2) and (5).

The processed Pepflogram is then displayed as if the photon were recording images under sparse conditions. This is because only some of the surviving photons survive through the scattering medium and it is possible to detect these surviving photons directly by photon counting imaging. Here, the photon counting method is a method used when acquiring an image of an object or a landscape in a photon-thin condition. Removing the estimated scattering medium in the perphol- ogy is like a sparse condition because the photons leave only a weakly intense light (ballistic ray). Therefore, a photon counting imaging method can be used to detect the living light. The photon counting method follows the Poisson distribution because it computes the number of rarely occurring photons. In order to control the number of photons ( N _p ) to be detected, the image obtained by removing the scattering medium from the pergologram is normalized. It then multiplies N _p and detects photons through Poisson random generation. The image of the detected photons is reconstructed as the original image. In order to obtain original images of higher quality, an integral image, which is one of the three-dimensional imaging methods, can be used. A method of acquiring a plurality of images having different perspective perceptions about an object or an image of an integrated image and displaying the three-dimensional image through the lens array. Therefore, in the perphol- ogy, a large number of fetograms obtained through the integrated image can be obtained. When these images are reproduced in three dimensions by the Computational Integral Imaging Reconstruction (CIIR) method, it is possible to restore the original images with better image quality.

에 의하여 정규화된 페플로그램이 사용되며, 이때의 전체 에너지는 생존 광자의 감지를 위한 단위 에너지를 갖는다. 광자계수 검출기의 모델은 수식(6)과 같이 표현된다.Photon count detectors capable of detecting photons by photon counting are mathematically modeled by the Poisson distribution. Taking account of the characteristics of such distributions is useful in event situations rarely occurring in units of time and space. This is because it has the advantage of being able to. As shown in Figure 4d, in order to detect the living photons from the processed Peplogram,

Normalized perphograms are used, where the total energy has a unit energy for the detection of the living photons. The model of the photon coefficient detector is expressed as Equation (6).

(수식 6)

(Equation 6)

는 검출하고자 하는 광자의 수,

는 색상 채널(Red(R)채널, Green(G)채널 및 Blue(B)채널), 그리고

는 개개의 색상 채널에 대한 생존 광자의 계수(coefficient)를 나타낸다.here

The number of photons to be detected,

(Red (R) channel, Green (G) channel and Blue (B) channel), and

Represents the coefficient of the surviving photons for each color channel.

)에 의해 조절된다. 그래서, 이미지는 단일 에너지를 가질 필요가 있고, 수식 (7)에 의하여 산란 매질을 통과한 물체 또는 풍경에 대한 전체 픽셀 세기의 합을 이용하여 정규화를 수행할 수 있다.A more detailed description of the mathematical model for extracting photons is that a new image extraction method is needed since the image after removing the estimated scattering medium has a small number of surviving photons. The photon counting method is used to detect the surviving photons from the perpetogram after removing the scattering medium. Generally, the photon counting method can be modeled by the Poisson distribution. In the modeling by the photon coefficient imaging method, the number of photons extracted is the number of photons to be extracted (

). Thus, the image needs to have a single energy, and normalization can be performed using the sum of the total pixel intensities for the object or landscape passing through the scattering medium by equation (7).

(수식 7)

(Equation 7)

)는 단일 에너지를 갖는다. 원하는 수의 광자를 추출하기 위해서 수식 (8)과 같은 푸아송 랜덤 생성(Poisson random generation) 방법이 적용될 수 있다.Here, the normalized image (

) Has a single energy. To extract a desired number of photons, a Poisson random generation method such as Equation (8) can be applied.

(8)

(8)

는 추출하고자 하는 광자의 수이고,

는 R채널, G채널 및 B채널 등의 색상 채널에 대한 빛의 계수(coefficient)를 의미한다.At this time,

Is the number of photons to be extracted,

Means a coefficient of light for a color channel such as an R channel, a G channel, and a B channel.

(수식9)

(수식10)

(수식11)From the equations (7) and (8), a reconstructed perphogram with the surviving photons is generated. However, since the generated perphogram can not contain a sufficient number of surviving photons to visualize the object, it is possible to use a plurality of restored perpetograms and a maximum likelihood estimation (MLE) Can be estimated. A likelihood function can be constructed from a plurality of reconstructed perphograms, and scenery can be estimated using Equation (9) to Equation (12).

(Equation 9)

(Equation 10)

(Equation 11)

(수식12)

(Equation 12)

By using Eq. (12), it is possible to estimate a two-dimensional landscape passing through a strong scattering medium. Thereafter, the three-dimensionally reconstructed perpetogram can be acquired using a computer integrated image reconstruction technique. Computer integrated image reproduction technology is passive 3D image sensing and visualization technology. Through a computer integrated image reproduction technique, a lenslet array or a camera array can be applied to a three-dimensional object to record a plurality of two-dimensional images having different perspective perspectives on the three-dimensional information. These two-dimensional images are called elemental images. Thereby, it is possible to provide a perfect parallax and a continuous viewpoint without a separate apparatus such as a pair of glasses. The three-dimensional image can be visualized by restoring at each depth using a computer integrated image reproduction technique.

Referring to FIG. 6, it can be seen that each element image is projected through a virtual pinhole array having a focal length f on the reproduction plane.

Thereafter, all the elemental images overlap each other, and the overlapping portion changes depending on the depth of the reproduction plane, resulting in another reproduction image. The process of computer integrated image restoration can be implemented as Equation (13) and Equation (14).

(13)

(13)

(14)

(14)

Here, N _x is the number of pixels in the X-axis direction in the element image, and N _Y Is the number of pixels in the Y axis direction in the element image. P is the distance between the virtual pinholes, C _x is the size of the image sensor in the X axis direction, and C _y is the size of the image sensor in the Y axis direction. z is the depth of the reproduced image, S _x is The number of moving pixels superimposed in the X-axis direction, and S _y is the number of moving pixels superimposed in the Y-axis direction. C _k1 denotes the restored _perpetogram of the K-th row and the l- th column, and O ( x , y ) is a matrix indicating the number of superimposed values.

에 각각 다른 계수가 적용될 수 있다. 즉 R채널, G채널 및 B채널에서 생존 광자에 대한

의 계수는 각각 1.4497, 1.1270 및 1로 설정된다. 이는 각각의 색상 채널의 평균 파장은 적색(R)이 685nm(red: 620~750nm), 녹색(G)이 532.5nm(green: 495~570nm) 그리고 파랑색(B)이 472.5nm(blue: 450~495nm)이기 때문이며, B채널을 기준으로 각각의 값을 비례적으로 정했기 때문이다.The total energy of a surviving photon coming from a three-dimensional object or scene behind the scattering medium is proportional to the wavelength of the light (light). For each basic color channel, a normalized Pepflogram

Respectively. That is, for the R-channel, G-channel and B-channel,

Are set to 1.4497, 1.1270 and 1, respectively. This means that the average wavelength of each color channel is 685 nm (red: 620 to 750 nm), green (G) is 532.5 nm (green: 495 to 570 nm), and blue color (B) is 472.5 nm ~ 495 nm), and each value is proportional to the B channel.

FIG. 7 is a diagram showing experimental results of visualized perfluorography according to another embodiment of the present invention. FIG. 7a shows a reconstructed perphogram with 200,000 surviving photons according to an embodiment of the present invention. 7B is a diagram showing experiment settings set according to an embodiment of the present invention. FIG. 7C is a diagram showing a three-dimensionally reconstructed perfluorogram when the depth z is 145 mm according to the embodiment of the present invention. FIG. 7D is a diagram showing a three-dimensionally restored perphogram when the depth z is 155 mm according to the embodiment of the present invention. FIG.

)를 조절함으로써 산란 매질을 제거한 후의 페플로그램으로부터 변화하는 생존 광자의 수를 측정할 수 있는데, 상기 생존 광자의 수는 입력 영상의 픽셀 수의 1% ~ 10% 사이의 범위 중에서 선택될 수 있다.In the perphogram, the number of photons to be extracted (

), It is possible to measure the number of surviving photons that change from the perphogram after removing the scattering medium, wherein the number of surviving photons can be selected from a range between 1% and 10% of the number of pixels of the input image .

)를 각각 색상 채널에

=200,000 로 설정하게 되면, 도 7의 a에 도시된 바와 같이 페플로그램을 복원할 수 있게 한다. 이때, 전술한 도 6의 b에서와 같이 탁한 물에서도 페플로그램으로부터 물체를 인식할 수 있기 때문에, 몇몇 노이즈 부분을 제외하고 화질은 대체적으로 개량되는 것을 확인할 수 있다. 그러나, 이렇게 얻어진 이미지는 깊이(두께) 정보가 없는 2차원 이미지이며, 화질은 여전히 만족스럽지 못하기 때문에, 개선된 화질을 가진 3차원 페플로그램을 획득하기 위해서 본 발명에서는 수동적인 3차원 집적 영상 기술(passive 3-D imaging technique of integral imaging)을 도입하였다. 수동적인 3차원 집적 영상 기술은 렌즈(lenslet)배열을 통하여 다양한 관점에서 다수의 2차원 이미지를 기록하고 표시하는 방법으로써, 3차원 이미지를 복원하는 데에 이용될 수 있다. 또한, 그림 6의 b에 도시된 바와 같이 합성 조리개 집적 영상 기술(synthetic aperture integral imaging)은 높은 분해능을 가진 요소 영상을 기록하는 데에 사용된다.The number of surviving photons to be extracted using equation (7)

) To the respective color channels

= 200,000, it is possible to restore the pergraph as shown in FIG. 7A. At this time, since the object can be recognized from the pepogram even in a cloudy water as shown in Fig. 6B, it can be confirmed that the image quality is improved substantially except for some noise portions. However, since the obtained image is a two-dimensional image without depth (thickness) information and the image quality is still unsatisfactory, in order to obtain a three-dimensional perphogram having an improved image quality, a passive three- And a passive 3-D imaging technique of integral imaging. A passive three-dimensional integrated imaging technique can be used to reconstruct a three-dimensional image by recording and displaying a plurality of two-dimensional images from various viewpoints through a lenslet array. In addition, as shown in Fig. 6 (b), synthetic aperture integral imaging is used to record element images with high resolution.

은 획득된 페플로그램으로부터

에 의해 계산될 수 있다. 여기서 샘플 평균값은 획득된 페플로그램으로부터 제거된다. 페플로그램의 전체 에너지는 픽셀 세기의 전체 합에 의해 정규화될 수 있다.Here, the focal length of the camera lens is 50 mm, the camera has 3,008 (H) x 2,000 (V) pixels, and the distance between the cameras is 2 mm. The front of the model vehicle such as an emblem or a hummer is 145 to 155 mm away from the camera. The distance from the camera to the wrecked ship model is about 175 mm. After white LED (light emitting diode) illumination was used as a passive light source and multiple perphograms were acquired with synthetic aperture integrated imaging technology,

From the obtained Pepflogram

Lt; / RTI > Where the sample mean value is removed from the acquired perphogram. The total energy of the perpetogram can be normalized by the total sum of pixel intensities.

With such an integrated imaging method, elemental perfluorograms of the order of 10 (H) X 10 (V) are obtained, and these three-dimensional object or landscape images are finally obtained by computerized integration It can be reconstructed on various planes by means of image reproduction technology. This technique is used to project an elemental image through a virtual pinhole array in the reproduction plane, and is used to superimpose all elemental images. The nested regions in different playback planes are different.

Also, as shown in Fig. 7c and Fig. 7d, the restored perphogram noise is expanded due to the low-pass filtering of the computer reproduction, since the depth of the object or scene passing through the scattering medium To obtain information, computer playback methods are performed at various depths.

8 is a diagram illustrating experimental results applied to various applications according to another embodiment of the present invention. FIG. 8A is a diagram showing a three-dimensionally reconstructed perpetogram applied to a vehicle model in hazy water and its element image according to an embodiment of the present invention. FIG. FIG. 8B is a diagram showing a three-dimensionally reconstructed perpetogram applied to a wrecked ship model in turbid water according to an embodiment of the present invention and its element image. FIG. 8C is a view showing the scenery of the misty weather, the two-dimensionally restored perpetogram, and the element image thereof according to the embodiment of the present invention.

Referring to Figs. 8A and 8B, a perphrogram is formed in which even a small bubble is displayed according to an embodiment of the present invention, even though an object in cloudy water is recorded. Also, referring to FIG. 6C, by performing perforation on actual conditions (for example, foggy weather), buildings in dense fog areas can be visualized by a two-dimensionally reconstructed perphogram do. Then, in order to evaluate the three-dimensional perphol- ogy, the mean square error (MSE) for the two- and three-dimensionally reconstructed pathograms is calculated. The mean square error of the two-dimensionally restored perphogram as shown in FIG. 6A is approximately 0.0876, and the mean square error of the three-dimensionally restored perphogram at that time is 0.0 > 0.0220 < / RTI > and 0.0218 as shown in FIG. As a result, it can be seen that the image quality of the three-dimensionally reconstructed Pepflogram is improved by four times that of the two-dimensionally reconstructed Pepflogram.

The preferred embodiments of the present invention have been described above. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the improved embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

300:

Claims (10)

Obtaining a Peplogram from light transmitted through the scattering medium by the intensity of the applied Beer-Lambert-Bouguer law;
Estimating a scattering medium by applying a specific estimation method to the acquired single cepstral flow;
Removing the estimated scattering medium in the single perphogram;
Detecting a photon by a specific calculation method from the perphogram in which the scattering medium is removed, and restoring the two-dimensional original image;
Superimposing the images using the restored two-dimensional original image, and restoring the superimposed image into a three-dimensional image using a 3D passive imaging technique; And reconstructing the original image. The method according to claim 1,
Wherein the step of estimating a scattering medium by applying a specific estimation method to the obtained single cepstralgram estimates a scattering medium based on a Maximum Likelihood Estimation method or a Bayesian estimation method, . The method according to claim 1,
The step of detecting a photon by a specific calculation method from the perphogram in which the scattering medium has been removed to restore a two-dimensional original image may be performed by photon counting imaging based on a Poisson distribution, And extracting the original image by using a calculation method by using the calculation method. The method according to claim 1,
Wherein the step of detecting photons from the perfluorogram from which the scattering medium is removed has the range of the scattering medium between 1% and 10% of the size of the input image. The method according to claim 1,
Wherein the step of detecting photons from the perlogogram in which the scattering medium is removed comprises measuring the number of living photons varying from the perlogogram,
Wherein the number of surviving photons ranges from 1% to 10% of the number of pixels of the input image. The method according to claim 1,
Wherein the natural image is used to reconstruct the original image into a three-dimensional image. The method according to claim 1,
And restoring the original image into a three-dimensional image using natural light or infrared rays. The method according to claim 1,
Wherein a visible ray and an infrared ray are used to reconstruct the original image into a three-dimensional image. The method according to claim 1,
Wherein a radar is used to reconstruct the original image into a three-dimensional image. A recording medium on which a program produced by the method according to any one of claims 1 to 9 is recorded.

Images