KR101140414B1 - Digital Camera Apparatus for Supporting Deblurring and Method thereof - Google Patents

Abstract

The present invention receives a captured image of a predetermined subject knowing an original image, obtains a motion blur kernel from the captured image and the original image, and uses the obtained motion blur kernel according to the input of a captured image of an unknown subject. A deblurring support digital photographing apparatus and method for restoring an original image.

Description

Digital Camera Apparatus for Supporting Deblurring and Method

The present invention relates to a digital photographing apparatus and method, and more particularly to a digital photographing apparatus and method for supporting image reconstruction.

Unlike a film shooting device, a digital shooting device can take pictures no less than an expert with a simple operation, can instantly check the pictures taken without developing and printing, and semi-permanently uses clean images without deformation. The disadvantage is that it is possible.

However, when shooting with such a digital photographing device, motion blurring is often caused in the captured image due to the movement of the photographing device that occurs at the same time as the user presses the shutter, especially when taking night photographs without the help of a flash. .

Such motion blurring may be used for computer graphics for adding naturalness to a shot image, but may have negative consequences of degrading the quality of the shot image.

In general, in order to remove such motion blurring, a method of restoring the blurred image file to an original image through a blind deblur technique or the like in a computer is used.

However, this is not only easy for non-professional users to use, but also has a disadvantage in that it is not possible to recover a high resolution original image in real time.

Accordingly, an object of the present invention is to provide a deblurring assisted digital photographing apparatus and method for eliminating blurring of a photographed image in order to overcome the problems described above.

Another object of the present invention is to provide a digital photographing apparatus and method for providing a raw image by deblurring a captured image in real time.

An embodiment of the present invention for achieving the above object is a deblurring method in a digital photographing apparatus, and receives a captured image of a predetermined subject knowing an original image, and obtains a motion blur kernel from the captured image and the original image. Acquiring and reconstructing the original image by using the obtained motion blur kernel according to an input of a captured image of an unknown subject.

According to another embodiment of the present invention, an operation unit for receiving and outputting user's instruction information to a deblurring-enabled digital photographing apparatus, a photographing unit for photographing a subject and outputting a photographed image according to a user instruction output from the operation unit; Receives a captured image of a predetermined subject knowing the original image from the photographing unit, obtains a motion blur kernel from the captured image and the original image, and uses the obtained motion blur kernel according to the input of a captured image of an unknown subject And a control unit for restoring the original image.

The present invention has the advantage that the user can conveniently use to restore the image taken in real time in the digital photographing apparatus.

In addition, the present invention has the advantage that it is possible to remove the blurring of a certain pattern according to the user's shooting habits and shooting environment.

1 is a diagram illustrating an internal configuration of a deblurring assisted digital photographing apparatus according to an exemplary embodiment of the present invention.
2 is a flowchart illustrating a process of acquiring a motion blur kernel in a deblurring method in a digital photographing apparatus according to an exemplary embodiment of the present invention.
3 is a flowchart illustrating a process of debluring in a deblurring method in a digital photographing apparatus according to an exemplary embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout.

In the following description of the present invention, detailed descriptions of well-known functions or configurations will be omitted if it is determined that the detailed description of the embodiments of the present invention may unnecessarily obscure the gist of the present invention.

Prior to describing the present invention, terms used throughout the specification may be sufficiently modified according to the intention, customs, etc. of the user or operator, and the definition of the terms should be made based on the contents throughout the specification of the present invention. will be.

The present invention is to solve the blurring generated in the photographed image, it is focused on the fact that blurring generated in the captured image repeatedly takes a similar form according to the habitual hand shake characteristics and the shooting environment characteristics of the user to shoot. It is. For example, vertical motion blurring may occur in images taken by a user having a habit of shaking his hand vertically when the shutter button is pressed, and in images taken on a windy day, This is to take advantage of the high probability that horizontal motion blurrings may occur.

Meanwhile, the mathematical relationship between the blurred image and the original image may be defined by Equation 1 below.

In Equation 1, B denotes a blurred image, L denotes a latent image as a latent sharp image, and K denotes a motion blur kernel or a point diffusion function. (Point Spread function: PSF), and N represents noise. In the following description, for convenience of description, K will be referred to as a motion blur kernel, and noise N will be assumed to be a known value.

According to Equation 1, the blurred image B is a modified image modeled by the convolution of the original image L and the motion blur kernel K. To restore the original image L, The motion blur kernel K is found and deconvolved into the blurred image B. FIG.

However, as described above, since the motion blurring with a certain pattern is repeatedly generated by the user's shooting habits or the shooting environment, the motion blur kernel K also has similar values according to the shooting habits and the shooting environment of the user. do.

Therefore, in the present invention, the motion blur kernel K patterned according to the user's shooting habit or shooting environment is acquired in advance, and any image captured by a specific user later or any image captured in a specific shooting environment is acquired. The original image is reconstructed using the obtained motion blur kernel.

In the detailed description to be described later, the image may include not only a still image but also a moving image.

In addition, although an embodiment of the present invention is applied to a digital photographing apparatus, the present invention is not limited to the digital photographing apparatus. That is, the digital photographing apparatus may be a cellular phone, a PDA, a PC, an IC recorder, a digital video camera, or the like with a camera. That is, the digital photographing device may be interpreted as any device capable of photographing a subject.

In addition, the deblurring method of the present invention to be described later can of course be implemented in a computer-readable code on the recording medium. The recording medium includes all kinds of recording devices in which data which can be read by a computer system is stored.

1 is a diagram illustrating an internal configuration of a deblurring assisted digital photographing apparatus according to a preferred embodiment of the present invention as described above.

As shown in FIG. 1, the digital photographing apparatus includes a photographing unit 110, an operation unit 120, a memory 130, a display unit 140, and a controller 150.

In detail, the photographing unit 110 may include an optical unit 111, an optical driver 113, an imaging device 115, and an imaging device controller 117.

The optical unit 111 receives an optical signal from a subject and provides the optical signal to the imaging device 120. The optical unit 111 may include at least one lens, such as a zoom lens for controlling the view angle to be narrowed or widened according to the focal length, and a focus lens for focusing the subject. In addition, the optical unit 111 may further include an aperture for adjusting the amount of light.

The optical driver 113 adjusts the position of the lens, opening and closing of the iris, and the like. You can focus by shifting the position of the lens.

The optical signal transmitted through the optical unit 111 reaches the light receiving surface of the imaging element 115 to form an image of the subject. The imaging device 115 is a photoelectric conversion device for converting an optical signal into an electrical signal. For example, a CCD or a CSI may be used. The sensitivity of the imaging device 115 may be adjusted by the imaging device controller 117.

The operation unit 120 receives and outputs a control signal from the outside such as a user. The operation unit 120 exposes the image pickup device 115 to light for a predetermined time, and releases a shutter button for inputting a shutter signal for taking a picture, a power button for supplying power, and an angle of view according to the input. It includes a wide-angle zoom button and a tele-zoom button for narrowing, and various function buttons for receiving a mode selection signal light such as a text input or a shooting mode, a playback mode, and the like.

The operation unit 120 may have a variety of buttons as described above, but is not limited thereto, and may be implemented in any form that a user can input, such as a keyboard, a touch pad, a touch screen, a remote controller, and the like.

In particular, according to an embodiment of the present disclosure, the manipulation unit 120 may receive a motion blur kernel acquisition mode selection signal and a deblurring request mode selection signal.

In addition, user information or shooting environment information input according to the present invention may be input through the manipulation unit 120, and although not shown in the drawing, a separate sensor is attached to the digital photographing apparatus to detect a shooting environment state. It can also be input to the control unit. For example, the sensor may detect wind speed, humidity, temperature, and a moving speed of the user.

In addition, the shutter speed and the like may be input to the controller 150 from the shutter button included in the operation unit 120 according to the present invention.

The memory 130 stores a program such as an operating system or an application system for driving the digital photographing apparatus, and data such as data or result data, image file, etc. necessary for performing an operation. In particular, according to an embodiment of the present invention, the memory 130 stores motion blurring kernel information mapped to one or more of user information, a shooting environment, and a shutter speed, and original image information known in advance.

In addition, the digital photographing apparatus includes a display unit 140 displaying an operation state, photographed image information, or a message for setting a user mode.

In addition, the digital photographing apparatus includes a controller 150 for processing an input photographed image signal and controlling each component in accordance with an external input signal.

In particular, the controller 150 according to an embodiment of the present invention includes a motion blur kernel acquisition module 151 and a deblurring module 152.

The motion blur kernel acquisition module 151 is driven by the motion blur kernel acquisition request mode selection signal input from the user through the manipulation unit 120, and when a captured image of a subject knowing the original image is input, Equation 1 Acquire a motion blur kernel using and store in the memory 130. In this case, the motion blur kernel acquisition module 151 maps and stores setting information such as user information, a shooting environment, or a shutter speed corresponding to the obtained motion blur kernel.

The deblurring module 152 is configured to store the motion blur kernel stored in the memory 130 when an unknown subject photographed image is input according to the deblur request mode selection signal input from the user through the manipulation unit 120. Is detected and the original image is restored through Equation 1.

Detailed operations of the motion blur kernel acquisition module 151 and the deblurring module 152 will be described in more detail with reference to FIGS. 2 through 3 in the deblurring method of the digital photographing apparatus to be described later.

According to an aspect of the present invention, there is provided a method of debbling a digital blur device, which comprises: receiving a captured image of a predetermined subject knowing an original image, acquiring a motion blur kernel from the captured image and the original image; And restoring the original image using the obtained motion blur kernel according to the captured image input.

2 is a flowchart illustrating a detailed embodiment of obtaining a motion blur kernel according to the present invention, and is operated by the motion blur kernel acquisition module 151 of the digital photographing apparatus.

Referring to FIG. 2, the motion blur kernel acquisition module 151 is driven by the motion blur kernel acquisition mode selection signal input in step 210. Then, the motion blur kernel acquisition module 151 performs a motion blur kernel detection menu setting operation in step 220. In step 330, which will be described later, the motion blur kernel acquisition module 151 is obtained from which user's captured image or under what circumstances. Setting information of whether the image is obtained from the photographed image is received from the user.

The motion blur kernel detection menu may also include an item for determining whether to input a photographed image by changing the shutter speed in stages, and an item for receiving selection information for selecting an original image for detecting the motion blur kernel. May be included. The original image may be, for example, a form in which four white dots are distributed on black paper.

On the other hand, when the motion blur kernel to be obtained is a one-time use, input of setting information such as user information or shooting environment information may be omitted.

Next, the motion blur kernel acquisition module 151 receives an image photographing a subject photographing the original image known in advance in step 230. The original image may be an original image set in step 220, and two or more captured images may be input.

In operation 240, the motion blur kernel acquisition module 151 determines whether the shutter speed is changed. That is, it is determined whether the shutter speed change is set in step 220. As a result of the determination in step 240, when the shutter speed change is set, the motion blur kernel module 151 receives the image captured at the changed shutter speed in step 250, and then proceeds to step 240. Herein, step 240 may be repeated until all the captured images corresponding to the set shutter speed steps are acquired when the shutter speed change setting is set.

On the other hand, if the shutter speed change is not set as the determination result of step 240 or if all the captured images of the desired shutter speed are acquired, the motion blur kernel module 151 already knows using Equation 1 in step 260. The motion blur kernel is generated from the original image and the captured image.

In operation 270, the motion blur kernel acquisition module 151 maps and stores the calculated motion blur kernel with the information set in operation 220.

3 is a flowchart illustrating a detailed embodiment of reconstructing an original image according to a captured image input after the above-described motion blur kernel is obtained, and is operated by the deblurring module 151 of the digital photographing apparatus.

Referring to FIG. 3, when the captured image of an unknown subject is input in step 310, the deblurring module 310 determines whether or not to blur in step 320. Step 320 may be determined according to whether a user inputs a deblurring request key input signal or whether a blurring degree of the input image is greater than or equal to a predetermined threshold. Alternatively, the determination may be based on whether or not the deblurring operation is set as a basic setting of digital photographing.

If it is determined in operation 320 that the blurring operation is determined, the blurring module 152 detects a motion blur kernel for selecting a motion blur kernel suitable for reconstructing the input image from among the motion blur kernels stored in operation 330. Obtain selection information. For example, if the input photographed image is an image photographed in a weather winded by a specific user A, the detection selection information may be user A and windy weather.

In addition, the detection selection information in step 330 may be information obtained by key input from the user in the current step, or may use information previously stored as blurring basic setting information.

Next, in operation 340, the deblurring module 152 detects a motion blur kernel mapped to the obtained detection selection information in a memory. That is, one or more motion blur kernels are matched by comparing the configuration selection information mapped to the motion blur kernel stored in step 270 and the detection selection information.

In operation 350, the deblurring module 152 restores the original image of the captured image from the captured image and the detected motion blur kernel using Equation 1. In this case, since there may be more than one motion blur kernel, there may be more than one original image to be reconstructed, and the blurring module 152 may select and store the sharpest original image among them, and store two or more restored original images. It may be.

Claims (10)

Receiving a captured image of a predetermined subject that knows the original image, and obtaining a motion blur kernel from the captured image and the original image;
Mapping and storing the obtained motion blur kernel with at least one setting information among a shutter speed at the time of shooting, photographed user information, and a shooting environment;
Acquiring detection selection information including at least one of a shutter speed, photographed user information, and photographing information when photographing an unknown subject;
And restoring the original image of the unknown subject by using a motion blur kernel corresponding to the obtained detection selection information.
delete delete The method of claim 1, wherein restoring the original image
And reconstructing the original image selectively according to whether blurring generated in the input original image is equal to or greater than a predetermined threshold value.
An operation unit for receiving and outputting user's instruction information;
A photographing unit which photographs a subject and outputs a photographed image according to a user's instruction output from the operation unit;
A memory,
Receiving a captured image of a predetermined subject knowing the original image from the photographing unit, obtaining a motion blur kernel from the captured image and the original image, and at least one of a shutter speed, photographed user information, and a photographing environment Receives setting information from the operation unit, maps the acquired motion blur kernel to the memory, and stores it in the memory, and according to the input of a captured image of an unknown subject, a shutter speed when photographing the unknown subject, photographed user information, and photographed image. And a control unit for acquiring detection selection information including at least one of the information and reconstructing the original image of the unknown subject using a motion blur kernel corresponding to the acquired detection selection information. Digital shooting device with blurring support.
delete delete delete The method of claim 5, wherein the control unit
And de-blurring the original image selectively according to whether blurring generated in the input photographed image is greater than or equal to a predetermined threshold value.
6. The method of claim 5,
And a sensor for detecting photographing environment information and outputting the photographing information to the controller.

Images