WO2022019513A1 - Method for compensating autofocus and electronic device compensating autofocus - Google Patents

Abstract

A method for compensating autofocus and an electronic device that compensates autofocus are provided. The method can comprise the steps of: obtaining a point spread function (PSF) corresponding to the current partition block from among a plurality of partition blocks generated from an input image; filtering the current partition block by means of the point spread function so as to remove blur from the current partition block; post-processing samples included in the filtered current partition block on the basis of a threshold value so as to remove overshoot from the filtered current partition block; and generating an output image comprising the post-processed current partition block.

Description

Method for compensating for autofocus and electronic device for compensating for autofocus

The present disclosure relates to a method for compensating for autofocus, an electronic device for compensating for autofocus, and a computer-readable storage medium storing program codes for performing the method for compensating for autofocus.

A digital photographing apparatus includes an image pickup device that converts an optical signal into an electrical signal. A live-view screen displaying an image currently generated by the imaging device may be provided to the user by using a signal detected by the imaging device. In a digital photographing apparatus such as a camera or a camcorder, it is necessary to accurately focus on a subject in order to capture a clear still image or a moving image. Accordingly, an auto-focus (Auto-Focus, AF) function for automatically focusing on a subject is widely installed in a photographing apparatus for photographing a subject. By mounting the AF function in the photographing apparatus, even a beginner can easily acquire a high-quality photographed image in which the subject is in focus.

In general, a photographing apparatus uses an actuator to implement an optical AF function. In order to use such an actuator, a photographing device of a certain size or larger is required, and thus it may be difficult to reduce the volume of a camera for a mobile phone. Therefore, a method capable of replacing the optical AF function while reducing the volume of a camera for mobile phone or the like is required. When an actuator is not used, the lens is fixed, and the degree of blur of the acquired image may vary according to the distance from the camera to the object.

Therefore, a technique for predicting a degree of blur according to an image and reconstructing an image using a point spread function (PSF) estimated from the predicted degree of blur is required. In this case, when the blur of the image is removed using the wrongly estimated point spread function, there is a limit to the removal of the blur, and artifacts such as color defects or ring effects This may occur. Therefore, it is required to estimate an appropriate point spread function according to various images and compensate for autofocus.

An embodiment of the present disclosure may provide a method and an electronic device for reconstructing an image by removing blur from an image according to blur levels of regions constituting the image.

An embodiment of the present disclosure provides a post-processing operation for correcting an overshoot occurring in a deconvolution filtering operation, thereby maintaining the resolution of an image while maintaining a defect such as a ring effect (ring effect). A method and an electronic device capable of reducing artifacts can be provided.

An embodiment of the present disclosure may provide a method and an electronic device capable of reducing a computational load of a processor and accelerating computational execution time by performing deconvolutional filtering in a spatial domain. .

A method for compensating for autofocus, disclosed as a technical means for achieving the above technical problem, is to obtain a point spread function (PSF) corresponding to a current partition block among a plurality of partition blocks generated from an input image. Step, filtering the current partition block using the point spread function to remove blur included in the current partition block, removing overshoot included in the filtered current partition block to post-processing the samples included in the filtered current partition block based on a threshold value, and generating an output image including the post-processed current partition block.

An electronic device for compensating for autofocus, disclosed as a technical means for achieving the above-described technical problem, includes a camera unit for acquiring an input image, a display unit for outputting an output image, and a program including at least one instruction It may include a storage unit for storing, and at least one processor executing at least one instruction stored in the storage unit. The at least one processor obtains a point spread function (PSF) corresponding to a current partition block from among a plurality of partition blocks generated from the input image, and calculates a blur included in the current partition block. Filters the current partition block using the point spread function to remove, and sets samples included in the filtered current partition block to a threshold value to remove overshoot included in the filtered current partition block It is possible to post-process based on , and generate an output image including the post-processed current partition block.

As a technical means for achieving the above-described technical problem, the computer-readable recording medium may store a program for executing at least one of the embodiments of the disclosed method in a computer.

1 is a schematic diagram of a method of compensating for auto-focus (AF) by an electronic device according to an embodiment of the present disclosure;

2 is a block diagram of an electronic device according to an embodiment of the present disclosure.

3 is a block diagram of a camera unit according to an embodiment of the present disclosure.

4 is a flowchart of a method for compensating for autofocus according to an embodiment of the present disclosure.

5 is a diagram for explaining an operation of filtering a current partition block using a point spread function (PSF) corresponding to the current partition block according to an embodiment of the present disclosure.

6 is a diagram for explaining a point spread function (PSF) corresponding to a current partition block according to an embodiment of the present disclosure.

7 is a diagram for explaining an operation of determining a threshold value in a post-processing operation according to an embodiment of the present disclosure.

8 is a diagram for explaining another operation of determining a threshold value in a post-processing operation according to an embodiment of the present disclosure.

9 is a diagram illustrating a result of compensating for autofocus of an input image by applying the method of compensating for autofocus according to an embodiment of the present disclosure.

10 is a diagram illustrating another result of compensating for autofocus of an input image by applying a method of compensating for autofocus according to an embodiment of the present disclosure.

In an embodiment of the present disclosure, a method for an electronic device to compensate for autofocus may be provided. The method includes: obtaining a point spread function (PSF) corresponding to a current partition block among a plurality of partition blocks generated from an input image; to remove blur included in the current partition block; Filtering the current partition block using a spreading function, post-processing samples included in the filtered current partition block based on a threshold value to remove overshoot included in the filtered current partition block , and generating an output image including the post-processed current partition block.

In an embodiment, the point spread function corresponding to the current partition block may be a preset point spread function or a point spread function calculated through an artificial neural network.

In an embodiment, the filtering of the current partition block using the obtained point spreading function includes: obtaining a Wiener filter from the obtained point spreading function; and using the obtained Wiener filter, the current It may include filtering the partition block.

In one embodiment, the Wiener filter may be a Wiener filter in a spatial domain.

In an embodiment, the Wiener filter on the spatial domain performs a Fourier inverse transform on the Wiener filter on the Fourier domain, and a value less than or equal to a preset intensity is removed from the coefficients of the inverse Fourier-transformed Wiener filter. can be obtained by

In an embodiment, obtaining the Wiener filter may include obtaining a Wiener filter on a Fourier domain and a Wiener filter on a spatial domain. The step of filtering the current partition block using the obtained Wiener filter includes selecting the obtained Wiener filter on the Fourier domain or the Wiener filter on the spatial domain, and filtering the current partition block using the selected Wiener filter. may include

In an embodiment, the Wiener filter may be a one-dimensional directional filter or a two-dimensional directional filter.

In an embodiment, when the Wiener filter is a one-dimensional directional filter, obtaining the Wiener filter from the obtained point spread function may include obtaining filter coefficients of the Wiener filter using a lookup table. have.

In one embodiment, the threshold may be determined based on a bit-depth of the sample.

In an embodiment, the threshold value may be determined based on at least one of standard deviations of samples of a predetermined region including the current sample.

In an embodiment of the present disclosure, an electronic device for compensating for autofocus may be provided. The electronic device includes a camera unit for acquiring an input image, a display unit for outputting an output image, a storage unit for storing a program including at least one instruction, and at least one for executing at least one command stored in the storage unit. may include a processor of The at least one processor obtains a point spread function corresponding to the current partition block among a plurality of partition blocks generated from the input image, and divides the current partition block by using the point spread function to remove blur included in the current partition block. Filter, post-process samples included in the filtered current partition block based on a threshold value to remove overshoot included in the filtered current partition block, and generate an output image including the post-processed current partition block. can

In an embodiment, the point spread function corresponding to the current partition block may be a preset point spread function or a point spread function calculated through an artificial neural network.

In an embodiment, the processor may execute at least one instruction to obtain a Wiener filter from the obtained point spread function, and filter the current partition block using the obtained Wiener filter.

In one embodiment, the Wiener filter may be a Wiener filter in the spatial domain.

In an embodiment, the Wiener filter on the spatial domain may be obtained by inverse Fourier transforming the Wiener filter on the Fourier domain, and removing values equal to or less than a preset intensity from the coefficients of the inverse Fourier transform Wiener filter.

In one embodiment, the processor executes at least one instruction to obtain a Wiener filter on the Fourier domain and a Wiener filter on the spatial domain, select the obtained Wiener filter on the Fourier domain or the Wiener filter on the spatial domain, and select the selected Wiener filter can be used to filter the current partition block.

In an embodiment, the Wiener filter may be a one-dimensional directional filter or a two-dimensional directional filter.

In an embodiment, when the Wiener filter is a one-dimensional directional filter, the processor may execute at least one instruction to obtain filter coefficients of the Wiener filter from the obtained point spread function using a lookup table.

In an embodiment, the threshold value may be determined based on at least one of a bit depth of a sample and a standard deviation of samples of a predetermined region including the current sample.

In an embodiment of the present disclosure, a computer-readable recording medium in which a program for implementing a method of compensating for autofocus by an electronic device is recorded may be provided.

Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. However, the present disclosure may be implemented in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present disclosure in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

The terms used in the embodiments of the present disclosure have been selected as currently widely used general terms as possible while considering the functions of the present disclosure, but this may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, etc. . In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding embodiment. Therefore, the terms used in this specification should be defined based on the meaning of the term and the contents of the present disclosure, rather than the simple name of the term.

The singular expression may include the plural expression unless the context clearly dictates otherwise. Terms used herein, including technical or scientific terms, may have the same meanings as commonly understood by one of ordinary skill in the art described herein.

In the present disclosure, when a part "includes" a certain component, it indicates that other components may be further included, rather than excluding other components, unless otherwise stated. In addition, terms such as "~ unit" and "~ module" described in this specification indicate a unit for processing at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

Throughout the present disclosure, when a part is "connected" with another part, this includes not only the case of being "directly connected" but also the case of being "electrically connected" with another element interposed therebetween. do.

As used herein, the expression “configured to (or configured to)” depends on the context, for example, “suitable for”, “having the capacity to” It can be used interchangeably with "," "designed to", "adapted to", "made to", or "capable of". The term “configured (or configured to)” may not necessarily refer to only “specifically designed to” hardware. Instead, in some circumstances, the expression “a system configured to” may indicate that the system is “capable of” with other devices or components. For example, the phrase "a processor configured (or configured to perform) A, B, and C" refers to a dedicated processor (eg, an embedded processor) for performing the operations, or by executing one or more software programs stored in memory; It may indicate a generic-purpose processor (eg, a CPU or an application processor) capable of performing corresponding operations.

Functions related to artificial intelligence according to the present disclosure are operated through a processor and a memory. The processor may consist of one or a plurality of processors. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a digital signal processor (DSP), or the like, a graphics-only processor such as a GPU, a VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU. One or a plurality of processors control to process input data according to a predefined operation rule or artificial intelligence model stored in the memory. Alternatively, when one or more processors are AI-only processors, the AI-only processor may be designed with a hardware structure specialized for processing a specific AI model.

A predefined action rule or artificial intelligence model is characterized in that it is created through learning. Here, being made through learning means that a basic artificial intelligence model is learned using a plurality of learning data by a learning algorithm, so that a predefined action rule or artificial intelligence model set to perform a desired characteristic (or purpose) is created can indicate the burden. Such learning may be performed in the device itself on which artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and/or system. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

The artificial intelligence model may include a plurality of neural network layers. Each of the plurality of neural network layers has a plurality of weight values, and a neural network operation is performed through an operation between the operation result of a previous layer and the plurality of weights. The plurality of weights of the plurality of neural network layers may be optimized by the learning result of the artificial intelligence model. For example, a plurality of weights may be updated so that a loss value or a cost value obtained from the artificial intelligence model during the learning process is reduced or minimized. The artificial neural network may include a deep neural network (DNN), for example, a Convolutional Neural Network (CNN), a Deep Neural Network (DNN), a Recurrent Neural Network (RNN), a Restricted Boltzmann Machine (RBM), There may be a Deep Belief Network (DBN), a Bidirectional Recurrent Deep Neural Network (BRDNN), or a Deep Q-Networks, but is not limited thereto.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

1 is a schematic diagram of a method of compensating for auto-focus (AF) by an electronic device according to an embodiment of the present disclosure;

The electronic device may be a device that acquires the input image or the input image 10 and outputs the output image or the output image 15 . The electronic device is, for example, a smart phone, a tablet personal computer (PC), a mobile phone, a video phone, an e-book reader, a desktop personal computer (PC), a laptop. PC (laptop personal computer), netbook computer (netbook computer), workstation, server, PDA (personal digital assistant), PMP (portable multimedia player), MP3 player, mobile medical device, camera, wearable device (wearable device), home appliances, and other mobile or stationary computing devices. However, the electronic device is not limited to the above-described example, and the electronic device may include all kinds of devices that acquire, process, and output an image or image.

In the present disclosure, 'auto-focus compensation' may refer to improving focus of a given image. In an embodiment, as the size of the sensor of the camera increases, a deterioration phenomenon (blurring, blur) may occur in which the outer edge of the image is not in focus during short-distance shooting. For example, the deterioration phenomenon may occur radially from the center of the image. Compensation of autofocus may represent an operation of reconstructing or restoring a deteriorated image using prior information on such deterioration phenomenon.

Referring to FIG. 1 , the electronic device may acquire an input image 10 through a built-in camera unit. In an embodiment, a plurality of partition blocks may be generated 110 from the obtained input image 10 . Each of the plurality of partition blocks may correspond to at least a portion of the input image 10 . In one embodiment, a particular partition block may be contiguous with a neighboring partition block. In another embodiment, at least a part of a specific partition block and neighboring partition blocks may overlap.

The electronic device according to an embodiment of the present disclosure may acquire ( 120 ) a point spread function (PSF) corresponding to a current partition block among a plurality of partition blocks generated from the input image 10 . The point spread function (PSF) may represent a function to a deterioration phenomenon or an optical transfer function of an imaging system. The point spread function (PSF) represents the impulse response of the imaging system to a point source. A point spread function (PSF) can be viewed as an extended form of an image representing a single point object. The degree of spread or blur of the point object may be a measure of the quality of the imaging system. In a space-invariant system, the point spread function (PSF) may be the same over the entire area of the image. An image of an object captured by a specific imaging system can be viewed as a result of convolution of an actual image of the object and a point spread function (PSF) of the imaging system.

In an embodiment, the electronic device may filter 130 the current partition block using the obtained point spread function (PSF). For example, a deconvolution filter may be used for the filtering 130 operation. Since the photographed image of the object can be expressed as a convolution value of the actual image of the object and the point spread function (PSF), theoretically, the acquired (photographed) input image 10 is deconvolved with the point spread function (PSF). In case of deconvolution, an actual image of the object may be obtained. It is assumed that the captured input image 10 includes blur. Accordingly, when the plurality of partition blocks of the input image 10 are filtered 130 using the point spread function (PSF), blur included in the plurality of partition blocks may be removed.

The electronic device according to an embodiment of the present disclosure may post-process 140 samples included in the current partition block filtered 130 based on a threshold.

In an embodiment, when the deconvolution filtering operation is performed, an overshoot in which the brightness of an edge of an included object or an edge of an included text is excessively enhanced may occur in the filtered image. A sample at a position where such overshoot occurs has a large difference in value between the input image 10 and the value after the filtering 130 . Accordingly, a threshold value for the difference δ between the value in the input image 10 of a specific sample and the value after the filtering 130 is set, and the electronic device receives the filtered image 130 based on the set threshold value. In the case of the processing 140 , overshoot included in the filtered partition block may be removed, and artifacts such as a ring effect may be reduced while maintaining image resolution.

Thereafter, the electronic device may generate an output image 15 including the post-processed partition blocks by fusing the plurality of partition blocks including the post-processed 140 samples. The output image 15 may correspond to the input image 10 for which autofocus is compensated.

2 is a block diagram of an electronic device 1000 according to an embodiment of the present disclosure.

Referring to FIG. 2 , the electronic device 1000 may include a camera unit 1100 , a processor 1300 , a storage unit 1500 , and a display unit 1700 . Not all of the components shown in FIG. 2 are essential components of the electronic device 1000 . The electronic device 1000 may be implemented by more components than those illustrated in FIG. 2 , or the electronic device 1000 may be implemented by fewer components than those illustrated in FIG. 2 .

The camera unit 1100 may include a digital photographing device. In an embodiment, the camera unit 1100 may acquire an input image or an input image. In an embodiment, the camera unit 1100 may include an actuator, and the storage unit 1500 of the electronic device 1000 may include instructions for implementing an optical autofocus function using the actuator. may be

The display unit 1700 may output an output image or an output image to the outside. The display unit 1700 may display and output a visual image to the outside. In an embodiment, the display unit 1700 may include a panel. The display unit 1700 includes, for example, a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, It may include at least one of a three-dimensional display (3D display) and an electrophoretic display (electrophoretic display). However, the display unit 1700 is not limited to the above-described examples, and may include all types of displays that output and display an image or an image.

The storage 1500 may store a program to be executed by the processor 1300 to be described later in order to control the operation of the electronic device 1000 . The storage 1500 may store a program including at least one instruction for controlling the operation of the electronic device 1000 . The storage unit 1500 may store instructions and program codes readable by the processor 1300 . In an embodiment, the processor 1300 may be implemented to execute instructions or codes of a program stored in the storage 1500 . The storage 1500 may store data input to or output from the electronic device 1000 .

The storage unit 1500 is, for example, a flash memory, a hard disk, a multimedia card micro type, or a card type storage unit (eg, SD or XD storage). ), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory) , a magnetic storage unit, a magnetic disk, and an optical disk may include at least one type of storage medium. However, the storage unit 1500 is not limited to the above-described example, and may include any type of storage medium in which data may be stored.

Programs stored in the storage 1500 may be classified into a plurality of modules according to functions. For example, the storage 1500 may include a point spread function (PSF) obtaining module 1510 , a filtering module 1520 , and a post-processing module 1530 . In an embodiment, the storage 1500 may further include an artificial neural network (ANN) or a database (DB).

The processor 1300 may control the overall operation of the electronic device 1000 . For example, the processor 1300 may generally control the camera unit 1100 , the display unit 1700 , the storage unit 1500 , and the like by executing programs stored in the storage unit 1500 .

The processor 1300 may include a hardware component that performs arithmetic, logic, input/output operations and signal processing. The processor 1300 is, for example, a central processing unit (Central Processing Unit), a microprocessor (microprocessor), a graphic processor (Graphic Processing Unit), ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), Programmable Logic Devices (PLDs), and Field Programmable Gate Arrays (FPGAs) may include, but are not limited thereto.

The processor 1300 may acquire an input image or an input image through the camera unit 1100 by executing at least one command stored in the storage unit 1500 .

The processor 1300 may generate a plurality of partition blocks from the input image by executing at least one instruction stored in the storage 1500 . Each of the plurality of partition blocks may correspond to at least a part of the input image. In an embodiment, a specific partition block may be in contact with a neighboring partition block, and at least a part of the specific partition block and the neighboring partition block may overlap. Each of the plurality of partition blocks may include a plurality of samples. All samples (pixel values) included in the input image may be included in at least one partition block.

The processor 1300 executes at least one instruction of the point spread function (PSF) acquisition module 1510 among the programs stored in the storage unit 1500, so that the current partition block among a plurality of partition blocks generated from the input image is A corresponding point spread function (PSF) may be obtained. A point spread function (PSF) corresponding to the current partition block may represent an imaging impulse response applied to point objects included in the current partition block. That is, the point spread function PSF corresponding to the current partition block may indicate the degree of blur of the current partition block. For example, in the case of short-distance imaging, the degree of blur or deterioration may increase from the center of the image toward the outside. The distance from the image center to each partition block is different for each partition block, and the point spread function (PSF) corresponding to the current partition block is determined according to the distance from the image center to the current partition block. (PSF) may vary. In an embodiment, a point spread function (PSF) corresponding to the current partition block may be applied to all samples in the current partition block.

In an embodiment, the point spread function corresponding to the current partition block among the plurality of partition blocks generated from the input image is a preset point spread function (PSF) or a point spread function calculated through an artificial neural network. (PSF).

The processor 1300 executes at least one instruction of the filtering module 1520 among the programs stored in the storage 1500 to remove blur included in the current partition block, corresponding to the current partition block. The current partition block may be filtered using the acquired point spread function (PSF). In an embodiment, a deconvolution filter may be applied to the filtering operation. Since the photographed image of the object can be expressed as a convolution result of the actual image of the object and the point spread function (PSF), a plurality of partition blocks of the obtained input image are deconvolved with the corresponding point spread functions (PSF). In the case of deconvolution, an actual image of an object can be obtained. Accordingly, blur included in the partition block may be removed through the operation of filtering the partition block using the obtained point spread function (PSF).

In an embodiment, the filtering of the partition block using the obtained point spread function (PSF) includes obtaining a Wiener filter from the obtained point spread function (PSF) and using the obtained Wiener filter to filter the partition block. The deconvolution filter may be in the form of a Wiener filter. For example, the Wiener filter W(f) in the frequency domain may be defined as in Equation 1 below.

Here, f is a coordinate component in the frequency domain, H(f) is a point spread function (PSF) in the frequency domain, and H ^* (f) is a conjugate function of H(f). N(f) is a ratio of noise to signal, and may be defined as a value obtained by dividing a power spectrum of noise by a power spectrum of an actual image.

By applying W(f) to the partition block corresponding to the point spread function H(f) and multiplying W(f) by the frequency components of the samples in the partition block, the frequency components of the filtered samples in the partition block are can be created Filtered samples in the corresponding partition block may be obtained by transforming the frequency component of the filtered samples in the corresponding partition block into the spatial domain. When samples in the corresponding partition block are filtered, samples from which blur is removed may be obtained. If this process is repeated for all samples in the partition block, all blurs included in the partition block may be removed.

In an embodiment, the Wiener filter may be a Wiener filter on a spatial domain. The Wiener filter on the spatial domain is to be generated by performing a Fourier inverse transform (IFT) on the Wiener filter on the Fourier domain (frequency domain), and removing a value less than or equal to a preset intensity from the coefficients of the Fourier inverse transformed Wiener filter. can For example, the Wiener filter c(x) in the spatial domain may be defined as in Equation 2 below.

Here, x is a coordinate component in the spatial domain, and w(x) is a function in the spatial domain obtained by inverse Fourier transform (w(x)= F ^-1 ( W (f))) of the Wiener filter W(f) in the frequency domain. to be. ε is a preset intensity, for example, ε may be set such that about 99% of the total intensity is included in c(x).

If c(x) is convolved with the block image of the partition block corresponding to the point spread function H(f) in the spatial domain, the samples in the partition block are filtered, that is, samples from which blur is removed. can be obtained If this process is repeated for all samples in the partition block, all blurs included in the partition block may be removed.

In one embodiment, the processor 1300 executes at least one instruction stored in the storage unit 1500 to obtain at least one of a Wiener filter on a Fourier domain and a Wiener filter on a spatial domain, and , may select the obtained Wiener filter on the Fourier domain or the Wiener filter on the spatial domain, and filter the partition block using the selected Wiener filter. In an embodiment, the electronic device 1000 filters the plurality of partition blocks using a winner filter on the Fourier domain or a winner on the spatial domain to remove blur included in the plurality of partition blocks generated from the input image. A plurality of partition blocks may be filtered using a filter, or some partition blocks among the plurality of partition blocks may be filtered using a Wiener filter on the Fourier domain and other partition blocks may be filtered using a Wiener filter on the spatial domain. .

In an embodiment, the Wiener filter on the spatial domain may have a smaller computational load on the processor 1300 and a faster computation speed than the Wiener filter on the frequency domain (Fourier domain). Accordingly, in an embodiment, in order to reduce a computing resource allocated to an operation in a filtering operation, only a Wiener filter on the spatial domain is used or a Fourier domain ( A Wiener filter on the frequency domain) or a Wiener filter on the spatial domain can be selectively used. For example, when the computational load applied to the processor 1300 is excessive, the filtering operation may be performed using a Wiener filter in the spatial domain in order to reduce the computational load corresponding to the filtering operation.

In an embodiment, the Wiener filter may be a one-dimensional directional filter or a two-dimensional directional filter. The dimension of the Wiener filter may be determined according to the dimension of the point spread function (PSF) H(f) used when generating the filter. When the Wiener filter W(f) or c(x) is a one-dimensional directional filter, f may be expressed as u or v in Equation 1, and x may be expressed as x or y in Equation 2 . When the Wiener filter W(f) or c(x) is a two-dimensional directional filter, it may be expressed as u, v instead of f in Equation 1, and may be expressed as x, y instead of x in Equation 2 .

In an embodiment, the operation of the processor 1300 filtering the partition block using the obtained point spread function (PSF) may include filtering the partition block further using a lookup table. For example, when the Wiener filter is a one-dimensional directional filter, the operation of obtaining the Wiener filter from the obtained point spread function may include the operation of obtaining filter coefficients of the Wiener filter using a lookup table.

A lookup table or lookup table is a data structure composed of an array or associative array, and can be used to replace an arithmetic operation with a simpler array indexing operation. In an embodiment, filter coefficients of the Wiener filter may be stored in the lookup table. That is, a previously stored winner filter can be called by using the lookup table. When the filter coefficients of the Wiener filter previously stored in the lookup table are used, the Wiener filter can be obtained from the point spread function without calculating Equation 1 or Equation 2 described above. In an embodiment, a partition block may be filtered using a Wiener filter obtained using a lookup table. When the lookup table is used, the calculation of Equation 1 or 2 may not be directly performed to obtain the Wiener filter on the Fourier domain or the Wiener filter on the spatial domain, and the processing speed of the processor 1300 may be further accelerated. have. Accordingly, it is possible to reduce the computational load of the overall autofocus control method and increase the computational speed. Meanwhile, the operations of obtaining a Wiener filter using the lookup table and filtering the plurality of partition blocks using the obtained Wiener filter may be applied when the directional filter is both one-dimensional and two-dimensional. In order to increase the operation speed and reduce the capacity of the lookup table itself, it may be more effective to perform the directional filter in one dimension.

The processor 1300 executes at least one instruction of the post-processing module 1530 among the programs stored in the storage 1500 to remove an overshoot included in the filtered partition block. Samples included in the partition block may be post-processed based on a threshold value.

In an embodiment, when the deconvolution filtering operation is performed, an overshoot phenomenon in which the brightness of an edge of an object included in the filtered image or an edge of a text is excessively enhanced may occur. A sample at a location where such overshoot occurs has a large difference between a value in the input image and a value in the filtered image. Therefore, when a threshold value for the difference δ between the value in the input image of a specific sample and the value in the filtered image is determined, and the filtered image is post-processed based on the determined threshold value, Overshoot can be eliminated. In this case, it is possible to reduce artifacts such as a ring effect while maintaining the resolution of the image.

For example, when the difference δ between the original value O(x,y) of a specific sample and the deconvolution-filtered sample value D(x,y) is greater than the determined threshold value TH, it is within the determined threshold value TH. δ can be adjusted. Here, (x,y) represents the coordinates of the sample on the image. Such an operation may be expressed as a pseudo code as shown in Table 1 below.

In an embodiment, the processor 1300 may determine a threshold value for post-processing samples included in the partition block by executing at least one instruction stored in the storage unit 1500 .

In an embodiment, the threshold TH may be determined based on a bit-depth of a sample. For example, when the bit depth of the sample is N bits, the threshold TH may be determined to be ^{2 (NM).} Here, M may be any integer less than or equal to N. For example, when the bit depth of the image is 8 bits and M is determined to be 5, the threshold value TH may be 8.

In an embodiment, the threshold value TH may be determined based on a standard deviation σ of samples included in a specific region from a predetermined region including a specific sample. This case will be described in detail with reference to FIGS. 7 and 8 to be described later.

Thereafter, the processor 1300 executes at least one instruction stored in the storage unit 1500 , and fuses partition blocks including post-processed samples, thereby generating an autofocus-compensated output image. can be obtained In an embodiment, the electronic device 1000 may output an output image for which autofocus is compensated by controlling the display unit 1700 .

3 is a block diagram of a camera unit 1100 according to an embodiment of the present disclosure.

Referring to FIG. 3 , the camera unit 1100 includes a lens assembly 1110 , a flash 1120 , an image sensor 1130 , an image stabilizer 1140 , a memory 1150 (eg, a buffer memory), or an image signal processor. (1160) may be included. The lens assembly 1110 may collect light emitted from a subject, which is an image capturing object. The lens assembly 1110 may include one or more lenses. According to an embodiment, the camera unit 1100 may include a plurality of lens assemblies 1110 . In this case, the camera unit 1100 may form, for example, a dual camera, a 360 degree camera, or a spherical camera. Some of the plurality of lens assemblies 1110 may have the same lens properties (eg, angle of view, focal length, auto focus, f number, or optical zoom), or at least one lens assembly may be a different lens assembly. It may have one or more lens properties different from the lens properties of . The lens assembly 1110 may include, for example, a wide-angle lens or a telephoto lens.

The flash 1120 may emit light used to enhance light emitted or reflected from the subject. According to an embodiment, the flash 1120 may include one or more light emitting diodes (eg, a red-green-blue (RGB) LED, a white LED, an infrared LED, or an ultraviolet LED), or a xenon lamp. can The image sensor 1130 may acquire an image corresponding to the subject by converting light emitted or reflected from the subject and transmitted through the lens assembly 1110 into an electrical signal. According to an embodiment, the image sensor 1130 is, for example, one image sensor selected from among image sensors having different properties, such as an RGB sensor, a black and white (BW) sensor, an IR sensor, or a UV sensor, the same It may include a plurality of image sensors having properties, or a plurality of image sensors having different properties. Each image sensor included in the image sensor 1130 may be implemented using, for example, a charged coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor.

In response to the movement of the camera unit 1100 or the electronic device 1000 including the same, the image stabilizer 1140 moves at least one lens or the image sensor 1130 included in the lens assembly 1110 in a specific direction or Operation characteristics of the image sensor 1130 may be controlled (eg, read-out timing may be adjusted, etc.). This makes it possible to compensate for at least some of the negative effects of the movement on the image being taken. According to an embodiment, the image stabilizer 1140 uses a gyro sensor (not shown) or an acceleration sensor (not shown) disposed inside or outside the camera unit 1100 to the camera unit 1100 or the electronic device 1000 . ) can be detected. The image stabilizer 1140 may be implemented as, for example, an optical image stabilizer. The memory 1150 may temporarily store at least a portion of the image acquired through the image sensor 1130 for the next image processing operation. For example, when image acquisition is delayed according to the shutter, or a plurality of images are acquired at high speed, the acquired original image (eg, a Bayer-patterned image or a high-resolution image) is stored in the memory 1150 ), and a corresponding copy image (eg, a low-resolution image) may be previewed through the display unit 1700 . Thereafter, when a specified condition is satisfied (eg, a user input or a system command), at least a portion of the original image stored in the memory 1150 may be acquired and processed by, for example, the image signal processor 1160 . According to an embodiment, the memory 1150 may be configured as at least a part of the storage unit 1500 or as a separate memory operated independently of the memory unit 1500 .

The image signal processor 1160 may perform one or more image processing on an image acquired through the image sensor 1130 or an image stored in the memory 1150 . The one or more image processes may include, for example, depth map generation, 3D modeling, panorama generation, feature point extraction, image synthesis, or image compensation (eg, noise reduction, resolution adjustment, brightness adjustment, blurring ( blurring), sharpening (sharpening), or softening (softening) Additionally or alternatively, the image signal processor 1160 may include at least one of the components included in the camera unit 1100 (eg, an image sensor). 1130), for example, exposure time control, readout timing control, etc. The image processed by the image signal processor 1160 is stored back in the memory 1150 for further processing. or may be provided as an external component (eg, the processor 1300, the storage 1500, or the display unit 1700) of the camera unit 1100. According to an embodiment, the image signal processor 1160 ) may be configured as at least a part of the processor 1300 or as a separate processor operated independently of the processor 1300. When the image signal processor 1160 is configured with the processor 1300 and a separate processor, the image At least one image processed by the signal processor 1160 may be displayed through the display unit 1700 as it is or after additional image processing is performed by the processor 1300 .

In an embodiment, the electronic device 1000 may include a plurality of camera units 1100 each having different properties or functions. In this case, for example, at least one of the plurality of camera units 1100 may be a wide-angle camera, and at least the other may be a telephoto camera. Similarly, at least one of the plurality of camera units 1100 may be a front camera, and at least the other may be a rear camera.

4 is a flowchart of a method for compensating for autofocus according to an embodiment of the present disclosure.

A method of compensating for autofocus according to an embodiment of the present disclosure may generate a plurality of partition blocks from an input image. Each of the plurality of partition blocks may correspond to at least a part of the input image. In an embodiment, a specific partition block may be in contact with a neighboring partition block, and at least a part of the specific partition block and the neighboring partition block may overlap. A plurality of partition blocks may include a plurality of samples. All samples included in the input image may be included in at least one partition block, respectively.

In operation S410, a point spread function (PSF) corresponding to the current partition block among a plurality of partition blocks generated from the input image may be obtained. A point spread function (PSF) corresponding to a specific partition block may represent an imaging impulse response applied to point objects included in the corresponding partition block. The operation of obtaining the point spread function (PSF) corresponding to the partition block is to obtain a preset point spread function (PSF) for each partition block, or to obtain a point spread function (PSF) calculated through an artificial neural network. It may include an operation to acquire.

In step S420 , in order to remove blur included in the partition block, the partition block may be filtered using the obtained point spread function. For example, in order to remove blur included in the current partition block, the current partition block may be filtered using a point spread function obtained to correspond to the current partition block. In an embodiment, a deconvolution filter may be applied to the filtering operation, and the deconvolution filter may be in the form of a Wiener filter in the frequency domain or in the form of a Wiener filter in the spatial domain. In an embodiment, in order to reduce a computing resource allocated to an operation in a filtering operation, only a winner filter on the spatial domain is used, or a winner on the Fourier domain (frequency domain) based on the computational load of the electronic device. A filter or a Wiener filter on the spatial domain may be selectively used. In an embodiment, the Wiener filter may be a one-dimensional directional filter or a two-dimensional directional filter determined according to the dimension of the point spread function H(f) used when generating the filter. In an embodiment, the filtering of the partition block using the obtained point spread function (PSF) includes obtaining a Wiener filter from the point spread function using a lookup table, and using the obtained Wiener filter to filter the partition block. It may further include the operation of filtering.

In operation S430, in order to remove overshoot included in the filtered partition block, samples included in the filtered current partition block may be post-processed based on a threshold value. In an embodiment, when a filtering operation is performed on a photographed image using a deconvolution filter, an overshoot phenomenon in which the brightness of an edge of an object included in the filtered image or an edge of a text is excessively enhanced This can happen. In one embodiment, the samples may be post-processed based on the determined threshold to remove the overshoot while maintaining the resolution of the image. For example, the threshold value may be determined based on the bit-depth of the sample or may be determined based on the standard deviation (σ) of samples of a predetermined region including the current sample.

In step S440, an output image including the post-processed partition block may be generated. In one embodiment, by fusing partition blocks containing post-processed samples, an autofocus compensated output image may be obtained.

FIG. 5 is a diagram for explaining an operation of filtering the current partition block 511 using a point spread function (PSF) 521 corresponding to the current partition block 511 according to an embodiment of the present disclosure.

Referring to FIG. 5 , a plurality of partition blocks 510 may be generated from an input image. The plurality of partition blocks 510 may have a rectangular grid shape. A current partition block 511 among the plurality of partition blocks 510 may include at least a portion of the input image, and all samples of the input image may be included in at least one partition block 511 .

A plurality of point spread functions (PSFs) 520 may correspond to a plurality of partition blocks 510 . In an embodiment, the point spread functions (PSFs) 520 and the partition blocks 510 may have a one-to-one correspondence. In an embodiment, a plurality of partition blocks 510 may correspond to one point spread function (PSF) 520 .

Referring to FIG. 5 , for example, when an input image is divided into 25 partition blocks 510 , 25 point spread functions 520 may also be generated to correspond to the plurality of partition blocks 510 . A specific point spread function 521 may be applied to all samples in the corresponding partition block 511 . For example, when the first point spread function 521 corresponds to the first partition block 511 , the first point spread function 521 may be applied to all samples located in the first partition block 511 .

Accordingly, the plurality of partition blocks 510 may be filtered using the point spread functions 520 corresponding to the plurality of partition blocks 510 , and the filtered image 530 may be obtained from the plurality of partition blocks. can be obtained.

6 is a diagram for explaining a point spread function (PSF) corresponding to a partition block according to an embodiment of the present disclosure.

Referring to FIG. 6 , a radial blur may be formed around a focal point F during short-distance imaging, and the greater the distance from the focal point F, the greater the degree of blur. For example, at the first point 60a at the upper left from the focus, it can be seen that blur is formed as if an image object is shadowed to the upper left, and at the second point 60b at the upper right from the focus. You can see the blur is formed as the image object is shadowed to the upper right corner. In addition, at the third point 60c at the lower right from the focus, blur is formed as if the image object is shadowed to the upper left, and at the fourth point 60d at the lower left from the focus, the image object moves to the lower left. You can see the blur forming like a shadow.

Referring to the third point 60c and the fourth point 60d, in the case of the fourth point 60d having a longer linear distance from the focal point F, the blur is higher than in the case of the third point 60c. You can see the big thing happened.

As such, since the degree of blur is formed differently depending on the distance and direction from the focus (F), the point spread function (PSF) used in the filtering operation to remove the blur included in the photographed image and the focus (F) It may be set differently depending on the distance and direction from the .

7 is a diagram for explaining an operation of determining a threshold value in a post-processing operation according to an embodiment of the present disclosure.

Referring to FIG. 7 , in an embodiment, the threshold value may be determined based on the standard deviation (σ) of samples included in a predetermined region 73 including a current sample 75 to be subjected to a post-processing operation. . For example, the region 73 may have M×N pixels, and the standard deviation of the values of the M*N samples included in the region 73 may be σ. In this case, the standard deviation σ may be determined as the threshold value TH.

In one embodiment, a value 'α*σ + β' obtained by applying a scale factor (α) and an offset factor (β) to the standard deviation (σ) for samples included in the region 73 is a threshold value (TH) can also be set to

In one embodiment, after the filtering operation with the standard deviation (σ) for samples included in the predetermined region 73 including the specific sample 75 and the original value O(x,y) of the specific sample 75 , The threshold value TH may be determined based on the difference δ between the values D(x,y) of . For example, a weighted sum 'w*σ+(1-w)*δ' in which a weight is applied to the standard deviation (σ) and the difference (δ) may be determined as the threshold value (TH). Here, the weight w may have an arbitrary value between 0 and 1.

8 is a diagram for explaining another operation of determining a threshold value in a post-processing operation according to an embodiment of the present disclosure.

Referring to FIG. 8 , in an embodiment, the threshold value is based on the standard deviation (σ) of some samples among samples included in a predetermined region 83 including a current sample 85 to be subjected to a post-processing operation. can be determined by For example, after calculating the standard deviation σ using only the four corner samples 83a , 83b , 83c , and 83d of the predetermined region 83 including the current sample 85 , the calculated standard deviation σ ) may be determined as the threshold value TH.

In an embodiment, the threshold value for the post-processing operation may be determined based on a bit-depth of a sample and a standard deviation of samples included in a predetermined region including the current sample. For example, in one embodiment, the operation of determining the threshold value may be expressed as a pseudo code as shown in Table 2 below.

Here, O(x,y) is the original value of the specific sample, D(x,y) is the value after the deconvolution filtering operation of the specific sample, bit_depth is the bit depth of the sample, and M is the bit depth of the sample It is an arbitrary integer less than or equal to, and A, B, C, and D are values of the four corner samples 83a , 83b , 83c , and 83d of the predetermined area 83 including the specific sample 85 . w may be a weight having an arbitrary value between 0 and 1.

Referring to Table 2, the difference δ between the original value O(x, y) of a specific sample (x, y) and the deconvolution-filtered sample value D(x, y) may be calculated.

The second threshold value TH2 may be determined based on the bit depth (bit_depth) of the sample. For example, when the bit depth of the sample is N bits, bit_depth may be N, and M may be any integer less than or equal to N. In this case, the second threshold value TH2 may be determined to be ^{2 (NM).}

The first threshold value TH1 may be determined based on standard deviations of samples A, B, C, and D included in a predetermined region including the current sample x and y. For example, the average value Avg of the samples A, B, C, and D may be calculated, and the standard deviation σ of the samples A, B, C, and D may be calculated therefrom. Referring to Table 2, when the standard deviation σ is smaller than the absolute value of the difference value δ obtained above, the first threshold value TH1 is the weighted sum 'w*σ+(1) -w)*δ'. In other cases, that is, when the standard deviation σ is greater than or equal to the absolute value of the difference value δ, the first threshold value TH1 may be determined as the standard deviation σ.

Thereafter, the final threshold value TH for the post-processing operation is a second threshold value TH2 determined based on a bit_depth of a sample and a first threshold value determined based on a standard deviation σ of at least one sample. It may be determined from the threshold value TH1. Referring to Table 2, in an embodiment, when the first threshold value TH1 is greater than the second threshold value TH2, the final threshold value TH may be determined as the second threshold value TH2. In this case, the final threshold value TH may be determined as the first threshold value TH1.

The determined final threshold TH may be used for post-processing of the sample. For example, the difference δ between the original value O(x, y) of the sample (x, y) and the deconvolution-filtered sample value D(x, y) may be adjusted within the determined final threshold value TH.

Meanwhile, the determination of the threshold value is not limited to the above-described methods, and may be determined using a variance value, an average value, a maximum value, and the like of samples.

9 and 10 are diagrams illustrating a result of compensating for autofocus of an input image by applying a method of compensating for autofocus according to an embodiment of the present disclosure.

9 shows an enlarged view of the right end of the photographed image. Referring to FIG. 9 , the right-direction blur included in the first portion 90a and the second portion 90b of the input image 90 results in the first portion 95a and the second portion ( 95b), it can be seen that each was removed.

10 shows an enlarged view of the upper left corner of the photographed image. Referring to FIG. 10 , the upper left blur included in the first portion 100a , the second portion 100b , the third portion 100c , and the fourth portion 100d of the input image 100 is output It can be seen that the first portion 105a , the second portion 105b , the third portion 105c , and the fourth portion 105d of the image 105 have been removed, respectively.

As such, according to an embodiment of the present disclosure, the actual image may be restored by removing the blur of the photographed image according to the blur levels of regions constituting the photographed image. In addition, by performing a post-processing operation to correct an overshoot occurring in a deconvolution filtering operation, artifacts such as a ring effect can be reduced while maintaining the image resolution. have.

In the method for compensating for autofocus or the electronic device for compensating for autofocus according to an embodiment of the present disclosure, the operation of filtering a partition block or post-processing samples included in the filtered partition block based on a threshold value , may be performed only for the Y channel, or may be performed for both YUV channels or RGB channels.

An embodiment of the present disclosure may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Additionally, computer-readable media may include computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media may typically include computer readable instructions, data structures, or other data in a modulated data signal such as program modules.

In addition, the computer-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory storage medium' is a tangible device and only indicates that it does not contain a signal (eg, electromagnetic wave). It does not distinguish between storage cases. For example, the 'non-transitory storage medium' may include a buffer in which data is temporarily stored.

According to an embodiment, the method according to various embodiments disclosed in this document may be included and provided in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or through an application store (eg Play Store™) or on two user devices (eg, It can be distributed (eg downloaded or uploaded) directly or online between smartphones (eg: smartphones). In the case of online distribution, at least a portion of the computer program product (eg, a downloadable app) is stored at least on a machine-readable storage medium, such as a memory of a manufacturer's server, a server of an application store, or a relay server. It may be temporarily stored or temporarily created.

The description of the present disclosure described above is for illustration, and those of ordinary skill in the art to which the present disclosure pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present disclosure. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may also be implemented in a combined form.

The scope of the present disclosure is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present disclosure. do.

Claims (15)

1. A method for an electronic device to compensate for autofocus, the method comprising:

   obtaining a point spread function (PSF) corresponding to a current partition block among a plurality of partition blocks generated from an input image;

   filtering the current partition block using the point spread function to remove blur included in the current partition block;

   post-processing samples included in the filtered current partition block based on a threshold value to remove overshoot included in the filtered current partition block; and

   generating an output image comprising the post-processed current partition block.
2. According to claim 1,

   Filtering the current partition block using the obtained point spread function comprises:

   obtaining a Wiener filter from the obtained point spread function; and

   Filtering the current partition block by using the obtained Wiener filter.
3. 3. The method of claim 2,

   wherein the Wiener filter is a Wiener filter in a spatial domain.
4. The method of claim 3, wherein the Wiener filter on the spatial domain comprises:

   A method, obtained by performing a Fourier inverse transform on a Wiener filter on a Fourier domain, and removing a value less than or equal to a preset intensity from the coefficients of the inverse Fourier transform Wiener filter.
5. 3. The method of claim 2,

   The obtaining the Wiener filter includes obtaining a Wiener filter on a Fourier domain and a Wiener filter on a spatial domain,

   Filtering the current partition block using the obtained Wiener filter includes:

   selecting the obtained Wiener filter on the Fourier domain or the Wiener filter on the spatial domain; and

   and filtering the current partition block using the selected Wiener filter.
6. The method of claim 2, wherein when the Wiener filter is a one-dimensional directional filter,

   The obtaining of the Wiener filter from the obtained point spread function includes obtaining filter coefficients of the Wiener filter using a lookup table.
7. According to claim 1,

   The threshold value is determined based on at least one of a bit-depth of a sample and a standard deviation of samples of a predetermined region including the current sample.
8. An electronic device for compensating for autofocus, comprising:

   a camera unit for acquiring an input image;

   a display unit for outputting an output image;

   a storage unit for storing a program including at least one instruction; and

   At least one processor for executing at least one instruction stored in the storage unit,

   the at least one processor,

   obtaining a point spread function (PSF) corresponding to a current partition block among a plurality of partition blocks generated from the input image;

   Filtering the current partition block using the point spread function to remove blur included in the current partition block,

   In order to remove overshoot included in the filtered current partition block, samples included in the filtered current partition block are post-processed based on a threshold;

   and generating an output image including the post-processed current partition block.
9. 9. The method of claim 8,

   The processor executes the at least one instruction,

   obtaining a Wiener filter from the obtained point spread function,

   and filtering the current partition block by using the obtained Wiener filter.
10. 10. The method of claim 9,

    The electronic device of claim 1, wherein the Wiener filter is a Wiener filter in a spatial domain.
11. The method of claim 10, wherein the Wiener filter on the spatial domain comprises:

    An electronic device obtained by performing a Fourier inverse transform on a Wiener filter in a Fourier domain, and removing values less than or equal to a preset intensity from coefficients of the inverse Fourier transform Wiener filter.
12. 10. The method of claim 9,

    The processor executes the at least one instruction,

    Obtaining a Wiener filter on a Fourier domain and a Wiener filter on a spatial domain,

    selecting the obtained Wiener filter on the Fourier domain or the Wiener filter on the spatial domain,

    and filtering the current partition block by using the selected winner filter.
13. The method of claim 9, wherein when the Wiener filter is a one-dimensional directional filter,

    The processor executes the at least one instruction,

    An electronic device for obtaining filter coefficients of the Wiener filter from the obtained point spread function using a lookup table.
14. The method of claim 8, wherein the threshold is

    The electronic device is determined based on at least one of a bit-depth of a sample and a standard deviation of samples of a predetermined region including the current sample.
15. A computer-readable recording medium in which a program for executing the method of any one of claims 1 to 7 on a computer is recorded.

Images