KR20200015001A - Apparatus and method for image processing - Google Patents

Abstract

Disclosed are an image processing apparatus and a method thereof. According to the present invention, the image processing method comprises the steps of: receiving a plurality of consecutive images; comparing feature points of a first image and a previous image inputted before the first image to obtain motion information of a background in the first image when the first image among a plurality of images is inputted; generating a background matching image generated by correcting the background in the first image based on the acquired motion information; acquiring a background image generated by removing noise components from the background matching image and a noise image by using a main component analysis algorithm; and outputting the background image from which noise components are removed. Accordingly, the image processing apparatus can remove various noises included in images captured through a thermal imaging camera.

Description

Apparatus and method for image processing

The present invention relates to an image processing apparatus and method, and more particularly, to an image processing apparatus and method for removing noise included in an image captured by a thermal imaging camera.

Electro-optical tracking equipment or devices that detect / track / acquire targets capture images using thermal imaging cameras.

However, an image captured by a thermal imaging camera provided in such equipment or apparatus includes various noises such as bad pixels, streaks, noise fixed on the screen, and the like.

Conventionally, various noise reduction methods such as non-uniformity correction (NUC), two-point correction (TPC), scene based NUC (SBNUC), feedback-integrated scene cancellation (FiSC), and weighted nuclear norm minimization (WNNM) are employed. Noise in the image taken by the thermal imaging camera was removed.

However, when the noise in the image is removed through the conventional noise removal method, the noise is regenerated according to the elapsed time, or there is a problem in that the screen is temporarily not seen during the noise removal in the image.

In addition, the conventional noise removal method is only for noise (fixed pattern (grid, cloud, etc.), shading phenomenon, dead pixels) belonging to a static characteristic that is fixed in time according to time-varying conditions among the noise included in the image. Improvements were possible, and there are limitations that cannot be improved for moving characteristics such as moving dots and lines, wavefront abberation, and long term variant (LTV) fixed pattern noise.

Therefore, there is a need to find a way to remove not only noise belonging to static characteristics but also noise having dynamic characteristics.

The present invention has been made in view of the above-described necessity, and an object of the present invention is a technique for improving noise included in an image captured by a thermal imaging camera.

The image processing method according to an embodiment of the present invention for achieving the above object is a step of receiving a plurality of consecutive images, when the first image of the plurality of images, the first image and the first image Obtaining motion information of the background in the first image by comparing feature points of a previous image input before the image, and generating a background matched image correcting the background in the first image based on the obtained motion information And acquiring a background image from which the noise component is removed and a noise image from the background matching image using a principal component analysis algorithm, and outputting the background image from which the noise component is removed.

The background matched image may include a motion noise component changed from a noise component included in the first image by the motion information.

The generating of the background registration image may be an image corrected to match the background in the first image with the background in the previous image based on the motion information.

The acquiring of the motion information may include comparing a feature point of the second image and the first image when the second image is input while the noise image is acquired or the background image is output. You can obtain motion information of your background.

The method may further include comparing the motion information of the background in the second image with a preset threshold value and if the motion information of the background in the second image is less than or equal to the preset threshold value, using the acquired noise image. The method may further include removing a noise component included in the image.

The generating may further include correcting the background in the second image based on the motion information of the background in the second image when the motion information of the background in the second image exceeds the preset threshold. An image can be generated.

The plurality of images may be images captured by a thermal image sensor.

According to another embodiment of the present disclosure, the image processing apparatus may include the first image and the first image when a first image of an input unit, an output unit, and a plurality of images, which receives a plurality of consecutive images, is input. Compares feature points of a previous image input before the image to obtain motion information of the background in the first image, and generates a background matched image correcting the background in the first image based on the obtained motion information, And a processor configured to control the output unit to output a background image from which the noise component has been removed from the background matching image and a noise image using an analysis algorithm, and then output the background image from which the noise component has been removed.

The background matched image may include a motion noise component changed from a noise component included in the first image by the motion information.

The processor may correct the background of the first image to match the background of the previous image based on the motion information.

When the second image is input while the noise image is acquired or the background image is output, the processor compares the feature points of the second image and the first image to determine motion information of the background in the second image. Can be obtained.

The processor may compare the motion information of the background in the second image with a preset threshold value and, if the motion information of the background in the second image is less than or equal to the preset threshold value, use the obtained noise image. The noise component included in the second image may be removed.

When the motion information of the background in the second image exceeds the preset threshold, the processor corrects the background in the second image based on the motion information of the background in the second image to generate a background matched image. Can be generated.

The plurality of images may be images captured by a thermal image sensor.

As described above, the image processing apparatus according to the present invention includes various noises (fixed-type noise, shadow-patterned noise, dead pixels, moving dots or string-shaped noises of varying shapes, screens, etc.) included in images captured by a thermal imaging camera. It is effective for removing noises of moving components.

1 is a block diagram of an image processing apparatus according to an embodiment of the present invention;
2A is a first detailed block diagram of a processor of an image processing apparatus according to an embodiment of the present invention;
FIG. 2B is a second detailed block diagram of a processor of an image processing apparatus according to another embodiment of the present invention; FIG.
3 is a detailed block diagram of an image processing apparatus according to an embodiment of the present invention;
4 is an exemplary diagram of generating a background matched image in an image processing apparatus according to an embodiment of the present invention;
5 illustrates an example of generating a background image from which noise components are removed by using a background matching image generated by an image processing apparatus according to an embodiment of the present invention;
6 is an exemplary diagram of measuring a noise component in an image using an SSNR noise measuring method according to an embodiment of the present invention;
FIG. 7 is an exemplary diagram illustrating a result of measuring a noise component between an image including a noise component and an image from which noise is removed by an image processing apparatus using the SNM noise measuring method according to an embodiment of the present invention; FIG.
8 is an exemplary view illustrating a result of measuring image quality using a Variant Structural Similarity (VSSIM) noise measuring method according to an embodiment of the present invention;
9 is an exemplary view illustrating an image in which noise is improved through a noise improvement method and a conventional noise improvement method of an image processing apparatus according to the present invention;
10 is a flowchart illustrating an image processing method for removing noise included in an image in an image processing apparatus according to an embodiment of the present invention.

Terms used herein will be briefly described, and the present invention will be described in detail.

The terminology used in the embodiments of the present invention has been selected as widely used as possible terms in consideration of the functions of the present invention, but may vary according to the intention or precedent of the person skilled in the art, the emergence of new technologies, etc. . In addition, in certain cases, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the invention. Therefore, the terms used in the present invention should be defined based on the meanings of the terms and the general contents of the present invention, rather than simply the names of the terms.

Embodiments of the present invention may be variously modified and have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the scope to the specific embodiment, it should be understood to include all transformations, equivalents, and substitutes included in the spirit and technical scope of the invention. In describing the embodiments, when it is determined that the detailed description of the related known technology may obscure the gist, the detailed description thereof will be omitted.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are only used to distinguish one component from another.

Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "consist" are intended to indicate that there is a feature, number, step, action, component, part, or combination thereof described in the specification, and one or more other It is to be understood that the present invention does not exclude in advance the possibility of the presence or the addition of features or numbers, steps, operations, components, parts, or combinations thereof.

In an embodiment of the present invention, the module or unit performs at least one function or operation, and may be implemented in hardware or software, or in a combination of hardware and software. In addition, a plurality of modules or a plurality of units may be integrated into at least one module except for a module or a unit that needs to be implemented by specific hardware, and may be integrated into at least one processor (not shown). Can be.

In an embodiment of the present invention, when a part is "connected" to another part, it is not only "directly connected" but also "electrically connected" with another element in between. Also includes. In addition, not only physical connection but also wireless connection is included. In addition, when a part is said to "include" a certain component, this means that it may further include other components, except to exclude other components unless otherwise stated.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

1 is a block diagram of an image processing apparatus according to an embodiment of the present invention.

As illustrated in FIG. 1, the image processing apparatus 100 includes an input unit 110, an output unit 120, and a processor 130.

The input unit 110 receives a plurality of consecutive images. Here, the image may be an image captured by a camera including a thermal image sensor (not shown).

The output unit 120 outputs a corrected image in relation to the image input through the input unit 110.

The processor 130 controls overall operations of the components constituting the image processing apparatus 100.

In particular, when the first image of the plurality of images is input through the input unit 110, the processor 130 compares the feature points of the inputted first image with the previous image input before the first image to determine the background of the background in the first image. Obtain motion information.

Subsequently, the processor 130 generates a background registration image correcting a background in the first image based on the obtained motion information.

Here, the background matched image may include a motion noise component changed from a noise component included in the first image by previously obtained motion information. The noise component included in the first image may be fixed noise.

Then, the processor 130 obtains a background image and a noise image from which noise components are removed from the parasitic background matching image by using a principal component analysis algorithm.

As such, when the background image from which the noise component is removed and the noise image are obtained from the first image, the processor 130 controls the output unit 120 to output the background image from which the noise component is removed.

Accordingly, the output unit 120 may output a background image from which the noise component is removed.

In detail, when the first image is input, the processor 130 obtains a feature point of an object included in each of the first image currently input and the previous image input before the first image using a feature point analysis algorithm. Thereafter, the processor 130 obtains motion information between the background in the first image and the background in the previous image by comparing the feature points acquired from the first image and the previous image.

Subsequently, the processor 130 generates a background registration image correcting a background in the first image based on the previously obtained motion information.

According to an embodiment, the processor 130 may generate a background matching image by correcting the background in the first image to match the background in the previous image based on the previously obtained motion information.

However, the present invention is not limited thereto, and the processor 130 generates a background matched image by correcting the background in the first image to be matched between the background in the first image and the background in the previous image based on the previously obtained motion information. can do.

Meanwhile, a noise component may be included in the first image. Therefore, when a background matched image is generated for the first image based on the motion information, the noise component included in the first image may be changed by the previously obtained motion information.

Therefore, the background matched image with respect to the first image may include a motion noise component changed from the noise component included in the first image by the motion information.

When a background matched image including such a motion noise component is generated, the processor 130 acquires a background image and a noise image from which the noise component is removed from the background matched image for the parasitic first image using a principal component analysis algorithm. do.

An operation of acquiring the background image and the noise image from which the noise component is removed from the background matching image by using the principal component analysis algorithm will be described in detail below.

Meanwhile, the processor 130 controls the output unit 120 to output a background image from which the noise component is removed and a background image from which the noise component is removed from the background matching image for the first image. Accordingly, the output unit 120 may output a background image from which noise components have been removed in relation to the first image.

On the other hand, if the second image is input while the background image from which the noise component is removed and the noise image are obtained from the background matching image with respect to the first image or the background image from which the noise component is removed is output, the processor 130 receives the second image. Motion information of the background in the second image is obtained by comparing feature points of the image and the first image.

Thereafter, the processor 130 determines whether the motion information of the background in the second image that is obtained is less than or equal to a preset threshold.

As a result of the determination, when the motion information of the background in the second image is less than or equal to a preset threshold value, the processor 130 removes the noise component included in the second image by using the noise image previously acquired in relation to the first image.

Thereafter, the processor 130 controls the output unit 120 to output a background image from which the noise component is removed from the second image. Accordingly, the output unit 120 may output a background image from which noise components have been removed in relation to the second image.

Meanwhile, when the motion information of the background in the second image exceeds a preset threshold, the processor 130 corrects the background in the second image based on the motion information of the background in the second image, as described above, to match the background. Create an image.

Subsequently, the processor 130 obtains a background image and a noise image from which noise components are removed from the background matching image generated with respect to the second image using a principal component analysis algorithm.

Thereafter, the processor 130 controls the outputter 120 to output the background image from which the noise component is removed with respect to the second image. Accordingly, the output unit 120 may output a background image from which noise components have been removed in relation to the second image.

Hereinafter, the processor 130 for obtaining the background image from which the noise component is removed from the input image will be described in detail.

2A is a first detailed block diagram of a processor of an image processing apparatus according to an embodiment of the present invention, and FIG. 2B is a second detailed block diagram of a processor of an image processing apparatus according to another embodiment of the present invention. .

As shown in FIG. 2A, the processor 130 may include a motion analyzer 131, a background matched image generator 132, an image acquirer 133, and a comparator 134.

When a plurality of consecutive images are input through the input unit 110, the motion analyzer 131 detects feature points of a background included in the input image using a feature point detection algorithm. Here, the input image may be an image captured by a camera including a thermal image sensor (not shown).

The feature point detection algorithm may be implemented through Equation 1 below.

Here, Dx, Dy 'and Dt are partial derivatives of the image for each of the variables x, y, and t, OM is an optical movemnet vector, and Vx and Vy may be local flow vectors.

Therefore, when the first image is input through the input unit 110, the motion analyzer 131 may detect a feature point of the background included in the first image by using Equation 1 described above.

Then, the motion analyzer 131 obtains motion information of the background included in the first image by comparing the feature points detected from the input first image with the feature points detected from the previous image input before the first image is input. do.

The background matched image generator 132 generates a background matched image correcting a background in the first image based on the motion information acquired through the motion analyzer 131. In detail, the background matching image generator 132 may generate a background matching image by correcting the background in the first image to match the background in the previous image based on the previously obtained motion information.

However, the present invention is not limited thereto, and the background matching image generation unit 132 corrects the background of the first image to be matched between the background of the first image and the background of the previous image based on the previously obtained motion information. The matched image may be generated. The background matched image may include a motion noise component that is changed from a noise component included in the first image by previously obtained motion information.

The image acquirer 133 acquires a background image and a noise image from which noise components are removed from the background match image of the first image generated by the background match image generator 132 using a principal component analysis algorithm.

Here, the principal component analysis algorithm may be implemented through Equation 2 below.

Where L is a low rank part of the image, S is a sparse part of the image, and λ is a coefficient to control the amount of sparsisy S.

Thereafter, the image acquirer 133 may output the background image from which the noise component obtained from the background matched image of the first image is removed to the outputter 120.

The comparison unit 134 determines whether the motion information of the input image is equal to or less than a preset threshold.

In detail, when the second image is input after the first image is input, the motion analyzer 131 is included in the second image using the feature point analysis algorithm as described above included in the input second image. Motion information on the generated background.

When the motion information of the background included in the second image is obtained, the comparator 134 compares the motion information of the background included in the second image with a preset threshold and moves the motion of the background included in the second image. It is determined whether the information is less than or equal to a preset threshold.

As a result of determination, if the motion information of the background included in the second image is less than or equal to a preset threshold value, the image acquisition unit 133 is included in the second image using the noise image acquired from the background matching image of the first image. Remove the noise component. Thereafter, the image acquirer 133 outputs the background image from which the noise component included in the second image is removed to the outputter 120.

Meanwhile, when the motion information of the background included in the second image exceeds the preset threshold value through the comparator 134, the background matched image generator 132 may include the first and second images. The background registration image of the second image may be generated based on the motion information of the background.

Thereafter, the image acquisition unit 133 acquires a background image and a noise image from which noise components are removed from the background matching image for the second image by using a principal component analysis algorithm.

Since the operation of the background matching image generating unit 132 and the image obtaining unit 133 has been described in detail above, the detailed description thereof will be omitted.

However, the present invention is not limited thereto, and as illustrated in FIG. 2B, the comparator 134 may include a motion noise component included in the background match image for the second image generated by the background match image generator 132. And a difference between the motion noise component included in the background matching image for the first image, which is the previous image, may be equal to or less than a preset threshold.

If the difference between the two motion noise components is less than or equal to a preset threshold, the image acquisition unit 133 removes the noise component included in the second image by using the noise image acquired from the background matching image for the first image. do. Thereafter, the image acquirer 133 outputs the background image from which the noise component included in the second image is removed to the outputter 120.

On the other hand, when the motion information of the background included in the second image exceeds the preset threshold value through the comparator 134, the image acquirer 133 uses the principal component analysis algorithm to generate the background matched image generator 132. The background image and the noise image from which the noise component is removed are obtained from the background matching image of the second image obtained through the operation.

Since the operation of the background matching image generating unit 132 and the image obtaining unit 133 has been described in detail above, the detailed description thereof will be omitted.

Hereinafter, the detailed configuration of the image processing apparatus 100 described above will be described in detail.

3 is a detailed block diagram of an image processing apparatus according to an embodiment of the present invention.

As described above, the input unit 110 receives an image captured by the camera 150 including a thermal image sensor (not shown). In addition, the input unit 110 may receive various user commands and transmit them to the processor 130.

Specifically, the input unit 110 receives a user's voice command through a microphone (not shown), or a user's operation command through a keypad (not shown) having various function keys, numeric keys, special keys, text keys, and the like. Can be input. In addition, the input unit 110 may receive an IR signal or an RF signal for controlling the operation of the image processing apparatus 100 from a device that externally controls the operation of the image processing apparatus 100.

The output unit 120 may include a display unit 121 and an audio output unit 122.

The display unit 121 outputs an image. In detail, the display 120 outputs a background image from which noise components included in the image input through the input unit 110 are removed.

The audio output unit 122 may output information related to the background image from which the noise component included in the image input through the input unit 110 has been removed in the form of audio.

The image processing apparatus 100 including the input unit 110, the output unit 120, and the processor 130 may have the communication unit 140, the camera 150, and the storage as shown in FIG. 3 in addition to the above-described configuration. The unit 160 may further include.

The communication unit 140 performs data communication with a server (not shown) associated with a peripheral device (not shown) such as a smart TV, a smart phone, a military aircraft, a ship, a medical device, etc., on which the image processing apparatus 100 is mounted.

The communication unit 140 may include a wireless communication module such as a short range communication module or a wireless LAN module, a high-definition multimedia interface (HDMI), a universal serial bus (USB), or the like.

Here, the short-range communication module is a component that performs short-range communication wirelessly with the above-described peripheral device (not shown), a server (not shown), and the like. The short range communication module may include at least one of a Bluetooth module, an infrared data association (IrDA) module, a Near Field Communication (NFC) module, a Wi-Fi (WIFI) module, and a Zigbee module. .

The wireless communication module is a module connected to an external network and performing communication according to a wireless communication protocol such as EEE. In addition, the wireless communication module may further include a mobile communication module for accessing and communicating with a mobile communication network according to various mobile communication standards such as 3rd generation (3G), 3rd generation partnership project (3GPP), long term evolution (LTE), and the like. It may be.

In addition, the communication unit 140 may further include a connector that provides an interface with various source devices such as USB 2.0, USB 3.0, HDMI, IEEE 1394, and the like.

The camera 150 captures an image. In detail, the camera 150 may capture a continuous thermal image through a thermal image sensor (not shown).

The storage unit 160 may store an image captured by the camera 150, a background match image generated in relation to the captured image, a background image obtained from the background match image, a noise image, and the like.

In addition, the storage 160 may further store an operation program for controlling the operation of the image processing apparatus 100.

Here, when the image processing apparatus 100 is turned on, the operation program may be a program that is read from the storage 160 and compiled to operate each component of the image processing apparatus 100. The storage unit 160 may be a memory card (eg, an SD card or a memory stick) that can be attached / removed / removed to the ROM 136, the RAM 136, or the image processing apparatus 100 to be described later. It may be implemented as at least one of volatile memory, volatile memory, a hard disk drive (HDD), or a solid state drive (SSD).

On the other hand, the processor 130 described above may be a processing device that can control the overall operation of the image processing apparatus 100, or to control the overall operation of the image processing apparatus 100.

Such a processor 130 may include a CPU 135, a ROM 136, a RAM 137, and a GPU 138, and include a CPU 135, a ROM 136, a RAM 137, and a GPU 138. ) May be connected to each other via a bus (139).

The CPU 135 accesses the storage 160 and performs booting using an OS stored in the storage 160. In addition, the CPU 135 performs various operations using various programs, contents, data, and the like stored in the storage 160.

The GPU 136 generates a display screen including various objects such as icons, images, texts, and the like. Specifically, the GPU 136 calculates attribute values such as coordinate values, shapes, sizes, colors, and the like in which each object is to be displayed according to the layout of the screen based on the received control command, and calculates objects based on the associated attribute values. Create a display screen with various layouts to include.

The ROM 137 stores a command set for system booting. When the turn on command is input and the power is supplied, the CPU 135 copies the OS stored in the storage unit 160 to the RAM 136 according to the command stored in the ROM 137, and executes the OS to boot the system. . When the booting is completed, the CPU 135 copies various programs stored in the storage unit 160 to the RAM 136 and executes the programs copied to the RAM 136 to perform various operations.

The processor 130 may be combined with each of the above-described components and implemented as a single-chip system (System-on-a-chip or System on chip, SOC, SoC).

4 illustrates an example of generating a background matched image in an image processing apparatus according to an exemplary embodiment.

As shown in FIG. 4A, when a first image is input, the image processing apparatus 100 obtains a feature point 410 for an object included in the input first image using a feature analysis algorithm. .

Thereafter, as illustrated in FIG. 4B, the image processing apparatus 100 compares the feature points of the previous image input before the first image is input with the feature points acquired from the first image, and thus the background in the first image. And motion information 420 between the background of the previous image.

Thereafter, as illustrated in FIG. 4C, the image processing apparatus 100 generates a background registration image 430 of correcting a background in the first image based on previously obtained motion information.

FIG. 5 is an exemplary diagram of generating a background image from which noise components are removed by using a background matching image generated by an image processing apparatus according to an embodiment of the present invention.

As illustrated in FIG. 5, when the first image is input, the image processing apparatus 100 may perform a background registration image 510 on the first image based on the input first image and the previous image input before the first image. ) Can be created.

Subsequently, the image processing apparatus 100 may obtain a background image 530 and a noise image 520 from which noise components are removed from the background matching image 510 for the first image using a principal component analysis algorithm.

That is, the image processing apparatus 100 may generate the background image 530 by removing the noise component included in the noise image 520 from the background matching image 510 as shown in FIG. 5.

Hereinafter, a noise measuring method for providing a result of removing noise included in an image input through the image processing apparatus 100 according to the present invention will be described.

Hereinafter, a method of measuring the degree of noise improvement for the background image corrected by the image processing apparatus 100 according to the present invention using the following noise measuring method will be described in detail.

In the present invention, the background image corrected by the image processing apparatus 100 according to the present invention using a subtraction signal-to-noise (SSNR), a sum of noise map (SNM), and a variant structural similarity (VSSIM) noise measurement method. We can measure the degree of noise improvement for.

The SSNR (Subtraction Signal-to-noise) noise measurement method detects noise patterns and noise movements included in an image and measures how little noise components are included in the image based on the detected noise pattern and noise movement information. It can be implemented through the following [Equation 3].

Here, m, n is the size of the image, Kt and Kt + 1 are continuous images including noise, and MAXI represents the maximum pixel brightness of the image.

6 is an exemplary diagram of measuring noise components in an image by using an SSNR noise measuring method according to an exemplary embodiment of the present invention.

As shown in (b) of FIG. 6, the noise component of the horizontal line is extracted from the image 610 captured by the camera including the thermal image sensor (not shown). A noisy image 620 may be obtained.

In addition, an SSNR value corresponding to 38.6801 dB may be measured from the noise image 620 including the noise component of the horizontal line by using the aforementioned SSNR noise measurement method.

In addition, as shown in (d) of FIG. 6, the horizontal and vertical lines, as shown in (d) of FIG. 6 from the image 630 taken through a camera including a thermal image sensor (not shown) as shown in (c) of FIG. A noise image 640 including a noise component and a dot noise component may be obtained.

In addition, an SSNR value corresponding to 34.5074 dB may be measured from the noise image 640 including the noise component of the horizontal and vertical lines and the dot noise component using the aforementioned SSNR noise measurement method.

The Sum of Noise Map (SNM) noise measurement method is a method of determining a value above a threshold value as noise using a noise map.

In detail, a sum of noise map (SNM) noise measurement method analyzes an edge region in a noisy image and expands or contracts the edge region or processes an image to emphasize the edge region, thereby generating an image in which noise components are emphasized.

Subsequently, a sum of noise map (SNM) noise measurement method generates a noise map image based on the noise component included in the image in which the noise component is emphasized.

Next, a sum of noise map (SNM) noise measurement method extracts a noise component having a level higher than or equal to a predetermined threshold value among noises included in a noise map image generated in a 3D domain, and measures the extracted noise component. to be.

7 is an exemplary diagram illustrating a result of measuring a noise component between an image including a noise component and an image from which noise is removed by an image processing apparatus using the SNM noise measuring method according to an exemplary embodiment of the present invention.

The first noise map image 714 illustrated in FIG. 7A is an image generated from the first image 710 including the noise component.

That is, in the SNM noise measurement method, an edge region is analyzed in the first image 710 including the noise component, and the image processing is performed to emphasize the analyzed edge region to generate the first noise map image 714 in which the noise component is emphasized. Can be.

Subsequently, as shown in FIG. 7B, the SNM noise measuring method measures a noise component having a level equal to or higher than a predetermined threshold value among noises included in the first noise map image 714 on the 3D domain. A noise value of 179.17 k [pixels] can be calculated. That is, a noise value of 179.17k [pixels] may be calculated for the noise component included in the first image 714 through the SNM noise measurement method.

Meanwhile, the second noise map image 724 illustrated in FIG. 7C is an image generated from the second image 720 from which the noise component is removed by the image processing apparatus 100 according to the present invention.

That is, the SNM noise measuring method analyzes an edge region in the second image 720 from which the noise component is removed through the image processing apparatus 100 according to the present invention, and processes the image to emphasize the analyzed edge region, thereby reducing the noise component. The highlighted second noise map image 724 may be generated.

Subsequently, as shown in FIG. 7D, the SNM noise measuring method measures a noise component having a level higher than or equal to a predetermined threshold value among noises included in the second noise map image 724 on a 3D domain. A noise value of 17.94 k [pixels] can be calculated. That is, a result value of 17.94k [pixels] may be calculated for the noise component included in the second image 724 through the SNM noise measurement method.

Through such a result value, when performing image processing using the image processing apparatus 100 according to the present invention, the noise component in the image photographed by a camera including a thermal image sensor (not shown) is significantly reduced. It can be seen that.

VSSIM (Varians Structural Similarity) noise measurement is a method of measuring the quality of an input image using a ground truth (GT) image.

Here, the measured image may be a previous image.

That is, VSSIM (Varians Structural Similarity) noise measurement method uses the characteristics of the previous image and the input image (luminance, contrast, and structural terms) to determine the noise in the input image. It can be a way of measuring the quality of

This VSSIM (Varians Structural Similarity) noise measurement method can be implemented through Equation 4 below.

Where l is brightness, c is contrast, and s represents structural terms.

In addition, μx0 and μx1 represent local,? X0 and? X1 represent standard deviations, and? X0x1 represent cross covariance with respect to the images μx0 and μx1.

And C1 = (0.01 * L) 2, C2 = (0.03 * L) 2, C3 = C2 / 2, where L is a specific dynamic range value.

The value calculated through Equation 4 exists between −1 and 1, and the closer to 1, the higher the quality of the image.

Using this VSSIM (Varians Structural Similarity) noise measurement method, the quality of the image including the noise and the quality of the image from which the noise is removed can be measured.

8 is an exemplary view illustrating a result of measuring image quality using a Variant Structural Similarity (VSSIM) noise measuring method according to an embodiment of the present invention.

The image illustrated in FIG. 8A is a first image 810 including noise, and the image illustrated in FIG. 8B is obtained by removing noise through the image processing apparatus 100 according to the present invention. The second image 820 may be input.

8C may be a first noise map image 811 for noise included in the first image 810, and the image illustrated in FIG. 8D may be viewed. The image processing process 100 according to the present invention may be a second image of the second map 820 of the noise-removed second image 820.

When the VSSIM of each of the first and second noise map images 811 and 821 is measured, the VSSIM value of the first noise map image 811 is 0.8814 and the VSSIM value of the second noise map image 821 is 0.9947. can be dl

Through this experiment, it can be seen that the second image 820 from which the noise is removed through the image processing apparatus 100 according to the present invention has a higher quality than the first image 810.

Hereinafter, the experimental results of the noise removing method and the conventional noise removing method according to the present invention will be described.

9 is an exemplary view illustrating an image in which noise is improved through a noise improvement method and a conventional noise improvement method of the image processing apparatus according to the present invention.

As shown in FIG. 9, the background image 930 from which the noise included in the image is removed through the image processing apparatus 100 according to the present invention, and the image included in the image using conventional noise removal methods (NUC, WNNM) are included. When comparing the background images 910 and 920 from which the noise is removed, the quality of the background image 930 from which the noise is included in the image is reduced by the image processing apparatus 100 according to the present invention. It can be seen that it is superior to the quality of the background images 910 and 920 from which the noise is removed.

That is, an enlarged region 911 in which specific regions 911, 921, 931 are enlarged in each of the background images 910 ˜ 930 from which noise is removed through the noise removing method and the conventional noise removing method of the image processing apparatus 100 according to the present invention. ', 921', 931 '), it can be seen that the noise removal method of the image processing apparatus 100 according to the present invention is improved in the noise included in the image compared to the conventional noise removal method.

That is, the noise removal method using the image processing apparatus 100 according to the present invention may remove various noise types included in an image than conventional noise removal methods (NCU, TPC, SBNUC, FiSC).

Specifically, noise reduction methods of conventional non-unifromity correction (NUC), two-point correction (TPC), scene-based NUC (SBNUC), feedback-integrated scene cancellation (FiSC), and weighted nuclear minimization (WNNM) are each image removed. It can be seen that only the noise included in the Baisc Types of Noise included in the included noise is improved, but the noise of the dynamic noise is not improved.

Since the noise cancellation method of each of the NCU, TPC, SBNUC, and FiSC is a known technique, detailed description thereof will be omitted.

On the other hand, when using the noise removal method according to the present invention, it can be seen that all the noise belonging to Baisc Types of Noise and Problematic Noise included in the image is improved.

Until now, image processing is performed using an operation of performing image processing for removing noise included in an image in the image processing apparatus 100 according to the present invention, and a noise removing method of the image processing apparatus 100 according to the present invention. The experimental results performed were described in detail.

Hereinafter, an image processing method for removing noise included in an image in the image processing apparatus 100 according to the present invention will be described in detail.

10 is a flowchart illustrating an image processing method for removing noise included in an image in an image processing apparatus according to an embodiment of the present invention.

As shown in FIG. 10, the image processing apparatus 100 receives a plurality of consecutive images (S1010). When an image is input, the image processing apparatus 100 obtains motion information by comparing feature points of the input image and the previous image (S1020).

Here, the plurality of continuous images may be images captured by a camera including a thermal image sensor (not shown).

Thereafter, the image processing apparatus 100 compares the previously obtained motion information with a preset threshold value (S1030).

As a result of the comparison, if the motion information is less than or equal to a predetermined threshold value, the image processing apparatus 100 generates a background registration image correcting a background in the first image based on the motion information (S1040). .

Here, the background matched image may include a motion component changed from a noise component included in the input image by previously obtained motion information. The noise component included in the input image may be fixed noise.

Thereafter, the image processing apparatus 100 obtains a background image and a noise image from which noise components are removed from a background match image generated with respect to the input image using a principal component analysis algorithm, and outputs the obtained background image (S1050, S1060).

On the other hand, if the motion information is less than or equal to a predetermined threshold in step S1030 described above, the image processing apparatus 100 stores a noise image (hereinafter referred to as a previous noise image) generated in relation to the input image before the input image is input. It is determined whether there exists a wealth (1070).

As a result of the determination, if the previous noise image does not exist in the storage unit, the image processing apparatus 100 performs the above-described steps S1040 to S1060.

On the other hand, if it is determined that the previous noise image generated in relation to the input image before the input image is present in the storage unit, the image processing apparatus 100 includes the previous noise image stored in the storage unit in the input image. The noise component is removed (S1080).

Thereafter, the image processing apparatus 100 may output the background image from which the noise component is removed through step S1060.

In detail, when the first image is input from among a plurality of consecutive images, the image processing apparatus 100 may use the feature analysis algorithm to determine a feature point of an object included in each of the first image and the previous image input before the first image. Acquire.

Thereafter, the image processing apparatus 100 obtains motion information between the background in the first image and the background in the previous image by comparing the feature points acquired from the first image and the previous image.

Thereafter, the image processing apparatus 100 generates a background matched image based on the previously obtained motion information. As described above, the background matched image may include a motion component changed from a noise component included in the first image by previously obtained motion information. The noise component included in the first image may be fixed noise.

When such a background matched image is generated, the image processing apparatus 100 obtains a background image and a noise image from which noise components are removed from the background matched image generated in relation to the first image by using a principal component analysis algorithm. Output the background image.

On the other hand, the image processing apparatus 100 obtains a noise image and a background image from which noise is removed from the background matching image for the first image, or when the second image is input while the corresponding background image is output, the input second image Motion information of the background in the second image is obtained by comparing the first and second feature points of the first image.

Thereafter, the image processing apparatus 100 compares the acquired motion information with a preset threshold and includes the second image using the noise image previously acquired in relation to the first image if the motion information is less than or equal to the preset threshold. Remove the noise component. Thereafter, the image processing apparatus 100 outputs a background image from which a noise component is removed from the second image.

On the other hand, if the motion information exceeds a preset threshold value, the image processing apparatus 100 obtains motion information of the background in the second image through the above-described embodiment, and based on the acquired motion information, the image processing apparatus 100 A background registration image is generated by correcting the background.

Thereafter, the image processing apparatus 100 may output a background image from which the noise component is removed from the background matching image by using a principal component analysis algorithm.

Meanwhile, the image processing method as described above may be implemented with at least one executable program, and the executable program may be stored in a non-transitory computer readable medium.

A non-transitory readable medium refers to a medium that stores data semi-permanently and is read by a device, not a medium that stores data for a short time such as a register, a cache, or a memory. Specifically, the above-described programs are random access memory (RAM), flash memory, read only memory (ROM), erasable programmable ROM (EPROM), electronically erasable and programmable ROM (EPROM), registers, hard disks, removable disks, and memory. It may be stored in various types of recording media readable by the terminal, such as a card, a USB memory, a CD-ROM, and the like.

So far I looked at the center of the preferred embodiment for the present invention.

While the above has been shown and described with respect to preferred embodiments of the present invention, the present invention is not limited to the specific embodiments described above, it is usually in the art to which the invention belongs without departing from the spirit of the invention claimed in the claims. Various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

100: image processing apparatus 110: input unit
120: output unit 121: display unit
122: audio output unit 130: processor
131: motion analysis unit 132: background registration image generation unit
133: image acquisition unit 134: comparison unit
140: communication unit 150: camera
160: storage unit

Claims (14)

In the image processing method,
Receiving a plurality of consecutive images;
Obtaining a motion information of a background in the first image by comparing feature points of the first image and a previous image input before the first image when a first image is input from the plurality of images;
Generating a background registration image by correcting a background in the first image based on the obtained motion information;
Obtaining a background image and a noise image from which noise components are removed from the background matching image using a principal component analysis algorithm; And
Outputting a background image from which the noise component is removed;
Image processing method comprising a. The method of claim 1,
The background registration image,
And a motion noise component changed from the noise component included in the first image by the motion information. The method of claim 1,
Generating the background registration image,
And an image corrected to match the background in the first image with the background in the previous image based on the motion information. The method of claim 1,
Acquiring the motion information,
When the second image is input while the noise image is obtained or the background image is output, motion information of the background in the second image is obtained by comparing feature points of the second image and the first image. Image processing method. The method of claim 4, wherein
Comparing the motion information of the background in the second image with a preset threshold value; And
Removing the noise component included in the second image by using the obtained noise image when the motion information of the background in the second image is less than or equal to the preset threshold value;
The image processing method further comprises. The method of claim 5, wherein
The generating step,
When the motion information of the background in the second image exceeds the preset threshold value, the background registration image is generated by correcting the background in the second image based on the motion information of the background in the second image. Image processing method. The method of claim 1,
The plurality of images,
The image processing method, characterized in that the image captured by the thermal image sensor. In the image processing apparatus,
An input unit configured to receive a plurality of consecutive images;
An output unit; And
When a first image of the plurality of images is input, motion information of the background in the first image is obtained by comparing feature points of the first image and a previous image input before the first image,
Generating a background registration image correcting a background in the first image based on the obtained motion information;
A processor configured to output a background image from which the noise component is removed from the background matching image and a noise image by using a principal component analysis algorithm, and then output the background image from which the noise component is removed;
Image processing apparatus comprising a. The method of claim 8,
The background registration image,
And a motion noise component changed from the noise component included in the first image by the motion information. The method of claim 8,
The processor,
And correct the background in the first image to match the background in the previous image based on the motion information. The method of claim 8,
The processor,
When the second image is input while the noise image is obtained or the background image is output, motion information of the background in the second image is obtained by comparing feature points of the second image and the first image. An image processing apparatus, characterized in that. The method of claim 11,
The processor,
The motion information of the background in the second image is compared with a preset threshold value, and when the motion information of the background in the second image is less than or equal to the preset threshold value, the motion image included in the second image is included in the second image. The image processing device, characterized in that to remove the noise component. The method of claim 12,
The processor,
When the motion information of the background in the second image exceeds the preset threshold value, the background registration image is generated by correcting the background in the second image based on the motion information of the background in the second image. Image processing device. The method of claim 8,
The plurality of images,
And an image photographed through a thermal image sensor.

Images