KR102456019B1 - Method and apparatus for target detection in infrared image using background modeling and binarization - Google Patents

Abstract

The present embodiments apply a local window of an optimized size to the reduced image of the infrared image, maximize the computational efficiency by using the integral image of the reduced image, and set two types of global thresholds according to the temperature of the target based on the background. Provided are an infrared image target detection apparatus and method capable of rapidly and accurately separating a target region by applying each to generate a binarized image.

Description

Apparatus and method for target detection of infrared images through background modeling and binarization

The technical field to which the present invention pertains relates to a background modeling and binarization method for target detection in an infrared image.

The content described in this section merely provides background information for the present embodiment and does not constitute the prior art.

The method of detecting and tracking an object in an infrared image is greatly affected by the performance of the algorithm by the accuracy of the detection result. The ability to quickly and accurately isolate a target area from an infrared image improves tracking.

The conventional infrared image detection method requires a lot of computation time because the entire image is considered in the process of modeling the background corresponding to the pixel.

In the conventional infrared image detection method, if the modeled background is removed from the original image, the average of the image from which the background has been removed becomes 0. There is a problem that a single threshold cannot be calculated.

Korean Patent Publication No. 10-1658474 (2016.09.12.)

Embodiments of the present invention apply a local window of an optimized size to the reduced image of the infrared image, maximize the computational efficiency using the integral image of the reduced image, and use two types of global windows according to the temperature of the target based on the background. The main purpose is to quickly and accurately isolate a target region through a method of generating a binarized image by applying a threshold to each.

Other objects not specified in the present invention may be additionally considered within the scope that can be easily inferred from the following detailed description and effects thereof.

According to one aspect of this embodiment, a background modeling unit for modeling a background using an original image photographed with infrared rays, a background removing unit for generating a core image by removing the modeled background from the original image, and the core image An infrared image comprising a binarization processing unit that divides the first core image and the second core image according to a temperature higher or lower than the background and performs binarization by calculating a first threshold value according to a warm-body target and a second threshold value according to a cold-body target A target detection device is provided.

The background modeling unit sets a size of a cell, sets a size of a window corresponding to an area for designating a range of the cell, reduces the original image by a preset reduction ratio to generate a reduced image, and the reduction An integral image can be generated from the image. The integral image may be an image obtained by accumulating the sum of pixels in the partial region.

The background modeling unit sets a center cell, sets a plurality of neighboring cells by skipping a predetermined pixel for cells of the same size in upper, lower, left, right, and diagonal directions of the center cell, and selects a plurality of neighboring cells from among the plurality of neighboring cells. The most similar cell can be selected.

The background modeling unit may calculate a sum of pixels of a region of a target cell to be compared in the process of selecting the most similar cell using a value of a pixel position corresponding to the integral image.

The background modeling unit may generate a background of a target pixel for background modeling by using a difference between the selected peripheral cell and the center cell, and may obtain a background image by generating a background for all pixels of the reduced image.

The background modeling unit may skip background modeling for an area corresponding to half the size of windows in the upper, lower, left, and right directions of the target pixel.

The background modeling unit may restore the background image to an original size using a reciprocal of the reduction ratio.

The binarization processing unit generates a first core image according to a positive value of the core image, generates a second core image according to a negative value of the core image, and uses the average and standard deviation of the first core image. The first threshold is calculated, the second threshold is calculated using the average and standard deviation of the second core image, a first binarized image is generated based on the first threshold, and the second threshold is based on the second threshold. to generate a second binarized image, and fuse the first binarized image and the second binarized image to obtain the target region.

The apparatus for detecting a target of the infrared image may further include a target selector configured to select a target in the target region on which the binarization has been performed.

According to another aspect of this embodiment, in the method for detecting a target by a target detecting apparatus of an infrared image, modeling a background using an original image photographed with infrared rays, removing the modeled background from the original image to obtain a core image binarization by dividing the core image into a first core image and a second core image according to a temperature higher or lower than the background, and calculating a first threshold according to a warm-body target and a second threshold according to a cold-body target It provides a target detection method comprising the step of performing.

The modeling of the background includes setting a size of a cell, setting a size of a window corresponding to an area for designating a range of the cell, and reducing the original image by a preset reduction ratio to generate a reduced image, , generate an integral image from the reduced image, set a central cell and a peripheral cell, select a peripheral cell most similar to the central cell using the integral image, and determine the difference between the selected peripheral cell and the central cell It can be used to create a background of a target pixel to be modeled as a background.

In the binarization, a first core image is generated according to a positive value of the core image, a second core image is generated according to a negative value of the core image, and a degree of dispersion of the first core image is used. to calculate the first threshold value, calculate the second threshold value using the variance of the second core image, generate a first binarized image based on the first threshold value, and generate a first binarized image based on the second threshold value A second binarized image may be generated, and the target region may be obtained by fusing the first binarized image and the second binarized image.

The target detection method may further include selecting a target from the target region in which the binarization has been performed.

As described above, according to the embodiments of the present invention, a local window of an optimized size is applied to the reduced image of the infrared image, the operation efficiency is maximized by using the integral image of the reduced image, and the target is based on the background. By applying the two types of global thresholds, respectively, according to the temperature of

Even if effects not explicitly mentioned herein, the effects described in the following specification expected by the technical features of the present invention and their potential effects are treated as described in the specification of the present invention.

1 is a diagram illustrating an apparatus for detecting a target of an infrared image according to an embodiment of the present invention.
2 is a flowchart illustrating a background modeling operation of an apparatus for detecting a target of an infrared image according to an embodiment of the present invention.
3 is a diagram illustrating a cell and a window of a reduced image processed by the apparatus for detecting a target of an infrared image according to an embodiment of the present invention.
4 is a diagram illustrating an integral image processed by the apparatus for detecting a target of an infrared image according to an embodiment of the present invention.
5 is a diagram illustrating a core image processed by the apparatus for detecting a target of an infrared image according to an embodiment of the present invention.
6 is a flowchart illustrating a binarization processing operation of an apparatus for detecting a target of an infrared image according to an embodiment of the present invention.
7 is a diagram illustrating a binary image processed by the apparatus for detecting a target of an infrared image according to an embodiment of the present invention.
8 is a flowchart illustrating a target detection method according to another embodiment of the present invention.

Hereinafter, in the description of the present invention, when it is determined that the subject matter of the present invention may be unnecessarily obscured as it is obvious to those skilled in the art with respect to related known functions, the detailed description thereof will be omitted, and some embodiments of the present invention will be described. It will be described in detail with reference to exemplary drawings.

The present embodiment detects a target in an image of surveillance and reconnaissance equipment or an image searcher, and can be applied to various image-based detection technologies, such as a detection-based tracking technology and a target area detection for target recognition.

In order to improve the target detection speed and performance, the infrared image target detection apparatus according to the present embodiment models a background using a reduced image rather than the entire original image, removes the modeled background from the original image, and then adjusts the temperature to high and low. The target region is detected by performing binarization based on the calculated two types of thresholds.

1 is a diagram illustrating an apparatus for detecting a target of an infrared image according to an embodiment of the present invention.

The infrared image target detection apparatus 100 assumes that the image is composed of a target, a background, and noise, and models the background from the image and removes it from the original image, leaving only the target area and noise and removing the noise using a threshold. to obtain the target area. After detecting the target area, the final target is selected using previously learned information or features extracted from the image.

The infrared image target detection apparatus 100 includes a background modeling unit 110 , a background removing unit 120 , a binarization processing unit 130 , and a target selecting unit 140 .

The background modeling unit 110 models the background by using the original image captured by infrared rays.

The background removing unit 120 generates a saliency image by removing the modeled background from the original image.

The binarization processing unit 130 divides the core image into a first core image and a second core image according to a temperature higher or lower than the background, and performs binarization by calculating the first threshold value according to the warm body target and the second threshold value according to the cold body target. do.

The target selection unit 140 selects a target from the target region on which the binarization has been performed.

2 is a flowchart illustrating a background modeling operation of an apparatus for detecting a target of an infrared image according to an embodiment of the present invention.

Background modeling is a process of finding a region excluding the target region in order to remove the region excluding the target region of interest from the original image. The more accurate the background is modeled, the more accurately the target can be detected, but this requires a lot of computation time.

In the background modeling process, reduced images and integral images are used to improve speed. In the resize image, the original image is reduced by 1/4, and an integral image is generated using the reduced image. Background modeling is performed using the generated integral image.

Background modeling sets the cell centered on the pixel to be modeled, and sets the cell of the same size as the cell at the upper, lower, left, right, and four-direction diagonal positions of the set center cell, skipping a certain pixel. The center cell and the most similar cell among the 8 surrounding cells are selected, and the background of the pixel to be modeled is calculated using the difference between the selected cell and the center cell. This process is performed for all the reduced pixels, and when the background modeling is completed, the completed image is restored to its original size.

FIG. 3 is a diagram illustrating cells and windows of a reduced image that the apparatus for detecting a target of an infrared image according to an embodiment of the present invention processes when modeling a background of an original image.

Window, padding, cell for modeling background B( _x0 , _y0 ) corresponding to one pixel I( _x0 , _y0 ) of original image I( _{x∈image width} , _{y∈image height} ) ) is composed of

The window is half-sized and constructed around I _(x0 , _y0) . A window is used as a section to designate a range of cells. The size and range of the window are also set as in Equation 1.

Cells are defined as C ₀ centered on I _(x0 , _y0) and C _1-8 within the window range. In this case, the size of the cell is calculated as in Equation 1.

The padding is set as in Equation 1 to provide a space between the center cell C ₀ and the neighboring cells C _{1 to 8} .

Referring to FIG. 2 , the procedure for background modeling is as follows.

The cell size is set using the expected size (width _target , height _target ) of the target in the input image (S210), and the window size is set (S220).

Thereafter, the input image is reduced using a resize rate (S230). For example, 1/4 scaling is performed. Thereafter, an integral image I' _{(x, y)} of the reduced image I _(x, y) is generated (S240).

4 is a diagram illustrating an integral image processed when the apparatus for detecting a target of an infrared image according to an embodiment of the present invention compares the similarity between cells.

When comparing the similarity between cells, the concept of the integral image for calculating the sum of pixel values (eg, gray values or brightness values) of the corresponding cells is expressed as Equation (2).

A method of obtaining the sum of pixel (eg, gray) values in a specific area using an integral image is described. When I' _(x , _y) is an integral image, it moves left and down with the upper left as the starting point. It can be created by adding all previous pixel values and the current pixel value. Therefore, the value of the lower-right pixel I' _{(image width-1} , _{image height-1)} of the integral image becomes the sum of all pixel values of I _(x , _y) .

In order to obtain the sum of the pixel values of the area A ₀ of FIG. 4 , it is calculated using the pixel sum of the areas A ₁ , A ₂ , A ₃ , and A ₄ of FIG. 4 as in Equation 2, and the sum of the areas is Equation 2 It is the same as the value of each pixel position of the integral image. The sum of regions A ₁ is equal to the integral image I' _(x1 , _y1) , the sum of regions A ₂ is equal to the integral image I' _(x2 , _y2) , and the sum of regions A ₃ is equal to the integral image I' _(x3 , _y3 ). ₎ , and the sum of the area A ₄ is equal to the integral image I' _(x4 , _y4) .

Referring back to FIG. 2 , after the integral image is generated, the following procedure is repeated.

A pixel I _(x0 , _y0) for background modeling is selected from the reduced image I _(x , _y) (S250). The center cell and the surrounding cells (C ₀ ~ C ₈ ) are set using the previously calculated cell and window sizes ( S260 ). The neighboring cell C _m most similar to C ₀ is extracted using Equation 3 (S270).

Equation 3 calculates the similarity of the cell regions of FIG. 3 to find the cell most similar to C ₀ as the center _cell . To find _{∈1, ... , 8)} .

For the extracted C _m and the center cell C ₀ , the background B _{(x0, y0)} of the selected pixel I _{(x0, y0} ) is obtained using Equations 4 and 5 (S280). This process is performed for all pixels of the reduced image I _{(x, y)} (S290), and the background image B _{(x, y)} is obtained. In this process, background modeling is not performed on an area corresponding to half the window size in the top, bottom, left, and right directions of the image I _{(x, y)} .

Background modeling is performed on the entire area corresponding to the size of 1/4 of the original image (reducing the original image), that is, the area in which the width and length are resized to 1/2 respectively. This is just a numerical value according to the number of operations, and the actual background modeling is performed for the entire area, but the window is moved using the slide window technique in the image reduced to 1/4, and when the window is determined, the cell is set again in the window to set the background method of modeling. In the set window, background modeling is performed only on the area corresponding to the central pixel of C ₀ .

For example, in order to perform background modeling on an image having a size of 640x480, it is necessary to perform central pixel background modeling by setting a window of 640x480 and calculating similar cells in the window. In this embodiment, the 40x480 image is reduced by half, each horizontally and vertically, reduced to a size of 320x240, and the operation is performed on the entire reduced area. can reduce

Equation 4 calculates the difference between the cell and the center cell found in Equation 3, calculates the difference between pixels corresponding to each cell, and takes the reciprocal of the absolute value to obtain all pixels in the cell.

Equation 5 is to obtain the modeled background pixel value using the result of Equation 4, by multiplying and adding diff _{r,c and} the pixel C _{0 r,c} corresponding to the center cell C ₀ , respectively, and adding all the values of diff r,c It is an expression of taking the reciprocal and making the multiplied value as the modeled background B _{(x0, y0)} of the image pixel I _{(x0, yo )} to be modeled.

Finally, the image is enlarged and restored to the original image scale by using the reciprocal of the resize rate of Equation 1 (S300).

5 is a diagram illustrating a core image processed by the apparatus for detecting a target of an infrared image according to an embodiment of the present invention.

As an actual infrared image, the original image (Fig. 5a) and the modeling image using the background modeling algorithm (Fig. 5b), the core image (Saliency Image, Fig. 5c), in which the modeled background is removed from the original image, are 3D 5d, 5e, and 5f are two-dimensional side views, each of which can confirm the gray level of the result compared to the gray level of the original image. Modeled background (side view, in FIG. 5 ) e) If the gray level is checked, it can be confirmed that the model is similar to the gray level of the original image.

Checking the image from which the background has been removed (core image, f of FIG. 5), considering the assumption that only the target image and noise remain when the background is removed from the original image, background modeling is performed well as the approximate average gray level is 0 it can be checked that

6 is a flowchart illustrating a binarization processing operation of an apparatus for detecting a target of an infrared image according to an embodiment of the present invention.

In the binarization process, a threshold value is calculated using a core image obtained by subtracting the background modeled through the background modeling process from the original image, and a binary image is generated using the threshold value.

In the infrared image, not only a hot source target that is hotter than the background, but also a cold source target that is colder than the background exists. For this reason, there is a problem in that it is difficult to detect a target when a single threshold is considered in the process of obtaining a threshold to detect a target region in a core image.

In the present invention, in order to overcome this problem, the core image is divided into a positive region and a negative region, each threshold is obtained, and binarization is performed using this. When calculating the threshold, the global threshold is calculated using the average and standard deviation of the target core image.

Referring to FIG. 6 , the operation for binarization is as follows.

A first core image (saliency positive) and a second core image (saliency negative) are generated by separating the positive part and the negative part of the core image (S610 and S620).

Then, the average (μ) and standard deviation (σ) of each image are obtained (S630, S640).

Thereafter, as shown in 5 and 6 of FIG. 11 , a first threshold value ( Threshold _sp ) and a second threshold value ( Threshold _sn ), which are respective global thresholds, are obtained using respective averages and standard deviations.

The first weight (K _p ) indicates how sensitively it will respond to the noise value included in the first core image by controlling the range of the standard deviation applied to the first threshold. Target information may be lost as well, and if the corresponding value is decreased, the loss of target information may be reduced, while noise suppression effect may be given.

The second weight (K _n ) indicates how sensitively to respond to the noise value included in the second core image by controlling the range of the standard deviation applied to the second threshold.

Finally, binarization is performed using the calculated threshold and the two binarized images are fused (S690).

7 is a diagram illustrating a binary image processed by the apparatus for detecting a target of an infrared image according to an embodiment of the present invention.

The image from which the modeled background is removed from the original image, that is, the core image (Saliency Image) is divided into a positive part and a negative part, respectively, a first core image (positive, FIG. 7 a) and a second core image (negative, FIG. 7 d) , and each binarization result (c and f in FIG. 7) is shown in a plan view.

In an actual infrared image, the target contains both a brighter and a darker portion than the background, so if a single threshold is obtained and binarization is performed from a general core image, the target cannot be effectively separated.

On the other hand, if the core image is divided into a negative part and a positive part as shown in b and e of FIG. 7 , it can be confirmed that the target is effectively separated as shown in c and f of FIG. 7 which is the result of each binarization.

8 is a flowchart illustrating a target detection method according to another embodiment of the present invention. The target detection method may be performed by an infrared image target detection apparatus.

In step S810, the target detection apparatus models the background using the original image captured by infrared rays.

In step S820, the target detection apparatus generates a core image by removing the modeled background from the original image.

In step S830, the target detection device divides the core image into a first core image and a second core image according to a temperature higher or lower than the background, and calculates a first threshold according to the warm-body target and a second threshold according to the cold-body target to perform binarization. carry out

In step S840, the target detection apparatus selects a target from the target region on which binarization has been performed.

By using a reduced image of 1/4 size in the background modeling process, it is possible to reduce the calculation time and reduce the phenomenon that the modeled background becomes the same as the original image. By using the integral image, it is possible to quickly perform an operation to find a similar cell during modeling.

In the binarization process, the core image, which is the background modeling result, is divided into a positive image and a negative image, and each threshold is used to effectively respond to the warm-body target and the cold-body target. In addition, by using the global threshold using the mean and standard deviation of the core image, it is possible to quickly obtain the threshold and perform binarization.

The target detection apparatus may be mounted in the form of software, hardware, or a combination thereof on a computing device or server provided with hardware elements. A computing device or server is all or part of a communication device such as a communication modem for performing communication with various devices or a wired/wireless communication network, a memory for storing data for executing a program, and a microprocessor for executing operations and commands by executing the program It can mean a variety of devices, including

The target detection apparatus may include at least one processor, a computer readable storage medium, and a communication bus.

The processor may control the target detection device to operate. For example, the processor may execute one or more programs stored in a computer-readable storage medium. The one or more programs may include one or more computer-executable instructions, which, when executed by a processor, may be configured to cause the target detection apparatus to perform operations according to an exemplary embodiment.

The computer-readable storage medium is configured to store computer-executable instructions or program code, program data and/or other suitable form of information. A program stored in a computer-readable storage medium includes a set of instructions executable by a processor. In one embodiment, the computer-readable storage medium includes memory (volatile memory, such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory device , or any other form of storage medium that can be accessed by the SAR sidelobe reduction device and can store desired information, or a suitable combination thereof.

A communication bus interconnects the various other components of the SAR side lobe reduction device, including a processor and a computer readable storage medium.

The target detection device may also include one or more input/output interfaces and one or more communication interfaces that provide interfaces for one or more input/output devices. The input/output interface and the communication interface are coupled to the communication bus. The input/output device may be coupled to other components of the device via an input/output interface.

The target detection apparatus may be implemented in a logic circuit by hardware, firmware, software, or a combination thereof, and may be implemented using a general-purpose or special-purpose computer. The device may be implemented using a hardwired device, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or the like. In addition, the device may be implemented as a system on chip (SoC) including one or more processors and controllers.

In FIGS. 2, 6, and 8, it is described that each process is sequentially executed, but this is only an exemplary description, and those skilled in the art can see Fig. 2 in the range that does not depart from the essential characteristics of the embodiment of the present invention. , Figs. 6, and 8 may be applied by changing the order of execution, or by executing one or more processes in parallel or adding other processes to various modifications and variations.

The operations according to the present embodiments may be implemented in the form of program instructions that can be performed through various computer means and recorded in a computer-readable medium. Computer-readable medium represents any medium that participates in providing instructions to a processor for execution. Computer-readable media may include program instructions, data files, data structures, or a combination thereof. For example, there may be a magnetic medium, an optical recording medium, a memory, and the like. A computer program may be distributed over a networked computer system so that computer readable code is stored and executed in a distributed manner. Functional programs, codes, and code segments for implementing this embodiment may be easily inferred by programmers in the art to which this embodiment belongs.

The present embodiments are for explaining the technical idea of the present embodiment, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The protection scope of this embodiment should be interpreted by the claims below, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present embodiment.

Claims (13)

a background modeling unit for modeling a background using an original image taken with infrared light;
a background removing unit for generating a core image by removing the modeled background from the original image; and
A binarization processing unit that divides the core image into a first core image and a second core image according to a temperature higher or lower than the background and performs binarization by calculating a first threshold value according to a warm body target and a second threshold value according to a cold body target do,
The background modeling unit sets a size of a cell, sets a size of a window corresponding to an area for designating a range of the cell, reduces the original image by a preset reduction ratio to generate a reduced image, and the reduction An integral image is generated from the image, and the integral image is an image obtained by accumulating the sum of pixels in the partial region. delete According to claim 1,
The background modeling unit,
set the center cell,
Setting a plurality of peripheral cells by skipping a certain pixel for cells of the same size in the upper, lower, left, right, and diagonal directions of the central cell,
The infrared image target detection apparatus, characterized in that selecting a cell most similar to the central cell from among the plurality of peripheral cells. 4. The method of claim 3,
The background modeling unit,
In the process of selecting the most similar cell, the sum of pixels in the area of the target cell to be compared is calculated using the value of the pixel position in the integral image. 4. The method of claim 3,
The background modeling unit,
generating a background of a target pixel to be background modeled by using the difference between the selected peripheral cell and the central cell;
The infrared image target detection apparatus, characterized in that the background image is obtained by generating a background for all pixels of the reduced image. 6. The method of claim 5,
The background modeling unit,
The apparatus for detecting a target in an infrared image, characterized in that the background modeling is skipped for an area corresponding to half the size of the windows in the upper, lower, left, and right directions of the target pixel. 6. The method of claim 5,
The background modeling unit,
The apparatus for detecting a target of an infrared image, characterized in that the background image is restored to its original size by using a reciprocal of the reduction ratio. According to claim 1,
The binarization processing unit,
generating a first core image according to the positive value of the core image;
generating a second core image according to a negative value of the core image;
calculating the first threshold using the average and standard deviation of the first core image;
calculating the second threshold using the average and standard deviation of the second core image;
generating a first binarized image based on the first threshold,
generating a second binarized image based on the second threshold,
The infrared image target detection apparatus, characterized in that by fusing the first binarized image and the second binarized image to obtain a target area. According to claim 1,
Infrared image target detection apparatus, characterized in that it further comprises a target selection unit for selecting a target in the target region on which the binarization has been performed. In the target detection method by the target detection apparatus of the infrared image,
modeling a background using the original image taken with infrared light;
generating a core image by removing the modeled background from the original image; and
Separating the core image into a first core image and a second core image according to a temperature higher or lower than the background, calculating a first threshold value according to a warm body target and a second threshold value according to a cold body target, and performing binarization; ,
The modeling of the background includes setting a size of a cell, setting a size of a window corresponding to an area for designating a range of the cell, and reducing the original image by a preset reduction ratio to generate a reduced image, , generate an integral image from the reduced image, set a central cell and a peripheral cell, select a peripheral cell most similar to the central cell using the integral image, and determine the difference between the selected peripheral cell and the central cell A target detection method, characterized in that the background of the target pixel to be modeled is generated using the background model. delete 11. The method of claim 10,
The step of performing the binarization is,
generating a first core image according to the positive value of the core image;
generating a second core image according to a negative value of the core image;
calculating the first threshold using the variance of the first key image;
calculating the second threshold using the variance of the second key image;
generating a first binarized image based on the first threshold,
generating a second binarized image based on the second threshold,
The target detection method, characterized in that by fusing the first binarized image and the second binarized image to obtain a target region. 11. The method of claim 10,
Target detection method, characterized in that it further comprises the step of selecting a target from the target region on which the binarization has been performed.

Images