KR101175752B1 - Bayesian rule-based target decision system for strapdown dual mode imaging seeker using location weighted or region growing moment modeling - Google Patents

Abstract

PURPOSE: A Bayesian rule base target determining system for a strapdown dual mode imaging seeker using location weighted or region expanding 2D moment modeling capable of determining a final target using a Bayesian rule is provided to precisely determine a miniature target and an expanded target by determining the final target using the Bayesian rule. CONSTITUTION: An infrared image sensor(30) and a visible light image sensor(50) respectively obtain an infrared ray image and a visible light image with respect to a specific area. A two dimensional moment modeling module(100) determines a first target candidate from the obtained infrared ray image through two dimensional moment modeling based on location weighting or region growing. An SIFT(Scale Invariant Feature Transform) target characteristic value extraction module(200) extracts a second target candidate by using an SIFT technique. A Bayesian rule application target determining module(300) determines a final target according to the priority of calculated probability by calculating the similarity of determined first and second target candidates using probability.

Description

BAYESIAN RULE-BASED TARGET DECISION SYSTEM FOR STRAPDOWN DUAL MODE IMAGING SEEKER USING LOCATION WEIGHTED OR REGION GROWING MOMENT MODELING}

An embodiment according to the concept of the present invention relates to a target determination technique of a strapdown searcher, and more particularly, to a system for determining a small target and an extended target based on Bayesian rules in a strapdown dual mode imager. .

A missile, one of the devices for intercepting a target, is a weapon with a device guided by a particular method until the target is reached. The missile is equipped with an image search device for processing target information, and the image search device detects, identifies, captures, and tracks a target based on an image, and transmits the result to a guided steering computer. Recently, researches on image fusion signal processing techniques using multiple sensors such as IR sensors and CCD sensors have been actively conducted to improve the target detection and identification rate. The image fusion signal processing technique obtains image information of a unique wavelength band from each of a plurality of single sensors, and fuses and analyzes the acquired image information to complement the target and background information which could not be seen by a single sensor alone. It is a technique that can be obtained by Conventional image information fusion method of a multi-sensor is a method for detecting a target by extracting a common area or feature point for the image information obtained by each of the multiple sensors, these methods are described in detail, such as 10-1051716. . However, the conventional image fusion signal processing method only detects a target by extracting a common region or feature point of image information to be fused, and thus does not reflect a difference or shape difference in brightness values due to environmental influences, and thus does not reflect a target. There was a problem that precise detection is not possible, and precise detection of multiple targets and small targets is also impossible.

The technical problem to be achieved by the present invention is to extract the features of the infrared image (Imaging Infrared, IIR) through the two-dimensional moment modeling based on location weighting or region expansion, and the Scale Invariant Feature Transform (SIFT) technique It is to provide a Bayesian rule-based targeting system that extracts features of a visible ray (VIS) image that is contrasted through and determines a final target by applying a Bayesian rule to the extracted features. .

The target system according to an exemplary embodiment of the present invention uses an infrared image sensor and a visible light sensor for acquiring an infrared image and a visible light image for a specific region, and two-dimensional moment modeling based on location weighting or area extension from the acquired infrared image. A point corresponding to the first target candidate determined from the 2D moment modeling module and the 2D moment modeling module for determining the first target candidate is set as a corresponding N × N region, and SIFT ( Applying the SIFT target feature extraction module and the Bayesian rule to derive the second target candidate using the Scale Invariant Feature Transform method, the similarity of the determined first target candidate and the second target candidate is calculated as probability. And a Bayesian rule applied target determination module for determining a final target according to the priority of the calculated probability. The.

According to an embodiment, N may be set to a real number of 15 or more.

The position-weighted two-dimensional moment modeling module according to the first embodiment of the present invention is based on a maximum brightness position acquisition unit for acquiring coordinates of a pixel having a maximum brightness value from the acquired infrared image and the obtained pixel maximum brightness position. An area setting unit for setting a predetermined area including four sub-areas having the same pixel unit, calculating an average pixel value of the sub-areas, and obtaining a maximum value coordinate of a pixel average of the predetermined area and the obtained pixel A center coordinate calculator configured to calculate a final pixel center coordinate from a maximum value coordinate of an average, and a two-dimensional moment modeling unit configured to determine the one-target candidate by performing position weighted two-dimensional moment modeling from the calculated center coordinates of the final pixel; .

The area extension two-dimensional moment modeling module according to the second embodiment of the present invention is a maximum brightness position obtaining unit for obtaining the coordinates of the pixel having the maximum brightness value from the acquired infrared image from the pixel coordinates having the obtained maximum brightness value Setting a predetermined pixel area, obtaining an equal value expansion unit that extends the brightness values of the pixels in the set pixel area to the obtained maximum brightness value, and calculating a threshold value by obtaining an average of the extended total pixels of the pixel area; A center coordinate calculation unit calculating a pixel average coordinate between a set threshold value and the maximum brightness value, and calculating a final pixel center coordinate according to the extended brightness value coordinates and the pixel average coordinates; and a center coordinate of the calculated final pixel. And a two-dimensional moment modeling unit for performing region extension two-dimensional moment modeling.

A Bayesian rule-based target determination method according to an embodiment of the present invention extracts features of an infrared image and a visible light image through position weighting or region extension 2D moment modeling and size invariant feature transformation, and extracts Bayesian to the extracted features. By applying the rules to determine the target, there is an effect that can determine very precisely the small target and the expanding target.

The detailed description of each drawing is provided in order to provide a thorough understanding of the drawings referred to in the detailed description of the invention.
FIG. 1 illustrates the results of two-dimensional Gaussian modeling of an ideal infrared image based on a maximum brightness value to help understand the target.
Figure 2 shows the result of the two-dimensional Gaussian modeling on the basis of the maximum brightness value of the actual infrared image to help the understanding of the target.
3 is an internal block diagram of a Bayesian rule-based target determination system of a strapdown dual mode image detector according to an embodiment of the present invention.
4 is an internal block diagram of a position weighted 2D moment modeling module according to an exemplary embodiment of the present invention.
FIG. 5 illustrates a diagram for describing a method in which the area setting unit illustrated in FIG. 4 sets a predetermined area from a pixel maximum brightness position.
6 is an internal block diagram of a region extension 2D moment modeling module according to another embodiment of the present invention.
FIG. 7 illustrates an example of a method of expanding a region in a 3 × 3 region by the same value expansion unit illustrated in FIG. 6.
FIG. 8 is a diagram illustrating a result of two-dimensional moment modeling of an infrared image by the two-dimensional moment modeling module illustrated in FIG. 1.
9 illustrates a result of extracting feature points and generating descriptors for the extracted feature points by applying a SIFT technique to a visible light image.
10 shows an example of the result of the target is finally selected according to the embodiment of the present invention.
11 is a flowchart illustrating a Bayesian rule-based target determination method of the strapdown dual mode image detector according to the first embodiment of the present invention in detail.
12 is a flowchart illustrating a Bayesian rule-based target determination method of the strapdown dual mode image detector according to the second embodiment of the present invention in detail.

It is to be understood that the specific structural or functional descriptions of embodiments of the present invention disclosed herein are only for the purpose of illustrating embodiments of the inventive concept, But may be embodied in many different forms and is not limited to the embodiments set forth herein.

Embodiments in accordance with the concepts of the present invention are capable of various modifications and may take various forms, so that the embodiments are illustrated in the drawings and described in detail herein. It should be understood, however, that it is not intended to limit the embodiments according to the concepts of the present invention to the particular forms disclosed, but includes all modifications, equivalents, or alternatives falling within the spirit and scope of the invention.

The terms first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The terms are intended to distinguish one element from another, for example, without departing from the scope of the invention in accordance with the concepts of the present invention, the first element may be termed the second element, The second component may also be referred to as a first component.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between. Other expressions describing the relationship between components, such as "between" and "immediately between," or "neighboring to," and "directly neighboring to" should be interpreted as well.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "comprise" or "have" are intended to indicate that the described feature, number, step, operation, component, part, or combination thereof exists, and one or more other features or numbers are present. It should be understood that it does not exclude in advance the possibility of the presence or addition of steps, actions, components, parts or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the meaning of the context in the relevant art and, unless explicitly defined herein, are to be interpreted as ideal or overly formal Do not.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, the present invention will be described in detail with reference to the preferred embodiments of the present invention with reference to the accompanying drawings.

FIG. 1 illustrates the results of two-dimensional Gaussian modeling of an ideal infrared image based on a maximum brightness value to help understand the target.

Referring to FIG. 1, (a) shows an ideal infrared image of a target, and (b) shows a two-dimensional image to help understand the target based on the maximum brightness value according to the brightness value distribution shown in (a). The results are Gaussian modeled.

That is, FIG. 1 illustrates the two-dimensional Gaussian modeling result for the ideal target image when the target maximum brightness value is near the center.

The moment m of the two-dimensional image pattern f (x, y) of the target M × N in the infrared image may be obtained as in Equation 1 below.

P is an attribute value on the horizontal axis and q is an attribute value on the vertical axis.

는 영상패턴 f를 구성하는 전체픽셀에 해당되고, 모멘트

,

는 영상패턴의 중심(centroid)인

에 해당한다.At this time, if the p = 0, q = 0, the moment

Corresponds to all pixels constituting the image pattern f, and the moment

,

Is the centroid of the image pattern

.

에에 대한 모멘트

는 하기 수학식 2와 같이 구할 수 있다.The center

Moment to

Can be obtained as in Equation 2 below.

는 하기 수학식 3과 같이 구할 수 있다.Where normalized center moment

Can be obtained as in Equation 3 below.

On the other hand, the result of the general second Gaussian modeling applied to Figure 8 to be described later is expressed to help the understanding of the target, it is made based on the following equation (4).

는 표적의 최대 밝기값을 나타내고, 상기 e는 지수함수이며, 상기 σ는 가우시안 분포의 폭을 나타내고, ｒ는 가로축의 크기, c는 세로축의 크기를 나타낸다.At this time,

Denotes the maximum brightness value of the target, e denotes an exponential function, σ denotes the width of the Gaussian distribution, r denotes the size of the horizontal axis, and c denotes the size of the vertical axis.

As described above, FIG. 1B illustrates a modeling result of an ideal infrared image of a target as a modeling result when the target maximum brightness value exists near the center.

In reality, however, the infrared image of the target is different from the ideal infrared image, and an infrared image having a different brightness or shape is obtained due to environmental influences.

Figure 2 shows the result of the two-dimensional Gaussian modeling on the basis of the maximum brightness value of the actual infrared image to help the understanding of the target.

That is, FIG. 2 is a diagram illustrating a result of performing two-dimensional Gaussian modeling on an image obtained differently and an acquired image under the influence of the real environment when acquiring an infrared image of a target.

Referring to FIG. 2, (a) shows an actual infrared image of the target, and (b) shows 2 to help understand the target based on the maximum brightness value according to the brightness value distribution shown in (a). Shows the result of performing the dimensional Gaussian modeling

When the second Gaussian modeling is performed according to the above-described Equations 1 to 4, as shown in (b) of FIG. 2, an accurate center cannot be set for the target.

Therefore, there is a need for a method capable of setting an accurate center of the target.

3 is an internal block diagram of a Bayesian rule-based target determination system of a strapdown dual mode image detector according to an embodiment of the present invention.

Referring to FIG. 3, the Bayesian rule-based target determination system (hereinafter, referred to as a 'target determination system') of the strapdown dual mode imager is an infrared image sensor (IIR) 30 and a visible light image sensor (CCD) 50. ), A 2D moment modeling module 100, a SIFT target feature value extraction module 200, and a Bayesian rule application target determination module 300.

A module in the present specification may mean hardware capable of performing functions and operations according to each name described in the present specification, and also refers to computer program code capable of performing specific functions and operations. It may also mean an electronic recording medium, such as a processor, on which computer program code is capable of performing specific functions and operations.

In other words, a module may mean a functional and structural combination of hardware for performing the technical idea of the present invention and software for driving the hardware.

The infrared image sensor 30 and the visible light image sensor 50 sense an infrared image and a visible light image, respectively, for which the subject is the same region.

That is, in order to acquire an image of the specific same region, the infrared sensor 30 and the visible light sensor 50 obtain an infrared image and a visible light image photographing one specific point (x, y) on a parallel collinear line, respectively.

In general, a small thermal target of an infrared image is brighter than a surrounding background, has the same maximum intensity value in several places, and has a shape that is not clear.

The infrared sensor 30 transmits the sensed infrared image to the 2D moment modeling module 100, and the visible light sensor 50 transmits the sensed visible light image to the SIFT target feature value extraction module 200.

The two-dimensional moment modeling module 100 detects a first target candidate most similar to an actual target based on the transmitted infrared image, and according to the first embodiment, position weighting 2 using a position weighting technique. It may be implemented as a dimensional moment modeling module, and may be implemented as an area extension two-dimensional moment modeling module using an area extension technique according to the second embodiment.

According to the first embodiment, the 2D moment modeling module 100 is implemented as a position weighted 2D moment modeling module 100-1 that performs 2D moment modeling using a position weighting technique.

4 is an internal block diagram of a position weighted 2D moment modeling module according to an exemplary embodiment of the present invention.

Referring to FIG. 4, the position weighted two-dimensional moment modeling module 100-1 includes a maximum brightness position obtaining unit 110-1, an area setting unit 130-1, a center coordinate calculating unit 150-1, and 2. The dimensional moment modeling unit 170-1 is included.

The position weighted two-dimensional moment modeling module 100-1 uses a method of converting the position of the maximum brightness value and the position of the average brightness value of the surrounding pixels instead of modeling the target moment from one pixel position in the small target.

That is, since the method is modeled to include not only the region of interest but also the surrounding region of the target, the target can be detected insensitive to the change in size and brightness when the small target is expanded.

The maximum brightness position acquirer 110-1 in the position weighted 2D moment modeling module 100-1 obtains the pixel maximum brightness position from the infrared image.

The area setting unit 130-1 sets a predetermined area for obtaining an average brightness value of adjacent pixels from the pixel maximum brightness position acquired from the maximum brightness position obtaining unit 110-1.

FIG. 5 illustrates a diagram for describing a method in which the area setting unit illustrated in FIG. 4 sets a predetermined area from a pixel maximum brightness position.

As shown in FIG. 5A, the area setting unit 130 is 3 × 3 from the position obtained from the maximum brightness position obtaining unit 110. The area is set and divided into four areas A ₁ , A ₂ , A ₃ , and A ₄ .

In this case, the area setting unit 130 are the respective areas (A _1, A _2, A _3, A ₄₎ pixels calculate the average, the respective areas (A _1, A _2, and A to _3, A ₄ The value of the largest pixel mean calculated for N) is obtained as the compensated coordinate value.

Each pixel average equation is represented by Equation 5 below.

In addition, as shown in (b) of FIG. 5, the area setting unit 130-1 sets four areas B1 by setting a 5 × 5 area from a position obtained from the maximum brightness position obtaining unit 110-1. , B2, B3, B4).

Similarly, the area setting unit 130-1 calculates a pixel average for each of the areas B ₁ , B ₂ , B ₃ , and B ₄ , and calculates the pixel averages of the areas B ₁ , B ₂ , B ₃ , and B ₄ . ₄ ) obtain a coordinate value whose pixel mean calculated for ₄ ) is compensated with the largest value.

Each pixel average calculation method uses the same method as in Equation 5 above.

As shown in FIG. 5C, the area setting unit 130-1 divides the area into four areas by setting a 7 × 7 area from the position acquired from the maximum brightness position obtaining unit 110-1. C ₁ , C ₂ , C ₃ , C ₄ ) to calculate each pixel average, and obtain a coordinate value whose pixel average is compensated for the largest value.

 In this case, the compensated coordinate value may be obtained using a method as in Equation 5.

Meanwhile, the center coordinate calculator 150-1 in the position weighted two-dimensional moment modeling module 100-1 may determine predetermined areas 3 × 3, 5 × 5, and 7 × 7 from the area setting unit 130-1. From the maximum coordinates of the pixel average obtained from each of the following to calculate the final center coordinate value (x, y in C _max (x, y)) as shown in Equation 6.

는 최대 밝기값,

는 3×3의 4개 분할 영역들(A₁, A₂, A₃, A₄)의 픽셀 평균값들 중 최대 밝기값,

는 5×5의 4개 분할 영역들(B₁, B₂, B₃, B₄)의 픽셀 평균값들 중 최대 밝기값,

는 7×7의 4개분할 영역들(C₁, C₂, C₃, C₄)의 픽셀 평균값들 중 최대 밝기값을 나타낸다.At this time,

Is the maximum brightness value,

Is the maximum brightness value among the pixel average values of the 4 divided regions A ₁ , A ₂ , A ₃ , and A ₄ ,

Is the maximum brightness value among the pixel average values of four divisions B ₁ , B ₂ , B ₃ , and B ₄ of 5 × 5,

Denotes a maximum brightness value among pixel average values of four division areas C ₁ , C ₂ , C ₃ , and C ₄ of 7 × 7.

The center coordinate value of the final pixel calculated from the center coordinate calculation unit 150-1 is used as a parameter of position weighted 2D moment modeling including a peripheral area brightness value.

The 2D moment modeling unit 170-1 generates a result of 2D moment modeling from the center coordinates of the final pixel calculated from the center coordinate calculation unit 150-1.

That is, the 2D moment modeling unit 170-1 may generate a position weighted 2D moment modeling result by substituting the center coordinates of the final pixel calculated from the center coordinate calculating unit 150-1 into Equation 2 above. have.

Accordingly, the position weighted 2D moment modeling module 100-1 may generate the first target candidate most similar to the small thermal target through the above method.

According to the second embodiment, the 2D moment modeling module 100 may be implemented as an area extension 2D moment modeling module 100-2 that performs 2D moment modeling using a region extension technique.

6 is an internal block diagram of a region extension 2D moment modeling module according to another embodiment of the present invention.

Referring to FIG. 6, the area extension 2D moment modeling module 100-2 detects a first target candidate most similar to an actual target based on the transmitted infrared image, and obtains a maximum brightness position obtaining unit ( 110-2), the same value expansion unit 130-2, a central coordinate calculation unit 150-2, and a two-dimensional moment modeling unit 170-2.

The area extension two-dimensional moment modeling module 100-2 uses the method of expanding the area to appropriately determine the initial center position of the moment when two or more same maximum brightness values are adjacent to each other. The most similar target candidates can be detected.

The maximum brightness position acquisition unit 110-2 in the area extension 2D moment modeling module 100-2 obtains the pixel maximum brightness position from the obtained infrared image.

The same value expansion unit 130-2 is a predetermined area, for example, 3 × 3, from a position obtained from the maximum brightness position obtaining unit 110-2. The area is set to extend to the same value as the maximum brightness value.

FIG. 7 shows that the same value expansion unit shown in FIG. 6 is 3 × 3. An example of a method of expanding a region in the region is shown.

Referring to FIG. 7, when the center of a pixel is A ₁ pixel, the same value expansion unit 130-2 expands to the A ₂ pixel coordinate including the same brightness value around the A ₁ pixel.

The central coordinate calculator 150-2 sets a threshold value by obtaining the total pixel average of the extended 3 × 3 region, and obtains a pixel average coordinate between the maximum brightness values from the set threshold value.

In addition, the center coordinate calculator 150-2 calculates the center coordinates of the final pixel according to the brightness value coordinates extended to the same value as the maximum brightness value and the pixel average coordinates.

A method of obtaining the center of the pixel coordinates in the extended 3x3 region is shown in Equation 7 below.

At this time, A _i is a coordinate including the maximum brightness value of the pixel, n is the number of expanded pixels.

은 하기 수학식 8과 같이 나타낼 수 있다.At this time, the 3 × 3 Average of pixel brightness values in a region

Can be expressed as Equation 8 below.

In addition, a method of calculating the center coordinates of the final pixel by calculating the average coordinates of the maximum brightness value coordinates and the pixel average coordinates by the center coordinate calculation unit 150-2 is shown in Equation 9 below.

The center coordinate value of the final pixel calculated from the center coordinate calculation unit 150-2 is used as a parameter of area extension two-dimensional moment modeling including a peripheral area brightness value.

The 2D moment modeling unit 170-2 generates an area extension 2D moment modeling result from the center coordinates of the final pixel calculated by the center coordinate calculating unit 150-2.

That is, by substituting the result of Equation 9 into Equation 2, the area-expanded two-dimensional moment modeling result can be generated.

FIG. 8 is a diagram illustrating a result of two-dimensional moment modeling of an infrared image by the two-dimensional moment modeling module illustrated in FIG. 1.

That is, FIG. 8A illustrates brightness values of the infrared image, and FIG. 8B illustrates a position weighted two-dimensional moment modeling module 100-1 according to the first embodiment. (C) shows the result of the two-dimensional moment modeling based on the region extension by the region extension two-dimensional moment modeling module 100-2 according to the second embodiment.

Referring back to FIG. 1, the SIFT target feature value extraction module 200 of the target determination system 10 includes a corresponding area setting unit 230 and an SIFT execution unit 250, and senses the visible light image sensor 50. The received visible light image to extract the SIFT-based target feature value.

The corresponding region setting unit 230 may be a position weighted 2D moment modeling module 100-1 according to the first embodiment or an area according to the second embodiment, for example, from the 2D moment modeling module 100 in the visible light image. A point corresponding to the first target candidate derived from the extended two-dimensional moment modeling module 100-2 is set as an N × N region.

The SIFT execution unit 250 derives a second target candidate which is a candidate point of a target by using a scale invariant feature transform (SIFT) technique for the N × N region set by the corresponding region setting unit 230.

At this time, the SIFT performer 250 acquires the blurred image by applying Gaussian filters having different sizes in steps, calculates by applying a Gaussian difference in the size invariant space, and then calculates the neighboring 8 Pixels and pixels that become the local maximum and local minimum in two adjacent spaces calculate the size-invariant feature point D (x, y, σ) as the second target candidate.

In this case, the Gaussian difference is performed as in Equation 10 below.

은 가우시안 함수이고, σ는 표준편차이며, k는 복수의 공간에 블러링을 적용하기 위한 상수 값이며, I(x,y)는 상기 가시광 영상 센서(50)부터 수신한 가시광 영상이다.remind

Is a Gaussian function, σ is a standard deviation, k is a constant value for applying blurring to a plurality of spaces, and I (x, y) is a visible light image received from the visible light image sensor 50.

9 illustrates a result of extracting feature points and generating descriptors for the extracted feature points by applying a SIFT technique to a visible light image.

Referring to FIG. 9, (a) is a result of extracting the size invariant feature points in the ROI by applying a SIFT technique to the visible light image, and (b) is a result of extracting the size invariant feature points in the ROI shown in (a). This is the result of generating the descriptor.

Referring back to FIG. 1, the Bayesian rule-applied target determination module 300 in the target determination system 10 may determine each candidate point extracted from the infrared image and the visible light image, for example, the first target candidate and the second target candidate. The similarity of feature values is determined based on a Bayesian Rule based probability.

The Bayesian rule is a probability-based reasoning method, and a method of obtaining post-information from prior information and new information is shown in Equation 11 below.

는 z사건이 현재 발생했을 때, x사건이 현재 일어날 확률이며, xi는 사건이 이전에 발생한 확률이다.From here,

Is the probability that x event will occur when z event occurs now, and xi is the probability that event occurred before.

That is, the Bayesian rule application target determination module 300 determines the similarity between the feature value Z1 extracted from the infrared image and the feature value Z2 extracted from the visible light image, based on the Bayesian rule. The probability of the target is calculated and the final target is determined as the priority of the calculated probability.

The Bayesian rule application target determination module 300 uses Equation 12 to determine the final target based on the Bayesian rule.

과

는 새롭게 추출된 집합이고

와

는 이전에 추출된 집합이며, 소문자인

과

는 현재의 측정값이다.Where capital letters

and

Is the newly extracted set

Wow

Is a previously extracted set,

and

Is the current measurement.

According to an embodiment, the weighted target determination module 300 may select the final target when the probability calculated according to Equation 12 is the maximum.

10 shows an example of the result of the target is finally selected according to the embodiment of the present invention.

(A) of FIG. 10 illustrates first target candidates I ₁ , I ₂ , I ₃ , I ₄ , and I ₅ in an infrared image, and (b) illustrates first target candidates I ₁ in a visible light image. , I ₂ , I ₃ , I ₄ , I ₅ ) to extract the SIFT descriptor corresponding to the second target candidates (V ₁ , V ₂ , V ₃ , V ₄ , V ₅ ) is determined to be determined.

In particular (a) it can be seen that I ₃ is selected as the final target among the first target candidates (I ₁ , I ₂ , I ₃ , I ₄ , I ₅ ) according to the priority of the basis-based probability.

As a result, the present invention can use the Bayesian rule-based targeting method in the strap-down dual mode imager 10 to accurately determine the small heat target and the extended target.

11 is a flowchart illustrating a Bayesian rule-based target determination method of the strapdown dual mode image detector according to the first embodiment of the present invention in detail.

1 to 5 and 8 to 11, the infrared sensor 30 and the visible light sensor 50 are positioned at one specific point (x, y) on parallel collinear lines in order to acquire an image of a specific same area. Infrared image and visible light image for each is obtained (S10).

The infrared sensor 30 transmits the sensed infrared image to the position weighted 2D moment modeling module 100-1 (S30), and the visible light sensor 50 transmits the sensed visible image to the SIFT target feature value extraction module 200. (S50).

The maximum brightness position acquisition unit 110-1 in the position weighted 2D moment modeling module 100-1 obtains the pixel maximum brightness position from the obtained infrared image and sends the obtained result to the area setting unit 130-1. Transmit (S100).

The area setting unit 130-1 sets a predetermined area for obtaining an average brightness value of adjacent pixels from the pixel maximum brightness location acquired from the maximum brightness location obtaining unit 110-1, and sets the predetermined area to four areas. Compensated coordinate values are obtained by dividing into regions and calculating pixel averages thereof (S130).

For example, as shown in FIG. 5A, the area setting unit 130-1 is 3 × 3 from a position obtained from the maximum brightness position obtaining unit 110-1. The region is set and divided into four regions A ₁ , A ₂ , A ₃ , and A ₄ to calculate respective pixel averages, and a coordinate value compensated with the largest pixel average is obtained (S130).

Subsequently, as shown in (b) of FIG. 5, the area setting unit 130-1 sets four areas by setting a 5 × 5 area from the position acquired from the maximum brightness position obtaining unit 110-1. Compensated coordinate values are obtained by dividing by (B1, B2, B3, B4) and calculating a pixel average thereof (S140).

Subsequently, as shown in FIG. 5C, the area setting unit 130-1 sets four areas by setting a 7 × 7 area from the position acquired from the maximum brightness position obtaining unit 110-1. By dividing (C ₁ , C ₂ , C ₃ , C ₄ ), each pixel average is calculated, and a coordinate value compensated with the largest pixel average is obtained (S150).

The center coordinate calculator 150-1 uses the maximum coordinate values of the pixel averages obtained from the predetermined areas 3 × 3, 5 × 5, and 7 × 7 by the area setting unit 130-1. In operation S170, the final center coordinate value is calculated according to Equation 9.

The 2D moment modeling unit 170-1 determines the first target candidate by modeling the position weighted 2D moment from the center coordinates of the final pixel calculated by the center coordinate calculating unit 150-1 (S190).

In addition, the 2D moment modeling unit 170-1 transmits the determined first target candidate to the SIFT target feature value extraction module 200 and the Bayesian rule application target determination module 300 (S190).

Meanwhile, the corresponding region setting unit 230 of the SIFT target feature value extraction module 200 may determine a point corresponding to the first target candidate transmitted from the position weighted two-dimensional moment modeling module 100-1 from the visible light sensor 500. The transmission visible light image is set to an N × N region (S330).

The SIFT execution unit 250 derives a second target candidate which is a candidate point of a target by using a scale invariant feature transform (SIFT) technique for the N × N region set by the corresponding region setting unit 230 (S350). Send to the Bayesian rule application target determination module 300.

The SIFT technique is a technique for detecting a size-invariant feature point of a target or a background, and acquires a blurred image by applying Gaussian filters having different sizes in stages, and Gaussian difference in the size-invariant space (Difference of Gaussian, DoG) After application and calculation, the process involves calculating the size-invariant feature points of neighboring pixels and pixels that become the maximum and minimum values in two neighboring spaces.

The Bayesian rule application target determination module 300 calculates the similarity between the feature value extracted from the infrared image and the feature value extracted from the visible light image as a probability based on the Bayesian rule, and the final target as the priority of the calculated probability. Determine (S400).

At this time, the Bayesian rule application target determination module 300 uses Equation 12 to apply the Bayesian rule.

12 is a flowchart illustrating a Bayesian rule-based target determination method of the strapdown dual mode image detector according to the second embodiment of the present invention in detail.

1 to 3, 6 to 10, and 12, the infrared sensor 30 and the visible light sensor 50 have one specific point (x, An infrared image and a visible light image for y) are respectively obtained (S10).

The infrared sensor 30 transmits the sensed infrared image to the region expansion 2D moment modeling module 100-2 (S30), and the visible light sensor 50 transmits the sensed visible image to the SIFT target feature value extraction module 200. (S50).

The maximum brightness position acquisition unit 110-2 in the area extension 2D moment modeling module 100-2 obtains the pixel maximum brightness position from the acquired infrared image and displays the obtained result as the same value extension 130-2. Transmit to (S200).

The same value expansion unit 130-2 is a predetermined area (eg, 3 × 3) from the acquisition result transmitted from the maximum brightness position obtaining unit 110-2. Region) and extends to the same value as the maximum brightness value (S230).

For example, referring to FIG. 7, when the center of the pixel is A ₁ pixel, the pixel coordinates are extended to A ₂ including the same brightness value around the A ₁ pixel.

The central coordinate calculation unit 150-2 sets a threshold value by obtaining the total pixel average of the extended 3 × 3 region, and obtains a pixel average coordinate between the maximum brightness values at the set threshold value (S250).

In operation S270, the central coordinate calculator 150-2 calculates the center coordinates of the final pixel based on the brightness value coordinates extended from the same value extension unit 130-2 and the pixel average coordinates.

The 2D moment modeling unit 170-2 determines the first group target candidate by modeling the region extension 2D moment from the center coordinates of the final pixel calculated from the center coordinate calculating unit 150-2 (S290).

In addition, the 2D moment modeling unit 170-2 transmits the determined first group target candidate to the SIFT target feature value extraction module 200 and the Bayesian rule application target determination module 300 (S290).

Since the SIFT target feature value extraction process and the Bayesian rule application process are the same as those described with reference to FIG. 11, a detailed description thereof will be omitted.

As a result, a Bayesian rule-based target determination method and a target determination system using the same extract the features of an infrared image and a visible light image through location weighting or region extension two-dimensional moment modeling and size-invariant feature transformation. By determining the final target by calculating the similarity of the extracted features as a probability based on the Bayesian rule, there is an effect that the small target and the extended target can be determined very precisely.

Although the present invention has been described with reference to one embodiment shown in the drawings, this is merely exemplary, and those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

10: target determination system
30: IIR
50: CCD
100: two-dimensional moment modeling module
100-1: Position Weighted 2D Moment Modeling Module
100-2: Region Expansion 2D Moment Modeling Module
110-1, 110-2: maximum brightness position acquisition unit
130-1: area setting unit
130-2: equal value extension
150-1, 150-2: center coordinate calculation unit
170-1, 170-2: two-dimensional moment modeling unit
200: SIFT target feature value extraction module 200
300: Bayesian rule application target determination module

Claims (3)

An infrared image sensor and a visible light image sensor for respectively obtaining an infrared image and a visible light image for a specific region;
A 2D moment modeling module for determining a first target candidate through position weighting or area extension based 2D moment modeling from the acquired infrared image;
A point corresponding to the first target candidate determined from the two-dimensional moment modeling module is set as a corresponding N × N region, and a second target candidate using the scale invariant feature transform (SIFT) technique for the set N × N region. SIFT target feature value extraction module for deriving; And
A Bayesian rule applied target determination module for calculating a similarity between the determined first target candidate and the second target candidate as a probability by applying a Bayesian rule, and determining a final target according to the priority of the calculated probability. Bayesian rule-based target determination system of a strap-down dual mode imager including; The method of claim 1, wherein the two-dimensional moment modeling module,
A maximum brightness position obtaining unit obtaining coordinates of a pixel having a maximum brightness value from the acquired infrared image;
A predetermined area including four sub areas having the same pixel unit is set around the obtained pixel maximum brightness position, and an average pixel value of the sub areas is calculated to determine a maximum value coordinate of the pixel average of the predetermined area. An area setting unit to obtain;
A center coordinate calculator configured to calculate a final pixel center coordinate from the acquired maximum value coordinate of the pixel average; And
And a two-dimensional moment modeling unit configured to determine the first target candidate by performing position-weighted two-dimensional moment modeling from the calculated center coordinates of the final pixel. The method of claim 1, wherein the two-dimensional moment modeling module,
A maximum brightness position obtaining unit obtaining coordinates of a pixel having a maximum brightness value from the acquired infrared image;
An equal value expansion unit that sets a predetermined pixel area from pixel coordinates having the obtained maximum brightness value, and extends brightness values of pixels in the set pixel area to the obtained maximum brightness value;
A threshold value is obtained by obtaining an extended total pixel average of the pixel area, a pixel average coordinate between the set threshold value and the maximum brightness value is calculated, and a final pixel according to the extended brightness value coordinate and the pixel average coordinate A center coordinate calculation unit calculating a center coordinate; And
A Bayesian rule-based target determination system of a strap-down dual mode image searcher including a;

Images