KR100649384B1 - Method for extracting image characteristic in low intensity of illumination environment - Google Patents

Abstract

A method for extracting the characteristic of an image in the low intensity of illumination environment is provided to separately remove the noise mixed at the relatively darker area than the entire dark area. A CCD(Charge Coupled Device) camera(101) as an image sensor photographs an external image and outputs a boosted analog video signal. The CCD camera is surrounded by an IR(Infrared) module(100) arranged with a circular shape. A frame graver(102) is activated by a control signal inputted by a microcomputer(104), digitizes the analog video signal photographed by the CCD camera and IR module and outputs the digital video signal to a memory(103) by frames. The memory is connected to the frame graver and stores the video information outputted from the frame graver.

Description

Method for extracting image characteristic in low intensity of illumination environment

1 is a diagram showing an image feature extraction apparatus to which the present invention is applied.

2 is a view for explaining an aspect for obtaining a difference image according to the present invention;

3 is a view for explaining an aspect for obtaining a feature point according to the present invention;

4 is a view for explaining an aspect for obtaining a change amount and a change direction of each feature point according to the present invention;

5 is a view for explaining an aspect for obtaining a feature vector according to the present invention.

6 is a diagram illustrating a method for extracting image features in a low light environment according to the present invention.

Explanation of symbols on the main parts of the drawings

100: IR LED module 101: CCD camera

102: frame grabber 103: memory

104: microcomputer

The present invention removes noise mixed in a relatively darker area in an image that is dark in an image taken in a low light environment, and thus an image feature extraction method in a low light environment to easily extract a feature vector in a low light environment. It is about.

Typically, a low-light image boosted by an IR LED (Infrared Luminescent Diode) and a color CCD camera in a low-light environment of less than 1 (lux), such as night, is used for Poisson Counting Noise (PCN) or FCN. (False Color Noise), etc. are mixed. In particular, a relatively darker portion of the low-light image contains a greater amount of Poisson Counting Noise (PCN) or False Color Noise (FCN).

On the other hand, the general image feature extraction method is a sample of images received from the outside through a color CCD camera and IR LED, filtered, and then quantized and blurred using a Gaussian mask. The feature vector of the image is extracted using the distance between each corresponding pixel of adjacent sample images and the amount of affine variation.

However, such a conventional image feature extraction method has a large amount of computation and is not easy to implement an algorithm in real time, and, as described above, a considerable amount of PCN (which is incorporated in a region that is particularly dark in low light image, as described above). Poisson Counting Noise (FSI) or False Color Noise (FCN) acts as a significant outlier in quantization using Gaussian masks, making it difficult to extract feature vectors from images.

The present invention was developed to solve the above-described problems, so as to separately remove the noise mixed in a relatively darker area in the image that is dark overall by shooting in a low light environment, and also to easily extract the feature vector Its purpose is to provide a method for extracting image features in low light environments.

Image feature extraction method in a low light environment of the present invention according to this object is

Shooting in low light and dividing the relatively darker areas of the overall dark image, especially the noises, and dividing the noises in the area according to their properties, filtering the amount of change in brightness and brightness for each feature point of the image. A feature vector is extracted by quantizing the change direction into a plurality of reference direction vectors having a setting range.

The preferred structure comprises the steps of: dividing the relatively bright and dark areas in the photographed low-light image, obtaining a sample image by dividing and filtering noises in the dark area according to each property among the divided areas, and the difference between adjacent sample images. Obtaining an image, and quantizing the amount of change in brightness and the direction of change in brightness of each feature point in the difference image into a plurality of reference direction vectors having a set range, and calculating a feature vector of the image.

A change amount of brightness and a change direction of brightness of each feature point in the difference image are calculated according to Equation 1 below.

는 밝기의 변화량(Gradient)이고,

는 밝기의 변화방향(orientationn)이며,

는 픽셀들의 위치좌표를 나타낸다.here,

Is the gradient of brightness,

Is the orientation of brightness,

Represents the position coordinate of the pixels.

The dividing of the light and dark areas of the image taken at the low light level comprises comparing the gray level value of each pixel and the preset gray level value in the photographed low light image, and relatively light and dark areas according to the comparison result. And dividing the region.

The dividing and filtering of the noises according to respective attributes may include, for example, determining whether or not the image moves in the dark region, and dividing and filtering the noises according to the attributes according to the determination result of the movement to generate a sample image. Characterized in that consisting of steps.

The presence or absence of the movement may be determined according to the degree of gray level change of the images in the adjacent dark area.

The generating of the sample image may include detecting different noises according to a determination result of the presence or absence of the motion, and generating a sample image by filtering noise corresponding to a filter suitable for the detected noise. It is done.

The method may further include determining whether the noise is located at the center of the image, and generating the sample image may generate a sample image by filtering noise corresponding to a different filter according to the determination result. It is done.

The different noise may be FCN noise and PCN noise.

The filter for filtering the different noises may use any one or more selected from a temporal filter, a median filter, and a lowpass filter.

인 것을 사용할 수 있다.The generating of the difference image is characterized by applying the same Gaussian mask to a sample image. In particular, the Gaussian mask has a covariance.

Can be used.

Hereinafter, the present invention will be described with reference to the accompanying drawings.

The image feature extracting apparatus shown in FIG. 1 is an example of an image feature extracting apparatus to which the present invention is applied. The image feature extracting apparatus may be, for example, an IR LED (Infrared Luminescent Diode) module 100, a CCD camera 101, a frame grabber 102, and a memory. 103, the microcomputer 104.

The present invention includes an IR LED module 100 together with a CCD camera 101 which is an image sensor for outputting a boosted analog video signal by taking an external image in order to capture an image according to a low light environment, The IR LED module 100 is disposed in a circular shape surrounding the CCD camera 101 to have a radially wide pattern.

The frame grabber 102 is connected to the CDD camera 101, the IR LED module 100, the memory 103, and the microcomputer 104, respectively, and is activated according to a control signal input from the microcomputer 104. The digital camera digitizes the analog image signal captured by the CCD camera 101 and the IR LED module 100 and outputs the image to the memory by dividing it by frame. The division of the frame may be performed by applying prime information to image information corresponding to the beginning and the end, and other conventional methods may be applied.

The memory 103 is connected to the frame grabber 102 to store image information output from the frame grabber 102 in a designated storage space. The storage space is designated by the microcomputer 104, and this information is received by the frame grabber 102 and stored in the corresponding storage space frame by frame. The memory 103 can also be used as a frame memory.

The microcomputer 104 controls the frame grabber 102 according to a user command input from the outside, and processes the image information stored in the memory 103 to extract a feature of the image according to the present invention and output it to an image processor. will be. A low light filter module and a pseudo scale invariant feature transform (PSIFT) module are provided therein, and the low light filter module may include a median filter, a low pass filter, and a temporal filter.

The microcomputer 104 activates the frame grabber 102 to classify the image stored in the memory 103 by dividing the frame grabber 102 into frames according to a preset reference gray value. Split into areas.

That is, a gray level value of each pixel in the image stored in the memory is compared with a preset gray level value, and as a result of the comparison, an area in which the gray level value of each pixel in the image stored in the memory is smaller than the preset gray level value is divided into dark areas. .

The low light filter module generates a sample image by dividing and filtering the noises mixed in the divided dark areas according to each property.

For example, the presence or absence of motion of an image belonging to a dark region is determined, and a sample image is generated by dividing and filtering noises according to each property according to the determination result of the movement. The presence or absence of the motion is determined according to the degree of gradation change of adjacent images, which is a general technique, and a detailed description thereof will be omitted.

Among the low light filter modules, the median filter filters PCN noise detected when there is motion, and in particular, PCN noise located at the center in a corresponding video frame.

The low pass filter filters the PCN noise detected when there is no motion, in particular, the PCN noise located at the center in the corresponding video frame.

The temporal filter filters the FCN noise detected when there is motion as a result of the determination of the presence or absence of the motion, and the filtering operation of each filter is a known technique, and a description thereof will be omitted herein.

The PSIFT module continuously receives the generated sample image, generates a difference image between two adjacent images, sets each feature point in the generated difference image, and changes the brightness variation vector and the brightness of each feature point. The feature vector is calculated by approximating the direction vector to a plurality of predetermined reference vectors. This is clearly different from the conventional method of calculating a feature vector using the distance between each corresponding pixel of adjacent sample images and the amount of affine among the sample images blurred using different Gaussian masks. The calculation operation will be described later immediately.

2 to 5 are provided to explain in more detail the operation of calculating the feature vector by the PSIFT module.

2 illustrates an operation of continuously receiving the sample image generated by the filtering and generating a difference image between two adjacent images.

인 것이 바람직하다.The operation is performed by continuously obtaining blurred sample images by applying a Gaussian mask of the same size to the input image as shown on the left side, and then adjacent sample images from the left sample images generated as described above. By subtracting the gray scale value of the two gray scale values, that is, generating a difference image composed of the difference values of the gray scale values of adjacent sample images. The Gaussian mask has a covariance

Is preferably.

3 shows a task of setting a feature point that is a feature in the difference image generated by the above task.

The operation is, for example, pixels belonging to the same position as the pixels in the difference image 301 and 303 that are different from the eight pixel values adjacent to the pixels i and j of the difference image 302, (I-1, j), (i + 1, j), (i-1, j + 1), (i, j + 1), (i + 1, j + 1), (i-1, The gray level values for the pixels at positions j-1), (i, j-1), and (i + 1, j-1) are compared with each other, and set as feature points if they are larger or smaller.

FIG. 4 is a task of obtaining a change amount vector of brightness and a change direction vector of brightness with respect to a feature point set by the work shown in FIG. 3.

The operation sets an 8 × 8 window around the position of the feature point to give a value to a pixel corresponding to the set feature point, and the gradient of 64 pixels in the window, that is, the feature point, is expressed by Equation 2 below. The change amount vector of the brightness and the orientation, i.e., the change direction vector of the brightness are obtained.

는 그래디언트(Gradient)이고,

는 오리엔테이션(Orientation)이며,

는 픽셀들의 위치좌표를 나타낸다. here,

Is a gradient,

Is an orientation,

Represents the position coordinate of the pixels.

FIG. 5 illustrates a plurality of reference direction vectors (eg, 10, 3, and 12) having a setting range (eg, A and B) for changing the amount of change in brightness and the direction of changing the brightness for each feature point obtained by the work illustrated in FIG. 4. , ...) to extract feature vectors.

For example, as shown in the drawing, when a plurality of reference direction vectors 10, 3, 12, ... are set to eight reference vectors indicating different directions, first, the amount of change in brightness for each feature point and Each change direction of brightness is sorted and sorted to determine the maximum value. Set this orientation value to canonical orientation.

에서 캐노니컬(canonical)오리엔테이션 값을 감산시켜 64개의 픽셀들에 대해 새로운 오리엔테이션

를 구한다. Then, the orientation obtained for the 64 pixels around the pixel set as the feature point

New orientation for 64 pixels by subtracting the canonical orientation value in

Obtain

와 각 픽셀들에 대한 그래디언트값을 이용하여 벡터(Scaled and Quantized orientation vector)를 얻고, 64개의 픽셀들에 대해 8개 방향의 오리엔테이션에 대한 스케일링 값의 합을 구한 후, 각 8개 방향에 대한 스케일링 값의 합을 특징 벡터로 설정한다. Next, the orientation value for each pixel is quantized into eight orientation values using the eight reference direction vectors. Then, quantized for 64 pixels

Using the gradient values for each pixel, we obtain a scaled and quantized orientation vector, sum the scaling values for eight orientations for 64 pixels, and then scale for each of the eight orientations. Set the sum of the values to a feature vector.

This process is repeatedly performed for each feature point, and the dimension of the finally obtained feature vector has 8-dimensions per feature point. As a result, the same image can be found by comparing a set of feature vectors corresponding to the number of feature points in the input image, and a description thereof is conventional and will be omitted herein.

6 is a diagram illustrating a method for extracting image features in a low light environment according to the present invention.

First, the method for extracting image features in a low light environment according to the present invention receives a low light image from the outside through the CCD camera and the IR LED module of the robot according to step 601. The IR LED module is used to solve this problem because accurate coordinate extraction is difficult due to the influence of external lighting, etc., and is arranged in a circular shape surrounding the CCD camera to have a radially wide pattern.

The low light image of the analog input in step S601 is digitized through a frame grabber or the like and is divided into frames and stored in a memory. The classification of each frame is performed by assigning prime information to digital image information corresponding to the beginning and the end, and other conventional methods may be applied.

In operation S602, the low illumination image inputted according to the operation S601 is divided into a dark area and a bright area. For example, the preset reference gray value and the gray value of each pixel constituting the low light image are divided. In comparison, a low-light image stored in the memory is obtained by making an area composed of pixels having a gradation value equal to or greater than a preset gradation value a bright area, and conversely, a region consisting of pixels having a gradation value equal to or greater than a preset gradation value as a dark area. Split into dark and light areas.

Next, when a dark region is divided among the low light images input in operation S602, noises mixed in the divided dark region are detected according to each property.

For example, the presence or absence of motion of the image in the dark region is determined (S603), and the sample is generated by dividing and filtering noises according to each property according to the result of the determination. It can be determined according to the degree, which is general and the detailed description thereof is omitted here.

In step S604, when it is determined that there is motion according to step S603, if it is determined that there is motion, the detection of PCN noise mixed in the image of the dark area is carried out, and whether such PCN noise is located at the center of the low light image. Or differently detected depending on whether the edge is located, the center PCN noise is detected by the median filter and the PCN noise located at the edge using a low pass filter.

In step S605, if it is determined that there is no motion according to step S603, if it is determined that there is no motion, the FCN noise is detected. The above-described PCN or FCN detection method is conventional and a detailed description thereof is omitted here. do.

Next, when noises belonging to the dark region are filtered according to each property according to steps S604 and S605, a desired sample image is generated according to the present invention.

Subsequent work is to use these sample images to produce feature vectors. That is, the difference image between two adjacent images is generated by successively receiving the generated sample image, and the brightness change vector and the change direction vector of brightness for each feature point in the generated difference image are approximated to a plurality of predetermined reference vectors. To produce feature vectors

인 가우시안 마스크(Gaussian Mask)를 사용하여 블러링(Blurring)된 표본 영상들을 연속적으로 얻는다. To this end, first, Gaussian filtering the sample image according to step S606. For example, a Gaussian mask of the same size may be applied to a sample image generated by the filtering at step S604 and step S605.

Blured sample images are successively obtained by using an Gaussian mask.

In operation S607, a difference image is generated by using a Gaussian filtered sample image continuously obtained in step S606. Among the Gaussian filtered sample images obtained continuously in step S607, the grayscale values of adjacent sample images. By subtracting the result, a difference image consisting of difference values of gray levels of adjacent sample images is generated accordingly.

In step S608, a feature point is set in the difference image frame generated in step S607. For example, with respect to pixels (i, j) of the difference image, eight adjacent pixel values in the same difference image are set. Pixels belonging to the same position as the pixels in another adjacent difference image, that is, (i-1, j), (i + 1, j), (i-1, j + 1), (i, j + 1 ), (i + 1, j + 1), (i-1, j-1), (i, j-1), and (i + 1, j-1) to compare gradation values for pixels If it is big or small, set it as a feature point.

In operation S609, a change amount vector of brightness and a change direction vector of brightness of each feature point set in step S608 are calculated, and a predetermined feature value is assigned to a pixel corresponding to each feature point set in step S608. After setting the 8 × 8 window centering on the position of the corresponding feature point, the gradient of 64 pixels in the predetermined window, that is, the amount of change in brightness and the orientation of the feature point according to Equation 3 below. ), That is, the direction of change in brightness.

는 밝기의 변화량인 그래디언트(Gradient)이고, 는 밝기의 변화방향인 오리엔테이션(Orientation)이며,

는 픽셀들의 위치좌표를 나타낸다.here,

Is the gradient, the change in brightness, Is orientation, the direction of change of brightness,

Represents the position coordinate of the pixels.

In operation S610, a feature vector is calculated using a change vector of brightness of each feature point, a change direction vector of brightness, and a plurality of reference vectors. For example, a plurality of reference vectors may use different directions. In the case of eight reference vectors to indicate, the maximum value is determined by first sorting and sorting the amount of change in brightness and the direction of change in brightness for each feature point. Then, this orientation value is set to canonical orientation.

에서 캐노니컬(canonical)오리엔테이션 값을 감산시켜 64개의 픽셀 들에 대해 새로운 오리엔테이션

를 구한다. Then, the orientation obtained for the 64 pixels around the pixel set as the feature point

New orientation for 64 pixels by subtracting the canonical orientation value from

Obtain

와 각 픽셀들에 대한 그래디언트값을 이용하여 벡터(Scaled and Quantized orientation vector)를 얻고, 64개의 픽셀들에 대해 8개 방향의 오리엔테이션에 대해 스케일링된 값의 합을 구한 후, 각 8개 방향에 대한 스케일링 값의 합을 특징 벡터로 설정한다. Subsequently, the orientation value for each pixel is quantized to eight orientation values using the eight reference direction vectors. Then, quantized for 64 pixels

And a scaled and quantized orientation vector are obtained using the gradient values for each pixel and the sum of the scaled values for the eight directions of orientation for 64 pixels, and then for each of the eight directions. Set the sum of scaling values as a feature vector.

This process is repeatedly performed for each feature point, and the dimension of the finally obtained feature vector has 8-dimensions per feature point. As a result, the same image can be found by comparing a set of feature vectors corresponding to the number of feature points in the input image, and a description thereof is conventional and will be omitted herein.

As described in detail above, the method for extracting image features in a low light environment according to the present invention removes noise that is mixed in a relatively darker area among images taken at low light and has a serious effect on image feature vector extraction, and has the same size. The difference image is obtained by using a Gaussian mask, and the feature vector in the difference image is calculated using the amount of change in brightness and the direction of change in brightness, so that the feature vector can be easily extracted from an image captured in a low light environment.

Although the invention has been described in detail only with respect to the specific examples described, it will be apparent to those skilled in the art that various modifications and variations are possible within the spirit of the invention, and such modifications and variations belong to the appended claims.

Claims (5)

A dividing step of dividing a relatively bright area and a dark area in the captured low light image; A sample image generation step of generating a sample image by dividing and filtering noises in a dark region according to each attribute among the divided regions; A difference image generation step of sequentially receiving the generated sample image and generating a difference image between two adjacent images; Image characteristics in a low light environment having a feature vector extraction step of extracting a feature vector by quantizing the amount of change of brightness and the direction of change of brightness of each feature point in the generated difference image into a plurality of reference direction vectors having a setting range. Extraction method. The method of claim 1, The dividing step Comparing a gray level value of each pixel in the photographed low light image with a preset gray level value; And dividing the relatively bright and dark areas according to the comparison result. The method of claim 1, The sample image generation step Determining the presence or absence of movement in the dark area; As a result of the determination, if there is motion, a sample image is generated by filtering PCN (Poisson Counting Noise), and if there is no motion, a sample image is generated by filtering FCN (False Color Noise). Image Feature Extraction Method in Low Light Environment. The method of claim 1, The difference image generating step Continuously generating the blurred sample images by applying the same Gaussian mask to the filtered sample image; And generating a difference image by using the blurred sample images. The method of claim 1, The change amount of brightness and the change direction of the brightness for each feature point are calculated according to Equation 4 below.

(here,

Is the gradient of brightness,

Is the orientation of brightness,

Is the positional coordinate of the pixels.)

Images