JP2013114381A - Object identification device and object identification method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reliably detect and identify a moving object from image data.SOLUTION: Image data obtained by photographing an object is sequentially acquired at a prescribed rate by an image data acquisition unit (100), relative positions of a plurality of pieces of sequentially acquired image data are detected by an inter-pixel position detection unit (102), differences between the plurality of pieces of image data are detected from the relative positions and are extracted as one or a plurality of object candidate regions by a candidate region extraction unit (103), amounts of movement of the candidate regions are detected over the plurality of pieces of image data by a movement amount detection unit (104), and resolutions of the candidate regions are improved on the basis of the amounts of movement by a resolution improving unit (105). Meanwhile, a model pattern of the object is preliminarily stored in a storage unit (107), and the candidate regions having improved resolutions are compared with the stored model pattern to identify the object by an object identification unit (106).

Description

  The present invention relates to an object identification device that is mounted on, for example, a launcher of a guided flying object or the guided flying object itself and used to identify and track an object, and more particularly to a technique for identifying a moving object.

  A method using an imaging device such as an infrared camera is known for a guidance system that guides a guided flying object launched from the ground side toward a flying object (hereinafter referred to as a moving object). In this method, the direction of a moving object is photographed with an imaging device, and image data based on infrared rays or the like acquired by photographing is image-processed to generate an image based on a luminance region distribution. Then, a high brightness area in the image is extracted as a moving object, and image feature values such as maximum brightness value, minimum value, average brightness, and size of each extracted area are calculated, and based on these image feature values. In addition, the region closest to the image pattern prepared in advance is identified as a moving object and guided for tracking.

  Actually, in many cases, objects that are confused with objects such as the surface of the earth or clouds enter the image, and there are many areas that have the same luminance as the moving object in the background of the moving object. In this case, it is difficult to accurately recognize the moving object by distinguishing it from the background using only the image features. Therefore, conventionally, in order to accurately identify a moving object even in such a state, a related area is extracted from each image data obtained continuously, and a feature amount reflecting the movement of each area is extracted, There is a technique for distinguishing a moving object from a background based on this.

  However, many of the existing technologies extract a high luminance area from the infrared image, define the coordinates of the center of gravity of the extracted area as the position of the area, and by moving the position of each area (center of gravity coordinates), It is detected as a moving object.

  However, in the above method, since the shape of the region changes due to a change in contrast or the influence of noise, the coordinates of the center of gravity easily change. For this reason, even when the area is stationary, the position of the center of gravity may change due to a change in contrast or the influence of noise, and an erroneous movement of the area may be detected. Such false detection does not affect much if the target moving object is nearby and the apparent movement is large, but if the target is far away and the apparent movement is small. It will have a serious impact.

  For example, the target detection device disclosed in Patent Document 1 can detect an apparently small object at a distance by matching a template created with a Gaussian distribution shape with an image on infrared image data. Since the original image of the object is small, an image having the same size exists in the same image, which may cause erroneous identification.

JP 2009-69019 A

  As described above, with the existing object identification technology, the position of the center of gravity may change due to the change in contrast or the influence of noise, and the movement of the wrong area may be detected. When the movement of the upper image is very small, there is a serious influence such as the possibility that the same image existing in the same image may be erroneously identified.

  The present embodiment has been made to solve the above problem, and can detect and identify a moving object more reliably from an image without being affected by a change in contrast or noise of the acquired image. An object is to provide an object identification device and an object identification method.

  In order to solve the above-described problem, according to the present embodiment, a plurality of image data obtained by photographing an object by an image data obtaining unit are sequentially obtained at a prescribed rate, and sequentially obtained by an inter-pixel position detecting unit. The relative position between the image data is detected, the difference between the plurality of image data is detected from the relative position by the candidate area extracting means, and one or more of the differences are extracted as candidate areas of the object, and moved. A moving amount of the candidate area is detected over the plurality of image data by a quantity detecting means, and a resolution of the candidate area is improved based on the moving amount by a resolution improving means. On the other hand, the model pattern of the object is stored in advance in the model storage means, and the object identification means identifies the object by comparing the candidate area whose resolution has been improved with the stored model pattern.

FIG. 2 is a block diagram illustrating a configuration of a target object identification device according to the first embodiment. The conceptual diagram for demonstrating operation | movement of the image data acquisition part in the target object identification apparatus shown in FIG. The conceptual diagram for demonstrating operation | movement of the position detection part between images in the target object identification apparatus shown in FIG. The conceptual diagram for demonstrating operation | movement of the candidate area | region extraction part in the target object identification apparatus shown in FIG. The conceptual diagram for demonstrating operation | movement of the movement amount detection part in the target object identification apparatus shown in FIG. The conceptual diagram for demonstrating operation | movement of the movement amount detection part in the target object identification apparatus shown in FIG. The conceptual diagram for demonstrating operation | movement of the resolution improvement part in the target object identification apparatus shown in FIG. The flowchart which shows the flow of a process of the target object identification apparatus shown in FIG. The block diagram which shows the structure of the target object identification apparatus in Embodiment 2. FIG.

  Hereinafter, the object position detection apparatus according to the present embodiment will be described with reference to the drawings.

(Embodiment 1)
First, Embodiment 1 will be described with reference to FIGS.

  FIG. 1 is a block diagram illustrating the configuration and functions of an object identification device according to the first embodiment. This object identification device includes an image data acquisition unit 100. The image data acquisition unit 100 is configured by an infrared imaging camera or the like, and acquires electromagnetic wave image data including a moving object by photographing toward the moving object. The acquired electromagnetic wave image data is extracted from a high luminance area. Sent to the unit 101.

  The high luminance area extraction unit 101 extracts an area exceeding the luminance threshold as the high luminance area with respect to the luminance information of the acquired electromagnetic wave image data, and the information on the extracted high luminance area is an inter-image position detection unit. 102. The inter-image position detection unit 102 detects a relative positional relationship between a plurality of pieces of image data out of the image data composed of the extracted high-luminance region. The detected relative positional relationship is the candidate region extraction unit 103. Sent to.

  The candidate area extraction unit 103 extracts a difference between the image data from the detected relative positional relationship between the image data, and uses the difference as a candidate area of the target object. Here, the image extracted as the candidate area Information on the difference between the data is sent to the movement amount detection unit 104. The movement amount detection unit 104 detects the movement amount of each candidate area that covers a plurality of extracted image data, and information on the movement amount of the detected candidate area is sent to the resolution improvement unit 105.

  The resolution improving unit 105 improves the resolution of the candidate area by superimposing the candidate areas over a plurality of image data from the information on the detected movement amount of the candidate area. The information is sent to the object identification unit 106. The object identifying unit 106 reads out the model pattern from the model storage unit 107 in which the model pattern of the moving object is stored in advance, and identifies whether or not the candidate area whose resolution has been increased is the object based on the model pattern. .

  Hereinafter, the operation of the object identification device of Embodiment 1 will be described in detail.

  The object identification device according to the first embodiment extracts and identifies an object from electromagnetic wave image data. First, the electromagnetic wave image data 300 is acquired by the image data acquisition unit 100. For example, an infrared camera is used for the image data acquisition unit 100. In this case, the electromagnetic wave image data 300 is an infrared image. The image data acquired by the image data acquisition unit 100 is output to the high luminance region extraction unit 101 at a constant rate.

  The high brightness area extraction unit 101 extracts pixels having brightness higher than a threshold value set from the input image data, and excludes information of other pixels. The object to be identified is assumed to be imaged with relatively high luminance in the image data. By excluding pixels with low luminance values at this stage, subsequent processing can be reduced. It is also possible to skip this process and go to the next process.

  The inter-image position detection unit 102 calculates the relative positional relationship between the image data from the plurality of image data obtained by the image data acquisition unit 100 or the image data from which the low luminance region is excluded by the high luminance region extraction unit 101. To do. For example, when the present apparatus is mounted on a guided flying object, the position of the background differs between image data because the shooting position changes from moment to moment.

  FIG. 2 shows how the background moves as the object identification device moves. As shown in FIG. 2, the position of the background differs between the images of the continuous image data 300a and 300b. In this case, since the background moves in the image, it is difficult to detect the moving object in the image. Therefore, the relative position of the background between the image data is calculated by the inter-image position detection unit 102.

  FIG. 3 shows a state of processing performed by the inter-image position detection unit 102. The inter-image position detection unit 102 sets a comparison area 400 for comparing the image of the image data 300a with the image of the image data 300b. A comparison scan 401 is performed while shifting the comparison area 400 pixel by pixel with respect to the image of the image data 300b, and a matching process for calculating a luminance correlation value at each position is performed. Then, an evaluation image in which the correlation values are arranged is created.

  Next, the inter-image position detection unit 102 searches the evaluation image for the position having the highest correlation value, obtains the maximum evaluation position, and calculates the relative position between the image data based on the position. Thereby, the relative position in pixel units between two pieces of image data can be detected. Furthermore, by using a quadratic curved surface, it is possible to detect a relative position in a minute subpixel order of one pixel or less.

  Assuming that the relative value in the evaluation image changes along the quadratic curved surface, three coordinate arrays of one pixel left and right and three coordinate arrays of one pixel above and below are centered on the maximum evaluation position obtained at the pixel level. The vertex position of the quadric surface can be obtained by solving the quadratic polynomial. The resolution is 1 pixel or less. For example, assuming that the coordinates of the maximum evaluation position obtained at the pixel level are (w, s), three points (w−1, s), (w, s), (w + 1, s) centered on the coordinates, By solving the quadratic polynomial from each of the three points (w, s-1), (w, s), and (w, s + 1), the vertex position of the quadric surface can be obtained with a resolution of one pixel or less. Therefore, the accuracy of detecting the relative position can be increased.

  The candidate area extraction unit 103 superimposes based on the relative position obtained by the inter-image position detection unit 102, and extracts an area remaining as a difference as an area where the object may exist and a candidate area. FIG. 4A shows an example of a result obtained by calculating a difference between the image data 300a and the image data 300b in FIG. 3, and corresponds to the difference image data 500a and 500b, respectively. Here, the candidate areas are 501a and 501b.

  However, in practice, there are a plurality of regions remaining as differences as shown in FIG. 4B (the shape of each candidate region on the diagram is schematically illustrated for easy explanation and is different from the actual one). In many cases, it is necessary to extract a true object from these remaining areas. The present embodiment solves this, and the remaining areas are identified in the subsequent processing units.

In the movement amount detection unit 104, first, a differential image indicating the luminance difference and the direction of the luminance difference between each pixel in the candidate area in the image data and its neighboring pixels is generated. In calculating the differential image, the following equation (1) is used. In equation (1), capital letter I is a luminance value for each pixel, and i and j are indexes for distinguishing the pixels.

  This value is calculated for each pixel. This value is referred to as information on the luminance difference direction, and is more generally represented by (Rx, Ry).

  Next, the movement amount detection unit 104 calculates the position of each region extracted by the candidate region extraction unit 103 as follows. FIG. 5 is a diagram illustrating an example of a template used for the process of calculating the position of the extracted region. For each pixel in the region extracted by the candidate region extraction unit 103, the movement amount detection unit 104 converts the luminance into a similar value assuming a bell-shaped two-dimensional luminance distribution such as a Gaussian distribution. Then, a similar value image obtained as a set of similar values is generated, and the position of each candidate area is calculated based on the similar value image.

  The Gaussian distribution has a vertex at the center and a value that inclines from the center toward the periphery. As shown in FIG. 5, the luminance difference direction is distributed so as to be directed from the periphery toward the apex. The arrows shown in FIG. 5 indicate the direction of luminance difference when it is assumed that “the luminance value is distributed in a Gaussian distribution around the object as a vertex” in the two-dimensional image. In FIG. 5, it is assumed that the center pixel has the maximum pixel value in a 5 × 5 pixel region.

The movement amount detection unit 104 sequentially scans the pixels of the differential image corresponding to the pixels extracted by the candidate area extraction unit 103 using the luminance difference direction of the Gaussian distribution shown in FIG. ) To calculate the similarity value. A similar value image is obtained by arranging the similar values.

  In Equation (2), K is a similar value, and θ is an angle formed by the luminance value direction and the template. In the lowermost expression of Expression (2), the difference between the luminance difference direction (Rθ) of the differential image and the luminance difference direction (Tθ) of the template shown in FIG. 6, that is, Dθ (= | Rθ−Tθ |) is 180 degrees. The value normalized by is subtracted from 1. Accordingly, the similarity value becomes 1 when the luminance difference directions coincide with each other, and the similarity value becomes 0 if the opposite directions are opposite. The contribution rate (α) is taken into consideration for the value. That is, the similarity value K is calculated by creating a coefficient corresponding to the matching rate in the luminance difference direction and accumulating the luminance differences based on the coefficient. By evaluating the similarity value K, the distribution in the luminance difference direction in the image frame can be matched with the Gaussian distribution, and the position of each candidate region can be calculated.

  A determination condition may be provided for the data to be added. Specifically, it is utilized that the luminance difference direction of each pixel of the Gaussian distribution is point symmetric. As can be seen from FIG. 5, the luminance difference direction of the AF pixel (upper left) is opposite to the luminance difference direction of the EJ pixel. Further, the luminance difference direction of the EF and AJ pixels is orthogonal to the luminance difference direction of the AF and EJ pixels, respectively. By using this fact, for example, if the coincidence ratios of the four pixels of AF, EF, AJ, and EJ in the luminance difference direction are each within 90 degrees, the determination condition is to add the luminance difference.

  Since the value addition is determined for each pixel in Expression (2), the similarity value may increase even when the coincidence rate is partially high. However, by providing the above determination condition, the circumferential direction In consideration of the above, it is possible to calculate the similarity value as a high value when the circumferential matching rate is high. That is, a high score can be given to the luminance distribution that rises from the surroundings toward the apex, and the position of the object can be captured more accurately.

  Further, the movement amount detection unit 104 searches for the value of the similarity value for each candidate region extracted by the candidate region extraction unit 103, and defines the coordinate (w, s) having the highest value as the position of each candidate region. As a result, when the luminance distribution of the object is assumed to be a Gaussian distribution, a position that is most likely to be the position of the object can be defined at the pixel level. At this stage, as shown in FIG. 4C, the maximum similarity value in each candidate region is excluded when it falls below a threshold arbitrarily set by the user, and only the regions 501a and 501b exceeding the threshold are extracted as candidate regions. It is also possible to do.

  Further, the movement amount detection unit 104 can detect the position at the sub-pixel as in the inter-image position detection unit 101. That is, it is assumed that the similarity value in the similarity value image changes along the quadric surface. Therefore, for example, three points (w-1, s), (w, s), (w + 1, s) centered on (w, s) obtained at the pixel level, and (w, s-1), ( By solving the quadratic polynomial using three points (w, s) and (w, s + 1), the vertex position of the quadric surface can be obtained with an accuracy of one pixel or less. Therefore, the accuracy of position detection can be further increased.

  Through the above processing, the position of each candidate area in each image data is extracted, and the amount of movement between the image data of each candidate area is sub-pixel united by taking into account the positional information and the relative positional relationship between the image data. Calculated.

  In the resolution improving unit 105, the resolution is improved by superimposing the images of candidate areas on a plurality of image data. If the images can be superimposed with a unit smaller than the pixels of the original image data and a resolution of sub-pixels, pixels smaller than the original image can be generated. FIG. 7 illustrates a state in which an image having a double resolution is generated by superimposing two images. When superposed, there are two values on the pixel 502 of the high resolution image. For example, as a method for uniquely determining the value of the pixel 502 of the high resolution image, a method of taking an average of them is the simplest.

  In the movement amount detection unit 104, since the movement amount of each candidate region is calculated in units of sub-pixels, the resolution can be increased by superimposing the movement amounts based on the above-described principle. In the above, an example in which two images are superimposed is shown, but it is possible to further increase the resolution of the image of the candidate region by overlaying not only two images but also more images.

  The object identifying unit 106 identifies whether or not the candidate area indicates the object by using the image of the candidate area whose resolution has been improved by the resolution improving unit 105. In this identification process, the model pattern of the identification / tracking target is stored in advance in the model storage unit 107, and a model having a high similarity to the stored model pattern is identified as the target. In the model storage unit 107, the model pattern of the object can be stored as electromagnetic wave image data, and can be stored parametrically as a bell-shaped distribution such as a Gaussian distribution. As a method of calculating the similarity, the luminance information may be matched, or the similarity may be calculated based on the distribution shape as described above.

  FIG. 8 is a flowchart illustrating a processing flow of the object identification device according to the first embodiment. That is, in this object identification device, when infrared image data including a moving object is acquired (step S1), an area exceeding the luminance threshold is extracted as a high luminance area with respect to the luminance information of the acquired infrared image data ( Step S2) calculates a relative positional relationship between a plurality of pieces of image data out of the image data composed of the extracted high-intensity region (Step S3), and calculates the image data based on the relative positional relationship between the detected image data. The difference is extracted as a candidate area for the object (step S4). Subsequently, the movement amount of the candidate area is detected in units of subpixels from the information on the difference between the image data extracted as the candidate area (step S5), and a plurality of pieces of image data are covered from the information on the movement amount of the detected candidate area. The resolution of the candidate area is improved by superimposing the candidate areas (step S6), the candidate area that has been increased in resolution is compared with the model pattern of the moving object, and the candidate area that has been increased in resolution from the comparison result Is identified as an object (step S7).

  As described above, in the object identification device according to the first embodiment, it is assumed that the luminance distribution around the target is distributed in a bell shape like a Gaussian distribution, and the bell shape is also used when specifying the target position from the candidate region. By matching with the distribution and defining the coordinates of the vertices of the Gaussian curve as the target position, the position detection at the sub-pixel is performed. Furthermore, by using the amount of movement in sub-pixel units, it is possible to improve the resolution of the image data, and it is possible to improve the discrimination performance to a level that allows discrimination even when the size of the object is small. .

(Embodiment 2)
The second embodiment will be described with reference to FIG.

  FIG. 9 is a block diagram illustrating a configuration of the object identification device according to the second embodiment. However, in FIG. 9, the same parts as those in FIG. 1 are denoted by the same reference numerals, and different parts will be mainly described here.

  This object identification device has a configuration in which an acquired altitude information acquisition unit 108 and a posture information acquisition unit 109 are added to the configuration of the object identification device in the first embodiment. The altitude information acquisition unit 108 acquires altitude information where the target object identification device exists. For example, the altitude information acquisition unit 108 acquires position information from the GPS, or the altitude information by a change in atmospheric pressure by a pressure gauge. Can be considered. The posture information acquisition unit 109 acquires the direction / posture information that the image data acquisition unit 100 is directed to, for example, a gyro sensor.

  The operation of the above configuration will be described below.

  First, image data is acquired by the image data acquisition unit 100 as in the first embodiment. At the same time, the altitude information acquisition unit 108 acquires the altitude at which the target object identification device exists, and the posture information acquisition unit 109 acquires the image. The direction / attitude information to which the data acquisition unit 100 is directed is acquired. These altitude and attitude information are input to the candidate area extraction unit 103, and candidate areas that cannot exist in the flight route can be excluded. For example, when an object flies over the water surface, the image may be reflected on the water surface and extracted as a candidate region. However, the candidate area observed on the ground surface can be excluded by the function of the exclusion function of the candidate area extraction unit 103, and erroneous identification in the identification processing in the object identification unit 106 at the subsequent stage can be prevented.

  According to the object identification device of the second embodiment, even when a reflection image of the object is present in the image data, it can be excluded and erroneous identification can be prevented.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  DESCRIPTION OF SYMBOLS 100 ... Image data acquisition part, 101 ... High brightness area extraction part, 102 ... Inter-image position detection part, 103 ... Candidate area extraction part, 104 ... Movement amount detection part, 105 ... Resolution improvement part, 106 ... Object identification part, 107: Model storage unit, 108: Altitude information acquisition unit, 109: Attitude information acquisition unit.

Claims (8)

Image data acquisition means for sequentially acquiring image data obtained by photographing an object at a prescribed rate;
An inter-image position detecting means for detecting a relative position between a plurality of image data sequentially acquired by the image data acquiring means;
Candidate area extraction means for detecting a difference between the plurality of image data from the relative position and extracting one or a plurality of the differences as candidate areas of the object;
A movement amount detecting means for detecting a movement amount of the candidate region over the plurality of image data;
Resolution improving means for improving the resolution of the candidate area based on the amount of movement;
Storage means for storing a model pattern of the object in advance;
An object identification apparatus comprising: identification means for comparing the candidate area with improved resolution and the model pattern to identify an object.   For each pixel forming the candidate area, the movement amount detection unit calculates a luminance difference direction from a difference between a luminance value of the pixel and a luminance value of a pixel surrounding the pixel, and calculates from the distribution of the luminance difference direction. 2. The object according to claim 1, further comprising: a function of calculating a coordinate of a maximum point of a luminance value to be a position of a candidate area and calculating a difference in position of the candidate area across the plurality of image data as a movement amount. Identification device.   The movement amount detection means uses the coordinates corresponding to the vertices of the distribution function in a region where the degree of the distribution in the luminance difference direction matches a specified distribution function exceeds a specified threshold as the position of a candidate region, The object identification device according to claim 1, further comprising a function of calculating a difference in position of the candidate area over image data as a movement amount.   4. The object identification device according to claim 3, wherein the movement amount detection means uses a bell-shaped function as the distribution function.   The object storage device according to claim 1, wherein the storage unit stores the model pattern as a bell-shaped function.   The image data further comprises a high luminance area extracting means for extracting only a high luminance area exceeding a luminance threshold with respect to the luminance information of the image data, and the inter-image position detecting means determines a relative position of the high luminance area over the plurality of image data. 6. The object identification device according to claim 1, wherein the object identification device is calculated as a relative position between the image data.   Further, the apparatus includes an altitude information acquisition unit that acquires altitude information and an attitude information acquisition unit that acquires attitude information, and the candidate area extraction unit narrows down the plurality of candidate areas based on the altitude information and the attitude information. The object identification device according to claim 1, wherein: The image data obtained by photographing the object is sequentially acquired at the specified rate,
Detect the relative position between multiple image data acquired sequentially,
Detecting a difference between the plurality of image data from the relative position and extracting the difference as one or a plurality of candidate areas of the object;
Detecting the amount of movement of the candidate region over the plurality of image data;
Improve the resolution of the candidate area based on the amount of movement;
A method for identifying an object, comprising: comparing a model pattern of the object stored in advance, a candidate area with improved resolution, and the model pattern;

Images