JP5325822B2 - Object identification device - Google Patents

Description

  The present invention relates to an object identification device.

  A conventional object identification device uses a subwavelength image data in a near-infrared wavelength region and image data in a two-wavelength region of image data in a far-infrared wavelength region, and uses a difference in infrared characteristics between the object and a flare. There is known a technique for identifying an object by using (see, for example, Patent Document 1). Further, there is known a technique for extracting main body information and flare image information of an object using image data in the near infrared region and image data in the middle infrared region, and identifying the object based on them (for example, , See Patent Document 2).

JP-A-6-174828 Japanese Patent No. 3431206

  The conventional object identification device has a similar absolute temperature and temperature distribution of the object and flare when the object is in a long-distance state (a state where the object is moving far away from the imaging device). As a result, misidentification may occur and misleading may occur, and reliability is lacking.

  Therefore, an object of the present invention is to provide a highly reliable object identification device that can accurately identify a target even when the object is in a traveling state.

  In order to achieve the above object, the object identification device according to claim 1 includes first image data that is image data in the first wavelength band and second image data that is image data in the second wavelength band. Image data acquisition means for acquiring image data from the same field of view; image data for setting one of the first image data and the second image data as primary target image data and the other as secondary target image data Setting means; first storage means for storing a first state pattern indicating the state of the object on the primary target image data; and second indicating the state of the object on the secondary target image data. Second storage means for storing the state pattern, first similarity calculation means for calculating a first similarity evaluation value indicating a degree of similarity with the first state pattern on the primary target image data, and Based on the first similarity evaluation value Candidate position extracting means for extracting a candidate position of an object; second similarity calculating means for calculating a second similarity evaluation value indicating similarity to the second state pattern at the candidate position; and the first similarity Identification means for identifying an object in the field of view based on the degree evaluation value and the second similarity evaluation value.

  According to the present invention, it is possible to provide a highly reliable object identification device that can accurately identify a target even when the object is in a traveling state.

The figure which shows the structure of the target object identification apparatus which concerns on Embodiment 1 of this invention. The flowchart which shows the process of the target object identification apparatus which concerns on Embodiment 1 of this invention. The figure which shows the long wavelength infrared image data and medium wavelength infrared image data which concern on Embodiment 1 of this invention. The figure explaining the calculation method of the 1st similarity evaluation value which concerns on Embodiment 1 of this invention. The figure explaining the extraction method of the candidate position which concerns on Embodiment 1 of this invention. The figure explaining the extraction method of the candidate position which concerns on Embodiment 1 of this invention. The figure which shows the structure of the target object identification apparatus which concerns on Embodiment 2 of this invention. The figure explaining the setting method of the object area | region which concerns on Embodiment 2 of this invention. The figure which shows the structure of the target object identification apparatus which concerns on Embodiment 3 of this invention. The figure which shows the structure of the target object identification apparatus which concerns on Embodiment 4 of this invention.

  Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1)
FIG. 1 is a configuration diagram showing the configuration of the object identification device according to Embodiment 1 of the present invention. As shown in FIG. 1, the object identification device includes an imaging system 100 that captures infrared image data of two wavelength bands (for example, a long wavelength band and a medium wavelength band) of the same field, and two of the infrared image data of the two wavelength bands. One (long-wavelength infrared image data) is set as primary target image data 104, and the other (medium-wavelength infrared image data) is set as secondary target image data 105. A first state storage unit 106 (first storage unit) that stores a first state pattern indicating a state of an object on the primary target image data 104 input from the unit 103; A second state storage unit 107 (second storage unit) that stores a second state pattern indicating the state of the target object, and indicates a similarity to the first state pattern on the primary target image data 104 A first similarity calculation unit 108 (first similarity calculation unit) that calculates one similarity evaluation value, and a candidate position extraction unit 109 (candidate) that extracts a candidate position of an object based on the first similarity evaluation value Position extraction means), a second similarity calculation unit 110 (second similarity) that calculates a second similarity evaluation value indicating the similarity to the second state pattern at the candidate position extracted by the candidate position extraction unit 109 Calculation means), and an evaluation identification unit 111 (identification means) for identifying the existence region of the object in the field of view based on the second similarity evaluation value calculated by the second similarity calculation unit 110.

  The imaging system 100 is driven by a gimbal mechanism (not shown) or the like and is directed to an object. An image in the field of view is guided to the long wavelength infrared sensor 202 and the medium wavelength infrared sensor 203 via the lens 200 and the dichroic mirror 201. The dichroic mirror 201 transmits long-wavelength infrared light and reflects medium-wavelength infrared light, and is disposed behind the lens 200. A long wavelength infrared sensor 202 is disposed in the transmission direction, and a medium wavelength infrared sensor 203 is provided in the reflection direction.

  The long wavelength infrared sensor 202 converts the guided optical image into long wavelength infrared image data 101 (first image data), and outputs it to the image data setting unit 103. On the other hand, the medium wavelength infrared sensor 203 converts the guided optical image into medium wavelength infrared image data 102 (second image data) and outputs it to the image data setting unit 103 (image data setting means). .

  FIG. 2 is a flowchart of processing of the object identification device according to Embodiment 1 of the present invention. Hereinafter, processing performed by the object identification device will be described with reference to FIG. The imaging system 100 is driven by a gimbal mechanism or the like and is directed to an object, and an image in the field of view is guided to the long wavelength infrared sensor 202 and the medium wavelength infrared sensor 203 via the lens 200 and the dichroic mirror 201 ( St1).

  The long-wavelength infrared sensor 202 converts the optical image guided and formed in St1 into the long-wavelength infrared image data 101 and outputs it to the image data setting unit 103. On the other hand, the medium wavelength infrared sensor 203 converts the optical image guided and formed in St1 into medium wavelength infrared image data 102, and outputs it to the image data setting unit 103 (image data acquisition means) (St2). FIG. 3 shows the long wavelength infrared image data 101 and the medium wavelength infrared image data 102 output to the image data setting unit 103.

  The image data setting unit 103 sets the primary target image data 104 and the secondary target image data 105 based on the input long wavelength infrared image data 101 and medium wavelength infrared image data 102 (St3). The primary target image data 104 is output to the first similarity calculation unit, and the secondary target image data 105 is output to the second similarity calculation unit.

  The setting method of the primary target image data 104 and the secondary target image data 105 includes a shape of a region where the signal intensity on the long-wavelength infrared image data 101 is equal to or greater than a predetermined threshold, and a signal intensity on the medium-wavelength infrared image data 102. Are compared as the primary target image data 104, and the other is set as the secondary target image data 105.

The complexity evaluation method binarizes into a region (T1) that is greater than or equal to a predetermined threshold and a region (T2) that is less than the threshold depending on the signal intensity on the long-wavelength infrared image data 101. Further, binarization is performed into a region (T3) where the signal intensity on the medium-wavelength infrared image data 102 is a predetermined value or more and a region (T4) where the signal intensity is less than the threshold. Complex on the binarized complexity E ₁ of the circumferential length L and from the area S region of the shape of each region T1, T2 equal to or larger than the threshold value of both infrared image data is calculated based on the following formula (1) It is evaluated by comparing the length. It is also possible to compare the complexity in terms of calculating the circularity E ₂ based on the following equation (2). That is, it can be evaluated that the shape is more complex as the value of the complexity E ₁ is larger and the value of the circularity E ₂ is smaller.

  Furthermore, when the wavelength band in which the shape of the region is complicated is known in advance, the data of the wavelength band can be used as the primary target image data 104. By setting the primary target image data 104 in this way, it is not necessary to perform the above-described complexity evaluation process, and the target can be identified more efficiently.

  Further, the long wavelength infrared image data 101 that is longer wavelength data may be set as the primary target image data 104, and the medium wavelength infrared image data 102 that is shorter wavelength data may be set as the secondary target image data 105. By determining the primary target image data 104 and the secondary target image data 105 in this way, it is not necessary to perform the above-described complexity evaluation process, and the target can be identified more efficiently.

  After St3, the first similarity character calculation unit 108 calculates the first similarity evaluation value at each coordinate on the primary target image data 104 input from the image data setting unit 103 (St4).

  As shown in FIG. 4, the calculation method of the similarity evaluation value includes the signal intensity 106a of the first state pattern stored in the first state storage unit 105 shown in FIG. The signal intensity of the area corresponding to the size of the area of the first state pattern in the coordinates is compared, and the similarity is calculated.

  As the signal intensity 106a of the first state pattern, a signal that stores a state pattern of an object acquired in advance may be used. Or you may memorize | store the state pattern of the target object identified immediately before by the evaluation identification part 111 mentioned later. In this case, it is particularly effective in the case of an object whose signal intensity shape gradually changes.

There is a possibility that the first similarity calculation unit 108 may be erroneously identified due to overlapping of imaging conditions, signal levels, and evaluation areas. Therefore, in order to reduce misidentification at the time of evaluating the similarity of the primary target image data 104, an object identification method based on vector information is used. The vector information is given as a signal intensity difference between pixels in the image area, and can be expressed by the following expression (3).

In Expression (3), I is the signal strength of the pixel, and the subscripts i and j are indexes for distinguishing the pixels. The magnitude of the vector R _{i, j} obtained from Equation (3) indicates the strength of the signal strength gradient between pixels. The vector direction indicates the connectivity of the pixels. That is, the vector R _{i, j} represents an orthogonal component with respect to the brightness gradient of the pixel.

The first similarity calculation unit 108 evaluates the statistic of the signal distribution shape using the input primary target image data 104 and the vector R _{i, j} . By using this statistic, it is possible to replace the target region extraction index from the absolute amount (intensity itself) to the relationship between the intensities. Accordingly, it is possible to make the first similarity calculation unit 108 have a robust property with respect to a change in signal level.

As a specific similarity calculation method based on the vector information, the first state pattern vector information Ra stored in the first state storage unit 105 and the primary target image data 104 The vector information Rb of the area corresponding to the size of the area of the first state pattern at each coordinate is compared. For example, it is calculated by the sum E ₃ of the inner products of the vectors as in the following equation (4). be able to. It can be said that the higher the degree of similarity this E ₃ is small.

Furthermore, when the size of the object in the image changes, it can be dealt with by comparing histograms relating to the intensity and direction of vector information, which is an invariable amount with respect to the change in size. For example, a histogram relating to the direction is created by dividing the entire 360 ° into sections such as how many vectors in the target area are oriented in the direction of 0 ° to 5 °, and counting the frequency of vectors corresponding to each section. To do. The calculation of the similarity between histograms, a histogram of the vector information Ra p, can be considered, such as using a Bhattacharyya distance E ₄ shown in the following equation when the histogram of the vector information and q (5).

here,

It is. Further, m represents a partition function, and p ^(u) represents the frequency of the u th section. At this time,

It is necessary to satisfy.

  The first similarity evaluation value at each coordinate calculated by the first similarity calculation unit 108 is input to the candidate position extraction unit 109, and the candidate position of the target object is extracted and determined based on the similarity evaluation value. (St5).

  Here, the candidate position of the object is determined based on the position showing a high degree of similarity. For example, the top N candidate positions are extracted in descending order of similarity. In addition, all the positions that show the degree of similarity equal to or higher than a predetermined threshold may be extracted. The candidate position extraction unit 109 inputs the top N candidate positions of the extracted similarities to the second similarity calculation unit 110.

  Similarly to the first similarity calculation unit 108, the second similarity calculation unit 110 calculates the similarity on the secondary target image data 105 based on the second state pattern stored in the second state storage unit 107. A second similarity evaluation value is calculated at the top N candidate positions (St6).

  The N first similarity evaluation values and the N second similarity evaluation values calculated by the first similarity calculation unit 108 and the second similarity calculation unit 110 in St4 and St6 are evaluated and identified. The position of the object is identified by the evaluation identification unit 111 by combining the evaluation values and input to the unit 111 (St7).

  Here, with respect to the first similarity evaluation value calculated by the first similarity calculation unit 108, positions with high similarity are likely to be hardened on the image data, and the number N of candidate positions extracted by the candidate position extraction unit 109 is determined. There is a case where sufficient identification cannot be made unless a certain amount is set.

  In such a case, the entire image can be evaluated more efficiently if the candidate positions are extracted with some dispersion. As a method for determining the degree of dispersion, for example, there is a method in which the distance between candidate positions is determined to be maintained at a predetermined distance or more.

As shown in FIG. 5, candidate positions 109a are extracted in descending order of similarity, and candidate positions are extracted from an area that is a predetermined distance ₁₁ away from the extracted candidate positions 109a. That is, only the candidate position 109a is within the circular area 109c with the radius l ₁ centered on the candidate position 109a. The predetermined distance l ₁ can be arbitrarily set according to the magnitude of the signal intensity of the first state pattern. That is, as shown in FIG. 6, the candidate position in the area 109d corresponding to the area of the first state pattern is only the candidate position 109a.

  As described above, in the first embodiment, a region where no signal is obtained or the signal intensity is saturated based on image data of a plurality of wavelength bands (long wavelength infrared image data 101, medium wavelength infrared image data 102). It is possible to interpolate the information of the existing area, so that a detailed temperature state can be observed. Further, even if there is an overlap between the object and the flare image in the course of flare emission, the object can be appropriately identified and detected. Therefore, the object identification performance is improved, and a highly reliable object identification apparatus can be provided.

  The circular region 109c is not limited to a circular shape, and may be a rectangular shape, for example.

(Embodiment 2)
Next, a second embodiment of the present invention will be described. FIG. 7 is a diagram showing a configuration of the object identification device according to Embodiment 2 of the present invention. The same components as those in the first embodiment are denoted by the same reference numerals, and redundant description is omitted. The second embodiment is different from the first embodiment in that the object identification apparatus shown in FIG. 1 further includes an evaluation area setting unit 112 (evaluation area setting means).

  The primary target image data 104 set in the image data setting unit 103 is input to the evaluation region setting unit 112, and a target region for calculating the first similarity evaluation value is set in the first similarity calculation unit 108. .

For example, as shown in FIG. 8, the target region setting method sets a minimum rectangle for the extracted region 104a in an image binarized by extracting a threshold band of a predetermined signal strength, as shown in FIG. This is defined as a primary evaluation area 112a. Further, an area expanded with a predetermined margin l ₂ is set as a secondary evaluation area 112b. By inputting the secondary evaluation area 112b to the first similarity calculation unit 108, it is possible to limit the area for calculating the first similarity evaluation value on the primary target image data.

  As described above, in Embodiment 2, since the number of similarity evaluation value calculation processes can be reduced, the identification efficiency of the object identification device can be improved.

(Embodiment 3)
Subsequently, Embodiment 3 of the present invention will be described. FIG. 9 is a diagram showing a configuration of the object identification device according to Embodiment 3 of the present invention. The same components as those in the first embodiment are denoted by the same reference numerals, and redundant description is omitted. The third embodiment is different from the first embodiment in that it includes a mutual evaluation identification unit 113 (identification means) instead of the evaluation region identification unit 111 shown in FIG.

  Similar to the first embodiment, N first similarity evaluation values and N second similarity evaluation values at N candidate positions are calculated and input to the mutual evaluation identifying unit 113. The mutual evaluation identification unit 113 evaluates the difference between the evaluation values. When a plurality of similar objects exist for one object, the difference between the evaluation values is calculated, and the most different candidate position is identified as the object.

  As described above, in the third embodiment, a target object can be identified by evaluating the evaluation values of a plurality of target objects with each other. Further, identification is possible even if the pattern information of the object is not held in advance.

(Embodiment 4)
Next, a fourth embodiment of the present invention will be described. FIG. 10 is a diagram showing a configuration of an object identification device according to Embodiment 4 of the present invention. The same components as those in the first embodiment are denoted by the same reference numerals, and redundant description is omitted. The fourth embodiment is different from the first embodiment in that the object identification apparatus shown in FIG.

  As in the first embodiment, N first similarity evaluation values and N second similarity evaluation values at N candidate positions are calculated and input to the evaluation identification unit 111 to determine whether the object is an object. At the same time, the information is input to the evaluation history storage unit 114 and stored. The storage capacity in the evaluation area storage unit 114 can be arbitrarily set by the user. For example, in the case of setting for the past M frames, the candidate positions and similarity evaluation values of the object for the previous M frames can be accumulated. Then, the evaluation identifying unit 111 identifies whether or not the object is an object by integrating the candidate positions of the objects for M frames stored in the evaluation value history storage unit 114 and the temporal change in the similarity evaluation value.

  As described above, in the fourth embodiment, by recording the evaluation value E of each target candidate region over a certain period of M frames, a change in shape over time can be evaluated as one feature amount. Identification can be reduced. Since an analog such as flare is emitted from the exhaust part of the object, its temperature attenuates, so that it works effectively in identifying the object and flare.

  In the present embodiment, the plurality of wavelength bands have been described as the long wavelength band and the medium wavelength band, but the wavelength bands may be different.

  Note that the present invention is not limited to the above-described embodiment, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above-described embodiments. For example, some constituent elements may be deleted from all the constituent elements shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

100 imaging system 101 long-wavelength infrared image data 102 medium-wavelength infrared image data 103 image data setting unit 106 first state storage unit 107 second state storage unit 109 candidate position extraction unit 110 second similarity calculation unit 111 evaluation identification unit 112 evaluation Area setting unit 113 Mutual evaluation identification unit 200 Lens 201 Dichroic mirror 202 Long wavelength infrared sensor 203 Medium wavelength infrared sensor

Claims (9)

Image data acquisition means for acquiring, in the same field of view, first image data that is image data in the first wavelength band and second image data that is image data in the second wavelength band;
Image data setting means for setting one of the first image data and the second image data as primary target image data and the other as secondary target image data;
First storage means for storing a first state pattern indicating a state of an object on the primary target image data;
Second storage means for storing a second state pattern indicating the state of the object on the secondary target image data;
First similarity calculation means for calculating a first similarity evaluation value indicating a similarity with the first state pattern on the primary target image data;
Candidate position extracting means for extracting candidate positions of the object based on the first similarity evaluation value;
Second similarity calculation means for calculating a second similarity evaluation value indicating a similarity with the second state pattern at the candidate position;
Identification means for identifying an object in the field of view based on the first similarity evaluation value and the second similarity evaluation value;
An object identification device characterized by comprising: The image data setting means includes
The primary target image data and the secondary target image data are set based on a comparison of the complexity of the shape of the region where the signal intensity on the first image data and the second image data is equal to or greater than a predetermined threshold. The object identification device according to claim 1, wherein: The image data setting means includes
2. The object identification device according to claim 1, wherein primary target image data and secondary target image data are set based on lengths of wavelength bands of the first image data and the second image data. The first state pattern and the second state pattern are:
The object identification device according to any one of claims 1 to 3, wherein the object state pattern is a state pattern of an object acquired in advance. The first state pattern and the second state pattern are:
The object identification device according to any one of claims 1 to 3, wherein the object identification pattern is a state pattern of the object identified by the identification means. An evaluation area setting means for setting an area in which the signal intensity on the primary target image data is equal to or greater than a predetermined threshold as a primary evaluation area, and an area obtained by extending the primary evaluation area with a predetermined margin as a secondary evaluation area; In addition,
The first similarity calculation means includes:
The first similarity evaluation value indicating a similarity with the first state pattern in the secondary evaluation area on the primary target image data is calculated. 6. The object identification device described in the paragraph. The candidate position extracting means includes
7. If there are a plurality of candidate positions for the object, the candidate positions of the object are extracted so that the distance between the candidate positions of the object is equal to or greater than a predetermined distance. The object identification device described in the paragraph. The identification means includes
The object identification according to any one of claims 1 to 7, wherein the first similarity evaluation value and the second similarity evaluation value are compared with each other, and the most different candidate position is set as an object. apparatus. The identification means includes
9. The object identification device according to claim 1, wherein an object is identified in the field of view based on a time change of the first similarity evaluation value and the second similarity evaluation value. .