JP6042146B2 - Object detection apparatus and object detection method - Google Patents

Description

  The present invention relates to an object detection device and an object detection method that are mounted on, for example, a launcher of a guided flying object or a guided flying object and are used for detecting and tracking an object, and in particular, the object itself is also moving. The present invention relates to a technique for detecting a moving object.

  A method of using an imaging device such as an infrared camera is known for a guidance system that guides a guided flying object launched from the ground or in the air toward a target object (hereinafter referred to as a moving object). . In this method, the direction of a moving object is photographed with an imaging device, and image data based on infrared rays or the like acquired by photographing is image-processed to generate an image based on a luminance region distribution. Then, a high brightness area in the image is extracted as a moving object, and image feature values such as maximum brightness value, minimum value, average brightness, and size of each extracted area are calculated, and based on these image feature values. In addition, the region closest to the image pattern prepared in advance is detected as a moving object and guided for tracking.

  However, in reality, when trying to detect a target from the sky, objects that are confused with objects such as the ground surface and clouds enter the image, and there is an area that shows the same luminance as the moving object in the background of the moving object. It often exists. In this case, it is difficult to accurately detect the moving object by distinguishing it from the background using only the image features. Therefore, conventionally, in order to accurately detect a moving object even in such a state, a related area is extracted from each image data obtained continuously, and a feature amount reflecting the movement of each area is extracted, There is a technique for distinguishing a moving object from a background based on this.

  However, many of the existing technologies extract a high luminance area from the infrared image, define the coordinates of the center of gravity of the extracted area as the position of the area, and by moving the position of each area (center of gravity coordinates), It is detected as a moving object.

  However, in the above method, since the shape of the region changes due to a change in contrast or the influence of noise, the coordinates of the center of gravity easily change. For this reason, even when the area is stationary, the position of the center of gravity may change due to a change in contrast or the influence of noise, and an erroneous movement of the area may be detected. Such false detection does not affect much if the target moving object is nearby and the apparent movement is large, but if the target is far away and the apparent movement is small. It will have a serious impact.

  For example, the target detection device disclosed in Patent Document 1 can detect an apparently small object in the distance by matching a template created with a Gaussian distribution shape with an image on infrared image data. Since the original image of the object is small, there is a possibility of erroneous identification when an image having the same size exists in the same image.

JP 2009-69019 A JP 2007-171077 A Japanese Patent No. 4152698

  As described above, with the existing object detection technology, the position of the center of gravity may change due to contrast changes or noise effects, and movement of the wrong area may be detected. If the movement of the upper image is very small, there is a serious influence such as the possibility that the same image existing in the same image may be erroneously detected.

  This embodiment has been made to solve the above problem, and can detect a moving object more reliably from the image without being affected by a change in contrast or noise of the acquired image. An object of the present invention is to provide an object detection apparatus and an object detection method capable of further improving the detection performance of an object flying in a low sky.

In order to solve the above-described problem, according to the present embodiment, electromagnetic wave image data that is moved by the image data acquisition unit is sequentially acquired at a specified rate, and the luminance band designation unit performs the luminance gradation of the electromagnetic wave image data. A part of the luminance band is designated, a region indicating the value of the designated luminance band is extracted from the electromagnetic wave image data by the region extracting unit, and a plurality of features are extracted from the shape feature of the region by the feature point extracting unit. Points are extracted, similar feature points are associated with the plurality of electromagnetic wave image data sequentially acquired by the feature point association unit, and the feature points are compared with each other from the feature point association result by the three-dimensional calculation unit. calculates the three-dimensional position, to estimate the most distance is close surfaces of the respective feature points in the three-dimensional three-dimensional space from a position between the characteristic point, the object detecting unit from the estimated surface Serial distance from a set of 3-dimensional positions of feature points to the manner of detecting the feature point as an object indicating a farthest 3D position.

FIG. 2 is a block diagram illustrating a configuration and functions of the object detection device according to the first embodiment. FIG. 5 is a diagram for explaining an operation example of an image data acquisition unit of the object detection device according to the first embodiment. The block diagram for demonstrating the luminance zone designation | designated example of the luminance zone designation | designated part of the target object detection apparatus which concerns on Embodiment 1. FIG. The figure for demonstrating the operation example of the area | region extraction part of the target object detection apparatus which concerns on Embodiment 1. FIG. The figure for demonstrating the operation example of the feature point extraction part of the target object detection apparatus which concerns on Embodiment 1. FIG. The figure for demonstrating the other operation example of the feature point extraction part of the target object detection apparatus which concerns on Embodiment 1. FIG. The figure for demonstrating the operation example of the matching process part of the target object detection apparatus which concerns on Embodiment 1. FIG. The figure for demonstrating the other operation example of the matching process part of the target object detection apparatus which concerns on Embodiment 1. FIG. The figure for demonstrating the operation example of the target object detection part of the target object detection apparatus which concerns on Embodiment 1. FIG. 5 is a flowchart showing a process flow of the object detection apparatus according to the first embodiment. FIG. 5 is a block diagram illustrating a configuration and functions of an object detection device according to a second embodiment.

  Hereinafter, the object position detection apparatus according to the present embodiment will be described with reference to the drawings.

(Embodiment 1)
First, Embodiment 1 will be described with reference to FIGS.

  FIG. 1 is a block diagram illustrating a configuration and functions of an object detection device according to the first embodiment. The object detection apparatus includes an image data acquisition unit 100 that acquires electromagnetic wave image data, a luminance band specification unit 101 that specifies a partial luminance band for the luminance gradation of the electromagnetic wave image data, and the electromagnetic wave image data. A region extracting unit 102 for extracting a region indicating the value of the luminance band designated by the luminance band designating unit 101, a feature point extracting unit 103 for extracting feature points from the electromagnetic wave image data, and a plurality of image data. A correspondence processing unit 104 that associates feature points, a three-dimensional calculation unit 105 that calculates a three-dimensional position of the feature points from the feature point correspondence information, and a feature point position is identified from the three-dimensional calculation result, and the distance is the longest. It is comprised from the target object detection part 106 which detects the feature point which has left | separated as a target object, and detects a target object from electromagnetic wave image data.

  Below, operation | movement of each process part of the target object detection apparatus of this Embodiment 1 is demonstrated.

  In the object detection apparatus of the first embodiment, first, the image data acquisition unit 100 acquires an infrared image as the electromagnetic wave image data 300 using, for example, an infrared camera, and outputs image data at a constant rate. The image data acquisition unit 100 has a function of acquiring image data of a plurality of wavelength bands from image data within the same visual field. For example, a configuration as shown in FIG. 2 is conceivable for acquiring infrared image data of two wavelength bands.

  The image data acquisition unit 100 illustrated in FIG. 2 directs the imaging direction in a direction in which the object is supposed to exist, captures an image of the object with the lens 200, and uses the dichroic mirror 201 disposed at the rear part of the lens 200 to obtain a medium wavelength. The configuration guides the infrared sensor 202 and the long wavelength infrared sensor 203. Here, it is assumed that the dichroic mirror 201 transmits long-wavelength infrared light and reflects medium-wavelength infrared light in the incident infrared image. A long wavelength infrared sensor 203 is disposed in the transmission direction, and a medium wavelength infrared sensor 202 is disposed in the reflection direction. Thereby, infrared images of two wavelengths are obtained simultaneously from the sensors 202 and 203. For guidance of the image of the object to both sensors 202 and 203, a medium-wavelength infrared sensor 202 and a long-wavelength infrared sensor 203 may be arranged at each focal position using a multifocal lens. .

  The luminance band designation unit 101 allows the user to arbitrarily designate a part of the luminance band with respect to the luminance gradation of the electromagnetic wave image data. For example, when the gradation of the electromagnetic wave image data is 256 gradations, it can be set to 200 to 250. Thereby, for example, a plurality of luminance bands (A, B) can be set as shown in FIG. 3B for the luminance level at each pixel position shown in FIG. When N luminance bands are set, N extraction result images from the region extraction unit 102 are output. The feature point extraction unit 103 to the association processing unit 104 sequentially repeat N times for the N outputs. Will be. For example, when the luminance band A and the luminance band B are set as shown in FIG. 3B, image data as shown in FIGS. 4A and 4B is output from the region extraction unit 102, Different regions 302a and 302b are obtained respectively.

  The area extraction unit 102 extracts the area of the luminance band designated by the luminance band designation unit 101. That is, the brightness of the area indicating a value outside the designated brightness band is set to 0.

  The feature point extraction unit 103 extracts a characteristic part from the region extracted by the region extraction unit 102 as a feature point. For example, the center of gravity of the extracted region or the position indicating the maximum luminance in the region is extracted as the feature point.

  FIG. 5 shows a case where the centroid 301 of the extraction region 302 is extracted as a feature point from the image data 300 as an extraction example of the feature point extraction unit 103. Actually, a plurality of feature points are extracted on the image data 300.

  Furthermore, as another method of extracting feature points, as shown in FIGS. 6A and 6B, a local region 303 is set in the image data, and a feature having a large luminance dispersion in the local region is characterized. It can be considered to extract the point 304. That is, a part where the luminance changes rapidly is extracted as a feature point.

  The association processing unit 104 executes processing for associating the feature points extracted by the feature point extraction unit 103 among a plurality of image data. As basic processing, as shown in FIG. 7, the similarity between the feature points between the image data 300a and 300b is calculated and compared, and the most similar feature points are associated with each other. That is, combinations of the number of feature points corresponding to which (x, y) coordinates of the image data 300a correspond to which (x, y) coordinates of the image data 300b are obtained. The similarity may be calculated by setting a region 305 of the same size around the feature point and calculating the correlation of the region.

  Here, when the movement direction of the target object detection apparatus is taken into consideration at the time of association, it is possible to improve the accuracy of association and reduce the processing amount. For example, when the image data acquired continuously is observed when flying substantially parallel to the ground surface, the feature points of the image data 300a are moved as shown in FIGS. 8 (a) and 8 (b). It appears that 306 is limited to a certain direction. The search range 307 is set as shown in FIG. 8C in consideration of the movement of the feature point along a position close to the moving direction. Thereby, the candidates for association can be narrowed down, and the accuracy can be further improved.

  The three-dimensional calculation unit 105 calculates the three-dimensional position of each feature point from the association information obtained by the association processing unit 104. In general, a technique for reconstructing a three-dimensional structure to be photographed from images photographed from different angles has been established, and if information on this correspondence can be obtained, three-dimensional position information of feature points can be obtained. Can do. Patent documents 2 and 3 are publicly known examples of these techniques. By using these techniques, the three-dimensional calculation unit 105 calculates the relative three-dimensional positional relationship between feature points.

  In the association processing unit 104, the association is performed and the similarity in the association is also calculated. When there are many feature points to be extracted, there is a high probability that erroneous associations are included, but by selecting associations with high similarity, the accuracy of calculation of the three-dimensional position can be improved. It is also possible to select feature points.

  By specifying a plurality of N brightness bands in the above-described brightness band specifying unit 101, the processing in the region extracting unit 102, the feature point extracting unit 103, and the association processing unit 104 is performed N times. Further, when M images are acquired in the image data acquisition unit 100, N × M times are performed. By performing it a plurality of times, the number of extracted feature points and the number of association information increase, and the accuracy of the three-dimensional information reconstructed in the three-dimensional calculation unit 105 can be improved.

  Next, the object detection unit 106 detects the object based on the three-dimensional position of the feature point calculated by the three-dimensional calculation unit 105. As shown in FIG. 9, a set 400 of feature points corresponding to the ground surface and a feature point 401 of the object are identified. Here, a point on the ground surface is a feature point that is close to the approximated two-dimensional surface as a feature point of an object on the ground surface, and a feature point that is far away from the surface is a target object. Will be detected. That is, the plane or curved surface that is closest to each feature point in the three-dimensional space is estimated, and the feature point 401 that is the most distant from the same surface is detected as an object.

  In addition, when the object is moving and the feature point on the ground surface and its three-dimensional position can be obtained stably, the feature point of the object fluctuates in time with respect to the feature point on the ground surface. It is also possible to identify the feature points of the object by comparing the temporal change amounts of the three-dimensional positions.

Further, if the ground surface can be regarded as a plane, the object can be detected using a homography matrix obtained from the association information obtained by the association processing unit 104. The homography matrix indicates the positional relationship between planes imaged on two images, and can be obtained from four sets of association information. For example, let k ₁ = [x ₁ , y ₁ , 1] ^T be the homogeneous coordinates of a certain feature point on the k = 1st image, and the homogeneous coordinates of the feature point in the k = 2nd image corresponding to it. When x ₂ = [x ₂ , y ₂ , 1] ^T , the matrix H that establishes the formula (1) is a homography matrix.
s · x ₂ = Hx ₁ (1)
Here, s is a scaling factor.

  Basically, it can be obtained if there are four sets of correspondence information, but if there is more correspondence information, by using the generally known RANSAC method, The most probable homography matrix H can be obtained.

After the homography matrix H is obtained, the coordinate value of the feature point is substituted into the equation (1), and the feature point indicating the correspondence having the largest distance D after the conversion is detected as the object.
D = || s · x ₂ −Hx ₁ || (2)
Since D = 0 holds for points on the same plane, it is considered that D is the maximum for an object that is not placed on the ground surface.

  Here, since the feature point on the ground surface is a stationary object, but the object is assumed to move, the three-dimensional position information corresponding to the object is information for detection and is an actual three-dimensional It should be noted that the position is not shown. The feature point having the deviating three-dimensional position information is tracked on the two-dimensional image data.

  FIG. 10 shows a flowchart of processing in the object detection apparatus of this embodiment. First, infrared image data is acquired by the image data acquisition unit 100 (step S1), and the region for each luminance band is determined by the region extraction unit 102 using an arbitrary luminance band preset by the luminance band specifying unit 101 (step S2). The feature points are extracted from each region by the feature point extraction unit 103 (step S4). Subsequently, the association processing unit 104 associates feature points between a plurality of image data (step S5), and the three-dimensional calculation unit 105 calculates the three-dimensional position of each feature point from the feature point association result. (Step S6), the object detection unit 106 classifies the three-dimensional positions of the feature points, detects the object (Step S7), and ends the series of processes.

  Here, when the number of images acquired by the image data acquisition unit 100 is M and the number of luminance bands specified by the luminance specifying unit 101 is N, the processes S3 to S5 are executed M × N times. Become.

  As described above, in the first embodiment, the relative three-dimensional positional relationship between feature points is calculated from the feature point association on the image, and the feature farthest from the set of feature points on the ground surface is calculated. Since the point is detected as an object, the object can be detected with higher accuracy. In addition, since the feature points are extracted according to the electromagnetic wave images of a plurality of wavelength bands and the settings of the plurality of luminance bands, the ground surface can be accurately grasped. As a result, even if an image similar to the image of the object is reflected two-dimensionally in the background of the object, it is detected by the three-dimensional position information, so that erroneous detection can be prevented and detection performance is improved. It is possible to provide an object detection device.

(Embodiment 2)
The object detection apparatus according to the second embodiment will be described with reference to FIG.

  FIG. 11 shows the configuration and function of an object detection apparatus according to the second embodiment. In the second embodiment, an altitude information acquisition unit 107 and a posture information acquisition unit 108 are added to the configuration of the object detection device in the first embodiment.

  In the following, the operation of each processing unit of the object detection device according to the second embodiment will be described mainly with respect to differences from the first embodiment.

  First, as in the first embodiment, the image data acquisition unit 100 acquires image data. At the same time, the altitude information acquisition unit 107 acquires the altitude at which the target object detection device exists, and the posture information acquisition unit 108 The direction / attitude information to which the image data acquisition unit 100 is directed is acquired. For example, the altitude information acquisition unit 107 can acquire position information from GPS, or can measure altitude by a change in atmospheric pressure using a pressure gauge. For the posture information acquisition unit 108, a gyro sensor or the like can be considered.

  If these altitude and posture information are input to the object detection unit 106, the direction of the object detection apparatus itself can be known, and thus the direction of the ground surface can be detected. For example, the object detection device may rotate during flight and change its posture, and if the direction of the ground surface is unknown and the number of feature points is small, the set of feature points on the ground surface cannot be specified. there is a possibility. However, if the ground surface direction is known, it becomes easier to specify a set of feature points that correspond to feature points on the ground surface and are close to the surface, and erroneous detection in the object detection unit 106 can be prevented.

  As described above, according to the target object detection device of the second embodiment, when detecting a target object based on three-dimensional information, it is possible to grasp its own position and posture, and further, feature points of the ground surface And the identification performance of the object can be improved, and erroneous detection can be prevented more reliably.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  DESCRIPTION OF SYMBOLS 100 ... Image data acquisition part, 101 ... Luminance zone designation part, 102 ... Area extraction part, 103 ... Feature point extraction part, 104 ... Association processing part, 105 ... Three-dimensional calculation part, 106 ... Object detection part, 107 ... Altitude information acquisition unit, 108 ... posture information acquisition unit.

Claims (11)

Image data acquisition means for sequentially acquiring moving electromagnetic wave image data at a prescribed rate;
A luminance band specifying means for specifying a part of the luminance band for the luminance gradation of the electromagnetic wave image data;
A region extracting unit for extracting a region indicating a value of a luminance band designated by the luminance band designating unit with respect to the electromagnetic wave image data;
Feature point extracting means for extracting a plurality of feature points from the shape feature of the region;
Feature point associating means for associating similar feature points across the plurality of electromagnetic wave image data acquired sequentially;
Three-dimensional calculation means for calculating a relative three-dimensional position between feature points from the result of the feature point association;
A surface closest to each feature point in the three-dimensional space is estimated from the three-dimensional position between the feature points, and the feature point indicating the three-dimensional position farthest from the estimated surface is the target. Object detection means for detecting as,
An object detection apparatus comprising:   The object detection apparatus according to claim 1, wherein the image data acquisition unit has a function of simultaneously acquiring electromagnetic wave data of a plurality of wavelength bands in the same field of view.   2. The object detection apparatus according to claim 1, wherein the region extraction unit sets a plurality of luminance bands to be extracted from the region in advance and acquires a plurality of region extraction results for each luminance band.   The object detection apparatus according to claim 1, wherein the feature point extraction unit has a function of using the center of gravity of the region extracted by the region extraction unit as a feature point.   2. The object detection apparatus according to claim 1, wherein the feature point extraction unit has a function of using a peak in the region extracted by the region extraction unit as a feature point.   2. The object detection according to claim 1, wherein the feature point extraction unit has a function of detecting, as a feature point, a center position of a local area having a large luminance dispersion in a local area having an arbitrarily set size. apparatus.   The object detection apparatus according to claim 1, wherein the feature point association means associates and associates with the traveling direction of the electromagnetic wave image data. The feature point association means has a function of holding the similarity between feature points in each association,
The object detection apparatus according to claim 1, wherein the three-dimensional calculation unit has a function of calculating a three-dimensional position using the association with the high similarity. The object detection unit has a function of detecting a feature point having the most movement amount as an object with reference to a history of the three-dimensional position of the feature point calculated by the three-dimensional calculation unit. Item 1. The object detection apparatus according to Item 1.   Further, the apparatus includes an altitude information acquisition means for acquiring altitude information and an attitude information acquisition means for acquiring attitude information, wherein the object detection means detects the direction of the ground surface from the altitude information and the attitude information and The object detection apparatus according to claim 1, wherein the object detection apparatus has a function of specifying a set of feature points and detecting a feature point most deviating from the set as an object. Acquire moving electromagnetic wave image data sequentially at a specified rate,
Specify a part of the luminance band for the luminance gradation of the electromagnetic wave image data,
Extracting a region indicating the value of the designated luminance band for the electromagnetic wave image data,
Extracting a plurality of feature points from the shape features of the region;
Corresponding similar feature points across the plurality of sequentially acquired electromagnetic wave image data,
Calculating a relative three-dimensional position between the feature points from the result of matching the feature points;
A surface closest to each feature point in the three-dimensional space is estimated from the three-dimensional position between the feature points, and the feature point indicating the three-dimensional position farthest from the estimated surface is the target. The object detection method characterized by detecting as follows.

Images