JP5347798B2 - Object detection apparatus, object detection method, and object detection program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device which detects a specific object without using a marker. <P>SOLUTION: An object detection device includes: a feature storage part 22 which stores a feature point of a background image which does not include a target object; an invariant feature storage part 32 which stores an invariant feature which shows the feature point of the target object in an invariant feature space; a comparison means 40 which compares the invariant feature which shows in the invariant feature space, the feature points obtained by subtracting from the feature point of the detection target image, the feature point corresponding to the background image which does not include the target object, with the invariant feature which shows the feature point of the target object in the invariant feature space and determines whether the target object is detected by the comparison. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to an object detection device, an object detection method, and an object detection program for detecting a specific object appearing in a predetermined space.

As a technique for identifying whether or not there is a desired object in a certain space, a marker having a predetermined shape is attached to the target object in advance, and the target object is determined according to the presence or absence of the marker in the background image of the space. In general, a marker detection method for determining the presence or absence of a mark is known.
For example, Patent Document 1 discloses a marker detection method that uses a marker generated from an image feature that does not appear in a background video, which is derived based on the feature point extracted from a background video that does not include a marker. Is disclosed.
According to this marker detection method, the same pattern as the marker generated by such a method is attached to the target object at the preparation stage, and this pattern is detected from a predetermined spatial image at the detection stage. It can be determined that the target object exists in the space.

International Publication No. 2008/090908 Pamphlet

  However, such a marker detection method requires more time and effort for manufacturing the marker, and the manufactured marker needs to be affixed to the target object. A method that could do this was desired.

  An object of the present invention is considered in view of the above circumstances, and an object thereof is to provide an object detection device, an object detection method, and an object detection program capable of detecting a specific object regardless of a marker.

  To achieve this object, the object detection device of the present invention includes a feature storage unit that stores feature points of a background image that does not include a target object, and an invariant feature that represents the feature points of the target object in an invariant feature space. An invariant feature storage unit for storing, an invariant feature representing a feature point obtained by subtracting a corresponding feature point of a background image not including the target object from a feature point of the detection target image in an invariant feature space, and the target object The comparison point is compared with the invariant feature represented in the invariant feature space to determine whether or not the target object has been detected.

  The object detection method of the present invention includes a step of storing feature points of a background image that does not include a target object, a step of storing invariant features representing the feature points of the target object in an invariant feature space, and a detection target A feature point obtained by subtracting a corresponding feature point of a background image not including the target object from a feature point of the image is represented in an invariant feature space, and a feature point of the target object is represented in an invariant feature space. And a step of determining whether or not the target object is detected by this comparison.

  Also, the object detection program of the present invention stores, in the object detection device, means for storing feature points of a background image not including the target object, and means for storing invariant features representing the feature points of the target object in an invariant feature space. And an invariant feature obtained by subtracting the corresponding feature point of the background image not including the target object from the feature point of the detection target image in the invariant feature space, and the feature point of the target object A comparison is made with the invariant feature represented in the variable feature space, and this comparison is made to function as a means for determining whether or not the target object has been detected.

  According to the object detection device, the object detection method, and the object detection program of the present invention, it is possible to detect a specific object regardless of the marker.

It is a block diagram which shows the structure of the object detection apparatus in 1st embodiment of this invention. It is a block diagram which shows the detailed structure of the object detection apparatus in 1st embodiment of this invention. It is an example figure which shows the image which a video input part inputs. It is the figure which represented typically the feature point extracted from the input image. It is a figure which shows a mode that the feature point was arrange | positioned in the feature space. It is a chart which shows the composition of feature point information. It is a figure which shows the difference of the feature point of a detection target image, and the feature point of a background image. It is explanatory drawing which shows the mapping method of the feature point when a base number is set to 1. It is explanatory drawing which shows the mapping method of the feature point when a base number is set to 2. It is explanatory drawing which shows the mapping method of the feature point when a base number is set to 3. It is a graph which shows the structure of invariant feature information. It is a first explanatory view showing a specific example of object detection according to the first embodiment of the present invention. It is the 2nd explanatory view showing the specific example of the object detection concerning a first embodiment of the present invention. It is 3rd explanatory drawing which shows the specific example of the object detection which concerns on 1st embodiment of this invention. It is a 4th explanatory view showing a specific example of object detection concerning a first embodiment of the present invention. It is a flowchart which shows the process sequence in the preparation stage in 1st embodiment of this invention. It is a flowchart which shows the process sequence in the detection stage in 1st embodiment of this invention. It is a block diagram which shows the detailed structure of the object detection apparatus in 2nd embodiment of this invention. It is a figure which shows the frequency distribution of the invariant feature represented by the invariant feature space. It is a 1st explanatory view showing a specific example of object detection concerning a second embodiment of the present invention. It is the 2nd explanatory view showing the example of object detection concerning a second embodiment of the present invention. It is 3rd explanatory drawing which shows the specific example of the object detection which concerns on 2nd embodiment of this invention. It is the 4th explanatory view showing the example of object detection concerning a second embodiment of the present invention. It is a flowchart which shows the process sequence in the preparation stage in 2nd embodiment of this invention. It is a flowchart which shows the process sequence in the detection stage in 2nd embodiment of this invention. It is a block diagram which shows the structure of the object detection apparatus in 3rd embodiment of this invention. It is a block diagram which shows the detailed structure of the object detection apparatus in 3rd embodiment of this invention. It is explanatory drawing for demonstrating the selection process of a peculiar feature. It is a flowchart which shows the process sequence in the preparation stage in 3rd embodiment of this invention. It is a flowchart which shows the process sequence in the detection stage in 3rd embodiment of this invention. It is a block diagram which shows the detailed structure of the object detection apparatus in 4th embodiment of this invention. It is explanatory drawing which shows the specific example of the object detection which concerns on 4th embodiment of this invention. It is a flowchart which shows the process sequence in the detection stage in 4th embodiment of this invention.

  Hereinafter, embodiments of an object detection apparatus, an object detection method, and an object detection program according to the present invention will be described with reference to the drawings.

[First Embodiment of Object Detection Apparatus and Object Detection Method]
First, a first embodiment of an object detection apparatus and an object detection method of the present invention will be described with reference to FIGS. 1 and 2.
FIG. 1 is a block diagram showing the configuration of the object detection apparatus of the present embodiment.
FIG. 2 is a block diagram showing a detailed configuration of the object detection apparatus of the present embodiment.

  As shown in FIG. 1, the object detection apparatus 1 a includes a video input unit 10, a feature extraction unit 20, an invariant feature conversion unit 30, and a comparison unit 40.

As shown in FIG. 2, the video input means 10 includes a video input unit 11 and a video storage unit 12.
The video input unit 11 inputs a target video (hereinafter referred to as “background image”) in a detection area that does not include the target object in a preparation stage before performing the detection process.
For example, as shown in FIG. 3A, an image of a background in which a target object (hereinafter referred to as a paper airplane) is not displayed corresponds to this.
The video input unit 11 inputs an image of the target object in the preparation stage.
As shown in FIG. 3B, the “target object image” is an image in which only the target object is projected. For example, the target object is imaged with a blue screen as a background, and a blue component is captured from the captured image. It corresponds to the one obtained by removing.

In the detection stage, the video input unit 11 inputs the target video of the area where the target object is detected. The video input at the detection stage is stored in the video storage unit 12 as a digitized frame image. A frame image refers to each still image frame.
Hereinafter, the image of the detection area input at the detection stage is referred to as a “detection target image”.
FIG. 3C shows an example of an image including a target object among the detection target images.

The video storage unit 12 stores an image (background image or target object image) input in the preparation stage and an image (detection target image) input in the detection stage.
The video storage unit 12 can store a number (for example, a serial number) assigned to each input frame image. This number uniquely identifies one frame image.

As shown in FIG. 2, the feature extraction unit 20 includes a feature extraction unit 21 and a feature storage unit 22.
The feature extraction unit 21 extracts a frame image from the video storage unit 12 and extracts an image feature including a characteristic pattern in the extracted frame image.
Specifically, in the preparation stage, the feature extraction unit 21 extracts a background image or a target object image from the video storage unit 12 to extract feature points, and stores the feature point information in the feature storage unit 22.

For example, as illustrated in FIG. 4A, a plurality of feature points of an object included in the image are extracted from the background image, and the feature point information is stored in the background storage area 221.
Further, as shown in FIG. 4B, the feature extraction unit 21 extracts feature points from the image of the target object, and stores the feature point information in the object storage area 222 of the feature storage unit 22.

  Furthermore, as shown in FIG. 4C, the feature extraction unit 21 extracts a detection point image from the video storage unit 12 and extracts a feature point at the detection stage, and inputs the feature point information to the feature storage unit 22. Store in the storage area 223.

As the image feature, for example, a graphic characteristic characteristic in numerical form can be used.
For this, for example, the method published in the 1998 IEEE Computer Vision Pattern Recognition Conference Proceedings can be used. This method can extract vertices of object shapes in images, intersections and end points of linear objects, and the like. A series of position coordinate information on the image of these points can be used as a graphic feature. A space in which feature points are arranged is called a feature space.

As another method, for example, there is a method described in “On the option detection of curves in noisy pictures” published in Montanari, 1971 Communications of ACM, Volume 14.
This can use the contents of the R table storing the distance from the reference point and the relative accuracy as features. At this time, by setting reference points for all feature positions and extracting features comprehensively, marker detection is robust against partial feature loss.
Further, as another feature extraction method, for example, there is a method characterized by the luminance value or color difference value of each pixel on the image.

Next, the feature extraction unit 21 assigns a serial number to each feature point.
FIG. 5 shows an example in which serial numbers are assigned to the feature points of the frame image shown in FIG.
As shown in the figure, serial numbers can be assigned in the order of 1, 2, 3, 4,...
Subsequently, the feature extraction unit 21 obtains the coordinates of the feature points. As coordinates, the X and Y axes can be set in the feature space, the distance from the Y axis can be set as the X coordinate, and the distance from the X axis can be set as the Y coordinate.

Then, the feature extraction unit 21 stores the serial numbers and coordinates of these feature points in each storage area of the feature storage unit 22. The feature storage unit 22 can store the serial number, coordinates, and the like as “feature point information” as shown in FIG.
As shown in the figure, the feature point information includes “frame image serial number” (a), “feature point serial number” (b), “feature point x coordinate” (c), and “feature point information”. The “y coordinate of point” (d) can be configured as an item.
The “frame image serial number” indicates a number assigned to the frame image from which the feature points are extracted.
The “feature point serial number” indicates a number assigned to each of a plurality of feature points extracted from one frame image of the feature points.
The “x coordinate of the feature point” indicates the x coordinate of the feature point in the feature space.
The “y-coordinate of the feature point” indicates the y-coordinate of the feature point in the feature space.

In the detection stage, the difference extraction unit 23 extracts feature point information of the detection target image from the input storage area 223 of the feature storage unit 22 and also extracts feature point information of the background image from the background storage area 221 and is located at the same coordinates. A process of deleting feature point information is performed.
FIG. 7 is a diagram schematically illustrating the feature space after the corresponding feature point of the background image is deleted from the feature points of the detection target image.
As shown in the figure, the difference extraction unit 23 can extract only feature points of objects (not limited to target objects) that do not originally exist in the background image.
Then, the feature point information of the feature points extracted by the difference extraction unit 23 is output to the invariant feature conversion means 30.

As shown in FIG. 2, the invariant feature conversion unit 30 includes an invariant feature conversion unit 31 and an invariant feature storage unit 32.
The invariant feature conversion unit 31 acquires an invariant feature by mapping the feature point to the invariant feature space.
First, the invariant feature conversion unit 31 takes out the feature point of the target object from the object storage area 222 of the feature storage unit 22 and maps it to the invariant feature space in the preparation stage. Then, the invariant feature information of the invariant feature obtained by this mapping process is stored in the object invariant storage area 321 of the invariant feature storage unit 32.
The invariant feature conversion unit 31 receives the feature point information output from the difference extraction unit 23 and maps the feature point to the invariant feature space at the detection stage. Then, the invariant feature information of the invariant feature acquired by this mapping process is stored in the difference invariant storage area 322 of the invariant feature storage unit 32.

Here, the process of mapping the feature points to the invariant feature space will be described in detail.
For example, when a point (feature point) is extracted as a characteristic part of an image and a series of position coordinate information in that space (feature space) is used as a graphic feature, a reference point is selected from the feature points on the feature space. By placing this reference point at the reference coordinates of the invariant feature space, and placing all other feature points in the invariant feature space using the same transformation rule, each feature point is invariant. It is mapped to the feature space. Hereinafter, this reference point is referred to as “base”. The number of “bases” can be determined according to a change in the posture of the object in the background video. This number is called a “basic number”.

When the target object is accompanied by a geometric change, that is, when the scene of the camera and the target object rotates and translates relatively and undergoes a posture change such as shear deformation, the base number corresponding to the posture change in advance By mapping the feature point of the target object to the invariant feature space based on the above, the invariant feature amount can be obtained from the positional relationship of the feature point group regardless of the relative positional relationship change.
Specifically, the base number is 1 when the posture change is only parallel movement, the base number is 2 when any of the posture changes of enlargement, reduction or rotation appears, and the base number when shear deformation is applied. The number can be three.

  In the present embodiment, for the sake of simplicity, geometric invariant features when the background is far away will be described. Also, it is easy to expand to invariant features with a higher degree of freedom, such as when the background is not far away.

Hereinafter, a method for mapping a feature point to an invariant feature space using a paper airplane as a target object will be described with reference to FIGS.
First, a method of mapping feature points when the base number is 1 will be described with reference to FIG.
As shown in FIG. 8A, a frame image obtained by capturing only the target object is prepared. That is, the video input unit 11 inputs an image of the target object in advance, and the video storage unit 12 stores this. Each feature point is given a serial number as shown in FIG.

As shown in the figure, the feature extraction unit 21 extracts feature points of the target object.
Here, the invariant feature conversion unit 31 determines one feature point as a base from among the plurality of extracted feature points, and moves so that this base comes to the coordinate (0, 0) of the invariant feature space. The movement amount is obtained, and all other feature points are moved to the invariant feature space by the movement amount.
For example, the feature point of serial number 1 is used as a base, and all feature points are translated so that the feature point of serial number 1 is at the coordinate (0, 0) in the invariant feature space. As a result, the feature points are mapped to the invariant feature space, and the invariant features as shown in FIG.

Then, in this way, one feature point is defined as a base, and as the base is moved to the origin of the invariant feature space, the process of translating all the feature points is performed sequentially with each feature point as a base. Each feature point is mapped to the invariant feature space by performing it every time.
For this reason, at the detection stage, the feature point of the detection target image is mapped to the invariant feature space with a basis number of 1, and this is detected depending on the input image even when the target object is translated. be able to.

Next, a method of mapping feature points when the base number is 2 will be described with reference to FIG.
In this case, the invariant feature conversion unit 31 determines two feature points as bases, moves so that the respective bases are located at two corresponding reference coordinates on the invariant feature space, and all the features are associated therewith. The point is moved to the invariant feature space while maintaining the relative positional relationship. And this movement is performed about each base which consists of the combination of 2 points | pieces which can be selected from all the feature points.
First, as shown in FIGS. 9A to 9B, an image of a target object is input in advance and its feature points are extracted.
Here, for example, when the feature point with serial number 1 is the first basis and the feature point with serial number 2 is the second basis, the first basis is set to the coordinates (0, 0) of the invariant feature space. As the second base moves to the coordinates (1, 0), all other feature points are moved according to the same conversion rule. Thereby, an invariant feature as shown in FIG.

In this way, two feature points are defined as bases, and as these bases are moved to two reference points in the invariant feature space, all the feature points are translated, rotated, or enlarged / reduced. By performing the process every time the combination of each feature point is sequentially determined as a base, each feature point is mapped to the invariant feature space.
Therefore, at the detection stage, the feature point of the detection target image is mapped to the invariant feature space with a basis number of 2 and compared, so that depending on the input image, the target object may be translated, rotated, enlarged / reduced. Even this can be detected.

Next, a feature point mapping method when the basis number is 3 will be described with reference to FIG.
In this case, the invariant feature conversion unit 31 determines three feature points as bases, moves so that each base comes to the corresponding three reference coordinates on the invariant space, and all the feature points are associated therewith. Are moved to the invariant feature space while maintaining the relative positional relationship. And this movement is performed about each base which consists of a combination of 3 points | pieces which can be selected from all the feature points.
First, as shown in FIGS. 10A to 10B, target object images are prepared and feature points are extracted.
Here, for example, when the feature point of serial number 2 is the first basis, the feature point of serial number 3 is the second basis, and the feature point of serial number 1 is the third basis, the first basis is The same transformation rule is moved by moving to the coordinate (0,0) of the invariant feature space, moving the second base to the coordinate (1,0), and moving the third base to the coordinate (0,1). To move all other feature points. Thereby, an invariant feature as shown in FIG.

In this way, three feature points are defined as bases, and as these bases are moved to the three reference points of the invariant feature space, all feature points are translated, rotated, enlarged / reduced, or By performing the process of shear deformation every time a combination of feature points is sequentially determined as a basis, each feature point is mapped to the invariant feature space.
Therefore, at the detection stage, the feature point of the detection target image is mapped to the invariant feature space with a basis number of 3, and the target object is translated, rotated, enlarged / reduced or sheared depending on the input image. This can be detected even in cases.

Thus, a one-to-one linear mapping from the original image space to the invariant feature space can be defined as an affine transformation. When all the feature point groups except the base are mapped to the invariant feature space using the same affine transformation characterized by the base, these feature point groups become invariant regardless of the relative positional relationship between the camera and the scene. In practice, however, it is not always possible to select the same base from the scene. Therefore, base selection is performed from a permutation combination of all three points of the feature point group, and the non-basis feature points for each base are set in the invariant feature space. Need to map.
The reason why these feature points are invariant to geometric deformation (for example, translation, rotation, enlargement / reduction, shear deformation, etc.) of the object is that the base selected from the feature points in the image including other objects. This is because the obtained invariant feature always matches.

The object invariant storage area 321 of the invariant feature storage unit 32 stores invariant feature information of the target object obtained through such invariant feature conversion.
In addition, the difference invariant storage area 322 of the invariant feature storage unit 32 stores invariant feature information based on the difference between the detection target image and the background image obtained by the same invariant feature conversion.
As shown in FIG. 11, “invariant feature information” includes “invariant feature space serial number” (a), “invariant feature serial number” (b), “invariant feature x-coordinate” (c), “ The y coordinate of the invariant feature "(d) can be configured as an item.

“Serial number of invariant feature space” indicates a number assigned to the invariant feature space.
The “invariant feature serial number” indicates a number assigned to each of a plurality of invariant features.
“Invariant feature x-coordinate” indicates the x-coordinate of the invariant feature in the invariant feature space.
“Y coordinate of invariant feature” indicates the y coordinate of the feature point in the invariant feature space.

In this embodiment, the method for mapping the feature points to the invariant feature space is the method shown in FIGS. 8 to 10, but the mapping method is not limited to this method, and bases of four or more points are used. You can also select and map.
In addition to the geometric invariants described above, mapping using invariants of object colors is also possible.
Even if the color of the object is the same, the color of the object is photographed with different colors depending on the light source color existing in the photographing environment. If the influence of the light source color variation can be separated and removed from the image, the actual object color can be obtained. The actual object color obtained may be used as the object color invariant. The specular reflection location is dominated by the influence of the light source color, and the luminance value is likely to be saturated in the light source color component. It may be.

  Other methods for estimating the object color from the image can be found in IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 27, NO. 2, FEBRUARY 2005, pp.178-193 by Robby T. Tan and Katsushi Ikeuchi. `` Separating Reflection Components of Textured Surfaces Using a Single Image '', IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENT, VOL.28, NO by Graham D. Finlayson, Steven D. Hordley, Cheng Lu, and Mark S. Drew .1, JANUARY 2006, pp.59-68, “On the Removal of Shadows from Images”, etc. may be used.

Moreover, a texture can be used as an invariant.
A numerical value or vector obtained by performing numerical calculation on the luminance distribution of the partial region of the image is used as a feature amount. Similar to the graphical invariant, the texture invariant is easily affected by the relative positional relationship between the camera and the object to be photographed. Therefore, a feature amount that is not easily affected is calculated and set as the texture invariant. For example, a feature quantity that is invariant to the distance and zoom between the camera and the object can be implemented by converting the focused partial image into polar coordinates and taking the power spectrum in the radial direction. Further, when the power spectrum is obtained again in the azimuth direction with respect to the power spectrum, the feature amount is invariable with respect to the rotation around the optical axis of the camera. In addition, `` Log-Polar Wavelet Energy Signatures for Rotation and Scale Invariant Texture Classification '' described in IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 25, NO. 5, MAY 2003 by Chi-Man Pun and Moon-Chuen Lee A method may be used.

  Further, as for geometric invariants, other geometric invariants as described in “Multiple View Geometry in Computer Vision” by Richard Hartley and Andrew Zisserman may be used. When observing the same scene with a plurality of cameras, it is possible to obtain information on the relative positional relationship in the distance or depth direction by the method described in the same document, but in this case, there are four points that are not on the same plane. Is selected as the basis, and the invariant feature space is three-dimensional, a three-dimensional geometric invariant can be created. In this case, one of the four base points selected from the feature point group is the origin of the invariant space, and the other base feature points are the position coordinates (1, 0, 0) and (0, 1) in the invariant space. , 0), (0, 0, 1), a transformation map is obtained, and other features are mapped to the invariant space using this transformation map.

As shown in FIG. 2, the comparison unit 40 includes a collation unit 41 and a determination unit 42.
The collation unit 41 extracts invariant feature information from the object invariant storage area 321 and the difference invariant storage area 322 of the invariant feature storage unit 32, and collates the arrangement of the invariant features.
That is, the coordinates of the “invariant feature based on the difference between the detection target image and the background image” are collated with the coordinates of the “invariant feature of the target object”.
And if the matching of the arrangement | positioning (coordinate) of a corresponding invariant feature is recognized as a result of collation, it will determine with the determination part 42 having detected the target object.
“Arrangement (coordinates) match” is accepted not only when all the arrangements (coordinates) of the corresponding invariant features are coincident but also when a predetermined number or more of the arrangements (coordinates) are confirmed. You can also

Here, a specific method of object detection according to the present embodiment will be described with an example.
First, the case where the target object is included in the detection target image will be described with reference to FIG.
First, the video input unit 11 inputs a detection target image shown in FIG.
Next, the feature extraction unit 21 extracts feature points from the detection target image, and the difference extraction unit 23 extracts a difference between the feature points of the detection target image and the background image (FIG. 12B).
Subsequently, the invariant feature conversion unit 31 maps the difference feature points to the invariant feature space to obtain the invariant feature group shown in FIG.
Then, the collation unit 41 collates the coordinates A in the invariant feature space of the invariant feature with the coordinates B in the invariant feature space of the invariant feature of the target object.
As a result, as shown in the figure, since the arrangement of both invariant features matches, the determination unit 42 can determine that the target object exists in the detection target image.

Moreover, even if the direction and size of the target object change, it can be detected in the same manner.
This is because, as described above, by selecting a plurality of bases for feature points (in this case, the number of bases is 2) and performing invariant feature conversion, geometric such as enlargement / reduction or rotation as well as translation is performed. This is because an invariant pattern with respect to a local change is acquired in the invariant feature space in advance.
For this reason, as shown in FIG. 13, even if the orientation and size of the target object change, this can be reliably detected.
Note that the invariant feature space A and the invariant feature space B shown in FIG. 13C schematically show the arrangement of the invariant features when the feature points of the original image are mapped as the basis number 2, respectively. is there. However, in these figures, for simplicity, the number of feature points to be mapped is omitted.

Further, as shown in FIG. 14, an object other than the target object (in the figure, a star-shaped object) may appear in the input detection target image.
In this case as well, the feature extraction unit 21 extracts feature points from the detection target image (FIG. 14A), and the difference extraction unit 23 extracts feature point differences from the background image (FIG. 14B). ).
Then, the invariant feature A based on the difference is compared with the invariant feature B of the target object.
However, in this case, as shown in FIG. 14C, the arrangement of both invariant features does not match.
Further, as shown in FIG. 15, there is a case where no object appears in the input detection target image.
Also in this case, the feature extraction unit 21 extracts feature points from the detection target image (FIG. 15A), and the difference extraction unit 23 extracts feature point differences from the background image (FIG. 15B).
Then, the invariant feature A based on the difference is compared with the invariant feature B of the target object, but these do not match as shown in FIG.
Therefore, in such a case, the determination unit 42 does not perform a determination to affirm the detection of the target object.

  There are cases where the target object (paper airplane) and an object other than the target object (star-shaped object) appear together in the input detection target image. In this case as well, verification of the invariant features of the target object is not possible. It can be detected because it matches.

Next, the operation (object detection method) of the object detection apparatus of the present embodiment will be described with reference to FIGS.
FIG. 16 is a flowchart illustrating a procedure in the preparation stage of the object detection method.
FIG. 17 is a flowchart illustrating a procedure in the detection stage of the object detection method.

<Procedure in preparation stage>
As shown in FIG. 16, in the object detection device 1a of the present embodiment, the video input unit 11 of the video input means 10 inputs a background image and a target object image (S101). The input image is stored in the video storage unit 12.
Next, the feature extraction unit 21 of the feature extraction unit 20 takes out an image from the video storage unit 12 and extracts feature points (S102). That is, the feature point of the background image and the feature point of the target object are extracted.
The feature point information of the background image is stored in the background storage area 221 of the feature storage unit 22 (S103).
The feature point information of the target object is stored in the object storage area 222 of the feature storage unit 22.

Subsequently, the invariant feature conversion unit 31 of the invariant feature conversion unit 30 extracts the feature point information of the target object from the object storage area 222, and maps the feature points into the invariant feature space (S104). Thereby, the feature points of the target object are converted into invariant features.
Then, the invariant feature information of the target object is stored in the object invariant storage area 321 of the invariant feature storage unit 32 (S105).

<Procedure in the detection stage>
As shown in FIG. 17, in the object detection device 1a of the present embodiment, the video input unit 11 of the video input unit 10 inputs a detection target image (S201). The input detection target image is stored in the video storage unit 12.
Next, the feature extraction unit 21 of the feature extraction unit 20 extracts the detection target image from the video storage unit 12 and extracts feature points (S202). The feature point information of the extracted detection target image is stored in the input storage area 223 of the feature storage unit 22.
Subsequently, the difference extraction unit 23 extracts the feature points of the detection target image from the input storage area 223 and also extracts the feature points of the background image stored in the background storage area 221 in the preparation stage.
Then, the difference extraction unit 23 deletes the corresponding feature point of the background image from the feature point of the detection target image, and obtains the difference of the feature point (S203).

Next, the invariant feature conversion unit 31 of the invariant feature conversion unit 30 maps the feature points extracted as the difference to the invariant feature space to acquire an invariant feature (S204). The obtained invariant feature information of the invariant feature is stored in the difference invariant storage area 322 of the invariant feature storage unit 32.
Then, matching with the invariant feature of the target object is performed (S205).
Specifically, the matching unit 41 of the comparison unit 40 extracts the invariant feature information from the difference invariant storage area 322 and extracts the invariant feature information from the object invariant storage area 321. Then, based on the extracted invariant feature information, the coordinates of these invariant features in the invariant feature space are collated.

As a result of the collation, when the coordinates of both invariant features match (S205: match), the determination unit 42 determines that the target object has been detected (S206). On the other hand, if the coordinates of the invariant features do not match (S205: mismatch), the target object is not detected. That is, in this case, the process proceeds to S207 without performing any special processing.
If the object detection is further continued (S207: YES), the process returns to S201 and the same processing is repeated. If the object detection is not continued (S207: NO), the object detection is terminated.

As described above, according to the object detection apparatus and the object detection method of the present embodiment, when the background feature is excluded from the features of the detection target image input at the time of detection and some feature points remain, It is assumed that the target object may have appeared, and the feature point is mapped to the invariant feature space, and the invariant feature obtained by mapping the feature point of the target object and the feature point of the target object to the invariant feature space Are collated.
When the matching of the coordinates of both invariant features is confirmed by this collation, detection of the target object can be recognized.
For this reason, the target object can be detected regardless of the marker, and the object detection can be accurately performed in accordance with the geometric deformation of the target object or the background image.
Therefore, it is possible to realize an object detection device excellent in convenience and reliability at a low cost.

[Second Embodiment of Object Detection Apparatus and Object Detection Method]
Next, a second embodiment of the object detection device and the object detection method of the present invention will be described with reference to FIG.
FIG. 18 is a block diagram showing a detailed configuration of the object detection apparatus of the present embodiment.
This embodiment is characterized by the invariant feature conversion unit 30 as compared to the first embodiment.
Specifically, in the object detection device 1b of the present embodiment, the invariant feature storage unit 32 of the invariant feature conversion unit 30 includes an object frequency storage area 323 and a difference frequency storage area 324.

That is, in the first embodiment, the detection of the target object is determined based on whether or not the arrangement (coordinates) of the invariant features match, whereas in the present embodiment, depending on whether or not the frequency of the invariant features matches. The detection of the target object is determined.
Other components are the same as those in the first embodiment.
Therefore, in FIG. 18, the same components as those in FIG. 2 are denoted by the same reference numerals, and detailed description thereof is omitted.

The invariant feature storage unit 32 of the invariant feature conversion unit 30 includes an object frequency storage area 323 and a difference frequency storage area 324 as shown in FIG.
The object frequency storage area 323 stores the frequency distribution based on the invariant feature of the target object obtained in the preparation stage.
Specifically, the invariant feature conversion unit 31 extracts invariant features (invariant feature information) of the target object stored in the object invariant storage area 321 in the preparation stage.
FIG. 19A is a schematic diagram showing the invariant feature of the target object extracted from the object invariant storage area 321 in the invariant feature space.

Next, the invariant feature conversion unit 31 attaches a grid-like mesh to the invariant feature space in which the invariant features exist, and divides it into a plurality of sections.
As shown in FIG. 19B, this section may be a partial area of the invariant feature space or the entire area. The calculation load can be reduced by narrowing down the region to a part, while the accuracy can be improved by targeting the entire region.
Subsequently, as shown in FIG. 19C, the invariant feature conversion unit 31 obtains the number of invariant features (frequency distribution) for each section.
Then, the invariant feature conversion unit 31 stores the obtained frequency distribution of the invariant features of the target object in the object frequency storage area 323.

The difference frequency storage area 324 stores the frequency distribution of invariant features based on the difference between the feature points of the detection target image and the background image obtained in the detection stage.
Specifically, the invariant feature conversion unit 31 takes out the invariant features stored in the difference storage area 322 and obtains the frequency distribution thereof. That is, the feature point after subtracting the corresponding feature point of the background image from the feature point of the detection target image is mapped to the invariant feature space, and the frequency distribution is obtained based on the invariant feature obtained thereby.
Then, the invariant feature conversion unit 31 stores the obtained invariant feature frequency distribution information in the difference frequency storage area 324.
Note that the “frequency distribution information” is configured by, for example, the position coordinates of each section in the invariant feature space and the number of invariant features arranged in each section.

In this way, the frequency distributions of the invariant features stored in the object frequency storage area 323 and the difference frequency storage area 324 are collated with each other, and if a match is recognized, it is determined that the target object has been detected.
Specifically, if the coincidence of the number of invariant features arranged in the corresponding section of both frequency distributions (that is, the coincidence of frequency distributions) is recognized, it is determined that the detection target object has been detected.
“Frequency distribution match” can be recognized not only when the frequencies of all the sections of the corresponding invariant feature match but also when the frequency matches in a predetermined number or more of the sections can be confirmed.

Here, a specific method of object detection in the present embodiment will be described with an example.
First, a case where a target object is included in a detection target image will be described with reference to FIG.
First, the video input unit 11 inputs a detection target image shown in FIG.
Next, the feature extraction unit 21 extracts feature points from the detection target image, and the difference extraction unit 23 extracts the difference between the feature points of the detection target image and the background image (FIG. 20B).
Subsequently, the invariant feature conversion unit 31 maps the difference feature points to the invariant feature space to obtain the invariant feature group shown in FIG.

Next, the invariant feature conversion unit 31 obtains the frequency distribution shown in FIG. 20D by dividing the invariant feature space in which the invariant feature exists and obtaining the number of invariant features.
The collation unit 41 collates the frequency distribution A of the invariant features with the frequency distribution B of the invariant features of the target object.
As a result, as shown in the figure, since the invariant feature numbers in the sections of both invariant features coincide, the determination unit 42 can determine that the target object exists in the detection target image.

In the present embodiment, even if the orientation or size of the target object changes, this can be reliably detected.
This is the same reason as described with reference to FIG. 13 in the first embodiment.
However, in the case of the first embodiment, complete matching of the coordinate values is the principle, but in the case of this embodiment, since the section based on the coordinates is a reference for comparison, the allowance for matching determination is set wider than in the first embodiment. can do.

In addition, as shown in FIG. 22, an object other than the target object (in the figure, a star-shaped object) may appear in the input detection target image.
In this case as well, the feature extraction unit 21 extracts feature points from the detection target image (FIG. 22A), and the difference extraction unit 23 extracts feature point differences from the background image (FIG. 22B). ).
Next, the invariant feature conversion unit 31 obtains an invariant feature by mapping the difference feature point to the invariant feature space (FIG. 22C).
Subsequently, the invariant feature conversion unit 31 obtains the frequency distribution shown in FIG. 22D by dividing the invariant feature space in which the invariant feature exists and obtaining the number of invariant features.
Then, the frequency distribution A of the invariant feature based on the difference is collated with the frequency distribution B of the invariant feature of the target object.
However, in this case, as shown in the figure, the frequency distributions do not coincide with both invariant features.

Further, as shown in FIG. 23, there is a case where no object appears in the input detection target image.
Also in this case, the feature extraction unit 21 extracts feature points from the detection target image (FIG. 23 (a)), and the difference extraction unit 23 extracts feature point differences from the background image (FIG. 23 (b)).
Next, the invariant feature converting unit 31 maps the difference feature points to the invariant feature space to obtain an invariant feature (FIG. 23C).
Subsequently, the invariant feature conversion unit 31 obtains the frequency distribution shown in FIG. 23D by dividing the invariant feature space in which the invariant feature exists and obtaining the number of invariant features.
Then, the invariant feature frequency distribution A based on the difference is collated with the invariant feature frequency distribution B of the target object. However, as shown in FIG.
Therefore, in such a case, the determination unit 42 does not perform a determination to affirm the detection of the target object.

In addition, there are cases where the target object (paper airplane) and the object other than the target object (star-shaped object) appear together in the input detection target image. In this case as well, the frequency distribution of the invariant features of the target object This can be detected by checking.
In this case, even if the frequency is not a perfect match, if the match is confirmed in a predetermined number or more sections of the frequency distribution of the invariant feature, or if the corresponding frequency number is approximate, the detection of the target object is positively accepted. Can do.
Moreover, the accuracy of object detection can be adjusted by changing the section width of the frequency distribution in the invariant feature space.

Next, the operation (object detection method) of the object detection apparatus of the present embodiment will be described with reference to FIGS.
FIG. 24 is a flowchart illustrating a procedure in the preparation stage of the object detection method.
FIG. 25 is a flowchart illustrating a procedure in the detection stage of the object detection method.

<Procedure in preparation stage>
As shown in FIG. 24, in the object detection device 1b of the present embodiment, the video input unit 11 of the video input unit 10 inputs the background image and the target object image (S301). The input image is stored in the video storage unit 12.
Next, the feature extraction unit 21 of the feature extraction unit 20 takes out an image from the video storage unit 12 and extracts feature points (S302). That is, the feature point of the background image and the feature point of the target object are extracted.
The feature point information of the background image is stored in the background storage area 221 of the feature storage unit 32 (S303).
The feature point information of the target object is stored in the object storage area 222 of the feature storage unit 22.

Subsequently, the invariant feature conversion unit 31 of the invariant feature conversion unit 30 extracts the feature point of the target object from the object storage area 222 and maps it to the invariant feature space (S304).
Next, the invariant feature information of the target object is stored in the object invariant storage area 321 of the invariant feature storage unit 32 (S305).
Then, the invariant feature conversion unit 31 extracts the invariant feature of the target object from the object invariant storage region 321, obtains its frequency distribution, and stores the frequency distribution information in the object frequency storage region 323 of the invariant feature storage unit 32 (S 306). .

<Procedure in the detection stage>
As shown in FIG. 25, in the object detection device 1b of the present embodiment, the video input unit 11 of the video input unit 10 inputs a detection target image (S401). The input detection target image is stored in the video storage unit 12.
Next, the feature extraction unit 21 of the feature extraction unit 20 extracts the detection target image from the video storage unit 12 and extracts feature points (S402). The feature point information of the extracted detection target image is stored in the input storage area 223 of the feature storage unit 22.
Subsequently, the difference extraction unit 23 extracts the feature points of the detection target image from the input storage area 223 and also extracts the feature points of the background image stored in the background storage area 221 in the preparation stage.
Then, the difference extraction unit 23 deletes the corresponding feature point of the background image from the feature point of the detection target image, and obtains the difference of the feature point (S403).

Next, the invariant feature converting unit 31 of the invariant feature converting unit 30 maps the feature points extracted as the difference to the invariant feature space to acquire the invariant feature (S404). The obtained invariant feature information of the invariant feature is stored in the difference invariant storage area 322 of the invariant feature storage unit 32.
Subsequently, the invariant feature conversion unit 31 extracts the invariant features from the difference invariant storage area 322, obtains the frequency distribution, and stores the frequency distribution information in the difference frequency storage area 326 (S405).
And it collates with the frequency distribution of the invariant feature of a target object (S406).
Specifically, the matching unit 41 of the comparison unit 40 extracts the invariant feature frequency distribution information from the difference frequency storage area 326 and extracts the invariant feature frequency distribution information from the object frequency storage area 323. Then, based on the extracted frequency distribution information of the invariant features, it is verified whether or not these frequency distributions match.

As a result of the collation, when both frequency distributions match (S406: match), it is determined that the determination unit 42 has detected the target object (S407). On the other hand, when both frequency distributions do not match (S406: mismatch), the target object is not detected. That is, in this case, the process proceeds to S408 without performing any special processing.
If the object detection is further continued (S408: YES), the process returns to S401 and the same processing is repeated. If the object detection is not continued (S408: NO), the object detection is terminated.

As described above, according to the object detection device and the object detection method of the present embodiment, it is possible to detect a target object without using a marker as in the first embodiment.
In particular, in this embodiment, the detection determination of the target object is performed based on whether or not the number of invariant features appearing in the invariant feature space matches each section.
For this reason, compared with 1st embodiment which calculates | requires a coordinate coincidence as a detection condition of a target object, tolerance can be given.
For this reason, for example, it is possible to invalidate a minute shift of invariant feature coordinates caused by image disturbance, noise, or the like.
In addition, regarding the allowable range of coincidence, it is possible to freely adjust the accuracy of object detection by changing the section width in the invariant feature space.

[Third Embodiment of Object Detection Device and Object Detection Method]
Next, a third embodiment of the object detection device and the object detection method of the present invention will be described with reference to FIGS.
FIG. 26 is a block diagram illustrating a configuration of the object detection device of the present embodiment.
FIG. 27 is a block diagram showing a detailed configuration of the object detection apparatus of the present embodiment.
As shown in FIG. 26, the object detection apparatus 1c of this embodiment is different from the first embodiment and the second embodiment described above in that it includes a unique feature selection unit 50.

Specifically, a part where the invariant feature did not appear from the invariant features in the invariant feature space derived from the feature points of the background (or a part where the number of invariant features is a predetermined number or less) is selected as a singular feature, If an invariant feature appears in the specific feature portion at the detection stage, the object detection is more positively determined that the target object is likely to appear.
That is, the present embodiment aims to further increase the determination accuracy in object detection of the object detection device of the above-described embodiment. Other components are the same as those in the first embodiment.
Accordingly, in FIGS. 26 and 27, the same components as those in FIGS. 1 and 2 are denoted by the same reference numerals, and detailed description thereof is omitted.

In the object detection device 1c of this embodiment, as shown in FIG. 27, the invariant feature storage unit 32 of the invariant feature conversion means 30 includes a background invariant storage region 325, a background frequency storage region 326, and an input frequency storage region 327. .
The background invariant storage area 325 stores invariant feature information obtained by mapping the feature points of the background image to the invariant feature space in the preparation stage.
The background frequency storage area 326 stores frequency distribution information based on the invariant features of the background image stored in the background invariant storage area 325 in the preparation stage.
The input frequency storage area 327 stores the invariant feature frequency distribution obtained by mapping the feature points of the detection target image to the invariant feature space in the detection stage.
Then, the frequency distribution information of the invariant feature acquired by this mapping is stored in the background frequency storage area 325 of the invariant feature storage unit 32.

As illustrated in FIG. 27, the unique feature selection unit 50 includes a unique feature selection unit 51 and a unique feature storage unit 52.
The singular feature selection unit 51 extracts the frequency distribution information of the invariant features related to the background image from the background frequency storage area 325 of the invariant feature storage unit 32 in the preparation stage.
Then, the singular feature selection unit 51 analyzes the extracted frequency distribution information of the invariant features, and selects a section having the invariant feature number equal to or less than a predetermined number as the singular feature.
FIG. 28 is a schematic diagram showing unique features derived from a background image.
As shown in the figure, a section having an invariant feature number of “0” can be selected as a unique feature.
As described above, the singular feature selection unit 51 can select a singular feature from the invariant features based on the portion of the feature space where the feature point group extracted from the predetermined image does not appear.

  This selection of singular features can be identified as the problem of finding a large blank from the distribution of invariant features in the invariant feature space. For example, “An algorithm for Finding Maximal” published in the 2003 Proc. An algorithm such as “Whitespace Rectangles at Arbitrary Orientations for Document Layout Analysis” may be used to extract a large blank area, or the center of the obtained rectangular area that does not include invariant features may be used as a singular feature.

  As another method, the invariant feature space is quantized with a mesh (partition) of a specific size to generate a one-dimensional or multidimensional histogram, and the center of the section where the occurrence frequency of the invariant feature is 0 is a unique feature. And so on. If there is no partition with a frequency of 0, the partition width may be reduced, a histogram may be taken, and if a partition with a frequency of 0 appears, a unique feature may be selected from the partition at this time. If a mesh with a frequency of 0 is not found, the histogram may be thresholded with a default value, and unique features may be selected from meshes that are less than or equal to the specified value.

The unique feature storage unit 50 stores information on unique features related to the background image selected by the unique feature selection unit 51. The unique feature information corresponds to coordinates indicating the position of each unique feature.
Then, in the detection stage, the matching unit 41 extracts the frequency distribution information of the invariant features of the detection target image from the input frequency storage area 326 of the invariant feature storage unit 32 and the uniqueness of the background image stored in advance in the unique feature storage unit 52. Feature information is extracted and collated.

As a result, the determination unit 42 can recognize that some object has been detected when one or more invariant features appear in the unique feature portion of the background image.
Thereby, when the detection of the target object is recognized to some extent by another determination method, the determination can be recognized more positively.
For this reason, a target object can be detected more correctly by combining with the object detection method according to the above-described embodiment.
That is, an object detection device with extremely excellent detection accuracy can be realized by adding a configuration unique to the present embodiment to the object detection devices 1a and 1b of the above-described embodiment.

Next, the operation (object detection method) of the object detection apparatus of this embodiment will be described with reference to FIGS. 29 and 30. FIG.
FIG. 29 is a flowchart illustrating a procedure in a preparation stage of the object detection method.
FIG. 30 is a flowchart illustrating a procedure in the detection stage of the object detection method.

<Procedure in preparation stage>
As shown in FIG. 29, in the object detection device 1c of the present embodiment, the video input unit 11 of the video input unit 10 inputs a background image and an image of the target object (S501). The input image is stored in the video storage unit 12.
The feature extraction unit 21 of the feature extraction unit 20 takes out an image from the video storage unit 12 and extracts feature points (S502). That is, the feature point of the background image and the feature point of the target object are extracted.
The feature point information of the background image is stored in the background storage area 221 of the feature storage unit 32.
Also, the feature point information of the target object is stored in the object storage area 222 of the feature storage unit 32 (S503).

Next, the invariant feature conversion unit 31 of the invariant feature conversion means 30 takes out the feature points of the target object from the object storage area 222 and maps the feature points to the invariant feature space (S504). Thereby, the feature points of the target object are converted into invariant features.
The invariant feature information of the target object is stored in the object invariant storage area 321 of the invariant feature storage unit 32 (S505).

The invariant feature conversion unit 31 extracts the invariant features of the target object from the object frequency storage area 321, obtains the frequency distribution, and stores the information in the object frequency storage area 323 of the invariant feature storage unit 32 (S506).
Further, the invariant feature conversion unit 31 extracts the feature points of the background image from the background storage area 221 of the feature storage unit 22 and maps them to the invariant feature space (S507). Thereby, the feature points of the background image are converted into invariant features, and the invariant feature information is stored in the background invariant storage area 325 of the invariant feature storage unit 32.
Next, the invariant feature conversion unit 31 obtains the frequency distribution of the invariant features of the background image and stores the information in the background frequency storage area 325 (S508).

  Then, the unique feature selection unit 51 extracts the frequency distribution of the invariant features of the background image from the background frequency storage area 325, selects the unique features, and stores the unique feature information in the unique feature storage unit 52 (S509).

<Procedure in the detection stage>
As shown in FIG. 30, in the object detection device 1c of this embodiment, the video input unit 11 of the video input unit 10 inputs a detection target image (S601). The input image is stored in the video storage unit 12.
Next, the feature extraction unit 21 of the feature extraction unit 20 extracts the detection target image from the video storage unit 12 and extracts feature points (S602). The feature point information of the extracted feature points is stored in the input storage area 223 of the feature storage unit 22.
Subsequently, the difference extraction unit 23 extracts the feature points of the detection target image from the input storage area 223 and also extracts the feature points of the background image stored in the background storage area 221 in the preparation stage.
Then, the difference extraction unit 23 deletes the corresponding feature point of the background image from the feature point of the detection target image, and obtains the difference of the feature point (S603).

The invariant feature conversion unit 31 of the invariant feature conversion unit 30 maps the feature points extracted as the difference to the invariant feature space to acquire an invariant feature (S604). The obtained invariant feature information of the invariant feature is stored in the difference invariant storage area 322 of the invariant feature storage unit 32.
And it collates with the frequency distribution of the invariant feature of a target object (S605).
Specifically, the matching unit 41 of the comparison unit 40 extracts the invariant feature frequency distribution information from the difference frequency storage area 326 and extracts the invariant feature frequency distribution information from the object frequency storage area 323. Then, based on the extracted frequency distribution information of the invariant features, it is verified whether or not these frequency distributions match.

As a result of the matching, if both frequency distributions match (S605: match), a unique feature is further checked (S606).
Specifically, the collation unit 41 extracts the unique feature information of the background image stored in the unique feature storage unit 52 and also extracts the frequency information of the invariant feature related to the detection target image stored in the input frequency storage region 327.
If one or more invariant features of the detection target image are detected in the specific feature portion of the background image, the determination unit 42 determines that the target object has been detected (S607).
On the other hand, if the collation does not match in S605 or S606, the target object is not detected, that is, in this case, the process proceeds to S608 without performing any special processing.
If further object detection is to be continued (S608: YES), the process returns to S601 and the same processing is repeated. If object detection is not continued (S608: NO), the object detection is terminated.

As described above, according to the object detection device and the object detection method of the present embodiment, a portion (or an invariant feature) in which an invariant feature did not appear among invariant features in the invariant feature space derived from the feature points of the background. If the invariant feature appears in the part of the singular feature in the detection stage, the object detection is more positive because the target object is likely to appear. Judgment is made.
Therefore, the determination accuracy in object detection can be further increased.

[Fourth Embodiment of Object Detection Apparatus and Object Detection Method]
Next, a fourth embodiment of the object detection device and the object detection method of the present invention will be described with reference to FIG.
FIG. 31 is a block diagram showing a detailed configuration of the object detection apparatus of the present embodiment.
As shown in FIG. 31, the object detection device 1 d of this embodiment is characterized in that the comparison unit 40 includes a number determination unit 43.
That is, the object is to detect the number of objects in addition to the detection of the objects themselves. Other components are the same as those in the first embodiment.
Therefore, in FIG. 31, the same components as those in FIG. 2 are denoted by the same reference numerals, and detailed description thereof is omitted.

In the object detection device 1d of the present embodiment, the comparison unit 40 includes a number determination unit 43, as shown in FIG.
Hereinafter, a specific method for determining the number of objects according to the present embodiment will be described with reference to FIG.
Here, the video input unit 11 inputs a frame image including a plurality of target objects (FIG. 32A) as a detection target image and stores it in the video storage unit 12.
The feature extraction unit 21 extracts feature points from the detection target image, and the difference extraction unit 23 extracts the difference between the feature points of the detection target image and the background image (FIG. 32B).
Subsequently, the invariant feature conversion unit 31 maps the difference feature points to the invariant feature space to obtain the invariant feature group shown in FIG. In the case of this embodiment, invariant feature conversion is performed with a base as one point.

Next, as shown in FIG. 32D, the invariant feature conversion unit 31 divides the invariant feature space in which these invariant features exist into meshes, and obtains a frequency distribution for each section.
Then, the number determination unit 43 of the comparison unit 40 extracts the frequency distribution of the invariant features of the target object stored in the object frequency storage area 323 of the invariant feature storage unit 32 and performs collation.
However, this collation is performed in the range of a certain region centered on the origin of the invariant feature space. Specifically, the feature point information of two points farthest from each feature point extracted from a single target object is extracted, and the maximum distance in the x-axis direction and the maximum distance in the y-axis direction between these two points are obtained. Each distance is defined as a collation range in the ± direction of the x-axis and a collation range in the ± direction of the y-axis direction with respect to the origin of the invariant feature space.
As a result, as shown in the figure, the number determination unit 43 recognizes that the number of invariant features for each section related to the detection target image is twice the number of invariant features in the corresponding section of the target object. Can do.
In such a case, the number determination unit 43 can determine that two target objects have been detected.

In the present embodiment, an example in which there are two target objects has been described. However, the number is not limited to this number, and any number of one or more can be detected.
Further, when a plurality of target objects appear in the detection target image, it is a condition that the target objects are separated from each other by a certain distance. Specifically, the condition is that the target object is separated by the distance between the two most distant feature points among the feature points extracted from the single target object.
As a result, it is possible to eliminate the number of errors caused by overlapping invariant features in the invariant feature space, and to accurately detect the number of target objects.

Next, the operation (object detection method) of the object detection apparatus of this embodiment will be described with reference to FIG.
FIG. 33 is a flowchart showing the procedure of the object detection method of the present embodiment.
In addition, about the process of the preparation stage in this embodiment, since it is the same as that of above-mentioned 2nd embodiment or 3rd embodiment, detailed description is abbreviate | omitted.

As shown in FIG. 33, in the object detection device 1d of the present embodiment, the video input unit 11 of the video input unit 10 inputs a detection target image (S701). The input detection target image is stored in the video storage unit 12.
Next, the feature extraction unit 21 of the feature extraction unit 20 extracts the detection target image from the video storage unit 12 and extracts feature points (S702). The feature point information of the extracted detection target image is stored in the input storage area 223 of the feature storage unit 22.
Subsequently, the difference extraction unit 23 extracts the feature points of the detection target image from the input storage area 223 and also extracts the feature points of the background image stored in the background storage area 221 in the preparation stage.
Then, the difference extraction unit 23 deletes the corresponding feature point of the background image from the feature point of the detection target image, and obtains the difference of the feature point (S703).

Next, the invariant feature converting unit 31 of the invariant feature converting unit 30 maps the feature points extracted as the difference to the invariant feature space to acquire invariant features (S704). The obtained invariant feature information of the invariant feature is stored in the difference invariant storage area 322 of the invariant feature storage unit 32.
And it collates with the frequency distribution of the invariant feature of a target object (S705).
Specifically, the matching unit 41 of the comparison unit 40 extracts the invariant feature frequency distribution information from the difference frequency storage area 326 and extracts the invariant feature frequency distribution information from the object frequency storage area 323. Then, based on the extracted frequency distribution information of the invariant features, it is verified whether or not these frequency distributions match.

As a result of the collation, when both frequency distributions match (S705: YES), the determination unit 42 determines that the target object has been detected (S706). However, in this case, it is determined that one target object has been detected.
On the other hand, if the collation does not match (S705: NO), the frequency distribution of the invariant features of the target object is multiplied by an integer to perform collation (S707).
Specifically, the number determination unit 43 extracts the frequency distribution of the invariant features of the target object stored in the object frequency storage area 323, and repeatedly performs the collation while multiplying the frequency of each section by an integral multiple within a predetermined range. As a range of integers used in this collation, for example, the minimum value can be 2 (or 1), and the maximum value can be in the range of the number of feature points that appear as differences.

As a result, when the frequency distributions match (S707: match), the number determination unit 43 determines that the integer number of target objects used at that time have been detected (S708).
On the other hand, if the collation does not match in S707 (S707: mismatch), the target object is not detected. That is, the process proceeds to S709 without performing any special processing.
If further object detection is to be continued (S709: YES), the process returns to S701 and the same processing is repeated. If object detection is not continued (S709: NO), the object detection is terminated.

As described above, according to the object detection device and the object detection method of the present embodiment, not only the target object but also the number thereof can be detected.
For this reason, it is possible to realize an object detection device that not only has the same operations and effects as the above-described embodiment, but is also more convenient.

[Object detection program]
Next, the object detection program will be described.
The object detection function of the computer (object detection device) in each of the above embodiments is realized by an object detection program stored in a storage unit (for example, a ROM (Read Only Memory) or a hard disk).

An object detection program is read by a computer control means (CPU (Central Processing Unit), etc.) to send commands to the components of the computer, and predetermined processing, for example, video input processing of the object detection device, feature extraction processing, Invariant feature conversion processing, unique feature selection processing, comparison processing, etc. are performed.
Thus, the object detection function is realized by cooperation between the object detection program that is software and each component of the computer (object detection device) that is hardware resource.

The object detection program for realizing the object detection function is stored in a computer ROM, hard disk, or the like, or may be stored in a computer-readable recording medium, for example, an external storage device or a portable recording medium. it can.
The external storage device is a memory expansion device that incorporates a recording medium such as a CD-ROM (Compact Disc-Read Only Memory) and is externally connected to the object detection device. On the other hand, the portable recording medium is a recording medium that can be mounted on a recording medium driving device (drive device) and is portable, and refers to, for example, a flexible disk, a memory card, a magneto-optical disk, and the like.

Then, the program recorded on the recording medium is loaded into a RAM (Random Access Memory) or the like of the computer and executed by the CPU (control means). By this execution, the function of the object detection device of each embodiment described above is realized.
Further, when the object detection program is loaded by the computer, the object detection program held by another computer can be downloaded to its own RAM or external storage device using a communication line. The downloaded object detection program is also executed by the CPU, and realizes the object detection function of the object detection device of each of the above embodiments.

As described above, according to the object detection device, the object detection method, and the object detection program of the present embodiment, it is possible to detect that a desired object exists in the space regardless of the marker.
In addition, object detection corresponding to the posture change of the target object and distortion of the input image is also possible.
Furthermore, it is possible to add a configuration that further increases the detection of the target object.
It is also possible to detect a plurality of target objects.

The embodiments of the object detection device, the object detection method, and the object detection program of the present invention have been described above. However, the object detection device, the object detection method, and the object detection program according to the present invention are limited to the above-described embodiments. However, it goes without saying that various modifications can be made within the scope of the present invention.
For example, in each of the above-described embodiments, the feature extraction unit 20 has the feature storage unit 23 and the invariant feature conversion unit 30 has the invariant feature storage unit 32. May be included, or the storage unit may be independent. Further, the storage unit may be realized by an external storage device.

  Since the present invention relates to object detection, the present invention can be used for apparatuses and devices that detect objects, for example, image management such as article management and physical security, robot vision, mixed reality UI, and content generation applications. .

DESCRIPTION OF SYMBOLS 1 Object detection apparatus 10 Image | video input means 20 Feature extraction means 22 Feature memory | storage part 23 Difference extraction part 30 Invariant feature conversion means 32 Invariant feature storage part 40 Comparison means 43 Number determination part 50 Singular feature selection means

Claims (5)

A video input unit for inputting a detection target image;
A feature extraction unit for extracting feature points from the input detection target image;
An invariant feature converter that represents the extracted feature points in an invariant feature space;
A feature storage unit that stores feature points of a background image that does not include a target object;
An invariant feature storage unit for storing an invariant feature arrangement in which feature points of the target object are represented in an invariant feature space;
All or a predetermined number or more of the invariant feature arrangements obtained by subtracting the corresponding feature points of the background image from the feature points of the detection target image in the invariant feature space are the feature points of the target object. Comparing means for determining that the target object has been detected when it matches the arrangement of the invariant features represented in the random feature space ;
In the case of comprising a singular feature selection means for selecting, as a singular feature, a portion having a frequency equal to or lower than a predetermined value in the frequency distribution of the invariant feature representing the feature point of the background image in the invariant feature space,
The comparison means includes
It is determined that the target object has been detected when an invariant feature represented by subtracting the corresponding feature point of the background image from the feature point of the detection target image in the invariant feature space appears in the portion of the singular feature. An object detection apparatus characterized by: The invariant feature storage unit includes:
  Storing a frequency distribution of invariant features representing feature points of the target object in an invariant feature space;
  The comparison means includes
  All or a predetermined number or more of the frequency distributions of invariant features represented by subtracting the corresponding feature points of the background image from the feature points of the detection target image in the invariant feature space are the feature points of the target object. It is determined that the target object has been detected if it matches the frequency distribution of the invariant features represented in the invariant feature space.
The object detection apparatus according to claim 1. The comparison means includes
  All or a part of the frequency distribution in the predetermined region of the invariant feature represented by subtracting the corresponding feature point of the background image from the feature point of the detection target image in the invariant feature space is the feature of the target object. If the frequency of invariant features representing points in the invariant space matches the frequency distribution obtained by multiplying by an integer, it is determined that the integer number of target objects has been detected.
  The object detection apparatus according to claim 2. Inputting a detection target image;
Extracting feature points from the input detection target image;
Expressing the extracted feature points in an invariant feature space;
Storing feature points of the background image not including the target object;
Storing an invariant feature arrangement representing feature points of the target object in an invariant feature space;
All or a predetermined number or more of the invariant feature arrangements obtained by subtracting the corresponding feature points of the background image from the feature points of the detection target image in the invariant feature space are the feature points of the target object. A comparison step of determining that the target object has been detected when it matches the arrangement of the invariant features represented in the random feature space, and
In the case of comprising a step of selecting, as a unique feature, a portion having a frequency equal to or less than a predetermined value from a frequency distribution of invariant features representing feature points of the background image in an invariant feature space
The comparison step includes
It is determined that the target object has been detected when an invariant feature represented by subtracting the corresponding feature point of the background image from the feature point of the detection target image in the invariant feature space appears in the portion of the singular feature. An object detection method characterized by: The object detection device
Means for inputting a detection target image;
Means for extracting feature points from the input detection target image;
Means for representing the extracted feature points in an invariant feature space;
Means for storing feature points of a background image not including a target object;
Means for storing an arrangement of invariant features representing feature points of the target object in an invariant feature space;
All or a predetermined number or more of the invariant feature arrangements obtained by subtracting the corresponding feature points of the background image from the feature points of the detection target image in the invariant feature space are the feature points of the target object. A comparison means for determining that the target object has been detected when the arrangement of the invariant features represented in the variable feature space matches ,
When functioning as a singular feature selection means for selecting a portion having a frequency equal to or less than a predetermined value from the frequency distribution of invariant features representing the feature points of the background image in an invariant feature space,
In the comparison means,
It is determined that the target object has been detected when an invariant feature represented by subtracting the corresponding feature point of the background image from the feature point of the detection target image in the invariant feature space appears in the portion of the singular feature. An object detection program characterized by causing