JP6435647B2 - Object recognition method, object recognition apparatus, and object recognition program - Google Patents

Description

  The present invention relates to an object recognition method, an object recognition apparatus, and an object recognition program.

  For example, a pattern including a plurality of light spots arranged in a grid pattern is projected onto a predetermined area, and light spots are projected based on the deviation of each light spot from the reference position in an image obtained by photographing the projected pattern. There is a technique for recognizing the shape of an object in a specified area (see, for example, Patent Document 1).

  In this kind of technology, the object on which the pattern is projected is obtained by obtaining the distance between the light source used for projecting the pattern and the light spot projected on the object from the magnitude of deviation from the reference position of each light spot. Recognize the shape of objects such as surface irregularities.

JP 2004-96457 A

  In the technology for recognizing the shape of an object based on the distance to each light spot projected on the object, for example, when a pattern is projected on an area including a plurality of objects having similar shapes, Even if there are differences in surface characteristics, it is difficult to distinguish individual objects.

  An object recognition method, an object recognition apparatus, and an object recognition program disclosed in the present disclosure are intended to provide a technique that enables objects having different surface characteristics to be distinguished from each other even though their shapes are similar.

According to one aspect, the object recognition method, from the first object image pattern including a plurality of light spots are projected, brightness, and the light spots or features of clarity of outline similar to each other are distributed The range is extracted as a candidate region having the possibility of indicating the first object, a first vector including features of a plurality of light spots included in the candidate region is obtained, and a plurality of second objects whose types of objects are known Among the second vectors registered in advance including the characteristics of the plurality of light spots included in the projected patterns, the second vector similar to the other second vectors to the first vector of the candidate area The second object identified and associated with the identified second vector is recognized as the type of the first object.

According to another aspect, the object recognizing device has a light spot similar to each other in brightness or contour clarity from an image of the first object on which a pattern including a plurality of light spots is projected. An extraction unit that extracts a distributed range as a candidate region having a possibility of indicating the first object, a calculation unit that obtains a first vector including features of a plurality of light spots included in the candidate region, and the type of the object are known Among the previously registered second vectors including the features of the plurality of light spots included in the pattern projected onto each of the plurality of second objects, the second vector other than the first vector of the candidate region And a recognizing unit that recognizes the second object associated with the identified second vector as the type of the first object.

According to still another aspect, the object recognition program distributes light spots whose characteristics of brightness or contour clarity are similar to each other from an image of a first object on which a pattern including a plurality of light spots is projected. A plurality of second objects whose types of objects are known by obtaining a first vector including features of a plurality of light spots included in the candidate area. A second vector that is similar to the first vector of the candidate region from the other second vectors among the previously registered second vectors including the characteristics of each of the plurality of light spots included in the projected pattern And the computer recognizes the second object associated with the identified second vector as the type of the first object.

  The object recognition method, the object recognition apparatus, and the object recognition program disclosed in the present disclosure can discriminate objects having different surface characteristics even if they have similar shapes.

It is a figure which shows one Embodiment of an object recognition apparatus. It is a figure which shows the example of the image image | photographed with the camera shown in FIG. It is a figure which shows the example of the feature-value which shows the characteristic of each light spot shown in FIG. It is a figure which shows the example of the frequency distribution of a feature-value. It is a figure which shows the example of the feature vector calculated | required by the calculation part shown in FIG. It is a figure which shows the example of feature database DB shown in FIG. It is a figure which shows operation | movement of the object recognition apparatus 10 shown in FIG. It is a figure which shows another embodiment of an object recognition apparatus. It is a figure which shows the example of the image image | photographed with the camera shown in FIG. It is a figure which shows the example of the frequency distribution of the feature-value of the light spot contained in the image shown in FIG. It is a figure which shows operation | movement of the extraction part shown in FIG. It is a figure which shows operation | movement of the recognition part shown in FIG. It is a figure which shows the hardware structural example of the object recognition apparatus shown in FIG. It is a figure which shows operation | movement of the object recognition apparatus shown in FIG. It is a figure which shows another example of the process which extracts a candidate area | region. It is a figure which shows another example of the process which specifies the kind of object.

  Hereinafter, embodiments will be described with reference to the drawings.

  FIG. 1 shows an embodiment of an object recognition apparatus. The object recognition apparatus 10 illustrated in FIG. 1 includes an extraction unit 11, a calculation unit 12, and a recognition unit 13, and receives an image IMG photographed by the camera CAM. The object recognition apparatus 10 is connected to the feature database DB, and can refer to information stored in advance in the feature database DB. Note that the feature database DB may be provided inside the object recognition apparatus 10.

  Prior to the description of the function and operation of each unit included in the object recognition device 10, the camera CAM and the projection device PR will be described.

  For example, the projection apparatus PR projects a pattern PT including a plurality of light spots indicated by white circles in FIG. 1 onto an area including two objects T1 and T2. The pattern PT projected by the projection device PR includes, for example, a plurality of light spots arranged in a grid pattern with a predetermined interval. The projection apparatus PR, for example, diffracts a light beam emitted from a light emitting element such as a semiconductor laser included in the projection apparatus PR by a diffraction grating included in the projection apparatus PR, thereby grating a plurality of light spots. The pattern PT arranged in a shape is projected. In FIG. 1, for ease of explanation, the case where the pattern PT is projected onto the spherical objects T1 and T2 is shown, but the shapes of the objects T1 and T2 are not limited to the shapes shown in FIG. For example, the surface may be uneven. Further, each of the objects T1 and T2 illustrated in FIG. 1 is an example of a first object included in a region where the pattern PT is projected.

  For example, the camera CAM captures an image IMG including a plurality of light spots projected on the surfaces of the objects T1 and T2 by photographing the pattern PT projected onto the area including the objects T1 and T2 by the projection device PR. . The camera CAM has sensitivity to a wavelength band including the wavelength of light emitted from a light emitting element included in the projection apparatus PR. The image IMG photographed by the camera CAM is transferred to the extraction unit 11 of the object recognition device 10.

  The camera CAM photographs the pattern PT projected on the objects T1 and T2 from the direction inclined with respect to the direction in which the projection apparatus PR projects the pattern PT. For this reason, in the image IMG photographed by the camera CAM, the position of each light spot included in the pattern PT depends on the distance between the projection device PR that is the light source and each light spot projected on the objects T1 and T2. , It will deviate from the reference position. Here, the reference position of each light spot is, for example, each light in the image IMG obtained when the pattern PT projected on the reference plane arranged at a predetermined distance from the projection device PR is photographed by the camera CAM. The position of the point. An example of an image photographed by the camera CAM will be described later with reference to FIG.

  The extraction unit 11 illustrated in FIG. 1 receives, for example, images IMG of the objects T1 and T2 on which the pattern PT is projected from the camera CAM. Then, the extraction unit 11 can indicate, from the received image IMG, the objects (for example, the objects T1 and T2) included in the area where the pattern PT is projected, in each of the ranges where the light spots having similar characteristics are distributed. Extracted as a candidate region having sex. For example, the extraction unit 11 detects from the image IMG a range in which a predetermined number or more of light spots having the same luminance are continuously distributed, and extracts the detected range as one of candidate regions. An example of candidate areas extracted by the extraction unit 11 will be described with reference to FIG.

  The calculation unit 12 illustrated in FIG. 1 receives information indicating each candidate region extracted from the image IMG by the extraction unit 11 from the extraction unit 11. Then, based on the information received from the extraction unit 11, the calculation unit 12 obtains a first vector indicating the characteristics of a plurality of light spots included in the candidate region for each candidate region. For example, based on the deviation of each light spot included in the candidate region from the reference position, the calculation unit 12 projects each light projected onto the objects T1 and T2 from the reference plane on which the pattern PT is projected by the projection device PR. The distance to the point is calculated, and information indicating the calculated distance is set as a part of the first vector. Here, the reference plane on which the pattern PT is projected is equivalent to the reference plane used for setting the position serving as the reference for each light spot included in the pattern PT, and is set at a predetermined distance from the projection device PR, for example. The In addition, for example, the calculation unit 12 calculates an average value of feature amounts such as luminance indicating the features of each of a plurality of light spots included in the candidate region, and uses the calculated value for each feature amount as a part of the first vector. To do.

  Here, the information obtained as a part of the first vector by the calculation unit 12 and indicating the distance from the reference plane to each light spot indicates the feature of the shape of the object onto which the light spot included in each candidate area is projected. . In addition, the average value of the feature amount such as luminance included in the first vector for each candidate area is the feature of the light spot included in each candidate area, that is, the color, material, and smoothness of the surface of the object on which the light spot is projected. It shows the characteristics of such properties. That is, since the first vector obtained by the calculation unit 12 for each candidate area is information indicating the shape of the object and the surface features included in each candidate area, in the following description, the first vector is referred to as a candidate area feature vector. May be. Further, an example of feature vectors obtained for each candidate area by the calculation unit 12 will be described with reference to FIG.

  In the feature database DB shown in FIG. 1, feature vectors indicating the features of a plurality of light spots included in the pattern PT projected onto each of a plurality of known objects are registered in advance. That is, the feature database DB is an example of a pre-registered second vector indicating the features of the plurality of light spots included in the pattern PT projected on each of the plurality of second objects whose types of objects are known. is there. Here, the feature vector registered in the feature database DB corresponding to each known object is light projected onto each object from the reference plane on which the pattern PT is projected as information indicating the feature of the shape of each object. Contains information indicating the distance to each of the points. In addition, the feature vector registered in the feature database DB corresponding to each known object includes information such as the brightness of the light spot projected on each object as information indicating the characteristics of the surface property of each object. Contains information to indicate. An example of feature vectors stored in the feature database DB will be described later with reference to FIG.

  The recognizing unit 13 illustrated in FIG. 1 specifies a second vector that is more similar to the feature vector obtained for each candidate region than the other second vectors from the second vectors registered in the feature database DB. Then, the recognition unit 13 recognizes the type of the object associated with the identified second vector as the type of the object indicated by the candidate area, and the pattern PT is projected on the information indicating the type of the recognized object. Output as a recognition result of objects included in the region. The operation of the recognition unit 13 will be described later with reference to FIGS. 5 and 6.

  FIG. 2 shows an example of an image IMG taken by the camera CAM shown in FIG. The image IMG shown in FIG. 2 includes a plurality of light spots included in the pattern PT shown in FIG. Note that each of the circles SP0, SP1, SP2 and the ellipses SP3, SP4 illustrated in FIG. 2 is an example of a light spot included in the image IMG. In the example of FIG. 2, each light spot included in the image IMG has a luminance that is equal to or higher than a predetermined value that is set in advance, and the higher the luminance is, the higher the density is. The pattern PT is not limited to the example in which the light spots are arranged in a matrix of 9 rows × 12 columns shown in FIG. 2, but the light spots arranged in n rows × m columns (n and m are positive integers). Each of n and m may be greater than the values 9,12.

  Each of circles C1 and C2 indicated by a broken line in FIG. 2 indicates a region corresponding to each of the objects T1 and T2 onto which the pattern PT illustrated in FIG. 1 is projected. Here, in the case where there is no light source for illuminating the objects T1 and T2 other than the projection apparatus PR shown in FIG. 1, the shapes indicating the respective outlines of the objects T1 and T2 included in the image IMG are shown as regions C1 and C1 in FIG. It is difficult to detect C2 as a clear shape like a circle. Therefore, the extraction unit 11 illustrated in FIG. 1 extracts regions indicating the objects T1 and T2 based on the difference in characteristics of each light spot included in the image IMG illustrated in FIG.

  As shown in FIG. 2, the characteristics such as the brightness of the light spots (for example, the light spots SP1 and SP3) and the shape of the light spots included in the area C1 corresponding to the object T1 in the image IMG are, for example, outside the area C1. This is different from the feature of the light spot (for example, the light spot SP0) included in the region. Similarly, in the image IMG, features such as the brightness of the light spot (for example, the light spots SP2 and SP4) and the shape of the light spot included in the area C2 corresponding to the object T2 are included in the area outside the area C2, for example. This is different from the characteristics of the light spot (for example, the light spot SP0).

  FIG. 3 shows an example of the feature amount indicating the feature of each light spot shown in FIG. 3A, 3B, and 3C show images of the light spots SP1, SP2, and SP3 included in the image IMG shown in FIG. Note that the coordinate axes X and Y shown in FIGS. 3A, 3B, and 3C are, for example, horizontal and vertical pixel arrangements in the image sensor included in the camera CAM shown in FIG. Indicates direction. 3D, 3E, and 3F are the luminances in the X axis direction of the light spots SP1, SP2, and SP3 shown in FIGS. 3A, 3B, and 3C, respectively. The distribution of. In FIGS. 3D, 3E, and 3F, the coordinate axis B indicates the luminance.

  The light spot SP1 shown in FIG. 3A is a circle with a diameter r1 having substantially uniform luminance. Then, as shown in FIG. 3D, the brightness of the light spot SP1 is 0 from the brightness B1 at a width w1 smaller than the radius of the light spot SP1 at a position in the X-axis direction corresponding to the contour of the light spot SP1. Shows a steep change.

  On the other hand, the light spot SP2 shown in FIG. 3B is a circle with a diameter r2 having a luminance that decreases as the distance from the center increases. Then, as shown in FIG. 3E, the luminance of the light spot SP2 gradually changes from the luminance B2 to the luminance 0 with a width w2 substantially equal to the radius of the light spot SP2 from the vicinity of the center of the light spot SP2.

  The light spot SP3 shown in FIG. 3C is an ellipse having a major axis in the direction indicated by the straight line L1 that intersects the coordinate axis X at an angle θ. The shape of the light spot SP3 as shown in FIG. 3C is expressed using, for example, the major axis DL, the single axis DS, the major axis direction, and the angle θ between the coordinate axis X. Further, as shown in FIG. 3F, the brightness of the light spot SP3 is 0 from the brightness B1 at a width w3 smaller than the radius of the light spot SP3 at the position in the X-axis direction corresponding to the outline of the light spot SP3. Shows a steep change.

  As shown in FIGS. 3A and 3D, the shape of the light spot SP1 is a circle having a diameter r1, and the brightness B1 at the center of the light spot SP1 and the change in brightness at the contour of the light spot SP1. The steepness is an example of the feature of the light spot SP1. The circle diameter r1, the central brightness B1, and the steepness of the change in brightness, that is, the clarity of the outline of the light spot SP1, is an example of the feature quantity indicating the feature of the light spot SP1. is there. Note that the clarity of the outline of the light spot is, for example, closer to the value 1 as the slope of the change in luminance indicated by the ratio between the width w1 and the luminance B1 shown in FIG. It is desirable that the value be closer to 0 as the slope of change in luminance is smaller.

  Similarly, the diameter r2 of the circle showing the shape of the light spot SP2 shown in FIGS. 3B and 3E, the luminance B2 at the center, and the clarity of the outline of the light spot SP2 indicate the characteristics of the light spot SP2. It is an example of a feature-value. Note that the clarity of the outline of the light spot SP2 is a value obtained from the slope of the change in luminance indicated by the ratio between the width w2 and the luminance B2 shown in FIG. It becomes a value smaller than the clarity of the outline of the light spot SP1 described in D).

  Further, the long axis DL and the single axis DS of the ellipse showing the shape of the light spot SP3 shown in FIG. 3C, the intersection angle θ between the long axis direction and the coordinate axis X, the brightness B1 at the center, and the contour of the light spot SP3. Is an example of a feature amount indicating the feature of the light spot SP3. Note that the clarity of the outline of the light spot SP3 is a value obtained from the slope of the change in luminance indicated by the ratio between the width w3 and the luminance B1 shown in FIG. The value is almost equal to the clarity of the outline of the light spot SP1 described in D).

  Although not shown in FIG. 3, the light spot SP0 included in the image IMG shown in FIG. 2 has a brightness smaller than the brightness B1 at the center of the light spot SP1 shown in FIG. A circular shape having a diameter substantially equal to that of the light spot SP1 and the clarity of the outline.

  For example, the extraction unit 11 illustrated in FIG. 1 analyzes the image IMG received from the camera CAM to obtain the feature amount described in FIG. 3 for each light spot included in the image IMG. For example, a frequency distribution as shown in FIG. 4 is obtained by counting up for each predetermined class.

  FIG. 4 shows an example of the frequency distribution of feature amounts. FIG. 4A schematically shows a frequency distribution obtained by aggregating the number of light spots having the luminance indicated by the horizontal axis B among a plurality of light spots included in the image IMG. FIG. 4B schematically shows a frequency distribution obtained by aggregating the number of light spots having the clarity shown by the horizontal axis CL among a plurality of light spots included in the image IMG. 4A and 4B, the vertical axis n indicates the frequency.

  The frequency distribution regarding the luminance of the light spot shown by a curve in FIG. 4A shows local maximums PB0, PB1, and PB2 at luminances B0, B1, and B2, respectively. Here, the luminance of the light spot included in the image IMG depends on the reflectance of the surface of the object on which the pattern PT shown in FIG. 1 is projected. Therefore, as in the example of FIG. 4A, the fact that the frequency distribution of the luminance of the light spot included in the image IMG has a plurality of maximums means that the image IMG is projected onto a plurality of objects having different reflectances. It shows that it contains a light spot. In the image IMG shown in FIG. 2, areas where light spots having similar brightness are continuously distributed are areas that reflect the pattern PT with similar reflectance, and are shown in FIG. 1. The individual objects included in the area where the pattern PT is projected are shown.

  For example, in the frequency distribution shown in FIG. 4A, the light spot having the luminance included in the range Rb1 centered on the luminance B1 is the light reflected at the reflectance similar to the light spot SP1 shown in FIG. Is a point. As described with reference to FIG. 2, the luminance of each light spot included in the area C1 shown in FIG. 2 shows a value similar to the luminance B1 of the light spot SP1. Therefore, the extraction unit 11 extracts the range in which the light spots having the luminance included in the range Rb1 illustrated in FIG. 4A are distributed in the image IMG illustrated in FIG. As a candidate area indicating T1, the area C1 shown in FIG. 2 can be extracted. Similarly, the extraction unit 11 extracts the range in which the light spots having the luminance included in the range Rb2 centered on the luminance B2 shown in FIG. As a candidate area indicating the object T2, the area C2 shown in FIG. 2 can be extracted. The brightness B0 shown in FIG. 4A indicates the brightness of the light spot SP0 shown in FIG. 2, and the light spot having the brightness included in the range Rb0 centered on the brightness B0, that is, the light spot SP0. Light spots showing similar features are distributed in a region C0 excluding candidate regions C1 and C2 from the image IMG. Therefore, the extraction unit 11 extracts a range in which light spots having luminance included in the range Rb0 centered on the luminance B0 are distributed in the image IMG, so that the region C0 is set as a candidate region indicating the background of the objects T1 and T2. Can be extracted.

  The extraction unit 11 illustrated in FIG. 1 extracts candidate regions that focus on other feature values obtained for each light spot instead of extracting candidate regions that focus on the brightness of the light spot described in FIG. May be performed.

  The frequency distribution regarding the intelligibility of the contour of the light spot shown by the curve in FIG. 4B shows the maximum PC0, PC1, and PC2 at the intelligibility CL0, CL1, and CL2, respectively. Here, the clarity of the outline of the light spot included in the image IMG depends on the material and properties of the surface of the object onto which the pattern PT shown in FIG. 1 is projected. For example, the surface of the projected object is smooth. The higher the clarity. Note that the clarity C0 at the center of the maximum PC0 shown in FIG. 4B corresponds to the clarity obtained as one of the features of the light spot SP0 shown in FIG. Further, the clarity C1 at the center of the maximum PC1 shown in FIG. 4B corresponds to the clarity obtained as one of the features of the light spot SP1 shown in FIG. The clarity C2 at the center of the maximum PC 2 shown in FIG. 4B corresponds to the clarity obtained as one of the features of the light spot SP2 shown in FIG.

  Therefore, as in the example of FIG. 4B, the fact that the frequency distribution of the clarity of the light spots included in the image IMG has a plurality of local maxima means that the image IMG has a plurality of surface materials and properties different from each other. It shows that it contains a light spot projected on each of the objects. In the image IMG shown in FIG. 2, regions where light spots having similar clarity are continuously distributed are regions that reflect the pattern PT with similar properties, and are shown in FIG. 1. The individual objects included in the area where the pattern PT is projected are shown.

  That is, the extraction unit 11 includes, for example, a range C2 in which light points having intelligibility included in the range Rc2 centered on the intelligibility CL2 illustrated in FIG. 4B are distributed in the image IMG illustrated in FIG. A candidate region indicating the object T2 shown in FIG. 1 can be extracted. Similarly, in the image IMG illustrated in FIG. 2, the extraction unit 11 has a range C1 in which light points having intelligibility included in the range Rc1 centered on the clarity CL1 illustrated in FIG. A candidate area indicating the object T1 shown in FIG. 1 can be extracted. In the example of FIG. 2, the light spots having the clarity included in the range Rc0 centered on the clarity CL0 shown in FIG. 4B, that is, the light spots having characteristics similar to the light spot SP0 are distributed. The range to be performed is a region C0 obtained by removing the candidate regions C1 and C2 from the image IMG. The extraction unit 11 may extract the region C0 in which the light spots having the clarity included in the range Rc0 centered on the clarity CLO in the image IMG are extracted as candidate regions indicating the backgrounds of the objects T1 and T2, for example. Good.

  Similarly, the extraction unit 11 determines a region of the image IMG in which light spots indicating similar features are continuously distributed based on the frequency distribution obtained for each feature amount indicating the feature of each light spot. May be extracted as

  The extraction unit 11 may focus attention sequentially on a plurality of types of feature amounts indicating the features of each light spot, and may extract candidate regions using the focused feature amounts, for example, a vector including a plurality of types of feature amounts You may extract a candidate area | region for every cluster produced | generated based on the mutual distance. Note that an example of the extraction unit 11 that extracts candidate regions by paying attention to feature amounts sequentially selected from a plurality of types of feature amounts will be described later with reference to FIGS. 8 to 11.

  Information indicating the position of the light spot in the image IMG and the characteristics of each light spot included in each of the candidate areas extracted from the image IMG by the extraction processing described with reference to FIGS. 2 to 4 is the calculation unit 12 illustrated in FIG. Passed to. Then, based on the information received from the extraction unit 11, the calculation unit 12 includes information representing the feature of the shape indicated by the distance from the projection device PR illustrated in FIG. 1 to the light spot included in each candidate region, A feature vector including information representing the surface feature indicated by the feature amount of the light spot is obtained.

  FIG. 5 shows an example of the feature vector obtained by the calculation unit 12 shown in FIG. The example of FIG. 5 shows the feature vector obtained by the calculation unit 12 based on the information of the light spot included in each of the candidate areas C0, C1, and C2 shown in FIG. Of the elements shown in FIG. 5, elements equivalent to those shown in FIG. 4 are denoted by the same reference numerals and description of the elements may be omitted.

  In FIG. 5, symbols Y1, Y2, Y3, Y4, and Yk (k is an integer of 5 or more) shown in the shape feature column are calculated based on the difference between the position of each light spot and the reference position. The example of the class used for the total of distance is shown. The classes Y1, Y2, Y3, Y4, and Yk are, for example, sections in which a range of distances that can be measured based on the displacement of the position of the light spot included in the pattern PT is divided into lengths of several millimeters to several centimeters. It corresponds to each of.

  The feature vectors of the candidate areas C0, C1, and C2 shown in FIG. 5 are obtained by using the distances calculated based on the difference between the position of each light spot and the reference position as information representing the shape features. It includes a frequency distribution obtained by counting for each of a plurality of classes including Y2, Y3, Y4, and Yk. In the example of FIG. 5, the light spots included in each of the candidate regions C1 and C2 illustrated in FIG. 2 are one, four, six, and zero in each of the distance classes Y1, Y2, Y3, and Y4. A case is shown in which all of the 86 light spots included in the candidate area C0 are distributed in the class Yk.

  In FIG. 5, the feature vectors shown corresponding to the candidate regions C0, C1, and C2 are, for example, average luminance values B0 and B1 as information indicating the feature amount of the light spot included in each candidate region. , B2 and average values CL0, CL1, CL2 of the intelligibility. The example of the feature vector shown in FIG. 5 shows a case where the brightness and clarity of the light spots included in each of the candidate areas C0, C1, and C2 show a normal distribution. That is, as information indicating the characteristics of the luminance distribution and the clarity distribution of the light spots included in the candidate area C0, the feature vectors of the candidate area C0 are the frequency distributions shown in FIGS. 4A and 4B, respectively. Luminance B0 and intelligibility CL0, which are local maximum values PB0 and PC0. Similarly, as information indicating the characteristics of the luminance distribution and the clarity distribution of the light spots included in the candidate area C1, the feature vector of the candidate area C1 includes the frequencies shown in FIGS. 4A and 4B, respectively. It includes the brightness B1 and the clarity CL1, which are the maximum values of the local maximums PB1 and PC1 of the distribution. In addition, as information indicating the characteristics of luminance distribution and clarity distribution of light spots included in the candidate area C2, the feature vector of the candidate area C2 is a frequency distribution shown in each of FIGS. 4A and 4B. Brightness B2 and clarity CL2 which are the maximum values of local maximums PB2 and PC2. In the example of FIG. 5, among the elements included as information indicating the characteristics of the light spot in the feature vectors of the candidate regions C1, C2, and C3, the shape and size of the light spot that is a feature other than brightness and clarity The illustration of the element indicating is omitted.

  The recognition unit 13 illustrated in FIG. 1 identifies feature vectors similar to each of the feature vectors illustrated in FIG. 5 from the feature vectors registered in the feature database DB, and associates them with the identified feature vectors. The object type is set as the recognition result of the object included in each candidate area.

  FIG. 6 shows an example of the feature database DB shown in FIG. Among the elements shown in FIG. 6, elements equivalent to those shown in FIG. 5 are denoted by the same reference numerals and description of the elements may be omitted.

  The feature database DB shown in FIG. 6 indicates the shape of an object by the distance to each light spot projected onto the object corresponding to each of a plurality of types of objects including “Ball A” and “Ball B”. A feature vector including information and information indicating the feature of the projected light spot is held.

  The feature database DB shown in FIG. 6 includes information on the light spots included in the image IMG obtained by capturing the pattern PT projected on each of the balls A and B as information representing the characteristics of the shapes of the balls A and B. The frequency distribution of the distance calculated based on the position is held. In the example of FIG. 6, the light spots projected onto the balls A and B are one, four, six, and zero in order of the distance classes Y1, Y2, Y3, and Y4, respectively. In the case of distribution, that is, the shape of the ball A and the shape of the ball B are substantially equal.

  In addition, the feature database DB shown in FIG. 6 includes information indicating the characteristics of the light spot projected on the ball A as information indicating the surface property of the ball A, that is, the average value Ba and the clarity of the brightness of the light spot. Information including the average value CLa. Similarly, the feature database DB shown in FIG. 6 has information indicating the characteristics of the light spot projected on the ball B as information indicating the surface property of the ball B, that is, the average value Bb of the brightness of the light spot. Information including the average value CLb of the degree is held.

  For example, the information indicating the feature of the shape included in the feature vector corresponding to the candidate region C1 in FIG. 5 is almost the same as the feature of the shape of the ball A and the feature of the ball B shown in FIG. It is. Therefore, the object T1 included in the candidate region C1 is a ball based on a comparison between information indicating the shape registered for each type of object in the feature database DB and information indicating the shape included in the feature vector of the candidate region C1. It is difficult to determine whether it is A or ball B. Similarly, based on the comparison between the information indicating the shape registered for each type of object in the feature database DB and the information indicating the shape included in the feature vector of the candidate region C2, the object T2 included in the candidate region C2 is a ball It is difficult to determine whether it is A or ball B.

  However, the feature vector obtained for each candidate area by the calculation unit 12 shown in FIG. 1 and the feature vector registered for each type of object in the feature database DB are both added to the information representing the feature of the shape of the object. Contains information that characterizes the nature of the surface.

  Therefore, the recognition unit 13, for example, the object included in the candidate region C1 based on the distance between the feature vector obtained for the candidate region C1 by the calculation unit 12 and each feature vector registered in the feature database DB. It is possible to specify an object having characteristics similar to.

  For example, the recognition unit 13 specifies a feature vector whose distance from the feature vector of the candidate area C1 is smaller than the distance from other feature vectors among the feature vectors registered in the feature database DB. And the recognition part 13 outputs the kind of object matched with the specified feature vector as a recognition result of the object T1 contained in the candidate area | region C1. For example, let us consider a case where the luminance B1 shown in correspondence with the candidate area C1 in FIG. 5 is closer to the luminance Ba corresponding to the ball A than the luminance Bb shown in FIG. In this case, the distance between the feature vector of the candidate area C1 and the feature vector corresponding to the ball A is smaller than the distance between the other feature vectors including the feature vector corresponding to the ball B and the feature vector of the candidate area C1. . That is, the recognizing unit 13 recognizes the object T1 included in the candidate area C1 as the ball A having the smallest distance between the feature vectors.

  Similarly, the recognizing unit 13 specifies a feature vector whose distance from the feature vector of the candidate region C2 is smaller than the distance from other feature vectors among the feature vectors registered in the feature database DB. Objects included in the candidate area C1 can be recognized. For example, when the brightness B2 shown corresponding to the candidate area C2 in FIG. 5 is closer to the brightness Bb corresponding to the ball B than the brightness Ba shown corresponding to the ball A in FIG. 13 recognizes the object T2 included in the candidate area C2 as the ball B.

  That is, the object recognition apparatus 10 including the extraction unit 11, the calculation unit 12, and the recognition unit 13 illustrated in FIG. 1 recognizes information using information indicating the characteristics of the surface of the object together with information indicating the characteristics of the shape of the object. Thus, objects having different surface properties can be distinguished from each other even if their shapes are similar.

  FIG. 7 shows the operation of the object recognition apparatus 10 shown in FIG. The processing of step S301 to step S304 shown in FIG. 7 shows the operation of the object recognition apparatus 10 shown in FIG. 1, and shows an example of an object recognition method and an object recognition program. For example, the processing shown in FIG. 7 is realized by a processor mounted on the object recognition apparatus 10 executing an object recognition program. Note that the processing shown in FIG. 7 may be executed by hardware mounted on the object recognition apparatus 10.

  In step S301, the extraction unit 11 included in the object recognition apparatus 10 determines the brightness of the light spot and the light spot as the characteristics of each light spot included in the image IMG received from the camera CAM as described in FIG. A feature amount including the clarity of the contour and the shape of the light spot is obtained.

  In step S302, the extraction unit 11 extracts each of the ranges in which light spots having similar feature amounts are distributed in the image IMG as candidate areas indicating objects included in the area where the pattern PT is projected. For example, as described with reference to FIGS. 2 to 4, the extraction unit 11 is shown as candidate areas C1 and C2 indicated by broken-line circles in FIG. 2 and ranges other than the candidate areas C1 and C2 in FIG. 2. Candidate area C0 is extracted.

  In step S303, the calculation unit 12 uses the feature information indicating the shape of the object and the characteristics of the surface of the object included in each candidate area based on the information on the light spot included in each candidate area extracted in the process of step S302. Ask for. For example, based on the position of each light spot and the characteristics of each light spot included in each of the candidate areas C0, C1, and C2 shown in FIG. Each of the feature vectors shown correspondingly is calculated.

  In step S304, the recognition unit 13 identifies each candidate vector by identifying a feature vector similar to the feature vector obtained for each candidate region in step S303 from the feature vectors registered in the feature database DB. Recognize objects contained in the region. For example, the recognition unit 13 recognizes an object included in each candidate region based on the distance between the feature vector obtained for each candidate region and each feature vector registered in the feature database DB. It should be noted that the recognition unit 13 calculates the distance to be used for specifying a feature vector similar to the feature vector obtained for each candidate region from among the feature vectors registered in the feature database DB. It is not limited to the distance obtained using the element. For example, when the recognition unit 13 cannot identify the type of the object based on the distance based on some elements included in the feature vector, the recognition unit 13 sets the candidate region based on the distance between the feature vectors calculated by adding other elements. The types of objects included may be narrowed down. The recognition unit 13 that recognizes an object by gradually narrowing down the types of objects included in the candidate area will be described later with reference to FIGS.

  As described above, the object recognizing device 10 shown in FIG. 1 determines the range in which light spots having similar feature amounts are distributed from the image IMG obtained by capturing the pattern PT projected on the region including the object as the candidate region. Extract as And the object recognition apparatus 10 recognizes the object contained in the extracted candidate area | region by specifying the feature vector similar to the feature vector calculated | required for every extracted candidate area | region in feature database DB. That is, the object recognition device 10 recognizes an object included in each candidate region using information indicating the feature of the shape of the object and information indicating the characteristic of the surface property of the object. Accordingly, the object recognition apparatus 10 shown in FIG. 1 can detect individual light spots even when the information indicating the shapes obtained from the positions of the light spots is similar to each other as in the candidate areas C1 and C2 shown in FIG. The types of the objects T1 and T2 can be discriminated based on the difference in the surface properties indicated by the characteristics.

  FIG. 8 shows another embodiment of the object recognition apparatus. 8 that are equivalent to the components shown in FIG. 1 are denoted by the same reference numerals and description of the components may be omitted.

  An object recognition apparatus 10a illustrated in FIG. 8 includes an extraction unit 11a corresponding to the extraction unit 11 illustrated in FIG. 1, a calculation unit 12, a recognition unit 13a corresponding to the recognition unit 13 illustrated in FIG. DB is included.

  The extraction unit 11a includes an evaluation unit 111, a classification unit 112, an extraction control unit 113, a specification unit 114, and a detection unit 115. The evaluation unit 111 receives an image IMG taken by the camera CAM.

  The evaluation unit 111 evaluates a plurality of types of feature amounts including the brightness of the light spot and the clarity of the outline of the light spot for each light spot included in the image IMG received from the camera CAM. For example, as described with reference to FIG. 3, the evaluation unit 111 uses the light spot length as information indicating the shape of the light spot together with the brightness of the center of the light spot and the clarity of the outline of the light spot for each light spot. Evaluate the length of the axis and minor axis and the inclination of the major axis. The feature amount obtained for each light spot by the evaluation unit 111 is passed to the classification unit 112.

  The classification unit 112 sequentially selects one of a plurality of types of feature amounts received from the evaluation unit 111, and classifies the light spots included in the image IMG into groups having similar feature amounts based on the selected feature amounts. To do. As described with reference to FIG. 4, the classification unit 112 calculates a frequency distribution for the selected feature amount by aggregating the light spots for each value indicating the selected feature amount, and based on the calculated frequency distribution, The light spots are classified by grouping the light spots whose values indicating the feature quantities are similar to each other. For example, as shown in FIG. 4A, when the frequency distribution of the luminance of the light spot shows a maximum in the luminance B0, B1, B2, the classification unit 112 converts the light spot included in the image IMG to the luminance B0. , B1 and B2 are classified into three groups including light spots having luminance approximate to each.

  When the light spot included in the image IMG is classified into a plurality of groups by the classification by the classification unit 112, the extraction control unit 113 passes information indicating the light spots belonging to each of the plurality of groups to the specifying unit 114. On the other hand, when the light spots included in the image IMG are not classified into a plurality of groups due to the classification by the classification unit 112, the extraction control unit 113 instructs the classification unit 112 to perform classification based on another feature amount. To do. That is, the extraction control unit 113 is an example of a control unit that causes the classification unit 112 to perform classification based on a feature amount different from the feature amount used for classification when the classification unit 112 does not classify the plurality of groups.

  Based on the information received from the extraction control unit 113, the specification unit 114 specifies a range in which the light spots belonging to the group are distributed in the image IMG for each group obtained by the processing of the classification unit 112. For example, the specifying unit 114 specifies a range in which the light spots belonging to the group are distributed in the image IMG based on information indicating the positions of the light spots belonging to the group in the image IMG. For example, based on information indicating the position of each light spot in the group corresponding to each of the luminances B0, B1, and B2 illustrated in FIG. 4A, the specifying unit 114 performs the regions C0 and C1 illustrated in FIG. , C2 is specified as a range in which the light spots belonging to each group are distributed in the image IMG. Information indicating a region identified as a range in which the light spots belonging to each group are distributed by the identifying unit 114 is passed to the detecting unit 115.

  The detection unit 115 detects, as candidate regions, a range including a predetermined number or more of light spots set to a value of, for example, about 5 to 6 among the ranges specified by the specifying unit 114, and information indicating the detected candidate regions Is passed to the calculation unit 12. Note that the predetermined number used for determining whether or not to detect as a candidate region in the detection unit 115 is not limited to the value 5 to 6, and for example, the lower limit of the size that the object registered in the feature database DB occupies in the image IMG, etc. Is set in advance based on In addition, the detection unit 115 excludes a range corresponding to the background of the objects T1 and T2 illustrated in FIG. 1 from the candidate regions, such as the region C0 illustrated in FIG. It is desirable.

  Information indicating the candidate area detected by the detection unit 115 is passed to the calculation unit 12 and also to the extraction control unit 113. Then, based on the information received from the detection unit 115, the extraction control unit 113 determines whether or not an area that can be extracted as a candidate area still remains in the image IMG. The unit 112 is instructed to execute classification based on the new feature amount.

  The operations of the extraction control unit 113, the specifying unit 114, and the detection unit 115 included in the extraction unit 11a will be described later with reference to FIGS.

  The recognition unit 13a illustrated in FIG. 8 includes a search unit 131 and a recognition control unit 132.

  The search unit 131 receives at least some elements of the feature vectors obtained for each candidate area from the recognition control unit 132, and includes elements similar to the received elements from the feature vectors registered in the feature database DB. Search for feature vectors. For example, the search unit 131 searches the feature vector registered in the feature database DB for a feature vector including an element whose difference from the element received from the recognition control unit 132 is equal to or less than a predetermined threshold. In the search unit 131, the threshold value used for determining the similarity between the element of the feature vector registered in the feature database DB and the element received from the recognition control unit 132 is included in the feature vector corresponding to each of different types of objects, for example. Is set to a value smaller than the difference between the elements. In addition, the search unit 131 returns information indicating the type of object associated with the feature vector obtained by the search on the feature database DB to the recognition control unit 132.

  The recognition control unit 132 receives the feature vector obtained by the calculation unit 12 for each candidate region, selects an element from the elements included in the received feature vector in the order described below, and a feature database based on the selected element Is instructed to the search unit 131.

  For example, the recognition control unit 132 first selects an element indicating the distance to each light spot included in the candidate region from among the elements included in the feature vector received from the calculation unit 12, and selects the selected element as the search unit 131. To pass. Here, the element indicating the distance to each light spot included in the candidate area is the number of light spots included in each class provided for the distance, that is, the distance to each light spot, as described in FIG. Information indicating the frequency distribution of the object, and the feature of the shape of the object onto which the light spot included in the candidate area is projected. Thereafter, when the search unit 131 returns information indicating a plurality of types of objects, the recognition control unit 132 sequentially selects elements that have not been selected from among the elements of the feature vector received from the calculation unit 12. . Then, based on whether or not an element similar to the selected element is included, the search unit 131 executes a process of narrowing down feature vectors associated with a plurality of types of objects obtained by the previous search.

  For example, when the information indicating a plurality of types of objects is returned by the search unit 131, the recognition control unit 132 first uses the light spot included in the feature vector received from the calculation unit 12 as an element used to narrow down the feature vector. The brightness of the light spot is selected from the feature amount. When the feature vectors are further narrowed down, the recognition control unit 132 is an element indicating the clarity of the outline of the light spot and the shape of the light spot included in the feature vector received from the calculation unit 12 as the feature amount of the light spot. Are sequentially selected, and the selected elements are passed to the search unit 131.

  In addition, when the information indicating the single object is returned from the search unit 131, the recognition control unit 132 sets the type of the object indicated by the information received from the search unit 131 to the information indicating the object included in the candidate area. Output as.

  The operations of the selection unit 131 and the recognition control unit 132 included in the recognition unit 13a will be described later with reference to FIG.

  Here, the operation of the extraction unit 11a illustrated in FIG. 8 will be described using the case where the image IMG illustrated in FIG. 9 is received as an example.

  FIG. 9 shows an example of an image IMG taken by the camera CAM shown in FIG. The image IMG illustrated in FIG. 9 includes a plurality of light spots included in the pattern PT projected by the projection apparatus PR illustrated in FIG. Note that each of the circular SP0, SP1, SP2, and the elliptical SP3 illustrated in FIG. 9 is an example of a light spot included in the image IMG. In the example of FIG. 9, each light spot included in the image IMG is indicated by shading with higher density as the luminance is higher. The pattern PT is not limited to the example in which the light spots are arranged in a matrix of 9 rows × 12 columns shown in FIG. 2, but the light spots arranged in n rows × m columns (n and m are positive integers). Each of n and m may be greater than the values 9,12.

  In addition, a region C1 surrounded by a broken line in FIG. 9 indicates a region specified by the specifying unit 114 shown in FIG. 8 as a range in which light spots having characteristics similar to the light spot SP1 are distributed in the image IMG. . Similarly, in each of the areas C2-1 and C2-2 surrounded by a broken line in FIG. 9, light points having characteristics similar to the light spot SP2 are distributed in the image IMG by the specifying unit 114 shown in FIG. An area specified as a range to be displayed is shown. A region C3 surrounded by a broken line in FIG. 9 indicates a region specified by the specifying unit 114 shown in FIG. 8 as a range in which light spots having characteristics similar to the light spot SP3 are distributed in the image IMG. . The closed curves indicating the regions C1, C2-1, C2-2, and the region C3 are for indicating the positions in the image IMG of the region specified by the specifying unit 114, and are not included in the image IMG. .

  The evaluation unit 111 illustrated in FIG. 8 includes, for example, the luminance of the center of the light spot and the clarity of the outline of the light spot as a plurality of feature amounts indicating the characteristics of each light spot included in the image IMG illustrated in FIG. And the eccentricity of the ellipse indicating the shape of the light spot, and the obtained feature amount is passed to the classification unit 112.

  Then, the classification unit 112 illustrated in FIG. 8 aggregates the light spots for each value indicating the feature amount selected from the plurality of feature amounts based on the information received from the evaluation unit 111, as illustrated in FIG. In addition, a frequency distribution for the selected feature amount is obtained.

  FIG. 10 shows an example of the frequency distribution of the feature amount of the light spot included in the image IMG shown in FIG. FIG. 10A schematically shows a frequency distribution obtained by aggregating a plurality of light spots included in the image IMG shown in FIG. 9 for each luminance indicated by the horizontal axis B. FIG. Similarly, FIG. 10B schematically shows a frequency distribution obtained by aggregating a plurality of light spots included in the image IMG shown in FIG. 9 for each clarity shown by the horizontal axis CL. FIG. 10C schematically shows a frequency distribution obtained by counting a plurality of light spots included in the image IMG shown in FIG. 9 for each eccentricity indicated by the horizontal axis ec. In FIGS. 10A, 10B, and 10C, the vertical axis n indicates the frequency.

  The frequency distribution with respect to the luminance of the light spot shown by a curve in FIG. 10A shows local maximums PB0 and PB1 at the luminance B0 of the light spot SP0 and the luminance B1 of the light spot SP1 shown in FIG. In the example of FIG. 10A, each light spot included in the region surrounded by the closed curve C1 in FIG. 9 has a brightness included in the range Rb1 including the brightness B1 of the light spot SP1, and the outside of the closed curve C1. The case where the light spot in FIG. 2 has the brightness included in the range Rb0 including the brightness B0 of the light spot SP0 is shown.

  Further, the frequency distribution of the clarity of the contour of the light spot shown by the curve in FIG. 10B is the clarity of the contour CL0 of the light spot SP0 and the clarity of the contour CL2 of the light spot SP2 shown in FIG. , Show maximum PC0 and PC2, respectively. In the example of FIG. 10B, the clarity of each light spot included in the region surrounded by the closed curves C2-1 and C2-2 in FIG. 9 is included in the range Rc2 including the clarity CL2 of the light spot SP2. Shows the case. Similarly, in the example of FIG. 10B, the clarity of the light spot outside the closed curves C2-1 and C2-2 in FIG. 9 is included in the range Rc0 including the clarity CL0 of the light spot SP0. Show.

  The frequency distribution of the eccentricity of the ellipse, which is the shape of the light spot shown by the curve in FIG. 10C, is the value 0 and the light spot SP3 which are the circular eccentricity indicating the light spot SP0 shown in FIG. In the ellipse eccentricity e3 indicating the maximum PE0 and PE3, respectively. In the example of FIG. 10C, the eccentricity of each light spot included in the region surrounded by the closed curves C2-1 and C2-2 in FIG. 9 is in the range Re3 including the eccentricity e3 of the light spot SP2. Indicates the case where it is included. Similarly, in the example of FIG. 10C, the eccentricity of the light spot outside the closed curves C2-1 and C2-2 shown in FIG. 9 is in the range Re0 including the value 0 that is the eccentricity of the light spot SP0. Indicates the case where it is included.

  FIG. 11 shows the operation of the extraction unit 11a shown in FIG. The process from step S310 to step S318 shown in FIG. 11 is an example of the process of step S302 included in the object recognition program shown in FIG. For example, the processing shown in FIG. 11 is realized by a processor installed in the object recognition apparatus 10a shown in FIG. 8 executing an object recognition program. Note that the processing shown in FIG. 11 may be executed by hardware mounted on the object recognition apparatus 10a.

  In step S310, for each light spot included in the image IMG, the evaluation unit 111 illustrated in FIG. 8 includes, for example, a plurality of points including the center luminance, the clarity of the outline, and the eccentricity of an ellipse that is the shape of the light spot. Find the features.

  In step S311, the classification unit 112 illustrated in FIG. 8 selects one of the feature quantities obtained in the process of step S310, and totals the light spots for each selected feature quantity value. A frequency distribution as shown in A), (B), and (C) is obtained. In the process of step S311, for example, the classification unit 112 first selects the luminance of the light spot as one of the feature amounts, obtains the frequency distribution shown in FIG. 10A, and then receives the frequency distribution from the extraction control unit 113. In accordance with the instructions, other feature amounts are sequentially selected.

  In step S312, the extraction control unit 113 illustrated in FIG. 8 determines whether or not a plurality of local maxima are indicated by the frequency distribution obtained in the process of step S311.

  As shown in FIG. 10A, when a frequency distribution having a plurality of maximums is obtained (affirmative determination in step S312 (YES)), the extraction control unit 113 selects the feature selected in step S311. It is determined that the image IMG includes an object that is distinguished from the background based on the amount. If the determination is affirmative (YES) in step S312, the process proceeds to step S313. On the other hand, in the case of negative determination (NO) in step S312, the process proceeds to step S317.

  In step S313, the classification unit 112 classifies the light spots included in the image IMG into groups corresponding to the maximums of the frequency distribution obtained in the process of step S311. For example, based on the frequency distribution about the luminance shown in FIG. 10A, the classification unit 112 converts the light spot included in the image IMG into a group of light spots having the luminance included in the range Rb0 including the luminance B0. And a group of light spots having the luminance included in the range Rb1 including the luminance B1.

  In step S314, the specifying unit 114 illustrated in FIG. 8 sequentially selects one of the groups generated by the process in step S313, and specifies a range in which the light spots included in the selected group are distributed in the image IMG. . For example, the specifying unit 114 specifies the region C1 illustrated in FIG. 9 as a range in which the light spots included in the group associated with the luminance range Rb1 illustrated in FIG. 10A are distributed in the image IMG. Further, the specifying unit 114 corresponds to the outside of the region C1 illustrated in FIG. 9 as a range in which the light spots included in the group associated with the luminance range Rb0 illustrated in FIG. 10A are distributed in the image IMG. Specify the range.

  In step S315, the detection unit 115 illustrated in FIG. 8 detects, as candidate regions, a range in which a predetermined number (for example, 5) or more of light spots are continuously distributed among the ranges specified in the process of step S314. For example, the detection unit 115 detects the region C1 shown in FIG. 9 as a candidate region because the number of light spots included in the region C1 specified by the specification unit 114 is larger than the predetermined number 5 set in advance. To do. Here, the region specified as the range in which the light spots belonging to the group associated with the luminance range Rb0 shown in FIG. 10A are distributed corresponds to the range obtained by removing the region C1 from the image IMG. The outer boundary of the marked area coincides with the boundary of the image IMG. When the boundary of the specified region matches the boundary of the image IMG, such as the region specified for the group associated with the luminance range Rb0, the detection unit 115 determines that the specified region corresponds to the background. to decide. That is, when the boundary of the identified area matches the boundary of the image IMG, the detection unit 115 does not detect the candidate area indicating the object regardless of the number of light spots in the identified area, and the background. An unclassified region that is not classified as a region indicating an object other than.

  In step S316, the extraction control unit 113 determines whether or not the processes in steps S314 and S315 have been executed for all groups generated in the classification process in step S313.

  If there is still a group for which the processes in steps S314 and S315 have not been completed (No determination in step S316 (NO)), the process returns to step S314. Then, the specifying unit 114 specifies a range in which the light spots belonging to the newly selected group are distributed in the image IMG. Thereafter, in step S315, a new candidate area is detected by the detection unit 115 in accordance with the number of light spots included in the newly specified range.

  On the other hand, when the process for all the groups generated in the classification process in step S313 is completed (Yes in step S316 (YES)), the process proceeds to step S317.

  In step S317, the extraction control unit 113 determines whether or not an unclassified region remains in the image IMG. For example, every time a candidate area is detected in the process of step S315, the extraction control unit 113 removes the light spot included in the detected candidate area from the light spot included in the image IMG. When the number of light spots remaining without being removed is equal to or larger than a predetermined number used for determination of candidate regions in the detection unit 115 and is continuously distributed in the image IMG, the extraction control unit 113 Let it be affirmative (YES) in step S317.

  If the determination in step S317 is affirmative (YES), the extraction control unit 113 proceeds to the process of step S318. On the other hand, if no unclassified region remains in the image IMG (No determination in step S317 (NO)), the extraction control unit 113 ends the process of extracting the candidate region.

  In step S318, the extraction control unit 113 determines whether or not the classification based on all the feature values obtained for each light spot in the process of step S310 is completed.

  If there is a feature amount that has not been used for classification yet (No determination in step S318), the process returns to step S311. In this case, for example, intelligibility is selected as a new feature amount by the classification unit 112 in accordance with an instruction from the extraction control unit 113, and a frequency distribution for intelligibility as shown in FIG. It is done.

  Then, by performing the process of step S313 based on the frequency distribution regarding the intelligibility, the classification unit 112 sets the light spots included in the image IMG to the group corresponding to the maximum PC0 illustrated in FIG. A group corresponding to the maximum PC 2 is classified. For example, in FIG. 10B, the classification unit 112 classifies the light spots having the clarity included in the range Rc0 including the clarity CL0 into a group corresponding to the maximum PC0, and is included in the range Rc2 including the clarity CL2. Are classified into groups corresponding to the maximum PC2.

  Thereafter, in step S314, the specifying unit 114 sets, for example, the region C2- shown in FIG. 9 as a range in which the light spots belonging to the group associated with the maximum PC2 shown in FIG. 10B are distributed in the image IMG. 1, C2-2 is specified. The area C2-1 shown in FIG. 9 includes a predetermined number (for example, 5) or more of light spots therein, so that candidate areas whose light spot characteristics are similar to each other by the detection unit 115 in the process of step S315. Detected as one of On the other hand, since the number of light spots included in the region C2-2 illustrated in FIG. 9 is less than the predetermined number, in the process of step S315, the detection unit 115 determines that the region C2-2 is not a candidate region indicating an object. Therefore, it is determined that noise is present, and is not detected as a candidate area.

  A value obtained by subtracting the number of light spots included in each of the areas C1 and C2-1 from the number of light spots included in the image IMG illustrated in FIG. 5) As described above, the extraction control unit 113 determines that an unclassified region remains. At the stage where the region C1 and the region C2-1 are detected as candidate regions, the eccentricity indicating the feature of the shape of the light spot among the feature amounts obtained for each light spot by the evaluation unit 111 is still classified. Not used.

  In this case, the extraction control unit 113 determines in step S318 that there is a feature amount that has not yet been used for classification, and returns to step S311 again. In step S311, the classifying unit 112 selects an eccentricity as a new feature quantity, and obtains a frequency distribution for the eccentricity as shown in FIG.

  Then, by performing the process of step S313 based on the frequency distribution of the eccentricity, the classification unit 112 sets the light spot included in the image IMG as a group corresponding to the maximum PE0 illustrated in FIG. The group is classified into a group corresponding to the maximum PE3. For example, in FIG. 10C, the classification unit 112 classifies the light spots having the eccentricity included in the range Re0 including the eccentricity e0 into a group corresponding to the maximum PE0 and includes the light spot in the range Rc2 including the eccentricity e3. Are classified into groups corresponding to the maximum PC2.

  Thereafter, in step S314, the specifying unit 114 sets, for example, the region C3 illustrated in FIG. 9 as a range in which the light spots belonging to the group associated with the maximum PE3 illustrated in FIG. 10C are distributed in the image IMG. Identify. Since the region C3 shown in FIG. 9 includes a predetermined number (for example, 5) or more of light spots therein, one of candidate regions whose light spot characteristics are similar to each other by the detection unit 115 in the process of step S315. Detected as one.

  That is, the extraction unit 11a illustrated in FIG. 8 repeats the processing from step S311 to step S318 illustrated in FIG. 11 while paying attention to each of the luminance, clarity, and eccentricity obtained as the characteristics of the light spot. Accordingly, the extraction unit 11a illustrated in FIG. 8 performs each feature amount by a simple process compared to, for example, clustering that uses all the feature amounts obtained as light spot features collectively. Candidate regions indicating objects distinguished from the background can be extracted without omission.

  Then, when the classification process focusing on all of the feature amounts obtained by the evaluation unit 111 is completed, the extraction control unit 113 determines whether or not the unclassified region remains, whether or not the determination is affirmative in step S318. The candidate area extraction process is terminated according to the determination (YES) route.

  The light spot information included in each candidate area extracted as described with reference to FIGS. 8 to 11 is passed to the calculation unit 12 illustrated in FIG. 8, and the calculation unit 12 obtains a feature vector for each candidate area. Used for processing.

  FIG. 12 shows the operation of the recognition unit 13a shown in FIG. The process from step S320 to step S327 shown in FIG. 12 is an example of the process of step S304 included in the object recognition program shown in FIG. In the processing shown in FIG. 12, for example, the processor mounted in the object recognition apparatus 10a shown in FIG. 8 executes a program every time the calculation unit 12 shown in FIG. It is realized with. Note that the processing shown in FIG. 12 may be executed by hardware mounted on the object recognition apparatus 10a.

  In step S320, the recognition control unit 132 of the recognition unit 13a illustrated in FIG. 8 selects the element indicating the frequency distribution of the distance from the feature vector received from the calculation unit 12 to each light spot in the candidate area, and selects the selected element. The element is passed to the search unit 131. For example, in FIG. 5, the recognition control unit 132 is included in a plurality of classes including classes Y1 to Y4 among elements included in the feature vector corresponding to each of the candidate areas C1 and C2 illustrated in FIG. A partial vector including an element indicating the number of light spots to be transmitted is passed to the search unit 131.

  In step S321, the search unit 131 searches for an element similar to the element indicating the frequency distribution of the distance to each light spot in the candidate region selected in the process of step S320 from the feature vectors registered in the feature database DB. Search for feature vectors to include. For example, the search unit 131 includes a partial vector including an element indicating the frequency distribution of the distance to each light spot received in the process of step S320, and an element indicating the frequency distribution of the distance in each feature vector registered in the feature database DB. The distance from the partial vector including is calculated. Then, the search unit 131 includes, in the feature vectors registered in the feature database DB, a feature vector for which a distance equal to or smaller than a preset threshold is obtained, including a feature similar to the feature vector corresponding to the candidate region. Search as a vector. In addition, the search unit 131 passes information indicating the type of object associated with each of the feature vectors searched from the feature database DB to the recognition control unit 132 as a recognition result candidate for the object included in the candidate area.

  For example, the search unit 131 uses a partial vector indicating the frequency distribution of distances included in the feature vector shown in correspondence with the candidate region C1 in FIG. 5 and each feature vector registered in the feature database DB shown in FIG. The distance from the partial vector indicating the frequency distribution of the included distance is obtained. In this case, the search unit 131 sets the feature vector whose distance from the partial vector indicating the frequency distribution of the distance included in the feature vector corresponding to the candidate region C1 in FIG. The feature vectors shown corresponding to the balls A and B are searched.

  In step S322, the recognition control unit 132 determines whether or not the search unit 131 has searched at least one feature vector from the feature database DB.

  When one or more feature vectors are searched by the search unit 131 (Yes in step S322 (YES)), the process proceeds to step S323.

  In step S323, the recognition control unit 132 determines whether one type of object included in the candidate area is specified by the search by the search unit 131.

  When the search unit 131 searches for a plurality of feature vectors including elements similar to the feature vector corresponding to the candidate region, the recognition control unit 132 has not yet identified one type of object included in the candidate region. to decide. In this case, the recognition control unit 132 executes the processes of steps S324 and S325 according to the negative determination (NO) route of step S323. For example, in the process of step S321, when both of the feature vectors shown in FIG. 6 corresponding to each of the balls A and B are searched by the search by the search unit 131, the process proceeds to step S324.

  On the other hand, when the search unit 131 searches for a single feature vector that includes an element similar to the feature vector corresponding to the candidate region, the recognition control unit 132 identifies one type of object included in the candidate region. Judge that it was done. In this case, the recognition control unit 132 executes the process of step S326 according to the affirmative determination (YES) route of step S323.

  In step S <b> 324, the recognition control unit 132 sequentially selects one of the elements indicating the feature of the light spot included in the feature vector received from the calculation unit 12, and passes it to the search unit 131. For example, the recognition control unit 132 sequentially selects elements indicating the brightness of the light spot, the clarity of the outline of the light spot, and the feature of the shape of the light spot included in the feature vector received from the calculation unit 12. The selected element is passed to the search unit 131.

  In step S325, the search unit 131 uses the elements received in step S324 to narrow down the feature vectors searched from the feature database DB in step S321. For example, the search unit 131 corresponds to each of the element (for example, luminance) passed in the process of step S324 and each of the balls A and B searched from the feature database DB shown in FIG. 6 in the process of step S321. The corresponding element included in the feature vector is compared. Then, the search unit 131 determines whether the difference between the element (for example, luminance) passed in the process of step S324 and the corresponding element of each feature vector searched in the process of step S321 is within a predetermined threshold. Based on the above, the feature vectors are narrowed down. Note that the threshold used for narrowing down feature vectors in the search unit 131 is set in advance to a value smaller than the difference between the corresponding feature amounts indicated by the light spots projected on different types of objects, for example, for each type of element. It is desirable that

  After the process of step S325 ends, the process returns to step S323. In step S323, the recognition control unit 132 determines whether one feature vector is specified by the narrowing-down process in step S325.

  For example, let us consider a case where the brightness Ba shown in FIG. 6 corresponding to the ball A is closer to the brightness B1 shown in FIG. 5 corresponding to the candidate area C1 than the brightness Bb corresponding to the ball B. . In this case, by the process of step S325, the search unit 131 uses the feature vector corresponding to each of the balls A and B as the feature vector of the ball A including the brightness Ba close to the brightness B1 included in the feature vector of the candidate area C1. Refine to.

  When a single feature vector similar to the feature vector passed from the calculation unit 12 is identified among the feature vectors registered in the feature database DB by the processing in step S325 (Yes in step S323 (YES) )), The process proceeds to step S326.

  In step S326, the recognition control unit 132 outputs, as the recognition result of the object included in the candidate area, the type of the object associated with the feature vector specified in the process of step S321 or step S325 in the feature database DB. . For example, if the feature vector registered in the feature database DB corresponding to the ball A is more similar to the feature vector of the candidate area C1 than the other in the process of step S325, the recognition control unit 132 selects the ball A as a candidate. It outputs as a recognition result of the object contained in the area | region C1.

  On the other hand, if the feature vector similar to the feature vector corresponding to the candidate area cannot be searched from the feature vectors registered in the feature database DB in the process of step S321, the process proceeds to step S327.

  In step S327, the recognition control unit 132 outputs information indicating that the object included in the candidate area cannot be recognized, and ends the process of specifying the type of the object.

  As described with reference to FIG. 12, the recognition unit 13a shown in FIG. 8 first shows the feature of the shape included in the feature vector obtained corresponding to the candidate region from the feature vector registered in the feature database DB. Search for feature vectors containing elements similar to the element. Then, the recognition unit 13a specifies a single feature vector similar to the feature vector corresponding to the candidate region by narrowing down the feature vector searched based on the shape feature based on the element indicating the feature of the light spot. To do. Thereby, for example, the recognition unit 13a has a single feature vector in the feature database DB that includes elements similar to the frequency distribution of distances included as elements indicating shape features in the feature vectors obtained in correspondence with the candidate regions. In this case, the comparison of the characteristics of the light spot can be omitted. That is, the recognizing unit 13a illustrated in FIG. 8 determines the distance between the feature vector corresponding to each candidate region and each feature vector registered in the feature database DB based on all the elements included in the feature vector. Similar feature vectors can be identified at a higher speed than in the case of calculation.

  The object recognition devices 10 and 10a of the present disclosure described above can be realized by using a computer device such as a personal computer, for example.

  FIG. 13 shows a hardware configuration example of the object recognition apparatus 10a shown in FIG. 13 that are equivalent to the components shown in FIG. 1 or FIG. 8 are denoted by the same reference numerals, and description of the components may be omitted.

  The computer device COM shown in FIG. 13 includes a processor 21, a memory 22, a storage device 23, a general-purpose interface 24, a display device 25, an optical drive device 26, and a network interface 28. The processor 21, the memory 22, the storage device 23, the general-purpose interface 24, the display device 25, the optical drive device 26, and the network interface 28 shown in FIG. 13 are connected to each other via a bus. Further, the general-purpose interface 24 is connected to a projection device PR1 having a function equivalent to that of the projection device PR shown in FIG. 1 and a camera CAM1 having a function equivalent to the imaging unit CAM shown in FIG.

  The processor 21, the memory 22, the storage device 23, and the general-purpose interface 24 illustrated in FIG. 13 are included in the object recognition device 10a.

  The projection apparatus PR1 shown in FIG. 13 projects a pattern PT1 including a plurality of light spots, for example, on the predetermined area including the person Q1 and the bed BD, similar to the pattern PT described in FIGS. . Further, the camera CAM1 shown in FIG. 13 captures, for example, a region where the pattern PT1 is projected by the projection device PR1 at a predetermined time interval, and each of a plurality of images IMG obtained by the capturing is displayed on the general-purpose interface 24. To the processor 21.

  The optical drive device 26 shown in FIG. 13 can be loaded with a removable disk 27 such as an optical disk, and reads and records information recorded on the mounted removable disk 27.

  The computer device CAM is connected to a network NW such as the Internet via the network interface 28, and can exchange information with the server device SV connected to the network NW.

  The memory 22 shown in FIG. 13 stores an application program for the processor 21 to execute the object recognition processing described with reference to FIGS. 14 to 16 together with the operating system of the computer apparatus COM. Note that an application program for executing the object recognition process can be recorded and distributed on a removable disk 27 such as an optical disk, for example. The processor 21 may store the read application program in the memory 22 and the storage device 23 by attaching the removable disk 27 to the optical drive device 26 and performing read processing. Further, the application program for executing the object recognition process may be downloaded from the server device SV, for example, via the network interface 28 and the network NW. The downloaded application program is read into the memory 22 and the storage device 23 to become an application program that can be executed by the processor 21.

  The processor 21 performs the functions of the extraction unit 11a, the calculation unit 12, and the recognition unit 13a illustrated in FIG. 8 by executing an application program for object recognition processing stored in the memory 22 or the like.

  The application program for object recognition processing may include information indicating feature vectors obtained in advance for each of a plurality of types of objects, for example, similar to the feature database DB shown in FIG. In this case, the feature vector obtained in advance for each of the plurality of types of objects is stored in a part of the storage area of the storage device 23, for example. That is, the same constituent elements as the feature database DB shown in FIG. 8 are realized by using a part of the storage area of the storage device 23. It should be noted that the feature database DB provided in the storage device 23 preferably includes feature vectors indicating the features of various objects including the bed BD and the person Q1 included in the region where the pattern PT1 is projected by the projection device PR1. .

  As described above, the object recognition device 10a shown in FIG. 8 is, for example, a combination of the processor 21, the memory 22, the storage device 23, and the general-purpose interface 24 included in the computer device COM shown in FIG. It can be realized by working.

  FIG. 14 shows the operation of the object recognition apparatus 10a shown in FIG. Each process of step S301 to step S304 and step S331 to step S332 illustrated in FIG. 14 is an example of a process included in the application program for the object recognition process. In addition, each process of step S301 to step S304 and step S331 to step S332 is executed by the processor 21 shown in FIG. Note that the processing in steps S301 to S304 shown in FIG. 14 is equivalent to the processing in steps S301 to S304 shown in FIG. The processor 21 executes each process shown in FIG. 14 every time a new image is taken by the camera CAM1 shown in FIG.

  In step S331, the processor 21 receives an image IMG taken by the camera CAM1 every time, for example, about several tens of milliseconds to several seconds, via the general-purpose interface 24 shown in FIG.

  In step S301, the processor 21 obtains a feature amount indicating the feature of each light spot included in the image IMG received in the process of step S331 as described in FIG.

  In step S302, the processor 21 sets, as the candidate region indicating the object included in the image IMG, the range in which the light spots having similar characteristics are distributed in the image IMG as described with reference to FIGS. 4 and 9 to 11. Extract. Note that the processor 21 may extract candidate regions by performing each process described in FIG. 15 instead of each process described in FIG. 11.

  In step S303, the processor 21 obtains a feature vector indicating the feature of the object onto which the light spot included in the candidate region is projected, as described in FIG. 5, for each candidate region extracted in the process of step S302. .

  In step S <b> 304, the processor 21 uses the information registered in the feature database DB provided in the storage device 23 to specify the type of an object that shows a feature similar to the feature vector of the candidate area. For example, as described with reference to FIGS. 5 and 6, the processor 21 recognizes an object included in the candidate region based on the distance between the feature vector of the candidate region and each feature vector registered in the feature database DB. Alternatively, the object may be recognized by the process shown in FIG. Further, the processor 21 may perform recognition of the object by the process described in FIG. 16 using the information stored in the process of step S332 described below.

  In step S332, for each candidate region, the processor 21 stores, in the memory 22 or the like, elements that greatly contributed to the specification of the object type compared to other elements included in the feature vector in the process of step S304. For example, the processor 21 is used to narrow down the object types retrieved from the feature database DB to one in the process of specifying the object type as an element that greatly contributes to the specification of the object type compared to other elements. The obtained element is stored in the memory 22 or the like. The information stored in the memory 22 or the like in the process of step S322 is used for the process of specifying the type of object included in each candidate area extracted from the image IMG newly taken by the camera CAM1. Note that the process of specifying the type of object included in each candidate area using the information stored in the memory 22 or the like in the process of step S322 will be described later with reference to FIG.

  Further, every time the processor 21 receives the image IMG from the camera CAM1, the processor 21 may display information indicating the type of the object specified for each candidate area in the process of step S304, for example, on the display device 25 illustrated in FIG. Good.

  Similar to the object recognition apparatus 10 shown in FIG. 1, the object recognition apparatus 10a shown in FIG. 13 takes into account the characteristics of the surface characteristics of the object as well as the shape of the object included in the area where the pattern PT1 is projected. Can be recognized. For example, the head of the person Q1 and an object such as a ball or stuffed toy equivalent to the head of the person Q1 are discriminated based on the difference between the surface characteristics of the head of the person Q1 and the surface characteristics of the ball. Is possible. Further, if, for example, feature vectors corresponding to the front head and the horizontal head of the person Q1 are registered in the feature database DB provided in the storage device 23, the person Q1 on the bed BD is registered. It is also possible to determine the orientation of the face.

  That is, the object recognition apparatus 10a shown in FIG. 13 is based on the characteristics of the surface of the head of the person Q1, for example, even when an object (for example, a stuffed animal) confusing with the head of the person Q1 is on the bed BD. Thus, the head of the person Q1 and the orientation of the head can be recognized. Then, the object recognition device 10a provides information indicating the orientation of the face of the person Q1 along with the positional relationship between the candidate region recognized as the bed BD and the candidate region recognized as the person Q1, to the operator of the object recognition device 10a. Can be presented. An operator who receives information indicating the positional relationship between the bed BD and the person Q1 and the head direction of the person Q1 from the object recognition device 10a can determine, for example, the posture and behavior of the person Q1. That is, the object recognition device 10a can provide the operator with information useful for determining the posture and behavior of the person Q1 who stays in the area where various objects are arranged. This is useful in fields such as watching people staying at facilities. That is, the object recognition apparatus 10a shown in FIG. 13 is useful in the field of a person watching service in a medical facility or a nursing facility.

  Similarly, the object recognition apparatus 10a shown in FIG. 13 is capable of recognizing various fixtures installed in a factory, an office, etc., and a person staying in the interior, and the positional relationship between the fixtures and fixtures. Information indicating a positional relationship with a person and a change in the positional relationship can be output. Therefore, the object recognition apparatus 10a shown in FIG. 13 can be used for a monitoring system that detects an intruder into a factory or an office.

  By the way, when the pattern PT1 shown in FIG. 13 or an object on which the pattern PT shown in FIG. 1 is projected includes an object having different surface characteristics, extraction is performed based on each of the characteristics of the light spots included in the image IMG. In some cases, the boundaries of the candidate regions that are displayed may be unclear. For example, in the example of the image IMG shown in FIG. 2, the difference between the clarity of the outline of each light spot included in the candidate area C1 and the clarity of the outline of each light spot included in the candidate area C0 is the light spot SP1. This is smaller than the difference between the clearness CL1 of the contour and the clearness CL2 of the contour of the light spot SP2. Therefore, in the frequency distribution of the intelligibility shown in FIG. 4B, the range in which the intelligibility of the outline of the light spot included in the candidate area C1 and the intelligibility of the outline of the light spot in the candidate area C0 are The distributed range may overlap each other. For this reason, the minimum of the frequency distribution indicating the boundary between the maximum PC1 corresponding to the candidate area C1 and the maximum PC0 corresponding to the candidate area C0 is less clear than the minimum between the maximum PC1 and the maximum PC2. . Then, when the boundary between the two maxima is unclear, such as the maxima PC1 and PC0 shown in FIG. 4, the candidate region extracted corresponding to one maxima is originally a candidate corresponding to the other maxima. It may contain light spots included in the region. That is, in the frequency distribution of the feature amount, when candidate regions are extracted corresponding to each of two maxima whose boundaries are unclear, they are extracted corresponding to each of two maxima whose boundaries are clear Compared to the case, the probability of the extracted candidate region is lower. In other words, in the frequency distribution of the feature quantity, candidate areas extracted corresponding to highly independent maxima that are clearly distinguished from other maxima are classified into candidate areas corresponding to other maxima. Since the possibility of including a point is low, it is a probable candidate area.

  Therefore, the processor 21 illustrated in FIG. 13 executes the processing illustrated in FIG. 15 as the processing in step S302 illustrated in FIG. 14, so that the frequency distribution of the feature amount of the light spot included in the image IMG exceeds the threshold value. Candidate areas corresponding to local maxima with high independence are selectively extracted.

  FIG. 15 shows another example of processing for extracting candidate regions. The processes in steps S310 to S312 and steps S314 to S318 and the processes in steps S341 and S342 shown in FIG. 15 are executed by the processor 21 shown in FIG. Note that among the steps shown in FIG. 15, the steps equivalent to those shown in FIG. 11 are denoted by the same reference numerals and description of the steps may be omitted. For example, steps S310 to S312 and steps S314 to S318 shown in FIG. 15 are equivalent to the steps indicated by the same reference numerals in FIG.

  In the flowchart shown in FIG. 15, in the case of an affirmative determination in step S312 equivalent to step S312 shown in FIG. 11, the processor 21 replaces the processing in step S313 shown in FIG. 11 with steps S341 and S342. Execute the process.

  In step S341, the processor 21 obtains an index value indicating the degree to which each maximum indicated by the frequency distribution is independent from the other maximum, based on the frequency distribution obtained in the process of step S311. For example, the processor 21 obtains a value obtained by dividing a value obtained by subtracting a local minimum value indicating the boundary between each local maximum and an adjacent local maximum by a maximum value of each local maximum by the maximum value of each local maximum. Obtained as an index value indicating independence with respect to adjacent maxima. For example, the index value indicating the independence of the maximum PC1 from the maximum PC0 shown in FIG. 4B is the minimum value of the frequency distribution indicating the boundary between the maximum PC1 and the maximum PC0 from the frequency corresponding to the clarity CL1. The value obtained by subtraction is a value obtained by dividing the value by the frequency corresponding to the clarity CL1. As shown in FIG. 4B, when the minimum value indicating the boundary between the maximum PC1 and the maximum PC0 in the frequency distribution is larger than half of the frequency corresponding to the clarity CL1, the maximum shown in FIG. The index value indicating the independence of PC1 from the maximum PC0 is a value smaller than 0.5, for example. On the other hand, as shown in FIG. 4B, when the minimum value indicating the boundary between the maximum PC1 and the maximum PC2 is less than half of the frequency corresponding to the clarity CL2, the maximum PC2 shown in FIG. An index value indicating independence from the maximum PC1 is, for example, a value larger than 0.5.

  In step S342, the processor 21 determines the maximum in which the index value equal to or greater than the threshold is obtained in the process of step S341 in the frequency distribution corresponding to the feature amount, and corresponds the light spot included in the image IMG to the determined maximum. Classify into groups with features For example, the processor 21 determines that the local maximum obtained by the index value larger than the threshold value set to a value of about 0.5 or larger than the value 0.5 is adjacent in the frequency distribution obtained in the process of step S311. It is determined as a local maximum independent of the local maximum. Then, the processor 21 classifies the light spots having the feature amount within the range corresponding to the determined maximum among the light spots included in the image IMG into a group corresponding to the determined maximum. On the other hand, among the light spots included in the image IMG, the processor 21 leaves the light spots having the characteristic values included in the range corresponding to the maximum where the index value equal to or less than the threshold is obtained without being grouped.

  For example, when the process of step S341 obtains a value larger than 0.5 as an index value indicating the independence of the maximal PC 2 shown in FIG. 4B, the processor 21 in FIG. The light spots having the clarity included in the range Rc2 are classified into a group corresponding to the maximum PC2. Here, the light spot classified into the group corresponding to the maximum PC2 by the process of step S342 shows a characteristic clearly different from other light spots not belonging to the group from the viewpoint of the clarity of the outline. Therefore, in the process of step S314, the range specified as the light spots classified into the group corresponding to the maximum PC2 are continuously distributed in the image IMG is an object having a feature distinguished from the background or the like in the image IMG. This is a probable candidate area indicating

  On the other hand, when a value less than 0.5 is obtained as an index value indicating the independence of the maximal PC 1 shown in FIG. 4B in the process of step S341, in the process of step S342, the processor 21 The classification of the light spots having the clarity included in the range Rc1 in FIG. 4B is suppressed. In this case, the processor 21 leaves the light spots having the clarity included in the range Rc1 in FIG. 4B as unlighted light spots. That is, it is difficult to distinguish the light spot having the clarity included in the range Rc1 and the light spot having the clarity included in the range Rc0 shown in FIG. Are treated as unclassified light spots. Therefore, an uncertain region including a light spot having a possibility of showing the characteristics of another object, such as a range in which the light spot having the clarity included in the range Rc1 in FIG. 4B is distributed in the image IMG, It is not extracted as a candidate area by the process of step S314. In other words, prior to the processing in step S314, the processing in steps S341 and S342 is executed by the processor 21, so that extraction of uncertain candidate regions can be suppressed and probable candidate regions can be selectively extracted.

  In addition, the index value calculated | required by the process of step S341 shown in FIG. 15 is not limited to the value calculated based on the maximum value and the minimum value indicated by the frequency distribution of the feature value, but is classified based on the feature value. Any value indicating the degree of grouping of each group may be used.

  Next, based on FIG. 16, the recognition result obtained by recognizing an object included in the image IMG previously captured by the object recognition device 10 a illustrated in FIG. 13 is converted into a newly captured image. A method used for recognition of included objects will be described.

  FIG. 16 shows another example of processing for specifying the type of object. Step S320 to step S327, step S351, and step S352 shown in FIG. 16 are an example of the process of step S304 shown in FIG. 14, and are executed by the processor 21 shown in FIG. Of the steps shown in FIG. 16, those equivalent to the steps shown in FIG. 12 are denoted by the same reference numerals and description of the steps may be omitted. For example, the processing from step S320 to step S327 shown in FIG. 16 is equivalent to the step shown by the same reference numeral in FIG. Further, each process shown in FIG. 16 is executed for each candidate area extracted in the process of step S302 shown in FIG.

  In the flowchart shown in FIG. 16, in the case of an affirmative determination (YES) in step S322 equivalent to step S322 shown in FIG. 12, the processor 21 performs steps S351 and S351 prior to the processing in step S323 shown in FIG. The process of step S352 is executed.

  In step S351, the processor 21 uses the camera CAM1 shown in FIG. 13 to select a candidate area equivalent to the candidate area to be processed in the object recognition process of the image IMG taken at the previous shooting timing, for example. It is determined whether or not an object has been recognized. For example, the processor 21 associates information stored in the memory 22 or the like with the process of step S332 illustrated in FIG. 14 with a candidate area similar to the position and shape of the candidate area to be processed. Whether or not there is an equivalent candidate region is determined based on whether or not the is included.

  When information stored in the memory 22 or the like includes information indicating feature vector elements that contribute to specifying the type of object included in the candidate area similar to the position and shape of the candidate area to be processed In addition, the processor 21 makes an affirmative determination (YES) in step S351. In this case, the processor 21 executes the process of step S352. On the other hand, in the case of a negative determination (NO) in step S351, the processor 21 omits the process of step S352.

  In step S352, based on the information stored in the memory or the like in the process of step S332, the processor 21 selects the feature vector element as the feature value of the light spot used for narrowing down the search result in the process of step S324. Set. For example, the processor 21 sets the order in which the elements of the feature vector indicated by the information stored in the memory or the like in step S332 are selected before other elements.

  For example, when the information stored in the memory 22 or the like indicates that one type of object in the candidate area is specified by narrowing down based on the clarity of the outline of the light spot, the processor 21 Set the order in which are first selected. Therefore, the object type candidates retrieved from the feature database DB in the process of step S321 for the corresponding candidate region of the newly received image IMG are first narrowed down based on the element indicating the clarity included in the feature vector. .

  Here, in the recognition process for the image IMG of the previous frame, the feature vector elements narrowed down to one type of object in the candidate region are the feature database DB among the features of the object included in the candidate region. The feature is markedly different from the other objects registered in.

  Therefore, by narrowing down the search results obtained in the process of step S321 using the elements indicated by the information stored in the memory 22 or the like in the process of step S332, candidates can be obtained rather than using other elements first. The type of object in the area can be quickly narrowed down to one. That is, prior to the process of step S323, the processing of steps S351 and S352 is executed by the processor 21, so that the types of objects included in the candidate area are selected rather than selecting elements to be used for narrowing down in a fixed order. One can be identified at high speed.

  From the above detailed description, features and advantages of the embodiment will become apparent. It is intended that the scope of the claims extend to the features and advantages of the embodiments as described above without departing from the spirit and scope of the right. Any person having ordinary knowledge in the technical field should be able to easily come up with any improvements and changes. Therefore, there is no intention to limit the scope of the inventive embodiments to those described above, and appropriate modifications and equivalents included in the scope disclosed in the embodiments can be used.

Regarding the above description, the following items are further disclosed.
(Additional remark 1) Extracting the range which the light spot which has a mutually similar characteristic distributes as a candidate area | region which has a possibility of showing the said 1st object from the image of the 1st object in which the pattern containing a several light spot was projected And
Obtaining a first vector indicating characteristics of a plurality of light spots included in the candidate region;
Among the second vectors registered in advance indicating the characteristics of the plurality of light spots included in the pattern projected onto each of the plurality of second objects whose types of objects are known, the first of the candidate regions An object recognition method characterized by identifying a second vector similar to another second vector and recognizing a second object associated with the identified second vector as the type of the first object .
(Appendix 2) In the object recognition method according to appendix 1,
The process of extracting the candidate area is as follows:
Evaluating a plurality of types of feature quantities including the brightness of each light spot included in the image and the clarity of the outline of the light spot;
When the light spots are classified into a plurality of groups based on one of the plurality of types of feature amounts, a range in which a plurality of light spots belonging to each of the plurality of groups is continuously distributed in the image Detected as a candidate area,
When the light spots are not classified into a plurality of groups based on one of the feature quantities, the light spots are classified based on the other one of the feature quantities and the candidate area is detected. An object recognition method characterized by that.
(Supplementary note 3) In the object recognition method according to supplementary note 2,
The process for evaluating the feature amount of each light spot is information indicating the shape and size of each light spot in addition to the brightness of each light spot included in the image and the clarity of the outline of the light spot. Is calculated as a feature value of each light spot,
The process of detecting the candidate area includes the clarity of the outline of the light spot and the shape and size of each light spot when the light spot is not classified into a plurality of groups by classification based on the luminance of each light spot. Are sequentially executed as a first feature value, and the light spot is classified based on the selected first feature value.
(Supplementary Note 4) In the object recognition method according to Supplementary Note 1,
The process for obtaining the first vector includes:
Information indicating the distance from the reference plane on which the pattern is projected to each light spot projected onto the first object is calculated based on the position of each light spot included in the candidate region in the image. As part of a vector,
It is executed by obtaining information indicating a plurality of types of feature amounts including brightness of each light spot included in the candidate region and clarity of the outline of the light spot as another part of the first vector,
The process of recognizing the type of the first object is
When one second vector including information indicating a distance similar to information indicating the distance between each light spot and the reference point included in the candidate region is specified, the specified second vector Recognizing the associated second object as the type of the first object indicated by the candidate area;
When a plurality of second vectors are specified, second information including information similar to information indicating a plurality of feature amounts of each light spot included in the first vector is selected from the specified second vectors. An object recognition method, which is executed by narrowing down a second vector used for recognizing the type of the first object by selecting a vector.
(Additional remark 5) Extracting the range which the light spot which has a mutually similar characteristic distributes from the image of the 1st object as which the pattern containing a several light spot was projected as a candidate area | region which has a possibility of showing the said 1st object An extractor to perform,
A calculation unit for obtaining a first vector indicating characteristics of a plurality of light spots included in the candidate region;
Among the second vectors registered in advance indicating the characteristics of the plurality of light spots included in the pattern projected onto each of the plurality of second objects whose types of objects are known, the first of the candidate regions A recognizing unit that identifies a second vector similar to the other second vector to the vector, and recognizes a second object associated with the identified second vector as the type of the first object;
An object recognition apparatus comprising:
(Supplementary note 6) In the object recognition device according to supplementary note 5,
The extraction unit includes:
An evaluation unit that evaluates a plurality of types of feature quantities including the brightness of each light spot included in the image and the clarity of the outline of the light spot;
A classification unit that classifies the light spots included in the image into groups based on one of the plurality of types of feature amounts;
When the light spots included in the image are classified into a plurality of groups by the classification unit, a specifying unit that specifies a range in which the light spots included in each of the plurality of groups are continuously distributed in the image;
A detection unit that detects a range including a predetermined number or more of light spots among the range specified by the specifying unit as the candidate region;
Control that causes the classification unit to perform a process of classifying the light spot based on another one of the plurality of types of feature amounts when the light spot included in the image is not classified into a plurality of groups by the classification unit. An object recognizing device characterized by comprising:
(Supplementary note 7) In the object recognition device according to supplementary note 6,
The said classification | category part classifies the said light spot based on the brightness | luminance of the said light spot before the process which classifies the said light spot based on another feature-value. The object recognition apparatus characterized by the above-mentioned.
(Supplementary note 8) In the object recognition device according to supplementary note 5,
The calculation unit includes:
Information indicating the distance from the reference plane on which the pattern is projected to each light spot projected onto the first object is calculated based on the position of each light spot included in the candidate region in the image. As part of a vector,
Obtaining information indicating a plurality of types of feature amounts including the brightness of each light spot and the clarity of the outline of the light spot included in the candidate area as another part of the first vector,
The recognition unit
When one second vector including information indicating a distance similar to the distance to each of the plurality of light spots included in the candidate region is specified, a second associated with the specified second vector is specified. Recognizing an object as the type of the first object indicated by the candidate area;
When a plurality of second vectors are identified, a second vector including information similar to information indicating the characteristics of each light spot included in the first vector is selected from the identified second vectors. And narrowing down the second vector used for recognizing the type of the first object.
(Supplementary note 9) In the object recognition device according to supplementary note 8,
Receiving a plurality of images of the first object projected with a plurality of light spots;
The recognition unit
When a plurality of second vectors are identified as including information indicating distances similar to the distances to each of the plurality of light spots included in the candidate region, the candidate region extracted from the previously received image The second vector used for recognizing the type of the first object is narrowed down using the feature quantity obtained by narrowing the number of second vectors similar to the first vector obtained correspondingly to a number smaller than other feature quantities. An object recognition apparatus characterized by that.
(Supplementary Note 10) Candidates that may indicate the first object in a range in which light spots having similar characteristics are distributed from an image (IMG) of the first object onto which a pattern including a plurality of light spots is projected Extract as a region,
Obtaining a first vector indicating characteristics of a plurality of light spots included in the candidate region;
Among the second vectors registered in advance indicating the characteristics of the plurality of light spots included in the pattern projected onto each of the plurality of second objects whose types of objects are known, the first of the candidate regions Identifying a second vector that is more similar to the second vector than the other second vector, and recognizing a second object associated with the identified second vector as the type of the first object;
An object recognition program that causes a computer to execute processing.

DESCRIPTION OF SYMBOLS 10,10a ... Object recognition apparatus; 11, 11a ... Extraction part; 12 ... Calculation part; 13, 13a ... Recognition part; 111 ... Evaluation part; 112 ... Classification part; 113 ... Extraction control part; Detection unit; 131 ... Search unit; 132 ... Recognition control unit; 21 ... Processor; 22 ... Memory; 23 ... Storage device; 24 ... General-purpose interface; 25 ... Display device; 26 ... Optical drive device; CAM, CAM1 ... camera; PR, PR1 ... projection device; PT, PT1 ... pattern; T1, T2 ... object; DB ... feature database; BD ... bed; Q1 ... person; COM ... computer device; SV: Server device

Claims (5)

Possibility of indicating the first object in a range in which light spots with similar characteristics of luminance and contour clarity are distributed from the image of the first object onto which a pattern including a plurality of light spots is projected. Extracted as a candidate area
Obtains a first vector including the features of the plurality of light spots included in the candidate region,
In the second vector the type of the object is registered in advance, including each of the features of the plurality of light spots included in the plurality of the pattern projected on each of the second object is known, the said candidate region A second vector that is similar to one second vector than another second vector is identified, and a second object associated with the identified second vector is recognized as the type of the first object. Method. The object recognition method according to claim 1,
The process of extracting the candidate area is as follows:
Evaluating a plurality of types of feature quantities including the brightness of each light spot included in the image and the clarity of the outline of the light spot;
When the light spots are classified into a plurality of groups based on one of the plurality of types of feature amounts, a range in which a plurality of light spots belonging to each of the plurality of groups is continuously distributed in the image Detected as a candidate area,
When the light spots are not classified into a plurality of groups based on one of the feature quantities, the light spots are classified based on the other one of the feature quantities and the candidate area is detected. An object recognition method characterized by that. The object recognition method according to claim 1,
The process for obtaining the first vector includes:
Information indicating the distance from the reference plane on which the pattern is projected to each light spot projected onto the first object is calculated based on the position of each light spot included in the candidate region in the image. As part of a vector,
It is executed by obtaining information indicating a plurality of types of feature amounts including brightness of each light spot included in the candidate region and clarity of the outline of the light spot as another part of the first vector,
The process of recognizing the type of the first object is
When one second vector including information indicating a distance similar to the information indicating the distance between each light spot and the reference plane included in the candidate region is specified, the specified second vector Recognizing the associated second object as the type of the first object indicated by the candidate area;
When a plurality of second vectors are specified, second information including information similar to information indicating a plurality of feature amounts of each light spot included in the first vector is selected from the specified second vectors. An object recognition method, which is executed by narrowing down a second vector used for recognizing the type of the first object by selecting a vector. Possibility of indicating the first object in a range in which light spots with similar characteristics of luminance and contour clarity are distributed from the image of the first object onto which a pattern including a plurality of light spots is projected. An extraction unit for extracting as a candidate area having,
A calculation unit for obtaining a first vector including the features of the plurality of light spots included in the candidate region,
In the second vector the type of the object is registered in advance, including each of the features of the plurality of light spots included in the plurality of the pattern projected on each of the second object is known, the said candidate region A recognizing unit that identifies a second vector that is more similar to one second vector than another second vector, and recognizes a second object associated with the identified second vector as the type of the first object;
An object recognition apparatus characterized by comprising: Possibility of indicating the first object in a range in which light spots with similar characteristics of luminance and contour clarity are distributed from the image of the first object onto which a pattern including a plurality of light spots is projected. Extracted as a candidate area
Obtains a first vector including the features of the plurality of light spots included in the candidate region,
In the second vector the type of the object is registered in advance, including each of the features of the plurality of light spots included in the plurality of the pattern projected on each of the second object is known, the said candidate region A second vector that is more similar to one vector than another second vector is identified, and a second object associated with the identified second vector is recognized as the type of the first object;
An object recognition program that causes a computer to execute processing.

Images