JP2015173344A - object recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object recognition device with high accuracy having higher recognition accuracy compared with that of a prior art, which can reduce the arithmetic processing cost and has enhanced robustness against appearance changes of an object.SOLUTION: An object recognition device comprises: a camera which captures an image of a recognition object; a plurality of light sources which are arranged in the periphery of the camera, and each of which blinks independently in a selective manner; a control unit which controls the blinking of each of the light sources so as to send an imaging trigger signal to the camera in synchronous or asynchronous with the blinking of the light source, thereby sending light source ID information to a code generation unit; the code generation unit which generates a code on the basis of a luminance value of each image captured by the camera upon turning on or off each of the light sources; a template storage unit which stores information on a predetermined reference object; and a code collation unit which collates a code generated from an image group obtained when capturing the images of the recognition object with a code generated from the information stored in the template storage unit so as to output a collation result.

Description

  The present invention relates to an object recognition apparatus in which an object (recognition target object) to be recognized is registered in advance, and an existing location is recognized from a camera-captured image.

  The process of registering a specific pattern of an object to be recognized in advance and detecting its presence and position from the image is one of the basic techniques of image processing, and is widely applied in image recognition and image inspection. Has been. Here, a pattern registered in advance is referred to as a template, and a process of matching a template with an image is referred to as template matching or pattern matching. An input image in which a recognition target is shown is called a reference image.

  In template matching, the similarity or difference is evaluated while scanning the template image on the reference image, and the position of the recognition object is detected. As this evaluation scale, SAD (Sum of Absolute Difference) or SSD (Sum of Squared Difference) is often used (for example, see Non-Patent Document 1). These are all evaluation scales based on the luminance value of the image.

  Chamfer matching has been proposed as a template matching algorithm based on the similarity of the shape of an edge (contour) rather than the luminance value of an image (see Non-Patent Document 2). As preprocessing for collation, a Sobel filter, a Canny edge detector, or the like is applied to the reference image to extract edges in the image. Then, the object is recognized by collating the edge of the template with the edge of the reference image.

  Patent Document 1 discloses a method for extracting the contour of an object using a plurality of light sources capable of active control. A plurality of light sources attached around the camera are blinked one by one, and an image of the object is acquired by the camera in synchronization therewith. Then, a shadow is generated or disappears at a place where the three-dimensional shape exists. Since this shadow flickering depends on the illumination direction and the physical shape of the object, the discontinuity of the depth of the object, that is, the contour of the physical shape, can be determined by examining the change in the flickering of each shadow in each image. It can be extracted as an edge.

JP 2004-288185 A

Edited by Masatoshi Okutomi et al., “Digital Image Processing”, Section 12.1 Pattern Detection, pp. 202-204, CG-ARTS Association, 2004 G. Borgfors et al. , “Hierarchical campaign matching: a parametric edge matching algorithm”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 10, no. 6, pp. 849-865, 1988.

  The luminance value of an image photographed by the camera depends not only on the shape of the object but also on the density of the texture of the object surface, the reflectance, the illumination conditions, and the like. For this reason, when trying to recognize an object by template matching based on a luminance value, there is a possibility that the recognition performance is deteriorated due to factors other than the shape of the object. Similarly, in object recognition by chamfer matching, if the contour of the object to be detected is not extracted as an edge due to factors other than the shape of the object, or a portion other than the contour, such as the texture of the object surface, is a false contour May be extracted. In such a situation, it is known that the recognition accuracy of chamfer matching is greatly reduced.

  The method of extracting the contour of the object by active control of the light source does not take into account the recognition of the object. Therefore, it is necessary to combine the object recognition with another means such as chamfer matching. That is, a plurality of images obtained by photographing an object by changing the direction of light irradiation have useful information regarding the three-dimensional shape of the object, but the information is directly used for matching processing. There wasn't.

  Furthermore, in the above-described general template matching framework, a template having the highest similarity is evaluated by changing the position of the template (and, in some cases, the size and angle) while evaluating the similarity or dissimilarity with the reference image. Find the position (and size, angle) of. At this time, the template is handled as a two-dimensional plane pattern. On the other hand, when the recognition target is a three-dimensional object having a depth, even when the relative positional relationship between the camera and the recognition target is slightly shifted, the appearance when the target is viewed from the camera may change greatly. is there. Specifically, as the depth of the recognition target object approaches the working distance (distance between the camera and the recognition target object), the change in appearance due to the change in the positional relationship increases. In order to detect such a three-dimensional object using a general template matching framework, it is sufficient to hold a large number of templates corresponding to various changes in appearance and repeat the matching process while switching the templates. There is a problem that the processing cost becomes enormous.

  The object of the present invention is to solve the above-mentioned problems, have high recognition accuracy compared to the prior art, reduce the processing cost, and highly accurate object recognition with improved robustness against changes in the appearance of the object To provide an apparatus.

  An object recognition apparatus according to an aspect of the present invention includes a camera that captures a recognition target, a plurality of light sources that are arranged around the camera and each selectively flashes independently, and controls the flashing of each light source. A control unit that sends an imaging trigger signal to the camera in synchronization or asynchronously with the blinking of each light source and sends light source ID information to the code generation unit, and the luminance of each image taken by the camera when each light source is turned on or off A code generation unit that generates a code from a value, a template storage unit that stores information on a predetermined reference object, and a code that is generated from information stored in the template storage unit that is generated from an image group obtained by photographing a recognition object And a code collating unit that collates and outputs a collation result.

  In the present invention, an image is acquired while changing the blinking of a plurality of light sources arranged around the camera, and a code is generated from the pixel values of the acquired images. Since this code has information related to the three-dimensional shape of the object, by executing this code collation, that is, template matching, the object is more accurate, faster, and more robust against disturbance than the prior art. Can be recognized.

It is a block diagram which shows the structure of the object recognition apparatus which concerns on Embodiment 1 of this invention. It is explanatory drawing which showed an example of the light source direction information table which the code | symbol production | generation part with which the object recognition apparatus shown in FIG. It is the schematic diagram which showed an example of the external appearance of the object recognition apparatus shown in FIG. It is the schematic diagram which showed the structure at the time of image | photographing a rectangular parallelepiped with the object recognition apparatus shown in FIG. It is the schematic diagram which showed an example of the image image | photographed with a camera, when using the object recognition apparatus shown in FIG. 1 and image | photographing a rectangular parallelepiped by irradiating light from the top with the structure shown in FIG. It is the schematic diagram which showed an example of the image image | photographed with a camera, when using the object recognition apparatus shown in FIG. 1 and image | photographing a rectangular parallelepiped by irradiating light from the top with the structure shown in FIG. It is the figure which showed an example of the image of the brightness | luminance gradient produced | generated by the code | symbol production | generation part with which the object recognition apparatus shown in FIG. 1 is equipped, Comprising: (a) is a perspective view of the image irradiated with light from the top, (b) FIG. 4 is a schematic diagram of an image of a luminance gradient at that time. It is the figure which showed an example of the image of the brightness | luminance gradient produced | generated by the code | symbol production | generation part with which the object recognition apparatus shown in FIG. 1 is equipped, Comprising: (a) is a perspective view of the image irradiated with light from the lower left, (b) FIG. 4 is a schematic diagram of an image of a luminance gradient at that time. It is the schematic diagram which showed the code | symbol produced | generated by the code | symbol production | generation part with which the object recognition apparatus shown in FIG. 1 is equipped. It is the schematic diagram which showed an example of the flow which the code | symbol production | generation part with which the object recognition apparatus which concerns on Embodiment 2 of this invention is equipped produces | generates a code | symbol. It is the schematic diagram which showed an example of the flow which the code | symbol production | generation part with which the object recognition apparatus which concerns on Embodiment 3 of this invention is equipped produces | generates a code | symbol. It is the schematic diagram which showed an example in which the code | symbol production | generation part with which the object recognition apparatus which concerns on Embodiment 3 of this invention is equipped collates a code | symbol and calculates a difference degree. It is the schematic diagram which showed an example of the flow which the code generation part with which the object recognition apparatus which concerns on Embodiment 4 of this invention is equipped produces | generates a ternary code. It is the schematic diagram which showed an example in which the code | symbol collation part with which the object recognition apparatus which concerns on Embodiment 4 of this invention is equipped collates ternary code and calculates a difference degree. It is the schematic diagram which showed an example of the flow which the code | symbol production | generation part with which the object recognition apparatus which concerns on Embodiment 5 of this invention is equipped produces | generates a gradient direction code | symbol. It is the schematic diagram which showed an example of the flow which the code | symbol production | generation part with which the object recognition apparatus which concerns on Embodiment 6 of this invention is equipped produces | generates the code | symbol based on a gradient direction feature-value. It is a block diagram which shows the structure of the object recognition apparatus which concerns on Embodiment 7 of this invention. It is a schematic diagram showing the image at the time of image | photographing a rectangular parallelepiped with the object recognition apparatus which concerns on Embodiment 8 of this invention, and the attention point in an image. It is explanatory drawing showing the gradient intensity | strength corresponding to two attention points shown in FIG. It is explanatory drawing which shows the procedure in which the code | symbol production | generation part with which the object recognition apparatus which concerns on Embodiment 8 of this invention is equipped calculates | requires stability based on the binary code | symbol of a gradient strength order. It is the schematic diagram which showed an example of the external appearance at the time of arrange | positioning the light source of the object recognition apparatus shown in FIG. 1 in concentric multiple circle shape. In the object recognition apparatus concerning Embodiment 9 of this invention, it is a figure which shows that the range which a shadow produces differs with the positional relationship of a camera, a light source, and an object, Comprising: It is explanatory drawing when the position of a camera and a light source is near. . In the object recognition apparatus concerning Embodiment 9 of this invention, it is a figure which shows that the range which a shadow produces differs with the positional relationship of a camera and a light source and an object, Comprising: Explanatory drawing when the position of a camera and a light source is separated It is. In the object recognition apparatus concerning Embodiment 9 of this invention, it is a figure which shows that the range which produces a shadow by the positional relationship of a camera, a light source, and an object is different, Comprising: Explanatory drawing when the position of an object and a background is separated It is.

  Hereinafter, embodiments according to the present invention will be described with reference to the drawings. In addition, in each following embodiment, the same code | symbol is attached | subjected about the same component. In the embodiment according to the present invention, an object is obtained by irradiating light from different directions on an object, and matching is performed using codes generated from these images, so that the object is compared with the related art. A highly accurate object recognition device with improved robustness against changes in the appearance of the object.

Embodiment 1 FIG.
FIG. 1 shows a configuration of an object recognition apparatus 100 according to the first embodiment. In FIG. 1, an object recognition apparatus 100 according to Embodiment 1 includes a camera 101, a plurality of N light sources 102-1 to 102-N, a control unit 103, a code generation unit 104, and a template storage unit 105. The code verification unit 106 is configured. These components will be described in detail below.

  In FIG. 1, when receiving a control signal from the control unit 103, the camera 101 captures an image of the recognition target object and sends image data of the captured image to the code generation unit 104. The light sources 102-1 to 102-N irradiate the recognition target with light. Each of the light sources 102-1 to 102-N receives the control signal from the control unit 103 and sequentially and selectively blinks one by one. That is, it is assumed that two or more of the N light sources 102-1 to 102-N are not lit simultaneously. Here, N light sources 102 are numbered in order from No. 1 to N, and these numbers are hereinafter referred to as light source IDs.

  The control unit 103 is a controller such as a digital computer, for example, which sends a control signal to one of the N light sources 102-1 to 102-N and simultaneously sends a control signal to the camera 101. In addition, the light source ID information is sent to the code generation unit 104. This operation is sequentially performed one by one for each of the N light sources 102-1 to 102-N. The code generation unit 104 generates a code from N pieces of image data received from the camera 101. Note that the code generation unit 104 associates the light source ID of each of the light sources 102-1 to 102-N with the irradiation direction of light from the light sources 102-1 to 102-N, as illustrated in FIG. Assume that the table is held in the internal memory 104m. Information in the light source direction information table is used in internal processing of the code generation unit 104 described later. Hereinafter, the irradiation direction of light from the light source is referred to as a light source direction. Note that the light irradiation direction indicates the angle of the position of each of the light sources 102-1 to 102-N around the camera 101 as shown in FIG. 3, for example.

  The template storage unit 105 stores the template code of the recognition target object. A code of a recognition object to be recognized by the object recognition apparatus 100 is generated in advance and held as a template code. The code matching unit 106 reads the template code stored in the template storage unit 105 and performs matching with the code of the reference image generated by the code generation unit 104, that is, template matching processing. By collating the template code with the code of the reference image, the position of the recognition object in the reference image is recognized, and the recognition result is output.

  FIG. 3 is a schematic diagram illustrating an appearance of the object recognition apparatus 100 according to Embodiment 1 and an example in which a light source ID is assigned to each of the light sources 102-1 to 102-N. In the example of FIG. 3, N = 8, and the eight light sources 102-1 to 102-8 are arranged so as to be equidistant from the optical axis of the camera 101 and equidistant from each other. The light source IDs of the respective light sources 102-1 to 102-8 are numbered in order from the light source 102-1 above the camera 101 in the clockwise direction. Note that the arrangement of the N light sources 102-1 to 102-8 does not necessarily need to be equidistant as shown in FIG. 3 and need not be equidistant from the optical axis of the camera 101. Moreover, it is not necessary to make it the housing | casing which integrated the camera 101 and the light sources 102-1 to 102-8 like the example of FIG. 3, and it is good also as a separate housing | casing.

  FIG. 4 is a schematic diagram showing a state in which the cuboid 202 is photographed by the object recognition device 100 from directly above the scene in a scene in which the cuboid 202 of the recognition target object is arranged on the flat background 201. FIG. 5A and FIG. 5B are schematic diagrams of images taken in this situation while selectively switching the blinking of the light sources 102-1 to 102-N included in the object recognition apparatus 100 sequentially. FIG. 5A shows an example in the case where light is irradiated from above the image to the rectangular parallelepiped 202, and FIG. 5B shows an example in the case of a captured image in which light is irradiated from the lower left direction. By irradiating the rectangular parallelepiped 202 with light, a shadow 203 is generated on the flat background 201 on the back of the rectangular parallelepiped 202. By changing the light irradiation direction of the light source 102, the position of the shadow 203 appearing around the rectangular parallelepiped 202 changes.

  Next, the flow in which the object recognition apparatus 100 according to Embodiment 1 of the present invention performs object recognition will be described.

  First, in order to register an object to be recognized by the object recognition apparatus 100 as a template, a template code is generated using a reference object.

  An object to be recognized (reference object) is placed in the field of view of the camera 101, and images are captured by the camera 101 while sequentially turning on the N light sources 102-1 to 102-N one by one under the control of the control unit 103. Acquire N data. It is assumed that the relative positional relationship between the camera 101 and the recognition target object remains unchanged while N pieces of image data are captured. The code generation unit 104 generates a code from the N pieces of image data thus obtained. The generated code is sent to the template storage unit 105 and stored in the template storage unit 105 as a template code.

  The flow for performing object recognition is the same up to code generation in the code generation unit 104. That is, a recognition target scene is arranged in the field of view of the camera 101, N reference images are photographed while sequentially switching the blinking of the light sources 102-1 to 102-N, and a code generation unit 104 generates a code. Thereafter, the code collation unit 106 performs collation between the code stored in the template storage unit 105 and the code generated from the N reference images, and recognizes the position of the object.

  The above is a flow in which the object recognition apparatus 100 pre-registers a template and a flow in which object recognition is executed.

  Next, a code generation procedure by the code generation unit 104 will be described.

First, an object is placed in the field of view of the camera 101, and an image is acquired by the camera 101 while selectively turning on the N light sources 102-1-1 to 102-N one by one. The N pieces of image data thus obtained are set as I ₁ to I _N , respectively. Thereafter, for each of the image data I ₁ to I _N , the intensity of the luminance gradient in the direction along the light source direction (the absolute value of the luminance difference between adjacent pixels is referred to as “luminance gradient intensity”) is calculated. image data of the brightness gradient (hereinafter, referred to as "intensity gradient image data".) Request _{'I N from'} I 1. Here, the light source direction is obtained by referring to a light source direction information table as shown in FIG. 6A and 6B are schematic diagrams of an image (a) photographed by the camera 101 and a luminance gradient image (b) thereof. 6A and 6B are schematic diagrams of luminance gradient images for two light source directions, respectively, as in FIGS. 5A and 5B. For example, the luminance gradient image of an image when light is irradiated from the lower left is an image expressing the absolute value of the luminance difference of the lower left pixel.

In the luminance gradient image, the gradient intensity increases at the boundary portion between the region brightly illuminated by the light irradiation and the shadow region, and an edge appears in this portion. This edge appears along the part where the depth of the object changes rapidly, that is, along the contour of the physical shape. Therefore, it can be considered that the difference in intensity gradient when light is applied from different directions provides information on the direction in which the depth shape of the object is changing. The code generated by the code generation unit 104 is expressed by an N-dimensional vector for each pixel. If the luminance gradient intensity at the coordinates (x, y) of the luminance gradient image data I ′ _{k with} the light source ID k is I ′ _k (x, y), the code u (x, y) assigned to the coordinates (x, y). ) Is expressed by the following equation.

  Here, the symbol T represents transposition. Codes are generated as described above for all pixels in the image. The above is the procedure in which the code generation unit 104 generates a code from the captured image.

  FIG. 7 is a schematic diagram in which the code generated by the code generation unit 104 is expressed as a grayscale image from N pieces of image data captured by the camera 101. In a pre-registration stage in which a photographed object is registered as a reference object, a range including a recognition target object (illustrated by a dotted line) is cut out from the code and stored in the template storage unit 105.

  Next, a procedure for recognizing the position of the object by collating the code in the code collating unit 106 will be described. As described above, the code collating unit 106 collates the code generated from the captured image (reference image) of the recognition target scene with the code stored in the template storage unit 105 and minimizes the difference between the codes. Find the position where.

  Now, let s (i, j) be the sign of image data at coordinates (i, j), which is composed of the gradient strength obtained from the reference image. The template code of coordinates (i, j) stored in the template storage unit 105 is t (i, j), and the size is set to horizontal W pixels and vertical H pixels. At this time, when the inner product of two code vectors is expressed as <,>, the degree of difference is obtained by the sum of the inner products, that is, the following equation.

Then, the position of the object of the recognition target is obtained as coordinates (x, y) that minimizes the above equation (1). If the size (scale) of the recognition target object on the image changes between pre-registration and object recognition, the code matching unit 106 may change the scale of the template code and evaluate the degree of difference. . For example, the template code t (i, j) is divided into components t ₁ (i, j), t ₂ (i, j),... And expanded by using a two-dimensional nearest neighbor method or a spline interpolation method for each component. -Template codes with different scales can be obtained by reduction. Matching is repeated while switching these template codes, and the position and scale at which the degree of difference is minimized are obtained.

  According to the first embodiment configured as described above, the code generated by the code generation unit 104 has information regarding the intensity gradient intensity when light is applied from different directions. This luminance gradient depends on the direction of light and the physical shape of the object. That is, in the code generation unit 104, effective information regarding the physical shape of the target object is assigned as a code for each pixel. If template matching is performed using this code, highly accurate object recognition based on the solid shape information of the object is performed. It can be realized.

  In the above embodiment, the blinking of each of the light sources 102-1 to 102-N is controlled and an imaging trigger signal is sent to the camera 101 in synchronization with the blinking of each of the light sources 102-1 to 102-N. The present invention is not limited to this, and an imaging trigger signal is sent to the camera 101 asynchronously with the blinking of each of the light sources 102-1 to 102-N (however, correspondence with each of the light sources 102-1 to 102-N is necessary). May be.

Embodiment 2. FIG.
The code generation unit 104 provided in the object recognition apparatus 100 according to Embodiment 1 may generate the order of the luminance gradient strength as a code in Embodiment 2. The difference from the code generation unit 104 provided in the object recognition apparatus 100 according to Embodiment 1 is that the gradient strength is calculated from N pieces of image data captured by the camera 101 and then the rank of the gradient strength is obtained.

）と表記すると、座標（ｘ，ｙ）に割り当てる符号ｒ（ｘ，ｙ）を次式で表す。 FIG. 8 is a diagram showing an example of the luminance gradient strength of a certain coordinate and its rank in the case of N = 8. Ranking is performed from 1 to 8 in order from the light source ID corresponding to the direction of the light source having the high intensity gradient intensity. The code generated by the code generation unit 104 according to Embodiment 2 is expressed by an N-dimensional vector for each pixel. The image data of the coordinates (x, y), the ranking source ID is the gradient strength of _{k r k (x, y)} ( although

), The code r (x, y) assigned to the coordinates (x, y) is expressed by the following equation.

  Codes are generated as described above for all pixels in the image data. The above is the procedure in which the code generation unit 104 generates a code from the captured image.

  The verification of the code by the code verification unit 106 is executed by evaluating the difference in the order of the brightness gradient intensities as the degree of difference. Now, the sign in the image data of the coordinates (i, j), which is constituted by the luminance gradient strength obtained from the reference image data, is expressed by the following equation.

  The template code of coordinates (i, j) stored in the template storage unit 105 is expressed by the following equation.

  The size is set to horizontal W pixels and vertical H pixels. The degree of difference at this time is obtained as the sum of the differences in the order of the luminance gradient intensities, that is, the following equation.

  Then, the position of the object of the recognition target is obtained as coordinates (x, y) that minimizes the above equation (2). When the size (scale) of the recognition target object on the image changes between pre-registration and object recognition, the scale of the template code varies as in the code matching unit 106 according to the first embodiment. And evaluate the degree of difference.

  According to the second embodiment configured as described above, since the code generated by the code generation unit 104 is the rank of the gradient strength, the gradient strength varies within a range in which the gradient strength rank does not change. And robust object recognition.

Embodiment 3 FIG.
The code generation unit 104 included in the object recognition apparatus 100 according to the first embodiment generates a binary code that represents the result of classifying the luminance gradient strength into two stages in the third embodiment, and the code matching unit. 106 may be configured to collate the binary code generated by the code generation unit 104 based on the Hamming distance.

  The code generated by the code generation unit 104 according to Embodiment 3 is expressed by an N-bit binary number for each pixel. The N bits constituting the code of each pixel correspond to the light source IDs 1 to N, respectively. The difference from the code generated by the code generation unit 104 according to the second embodiment is that the rank of the luminance gradient strength for each pixel is obtained, and then the rank is classified into two stages, upper and lower, and the result is binarized. It is a point used as a sign. The light source ID that includes the rank of the luminance gradient intensity at the upper level sets the corresponding binary code bit to 1, and the light source ID that is included at the lower level sets the corresponding binary code bit to 0. By such a procedure, a binary code consisting of N bits is generated for all the pixels in the image data.

FIG. 9 shows an example in which a binary code is generated by classifying a luminance gradient intensity of a pixel, its rank, and further rank into two stages for N = 8. The flow of processing until the gradient strength rank rk is _obtained is the same as that shown in FIG. In this embodiment, after obtaining the rank of the gradient intensity, 1 is set in the corresponding bit for the upper four light source IDs (2, 3, 4, 6), and the lower four light source IDs (1, For 5, 7, 8), set the corresponding bit to 0. The 8 bits thus obtained becomes the binary code of this pixel. Code verification by the code verification unit 106 is performed by evaluating the degree of difference based on the Hamming distance between the code generated from the reference image and the template code. The Hamming distance is the number of bits having different values when two binary numbers having the same bit length are compared.

  FIG. 10 shows an example of obtaining the Hamming distance for a certain pixel from the binary code of the template and the binary code generated from the reference image. A portion where the values are the same is marked with a circle, and a portion where the values are different is marked with a cross. In the case of the example in FIG. 10, the number of bits having different values is four, so the hamming distance of this pixel is four.

Now, let S be a code generated from the reference image data. In addition, the template code stored in the template storage unit 105 is T, and its size is W × H. When the Hamming distance between two binary numbers b ₁ and b ₂ is expressed as H (b ₁ , b ₂ ), the degree of difference is obtained by the sum of the Hamming distances, that is, the following equation.

Then, the position of the object of the recognition target is obtained as coordinates (x, y) that minimizes the above equation (3). The Hamming distance H (b ₁ , b ₂ ) can be obtained at high speed by the following procedure. The number of digits whose value is 1 when the decimal number a (0 ≦ a <2 ^N ) is converted into a binary number is stored in the table t (a) of size 2 ^N in advance. For example, when the decimal number 151 is converted into a binary number, it is 10010111, and the number of digits that become 1 is 5, so t (151) = 5.

If a value obtained by converting the exclusive OR of the binary numbers b ₁ and b ₂ into a decimal number is expressed as XOR ₁₀ (b ₁ , b ₂ ), the Hamming distance between the two binary numbers b ₁ and b ₂ is By referring to the table t (·), the following equation is obtained.

  Thus, by creating a table in advance, it is possible to calculate the Hamming distance at high speed.

  In addition, when the size (scale) of the recognition target object on the image changes between pre-registration and object recognition, the scale of the template code T (i, j) is changed in the code matching unit 106 to make a difference. The degree should be evaluated. For example, templates with different sizes can be obtained by enlarging / reducing T (i, j) by the two-dimensional nearest neighbor method. Matching is repeated while switching between these templates, and the position and scale at which the degree of difference is minimized are obtained.

  According to the third embodiment configured as described above, the code generated by the code generation unit 104 is a binarized result of classifying the gradient strength rank into two stages, upper and lower, It is possible to perform object recognition robustly not only with respect to fluctuations in gradient strength caused by disturbances such as object position deviation and noise, but also with respect to changing the order.

  Furthermore, since the evaluation of the degree of difference by the code matching unit 106 is based on the Hamming distance, it can be realized as a calculation based on a bit calculation and a table reference, so that a code check can be executed at high speed on a digital computer.

Embodiment 4 FIG.
The code generation unit 104 provided in the object recognition apparatus 100 according to the first embodiment is a ternary code that represents the result of classifying the luminance gradient strength into the upper, middle, and lower levels in the fourth embodiment. The code verification unit 106 may be configured to verify the ternary code generated by the code generation unit 104. The code generated by the code generation unit 104 according to the fourth embodiment is expressed by an N-digit ternary code per pixel.

FIG. 11 is a diagram illustrating an example of a flow of generating an N-digit ternary code from a luminance gradient intensity at a certain coordinate in the case of N = 8. The flow of processing until the luminance gradient strength rank rk is _obtained is the same as that shown in FIGS. In the present embodiment, after obtaining the rank of the luminance gradient intensity, 1 is set in the corresponding digit for the two light source IDs (2, 3) with the highest rank, and the middle four light source IDs (1, 4, 4). 5 and 6), the symbol * is set, and for the lower two light source IDs (7 and 8), the corresponding bits are set to 0. In FIG. 11, the numbers of light source IDs classified into upper, middle, and lower levels are 2, 4, and 2, respectively, but the distribution of the numbers classified into upper, middle, and lower levels is not limited to this.

  Next, a procedure for collating the ternary code generated by the code generation unit 104 according to Embodiment 4 of the present invention with the code verification unit 106 will be described. The difference from the collation in the code collation unit 106 according to Embodiment 3 of the present invention is that the code * corresponding to the light source ID classified in the middle rank is regarded as a wild card. A wild card is a code that matches any code (the degree of difference is 0).

  FIG. 12 shows an example of collating a ternary code of a template with a ternary code generated from a reference image for a certain pixel. A portion where the values are the same is marked with a circle, and a portion where the values are different is marked with a cross. As described above, the wildcard code * is defined as a code that matches any code. In the case of the example in FIG. 9, the number of bits having different values is 2, so the degree of difference between the pixels is 2.

In the code matching unit 106 provided in the object recognition apparatus 100 according to Embodiment 3 of the present invention, the degree of difference is defined based on the Hamming distance. In the present embodiment, the concept of the degree of difference is extended to a ternary code as shown in FIG. 12, and this is called an extended Hamming distance. The ternary code generated from the reference image is S ′, the template ternary code stored in the template storage unit 105 is T ′, and the size is W × H. When the extended Hamming distance of the _two ternary codes s ₁ and s ₂ is expressed as H ′ (s ₁ , s ₂ ), the position of the object is obtained as coordinates (x, y) that minimize the following expression.

  When the size (scale) of the recognition target object on the image changes between pre-registration and object recognition, the scale of the template code is changed as in the code matching unit 106 according to the third embodiment, What is necessary is just to evaluate a difference degree.

  According to the fourth embodiment configured as described above, the code generated by the code generation unit 104 matches a code that becomes a wild card with respect to the light source ID having the middle gradient intensity, that is, matches any code. Therefore, object recognition can be performed robustly without affecting the evaluation of the degree of difference even when the gradient strength changes in the middle order (around N / 2). Become.

Embodiment 5 FIG.
The code generation unit 104 provided in the object recognition apparatus 100 according to the first embodiment obtains the light source direction that maximizes the luminance gradient in the fifth embodiment, encodes this direction, and the code verification unit 106 You may comprise so that a dissimilarity may be evaluated based on the difference of the direction (angle) of a gradient.

  FIG. 13 is a diagram illustrating an example of a flow of generating a light source direction code from a luminance gradient intensity at a certain coordinate in the case of N = 8. Here, the flow until the luminance gradient intensity is obtained is the same as that shown in FIGS. In the code generation unit 104 according to the present embodiment, the light source direction that maximizes the gradient intensity is obtained, and the corresponding light source ID is used as the code of this pixel (light source direction code).

The code matching unit 106 evaluates the degree of difference based on the angle difference in the light source direction between the code generated from the reference image and the template code. The code generated from the reference image is S ″, the template code stored in the template storage unit 105 is T ″, and the size is W × H. The corresponding light source direction is obtained by referring to the light source direction information table in the internal memory 104m shown in FIG. 2 using the light source direction at the coordinates (i, j) of the reference image, that is, the code S ″ (i, j) as a key. 2 is represented as θ _s (i, j) .Similarly, the light source direction at the coordinates (i, j) of the template code, ie, the light source direction information table shown in FIG. The obtained light source direction is _expressed as θ _t (i, j). Then, the position of the object is obtained as a sum of angular differences between the light source direction θ _s of the reference image and the light source direction θ _t of the template, that is, coordinates (x, y) that minimize the following equation.

  Here,% is an operator representing a remainder. Also, min (X, Y) means a minimum value selection function that selects the smaller of X and Y. When the size (scale) of the recognition target object on the image changes between pre-registration and object recognition, the code matching unit 106 sets the scale of the template code in the same manner as described in the third embodiment. It can be varied and the degree of difference can be evaluated.

According to the fifth embodiment configured as described above, the code generation processing in the code generation unit 104 selects the light source ID having the highest luminance gradient intensity among the N luminance gradient intensity values for each pixel. Therefore, the code can be generated at high speed. Further, since the code generated by the code generation unit 104 can be expressed by log ₂ N bits for each pixel, it can be a compact code.

Embodiment 6 FIG.
The code generation unit 104 provided in the object recognition apparatus 100 according to Embodiment 1 of the present invention obtains the direction of the luminance gradient for each pixel in the image data obtained by lighting each light source in Embodiment 6, and performs encoding. The code matching unit 106 may be configured to evaluate the degree of difference based on the difference in the direction (angle) of the luminance gradient. The code generated by the code generation unit 104 according to the present embodiment is expressed by an N-dimensional vector for each pixel. The flow for generating the code is as follows.

  In the present embodiment, in order to simplify the description, it is assumed that the light sources 102-1 to 102-8 included in the object recognition apparatus 100 are arranged at equal intervals in a concentric manner as shown in FIG. Even in the case where the arrangement of the light sources 102-1 to 102-8 is not equidistant, the processing described below can be realized by appropriately making necessary changes according to the intervals of the light sources 102-1 to 102-8. It is. In advance, the entire circumference of 360 degrees is divided into equal intervals of (360 / N) degrees, and codes from 0 to N-1 are assigned to the respective ranges, which are used as gradient direction feature quantities.

First, image data of an image is acquired by the camera 101 while selectively turning on the N light sources 102-1 to 102-N included in the object recognition device one by one sequentially. The N pieces of image data obtained in this way are set as I ₁ to I _N , respectively. Thereafter, the direction of the luminance gradient is determined for each pixel for each of the image data I ₁ to I _N. Assuming that the luminance gradient in the x-axis direction (lateral direction) of the image data I is I ′ _x and the luminance gradient in the y-axis direction (vertical direction) is I ′ _y , the gradient direction of the coordinates (x, y) is obtained by the following equation. It is done.

  Here, the function “atan2” represents an arctangent function having a four-quadrant expression, and is prepared as a standard in a general programming language such as C language. By quantizing the gradient direction information obtained in this way into N stages, the gradient direction feature quantity is obtained and used as the code of the image data at the coordinates (x, y).

（ただし

）と表記すると、座標（ｘ，ｙ）の画像データに割り当てる符号

を次式で表す。 FIG. 14 shows the luminance gradient direction of a certain pixel and the luminance gradient direction feature amount corresponding to N = 8. The correspondence relationship between the luminance gradient direction and the feature amount is shown on the right side of FIG. The gradient direction feature amount in the image data of coordinates (x, y) of the photographed image data I _k when the light source 102-k with the light source ID k is turned on.

(However,

) Is a code assigned to image data at coordinates (x, y)

Is expressed by the following equation.

  Codes are generated as described above for all pixels in the image. The above is the procedure in which the code generation unit 104 generates a code from the captured image. Code verification by the code verification unit 106 is performed by evaluating the difference in luminance gradient direction (angle) as the degree of difference. Now, the sign in the image data of the coordinates (i, j), which is constituted by the feature quantity in the luminance gradient direction obtained from the reference image data, is expressed by the following equation.

  Further, the template code of the image data at the coordinates (i, j) stored in the template storage unit 105 is expressed by the following equation.

  The size is set to horizontal W pixels and vertical H pixels. At this time, the degree of difference is obtained as a sum of differences in the gradient direction, that is, as the following equation.

  And the position of an object is calculated | required as a coordinate (x, y) which minimizes said Formula (4).

を成分

に分割し、成分毎に２次元最近傍法を用いて拡大又は縮小することにより、異なるスケールのテンプレート符号が得られる。これらのテンプレート符号を切り替えながら照合を繰り返し、相違度が最小となる位置とスケールを求める。 If the size (scale) of the recognition target object on the image data changes between pre-registration and object recognition, the code collating unit 106 may change the scale of the template code and evaluate the degree of difference. Good. For example, template code

The ingredients

The template codes of different scales can be obtained by dividing the image into two and enlarging or reducing each component using the two-dimensional nearest neighbor method. Matching is repeated while switching these template codes, and the position and scale at which the degree of difference is minimized are obtained.

  According to the sixth embodiment configured as described above, the luminance gradient strength and its order may vary due to factors such as the environment such as illumination and the state of the object surface. However, since the direction of the luminance gradient is strongly dependent on the shape of the object and has a feature that it is less likely to fluctuate due to the environment, such a configuration can improve the recognition accuracy of the object recognition apparatus.

Embodiment 7 FIG.
The code verification unit 106 provided in the object recognition apparatus 100 according to Embodiments 1 to 6 of the present invention employs a method of enlarging or reducing the template code in order to cope with a change in the scale of the object. As another method, in the seventh embodiment, a captured image when each of the N light sources 102-1 to 102-N is turned on is held in the template storage unit, and these N images are stored. The code may be generated while enlarging or reducing the data when collating.

  FIG. 15 shows the configuration of an object recognition apparatus 100 according to Embodiment 7 of the present invention. The object recognition device 100 according to the seventh embodiment sequentially turns on the N light sources 102-1 to 102-N sequentially in comparison with the object recognition device 100 according to the first embodiment in FIG. The difference is that N pieces of captured luminance image data are stored in the template storage unit 105 based on a control signal from the control unit 103. The code generation unit 104 generates a code for the template image data based on the light source ID from the control unit 103, the image data from the camera 101, and the template image data from the template storage unit 105. A code is generated and output to the code verification unit 106. The code collating unit 106, when collating each code of the template image data and the reference image data, the code generated by the code generating unit 104 from the N template image data read from the template storage unit 105, and the recognition target object The code generated by the code generation unit 104 from the reference image obtained by capturing the image is collated.

  When the size (scale) of the recognition target object on the image changes between pre-registration and object recognition, the code generation unit 104 enlarges or reduces the N template images read from the template storage unit 105. . Since these template image data are luminance images, a smooth interpolation method such as a two-dimensional spline interpolation method generally used as a method for enlarging or reducing an image can be used for enlarging or reducing. A code is generated from N pieces of image data enlarged or reduced by the interpolation method, and this is used as the code of the template image data. The procedure for generating the code from the N pieces of image data is the same as the procedure of the code generation unit 104 provided in the object recognition apparatus 100 according to Embodiments 1 to 6. Further, the generation of the code for the reference image data and the collation with the code of the template image data executed thereafter are the same as the procedures in the first to sixth embodiments.

  According to the seventh embodiment configured as described above, the template storage unit 105 holds the information of the reference object in the form of N pieces of luminance image data, so that the luminance image can be enlarged or reduced by smooth interpolation. Yes. Since collation is performed using a code generated from a smooth enlarged or reduced image, it is possible to improve the reliability of recognition processing for a scale change of an object.

Embodiment 8 FIG.
The code generation unit 104 included in the object recognition apparatus 100 according to Embodiments 1 to 7 of the present invention generates a code for template image data stored in the template storage unit 105 in Embodiment 8, and In addition to the code generated in the form, information for enabling or disabling the code for each pixel, so-called mask information, is generated, and the code matching unit 106 refers to the mask information and refers to only the pixel for which the code is valid. You may comprise so that the code | symbol produced | generated from image data and the code | symbol of template image data may be collated.

  First, a method based on luminance gradient strength will be described as a method for generating mask information.

  The luminance gradient intensity calculated from each captured image data tends to take a large value at the contour portion derived from the three-dimensional shape of the object. Therefore, the validity or invalidity of the code is set for each pixel based on the luminance gradient intensity. Specifically, the code generation unit 104 generates binary mask information m (x, y). If the code of the coordinates (x, y) is valid, the mask information m (x, y) = 1. If invalid, mask information m (x, y) = 0. The validity or invalidity of the code of each coordinate is determined as follows, for example.

が成り立てばマスク情報ｍ（ｘ，ｙ）＝１とし、そうでなければマスク情報ｍ（ｘ，ｙ）＝０とする。 One method is to obtain luminance gradient image data I ′ _k of N pieces of photographed image data I _k (k is 1 to N), and gradient intensity I ′ _k (x, y) in the image data at coordinates (x, y). ) If there is at least one k that is greater than _a certain threshold τ _a , that is,

If the above holds, mask information m (x, y) = 1 is set, otherwise mask information m (x, y) = 0 is set.

が成り立てばマスク情報ｍ（ｘ，ｙ）＝１とし、そうでなければマスク情報ｍ（ｘ，ｙ）＝０とする。 In another method, at least one k such that the difference between the maximum value and the minimum value of the gradient intensity I ′ _k (x, y) in the image data at the coordinates (x, y) is greater than a threshold value τ _b. If it exists, ie

If the above holds, mask information m (x, y) = 1 is set, otherwise mask information m (x, y) = 0 is set.

  In addition to the method of determining the value of the luminance gradient intensity or its difference with the threshold value as described above, the gradient intensity value or its value is set so that the number of effective pixels in the total number of pixels is a certain ratio. You may make it select the magnitude | size of a difference from a high-order pixel.

  Next, a method based on the stability of each pixel will be described below as another method for generating mask information.

FIG. 16 is a schematic diagram of a captured image when the object recognition apparatus 100 captures a recognition target object in the scene as shown in FIG. Here, it is assumed that the background 201 is sufficiently flat with respect to the depth of the rectangular parallelepiped 202 of the recognition target and is regarded as a plane. In FIG. 16, a point (x ₁ , y ₁ ) is a point located at the boundary between the rectangular parallelepiped 202 of the recognition object and the background 201, and a point (x ₂ , y ₂ ) is a point located in the background 201. Further, it is assumed that light source IDs are assigned to the object recognition apparatus 100 in order from 1 to 8 with respect to adjacent light sources as shown in FIG. 3 (N = 8 in FIG. 3). .

Now, as viewed from the camera 101, recognized when irradiated with light from the lower left side of the cuboid 202 of the object, since the shade is generated in the upper right side of the rectangular parallelepiped 202, the point of the captured image data at this time (x _1, y ₁₎ The gradient strength of increases. When light is irradiated from the lower side of the rectangular parallelepiped 202, a shadow is generated on the upper side, and when light is irradiated from the lower right side, a shadow is generated on the upper left side. Similarly, the luminance gradient of the point (x ₁ , y ₁ ) Strength increases. That is, as shown in the upper part of FIG. 17, the gradient intensities corresponding to the respective light source IDs tend to take a large value or a small value between adjacent ones. Here, it is assumed that the light source 102-1 having the light source ID 1 and the light source 102-8 having the light source ID 8 are adjacent to each other. On the other hand, even if the light sources 102-1 to 102-8 are selectively switched, no shadow is generated or disappears in the vicinity of the point (x ₂ , y ₂ ). Therefore, the point (x The gradient strength of ₂ , y ₂ ) is a random value, for example, only a gradient change due to noise occurs. As described above, whether or not the gradient intensity is close between adjacent light sources is used as an index of stability.

The stability is defined by the following calculation, for example. It is assumed that the luminance gradient strength of the point (x ₁ , y ₁ ) is obtained as shown in the upper part of FIG.

  First, an N-bit binary number b which is a binary code in which the gradient strength is classified into two stages is generated. Next, a binary number b 'is generated by rotating the binary number b to the left by one bit. Here, the binary left rotate is an operation to move the most significant bit overflowed by the bit shift to the least significant bit. Thereafter, the number of bits having the same value between the binary numbers b and b ', that is, the number obtained by subtracting the Hamming distance between the binary numbers b and b' from the light source number N is obtained and used as the stability. The hamming distance between the two binary numbers b and b 'can be obtained at high speed by the method described in the third embodiment.

  If the stability obtained in this way is greater than a preset threshold value, mask information m (x, y) = 1, otherwise mask information m (x, y) = 0. Alternatively, the stability may be selected from the higher-order pixels so that the number of effective pixels in the total number of pixels is a certain ratio.

  Next, the description returns to each component in the present embodiment.

  The template storage unit 105 stores the code generated by the code generation unit 104 and the mask information m (x, y) obtained by the above-described procedure. The code matching unit 106 introduces mask information m (x, y) for evaluation of the degree of difference. For example, mask information is incorporated into the evaluation expression for the degree of difference of the code matching unit 106 according to Embodiment 1 of the present invention as follows. That is, coordinates (x, y) that minimize the following expression are output as the position of the object.

  Similarly, the mask information m (x, y) can be incorporated into the evaluation formula for the degree of difference for each pixel in the evaluation formula for the degree of difference in the code matching unit 106 according to Embodiments 2 to 7 of the present invention.

  According to the eighth embodiment configured as described above, the code generation unit 104 generates mask information that enables only pixels with high luminance gradient strength or pixels with high stability, and the code verification unit 106 Then, based on the mask information m (x, y), only the effective pixels are used to evaluate the code dissimilarity. Since the intensity gradient strength or stability increases at the contour portion derived from the three-dimensional shape of the object, only such pixels are enabled and used for code verification, improving recognition accuracy and speeding up the matching process. Can be realized.

Embodiment 9 FIG.
The light sources 102 provided in the object recognition apparatus 100 according to Embodiments 1 to 8 of the present invention may be arranged in a concentric multiple circle shape including two or more concentric multiple circles in the ninth embodiment. Furthermore, an optimization function for selecting in advance a light source optimal for the purpose of code generation and collation from among a plurality of light sources provided in the object recognition apparatus 100 may be provided.

  FIG. 19 shows an example in which eight light sources are arranged in double concentric circles. Light sources 102-11 to 102-18 are disposed as light sources on the inner side (side closer to the camera 101), and light sources 102-21 to 102-28 are disposed as outer light sources.

  First, the effect obtained by arranging the light sources 102-11 to 102-28 in multiple circles will be described below. 20A, 20B, and 20C are diagrams illustrating the influence of the positional relationship between the camera 101 and the light source (102, which is one of 102-1 to 102-8), on the appearance of the shadow 203. FIG. As shown in FIG. 20A, when the rectangular parallelepiped 202 of the recognition object is in contact with the background 201 and the distance between the camera 101 and the light source 102 is short, the size of the shadow 203 formed by light irradiation becomes small. If the shadow 203 is too small, there is a possibility that a large luminance gradient strength cannot be obtained at the boundary between the rectangular parallelepiped 202 and the shadow 203 in the image. Therefore, as shown in FIG. 20B, when the light source 102 that is distant from the camera 101 is used, the size of the shadow 203 that can be generated by the light irradiation increases, so that a clear luminance gradient is obtained at the boundary between the rectangular parallelepiped 202 and the shadow 203.

  However, when the rectangular parallelepiped 202 of the recognition target is physically separated from the background 201 as shown in FIG. 20C, depending on the positional relationship among the camera 101, the light source 102, and the rectangular parallelepiped 202 of the recognition target, the recognition target A shadow 203 may appear at a position away from the rectangular parallelepiped 202. In the captured image in such a case, a clear luminance gradient does not appear between the rectangular parallelepiped 202 of the recognition target object and the background 201, and on the contrary, a large luminance gradient occurs in the vicinity of the shadow 203 generated in the background 201 that is not the recognition target. . Therefore, in the arrangement condition of FIG. 20C, it is better to make the distance between the camera 101 and the light source 102 closer.

  That is, it is desirable that the optimum distance between the camera 101 and the light source 102 is appropriately set according to conditions such as the shape of the recognition object and the position of the camera 101. In order to select the optimal light source 102 for object recognition, each of the light sources 102-1 to 102-8 is selected in addition to a method of visually confirming a photographed image by turning on the light sources 102-1 to 102-8. A method is conceivable in which the size and position of the shadow 203 and the intensity gradient intensity that are generated when the blinking is performed selectively and sequentially are analyzed from the image.

  Next, the optimum light irradiation direction will be described. Depending on the shape of the recognition object, a clear luminance gradient may not occur in a specific light source direction. This may occur, for example, when the shape change in the irradiation direction of the light sources 102-1 to 102-8 is round and the depth changes gently.

Therefore, an object recognition apparatus having a plurality of M light sources (M> N) 102-1 to 102-M is used. In the step of creating a template using the reference object, for example, the brightness gradient intensity is compared between M image data captured by selectively switching the blinking of the M light sources 102-1 to 102-M sequentially, N pieces of image data are selected. As an evaluation scale for luminance gradient strength, for example, the difference between the maximum and minimum luminance gradient strengths of a certain captured image _Ik , the variance of luminance gradient strength, and the like can be used. Light source IDs 1 to N are assigned to the N light sources 102-1 to 102-N corresponding to the optimal N pieces selected in this way. All the subsequent processes may use N pieces of captured image data corresponding to the selected N light sources 102-1 to 102-N.

  The code generation unit 104 creates a template code from the N pieces of image data and stores the template code in the template storage unit 105. At the time of object recognition, the code collation unit 106 collates the code of the reference image data generated by the code generation unit 104 using the N pieces of image data and the code of the template image data stored in the template storage unit 105. To do. For code verification, the procedure of the code verification unit 106 according to Embodiments 1 to 7 of the present invention can be used.

  According to the ninth embodiment configured as described above, a plurality of light sources 102-11 to 102-28 are arranged in a concentric multiple circle shape, so that the camera is irradiated with light from one direction. Since the angle between the optical axis and each light source optical axis can be adjusted, the size of the shadow caused by the undulation of the object can be set appropriately, and the code used for object recognition can be generated with high accuracy. it can.

  Furthermore, based on a large number of light sources with different illumination angles arranged in advance, a combination of arrangements in which the light / dark difference depending on the direction becomes large is obtained, and only the light source 102 in the arrangement is controlled to blink, thereby being sensitive to the light / dark difference. Recognition can be realized, and the reliability of matching can be improved.

Embodiment 10 FIG.
The light source 102 provided in the object recognition apparatus 100 according to Embodiments 1 to 9 of the present invention can actively control the light emission luminance in Embodiment 10, and the object recognition apparatus 100 is a camera 101. Re-photographing may be performed by adjusting the light emission luminance of the light source 102 according to the brightness of the photographed image. Alternatively, the camera 101 can actively control at least one of shutter speed and gain, and the object recognition apparatus 100 adjusts the shutter speed and gain of the camera 101 according to the brightness of the image captured by the camera 101. Then, re-shooting may be performed.

  The object recognition apparatus 100 according to the embodiment of the present invention generates a code based on the luminance gradient strength and direction of the captured image when each of the light sources 102-1 to 102-N is selectively turned on sequentially. Object recognition is performed using. That is, in order to perform highly accurate object recognition, it is important that codes are generated accurately, that is, a sufficiently large luminance gradient is obtained. For this purpose, it is necessary to appropriately set parameters such as the light emission luminance of the light source 102 or the shutter speed and gain of the camera 101 to obtain high contrast while suppressing saturation of pixel values.

  Adjustment of the light emission luminance of the light sources 102-1 to 102-N or the shutter speed and gain of the camera 101 is performed as follows. At the stage of creating a template using a reference object, set appropriate parameters and try shooting once. The luminance value distribution is examined for the captured image data. If the brightness value is saturated, the image is too bright, so the light emission brightness of the light sources 102-1 to 102-N is lowered, the shutter speed of the camera 101 is increased, or the gain of the camera 101 is lowered. Re-shoot. On the other hand, if the luminance value is biased toward a smaller value, the image is too dark, so the light emission luminance of the light sources 102-1 to 102-N is increased, the shutter speed of the camera 101 is decreased, or the gain of the camera 101 is increased. Raise the value and re-shoot. The above-described procedure is repeated for the re-captured image until the luminance value distribution is determined to be appropriate. In this way, an appropriate parameter setting is obtained.

  According to the tenth embodiment configured as described above, the object recognition apparatus 100 actively controls parameters such as the luminance of the light sources 102-1 to 102-N and the shutter speed and gain of the camera 101 to obtain appropriate luminance. Since an image having a distribution can be obtained, the reliability of the recognition process is improved.

Summary of embodiment.
The aspects and effects of the embodiments described above are as follows.

An object recognition apparatus according to a first aspect is:
A camera that captures the recognition object;
A plurality of light sources arranged around the camera and each selectively flashing independently;
A control unit for controlling the blinking of each light source to send an imaging trigger signal to the camera synchronously or asynchronously with the blinking of each light source, and to send light source ID information to the code generation unit;
A code generation unit that generates a code from the luminance value of each image captured by the camera when each of the light sources is turned on or off;
A template storage unit storing information of a predetermined reference object;
A code collation unit that collates a code generated from an image group obtained by photographing the recognition object with a code generated from information stored in the template storage unit and outputs a collation result is provided.

  With this configuration, template matching can be performed with high accuracy even when the recognition target has a depth and the change in appearance in the image due to the change in the viewpoint of the camera is large.

  The object recognition device according to the second aspect is the object recognition device according to the first aspect, wherein the code generated by the code generation unit is based on a luminance value of an image captured by selectively switching blinking of the light source. It is characterized by expressing the calculated gradient strength.

  With this configuration, template matching using depth information of the recognition object (shape information in the depth direction when viewed from the camera) can be realized, so that recognition accuracy is improved.

  In the object recognition device according to the third aspect, in the object recognition device according to the first or second aspect, the code generated by the code generation unit is an image of an image captured by selectively switching blinking of the light source. It is characterized by expressing the order of the intensity of the brightness gradient calculated from the brightness value.

  With this configuration, template matching can be executed robustly against fluctuations in gradient strength.

  The object recognition device according to a fourth aspect is the object recognition device according to any one of the first to third aspects, wherein the code generation unit has the number of bits equal to the number of light sources and each bit is a light source. Is generated for each pixel, a luminance gradient is calculated for each of the images captured by selectively switching the light source to blink, and the intensity of the luminance gradient is classified into a higher rank. Different bit values are assigned to the bits corresponding to the light sources and the bits corresponding to the light sources classified in the lower order.

  With this configuration, template matching can be executed at high speed as a bit operation. Further, by binarizing the gradient strength, the robustness against the gradient strength rank fluctuation is improved.

  An object recognition device according to a fifth aspect is the object recognition device according to any one of the first to fourth aspects, wherein the code matching unit is a binary generated from an input image group by the code generation unit. The degree of difference is calculated based on the Hamming distance between the code and a binary code registered in advance as a template.

  With this configuration, the degree of difference for each pixel can be obtained by referring to the table, so that the code matching process can be executed at high speed.

  The object recognition device according to a sixth aspect is the object recognition device according to any one of the first to fifth aspects, wherein the code generation unit uses a ternary code whose number of digits matches the number of light sources as a pixel. The ternary code is generated every time, and the luminance gradient is calculated for each image captured by selectively switching the light source to blink, and the intensity of the luminance gradient is divided into the upper, middle and lower levels. It is characterized by expressing the classified result.

  The combination of this configuration and the object recognition apparatus according to the seventh aspect does not affect the degree of difference even if the order of the gradient strength is changed, so that the recognition robustness is improved.

  The object recognition device according to a seventh aspect is the object recognition device according to the sixth aspect, wherein the code matching unit includes a ternary code generated from an input image group by the code generation unit and a ternary code of a template. The degree of difference is calculated by collation, and the evaluation value of the degree of difference is lowered when the codes of the upper and lower levels of the intensity of the luminance gradient match each other, and the medium code contributes to the calculation of the degree of difference. It is characterized by not letting it.

  With this configuration, even when a change in gradient strength ranking occurs between the template and the reference image, robustness matching can be realized because the difference is not affected unless the higher order and the lower order are changed.

  The object recognition device according to an eighth aspect is the object recognition device according to any one of the first to seventh aspects, wherein the code generation unit generates the code for each pixel, and blinks the light source. It is characterized in that the largest direction of the intensity of the luminance gradient calculated from each of the images photographed by selectively switching is encoded as the direction of the pixel.

With this configuration, the code size per pixel can be a compact code of log ₂ N bits.

  The object recognition device according to a ninth aspect is the object recognition device according to any one of the first to seventh aspects, wherein the code generation unit assigns a code whose number of dimensions matches the number of the light sources for each pixel. The direction of the luminance gradient calculated from each of the images captured by selectively switching the blinking of the light source is used as a feature amount assigned to each dimension of the code.

  With this configuration, it is possible to generate a code that is strong against variations in the environment and the recognition target, using the fact that the direction of the luminance gradient does not change even when the light irradiation direction is changed.

  An object recognition apparatus according to a tenth aspect is the object recognition apparatus according to the eighth or ninth aspect, wherein the code matching unit is based on an evaluation criterion such that the smaller the difference in the direction of the luminance gradient, the higher the matching degree. It is characterized by performing code verification.

  With this configuration, it is possible to improve the recognition accuracy by using the property that the luminance gradient strength and its order are likely to change due to variations in the environment and the recognition object, but the gradient direction is difficult to change.

  An object recognition apparatus according to an eleventh aspect is the object recognition apparatus according to any one of the first to ninth aspects, wherein the code generation unit is an image captured by selectively switching blinking of each light source. On the basis of the luminance value, the validity or invalidity of the code is determined for each pixel, and information regarding validity or invalidity is generated.

  With this configuration, it is possible to achieve both high precision and high speed of matching by selecting only pixels that are effective for matching.

  An object recognition apparatus according to a twelfth aspect is the object recognition apparatus according to the tenth aspect, wherein the code generation unit is a magnitude of intensity of a luminance gradient of an image captured by selectively switching blinking of each light source. Based on the above, the validity or invalidity of the code is determined for each pixel, and information regarding validity or invalidity is generated.

  With this configuration, the gradient strength increases at the contour portion derived from the physical shape, so only high-gradient pixels can be selected and used for code matching to achieve both high accuracy and high speed matching. can do.

  An object recognition apparatus according to a thirteenth aspect is the object recognition apparatus according to the tenth aspect, wherein the code generation unit is based on the stability of each pixel calculated based on a plurality of images obtained by photographing the same object. Thus, the validity or invalidity of the code is determined for each pixel, and information regarding validity or invalidity is generated.

With this configuration, the degree of stability of each pixel is estimated from a plurality of template images obtained by photographing the same object, and a pixel having a high stability is determined to be valid and a low pixel is determined to be invalid. By using it for collation, both high precision and high speed of matching can be achieved.

  In the object recognition device according to a fourteenth aspect, in the object recognition device according to any one of the first to twelfth aspects, the code matching unit is configured to input images only for pixels set to be valid by the code generation unit. And a template code are collated.

  With this configuration, only stable pixels whose luminance gradient does not change even when the light irradiation direction is changed are used for matching. That is, by masking pixels whose intensity changes are unstable, improvement of matching accuracy and speeding up of processing are realized at the same time.

  An object recognition device according to a fifteenth aspect is the object recognition device according to any one of the first to fourteenth aspects, wherein the plurality of light sources arranged are arranged in a plurality of concentric multiple circles. Features.

  With this configuration, by arranging a plurality of light sources in a concentric multiple circle shape, the angle between the camera optical axis and each light source optical axis can be adjusted when irradiating a light beam on the object from one direction. The size of the shadow caused by the undulation of the object can be set appropriately.

  The object recognition device according to a sixteenth aspect is the object recognition device according to any one of the first to fifteenth aspects, wherein the code generation performed by the code generation unit and the code verification performed by the code verification unit 1 has a function of optimizing the number and directions of light sources to be used in advance.

  With this configuration, based on a large number of light sources with different irradiation angles arranged in advance, a combination of arrangements in which the difference in brightness depends on the direction is obtained, and only the light sources in that arrangement are controlled to blink, thereby reducing the difference in brightness. Sensitive recognition is realized and matching reliability is improved.

  An object recognition device according to a seventeenth aspect is the object recognition device according to any one of the first to sixteenth aspects, and includes a plurality of light sources capable of actively controlling light emission luminance, and is photographed in advance. The brightness of each pixel in the obtained image is checked, and the image is re-photographed by controlling the light emission luminance of each light source according to the brightness.

  With this configuration, it is possible to obtain an image with an appropriate brightness range by taking a picture once before measurement and controlling to reduce the intensity of the light source if it is too bright and to increase if it is too dark. This improves the reliability of measurement.

  An object recognition device according to an eighteenth aspect is the object recognition device according to any one of the first to seventeenth aspects, wherein the camera can actively control at least one of shutter speed and gain. A camera is characterized in that the brightness of each pixel in an image photographed in advance is checked, and the shutter speed and gain of the camera are controlled according to the brightness, and re-photographing is performed.

  With this configuration, you can take a picture before measurement, and if it is too bright, control the camera to increase the shutter speed or decrease the gain, and if it is too dark, control the shutter speed to decrease or increase the gain. Thus, an image having an appropriate brightness range can be obtained, and measurement reliability is improved.

  The present invention can provide an object recognition apparatus that registers an object (recognition target object) to be recognized in advance and recognizes an existing location from a camera-captured image. This provides a highly accurate object recognition device that is more robust against changes in the appearance of the object than the prior art.

  DESCRIPTION OF SYMBOLS 100 Object recognition apparatus, 101 Camera, 102, 102-1 to 102-28, 102-N Light source, 103 Control part, 104 Code generation part, 105 Template storage part, 106 Code collation part, 201 Background, 202 Cuboid, 203 Shadow .

Claims (18)

A camera that captures the recognition object;
A plurality of light sources arranged around the camera and each selectively flashing independently;
A control unit for controlling the blinking of each light source to send an imaging trigger signal to the camera synchronously or asynchronously with the blinking of each light source, and to send light source ID information to the code generation unit;
A code generation unit that generates a code from the luminance value of each image captured by the camera when each of the light sources is turned on or off;
A template storage unit storing information of a predetermined reference object;
An object recognition apparatus comprising: a code collation unit that collates a code generated from an image group obtained by photographing the recognition object with a code generated from information stored in the template storage unit and outputs a collation result.   The code generated by the code generation unit represents intensity of a gradient calculated from a luminance value of an image captured by selectively switching blinking of the light source. The object recognition apparatus described.   The code generated by the code generation unit represents a rank of intensity of a luminance gradient calculated from a luminance value of an image captured by selectively switching blinking of the light source. Item 3. The object recognition apparatus according to Item 1 or 2.   The code generation unit generates a binary code for each pixel in which the number of bits matches the number of light sources and each bit corresponds to a light source, and each of the images photographed by selectively switching blinking of the light sources A brightness gradient is calculated with respect to the bit, and different bit values are assigned to the bit corresponding to the light source classified as the higher level and the bit corresponding to the light source classified as the lower level, respectively. The object recognition apparatus of any one of Claim 1 to 3.   The code collating unit calculates a degree of difference based on a Hamming distance between a binary code generated from an input image group by the code generating unit and a binary code registered in advance as a template. Item 5. The object recognition device according to any one of Items 1 to 4.   The code generation unit generates, for each pixel, a ternary code whose number of digits matches the number of light sources, and the ternary code is a luminance gradient for each image captured by selectively switching blinking of the light source. The object recognition according to any one of claims 1 to 5, characterized in that the result of calculating the intensity gradient and classifying the intensity of the luminance gradient into upper, middle and lower levels is expressed. apparatus.   The code collating unit calculates the degree of difference by collating the ternary code generated from the input image group by the code generating unit and the ternary code of the template. The object recognition apparatus according to claim 6, wherein the evaluation value of the dissimilarity is lowered when the codes of each match, and the middle code is not contributed to the calculation of the dissimilarity.   The code generation unit generates the code for each pixel, and encodes the largest direction of the intensity of the luminance gradient calculated from each of the images photographed by selectively switching the blinking of the light source as the direction of the pixel. The object recognition apparatus according to any one of claims 1 to 7, wherein the object recognition apparatus is characterized in that:   The code generation unit generates a code having a number of dimensions that matches the number of the light sources for each pixel, and indicates the direction of the luminance gradient calculated from each image captured by selectively switching blinking of the light sources. The object recognition apparatus according to any one of claims 1 to 7, wherein a feature amount assigned to a dimension is used.   10. The object recognition apparatus according to claim 8, wherein the code matching unit performs code matching based on an evaluation criterion such that the degree of coincidence increases as the difference in the direction of the luminance gradient decreases.   The code generation unit determines whether the code is valid or invalid for each pixel based on a luminance value of an image captured by selectively switching blinking of each light source, and generates information regarding validity or invalidity. The object recognition device according to any one of claims 1 to 9.   The code generation unit determines whether the code is valid or invalid for each pixel based on the magnitude of the intensity of the luminance gradient of the image photographed by selectively switching the blinking of each light source, and information on validity or invalidity The object recognition apparatus according to claim 10, wherein the object recognition apparatus generates the object recognition apparatus.   The code generation unit determines whether the code is valid or invalid for each pixel based on the stability of each pixel calculated based on a plurality of images obtained by photographing the same object, and generates information regarding validity or invalidity. The object recognition apparatus according to claim 10.   The object recognition according to any one of claims 1 to 12, wherein the code collating unit collates a code of an input image with a template code only for pixels that are set to be valid by the code generating unit. apparatus.   The object recognition apparatus according to claim 1, wherein the plurality of light sources are arranged in a plurality of concentric multiple circles.   The code generation performed by the code generation unit and the code verification performed by the code verification unit have a function of optimizing the number and directions of light sources to be used in advance. The object recognition apparatus of any one of Claims.   Equipped with multiple light sources that can actively control the emission brightness, check the brightness of each pixel in the pre-captured image, and control the emission brightness of each light source according to the brightness and re-shoot The object recognition apparatus according to any one of claims 1 to 16, wherein   The camera is a camera capable of actively controlling at least one of shutter speed and gain, and examines the brightness of each pixel in an image captured in advance, and the shutter of the camera according to the brightness. 18. The object recognition apparatus according to claim 1, wherein re-photographing is performed while controlling speed and gain.

Images