JP2017211302A - Image recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image recognition device which can detect the amount of change in the direction of the optical axis of a camera by a monocular camera.SOLUTION: The image recognition device includes: a camera 10 as imaging means for imaging a hand of an operator as an imaging object from a predetermined direction; and a controller for detecting the amount of change of the hand in the predetermined direction from the luminance change or the size change of the hand on the basis of image information in a plane perpendicular to the predetermined direction taken by the camera 10.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image recognition apparatus.

  As a conventional technique, there has been proposed an image recognition apparatus using a method for determining a position of an object that can increase the detection speed using a stereo camera. This position determination method uses a stereo camera in the first two measurements, but in the third measurement, only one camera is used, and the other camera The current position is predicted from the movement trajectory of the object, and then the image position is predicted. In this manner, the position of the object in the three-dimensional orthogonal coordinate system of X, Y, Z is determined by stereo measurement using the measured image position of one camera and the predicted image position of the other camera. After the third time, this operation is performed by alternately switching the actually measured cameras. Thereby, it is supposed that the measurement speed in the image recognition apparatus can be improved.

JP 2007-147424 A

  However, in the image recognition apparatus of Patent Document 1, it is necessary to perform stereo measurement with a stereo camera in order to detect the amount of change in the optical axis direction of the camera of the imaging object. Therefore, the cost of the stereo measurement apparatus is increased, and it is necessary to detect the three-dimensional image position based on the image information obtained by alternately switching the cameras.

  Accordingly, an object of the present invention is to provide an image recognition apparatus capable of detecting the amount of change in the optical axis direction of a camera with a monocular camera.

[1] In order to achieve the above object, the imaging object is based on imaging means for imaging an imaging object from a predetermined direction and image information in a plane orthogonal to the predetermined direction imaged by the imaging means. And a control unit that detects a change amount of the imaging object in the predetermined direction based on a change in luminance or size of the object.

[2] The image recognition apparatus according to [1], wherein the imaging unit is a camera, and the predetermined direction is an optical axis direction of the camera.

[3] Further, the imaging object is an operator's hand, and the control unit identifies a reciprocal motion in a triaxial direction including an optical axis direction of the camera. 2] may be used.

  According to the image recognition apparatus of the present invention, it is possible to provide an image recognition apparatus that can detect the amount of change in the optical axis direction of the camera with a monocular camera.

FIG. 1 is a schematic perspective view showing an example of an arrangement of an image recognition device according to an embodiment of the present invention. FIG. 2 shows an example of a block diagram of the image recognition apparatus according to the embodiment of the present invention. FIG. 3A is a frame image S (n) showing an example of an imaging screen viewed from the arrow A in FIG. 1, and FIG. 3B is an example of a background image of the imaging screen viewed from the arrow A in FIG. Is a frame image S (0) showing 4A is a frame image S ₁ (n) obtained by removing the frame image S (0), which is the background image, from the captured frame image S (n), and FIG. 4B is an imaging target object. This is the next frame image S ₁ (n + 1) when a hand moves in the Z direction (upward). FIG. 5A is a frame image S ₂ (n) obtained by binarizing the captured frame image S (n) by removing the frame image S (0) as the background image, and FIG. This is the next frame image S ₂ (n + 1) when the hand that is the imaging object moves in the Z direction (upward). FIG. 6 is a correspondence table showing the relationship between the movement direction of the hand and the position of the center of gravity, the brightness, and the size when the hand position is the camera center. FIG. 7 is a correspondence table showing the relationship between the hand movement direction and the gravity center position, brightness, and size when the hand position is on the right side from the camera center. FIG. 8 is a correspondence table showing the relationship between the hand movement direction and the gravity center position, brightness, and size when the hand position is on the left side from the camera center. FIG. 9 is a correspondence table showing the relationship between the movement direction of the hand and the position of the center of gravity, the luminance, and the size when the hand position is forward from the camera center. FIG. 10 is a correspondence table showing the relationship between the hand movement direction and the gravity center position, brightness, and size when the hand position is behind the camera center. FIG. 11 is an example of an imaging screen when the palm rotates from the downward direction to the right side in the imaging screen viewed from the direction of arrow A in FIG. 1.

(Embodiment of the present invention)
The image recognition apparatus 1 according to the embodiment of the present invention includes a camera 10 as an imaging unit that images an operator's hand 80 that is an imaging target from a predetermined direction, and a predetermined direction captured by the camera 10. And a control unit 30 that detects a change amount of the operator's hand 80 in a predetermined direction based on a change in luminance or size of the operator's hand 80 based on image information in the orthogonal plane. Yes.

  The predetermined direction is set in the optical axis direction (Z-axis direction) of the camera 10.

  FIG. 1 is a schematic perspective view showing an example of an arrangement of an image recognition device according to an embodiment of the present invention. FIG. 2 shows an example of a block diagram of the image recognition apparatus according to the embodiment of the present invention.

  As shown in FIG. 1, in the image recognition apparatus 1, a camera 10 as an imaging unit is arranged in the upward direction, and images the A direction shown in the figure, that is, the optical axis direction (Z-axis direction) of the camera 10. In the present embodiment, the camera 10 is a single-eye system. The camera 10 includes a semiconductor image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor). The infrared illumination 20 illuminates a range that is substantially the same as the imaging range 60 of the camera 10. In addition, although the light of the light to image can use the light of arbitrary frequencies, it illuminates and images using infrared light and near-infrared light. Thereby, an operator does not see illumination light at the time of imaging, and the uncomfortable feeling at the time of imaging can be prevented or reduced.

(Camera 10)
As shown in FIG. 1, the camera 10 captures an image of the operator's hand 80 from below, and image data that is image information in a plane orthogonal to the optical axis direction (Z-axis direction) of the camera 10 is displayed in the frame image 100. Are output to the control unit 30 at a predetermined frame period. The frame image 100 is a frame image including a background 110 as shown in FIG. As shown in FIG. 2, the frame image 100 outputs the frame image S (n) of the nth frame at a certain time to the control unit 30.

  Note that only a background image as shown in FIG. 3B to be described later is captured in advance, and this frame image S (0) is used as the background image.

(Lighting 20)
As shown in FIGS. 1 and 2, a lighting unit 20 is connected to the control unit 30. Note that a plurality of lights may be connected to the control unit 30. As shown in FIG. 1, the illumination 20 illuminates the imaging target by irradiating infrared light onto a range that covers the imaging range (angle of view) of the camera 10.

(Configuration of control unit 30)
As shown in FIG. 2, one camera 10 is connected to the control unit 30. In addition, the illumination unit 20 is connected to the control unit 30.

  The control unit 30 includes, for example, a CPU (Central Processing Unit) that performs operations and processing on acquired data according to a stored program, a RAM (Random Access Memory) that is a semiconductor memory, a ROM (Read Only Memory), and the like. Microcomputer. For example, a program for operating the control unit 30 is stored in the ROM. For example, the RAM is used as a storage area for storing a plurality of pieces of image information, temporary calculation results, and the like. In addition, the control unit 30 has a means for generating a clock signal therein, and executes processing based on the clock signal.

  The control unit 30 includes an image processing unit 31, a centroid detection unit 32, a luminance detection unit 33, a size detection unit 34, and an operation identification unit 35.

(Image processing unit 31)
The image processing unit 31 performs image processing such as background removal and binarization in advance in order to perform post-process centroid detection, luminance detection, size detection, and the like for each frame image S (n) input from the camera 10. It is the pre-process for performing.

  FIG. 3A is a frame image S (n) showing an example of an imaging screen viewed from the arrow A in FIG. 1, and FIG. 3B is an example of a background image of the imaging screen viewed from the arrow A in FIG. Is a frame image S (0) showing The frame image S (0), which is the background image, is a reference image for the difference image calculation process, and is preferably an image that does not change with time.

FIG. 4A shows a frame image S ₁ (n) obtained by removing the frame image S (0) as the background image from the captured frame image S (n). That is, the frame image S ₁ (n) is calculated by subtracting the frame image S (0) obtained by capturing only the background from the frame image S (n) input from the camera 10.

(Centroid detection unit 32)
The centroid detection unit 32 detects the position of the centroid G by calculating the position of the centroid G based on the frame image S ₁ (n) with the background removed. The X coordinate of the position of the center of gravity G is an average value of the numerical values in FIG. The average value can be calculated by dividing the sum of the X coordinate values of the pixels where the hand 80 is present by the number of pixels where the hand 80 is present. Similarly, the Y coordinate of the position of the center of gravity G is the average value of the numerical values in FIG. The average value can be calculated by dividing the sum of the Y coordinate values of the pixels where the hand 80 is present by the number of pixels where the hand 80 is present. As shown in FIG. 4A, the center of gravity G (x, y) is calculated by the center of gravity calculation described above, with the origin at the lower left of the frame image S ₁ (n), the right direction as X, and the upper direction as Y. calculate.

Note that the frame image S ₁ (n) in FIG. 4A as viewed in the direction of arrow A in FIG. 1 is the X direction in the right direction, the Y direction in the forward direction, and the Z direction in the upward direction.

(Luminance detection unit 33)
The luminance detection unit 33 performs luminance detection based on the frame image S ₁ (n) with the background removed. In the present embodiment, as shown in FIG. 1, the illumination 20 is applied to the operator's hand 80 from the camera 10 side. Therefore, the captured frame image S ₁ (n) shown in FIG. 4A has high palm portion brightness and low background portion brightness.

Here, FIG. 4B is a frame image S ₁ (n + 1) of the next frame when the hand that is the imaging target moves in the Z direction (upward). When the operator's hand 80 is moving in the Z direction (upward) with time, the operator's hand 80 is captured small in the subsequent frame image. Therefore, a change in luminance of the operator's hand 80 can be detected. In the case of the configuration shown in FIG. 1, the luminance change when the operator's hand 80 moves in the Z direction (upward) decreases the luminance, and conversely, the hand 80 moves downward. The luminance change increases the luminance.

  For example, in the configuration as shown in FIG. 1, if the operator's hand 80 is imaged from below and the illumination is natural light (external light) from above, the imaging condition is so-called backlight. In such a case, the luminance change when the operator's hand 80 is moving in the Z direction (upward) increases the luminance, and conversely, the luminance change when the hand 80 is moving downward is The brightness decreases.

  In any case, by setting the imaging direction and illumination direction of the imaging object in advance, the movement of the operator's hand 80 that is the imaging object in the Z direction (up or down) can be detected by a change in luminance. Further, the moving direction (up or down direction) of the operator's hand 80 can be specified from the detected luminance change by the set imaging direction and illumination direction.

(Size detector 34)
The size detection unit 34 performs size detection based on the frame image S ₂ (n) obtained by removing the background and binarizing. FIG. 5A shows a frame image S ₂ (n) that is binarized by removing the frame image S (0) that is the background image from the captured frame image S (n).

As shown in FIG. 1, when the illumination 20 is applied to the operator's hand 80 from the camera 10 side, the captured frame image S ₁ (n) shown in FIG. Is high and the brightness of the background is low. The size detection unit 34 binarizes the image processing unit 31 to generate a frame image S ₂ (n) as shown in FIG.

The size detection unit 34 can detect the size of the hand 80 by calculating the area of the region of the hand 80 of the binarized frame image S ₂ (n) as shown in FIG. . That is, if the region of the hand 80 of the binarized frame image S ₂ (n) is 1 (High) and the background portion is 0 (Low), 1 (High) in the frame image S ₂ (n) ), The size of the hand 80 can be detected.

Here, FIG. 5B is the next frame image S ₂ (n + 1) when the hand 80 that is the imaging target moves in the Z direction (upward). When the operator's hand 80 is moving in the Z direction (upward) with time, the operator's hand 80 is captured small in the subsequent frame image. Therefore, a change in the size of the operator's hand 80 can be detected. In the case of the configuration shown in FIG. 1, when the operator's hand 80 moves in the Z direction (upward), the size change becomes smaller, and conversely, the hand 80 moves downward. The size change increases the size.

(Operation identification unit 35)
FIG. 6 is a correspondence table showing the relationship between the direction of hand movement, the position of the center of gravity, the brightness, and the size when the hand position is the camera center. FIG. 7 is a correspondence table showing the relationship between the hand movement direction and the gravity center position, brightness, and size when the hand position is on the right side from the camera center. FIG. 8 is a correspondence table showing the relationship between the hand movement direction and the gravity center position, brightness, and size when the hand position is on the left side from the camera center. FIG. 9 is a correspondence table showing the relationship between the movement direction of the hand and the position of the center of gravity, the luminance, and the size when the hand position is forward from the camera center. FIG. 10 is a correspondence table showing the relationship between the direction of hand movement, the position of the center of gravity, the brightness, and the size when the hand position is behind the camera center. As shown in FIG. 1, the directions of right, left, front, back, up and down are set.

  The center-of-gravity position G (x, y) of the operator's hand 80, which is the imaging object detected by the center-of-gravity detection unit 32, the luminance change detected by the luminance detection unit 33, and the size detected by the size detection unit 34 Based on the change, the movement and movement of the hand 80 can be identified.

(When hand 80 is centered on camera)
When the position of the hand 80 moves on the Z axis, which is the center of the camera, the change shown in the correspondence table shown in FIG. 6 is shown. When the hand 80 moves upward with the position of the hand 80 at the center of the camera, the position of the center of gravity of the hand 80 does not change, the brightness decreases, and the size decreases. On the other hand, when the hand 80 is moved downward with the position of the hand 80 at the center of the camera, the position of the center of gravity does not change, the brightness increases, and the size increases. Further, when the hand 80 moves in the left direction, the position of the center of gravity moves to the left, and the luminance and size do not change. When the hand 80 moves to the right, the position of the center of gravity moves to the right, and the brightness and size remain unchanged. Further, when the hand 80 moves in the forward direction, the position of the center of gravity moves in the forward direction, and the luminance and the size remain unchanged. When the hand 80 moves backward, the position of the center of gravity moves backward, and the brightness and size remain unchanged.

(When the position of the hand 80 is on the right side from the camera center)
When the position of the hand 80 moves on the right side from the camera center, the change shown in the correspondence table shown in FIG. 7 is shown. If the hand 80 moves upward while the position of the hand 80 is on the right side from the camera center, the center of gravity of the hand 80 moves to the left (center), the brightness decreases, and the size decreases. On the other hand, when the hand 80 moves downward with the position of the hand 80 on the right side from the camera center, the position of the center of gravity moves to the right (outside), the brightness increases, and the size increases. Further, when the hand 80 moves in the left direction, the position of the center of gravity moves to the left, and the luminance and size do not change. When the hand 80 moves to the right, the position of the center of gravity moves to the right, and the brightness and size remain unchanged. Further, when the hand 80 moves in the forward direction, the position of the center of gravity moves in the forward direction, and the luminance and the size remain unchanged. When the hand 80 moves backward, the position of the center of gravity moves backward, and the brightness and size remain unchanged.

(When hand 80 is on the left side from the camera center)
When the position of the hand 80 moves on the left side from the camera center, the change shown in the correspondence table shown in FIG. 8 is shown. If the hand 80 moves upward while the position of the hand 80 is on the left side from the camera center, the center of gravity of the hand 80 moves to the right (center), the brightness decreases, and the size decreases. On the other hand, when the hand 80 moves downward with the position of the hand 80 on the left side from the center of the camera, the position of the center of gravity moves to the left (outside), the brightness increases, and the size increases. Further, when the hand 80 moves in the left direction, the position of the center of gravity moves to the left, and the luminance and size do not change. When the hand 80 moves to the right, the position of the center of gravity moves to the right, and the brightness and size remain unchanged. Further, when the hand 80 moves in the forward direction, the position of the center of gravity moves in the forward direction, and the luminance and the size remain unchanged. When the hand 80 moves backward, the position of the center of gravity moves backward, and the brightness and size remain unchanged.

(When the position of the hand 80 is the front side from the camera center)
When the position of the hand 80 moves forward from the center of the camera, the change shown in the correspondence table shown in FIG. 9 is shown. If the hand 80 moves upward while the position of the hand 80 is on the front side from the camera center, the position of the center of gravity of the hand 80 moves toward the back (center), the brightness decreases, and the size decreases. On the other hand, when the hand 80 moves downward with the position of the hand 80 on the front side from the camera center, the center of gravity moves toward the front (outside), the brightness increases, and the size increases. Further, when the hand 80 moves in the left direction, the position of the center of gravity moves to the left, and the luminance and size do not change. When the hand 80 moves to the right, the position of the center of gravity moves to the right, and the brightness and size remain unchanged. Further, when the hand 80 moves in the forward direction, the position of the center of gravity moves in the forward direction, and the luminance and the size remain unchanged. When the hand 80 moves backward, the position of the center of gravity moves backward, and the brightness and size remain unchanged.

(When the position of the hand 80 is behind the camera center)
When the position of the hand 80 moves rearward from the center of the camera, a change like the correspondence table shown in FIG. 10 is shown. If the hand 80 moves upward while the position of the hand 80 is on the rear side from the camera center, the center of gravity of the hand 80 moves forward (center), the brightness decreases, and the size decreases. On the other hand, when the hand 80 moves downward with the position of the hand 80 on the rear side from the center of the camera, the position of the center of gravity moves toward the rear (outside), the brightness increases, and the size increases. Further, when the hand 80 moves in the left direction, the position of the center of gravity moves to the left, and the luminance and size do not change. When the hand 80 moves to the right, the position of the center of gravity moves to the right, and the brightness and size remain unchanged. Further, when the hand 80 moves in the forward direction, the position of the center of gravity moves in the forward direction, and the luminance and the size remain unchanged. When the hand 80 moves backward, the position of the center of gravity moves backward, and the brightness and size remain unchanged.

  From the correspondence tables of FIGS. 6 to 10 described above, the motion identification unit 35 can detect the moving direction of the operator's hand 80 that is the imaging target. This detection is possible regardless of whether the position of the hand 80 is the camera center, the right side from the camera center, the left side, or the front or back position from the camera center.

  The motion identification unit 35 can detect that the operator's hand 80 that is the imaging target has moved upward or downward by detecting a change in luminance.

  The motion identification unit 35 can detect that the operator's hand 80 that is the imaging target has moved upward or downward by detecting a change in size.

  The motion identification unit 35 detects changes in the center of gravity of the operator's hand 80, which is the imaging object, to the left, right, front, and back, thereby changing the left, right, front, and back, respectively. Can be detected.

(When hand 80 moves with rotation)
FIG. 11 is an example of an imaging screen when the palm 81 of the hand 80 is rotated from the downward direction to the right side on the imaging screen as viewed in the direction of arrow A in FIG. As an example, when the hand 81 rotates from the downward direction to the right direction, the luminance increases and the size decreases, so that it can be distinguished from a downward movement in which the luminance increases and the size increases.

(Effect of embodiment)
The embodiment of the present invention has the following effects.
(1) An image recognition apparatus 1 according to an embodiment of the present invention includes a camera 10 as an imaging unit that images an operator's hand 80 that is an imaging target from a predetermined direction, and a predetermined image captured by the camera 10. And a control unit 30 for detecting a change amount of the operator's hand 80 in a predetermined direction based on a change in brightness or a size of the operator's hand 80 based on image information in a plane orthogonal to the direction of the operator's hand. It is configured. Since the movement of the hand 80 is detected by combining the brightness change and size change information of the operator's hand 80, even the monocular system with one camera can detect the amount of change in the optical axis direction of the camera. It becomes.
(2) Since it is possible to detect the movement of the camera in the optical axis direction, it is possible to recognize three-axis (three-dimensional) movement by the monocular camera system.
(3) Since it is a monocular camera system, it is possible to configure an image recognition apparatus at a lower cost than a stereo camera or a TOF camera.
(4) By using an AND condition with not only the luminance change but also the size change, the influence of ambient light can be suppressed.
(5) By using an AND condition with not only luminance change but also size change, it is possible to suppress the influence of camera shake (rotation).
(6) Since it is possible to detect the movement of the operator's hand 80 as described above, the control unit 30 recognizes the motion of the gesture operation (for example, the recognizing operation) in three axes (three dimensions) by the monocular camera system. Is possible.

  As mentioned above, although some embodiment and modification of this invention were demonstrated, these embodiment and modification are only examples, and do not limit the invention based on a claim. These novel embodiments and modifications can be implemented in various other forms, and various omissions, replacements, changes, and the like can be made without departing from the scope of the present invention. it can. In addition, not all combinations of features described in these embodiments and modifications are necessarily essential to the means for solving the problems of the invention. Furthermore, these embodiments and modifications are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 1 ... Image recognition apparatus 10 ... Camera 20 ... Illumination 30 ... Control part 31 ... Image processing part 32 ... Gravity center detection part 33 ... Luminance detection part 34 ... Size detection part 35 ... Motion identification part 60 ... Imaging range 80 ... Hand 81 ... Palm 100 ... frame image 110 ... background G ... centroid _{S (0), S 1 (} n), S 2 (n) ... frame image

Claims (3)

Imaging means for imaging an imaging object from a predetermined direction;
Control for detecting a change amount of the imaging object in the predetermined direction based on a change in luminance or size of the imaging object based on image information in a plane orthogonal to the predetermined direction imaged by the imaging means. And
An image recognition apparatus comprising:   The image recognition apparatus according to claim 1, wherein the imaging unit is a camera, and the predetermined direction is an optical axis direction of the camera.   The image recognition apparatus according to claim 1, wherein the object to be imaged is an operator's hand, and the control unit identifies a recognizing operation in a triaxial direction including an optical axis direction of the camera. .