JP2017224162A - Gesture determination device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a gesture determination device capable of reducing processing load in association with determination of a gesture.SOLUTION: A gesture determination device 1 is configured by comprising a control unit 5 that: determines a gesture performed by a hand 9 on the basis of a plurality of images 35 captured by an imaging unit 3; and sets, when unable to recognize the hand 9 in one of the images 35, a recognition area 50 for recognizing an object including the hand 9 in a frame 36 of another one of the images 35 obtained after the one image 35.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a gesture determination device.

  As conventional techniques, an imaging unit that images an operation object and a predetermined movable object that can be moved by the action of an external force, and the positions of both objects based on the operation object and the captured image of the movable object by the imaging unit A positive determination result is obtained by the position specifying means for specifying the position, the synchronous movement determining means for determining whether or not the positions of the two objects specified by the position specifying means have moved in synchronization, and the synchronous movement determining means. In this case, there is known a gesture recognition device including a gesture recognition control unit that invalidates a gesture recognition result based on a captured image of an operation object (see, for example, Patent Document 1).

  This gesture recognizing device can avoid erroneously recognizing an operation of moving a movable object as a gesture.

JP 2014-126879 A

  However, since the conventional gesture recognition apparatus performs recognition processing on the entire imaged region, the processing load is large.

  Accordingly, an object of the present invention is to provide a gesture determination device that can suppress a processing load associated with gesture determination.

  One aspect of the present invention is to determine a gesture made by a recognition target based on a plurality of images captured by an imaging unit, and when the recognition target cannot be recognized in one image, the image acquired after the one image Provided is a gesture determination device including a gesture determination unit that sets a recognition region for recognizing an object including a recognition target at an edge.

  According to the present invention, it is possible to suppress a processing load associated with gesture determination.

FIG. 1A is a schematic diagram illustrating an example of the inside of a vehicle on which the gesture determination device according to the embodiment is mounted, and FIG. 1B is a block diagram illustrating an example of the gesture determination device. 1 (c) is a block diagram illustrating an example of a vehicle communication system to which a gesture determination device is connected. FIG. 2A is a schematic diagram illustrating an example of an image generated by the gesture determination device according to the embodiment, and FIG. 2B is a schematic diagram illustrating an example of a recognition area, and FIG. ) Is a schematic diagram illustrating an example of a determination region, and FIG. 2D is a schematic diagram illustrating an example of a recognition region according to a modification. FIG. 3 is a flowchart illustrating an example of the operation of the gesture determination device according to the embodiment.

(Summary of embodiment)
The gesture determination device according to the embodiment determines a gesture made by a recognition target based on a plurality of images captured by the imaging unit and acquires the recognition target after the one image when the recognition target cannot be recognized in the one image. A gesture determination unit that sets a recognition area for recognizing an object including a recognition target at the edge of the image is schematically configured.

  This gesture determination apparatus sets a recognition area for recognizing an object including the recognition target at the edge of the image without performing recognition processing of the recognition target for the entire image. Therefore, the gesture determination device can suppress the processing load associated with the determination of the gesture as compared with the case where the recognition process is performed on the entire image.

[Embodiment]
(Outline of gesture determination device 1)
FIG. 1A is a schematic diagram illustrating an example of the inside of a vehicle on which the gesture determination device according to the embodiment is mounted, and FIG. 1B is a block diagram illustrating an example of the gesture determination device. 1 (c) is a block diagram illustrating an example of a vehicle communication system to which a gesture determination device is connected. FIG. 2A is a schematic diagram illustrating an example of an image generated by the gesture determination device according to the embodiment, and FIG. 2B is a schematic diagram illustrating an example of a recognition area, and FIG. ) Is a schematic diagram illustrating an example of a determination region, and FIG. 2D is a schematic diagram illustrating an example of a recognition region according to a modification. Note that, in each drawing according to the embodiment described below, the ratio between figures may be different from the actual ratio. Moreover, in FIG.1 (b) and FIG.1 (c), the flow of main information is shown by the arrow. Furthermore, “A to B” indicating a numerical range is used in the meaning of A or more and B or less.

  The gesture determination device 1 enables, for example, operation of an electronic device mounted on the vehicle 8 based on a gesture made. The operator can operate the function of the electronic device corresponding to the gesture by performing a predetermined gesture. This electronic device is, for example, the navigation device 84 and the music and video playback device 85 shown in FIG. As an example, the gesture determination device 1 and the electronic device constitute a vehicle communication system 81 as shown in FIG.

  In the present embodiment, a gesture made by the operator's hand 9 as a recognition target will be described. However, the present invention is not limited to this, and is a gesture by a part of the operator's body other than the hand 9 or an object such as a pen. May be.

  As shown in FIG. 1A to FIG. 2C, the gesture determination device 1 determines a gesture made by the hand 9 based on a plurality of images 35 captured by the imaging unit 3 and controls the hand 9. If the image 35 cannot be recognized, the control unit 5 serving as a gesture determination unit that sets a recognition area 50 for recognizing an object including the hand 9 on the edge 36 of the image 35 acquired after the one image 35 is obtained. In general, it is structured.

(Configuration of the imaging unit 3)
The imaging unit 3 is an imaging device using a semiconductor element such as a CCD image sensor (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

  As an example, the imaging unit 3 is disposed in the vicinity of a touch pad 88 disposed on a floor console 80a between a driver seat and a passenger seat of the vehicle 8, as shown in FIG. Therefore, the imaging area 30 is an area above the touch pad 88. The imaging unit 3 images, for example, a gesture performed above the touch pad 88. Moreover, this imaging part 3 is comprised so that near infrared rays may be irradiated to the imaging region 30 direction, for example, and it may image.

  The arrangement of the imaging unit 3 is not limited to the floor console 80a. For example, the center console 80b may be used so that a gesture made on the front surface of the display device 87 can be captured.

The imaging unit 3 periodically images the imaging region 30, generates image information S ₁ as information on the captured image, and outputs the image information S ₁ to the control unit 5. An image obtained by imaging the imaging region 30 is an image 35 shown in FIGS. 2A to 2D as an example.

(Configuration of control unit 5)
The control unit 5 includes, for example, a CPU (Central Processing Unit) that performs operations and processes on acquired data according to a stored program, a RAM (Random Access Memory) that is a semiconductor memory, a ROM (Read Only Memory), and the like. Microcomputer. For example, a program for operating the control unit 5 and gesture information 55 are stored in the ROM. For example, the RAM is used as a storage area for temporarily storing calculation results and the like.

  For example, the control unit 5 extracts the hand 9 from the difference between the image 35 (background image) that does not include the hand 9 and the image 35 that includes the hand 9, and extracts the hand 9 and the gesture information 55. It is configured to determine a gesture by a pattern matching method based on it. Note that a method other than the pattern matching method may be used for the determination of the hand 9 and the determination of the gesture. The background image may be, for example, an image captured first after the gesture determination device 1 is activated, or may be a preset image, or an image that is regularly updated during traveling or the like. It may be.

  The gesture information 55 is information regarding a gesture performed by the hand 9, for example. The gesture performed by the hand 9 is, for example, a gesture for moving the hand 9 up and down or left and right, a gesture performed by the hand 9 so as to draw a circle, and the like. Note that the vertical and horizontal directions of the vehicle 8 are based on the vertical and horizontal directions unless otherwise specified.

For example, as shown in FIG. 2C, the gesture for moving the hand 9 up and down or left and right extends the hand 9 and faces the palm toward the imaging unit 3, that is, the hand 9 is held on the touch pad 88. To be done. As an example, this gesture is a gesture for scrolling the map image displayed on the display device 87 by the navigation device 84 in the moving direction of the hand 9. The navigation device 84, for example, to scroll the map image in the moving direction of the hand 9 based on the operation information S _2.

The gesture performed so as to draw a circle with the hand 9 is, for example, a gesture performed with a finger raised. This gesture is, for example, a gesture for adjusting the volume of the voice guide of the navigation device 84, the volume of the music and the video playback device 85, and the like. For example, when the operator raises his finger and performs a gesture for drawing a circle in the clockwise direction on the imaging unit 3, the volume increases, and when the gesture for drawing a circle in a counterclockwise direction is performed, the volume decreases. In the case of the air conditioner 86, for example, the set temperature and the air volume is changed based on the operation information S _2.

  As shown in FIG. 2A, when the hand 9 cannot be recognized in the image 35, the control unit 5 sets a recognition area 50 on the edge 36 of the image 35 and performs a recognition process only on the recognition area 50. That is, the control unit 5 does not perform the recognition process using the entire image 35 as a recognition area, but performs the recognition process using only the recognition area 50 that is set in a limited manner at the edge 36 of the image 35. In other words, the recognition area 50 is formed around the central area 51 of the image 35.

  In order for the hand 9 to be captured as the image 35, the hand 9 must have entered the imaging region 30. When the captured image 35 does not include the hand 9, the vicinity of the edge 36 of the image 35 is more likely than the possibility that the hand 9 is recognized at the center of the captured image 35 next to the captured image 35. Is more likely to be recognized. This is because, for example, as shown in FIG. 2B, the operator sitting in the driver seat or the passenger seat holds the hand 9 over the touch pad 88 from above the left and right directions of the imaging region 30. This is because the direction in which the hand 9 appears in the image 35 is determined in advance in the left-right direction of the imaging region 30.

  Therefore, the control unit 5 is more likely to recognize the hand 9 when performing the recognition process on the recognition region 50 of the edge 36 than performing the recognition process on the central region 51 of the image 35. The load is reduced.

  The width of the recognition area 50 may be any width that allows an object to be recognized, and is about 1/10 to 1/4 of the vertical side (short side) of the image 35 as an example. This width may be different between the vertical side and the horizontal side of the image 35. The vertical side may be wider than the horizontal side.

  When the object including the hand 9 is recognized in the recognition area 50 set on the edge 36 of the image 35, the control unit 5 is configured to set a determination area 52 for determining a gesture on the entire image 35. . In other words, the control unit 5 cancels the recognition area 50 in order to determine a gesture when not only the hand 9 as an object in the recognition area 50 is recognized but also the recognition of the object due to a change in luminance in the recognition area 50 or the like. Thus, the entire image 35 is set in the determination area 52.

  As a modification, when the direction in which the hand 9 is reflected in the image 35 is determined in advance, the control unit 5 partially sets the recognition area 50 on the edge 36 of the image 35 corresponding to the direction in which the hand 9 is reflected. Specifically, as illustrated in FIG. 2D, the control unit 5 sets the recognition area 50 only on the vertical side that is likely to pass when the hand 9 is reflected in the imaging area 30. The recognition area 50 of this modification is a first recognition area 50a and a second recognition area 50b on the left side of the drawing in FIG. According to this modification, since the recognition area 50a and the recognition area 50b are smaller than the above-described recognition area 50, the gesture determination device 1 can further reduce the processing load.

  As a modification, the control unit 5 may be configured to set the recognition region 50 at the edge 36 of the first image 35 acquired after the control unit 5 is activated. In this modification, the recognition area 50 is set continuously from the first image 35. In this modification, the condition is that the hand 9 is not located in the imaging region 30.

  The condition is, for example, based on vehicle information acquired from the vehicle control unit 83 of the vehicle communication system 81 and operated with the left hand when the passenger is not seated in the front passenger seat from the opening and closing of the door. The operating device to be operated (such as an ignition switch or a lever) is being operated.

(Configuration of vehicle communication system 81)
The vehicle communication system 81 is a communication system using a standard such as CAN (Controller Area Network) as an example. In the vehicle communication system 81, for example, the gesture determination device 1 and the navigation device 84 communicate with each other via a vehicle LAN (Local Area Network) 82.

  For example, the vehicle communication system 81 includes a gesture determination device 1, a vehicle control unit 83, a navigation device 84, a music and video playback device 85, an air conditioner 86, a display device 87, and a touch as shown in FIG. The pad 88 and the like are configured to be able to communicate. The vehicle control unit 83 is, for example, a microcomputer that controls the vehicle communication system 81.

When the gesture determination device 1 operates the navigation device 84, the operation information S ₂ output from the gesture determination device 1 is input to the navigation device 84 via the vehicle LAN 82. The navigation device 84 performs an operation based on the operation information S ₂ input.

  Below, an example of operation | movement of the gesture determination apparatus 1 of this Embodiment is demonstrated according to the flowchart of FIG.

(Operation)
When the power of the vehicle 8 is turned on, the imaging unit 3 of the gesture determination device 1 periodically outputs image information S ₁ that is information of the captured image 35 of the imaging region 30. Control unit 5 acquires the image 35 on the basis of image information _{S 1} (Step1).

  The control unit 5 confirms whether the hand 9 that is the recognition target is recognized based on the image 35. When the hand 9 is not recognized (Step 2: No), the control unit 5 sets the recognition area 50 in the image 35 acquired thereafter (Step 3).

  The control unit 5 sets a recognition area 50 for each acquired image 35 and checks whether an object is recognized in the recognition area 50. When “Yes” in Step 4 is established, that is, when an object is recognized in the recognition area 50 (Step 4: Yes), the control unit 5 sets the entire image 35 in the determination area 52 (Step 5), and determines the gesture. Processing is performed (Step 6).

Control unit 5, the determination processing of the gesture is completed, and outputs to the electronic device connected generated by the operation information S ₂ based on the determination result (Step7), and ends the determination process of the gesture. The operation information S _2, as an example, is also output when the gesture is not determined.

  Here, when the hand 9 is recognized in Step 2 (Step 2: Yes), the control unit 5 advances the process to Step 6 and performs a gesture determination process.

  If the hand 9 is recognized and disappears from the image 35, that is, if the hand 9 cannot be extracted from the image 35 due to a frame-out or the like, the control unit 5 stops the gesture determination process.

(Effect of embodiment)
The gesture determination apparatus 1 according to the present embodiment can suppress the processing load associated with gesture determination. Specifically, the gesture determination device 1 does not perform recognition processing for the recognition target for the entire image 35, and sets the recognition region 50 limited to the edge 36 of the image 35. Therefore, the gesture determination device 1 can suppress the processing load associated with the determination of the gesture as compared with the case where the recognition process is performed on the entire image.

  Since the gesture determination device 1 can operate the operation target by performing a gesture so that the hand 9 is held over the touch pad 88, the movement of the line of sight is suppressed and the operation load is reduced as compared with the case of operating only with the touch pad. Can be reduced.

  The gesture determination device 1 described above is configured to detect a gesture performed above the touch pad 88, but is not limited thereto, and detects a gesture performed near the steering, the center console 80b, or the like. It may be configured to.

  The gesture determination apparatus 1 according to the above-described embodiment and modification is, for example, a program executed by a computer, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like, depending on applications. It may be realized by.

  As mentioned above, although some embodiment and modification of this invention were demonstrated, these embodiment and modification are only examples, and do not limit the invention based on a claim. These novel embodiments and modifications can be implemented in various other forms, and various omissions, replacements, changes, and the like can be made without departing from the scope of the present invention. In addition, not all combinations of features described in these embodiments and modifications are necessarily essential to the means for solving the problems of the invention. Further, these embodiments and modifications are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 1 ... Gesture determination apparatus, 3 ... Imaging part, 5 ... Control part, 8 ... Vehicle, 9 ... Hand, 30 ... Imaging area, 35 ... Image, 36 ... Edge, 50 ... Recognition area, 50a ... 1st recognition area, 50b ... 2nd recognition area, 51 ... Center area, 52 ... Determination area, 55 ... Gesture information, 80a ... Floor console, 80b ... Center console, 81 ... Vehicle communication system, 82 ... Vehicle LAN, 83 ... Vehicle control part, 84 ... Navigation device, 85 ... Video playback device, 86 ... Air conditioner, 87 ... Display device, 88 ... Touchpad

Claims (4)

  When the gesture made by the recognition target is determined based on a plurality of images captured by the imaging unit and the recognition target cannot be recognized in one image, the recognition target is placed on the edge of the image acquired after the one image. A gesture determination apparatus including a gesture determination unit that sets a recognition region for recognizing an object including the object. The gesture determination unit sets the recognition region at the edge of the first image acquired after the gesture determination unit starts.
The gesture determination apparatus according to claim 1. The gesture determination unit, when a direction in which the recognition target is reflected in an image is determined in advance, partially sets the recognition area at an edge of the image corresponding to the reflection direction;
The gesture determination apparatus according to claim 1 or 2. The gesture determination unit sets a determination region for determining the gesture in the entire image when the object including the recognition target is recognized in the recognition region set as an edge of the image.
The gesture determination apparatus according to any one of claims 1 to 3.

Images