JP2015141616A - Operation input device, operation input method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an operation input device for improving the detection accuracy of a gesture by suppressing the influence of disturbance, and to provide an operation input method and a program.SOLUTION: The operation input device includes infrared irradiation means for irradiating an infrared ray, imaging means for imaging an image from the infrared ray, image processing means for detecting the shape of an indication object from the imaged image, infrared control means for adjusting the irradiation intensity of the infrared ray, storage means for respectively storing first luminance of a first area where the infrared ray is irradiated in the imaged image in the case that the irradiation intensity is adjusted, and second luminance of each of a plurality of second areas, luminance operation means for calculating third luminance of each of the second areas, and calculating a luminance correction value to correct the first luminance in the case that the absolute value of difference between the third luminance and the second luminance, which corresponds to at least one of the second areas is equal to or more than a predetermined value, determination means for determining operation contents by the indication object from the shape, and instruction means for making an operation object device execute a function corresponding to the operation contents.

Description

  The present invention relates to an operation input device, an operation input method, and a program.

  In recent years, automobiles are equipped with many devices such as a car navigation device, an audio system, and an air conditioner. An operation by pressing a normal button when operating these devices by the driver is accompanied by movement of the driver's line of sight. Therefore, since the operation of the device by the button operation while driving the automobile is an incentive leading to the driver's carelessness, it is desirable that the device can be operated while looking forward.

  As such an operation input device that enables the device to be operated while looking forward, the camera captures the shape and gesture (hereinafter referred to as a gesture) of the driver's hand with the camera, determines the gesture, An apparatus for performing the above operation has been proposed (see Patent Document 1). The operation input device described in Patent Document 1 uses a visible light camera as a camera mainly during the daytime and uses an infrared camera at night to capture and discriminate gestures.

JP 2009-104297 A

  However, when using an infrared camera, the operation input device described in Patent Literature 1 is affected by disturbance due to sunlight during the daytime, and thus has a problem that gesture detection accuracy is poor. In addition, when an infrared camera is used at night and a visible light camera is used during the day, two cameras are required, leading to an increase in cost.

  The present invention has been made in view of the above, and an object of the present invention is to provide an operation input device, an operation input method, and a program that improve the accuracy of gesture detection by suppressing the influence of disturbance.

  In order to solve the above-described problems and achieve the object, the present invention includes an infrared irradiation unit that irradiates infrared rays, an imaging unit that detects infrared rays and images an image, and the image captured by the imaging unit. Image processing means for detecting the shape of the pointing object from a certain captured image, infrared control means for adjusting the irradiation intensity of infrared rays emitted from the infrared irradiation means, and the irradiation intensity adjusted by the infrared control means A first area used for detecting the shape by the image processing unit in a first area irradiated with infrared rays by the infrared irradiation unit in the captured image, and disposed outside the first area in the captured image. Storage means for storing the second luminance of each of the plurality of second areas, and the plurality of second errors in the captured image captured by the imaging means. If the absolute value of the difference between the third luminance and the second luminance corresponding to at least one of the plurality of second areas is greater than or equal to a predetermined value, A brightness calculation unit that calculates a brightness correction value based on the brightness correction value, and a determination unit that determines an operation content by the pointing object from the shape detected by the image processing unit. And command means for causing the operation target device to execute a function corresponding to the operation content.

  ADVANTAGE OF THE INVENTION According to this invention, the influence of a disturbance can be suppressed and the detection accuracy of a gesture can be improved.

FIG. 1 is a diagram illustrating an example of the overall configuration of the operation input system according to the embodiment. FIG. 2 is a diagram illustrating an example of a block configuration of the input controller according to the embodiment. FIG. 3 is a flowchart illustrating an operation example of luminance adjustment processing of the infrared LED in the input controller according to the embodiment. FIG. 4 is a diagram illustrating an example of an image captured by the camera unit of the embodiment and a sample area. FIG. 5 is a flowchart illustrating an operation example of gesture operation recognition processing in the input controller according to the embodiment. FIG. 6 is a diagram illustrating an example of a binarized image captured by the input controller according to the embodiment. FIG. 7 is a diagram illustrating a contour extraction image obtained by performing distance conversion and contour extraction from a binarized image captured by the input controller according to the embodiment.

  Hereinafter, embodiments of an operation input device, an operation input method, and a program according to the present invention will be described in detail with reference to the drawings. Further, the present invention is not limited by the following embodiments, and components in the following embodiments can be easily conceived by those skilled in the art, are substantially the same, and have a so-called equivalent range. Is included. Furthermore, various omissions, substitutions, and changes of the constituent elements can be made without departing from the scope of the following embodiments.

(Operation input system configuration)
FIG. 1 is a diagram illustrating an example of the overall configuration of the operation input system according to the embodiment. The overall configuration of the operation input system 1 according to the present embodiment will be described with reference to FIG. In the following description, a case where the operation input system 1 shown in FIG. 1 is mounted on an automobile will be described as an example. However, the present invention is not limited to this and may be mounted on a train or an airplane. .

  An operation input system 1 shown in FIG. 1 is a system that recognizes a driver's gesture and operates an operation target device according to an operation command corresponding to the gesture. The operation input system 1 is configured by connecting an input controller 10 (operation input device) and a car navigation device 20 via a communication cable 30.

  The input controller 10 is a device that recognizes a driver's gesture, generates an operation command corresponding to the gesture, and transmits the operation command to the car navigation device 20 via the communication cable 30. The input controller 10 includes a plurality of infrared LEDs 401 that emit infrared rays, a camera 411 that captures an image within a predetermined angle of view, and an infrared transmission filter 412 that is attached to the incident side of the optical system of the camera 411 and transmits only infrared rays. And. Moreover, the input controller 10 is installed so that it may become an image pick-up direction so that the window of a motor vehicle may not enter into the image picked up. For example, it is desirable that the input controller 10 be installed on a dashboard in a car of an automobile and be installed so as to have an imaging direction from the dashboard toward a roof in the car diagonally above. Thereby, in gesture operation recognition processing to be described later, it is possible to suppress a decrease in gesture detection accuracy due to disturbance such as sunlight entering from the window of the automobile.

  The camera 411 may be, for example, a camera that detects normal visible light, but is a camera that can detect at least infrared rays. The camera 411 includes an optical system including a lens (not shown) and the like, and a solid-state imaging device (imaging device) that generates an image by converting infrared light that has passed through the infrared transmission filter 412 and entered the optical system into an electrical signal. It is equipped with. The solid-state imaging device is realized by, for example, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor.

  As described above, since the camera 411 capable of detecting at least infrared rays and the infrared transmission filter 412 are used, the cost can be reduced as compared with the case of using an infrared camera that detects only infrared rays with high accuracy.

  As shown in FIG. 1, a plurality of infrared LEDs 401 are provided so as to surround the camera 411. However, the arrangement of the plurality of infrared LEDs 401 is not limited to the arrangement shown in FIG. Absent. Further, the infrared LED 401 may be one instead of plural.

  The car navigation device 20 is a device as an example of a device to be operated by the input controller 10, for example, a device that displays a current location of a vehicle such as a mounted car and route guidance to a destination. The car navigation device 20 executes a function corresponding to the operation command according to the operation command generated by the input controller 10 and received via the communication cable 30. The car navigation apparatus 20 includes a display screen 421 that displays a current location of a vehicle such as an automobile and route guidance to a destination.

  The communication cable 30 is a cable corresponding to the communication standard of the input controller 10 and the car navigation device 20, and is, for example, a USB cable or an Ethernet (registered trademark) cable. Note that communication between the input controller 10 and the car navigation device 20 is not limited to wired communication using the communication cable 30, and may be wireless communication based on standards such as WiFi (Wireless Fidelity) or Bluetooth (registered trademark). Good.

  1 shows the car navigation device 20 as an example, but the operation target device is not limited thereto, and may be an audio system, an air conditioner, or other in-vehicle devices.

  In FIG. 1, the input controller 10 and the car navigation device 20 as the operation target device are configured as separate units, but the present invention is not limited thereto, and may be configured as an integrated type. Good. In this case, when the car navigation device 20 is installed on the dashboard, the position of the camera 411 is preferably provided at a position that is an imaging direction toward the roof in the vehicle obliquely above the dashboard.

(Block configuration of input controller)
FIG. 2 is a diagram illustrating an example of a block configuration of the input controller according to the embodiment. A block configuration of the input controller 10 of the present embodiment will be described with reference to FIG.

  As shown in FIG. 2, the input controller 10 includes an infrared LED unit 11 (infrared irradiation means), a camera unit 12 (imaging means), a communication interface (I / F) 13, and a control unit 14. Yes.

  The infrared LED unit 11 includes the infrared LED 401 shown in FIG. 1 and irradiates infrared rays according to a command from the LED control unit 142 described later.

  The camera unit 12 is a unit that captures an image and transmits the captured image to the control unit 14. The camera unit 12 is realized by a camera 411 equipped with the infrared transmission filter 412 shown in FIG.

  The communication interface 13 is an interface corresponding to the communication standard of the car navigation device 20 that is the operation target device, and is, for example, a USB interface or an Ethernet interface.

  The control unit 14 is responsible for overall control of the input controller 10. Specifically, the control unit 14 adjusts the intensity of the infrared light emitted from the infrared LED unit 11, performs various processes on the image captured by the camera unit 12, determines the driver's gesture based on the captured image, and The transmission of the operation command corresponding to the determined gesture to the car navigation device 20 is controlled. The control unit 14 includes a luminance calculation unit 141 (luminance calculation unit), an LED control unit 142 (infrared control unit), an image processing unit 143 (image processing unit), a gesture determination unit 144 (determination unit), and a command generator. A unit 145 (instruction unit) and a storage unit 146 (storage unit).

  In the image captured by the camera unit 12, the luminance calculation unit 141 is an average value of luminance in an area corresponding to an infrared irradiation area from the infrared LED unit 11 and an average value of luminance in a predetermined area outside the irradiation area. Is calculated. The luminance calculation unit 141 calculates a luminance difference between the average value of the luminance of the area corresponding to the irradiation region and the average value of the luminance of a predetermined area outside the irradiation region.

  The LED control unit 142 controls the intensity of infrared rays emitted from the infrared LED unit 11.

  The image processing unit 143 binarizes the image captured by the camera unit 12 and determines whether or not a gesture by the driver's hand is included in the binarized image.

  The gesture determination unit 144 determines what kind of gesture the gesture detected by the image processing unit 143 is. Note that the gesture is not limited to that by the driver's hand, and may be an indication object (including a hand) for giving a wide range of operation instructions.

  The command generation unit 145 generates an operation command for the car navigation device 20 as an operation target device corresponding to the gesture determined by the gesture determination unit 144.

  When the infrared light irradiated by the LED control unit 142 is controlled to a predetermined intensity, the storage unit 146 has a luminance maximum value in an area corresponding to the infrared irradiation area in the image captured by the camera unit 12 and the outside of the irradiation area. The average value of the brightness of the predetermined area is stored. The storage unit 146 is realized by a rewritable volatile or nonvolatile storage device such as a RAM (Random Access Memory) or an SSD (Solid State Drive).

  The luminance calculation unit 141, the LED control unit 142, the image processing unit 143, the gesture determination unit 144, and the command generation unit 145 are executed by a CPU (Central Processing Unit) or the like that executes a program stored in a ROM (Read Only Memory) or the like (not shown). It is a function or means realized by doing. Note that some or all of the luminance calculation unit 141, the LED control unit 142, the image processing unit 143, the gesture determination unit 144, and the command generation unit 145 may be realized by a hardware circuit instead of a program that is software. Further, the luminance calculation unit 141, the LED control unit 142, the image processing unit 143, the gesture determination unit 144, and the command generation unit 145 are conceptually configured functions, and are not limited to such a configuration. Absent.

(Brightness adjustment processing operation)
FIG. 3 is a flowchart illustrating an operation example of luminance adjustment processing of the infrared LED in the input controller according to the embodiment. FIG. 4 is a diagram illustrating an example of an image captured by the camera unit of the embodiment and a sample area. With reference to FIGS. 3 and 4, a description will be given of luminance adjustment processing for infrared rays emitted from the infrared LED unit 11 in the input controller 10 of the present embodiment.

<Step S11>
The LED control unit 142 of the control unit 14 irradiates the infrared LED unit 11 with an infrared ray having a predetermined intensity with a certain extent. The camera unit 12 captures an image in the imaging direction toward the roof of the automobile, and transmits the image to the luminance calculation unit 141 of the control unit 14. The image picked up by the camera unit 12 is a black and white image picked up by infrared rays reflected from the roof of an automobile among the infrared rays irradiated from the infrared LED unit 11 through the infrared transmission filter 412. Then, the process proceeds to step S12.

<Step S12>
As shown in FIG. 4, the luminance calculation unit 141 is a predetermined area included in an irradiation area 251 that is an infrared area irradiated on the roof from the infrared LED unit 11 in the captured image 201 that is a received image. An average value of luminance in the irradiation area sample area 202 (first area) (hereinafter referred to as an irradiation unit luminance average value) (fourth luminance) is calculated. In addition, the luminance calculation unit 141 in the captured image 201 is an average value of luminance of the peripheral sample areas 203a to 203d (second area) that is a predetermined area outside the irradiation region 251 (hereinafter referred to as a peripheral luminance average value). ) Is calculated. The luminance calculation unit 141 extracts a maximum average value (hereinafter referred to as a peripheral luminance maximum average value) (fifth luminance) from the calculated average luminance values of the peripheral sample areas 203a to 203d. In this case, the luminance of the irradiation region sample area 202 in the irradiation region 251 irradiated with infrared rays is high, and the luminance of the peripheral portion sample areas 203a to 203d outside the irradiation region 251 not irradiated with infrared rays is low. And the brightness | luminance calculating part 141 calculates the brightness | luminance difference which is a difference of an irradiation part brightness | luminance average value and a peripheral part brightness | luminance maximum average value. Then, the process proceeds to step S13. The peripheral sample areas 203a to 203d are simply referred to as the peripheral sample area 203 when referred to without distinction or collectively.

  As in the peripheral part sample areas 203a to 203d shown in FIG. 4, the peripheral part luminance is obtained in a state in which much of the luminance information of the area outside the irradiation area 251 is included by using a plurality of sample areas for calculating the average value of the luminance. The maximum average value can be calculated. Thereby, the brightness | luminance calculating part 141 can calculate the brightness | luminance difference which is a difference of an irradiation part brightness | luminance average value and a peripheral part brightness | luminance maximum average value accurately.

  As shown in FIG. 4, one irradiation area sample area 202 and four peripheral sample areas 203a to 203d are provided. However, the present invention is not limited to this, and other numbers of sample areas are configured. It is good also as what to do. Further, as shown in FIG. 4, the peripheral sample areas 203 a to 203 d are arranged at the four corners of the captured image 201, but the present invention is not limited to this, and any area outside the irradiation region 251 can be used. It is good also as what is arrange | positioned in any. However, if the captured image 201 includes a window portion of the automobile, the peripheral sample areas 203a to 203d should be configured at positions avoiding the window portion in order to reduce the influence of infrared luminance due to sunlight. Is desirable.

<Step S13>
The LED control unit 142 compares the luminance difference with a predetermined target value. As a result of the comparison between the luminance difference and the target value by the LED control unit 142, when the luminance difference is larger than the target value (step S13: large), the process proceeds to step S14, and when the luminance difference is smaller than the target value (step S13: Small), the process proceeds to step S15, and if the luminance difference is the same as the target value (step S13: the same), the process proceeds to step S16.

  Note that the LED control unit 142 performs a comparison between the luminance difference and the predetermined target value, but whether or not the luminance difference is within the predetermined target range is determined as to whether or not the luminance difference is within the predetermined target range. It is a concept including what determines whether or not. In this case, when the luminance difference is larger than the predetermined target range, the process proceeds to step S14. When the luminance difference is smaller than the predetermined target range, the process proceeds to step S15, and when the luminance difference is within the predetermined target range. The process proceeds to step S16.

<Step S14>
The LED control unit 142 reduces the intensity of the infrared light emitted from the infrared LED unit 11 by a predetermined amount. Then, returning to step S11, the camera unit 12 captures an image again.

<Step S15>
The LED control unit 142 increases the intensity of infrared rays emitted from the infrared LED unit 11 by a predetermined amount. Then, returning to step S11, the camera unit 12 captures an image again.

  In this way, by repeating the operations of step S14 and step S15, the intensity of the infrared rays emitted from the infrared LED unit 11 is adjusted so that the luminance difference matches a predetermined target value.

<Step S16>
When the difference in luminance and the predetermined target value are the same, the LED control unit 142 maintains the irradiation intensity of infrared rays from the infrared LED unit 11 at that time, and the luminance calculation unit 141 displays the luminance of the irradiation area sample area 202. The maximum value (first luminance) is obtained, and the maximum luminance value is stored in the storage unit 146. Further, the LED control unit 142, when the luminance difference and the predetermined target value are the same, the peripheral luminance average values (first values) of the peripheral sample areas 203a to 203d calculated by the luminance calculation unit 141 in step S12. 2 luminance) is stored in the storage unit 146. Here, the peripheral luminance average value stored in the storage unit 146 is referred to as a “reference luminance average value”.

  The brightness adjustment process is executed as described above. The brightness adjustment process may be executed as an initial operation when the operation input system 1 is started, and thereafter executed every predetermined time.

(Gesture operation recognition processing)
FIG. 5 is a flowchart illustrating an operation example of gesture operation recognition processing in the input controller according to the embodiment. FIG. 6 is a diagram illustrating an example of a binarized image captured by the input controller according to the embodiment. FIG. 7 is a diagram illustrating a contour extraction image obtained by performing distance conversion and contour extraction from a binarized image captured by the input controller according to the embodiment. A gesture operation recognition process in the input controller 10 of the present embodiment will be described with reference to FIGS.

<Step S31>
The camera unit 12 captures an image in the imaging direction toward the roof of the automobile, and transmits the image to the luminance calculation unit 141 of the control unit 14. Then, the process proceeds to step S32.

<Step S32>
The luminance calculation unit 141 calculates the peripheral luminance average value (third luminance) of each of the peripheral sample areas 203a to 203d in the captured image 201 (see FIG. 4) that is the received image. In addition, the luminance calculation unit 141 calculates the absolute value of the difference between the calculated peripheral luminance average value and the reference luminance average value stored in the storage unit 146 for each peripheral sample area 203. And the brightness | luminance calculating part 141 determines whether the calculated absolute value is more than a predetermined value, respectively. When the absolute values corresponding to all the peripheral sample areas 203 are greater than or equal to a predetermined value, the brightness calculation unit 141 causes a change in the brightness of the external environment such as a car on which the operation input system 1 is mounted enters a tunnel. The process proceeds to step S33. On the other hand, if at least one of the absolute values corresponding to the peripheral sample area 203 is less than the predetermined value, the luminance calculation unit 141 determines that there is no change in the brightness of the external environment, and proceeds to step S34. .

<Step S33>
The luminance calculation unit 141 calculates an average value of the difference between the peripheral luminance average value and the reference luminance average value for each peripheral sample area 203, and uses the average value as the luminance correction value. The luminance calculation unit 141 corrects the luminance maximum value stored in the storage unit 146 by adding a luminance correction value, and the luminance maximum value stored in the storage unit 146 is corrected. Update to Then, the process proceeds to step S34.

  For example, consider a case where the reference luminance average values of the peripheral sample areas 203a to 203d are “120”, “123”, “124”, and “125”. First, the brightness of the external environment becomes dark, and the luminance calculation unit 141 calculates the peripheral luminance average values of the peripheral sample areas 203a to 203d as “92”, “93”, “94”, and “93”. Shall be. In this case, the absolute value of the difference between the peripheral luminance average value and the reference luminance average value for each peripheral sample area 203 calculated by the luminance calculation unit 141 is “28”, “30”, “30”, “ 32 ". Here, if the predetermined value to be compared is “10”, all the absolute values are equal to or larger than the predetermined value, so the luminance calculation unit 141 determines that the brightness of the external environment has changed. Become. In this case, the difference between the peripheral luminance average value and the reference luminance average value for each peripheral sample area 203 calculated by the luminance calculation unit 141 is “−28”, “−30”, “−30”, “ −32 ”. Then, the luminance calculation unit 141 calculates “−30”, which is an average value of these differences, as a luminance correction value. Assuming that the maximum luminance value stored in the storage unit 146 is “135”, for example, the luminance calculation unit 141 adds the luminance correction value “−30” to the maximum luminance value “135”. Correction is performed to obtain a corrected luminance maximum value “105”.

  On the other hand, the brightness of the external environment becomes bright, and the brightness calculation unit 141 calculates the peripheral brightness average values of the peripheral sample areas 203a to 203d as “137”, “141”, “143”, and “143”. Shall be. In this case, the absolute values of the difference between the peripheral luminance average value and the reference luminance average value for each peripheral sample area 203 calculated by the luminance calculation unit 141 are “17”, “18”, “19”, “ 18 ”. Here, if the predetermined value to be compared is “10” as described above, all the absolute values are equal to or larger than the predetermined value, so that the luminance calculation unit 141 has changed in the brightness of the external environment. It will be judged as a thing. In this case, the difference between the peripheral luminance average value and the reference luminance average value for each peripheral sample area 203 calculated by the luminance calculation unit 141 is “17”, “18”, “19”, and “18”, respectively. Become. Then, the luminance calculation unit 141 calculates “18”, which is an average value of these differences, as a luminance correction value. Assuming that the maximum luminance value stored in the storage unit 146 is “135” as described above, the luminance calculation unit 141 adds the luminance correction value “18” to the maximum luminance value “135”. Thus, the corrected maximum luminance value “153” is obtained.

  In step S32, the luminance calculation unit 141 determines that the brightness of the external environment has changed when the absolute values corresponding to all the peripheral sample areas 203 are equal to or greater than the predetermined value. However, the present invention is not limited to this. For example, the luminance calculation unit 141 may determine that the brightness of the external environment has changed when the absolute value corresponding to more than half of the plurality of peripheral sample areas 203 is greater than or equal to a predetermined value. . In this case, in step S33, the luminance calculating unit 141 calculates the average value of the difference between the peripheral average value and the reference luminance average corresponding to the peripheral sample area 203 whose absolute value is equal to or greater than the predetermined value, and calculates the average value. A brightness correction value may be used.

  In step S33, the luminance calculation unit 141 calculates the average value of the difference between the peripheral luminance average value and the reference luminance average value for each peripheral sample area 203, and uses this average value as the luminance correction value. It is not limited to this. For example, the luminance calculation unit 141 may use, as the luminance correction value, the maximum value or the minimum value among the differences between the peripheral luminance average value for each peripheral sample area 203 and the reference luminance average value, or the average of the differences A value obtained by multiplying a value, a maximum value or a minimum value by a predetermined coefficient, or a value obtained by adding a predetermined value may be used as the luminance correction value.

<Step S34>
The luminance calculation unit 141 binarizes the image by comparing the pixel value of each pixel constituting the received image with the maximum luminance value stored in the storage unit 146, and binarized image 201a shown in FIG. Is generated. For example, if the pixel value of each pixel constituting the received image is larger than the maximum luminance value, the luminance calculation unit 141 converts the value to a pixel having a value of “1”, and if the pixel value is equal to or less than the maximum luminance value, Is converted to a pixel having “0” to generate a binarized image 201a. Then, the process proceeds to step S35.

<Step S35>
A binarized image 201a illustrated in FIG. 6 illustrates an example of binarizing a captured image, and includes blocks 301 to 303 as blocks that are blocks of pixels having a pixel value “1”. When the driver holds his hand in front of the camera unit 12 of the input controller 10, the infrared light emitted from the infrared LED 401 hits his hand. Before the hand is held up, the infrared light emitted from the infrared LED 401 hits the roof in the vehicle, but when the hand is held up, it is reflected at the position of the hand shorter than the distance to the roof and enters the camera unit 12. . Therefore, in the image captured by the camera unit 12 with the hand held up, the brightness of the hand region portion is the brightness of the portion corresponding to the roof in the image captured by the camera unit 12 with the hand held up. It becomes a high value in comparison. Therefore, when the image captured by the camera unit 12 is binarized, the pixel value of the pixel in the hand region portion is “1”, and the pixel value of the pixel in the portion other than the hand region is “0”.

  However, if the captured image is binarized by the above-described method, fine noise may be generated. Therefore, it is desirable to apply a filter for noise removal. In order to remove noise from the binarized image, for example, filter processing is performed using a median filter. Here, the median filter refers to the pixel value of the pixel in the n × n local region in the filter target image arranged in ascending order, and the pixel value corresponding to the middle among the arranged pixel values is the pixel of the pixel in the center of the local region. A filter that replaces as a value. Even after such a filter for noise removal is applied, there are cases where blocks other than the hand region block remain. Therefore, when there are a plurality of blocks in the binarized image 201a, the image processing unit 143 calculates the area of each block and extracts the block having the maximum area as a candidate for a block in the hand region. In FIG. 6, the image processing unit 143 extracts the block 301 having the largest area among the blocks 301 to 303 as a candidate for the hand region block. Then, the process proceeds to step S36.

<Step S36>
The image processing unit 143 compares the area of the block 301 that is a candidate for the hand region in the binarized image 201a with a predetermined threshold. If the area is larger than the threshold, the block 301 corresponds to the hand region. Is determined. If the area of the block 301 that is a candidate for the hand region in the binarized image 201a is equal to or smaller than a predetermined threshold, the image processing unit 143 determines that the block 301 is not a block of the hand region, and performs gesture operation recognition processing. finish. On the other hand, if the area of the block 301 that is a candidate for the hand region is larger than the predetermined threshold, the image processing unit 143 determines that the block 301 is a block of the hand region, and proceeds to step S37.

<Step S37>
The image processing unit 143 obtains a contour extraction image 201b including a contour extraction block 301a shown in FIG. 7 in which the contour line of the block 301 determined as a hand region block is extracted from the binarized image 201a. Specifically, the image processing unit 143 performs distance conversion processing on an image obtained by deleting the blocks 302 and 303 that are blocks other than the block 301 in the binarized image 201a. Here, the distance conversion process is a process of replacing the pixel value of the pixel having a pixel value “1” in the binarized image with the distance to the pixel having the closest pixel value “0” from the pixel. is there. By connecting pixels with a distance of “1” in the distance-converted image, the contour line of the figure in the image can be obtained. The image obtained in this way is the contour extraction image 201b. Further, the image processing unit 143 calculates the length of the contour line of the contour extraction block 301a. Then, the process proceeds to step S38.

<Step S38>
Here, the gesture determination unit 144 determines, for example, which gesture is “go”, “choki”, or “par” as a gesture by the driver's hand. When the hand gesture is “par”, the outline of each finger appears, so the outline becomes longer. When it is “Goo”, the shape of the hand becomes an ellipse, so the outline becomes shorter. There is a feature that becomes. Here, (length of contour line) / (square root of area) is defined as the feature amount. In the case of a complex figure such as “Par”, the above-mentioned feature amount is large. In the case of a simple figure such as “Goo”, the feature amount is small. Since this value is an intermediate value, it is possible to determine the gestures of “goo”, “choki”, and “par” by adopting this feature amount. The gesture determination unit 144 calculates the above-described feature amount from the area of the block 301 calculated by the image processing unit 143 and the length of the contour line of the contour extraction block 301a, and based on the feature amount, the gesture of the hand is calculated. Is “goo”, “choki” or “par”. Then, the process proceeds to step S39.

  As described above, “GOO”, “CHOKI”, and “PAR” are given as examples of gestures determined by the gesture determination unit 144, but the gestures are not limited thereto. For example, the gesture determination unit 144 may determine a gesture by detecting the number of fingers that are raised.

  In addition, the feature amount for determining the gesture by the gesture determination unit 144 is (contour length) / (square root of area), but is not limited to this. Any feature amount may be used as long as it is a possible feature amount.

  In addition, the gesture determination unit 144 is not limited to static driver gestures such as “goo”, “choki”, and “par” as described above, but compared to past gestures, It is good also as what determines dynamic gestures, such as a motion, its direction, and bending and stretching of a finger.

<Step S39>
The command generation unit 145 generates an operation command for the car navigation device 20 corresponding to the gesture determined by the gesture determination unit 144. Then, the process proceeds to step S40.

<Step S40>
The command generation unit 145 transmits the generated operation command to the car navigation device 20 via the communication interface 13 and the communication cable 30. The car navigation device 20 executes a function corresponding to the operation command in accordance with the received operation command.

  For example, when the gesture determined by the gesture determination unit 144 is “par”, the car navigation device 20 changes the map displayed on the display screen 421 to a city map. Further, the car navigation device 20 enlarges the map displayed on the display screen 421 when the gesture is “Choki”, and reduces the map displayed on the display screen 421 when the gesture is “Goo”. To do.

  The gesture operation recognition process as described above is repeatedly executed. By performing the gesture operation recognition process as described above, it is possible to operate the car navigation device 20 that is the operation target device by a driver's gesture.

  In step S35, a block having some maximum area is extracted by the image processing unit 143. If the area of the block is equal to or smaller than a predetermined threshold value in step S36, the driver performs some gesture operation. There is a high possibility that it has been tried, and there is a high possibility that the brightness adjustment processing has not been properly executed. As described above, when the area of the block is equal to or smaller than the predetermined threshold by the image processing unit 143 in step S36 and the gesture operation recognition process is completed, the control unit 14 desirably executes the brightness adjustment process.

  As described above, the input controller 10 stores the peripheral brightness average value of each peripheral sample area 203 in the brightness adjustment process and sets it as the reference brightness average value. In the gesture operation recognition process, the input controller 10 determines the peripheral brightness average value and the reference brightness value. Whether or not the brightness of the external environment has changed is determined based on the absolute value of the difference from the average brightness value. When the input controller 10 determines that the brightness of the external environment has changed, the input controller 10 calculates the brightness correction value based on the difference between the peripheral brightness average value and the reference brightness average value, and the brightness correction value is used to determine the brightness. The maximum value is to be corrected. As a result, when the brightness of the external environment changes, such as when a car enters or exits a tunnel, the maximum luminance value is corrected, and a gesture is detected based on the corrected maximum luminance value. Therefore, the influence of disturbance can be suppressed and the gesture detection accuracy can be improved.

  Further, in the infrared image captured by the infrared LED unit 11, the luminance difference that is the difference between the luminance of the irradiation region sample area 202 and the luminance of the peripheral sample areas 203a to 203d becomes a target luminance difference. The irradiation intensity of infrared rays emitted from the infrared LED unit 11 is adjusted. Thereby, since the irradiation intensity of infrared rays is adjusted according to the change in the surrounding environment of the input controller 10, the influence of disturbance on the gesture operation recognition process can be suppressed, and the gesture detection accuracy can be improved. In addition, since infrared rays are used without using visible light, they can be used at night and are not easily affected by sunlight as disturbances even in the daytime, so that the reliability of the device of the input controller 10 is improved. . Furthermore, since the irradiation intensity of infrared rays emitted from the infrared LED unit 11 is adjusted so as to have a target luminance difference as described above, an increase in power consumption can be suppressed.

  In addition, the operation input system 1 according to the present embodiment does not require the use of two cameras, such as using a visible light camera during the daytime and using an infrared camera during the nighttime, thereby suppressing an increase in cost. can do.

  In step S12 shown in FIG. 3 and step S32 shown in FIG. 5, the average value of the luminance of each peripheral sample area 203 is calculated as the peripheral luminance average value. However, the present invention is not limited to this. It is not a thing. For example, the maximum value or minimum value of the brightness of each peripheral sample area 203 may be calculated.

  Further, in step S16 shown in FIG. 3, the maximum luminance value that is the maximum luminance value of the irradiation region sample area 202 is obtained, and in step S33 shown in FIG. 5, the maximum luminance value is corrected by the luminance correction value. In step S34, the maximum luminance value is used as a reference value for image processing (binarization) on the captured image, but the present invention is not limited to this. For example, in step S16, an average value of luminance of the irradiation region sample area 202 may be obtained, or a value obtained by multiplying the maximum value or average value of the luminance by a predetermined coefficient or adding a predetermined value is obtained. May be. In this case, in step S33 shown in FIG. 5, the value obtained in step S16 is corrected by the brightness correction value, and in step S34, the value obtained in step S16 is used as a reference value for image processing on the captured image. It should just be used.

DESCRIPTION OF SYMBOLS 1 Operation input system 10 Input controller 11 Infrared LED unit 12 Camera unit 13 Communication interface 14 Control unit 20 Car navigation apparatus 30 Communication cable 141 Luminance calculation part 142 LED control part 143 Image processing part 144 Gesture determination part 145 Command generation part 146 Storage part 201 Captured Image 201a Binary Image 201b Contour Extracted Image 202 Irradiation Area Sample Area 203, 203a-203d Peripheral Sample Area 251 Irradiation Area 301-303 Block 301a Contour Extraction Block 401 Infrared LED
411 Camera 412 Infrared transmission filter 421 Display screen

Claims (8)

Infrared irradiation means for irradiating infrared rays;
An image pickup means for picking up an image by detecting infrared rays;
Image processing means for detecting the shape of the pointing object from the captured image that is the image captured by the imaging means;
An infrared control means for adjusting the irradiation intensity of infrared rays emitted from the infrared irradiation means;
A first luminance used for detection of the shape by the image processing unit in a first area irradiated with infrared rays by the infrared irradiation unit in the captured image when the irradiation intensity is adjusted by the infrared control unit; And storage means for storing the second luminance of each of the plurality of second areas arranged outside the first area in the captured image;
3rd brightness | luminance of each of these 2nd area in the said picked-up image imaged by the said imaging means is calculated | required, The said 3rd brightness | luminance corresponding to at least any one of these 2nd area, and said 2nd brightness | luminance, A luminance calculation means for calculating a luminance correction value based on the difference when the absolute value of the difference is equal to or greater than a predetermined value, and correcting the first luminance by the luminance correction value;
Determination means for determining the operation content by the pointing object from the shape detected by the image processing means;
Command means for causing the operation target device to execute a function corresponding to the operation content;
An operation input device comprising:   The luminance calculation means corrects the first luminance when an absolute value of a difference between the third luminance and the second luminance corresponding to all of the plurality of second areas is equal to or greater than the predetermined value. The operation input device according to claim 1. The luminance calculation means calculates a luminance difference between the fourth luminance of the first area and the fifth luminance of the plurality of second areas,
The infrared control means adjusts the irradiation intensity so that the luminance difference becomes a predetermined target value,
The operation input device according to claim 1, wherein the luminance calculation unit causes the storage unit to store the first luminance and the second luminance when the luminance difference reaches the predetermined target value.   The operation input device according to claim 1, wherein the first luminance is a maximum luminance value of the first area.   The operation input device according to claim 1, wherein the second luminance and the third luminance are average values of luminance of the second area.   The image processing unit generates a binarized image by comparing a pixel value of each pixel constituting the captured image captured by the imaging unit with the first luminance, and generates the binarized image from the binarized image. The operation input device according to claim 1, wherein the shape of the pointing object is detected. Irradiating with infrared rays; and
Detecting and capturing infrared light to generate an image;
Detecting the shape of the pointing object from the captured image that is the captured image;
Adjusting the irradiation intensity of the infrared rays to be irradiated;
A first luminance used for detecting the shape of a first area irradiated with infrared rays in the captured image when the irradiation intensity is adjusted, and a plurality of pixels arranged outside the first area in the captured image Storing each second luminance of the second area;
An absolute value of a difference between the third luminance and the second luminance corresponding to at least one of the plurality of second areas is obtained by obtaining a third luminance of each of the plurality of second areas in the captured image. Is a predetermined value or more, calculating a luminance correction value based on the difference, and correcting the first luminance by the luminance correction value;
Determining the operation content by the pointing object from the detected shape;
Causing the operation target device to execute a function corresponding to the operation content;
An operation input method comprising: Image processing means for detecting the shape of the pointing object from a picked-up image that is an image picked up by detecting infrared rays by the image pickup means;
Infrared control means for adjusting the irradiation intensity of infrared rays emitted from the infrared irradiation means;
A first luminance used for detection of the shape by the image processing unit in a first area irradiated with infrared rays by the infrared irradiation unit in the captured image when the irradiation intensity is adjusted by the infrared control unit; And each 2nd brightness | luminance of the several 2nd area arrange | positioned outside the said 1st area in the said captured image is memorize | stored in a memory | storage means, The said 2nd multiple area in the said captured image imaged by the said imaging means When the absolute value of the difference between the third luminance and the second luminance corresponding to at least one of the plurality of second areas is equal to or greater than a predetermined value, A luminance calculation unit that calculates a luminance correction value based on the luminance correction value and corrects the first luminance by the luminance correction value;
Determination means for determining the operation content by the pointing object from the shape detected by the image processing means;
Command means for causing the operation target device to execute a function corresponding to the operation content;
A program for realizing the above on a computer.