JP5172485B2 - Input device and control method of input device - Google Patents

Description

  The present invention relates to an input device that detects a user's contact with an input surface and generates an input signal corresponding to the detection result.

  Conventionally, various input devices have been used as devices for performing input operations on electronic devices. In particular, in recent years, touch panel type input devices capable of intuitive input operations have been widely used. In addition, an input device has been proposed that can not only accept an input using a touch panel type, but also can perform an input operation more conveniently according to the application.

  For example, in an input device that operates an in-vehicle device, when the touch panel method is adopted, there is a risk that the driver's eyes may deviate from the traveling direction of the vehicle when the driver views the display on the touch panel.

  Therefore, in Patent Documents 1 and 2 below, an operation unit that performs operation input and a display unit that performs display are separated. Then, in order to make the driver recognize which switch is being operated, an image of the switch is displayed on the display unit, and an image of the position of the hand on the operation unit is superimposed on the switch image. Is displayed.

  According to the above configuration, by arranging the display unit at a position where the line of sight can be seen without much movement from the traveling direction of the vehicle, the in-vehicle device can be operated relatively safely even during traveling. .

  Further, in the conventional touch panel, only one point on the panel can be detected, but a plurality of points in contact with the panel cannot be detected at the same time. For example, when a plurality of fingers are simultaneously brought into contact with the panel, it has not been possible to detect to which finger position an input operation has been performed.

On the other hand, there has been proposed an input device that can simultaneously detect a plurality of points in contact with the panel, that is, capable of simultaneous multipoint input. For example, in a display device called a liquid crystal display device with a built-in optical sensor, an image sensor is incorporated in the liquid crystal display device, whereby an object in contact with the liquid crystal display panel can be captured as image data. Then, by analyzing this image data, a plurality of points in contact with the liquid crystal display panel can be detected simultaneously.
JP 2007-286696 A (published November 1, 2007) Japanese Patent Laid-Open No. 10-269012 (released on October 9, 1998)

  However, there has been no input device that can perform various input operations with a simple operation from a state in which a finger is in contact with the input surface of the input device. For example, in a touch panel type input device, when the finger is not in contact with the input surface of the input device, when the state is changed from the state in which the finger is in contact, the position of the finger in contact with the input surface is determined. Input is to be performed.

  A technique for performing gesture input using a photosensor built-in liquid crystal display device is known. In gesture input, an input operation corresponding to the locus can be performed by drawing a predetermined locus with the finger while the finger is in contact with the input surface of the input device.

  However, when performing gesture input, the user needs to learn a gesture preset in the input device. Therefore, gesture input is easy to apply to gestures that are easy to remember intuitively, such as enlargement, reduction, and scrolling of the display screen. It is difficult to apply to functions that are difficult to express with intuitive gestures, such as when a route guide is executed.

  The present invention has been made in view of the above problems, and its purpose is to perform various input operations with simple operations from a state where a finger is in contact with the input surface of the input device. An input device that can be used and an input device control method.

  In order to solve the above-described problems, the input device of the present invention identifies and detects contacts that are simultaneously made to a plurality of locations on the input surface that accepts an input operation, and generates an input signal according to the detection result. An input device for detecting that each of a plurality of fingers is in contact with the input surface; and a finger detected by the contact detection means as being in contact with the input surface. , And an input detection means for generating an input signal by detecting a state change from at least one finger contact to non-contact.

  Moreover, in order to solve the said subject, the control method of the input device of this invention identifies and detects the contact performed simultaneously with respect to the several location of the input surface which receives input operation, and according to the result of this detection A method of controlling an input device that generates an input signal, wherein the input surface is contacted in a contact detection step of detecting that each of a plurality of fingers is in contact with the input surface, and the contact detection step. And an input detection step of generating an input signal by detecting a change in state of at least one finger from contact to non-contact among the detected fingers.

  The above configuration is based on the premise that each contact that is simultaneously made to a plurality of locations on the input surface that accepts an input operation is identified and detected. This is a so-called multi-point simultaneous detection technique. In the present invention, by using such a multi-point simultaneous detection technique, various operations can be performed by simple operations from a state where the finger is in contact with the input surface. An object of the present invention is to provide an input device capable of performing an input operation.

  That is, according to the above configuration, after receiving the contact of a plurality of fingers on the input surface, the input signal is generated by detecting that the finger detecting the contact has moved away from the input surface. Thereby, it becomes possible to perform an input operation from a state in which a finger is in contact with the input surface.

  Here, in general, the input device detects the input operation performed by the user and generates an input signal corresponding to the detected input operation, thereby controlling the operation of the peripheral device, Accept input. For example, in the touch panel, a position where the user touches the panel is detected, and an input signal corresponding to the detected position is generated.

  That is, in the conventional input device, the input operation is performed by touching the input surface, whereas in the input device of the present invention, after touching the input surface with a plurality of fingers, the finger is separated from the input surface. Perform input operations. Thereby, various input operations can be performed with simple operations from a state where the finger is in contact with the input surface of the input device.

  Moreover, according to said structure, the new input method which was not in the past can be provided. That is, according to the above configuration, first, a plurality of finger contacts are received with respect to the input surface. Therefore, at this time, for example, it is possible to acquire information such as the number of fingers touching the input surface, the position on the input surface, the type (how many fingers), the fingerprint, and the palm print.

  In addition, when detecting that the finger has been removed from the input surface, the number of fingers away from the input surface, the position on the input surface, the type (how many fingers), the timing at which the finger has moved away from the input surface, etc. Information can be acquired.

  In the above input device, an input signal can be generated based on such a variety of information, so that a variety of input operations not possible in the past can be performed. For example, when an input signal is generated according to the number of fingers touching the input surface and the number of fingers away from the input surface, the user can determine the number of fingers touching the input surface and the number of fingers away from the input surface. Different input operations can be performed by changing the number. In addition, for example, a user can be identified by reading a fingerprint or palm print of a finger in contact with the input surface, and therefore, a different input signal can be generated for each user.

  Further, the contact detection means determines the type of each finger that is in contact with the input surface, and the input detection means changes the state from contact to non-contact based on the determination result of the contact detection means. It is preferable to generate an input signal corresponding to the detected finger type.

  According to said structure, the input signal according to the classification of the finger | toe away from the input surface is produced | generated. Thus, for example, when five fingers from the little finger to the thumb are brought into contact with the input surface and then the index finger is separated from the input surface, no matter what position on the input surface the finger is in contact with, A corresponding input signal is generated. That is, according to said structure, the same input operation can be performed irrespective of the input position with respect to an input surface.

  Further, the input device has a finger-command correspondence in which when the input signal corresponding to the type of the finger is generated, the input signal is associated with a command for causing the motion execution device to execute a predetermined motion. It is preferable to provide a device control means for transmitting a command corresponding to the input signal to the operation executing device based on the information.

  According to said structure, the input signal according to the classification of the finger | toe away from the input surface is produced | generated, and also the command corresponding to an input signal is transmitted to the operation | movement execution apparatus which performs this command. That is, according to the above configuration, the action execution device is operated by a command according to the type of the finger separated from the input surface. Therefore, the user of the input device can cause another device to execute a desired operation regardless of the position on the input surface. Note that there may be one or a plurality of operation execution devices, the operation execution devices may be integrated with the input device, or may be present at a position physically separated from the input device. .

  This is suitable, for example, when performing an input operation of an in-vehicle device. That is, in the techniques of Patent Documents 1 and 2, since it is necessary to put a finger on the position of the button displayed on the display unit, it is necessary to look at the display unit when operating the button. Therefore, since the line of sight deviates from the traveling direction of the car, the operation of the on-vehicle device while driving is also accompanied by danger.

  On the other hand, according to the above configuration, since the command is transmitted in response to the finger separated from the input surface, the position touching the input surface may be any position, and therefore, the input surface is not looked at. Can also perform input operations. Therefore, unlike the techniques of Patent Documents 1 and 2, it is not necessary to match the position of the finger to the button by looking at the display unit, and the in-vehicle device can be safely operated even during driving.

  Further, the contact detection means determines whether the finger in contact with the input surface is a right hand or a left hand, and the input detection means is based on the determination result of the contact detection means, Generate an input signal according to how many fingers of the right hand or left hand the detected finger from the contact to non-contact state, the device control means, the input according to the type of each finger of the right hand or left hand When the signal is generated, the instruction corresponding to the input signal is based on the finger-command correspondence information in which the input signal is associated with the instruction for causing the action execution device to execute a predetermined action. It is preferable to transmit to the operation execution device.

  According to the above configuration, the right finger and the left hand finger are identified and detected, and the instructions are associated with each finger, so the maximum number of instructions that can be assigned to the fingers increases. It is not always necessary to assign a command to all fingers.

  In addition, the input device extracts a command corresponding to each finger based on the finger-command correspondence information, and the command corresponding to the finger or the command at the position of each finger detected by the contact detection unit It is preferable that display means for displaying information indicating an operation executed by the operation executing device is provided.

  According to said structure, the information corresponding to the instruction | command corresponding to each finger | toe or the operation | movement which an operation | movement execution apparatus performs by this instruction | indication is displayed on the position of each finger | toe. Therefore, the user of the input device can easily recognize what instruction is generated and what operation is performed with which finger.

  Further, the apparatus control means transmits the command when the input detection means detects that the finger is again in contact with the display surface after detecting the change in state from the contact of the finger to the non-contact. Preferably it is done.

  According to the above configuration, the command is transmitted when the contact state of the finger is switched to contact-non-contact-contact. In other words, in this case, the command is not transmitted when the contact state of the finger is switched from contact to non-contact. Thereby, for example, when the user finishes the input operation on the input device and releases his / her hand from the input surface, it is possible to prevent a command not intended by the user from being transmitted.

  In addition, the input device includes an imaging unit that captures an object on the input surface as image data, and the contact detection unit includes a plurality of fingers on the input surface based on the image data captured by the imaging unit. It is preferable to detect contact.

  According to the above configuration, since the object on the input surface is captured as image data by the imaging unit, when a hand touches the input surface, each finger is input to the input surface by analyzing the image data. It is possible to detect whether or not it is touching.

  Note that the imaging unit only needs to be able to acquire image data that can determine whether an object on the input surface is in contact with the input surface. As the imaging unit, for example, a liquid crystal display device with a built-in optical sensor, an infrared camera, or the like can be applied.

  Moreover, it is preferable that the said imaging part is comprised with the optical sensor built-in liquid crystal display device, and the said input surface is a display surface of the said optical sensor built-in liquid crystal display device.

  According to said structure, since an image can be displayed on an input surface, the convenience for a user can be improved by displaying the information etc. for assisting a user's input. Further, according to the above configuration, since the image display and the finger detection on the input surface are performed by one liquid crystal display device with a built-in optical sensor, the input device can be downsized.

  Here, as described in Patent Documents 1 and 2, in the conventional input device, an input operation is performed by placing a finger on the position of the button. For this reason, even if the display unit is located close to the direction of travel of the car, there is no change in the need to look at the display unit. There was a problem of being.

  In addition, as a conventional technique for performing an input without looking at a display or the like, a gesture input can be mentioned. However, when performing a gesture input, the user needs to learn a gesture set in advance in the input device. For this reason, gesture input is easy to apply to gestures that are easy to remember intuitively, such as enlargement, reduction, scrolling, etc. of the display screen. It is difficult to apply to functions that are difficult to express with intuitive gestures, such as when a route guide is executed.

  Therefore, there is a demand for an input device that can perform various inputs with a simple operation without looking at a display or the like.

  In order to solve the above-described problems, an input device according to the present invention is an input device that detects contacts made simultaneously to a plurality of locations on an input surface that accepts an input operation, and generates an input signal according to the detection result. A type discriminating unit for discriminating each type of the plurality of fingers in contact with the input surface, an input detecting unit for detecting that at least one finger has performed a predetermined input operation, and the type discriminating unit. An input signal generating means for specifying the type of the finger detected by the input detecting means as the finger performing the input operation based on the determination result of the input and generating an input signal according to the specified finger type. It is characterized by being.

  In order to solve the above problems, the input device control method of the present invention detects simultaneous contact with a plurality of locations on the input surface that accepts an input operation, and generates an input signal according to the detection result. A method for controlling an input device, a type determining step for determining each type of a plurality of fingers in contact with the input surface, and an input for detecting that at least one finger has performed a predetermined input operation Based on the determination result of the detection step and the type determination step, the type of the finger detected as the finger that performed the input operation in the input detection step is specified, and an input signal corresponding to the specified type of finger is generated And an input signal generation step.

  According to the above configuration, each type of the plurality of fingers in contact with the input surface is determined, and it is detected that at least one finger has performed a predetermined input operation. Then, an input signal corresponding to the type of finger detected to have performed the input operation is generated.

  That is, according to the above configuration, an input signal corresponding to the type of finger is generated regardless of the position where the input surface is touched. Therefore, in the input device, various inputs can be performed by simple operations without looking at a display or the like.

  The predetermined input operation is not particularly limited as long as the input device can detect the input operation. For example, it may be detected as the predetermined input operation that the finger contact state on the input surface has changed to contact-non-contact, or the finger contact state on the input surface has changed to contact-non-contact-contact. May be detected as the predetermined input operation. Further, for example, a change in strength with which the finger is pressed against the input surface may be detected as the predetermined input operation, or a predetermined locus drawn by the finger on the input surface is detected as the predetermined input operation. May be.

  The input device may be realized by a computer. In this case, a control program for causing the input device to be realized by the computer by causing the computer to operate as each means of the input device, and recording the program. Such computer-readable recording media also fall within the scope of the present invention.

  As described above, the input device according to the present invention detects that the input surface is in contact with the input detection surface by the contact detection unit that detects that each of a plurality of fingers is in contact with the input surface. Among these fingers, there is provided an input detecting means for generating an input signal by detecting a state change from contact to non-contact of at least one finger.

  In addition, as described above, the input device control method according to the present invention includes the contact detection step for detecting that each of a plurality of fingers is in contact with the input surface, and the input surface in the contact detection step. And an input detection step of generating an input signal by detecting a change in the state of at least one finger from contact to non-contact among the fingers detected to be in contact with.

  Therefore, after touching the input surface with a plurality of fingers, the input operation can be performed by moving the finger away from the input surface. With this, it is possible to perform various input operations.

  In addition, as described above, the input device according to the present invention includes a type determination unit that determines the type of each of a plurality of fingers that are in contact with the input surface, and that at least one finger has performed a predetermined input operation. Based on the detection result of the input detection means to be detected and the type determination means, the type of the finger detected by the input detection means as the finger that performed the input operation is specified, and the input signal corresponding to the specified type of finger It is the structure provided with the input signal generation means which produces | generates.

  As described above, the control method for the input device according to the present invention includes a type determination step for determining the type of each of a plurality of fingers in contact with the input surface, and at least one finger performs a predetermined input operation. Based on the determination result of the input detection step and the type determination step, the finger type detected as the finger that performed the input operation in the input detection step is specified, and the specified finger type And an input signal generation step for generating a corresponding input signal.

  Therefore, an input signal corresponding to the type of the finger is generated regardless of the position touched on the input surface, and it is possible to perform various inputs with a simple operation without looking at the display or the like. .

[Summary of the Invention]
An embodiment of the present invention will be described below with reference to FIGS. Here, an outline of the input device according to the present embodiment will be described with reference to FIGS. 2 and 3 are diagrams illustrating an example of the operation of the input device 1 according to an embodiment of the present invention. As shown in FIG. 1A, the input device 1 includes a display panel 2, and button images a to e are displayed on the display panel 2.

  Here, when the user places his / her hand on the display panel 2, as shown in FIG. 4B, each of the button images a to e moves to the position of each fingertip of the user. In the illustrated example, button images a to e are displayed at positions corresponding to the user's little finger, ring finger, middle finger, index finger, and thumb, respectively.

  In this state, the input device 1 detects the movement of the user's finger and accepts an input operation corresponding to each of the button images a to e. For example, when the user performs an input operation with the index finger in the state of FIG. 5B, the input device 1 detects the movement of the index finger and inputs to the button image d as shown in FIG. Judge that there was an operation.

  As described above, the input device 1 receives an input operation at a position where the user touches the hand. Therefore, as shown in FIG. 3 (a), even when the user touches the display panel 2 below, the buttons are located at positions corresponding to the user's little finger, ring finger, middle finger, index finger, and thumb. Images a to e are displayed.

  Further, as shown in FIG. 3B, even if the user's hand is small and the hand touching the display panel 2 is the right hand, the user's little finger, ring finger, middle finger, index finger, and thumb Button images a to e are displayed at positions corresponding to the respective buttons. Then, the input device 1 detects the movement of the user's finger and accepts an input operation corresponding to each of the button images a to e.

  In other words, the conventional input device accepts input by a button or the like provided at a predetermined position, whereas in the input device 1, the position touched by the user is a button and accepts input. Therefore, when the input device 1 is used, an input operation can be easily performed by an adult or a child, a right-handed person or a left-handed person, and without looking at the display panel 2.

[Application example]
The input device 1 can be applied as an input device for various electronic devices. Here, as an example, an example in which the input device 1 is applied as an input device for an in-vehicle device such as an audio or an air conditioner will be described with reference to FIG.

  FIG. 4 is a diagram illustrating an example of an appearance when the input device 1 is applied as an input device of an in-vehicle device. The installation position of the input device 1 is not particularly limited, but it is preferable to install the input apparatus 1 at a position where the user can easily see the display panel 2. Moreover, since the input device 1 can perform an input operation with either the right hand or the left hand, the input device 1 is preferably installed at a position where the hand can be reached from the passenger seat or the driver seat.

  Specifically, as shown in FIG. 5A, the input device 1 is preferably installed on a center console or an instrument panel. Thereby, the display panel 2 can be easily seen from the passenger seat and the driver seat, and from the rear seat, and the vehicle-mounted device can be operated from either the passenger seat or the driver seat.

  FIG. 2B is an enlarged view of the input device 1 shown in FIG. In the illustrated example, the surface of the display panel 2 is formed in a dome shape. The surface of the display panel 2, that is, the input surface of the input device 1 may have any shape as long as the user has enough space to place five fingers. For example, similarly to a general touch panel, the input surface may be a rectangular plane. In this case, a display panel that is widely distributed can be used, which is advantageous in terms of cost. Further, when the input surface is dome-shaped as in the illustrated example, there is an advantage that it fits with the shape of the hand.

  Further, as shown in the figure, button images a to f are displayed on the display panel 2 along the outer periphery thereof. Since the button images a to f move to positions corresponding to the fingertips of the hand when the user touches the display panel 2, the initial display positions of the button images a to f are any positions. Alternatively, the button images a to f may be displayed only when the hand is touched.

  However, when the button images a to f are displayed in a state before the hand is touched as in the illustrated example, the user confirms what functions can be operated and then enters the input device 1. The input operation can be performed. Therefore, it is preferable to display the button images a to f before touching them.

  Moreover, since the input device 1 accepts an input operation on the display panel 2, the display panel 2 can also be used as a display output device of a peripheral device when no input operation is performed. For example, in the illustrated example, a map image output by the car navigation system is displayed on the display panel 2. Therefore, it is preferable that the initial display positions of the button images a to f are positions that do not hinder the display of the display panel 2.

  It is not necessary to move all the button images displayed on the display panel 2 to a position corresponding to the user's finger. For example, among the button images a to f, only the button image corresponding to the currently operable function may be moved to the position corresponding to the user's finger. In this case, the user can grasp the functions that can be operated with the input device 1 by looking at the button image displayed on the display panel 2, and can currently operate by touching the display panel 2. Capability can be grasped.

[Hardware configuration]
Next, the hardware configuration of the input device 1 will be described with reference to FIG. FIG. 5 is a block diagram illustrating an example of a hardware configuration of the input device 1. As illustrated, the input device 1 includes a display panel 2, an image acquisition unit (imaging unit) 3, a backlight 4, a CPU (Central Processing Unit) 5, a main storage unit 6, and an auxiliary storage unit 7. The peripheral device P is communicably connected.

  The display panel 2 and the backlight 4 are for displaying images. That is, the display panel 2 includes a color filter and a liquid crystal layer that blocks the light emitted from the backlight 4 and adjusts the amount of light reaching the color filter. An arbitrary image is displayed on the display panel 2 by driving the backlight 4 and the liquid crystal layer under the control of the CPU 5. Further, the surface of the display panel 2 that is exposed to the outside of the input device 1 also functions as an input surface that accepts user input operations.

  The image acquisition unit 3 acquires an object on the display panel 2 as an image, and based on the image acquired by the image acquisition unit 3, it is determined whether or not the finger touches the display panel 2. Note that the image acquisition unit 3 is incorporated in the display panel 2, and generates an image by receiving light reflected by an object on the display panel 2 from light emitted from the backlight 4. A known imaging device such as a CCD (charge-coupled device) can be applied to the image acquisition unit 3.

  That is, the display panel 2, the image acquisition unit 3, and the backlight 4 constitute a so-called photosensor built-in liquid crystal display device. In the input device 1, the display panel 2, the image acquisition unit 3, and the backlight 4 detect a user input operation on the display panel 2 and display an image.

  Here, a more detailed configuration of the display panel 2 and the image acquisition unit 3 will be described with reference to FIG. FIG. 6 is a diagram schematically illustrating the configuration and functions of the display panel 2 and the image acquisition unit 3.

  As illustrated, the display panel 2 includes a panel unit 2a, and a gate driver 2b and a source driver 2c provided around the panel unit 2a. An optical sensor driving circuit 3b is provided around the panel portion 2a. In the panel unit 2a, pixels composed of RGB sub-pixels are arranged, and a photosensor circuit 3a is provided for each pixel.

  The gate driver 2b and the source driver 2c apply a liquid crystal layer (not shown) disposed behind each pixel constituting the panel unit 2a (inside the input device 1) according to a display signal input to the display panel 2. Driving is performed to adjust the amount of light incident on the RGB sub-pixels from the backlight 4 (not shown) disposed further inside the input device 1 of the liquid crystal layer. Thereby, the image according to the said display signal is displayed on the panel part 2a.

  On the other hand, the image acquisition unit 3 includes an optical sensor circuit 3a and an optical sensor driving circuit 3b that drives the optical sensor circuit 3a. The optical sensor circuit 3a is driven by the optical sensor driving circuit 3b, whereby an image on the panel unit 2a is read, and the read image is output as a read signal (image data).

  In this embodiment, an example in which an image is displayed by a combination of the display panel 2 and the backlight 4 will be described. However, the input device 1 can display an image and detect an input operation. What is necessary is just and it is not limited to this example. For example, instead of the display panel 2 and the backlight 4, an image may be displayed using an organic EL (Electro Luminescence) display device.

  The main storage unit 6 is a ROM (Read Only Memory) that stores data necessary for the operation of the input device 1, such as program data of a basic input output system (BIOS) or program data of an operating system (OS). ) And a RAM (Random Access Memory) that functions as a working memory when information processing by the CPU 5 is performed.

  The auxiliary storage unit 7 stores various data used by the input device 1 such as program codes (execution format program, intermediate code program, source program) of a control program loaded into the RAM and executed by the CPU 5. Yes. The auxiliary storage unit 7 may be anything as long as the CPU 5 can write and read data. For example, tape systems such as magnetic tape and cassette tape, disk systems including magnetic disks such as floppy (registered trademark) disks / hard disks and optical disks such as CD-ROM / MO / MD / DVD / CD-R, IC cards (memory A card system such as an optical card) or a semiconductor memory system such as a mask ROM / EPROM / EEPROM / flash ROM can be used as the auxiliary storage unit 7.

  As described above, the input device 1 includes the CPU 5 that executes the instructions of the control program that implements each function of the input device 1, the ROM that stores the program, the RAM that expands the program, the auxiliary that stores the program and various data. By providing the storage unit 7, for example, functions as shown in FIGS. 2A to 2C are realized.

  In the input device 1, it is only necessary that the CPU 5 can use necessary data, and the data does not necessarily have to be stored in a storage medium built in or externally attached to the input device 1. For example, the input device 1 may be configured to be connectable to a communication network, and necessary data may be supplied to the CPU 5 via the communication network.

  The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Also, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used.

  In this embodiment, an example in which the function of the input device 1 is realized by causing the CPU 5 to execute a program will be described. However, the function of the input device 1 can also be realized using hardware such as a circuit or a chip. It is.

[Images Acquired by Image Acquisition Unit 3]
Here, an image acquired by the image acquisition unit 3 will be described with reference to FIG. FIG. 7 is a diagram illustrating an example of an image acquired by the image acquisition unit 3. More specifically, FIG. 5A shows an example of an image acquired by the image acquisition unit 3 while the user places his / her hand on the display panel 2, and FIG. 5B shows only the index finger. 2 shows an example of an image acquired by the image acquisition unit 3 in a state separated from 2, and FIG. 8C shows an example of an image acquired by the image acquisition unit 3 with the index finger again in contact with the display panel 2. Is shown.

  The image acquisition unit 3 captures an object in contact with the display panel 2 as an image. Therefore, when the entire hand is in contact with the display panel 2, five fingers and the palm are captured as an image, as shown in FIG. Generally, since the external shape of a person's hand is common, it can be determined from this image whether the hand in contact with the display panel 2 is the right hand or the left hand, and the finger in the image The type (thumb, forefinger, middle finger, ring finger, little finger) can also be determined.

  Here, when the index finger is separated from the display panel 2, the amount of light emitted from the backlight 4 and reflected by the index finger and incident on the image acquisition unit 3 is significantly reduced. Therefore, when the index finger is separated from the display panel 2, the four fingers and the palm excluding the index finger can be recognized as an image as shown in FIG. When the index finger comes into contact with the display panel 2 again, as shown in FIG. 5C, the five fingers and the palm are recognized as an image as in FIG.

  The input device 1 detects that the image acquired by the image acquisition unit 3 has changed from the state of FIG. 7A to the state of FIG. 7C, and determines that an input operation has been performed with the index finger. In the input device 1, the same determination is made when the finger whose contact state with the display panel 2 has changed is other than the index finger. That is, the input device 1 determines that an input operation has been performed with the finger when the finger touching the display panel 2 once leaves the display panel 2 and then touches the display panel 2 again.

[Main part configuration]
Next, a more detailed configuration of the input device 1 will be described with reference to FIG. FIG. 1 is a block diagram illustrating a main configuration of the control unit 11 provided in the input device 1. As illustrated, the control unit 11 includes a peripheral device control unit (device control unit) 12, an input detection unit (input detection unit, input signal generation unit) 13, an image analysis unit (contact detection unit, type determination unit) 14, A button coordinate changing unit 15, a display control unit 16, and a button display unit (display unit) 17 are provided. Further, as illustrated, the input device 1 includes a storage unit 21, and a command table 22 and a button coordinate table 23 are stored in the storage unit 21.

  In addition, each structure with which the control part 11 is provided has shown the function implement | achieved when CPU5 runs a program. The data stored in the storage unit 21 is basically stored in the auxiliary storage unit 7 and is read out to the main storage unit 6 when the CPU 5 uses these data.

  The peripheral device control unit 12 transmits a command to the peripheral device P to be operated by the input device 1 and controls its operation. As the peripheral device P, any device operated by the input device 1 can be applied. Here, since it is assumed that the input device 1 is mounted on a car, an example in which there are three peripheral devices P, that is, an audio device, an air conditioner (air conditioner), and a car navigation system (car navigation system) will be described.

  The input detection unit 13 detects that the contact state of each finger that has detected that the image analysis unit 14 is in contact with the display panel 2 has changed, and outputs an input signal corresponding to the finger that has detected the change in the contact state. Generate. Specifically, when the finger detected to be in contact with the display panel 2 is in a non-contact state, the input detection unit 13 generates operation finger data indicating the finger. The operation finger data is sent to the peripheral device control unit 12, and the peripheral device control unit 12 generates a command corresponding to the finger type indicated by the operation finger data and transmits the command to the peripheral device P.

  The image analysis unit 14 receives the image acquired by the image acquisition unit 3 and analyzes the received image. Specifically, the image analysis unit 14 determines whether the image is a hand image. When the image is a hand image, the image analysis unit 14 detects whether the hand is the right hand or the left hand and obtains the position of the fingertip of the hand.

  For example, by storing a standard hand image in advance and comparing the image with a standard hand image, the image acquisition unit 3 determines whether the image is a hand image. It is possible to determine whether the hand is the right hand or the left hand and where the fingertip is.

  In addition, when the image is a hand image, the image acquisition unit 3 may determine whether the hand is the right hand or the left hand based on the fingertip interval. This will be described with reference to FIG. FIG. 8 is a diagram illustrating a method for determining whether the left hand or the right hand is based on the fingertip interval.

  First, when a hand image is acquired, the position of the fingertip can be specified by pattern matching or the like. When the fingertip position is specified, the interval between the specified fingertip positions can be calculated.

  Here, as shown in the drawing, the distance b from the fingertip of the thumb to the fingertip of the index finger is generally longer in the human hand than the distance a from the fingertip of the little finger to the fingertip of the ring finger. Therefore, when the positions of the five fingertips are identified, by comparing the distance between the two fingertip positions located at both ends, it is possible to determine which end finger is the thumb and which end finger is the little finger. It is possible to determine whether it is a right hand or a left hand. For example, in the illustrated example, since the interval b between the right end fingertip positions is longer than the interval a between the left end fingertip positions, it is possible to determine that the right side is a thumb, thereby determining that the image is a left hand image. be able to.

  In addition, since the size of the hand, the thickness of the finger, the length, etc. are each person, when the image acquired by the image acquisition unit 3 and the standard hand image stored in advance are directly compared In some cases, the image analysis unit 14 may erroneously recognize.

  Therefore, it is preferable that the image analysis unit 14 standardizes the image acquired by the image acquisition unit 3. The standardization refers to correction performed so that the size of the hand, the thickness of the finger, the length, and the like are within a predetermined range. By comparing the corrected image with a standard hand image, the image analysis unit 14 can determine whether the image acquired by the image acquisition unit 3 is a hand image. When the image acquired by the image acquisition unit 3 is a hand image, the image analysis unit 14 determines whether the image is a right-hand image or a left-hand image, and where is the fingertip position. Can be reliably determined.

  Then, the image analysis unit 14 generates, for each finger in contact with the display panel 2, data indicating the position on the display panel 2, the type of the finger, and the left or right hand finger, The data is sent to the input detection unit 13.

  The button coordinate changing unit 15 updates the coordinates indicating the display position of the button image stored in the button coordinate table 23 of the storage unit 21 according to the position of the fingertip detected by the image analysis unit 14. As a result, the button image moves following the movement of the finger. The coordinates updated by the button coordinate changing unit 15 are also used for detecting an input operation by the input detecting unit 13. In the following, the coordinates updated by the button coordinate changing unit 15 are referred to as update coordinates.

  The display control unit 16 controls the operations of the display panel 2 and the backlight 4 to display an arbitrary image on the display panel 2. Further, the display control unit 16 receives the control screen data from the button display unit 17 and causes the display panel 2 to display the control screen data.

  The button display unit 17 displays a button image indicating a function assigned to each finger at a position corresponding to the finger. Specifically, the button display unit 17 reads the button image stored in the command table 22 of the storage unit 21 and reads the initial coordinates or the updated coordinates stored in the button coordinate table 23 of the storage unit 21. Then, the button display unit 17 generates control screen data in which the read button image is arranged at the position of the read initial coordinate or update coordinate, and outputs the control screen data to the display control unit 16 for display.

[Command table 22]
Here, the command table 22 will be described. The command table 22 is a table in which a finger and a command corresponding to the finger are associated with each other. Therefore, the peripheral device control unit 12 can specify a command corresponding to the finger specified by the input detection unit 13 by referring to the command table 22. Note that the command is a command for causing the peripheral device P to execute a predetermined operation, and is different for each peripheral device P. Therefore, the command table 22 is prepared for each peripheral device P.

  Further, each command and a button image corresponding to the command are stored in the command table 22 in association with each other. The button image is an image indicating an operation performed by the peripheral device P in accordance with each command. By displaying the button image, the user is assigned what command to each finger (what operation is performed). Can be recognized).

  The button image may be any button image that can be recognized by the user as to what operation is performed by the operation of each finger. In addition to still images, for example, moving images, characters, symbols, etc. It can also be a button image.

  For example, the command table 22 may be configured as shown in FIG. FIG. 9 is a diagram illustrating an example of the command table 22. In the command table 22 shown in FIG. 9, each of the left and right fingers is associated with commands and button images corresponding to four modes of main, audio, air conditioner, and car navigation.

  The main mode is a mode for selecting which peripheral device P of the audio, the air conditioner, and the car navigation is to be operated. Therefore, in the main mode, there are an audio command for shifting to an audio mode for operating audio, an air conditioner command for shifting to an air conditioner mode for operating an air conditioner, and a car navigation command for shifting to a car navigation mode for operating a car navigation system. It is stored in association with the finger.

  Specifically, the audio command is associated with the driver seat L1 and the passenger seat R1, the air-conditioner command is associated with the driver seat L2 and the passenger seat R2, and the car navigation command is It is associated with the seat L3 and the passenger seat R3.

  L1 to L5 indicate the left thumb, index finger, middle finger, ring finger, and little finger, respectively. Similarly, R1 to R5 indicate the right thumb, index finger, middle finger, ring finger, and little finger, respectively. That is, here, since it is assumed that the input device 1 is disposed on the center console of the right-hand drive automobile, when the input device 1 is operated from the passenger seat, it is operated with the right hand, When the input device 1 is operated from the right side, it is operated with the right hand. Therefore, the command table 22 in the illustrated example defines two finger assignments for the driver seat and the passenger seat.

  Similarly, in the audio mode, a stop command for stopping audio playback is displayed for L1 and R1, a playback command for starting audio playback is played for L2 and R2, and the next song is played for L3 and R3. The one-track advance command to be performed is associated with a volume UP command for increasing the audio reproduction volume to L4 and R4, and a volume DOWN command for decreasing the audio reproduction volume with L5 and R5.

  Further, in the air conditioner mode, L1 and R1 have an outside air command for executing the intake of outside air, L2 and R2 have a circulation command for circulating the air inside the vehicle, and L3 and R3 have a temperature UP for raising the set temperature. The command is associated with a temperature DOWN command for lowering the set temperature to L4 and R4.

  Further, in the car navigation mode, L1 and R1 include a home command for starting a route guide to the home, L2 and R2 include a wide-area command for switching the displayed map to a smaller scale map, and L3 and R3 include A detailed command for switching the map to be displayed to a map with a larger scale is associated.

  In the present embodiment, since it is assumed that the input device 1 is operated from both sides of the driver seat and the passenger seat, even when operating from the driver seat with the left hand, even when operating with the right hand from the passenger seat, It is preferable to be able to operate with the same feeling. Therefore, in the present embodiment, the same command is associated with each of L1 to L5 and R1 to R5. However, another command may be associated with L1 to L5 and R1 to R5. As a result, it is possible to perform various inputs not only according to the type of finger but also according to whether the right hand or the left hand.

  Further, as shown in the figure, a button image is associated with each command. This button image is read by the button display unit 17 and displayed on the display panel 2 by the display control unit 16. Thereby, the user can easily recognize what function is executed when the user performs an operation with which finger.

  Any command may be assigned to each finger, but it is considered that there are users who are not familiar with the input operation with the little finger or ring finger. It is preferable to assign it to the middle finger.

[Button coordinate table 23]
Next, the button coordinate table 23 will be described. The button coordinate table 23 is a table indicating coordinates for displaying a button image. The button coordinate table 23 stores initial coordinates that are coordinates for displaying a button image in a state before the hand is placed on the display panel 2. Until the hand is placed on the display panel 2, the button image is displayed at the position indicated by the initial coordinates in the button coordinate table 23.

  The button coordinate table 23 stores updated coordinates indicating the coordinates of each finger after the hand is placed on the display panel 2. As described above, the updated coordinates are coordinates updated by the button coordinate changing unit 15. After the hand is placed on the display panel 2, the button image is displayed at the position indicated by the updated coordinates in the button coordinate table 23.

  The button coordinate table 23 may be configured as shown in FIG. 10, for example. FIG. 10 is a diagram illustrating an example of the button coordinate table 23. As shown in the figure, the button coordinate table 23 stores initial coordinates and update coordinates corresponding to each of the driver seats, that is, the fingers L1 to L5, and each of the passenger seats, that is, the fingers R1 to R5. The initial coordinates and update coordinates corresponding to are stored.

  The initial coordinates are set in advance as the display position of the button image before the user touches the display panel 2, and the value thereof is fixed. On the other hand, the update coordinate indicates a position where the button image is displayed when the user touches the display panel 2 and is updated as needed according to the position of the finger on the display panel 2.

  Thus, in the input device 1, the display position of the button image on the display panel 2 is represented using coordinates. This will be described with reference to FIG. FIG. 11 is a diagram showing coordinates set on the display panel 2 and display positions of button images displayed on the display panel 2. In the illustrated example, a vertical axis from 0 to 200 and a horizontal axis from 0 to 200 are set, and the position on the display panel 2 is thereby changed from (0, 0) to (200, 200). Can be represented by coordinates.

  For example, when the button coordinate table 23 of FIG. 10 is used, the initial coordinates of L1 to L5 are (180, 120), (160, 170), (100, 200), (40, 170), and (10, 120). Accordingly, as shown in FIG. 11, the button image is displayed at the coordinates of (180, 120), (160, 170), (100, 200), (40, 170), and (10, 120). Become.

  As shown in FIG. 11, when a hand is placed on the display panel 2, the coordinates of the position of the fingertip are stored in the button coordinate table 23 as update coordinates. In the example of FIG. 11, the position coordinates of the fingertip are (160, 90), (130, 130), (110, 150), (70, 140), and (40, 110). Therefore, as shown in FIG. 10, the updated coordinates of L1 to L5 are also (160, 90), (130, 130), (110, 150), (70, 140), and (40, 110).

  Thereby, as shown in FIG. 11, the display position of the button image is the updated coordinates (160, 90), (130, 130), (110, 150), (70, 140), and (40, 110). Will change.

  Note that it is not necessary to match the position of the fingertip with the display position of the button image, and the button image may be displayed so that the user can recognize which command (function) corresponds to the operation with which finger. . However, when the button image is displayed at the exact same position as the fingertip coordinate, the button image is hidden by the finger, and in this application example, the button image is displayed at a position shifted from the fingertip coordinate. The amount of shift between the fingertip coordinates and the button image display position may be fixedly determined according to the size of the button image, but for each finger type, the approximate movement direction and the movable range Therefore, it is preferable to change the shifting direction and the shifting amount according to the type of the finger.

[Process flow from startup to control mode]
Next, the flow of processing until the input operation on the peripheral device P can be performed after the input device 1 is activated will be described with reference to FIG. Here, a state in which an input operation on the peripheral device P can be performed is referred to as a control mode. FIG. 12 is a flowchart illustrating an example of processing from activation of the input device 1 to control mode.

  When the car engine is started, a current is supplied to the input device 1 and the input device 1 is activated. In the input device 1 in the activated state, first, control data is read (S1). The control data is data such as a program necessary for the operation of the input device 1. By reading the control data, the function of each block shown in FIG. 1 becomes executable, and the input device 1 shifts to the main control mode (S2).

  In the main control mode, selection of which peripheral device P to operate is performed. That is, when an input for operating the audio device is performed in the main control mode, the input device 1 shifts to the audio control mode (S3). On the other hand, when an input for operating the air conditioner is performed, the input device 1 shifts to the air conditioner control mode (S4). If an input for operating the air conditioner is made, the input device 1 shifts to the car navigation control mode (S5).

  In the main control mode, audio control mode, air conditioner control mode, and car navigation control mode, only the data in the command table 22 referred to by the input device 1 changes, and the basic operation of the input device 1 is the same. That is, the main control mode, the audio control mode, the air conditioner control mode, and the car navigation control mode are all control modes in which an input operation to the peripheral device P can be performed.

[Process flow in control mode]
Next, the flow of processing in the control mode will be described with reference to FIG. FIG. 13 is a flowchart illustrating an example of processing in the control mode.

  When the input device 1 shifts to the control mode, the input detection unit 13 sets a flag to 0 (S11). This flag is used to determine the number of fingers that are in contact with the display panel 2. When it is determined that five fingers are in contact with the display panel 2, the flag is set to 1. Is set.

  When the input device 1 shifts to the control mode, the button display unit 17 reads initial coordinates from the button coordinate table 23. That is, when the input device 1 shifts to the control mode, initial coordinates are set in the button coordinate table 23 (S12).

  The initial coordinates are determined according to the hand image last input to the input device 1. That is, the button display unit 17 stores, in an arbitrary storage unit such as the auxiliary storage unit 7, whether the last input coordinate is the left-hand or right-hand coordinate, and the same hand on the same side. Read the initial coordinates of.

  Subsequently, the button display unit 17 reads out a button image corresponding to each finger from the command table 22. The button image to be read out here is determined according to which peripheral device P is in the control mode in which the input device 1 operates. For example, when the input device 1 is in the audio control mode, the button display unit 17 reads out each button image corresponding to the audio mode in FIG. Then, the button display unit 17 instructs the display control unit 16 to display the read button image at the initial coordinate position (S13).

  When the flag is set to 0, the input detection unit 13 sends an instruction to the image analysis unit 14 to start image analysis. Upon receiving the image analysis start instruction, the image analysis unit 14 instructs the image acquisition unit 3 to capture an image (S14). Then, the image acquisition unit 3 that has received the image capture instruction captures the image of the object on the display panel 2 and sends the captured image to the image analysis unit 14.

  Next, when receiving an image from the image acquisition unit 3, the image analysis unit 14 determines whether or not the received image is a hand image (S15). The image analysis unit 14 can determine whether or not the received image is a hand image, for example, by comparing a standard hand image stored in advance with the image.

  Here, when the image analysis unit 14 cannot determine that the image received from the image acquisition unit 3 is a hand image (NO in S15), the image analysis unit 14 transmits the fact to the input detection unit 13 and receives the transmission. The input detection unit 13 returns to the process of S11. That is, the processes from S11 to S15 are repeated until the hand is brought into contact with the display panel 2. Even when the hand that has been touched is released from the display panel 2, since the hand image is not detected in S15, the process returns to S11. Therefore, when the touched hand is released from the display panel 2, the display position of the button image is returned to the initial coordinate position by the processing from S11 to S13.

  On the other hand, when it is determined that the image received from the image acquisition unit 3 is a hand image (YES in S15), it is determined whether the image is a right hand image or a left hand image (S16). . Whether the image is a right hand image or a left hand image can also be determined by comparing a standard hand image stored in advance with the received image.

  Subsequently, the image analysis unit 14 detects the coordinates of the fingertip from the received image (S17). Specifically, the image analysis unit 14 specifies how many fingers of the image are based on the determination result of whether the image in S16 is a right-hand image or a left-hand image. Then, the image analysis unit 14 acquires the position of the identified tip portion of each finger on the display panel 2 as coordinates.

  That is, here, the image analysis unit 14 acquires the coordinates on the display panel 2 of the fingertips of the thumb, index finger, middle finger, ring finger, and little finger. Then, the image analysis unit 14 sends the obtained fingertip coordinates of each finger to the input detection unit 13. Here, the fingertip coordinates of each finger acquired by the image analysis unit 14 are referred to as detection coordinates.

  The input detection unit 13 that has received the detected coordinates checks whether or not the received detected coordinates include five fingertip coordinates (S18). That is, the input detection unit 13 confirms whether or not the detected coordinates include the coordinates of the fingertips of the thumb, index finger, middle finger, ring finger, and little finger.

  Here, when it is not confirmed that five fingertip coordinates are included (NO in S18), the input detection unit 13 checks whether or not the flag is 1 (S26). Here, since the flag is set to 0 in S11, the input detection unit 13 determines that the flag is not 1 (NO in S26), and returns to the process of S14.

  On the other hand, when it is confirmed that five fingertip coordinates are included (YES in S18), the input detection unit 13 checks whether or not the timer is started (S19). If the timer has started (YES in S19), this is stopped (S20) and the process proceeds to S21. If the timer has not started (NO in S19), the process of S21 is continued. Proceed to

  This timer is used to determine whether or not to determine that the change in the contact state is an input operation when the input detection unit 13 detects that the contact state of the finger has changed. Although not shown in FIGS. 1 and 5, the timer is provided in the input device 1 so that the input detection unit 13 can refer to it. Here, since the timer has not been started, the process proceeds to S21 after S19.

  In S21, the input detection unit 13 checks whether or not the flag is 1. Here, since the flag is set to 0 in S11, the input detection unit 13 determines that the flag is not 1 (NO in S21), and proceeds to the process of S22.

  In S22, the updated coordinates are set in the button coordinate table 23. Specifically, the input detection unit 13 instructs the image analysis unit 14 to transmit the detected coordinates to the button coordinate change unit 15. Next, upon receiving the instruction, the image analysis unit 14 sends the detected coordinates to the button coordinate change unit 15. Then, the button coordinate changing unit 15 updates the updated coordinates in the button coordinate table 23 using the detected coordinates received from the image analyzing unit 14, whereby the updated coordinates are set in the button coordinate table 23.

  Here, when the updated coordinates of the button coordinate table 23 are updated, the button display unit 17 reads the updated updated coordinates, and the control screen data in which the button image is arranged at the position of the read initial coordinates or updated coordinates. And the generated control screen data is output to the display control unit 16 and displayed on the display panel 2 (S23).

  Then, the input detection unit 13 sets the flag to 1. That is, five fingertip coordinates are detected, and the flag is set to 1 when an instruction to display control screen data in which a button image is arranged at a position corresponding to each detected fingertip coordinate is issued. And the input detection part 13 which set the flag to 1 returns to the process of S14.

  The flag only needs to be set to 1 when five fingers are in contact with the display panel 2, and the timing for setting the flag to 1 is not limited to the example illustrated. For example, the flag may be set to 1 when it is determined in S18 that five fingers are in contact.

  When the input detection unit 13 detects five fingertip coordinates with the flag set to 1, the process proceeds from S15 to S21 to S25. In S25, the input detection unit 13 confirms whether or not the input finger that has performed the input operation is specified. Since the input finger is specified in S30 described later, the input finger is not specified at this stage. Therefore, the input detection unit 13 determines that no input finger is specified (NO in S25), and returns to the process of S22.

  In S22 and S23, as described above, the updated coordinates are set in the button coordinate table 23, and the button image is displayed based on the updated coordinates. That is, in the input device 1, when the user moves the finger position with five fingers in contact with the display panel 2, the button image display position also moves following the finger position. It is like that. Thereby, even when the user's finger moves during the input operation, the button image corresponding to each finger can always be displayed at an appropriate position. In this case, since the flag is already set to 1, the input detection unit 13 does not change the flag in S24 and returns to the process of S14.

  On the other hand, when the flag is set to 1 and the input detection unit 13 cannot detect the five fingertip coordinates, the process proceeds to S27 through S18 and S26. In S27, the input detection unit 13 confirms whether or not four fingertip coordinates are detected. The fact that the flag is set to 1 indicates that five fingertip coordinates have been detected. Therefore, in S27, it is confirmed whether or not the detected fingertip coordinates have changed from five to four. Will be.

  Here, if the detection of the four fingertip coordinates cannot be confirmed (NO in S27), the input detection unit 13 returns to the process of S14. On the other hand, when the detection of the four fingertip coordinates is confirmed (YES in S27), the input detection unit 13 starts a timer (S28). That is, the input detection unit 13 starts the timer when detecting four fingertip coordinates (when detecting a state where one finger is not in contact).

  Subsequently, the input detection unit 13 compares the detected four fingertip coordinates with the updated coordinates stored in the button coordinate table 23, and detects a finger for which no coordinate is obtained (S29). That is, in S29, it is detected which of the five fingers has been separated from the display panel 2.

  And the input detection part 13 specifies the finger | toe from which the coordinate was not obtained, ie, the finger | toe distant from the display panel 2, as the input finger which performed input operation (S30). Thereafter, the process returns to S14 again, and an image is captured.

  Here, when the captured image is a hand image and five fingertip images are detected, the process proceeds to S19 through S15 to S18. Here, since the timer is started in S28, the process proceeds to S20, and the input detection unit 13 stops the timer.

  Thereby, the time from when the non-contact state of one finger is detected (YES in S27) to when it is detected that the finger is again in contact (YES in S18) is determined by the timer. It is measured.

  Here, since the flag is set to 1 and the input finger is specified in S30, the process proceeds from S21 to S25. In S25, the input detection unit 13 determines that the input finger has been specified. Determination is made (YES in S25), and the process proceeds to S31.

  In S31, the input detection unit 13 confirms whether or not the time measured by the timer is within a predetermined time range set in advance. Here, when the time measured by the timer is outside the predetermined time range (NO in S31), the input detection unit 13 returns to the process of S22. On the other hand, when the time measured by the timer is within the predetermined time range (YES in S31), the input detection unit 13 proceeds to the process of S32.

  In S32, a command is output to the peripheral device P. Specifically, the input detection unit 13 sends input finger data indicating the specified input finger to the peripheral device control unit 12. Then, the peripheral device control unit 12 that has received the input finger data reads a command associated with the input finger indicated by the received input finger data from the command table 22 and transmits the command to the peripheral device P that executes the command. .

  In addition, when the peripheral device control unit 12 transmits a command, the peripheral device control unit 12 transmits the command to the input detection unit 13, and the input detection unit 13 that has received the notification returns the flag to 0 (S33) and returns to the processing of S14. .

  Although not shown in FIG. 13, when the input finger is specified, the display mode of the button image corresponding to the specified input finger is changed. Specifically, the input detection unit 13 also transmits the input finger data to the button display unit 17 in S32. And the button display part 17 produces | generates the control screen data which changed the display mode of the button image corresponding to the finger | toe specified by the said input finger data.

  As a result, since the display mode of the button image corresponding to the finger whose input operation has been detected by the input device 1 changes, the user can recognize at a glance whether or not his input operation has been correctly recognized by the input device 1. Can do.

  The button image display mode may be changed as long as the user can visually recognize the change. For example, the color, saturation, brightness, size, shape, or the like may be changed. The button image to be displayed may be changed to another button image. Further, as in the example of FIG. 2C, the color of the button image may be reversed. Furthermore, the selected command (or the operation executed by the command) is output by voice so that it is possible to cope with the case where it is difficult to operate while looking at the display panel 2 such as during driving of the car. You may make it do.

  As described above, in the input device 1, when one of the five fingers brought into contact with the display panel 2 is released from the display panel 2, the input finger is determined and the user releases the display panel 2. By bringing the finger into contact with the display panel 2 again, the command associated with the input finger is transmitted to the peripheral device P.

  Therefore, the user does not need to put his finger on the position of the button as in the past, and can operate the peripheral device P simply by touching an arbitrary position on the display panel 2. This is useful when it is preferable to operate the peripheral device P without looking at the display panel 2, for example, during driving of a car.

  Further, as described above, the input device 1 displays a command (function of the peripheral device P) that is executed when an input operation is performed with the finger at a position corresponding to each finger that is in contact with the display panel 2. Is done. Therefore, for example, when the vehicle is stopped or in the passenger seat, the user can reliably operate the peripheral device P while viewing the display on the liquid crystal panel.

  Furthermore, in the input device 1, based on the elapsed time from when the non-contact state of one finger measured by the timer is detected until it is detected that the finger is again in the contact state. The presence / absence of an input operation is determined. That is, in the input device 1, when the elapsed time is not within a predetermined time range, it is determined that an input operation has not been performed.

  Thereby, it can prevent that input operation which a user does not intend is performed. For example, it is possible to avoid the problem of erroneously recognizing a momentary lift of a finger that occurs when a hand in contact with the display panel 2 is moved as a button operation.

  In the example of FIG. 13, the input operation is not accepted unless five fingertip coordinates are detected in S18. That is, the user can perform an input operation by bringing the hand into contact with the display panel 2 so that five fingertip coordinates are detected.

  This is because the input device 1 displays button images at each fingertip coordinate (or each position shifted by a predetermined distance from each fingertip coordinate), and the maximum number of button images is set to five. Because it is. That is, when five fingertip coordinates are detected, the number of button images to be displayed is always less than or equal to the number of fingertip coordinates, so that each button image can be reliably displayed at a position corresponding to each fingertip coordinate. it can.

  Further, when the positions of the user's fingertips on the display panel 2 are close to each other, there is a possibility that the display position of the button image overlaps and the visibility of the user is lowered. Further, in this case, there is a possibility that an erroneous operation of the finger that has performed the input operation occurs and an input operation that is not intended by the user is performed.

  However, when five fingertip coordinates are detected, the position of each fingertip of the user on the display panel 2 is separated to the extent that the input device 1 can be identified. Although depending on the detection accuracy of the fingertip position, when the position of each fingertip can be specified by pattern matching or the like, the positions of the fingertips are sufficiently separated from each other, and thus the above-described problems are unlikely to occur.

  In order to surely solve the above problem, for example, even when five fingertip coordinates are detected in S18 of FIG. 13, the distance between the detected fingertip coordinates and the adjacent fingertip coordinates is predetermined. When fingertip coordinates that are less than or equal to the value are included, it may be determined that five fingertip coordinates have not been detected (NO in S18).

[Modified example of input operation]
In the above embodiment, an example in which an input finger is determined when the detected fingertip coordinates change from five to four, and a command is output when the detected fingertip coordinates return to five. However, the determination of the input finger is not limited to this example as long as it is performed based on the change in the state of contact / non-contact of the finger.

  For example, an input finger may be determined and a command may be output when the coordinates of the fingertip change from five to four and again five. Further, for example, when the coordinates of the fingertip are changed from four to three and again to four, the input finger may be determined and a command may be output.

  That is, the input device 1 detects that one finger has changed from a state in which two or more fingers are in contact with the display panel 2 to a non-contact state, and sets the detected finger to the input finger. It can be set as the structure judged. Further, the input device 1 detects that one finger has changed from a state in which two or more fingers are in contact with the display panel 2 to a non-contact state and then returned to the contact state, and the detection is performed. It is good also as a structure which determines the done finger as an input finger. And the timing which determines an input finger, and the timing which outputs a command may be the same, and may differ.

  Further, the command may be determined by simultaneously changing the contact state of a plurality of fingers. For example, when the index finger and the thumb are separated from the display panel 2 from the state in which five fingers are in contact with the display panel 2, and the index finger and the thumb are brought into contact with the display panel 2 again, the index finger and the thumb are combined. Alternatively, a previously defined command may be output. This makes it possible to perform more various input operations.

  Further, the input device 1 may output a command when it is detected that the finger has changed from the contact state to the non-contact state. However, in this case, when the user finishes the input operation and releases his / her hand from the display panel 2, there is a possibility that a command is generated by mistake. Therefore, it is preferable to determine the input finger when it is detected that the finger that has become in the non-contact state is in the contact state again as in the above embodiment.

  In the above-described embodiment, the example of generating the command associated with each finger by determining the type of the finger that performed the input operation has been described. However, the command is assigned regardless of the finger type. Also good.

  For example, when there are three types of commands to be input, the commands may be assigned to the third fingertip coordinates from the right among the detected fingertip coordinates. In this case, for example, when the user touches the display panel 2 with five fingers of the right hand, commands are assigned to the thumb, index finger, and middle finger, and the user displays the index finger, middle finger, and ring finger of the right hand. When the panel 2 is brought into contact, commands are assigned to the index finger, the middle finger, and the ring finger, respectively. Accordingly, the user can perform an input operation by various input methods.

  Further, a command may be generated according to a position where an input operation on the display panel 2 is performed. For example, when the user places his / her hand on the display panel 2, the position, orientation, size, etc. of the hand are detected, and a position corresponding to the detected position, orientation, size, etc. (for example, the fingertip of each finger) ) Display the button image. Then, when the user touches the displayed button image, a command previously associated with the button image may be generated. In this case, after placing a hand on the display panel 2, an input operation similar to that of a conventional touch panel becomes possible. That is, regardless of the finger touching the button image, a command associated with the button image is generated.

  This is suitable when the input device 1 is applied to an input device having a plurality of input keys such as a keyboard. That is, when the user places his / her hand on the display panel 2, a button image is displayed at a position that is easy for the user to operate according to the position and size of the placed hand, and an input operation on the displayed button image is accepted. Thus, the user can smoothly perform an input operation regardless of the size or shape of the hand.

  Further, in the above-described embodiment, an example in which an input operation is performed after the hand is placed on the display panel 2 has been described. In addition, an operation input similar to that of a conventional touch panel may be performed. .

  That is, when it is detected that the button image displayed at the initial coordinates is touched, a command associated with the button image may be transmitted. For example, the command defined in the button image may be output by performing a tile touch on the button images a to e in the state of FIG. As a result, the user can perform an input operation with the same operation method as a conventional touch panel, and smoothly perform an input operation with an operation method that is conventionally used while viewing the display panel 2 when the user is not driving. Will also be possible.

[Modification of input operation detection method]
In the above-described embodiment, an example in which contact / non-contact of the finger with respect to the display panel 2 is determined based on an image acquired by the image acquisition unit 3 has been described, but the input device 1 determines which finger has been operated. This is not limited to this example.

  For example, an infrared camera is provided behind an arbitrary display device such as a liquid crystal display panel, an EL display panel, a plasma display panel, and the like. You may make it judge non-contact. Further, for example, it is possible to determine contact / non-contact of the finger with respect to the display panel 2 using a sensor such as a pressure sensor or a temperature sensor.

[User authentication]
In the input device 1, user authentication may be performed using the image acquired by the image acquisition unit 3. For example, user authentication can be performed by collating a fingerprint or palm print of a user acquired in advance with an image acquired by the image acquisition unit 3. As a result, it is possible to set restrictions on the input operation of the peripheral device P for each user.

  For example, only the parents can perform the input operation of the car navigation function, and other functions can be operated by the child. Thereby, it can prevent that a child operates a car navigation function and a driving | operation of parents is prevented. In addition, when a child touches the display panel 2, a button image is displayed at a position that matches the child's hand, and input operations can be performed at the position of the hand, so even a child can input easily. The operation can be performed. Therefore, even if the user is a child, it is possible to play back favorite music by operating an audio device, for example.

  For example, when the input device 1 is applied as an input device of an electronic device such as a personal computer or an ATM (Automatic Teller Machine), the user is authenticated without performing a special operation such as inputting a password or reading a fingerprint. The input operation can be validated only when the authentication is successful.

  Furthermore, the input operation can be limited according to the size of the hand. For example, when the size of the hand is smaller than a predetermined size, it is possible to prevent the child from operating the electronic device by mistake by not accepting the input operation.

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims. That is, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

  According to the input device of the present invention, an input operation can be easily performed simply by touching the display panel. Therefore, the input device can be applied to any input device that performs an input operation by touching with a finger.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 illustrates an embodiment of the present invention and is a block diagram illustrating a configuration of a main part of an input device. It is a figure which shows an example of operation | movement of the said input device. It is a figure which shows an example different from the above of the operation | movement of the said input device. It is a figure which shows an example of the external appearance at the time of applying the said input device as an input device of vehicle equipment. It is a block diagram which shows an example of the hardware constitutions of the said input device. It is a figure which shows the outline of a structure and function of a display panel with which the said input device is provided, and an image acquisition part. It is a figure which shows an example of the image which the image acquisition part with which the said input device is provided. It is a figure explaining the method of judging whether it is a left hand or a right hand based on the space | interval of a fingertip. It is a figure which shows an example of the command table with which the said input device is provided. It is a figure which shows an example of the button coordinate table with which the said input device is provided. FIG. 4 is a diagram illustrating coordinates set on a display panel included in the input device and display positions of button images displayed on the display panel 2. It is a flowchart which shows an example of the process from starting of the said input device to control mode. It is a flowchart which shows an example of the process which the said input device performs in the said control mode.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Input device 2 Display panel 3 Image acquisition part (imaging part)
11 Control Unit 13 Input Detection Unit (Input Detection Unit, Input Signal Generation Unit)
14 Image analysis unit (contact detection means, type discrimination means)
15 Button coordinate change part 17 Button display part (display means)
22 Command table

Claims (4)

An input device that identifies and detects contact with each of a plurality of locations on an input surface that receives an input operation, and generates an input signal according to the detection result,
Contact detecting means for detecting that each of a plurality of fingers is in contact with the input surface;
Of the fingers detected as being in contact with the input surface by the contact detection means, the input detection means for detecting a state change from contact to non-contact of at least one finger to generate an input signal ,
The contact detection means determines the type of each finger in contact with the input surface;
The input detection means generates an input signal according to the type of the finger that has detected the state change from contact to non-contact based on the determination result of the contact detection means,
When the input signal corresponding to the type of the finger is generated, the input signal is input based on the finger-command correspondence information in which the command for causing the motion execution device to execute a predetermined motion is associated with the input signal. Device control means for transmitting a command corresponding to the signal to the operation execution device,
The contact detection means determines whether the finger in contact with the input surface is a right hand or a left hand,
The input detection unit generates an input signal according to how many fingers of the right hand or the left hand are detected based on the determination result of the contact detection unit,
When the input signal corresponding to the type of each finger of the right hand or the left hand is generated, the device control means is associated with the input signal and a command for causing the operation execution device to execute a predetermined operation. Based on the finger-command correspondence information, a command corresponding to the input signal is transmitted to the operation execution device,
A command corresponding to each finger is extracted based on the finger-command corresponding information, and the command corresponding to the finger or the operation executed by the motion execution device according to the command is detected at the position of each finger detected by the contact detection means. Display means for displaying information indicating
The device control means transmits the command when the input detection means detects that the finger is in contact with the input surface again after detecting the change in state from contact to non-contact of the finger,
An imaging unit that captures an object on the input surface as image data,
The contact detection means detects that each of a plurality of fingers is in contact with the input surface based on image data captured by the imaging unit,
The imaging unit is composed of a liquid crystal display device with a built-in optical sensor,
The input surface is a display surface of the optical sensor built-in liquid crystal display device,
The display means displays a command corresponding to a finger, which is displayed in advance at a predetermined position, or the information indicating the operation performed by the motion execution device according to the command. An input device characterized in that when the contact is detected, it is displayed at the position of each finger detected by the contact detection means . According to the contact position detected by the contact detection means, the display position of the information displayed on the position of each finger by the display means, or the display position of the information indicating the operation executed by the action execution device according to the instruction. The input device according to claim 1, further comprising means for updating . The input detection means generates an input signal only when the contact detection means detects that five fingers of the thumb, index finger, middle finger, ring finger, and little finger are in contact with each other. Item 3. The input device according to Item 1 or 2. The display means displays information corresponding to a function that can be operated at present, out of the information corresponding to a finger displayed at a predetermined position or the information indicating the operation executed by the operation execution device according to the instruction. the input device according to any one of claims 1 to 3, characterized in that to display the position of each finger where the contact detecting means has detected.