JP2006010489A - Information device, information input method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information system, an information input method, and a program allowing to input information by quickly detecting movements of an indicating object such as a finger of a user. <P>SOLUTION: A cellular phone device 50 to which information is inputted in a noncontact manner by movements of a finger comprises a diaphragm 1, a stop control section 4 which controls the aperture of the diaphragm 1, a photographing section 3 which photographs a finger, a key contact detection section 11 which decides whether the finger is within a predetermined distance range or not by changing the aperture with the stop control section 4 and comparing a plurality of finger images photographed by the photographing section 3 for each aperture, and a key operation decision section 13 which decides what key has been operated when the finger has been decided to be within the predetermined distance range. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an information device such as a mobile phone device, and more particularly to an information device that can input information without contact.

  In recent years, downsizing of information devices such as mobile phone devices, PDA (Personal Digital Assistant) devices, and personal computer devices is progressing. In order to realize further downsizing of an information device, it is necessary to remove an input unit such as a numeric keypad and a keyboard for inputting information such as characters and numbers from the device configuration.

  As a device that can input information to an information device without using such an input unit, for example, a transparent touch panel is placed on a display unit such as an LCD (Liquid Crystal Display) device. An information device that displays an image for guiding input such as a keyboard on the screen of the display unit, and the user can input desired information by actually touching the touch panel as if touching a desired key on the image. It has been put into practical use.

  However, generally, a transparent touch panel has a high reflectance, so that it is difficult to see the display on the display unit arranged below under external light such as outdoors, and when a finger touches the touch panel for input. In the touch panel, dirt due to sebum adheres to the touch panel, and furthermore, the range that can be operated with a finger is limited to the size of the display unit. There is a problem that the possible range is also reduced, and the operability is thereby deteriorated, and the information device having such a configuration is not necessarily widely accepted by users.

  In view of such problems, the user moves an object such as a finger, pen, or pointing stick in a space without touching a device such as a touch panel, and the information device detects the movement of the object, thereby detecting how the object moves. There have been proposed information devices capable of inputting various information in a non-contact manner due to the difference.

As an example of such an information device, a plurality of light emitting diodes (hereinafter referred to as LEDs) are two-dimensionally lit in order, and the position of the reflected light on the finger of the light beam emitted from each LED is a position detection element. An information device that can detect the coordinates of a finger three-dimensionally and input various information based on the detected movement of the finger has been proposed (for example, a patent) See reference 1.)
JP 7-334299 A

  However, in the conventional information device as described above, the LEDs arranged two-dimensionally are turned on one by one in order, and the position of the reflected light from the finger corresponding to each LED is detected, and then the finger Since the three-dimensional coordinates are calculated, it takes time to detect the position of the finger. When the finger is moved quickly, it is difficult to follow the movement, and there is a problem that the position cannot be accurately detected.

  The present invention has been made in view of such a problem, and an information apparatus and an information input capable of accurately detecting the position and inputting information even when a pointing object such as a user's finger is quickly moved An object is to provide a method and a program.

  An information device of the present invention is an information device that generates information in a non-contact manner by the movement of an object, and includes a diaphragm unit, an aperture degree adjustment unit that adjusts the aperture of the diaphragm unit, a photographing unit that photographs an object, Whether or not the object is within a predetermined region by comparing a plurality of images having different opening degrees obtained by photographing the object with the photographing unit while varying the opening degree of the aperture unit by the opening degree adjusting unit. A distance determining unit for determining and an information generating unit for generating predetermined information corresponding to the position of the object when it is determined that the object is in the predetermined region are provided.

  With such a configuration, a plurality of images with different opening degrees are obtained by opening and closing the aperture, and the pointing object such as the user's finger is moved by a relatively simple process of comparing the plurality of images with different opening degrees. It is possible to provide an information apparatus capable of generating and inputting information by accurately detecting the position even if the apparatus is moved quickly.

  In addition, the information generation unit includes a coordinate calculation unit that calculates the two-dimensional coordinates of the object from one image selected from a plurality of images with different degrees of opening of the object, and the information generation unit includes different information depending on the two-dimensional coordinates of the object The structure which generate | occur | produces may be sufficient.

  According to such a configuration, since different information is generated based on the two-dimensional coordinates of the pointing object such as the user's finger, different information is generated by moving the pointing object vertically and horizontally toward the device. Is possible.

  Furthermore, the information generation unit includes an information storage unit that stores information generated in correspondence with the two-dimensional coordinates of the object, and the information generation unit is based on the information of the two-dimensional coordinates of the object and the information stored in the information storage unit. It may be configured to generate different information.

  According to such a configuration, since information to be generated can be provided in advance in the information storage unit in association with the two-dimensional coordinates of the object, information to be generated can be quickly changed. It becomes possible.

  Moreover, the structure provided with the display part which displays the information which generate | occur | produces from an information generation part corresponding to the two-dimensional coordinate of an object based on the information memorize | stored in the information storage part in the shape of a keyboard may be sufficient.

  According to such a configuration, since information that can be input in accordance with the coordinates of the pointing object such as the user's finger is displayed on the display unit, the user can indicate the pointing object such as the finger while viewing the displayed image. Can be moved to generate desired information.

  Furthermore, a trajectory generation unit that generates a movement trajectory of the object is provided, and the distance determination unit determines whether or not the object is in a trajectory display region that is larger than a predetermined region. The display unit may be configured to display the movement trajectory of the object when it is determined that the object is inside.

  According to such a configuration, when the pointing object such as a finger is outside the predetermined area, the movement trajectory of the pointing object is displayed on the display unit. Information that the user wants to input can be selected, and information can be easily generated.

  Further, the distance determination unit may be configured to determine whether or not the object is in a predetermined region based on the difference in the degree of focus in a plurality of images with different opening degrees of the object.

  According to such a configuration, it is possible to generate information by determining whether or not the object is in a predetermined region by a simple configuration in which the degrees of focus of images are compared.

  Further, the distance determination unit may be configured to calculate the degree of focus in a plurality of images having different opening degrees of an object based on a coefficient when DCT processing is performed.

  According to such a configuration, the degree of focus is calculated based on the coefficient using DCT processing widely used in image compression technology and the like, and the object is within a predetermined area based on the degree of focus. Since it is determined whether or not there is an information device, it is possible to provide an information device that can be easily configured using a DCT processing block or the like used in an image compression technique such as MPEG.

  In addition, the distance determination unit may be configured to calculate the degree of focus in a plurality of images with different opening degrees of an object by adding coefficient values of a predetermined frequency when DCT processing is performed.

  According to such a configuration, since the degree of focus is calculated by integrating the coefficient values of the predetermined frequency, it is possible to generate information more suitable for detecting a human finger.

  Further, the imaging unit captures three images with different opening degrees, and the distance determination unit has a larger opening degree than the first image and the first image among the three images with different opening degrees. It is determined whether or not the object is in the locus display area using the second image having a small opening degree, and the opening degree is smaller than that of the first image, the second image, and the second image. A configuration may be used in which it is determined whether or not the object is within a predetermined region using the third image.

  According to such a configuration, it is further determined whether or not the object is in the predetermined region and the region outside the region by a simple configuration of taking three images and comparing the images. can do.

  Furthermore, the photographing unit photographs three images having different opening degrees, and the distance determining unit is configured to select the smallest image of the first image and the first image among the three images having different opening degrees. It is determined whether or not the object is in the trajectory display area using the second image having a large opening degree, and using the second image and the third image having a larger opening degree than the second image. It may be configured to determine whether or not the object is within a predetermined area.

  According to such a configuration, the object is in the predetermined region and the region outside the predetermined region by a simple configuration in which three images are captured and compared. It can be determined whether or not.

  Further, the diaphragm unit may have a liquid crystal shutter.

  According to such a configuration, it is possible to further improve the durability of the throttle portion.

  Further, the opening degree adjusting unit may include a light amount adjusting unit that adjusts the amount of light so that the amount of light transmitted through the liquid crystal shutter is constant.

  According to such a configuration, since the amount of light passing through the liquid crystal shutter can be made constant, preprocessing such as brightness adjustment is not necessary when images taken with different apertures are compared. .

  Next, the information input method of the present invention includes a first step of capturing a first image of an object with a first aperture opening degree, and a second aperture opening degree larger than the first aperture opening degree. A second step of capturing a second image of the object, a third step of calculating a degree of focus of each of the first image and the second image, and a matching of each of the first image and the second image. A fourth step for determining whether or not the object is within the predetermined area by comparing the degrees of focus; and a predetermined step corresponding to the position of the object when it is determined that the object is within the predetermined area And a fifth step of inputting the above information.

  By such a method, a plurality of images having different opening degrees are obtained by opening and closing the diaphragm, and a plurality of images having different opening degrees are compared with each other. It is possible to provide an information input method capable of quickly detecting movement and inputting information.

  Next, the program of the present invention has a first step of inputting an image captured at a first aperture aperture of an object to a computer, and a second aperture aperture greater than the first aperture aperture. A second step of inputting the photographed image; a third step of calculating a degree of focus of each of the image photographed with the first aperture opening degree and the image photographed with the second aperture opening degree; A fourth determination is made as to whether or not the object is within a predetermined region by comparing the degrees of focus of the images shot with the first aperture opening degree and the images taken with the second aperture opening degree. And a fifth step of generating predetermined information corresponding to the position of the object when it is determined that the object is within the predetermined area.

  With such a program, it is possible to quickly detect the movement of a pointing object such as a user's finger by a relatively simple process of inputting a plurality of images having different opening degrees and comparing the plurality of images having different opening degrees. A program capable of generating information can be provided.

  As described above, by using the information apparatus, the information input method, and the program of the present invention, the pointing object such as the user's finger can be quickly moved by a relatively simple process of comparing a plurality of images having different opening degrees. It is possible to provide an information device, an information input method, and a program that can input information by accurately detecting the position even when moved.

  Hereinafter, a first embodiment of the present invention will be described in detail with reference to the drawings.

(First embodiment)
First, an information device and an information input method according to the first embodiment of the present invention will be described. In the first embodiment of the present invention, a mobile phone device will be described as an example of the information device.

  FIG. 1 is a diagram for explaining an outline of the mobile phone device 50 according to the first embodiment of the present invention.

  In FIG. 1, the mobile phone device 50 according to the first embodiment of the present invention displays a photographing unit 3 that photographs a user's finger 51, an input guidance display 52 that prompts the user to input, and a finger locus 53. A display unit 15 is provided. The user moves the finger 51 within the range of the angle of view θ of the photographing unit 3 to input a telephone number to the mobile phone device 50 and various information such as numbers and characters such as e-mail or address book. It is possible. In FIG. 1, the direction of the photographing optical axis of the photographing unit 3 is shown as a Z-axis direction, and directions orthogonal to the Z-axis are shown as an X-axis and a Y-axis, respectively.

A case where the user inputs various information such as numbers and characters to the mobile phone device 50 will be described. First, a keyboard-like input guidance display 52 simulating a numeric keypad display is displayed on the display unit 15. Further, when the user's finger 51 enters the area A ₁ within a predetermined distance range from the imaging unit 3, the locus 53 of the finger 51 is displayed on the screen of the display unit 15.

While viewing the trajectory 53, the finger 51 so that the trajectory 53 of his / her finger 51 overlaps the key display (for example, “1” etc.) corresponding to the information he / she wants to input in the input guidance display 52 of the display unit 15. Move. When the locus 53 of the finger 51 is overlapped on the display the desired key, the user is further closer to the finger 51 to the imaging unit 3, moving the finger 51 to reach the area A ₂ in FIG. 1 (hereinafter, this operation The information corresponding to the key display can be input to the mobile phone device 50.

That is, according to the mobile phone device 50 in the first embodiment of the present invention, the user operates the virtual keyboard 54 that exists in the space of the area A ₂ without bringing the finger 51 into contact with the mobile phone device 50. It is possible to input information in a non-contact manner. According to the cellular phone device 50 in the first embodiment of the present invention, it is not necessary to provide an input unit such as a numeric keypad in the cellular phone device 50, so that the size of the entire device can be further reduced.

  Next, the configuration of the mobile phone device 50 according to the first embodiment of the present invention will be described in more detail. FIG. 2 is a block diagram showing a configuration of the mobile phone device 50 according to the first embodiment of the present invention.

  The cellular phone device 50 captures an image signal by photographing a diaphragm 1 that can be controlled to be opened and closed, a single-focus lens 2 that collects light beams that have passed through the diaphragm 1, and a light image of a subject that has passed through the diaphragm 1 and the lens 2. A photographing unit 3 having an imaging device 33 such as a CCD to be output, a synchronizing signal generating unit 5 for generating a predetermined synchronizing signal, and an aperture for adjusting the aperture of the diaphragm 1 based on the synchronizing signal generated by the synchronizing signal generating unit 5 An image signal corresponding to the difference in aperture of the diaphragm 1 is acquired from the photographing unit 3 based on the synchronization signal generated by the adjusting unit 4 and the synchronization signal generator 5, and the image information is converted into the difference in aperture of the diaphragm 1. Accordingly, the distribution unit 6 distributes and outputs to the storage unit, the first image storage unit 7 that stores the image information distributed by the distribution unit 6, the second image storage unit 8 and the third image storage unit 9, 2 image storage unit 8 and A finger position detection unit 10, a first image storage unit 7, a second image storage unit 8, and a third unit that are connected to the third image storage unit 9 and detect the position of the finger 51 from the stored image information. Key contact detection unit 11, finger position detection unit 10, and key contact detection unit 11 that detect whether or not the finger 51 is pressed by a method described later from the image information stored in the image storage unit 9. A finger position storage unit 16 that stores finger position information indicating the positional relationship between the finger 51 and the imaging unit 3, and a locus generation unit that generates a movement locus of the finger 51 extracted by the finger position detection unit 10. 12, the key arrangement storage unit 14 for storing the contents of information to be input corresponding to the position of the finger 51, the position of the finger 51 detected by the finger position detection unit 10, the finger 51 detected by the key contact detection unit 11 Presence / absence of pressing operation and key arrangement storage unit 4 is used to determine which key operation the user has performed, to generate information corresponding to the key operation, and to output the key operation determination unit 13 and the finger 51 generated by the trajectory generation unit 12. The display unit 15 that displays the input guidance display 52 to the user based on the information stored in the locus 53 and the key arrangement storage unit 14, and the information corresponding to the key operation output from the key operation determination unit 13 are received. A mobile communication processing unit 60 that performs various known processes such as communication and information processing as a mobile phone device is provided.

  As long as the aperture can be adjusted, the aperture 1 may be configured by using a mechanically known aperture or by using a liquid crystal shutter. By using a liquid crystal shutter as the diaphragm 1, the durability is superior to that of a mechanical diaphragm, and the diaphragm adjustment unit 4 not only adjusts the aperture of the diaphragm 1 but also keeps the amount of light transmitted through the diaphragm 1 constant. As described above, it is possible to change the transmittance of the liquid crystal shutter. According to such a configuration, it is not necessary to perform extra processing such as luminance correction when comparing image information as will be described later, and it is desirable to use a liquid crystal shutter as the diaphragm 1 in view of practicality. .

  The diaphragm adjusting unit 4 may be a mechanism that adjusts the aperture when a mechanical diaphragm is used as the diaphragm 1, and when a liquid crystal shutter is used as the diaphragm 1, the opening / closing control and A control circuit for adjusting the transmittance may be configured.

  In the first embodiment of the present invention, description will be made assuming that the diaphragm adjusting unit 4 can adjust the aperture of the diaphragm 1 so as to have three states of “large”, “medium”, and “small”. . In order to control the transmittance of the liquid crystal shutter so that the diaphragm adjusting unit 4 keeps the light amount of the light beam transmitted through the liquid crystal shutter constant, the transmittance corresponding to the aperture is set to be constant. However, although not shown in the figure, it has a sensor unit for measuring the amount of light transmitted through the liquid crystal shutter, and the transmittance of the liquid crystal shutter so that the output from the sensor unit is constant. The structure which adjusts may be sufficient.

  The lens 2 may be a single-focus simple lens system that is widely installed in portable imaging devices such as a mobile phone device with a camera.

  As the image pickup element 33, a widely known element such as a CCD or a CMOS sensor can be used.

  The imaging unit 3 includes an aperture 1, a lens 2, and an image sensor 33, and the range of viewable angles of view is an area represented by the view angle θ in FIG.

FIG. 3 is a diagram showing a focusable range due to a difference in aperture of the diaphragm 1 in the mobile phone device 50 according to the first embodiment of the present invention. As shown in FIG. 3, in the mobile phone device 50 according to the first embodiment of the present invention, the depth of field when the aperture of the diaphragm 1 of the photographing unit 3 is “large” is expressed as a distance range Δ. ₃ is assumed. Further, when the aperture of the diaphragm 1 as "medium", the depth of field becomes deep, it is assumed that in focus in a range of distance range delta _2. Further, it is assumed that the aperture of the diaphragm 1 becomes deeper most depth of field in the case of the "small", focal point matching the range of the distance range delta _1. In the first embodiment of the present invention, the distance range Δ ₂ is a range in which the locus 53 of the finger 51 is displayed on the display unit 15 (region A ₁ in FIG. ₁ ), and the distance range Δ ₃ is a virtual range. This is a range (area A ₂ in FIG. 1) in which it is determined that the keyboard 54 has been pressed.

  Returning to FIG. 2 again, an oscillation circuit using crystal or the like can be used as the synchronization signal generator 5.

  The first image storage unit 7, the second image storage unit 8, the third image storage unit 9, and the key arrangement storage unit 14 are each appropriately selected from known storage means such as a hard disk device (HDD) or a semiconductor memory. The selected one can be used.

  In the cellular phone device 50 according to the first embodiment of the present invention, image information (hereinafter referred to as first image information) when the aperture size of the aperture 1 is “small” in the first image storage unit 7. However, image information when the aperture size of the diaphragm 1 is “medium” in the second image storage unit 8 (hereinafter referred to as second image information) is further added to the third image storage unit 9. It is assumed that image information (hereinafter referred to as third image information) when the size of the aperture of the diaphragm 1 is “large” is stored.

  The distribution unit 6 uses the first image storage unit 7, the second image storage unit 8, or the third image storage unit 6 based on the image signal output from the imaging unit 3 based on the synchronization signal generated by the synchronization signal generation unit 5. A dedicated circuit that distributes to the image storage unit 9 can also be used. As described above, when there is only one storage unit, an area in which the image information is to be stored is allocated according to the aperture of the diaphragm 1. The function of the sorting unit 6 may be realized by software that stores additional information indicating the aperture of the diaphragm 1 in addition to the image information.

The finger position detection unit 10 uses the second image information and the third image information described above to determine the position of the user's finger 51 using the difference in the object depth due to the difference in the aperture of the diaphragm 1. from the imaging unit 3 in 3 detects whether or not within a predetermined distance range Δ _2, when the user's finger 51 has entered the predetermined distance range Δ _2, stored in the finger position storage unit 16 the finger The key operation flag, which is position information, is turned ON, and the two-dimensional position coordinates (position coordinates on the XY plane in FIG. 1) of the finger 51 are detected and output to the locus generation unit 12 and the key operation determination unit 13. The function of the finger position detection unit 10 will be described later, but can be realized using either software or a dedicated circuit.

The key touch detection unit 11 uses the first image information, the second image information, the third image information, and the finger position information described above, and the user's finger 51 moves the distance range Δ in FIG. ₃ is detected, and the result is output to the key operation determination unit 13. Although the function of the key touch detection unit 11 will be described later, it can be realized by using either software or a dedicated circuit.

Locus generating unit 12, when the finger 51 with a finger position detecting section 10 is determined to have entered from the imaging unit 3 to the distance range Δ _2, acquires the two-dimensional coordinates of the finger 51 which is output from the finger position detecting section 10 Then, the locus 53 of the finger 51 is displayed on the display unit 15. The function of the locus generation unit 12 can be realized by using either software or a dedicated circuit.

  In the key arrangement storage unit 14, information in which the two-dimensional coordinates of the keyboard-like input guidance display 52 to be displayed on the screen of the display unit 15 and the information to be input are associated as shown in FIG. 4. The key arrangement information 61 is stored. FIG. 4 is a diagram showing an example of the key arrangement information 61 of the mobile phone device 50 according to the first embodiment of the present invention.

Key operation determination unit 13, when the finger 51 in the key touch detection unit 11 is determined to fall within the distance range delta ₃ in FIG. 2, obtaining a two-dimensional coordinate of the finger 51 which is output from the finger position detecting section 10 Then, with reference to the key arrangement information 61 stored in advance in the key arrangement storage unit 14, information to be input corresponding to the two-dimensional coordinates of the finger 51 is determined, and the information is output. For example, when information indicating that the finger 51 is at the position indicated by the position coordinate A in the key arrangement information 61 illustrated in FIG. 4 is output from the finger position detection unit 10, information on the key to be input correspondingly Becomes “5”, and the information “5” is output from the key operation determination unit 13 to the mobile communication processing unit 60. At this time, the key operation determination unit 13 changes the color of the key portion corresponding to “5” of the input guidance display 52 displayed on the display unit 15, or by a voice generation device (not shown) provided separately. A configuration with higher practicality can be realized if the configuration is such that the user is informed (for example, the virtual keyboard 54 has been pressed) by generating sound or the like.

  The information output from the key operation determination unit 13 is sent to the mobile communication processing unit 60, and various information processing such as communication call in the mobile communication processing unit 60 and text input in mail processing is performed.

  As the display unit 15, a display unit appropriately selected from known display devices such as LCD, EL, or PDP can be used.

  Here, functions of the finger position detection unit 10 and the key contact detection unit 11 in the mobile phone device 50 according to the first embodiment of the present invention will be described in more detail.

Finger position detecting section 10 and a key touch detection unit 11 of the present invention, by utilizing a difference in the object depth delta due to the difference of the opening degree D of the diaphragm 1, the finger 51 a distance ranging delta ₂ within or in Figure 3 is the delta ₃ It can be determined whether or not it is within the distance range.

  FIG. 5 is a diagram illustrating the basic principle of the function of the finger position detection unit 10 or the key contact detection unit 11 of the mobile phone device 50 according to the first embodiment of the present invention. As shown in FIG. 5, it is assumed here that the object 31 passes through the lens 32 of the aperture D and forms an image on the imaging surface of the image sensor 33. At this time, when the distance from the object 31 to the lens 32 is the distance u, the distance from the focal point F of the lens 32 is the focal distance f, and the pixel pitch (resolution) of the image sensor 33 is the pitch ε, the object depth Δ is ,

It is expressed.

  For example, when using a CCD with a pixel pitch ε = 0.001 mm, using a lens with a focal length f = 1 mm, and changing the aperture (aperture) D to 0.07 mm, 0.1 mm, and 0.2 mm, the focus FIG. 6 shows a change in the value of the object depth Δ with respect to the distance u from the camera at the alignment position. FIG. 6 is a diagram illustrating a change in the value of the object depth Δ with respect to the distance u from the camera at the focusing position when the diaphragm D is changed in the mobile phone device 50 according to the first embodiment of the present invention. is there.

  As shown in FIG. 6, for example, when the distance u from the lens 32 to the object 31 at the focusing position is 50 mm, the object depth Δ is about 10 mm when the diaphragm is opened (D = 0.2 mm). However, when the aperture is slightly reduced (D = 0.1 mm), it is about 30 mm, and when the aperture is further reduced (D = 0.07 mm), it is about 70 mm. By designing the lens 2 and the aperture 1 in various ways, it is possible to realize an arbitrary angle of view θ and a distance range, and it is possible to realize virtual keyboards 54 having various sizes according to the user's request. .

  The finger position detection unit 10 or the key contact detection unit 11 in the cellular phone device 50 of the present invention uses the difference in the object depth Δ due to the difference in the aperture D of the diaphragm. The operation principle of the finger position detection unit 10 and the key contact detection unit 11 of the mobile phone device 50 according to the first embodiment of the present invention will be described in more detail with reference to FIG. FIG. 7 is a diagram for explaining the operation principle of the finger position detection unit 10 and the key contact detection unit 11 in the mobile phone device 50 according to the first embodiment of the present invention. As shown in FIG. 7, the finger 71 is located 50 mm away from the lens 2 of the mobile phone device 50, the finger 72 is located 60 mm away from the lens 2, and the finger 73 is located 70 mm away from the lens 2. It is assumed that there is a finger 74 at a position 90 mm away from the lens 2. Note that FIG. 7 does not accurately reflect the distance relationship described above, and is a diagram schematically showing the position of each finger.

Further, the lens 2 mounted on the mobile phone device 50 has an object depth when the aperture u is “small” when the distance u from the camera at the focusing position is 50 mm as described above. Δ ₁ ≈10 mm (Focus position is in the range of ± 5 mm (strictly speaking, the focus range closer to the camera than the focus position is slightly larger than the focus range farther from the camera than the focus position) Although it is small, the discussion will be continued assuming that the in-focus range is the same before and after the focus position.)), And the object depth Δ ₂ = 30 mm with the aperture in the “medium” state (in-focus in the range of the focus position ± 15 mm) It is assumed that the object depth Δ ₃ = 70 mm (focusing at the focusing position ± 35 mm) when the aperture is “large”.

  FIG. 8 shows an example of image information to be taken by configuring in this way. FIG. 8 is a diagram showing an example of image information photographed by the mobile phone device 50 according to the first embodiment of the present invention.

FIG. 8A is a diagram illustrating an example of image information (first image information) when the aperture of the diaphragm is “small”. As with the first image information 81 shown in FIG. 8A, focused image information can be obtained with the fingers 71, 72, and 73, but the finger 74 is not clear, that is, a blurred image. Is obtained. That is, when the aperture of the diaphragm is "small", to the distance 50mm from the camera position of the focus, the object depth delta ₁ is a ± 35 mm, the finger 71 (distance 50mm), the finger 72 (Distance 60mm ) And finger 73 (distance 70 mm), it is possible to obtain focused image information. However, since the finger 74 (distance 90 mm) exceeds the in-focus range, a blurred image is obtained.

Next, when the aperture of the diaphragm is “medium”, the second image information 82 as shown in FIG. 8B can be obtained. The second image information 82 is in focus because the finger 71 (distance 50 mm) and the finger 72 (distance 60 mm) are at the focusing position 50 mm ± 15 mm, and the finger 73 (distance 70 mm) and the finger 74 (distance 70 mm) the distance 90 mm), since more than an object depth delta _2, since not focused, the image information blurred is obtained.

Further, when the aperture of the diaphragm is “large”, the third image information 83 as shown in FIG. 8C can be obtained. The third image information 83 is in focus because the finger 71 (distance 50 mm) is at a focus position of 50 mm ± 5 mm. However, the finger 72 (distance 60 mm), the finger 73 (distance 70 mm), the finger 74 (distance 90 mm), since more than an object depth delta _3, since not focused, the image information blurred is obtained.

The finger position detection unit 10 includes second image information 82 photographed with the aperture opening degree of “medium”, and third image information 83 photographed with the aperture opening degree of “large”. Compare Specifically, the finger position detection unit 10 compares the degree of focusing of the second image information 82 with the degree of focusing of the third image information 83 and the degree of focusing is different. , distance range Δ it has entered the finger 72 can know the in _2, it is possible to output a vertical and horizontal two-dimensional coordinates. Note that the calculation of the two-dimensional coordinates of the finger 72 can be easily performed by using the center coordinates and barycentric coordinates of the portions with different degrees of focus.

Further, the finger position detecting section 10, when detecting that the finger has entered the distance range Δ _2, turns ON the key operation flag is a finger position information stored in the finger position storage unit 16. Further, when the finger goes out of the distance range delta _2, the key operation flag to OFF.

  For the method of calculating the degree of blur in image information and determining the degree of focus, for example, “Depth distribution detection using moving images with continuously changing camera focus”, Akiyuki Kudo, Hidetoshi Miike, Information Processing Society of Japan, Computer Vision Study Group 98-2, published on January 18, 1996, “A New Sense for Depth of Field”, A.I. Pentlad, IEEE Trans. On Pattern Analysis and Machine Intelligence, vol. PAMI-9, No. 4 The method proposed in July 1987 can be used.

  It is also possible to use the method that the inventors have already proposed in JP-A-2001-227914. According to such a method, pixels constituting an image are divided into small blocks of vertical × horizontal = 8 × 8 pixels, and a coefficient obtained as a result of discrete cosine transform (hereinafter referred to as DCT) for each small block is a frequency. By arranging the components in a matrix and taking the square sum of the coefficients in a predetermined frequency region, it is possible to compare the degree of focusing of the images.

State placed within the fingers 72 from within a distance range delta ₂ further distance range delta _3, i.e. at the position of the finger 71 shown in FIG. 7, the aperture of the diaphragm 1 "large", "medium", When changed to “small”, as shown in FIGS. 8A, 8 </ b> B, and 8 </ b> C, an image in which any image of the finger 71 is focused is obtained.

That is, the key contact detection unit 11 includes first image information 83 in which the aperture of the diaphragm is “small”, second image information 82 in which the aperture of the diaphragm is “medium”, Compared with the third image information 83 in which the aperture of the aperture is “large”, the first image information 81, the second image information 82, and the third image information 83 are in focus. and by extracting the portion without the substantial difference in the degree, as well as identifying the image information of the portion of the entering finger 71 a distance range Δ _3, the finger 71 enters the distance range Δ ₃ Can be determined.

Here, when the finger is away than the distance range delta _1, i.e., when in the position of the finger 74 in FIG. 7, the first image information 81 as shown in FIG. 8, the second image information 82, In any of the third image information 83, an image having no difference in the degree of focus is obtained, but when the finger 71 is in the position, the key operation flag stored in the finger position storage unit 16 is ON, when in the position of the finger 74, since the key operation flag finger position storage unit 16 is OFF, the key touch detection unit 11 can detect whether or not entered the finger distance range Δ _3.

  In the mobile phone device 50 according to the first embodiment of the present invention, the mobile phone device 50 includes the first image storage unit 7, the second image storage unit 8, the third image storage unit 9, and the finger position. Although the configuration including the five storage units of the storage unit 16 and the key arrangement storage unit 14 is shown, this is a configuration in which the cellular phone device 50 according to the first embodiment of the present invention includes five storage media. It is not meant to be limited to. For example, the cellular phone device 50 has a single storage medium, and an area in which image information is stored and an area in which key arrangement is stored are allocated for each aperture of the diaphragm in the storage medium. It goes without saying that each piece of image information stored in the storage medium may include information indicating the aperture of the corresponding diaphragm as additional information.

  Furthermore, the structure which remove | excluded the portable communication process part 60 from the mobile telephone apparatus 50 in the 1st Embodiment of this invention, ie, the imaging | photography part 3, the aperture adjustment part 4, the synchronous signal generation part 5, the distribution part 6, 1st, Image storage unit 7, second image storage unit 8, third image storage unit 9, finger position detection unit 10, key contact detection unit 11, locus generation unit 12, key operation determination unit 13, and key arrangement storage unit 14. It is also possible to configure an input device including the finger position storage unit 16 and the display unit 15. If such an input device is configured, it is possible to perform various inputs such as numbers and characters by detecting the movement of the finger in a non-contact manner with respect to the information device connected to the outside.

  Next, the operation of the mobile phone device 50 according to the first embodiment of the present invention will be described. FIG. 9 is a flowchart showing the operation steps of the mobile phone device 50 according to the first embodiment of the present invention.

  As shown in FIG. 9, the cellular phone device 50 first sets the aperture of the diaphragm 1 to the “small” state by the diaphragm adjustment unit 4 and photographs the resulting optical image with the photographing unit 3 (S <b> 1). The distribution unit 6 stores the image information captured in step S1 in the first image storage unit 7 as the first image information 81 (S2).

  Next, the aperture adjusting unit 4 sets the aperture of the aperture 1 to the “medium” state, and the obtained optical image is captured by the imaging unit 3 (S3). The sorting unit 6 stores the image information captured in step S3 in the second image storage unit 8 as the second image information 82 (S4).

  Next, the aperture adjustment unit 4 sets the aperture of the aperture 1 to the “large” state, and the obtained optical image is captured by the imaging unit 3 (S5). The distribution unit 6 stores the image information captured in step S5 in the third image storage unit 9 as the third image information 83 (S6).

  Note that the present invention is not limited to the above-described order of photographing the three pieces of image information, and may be photographed in any order.

  Next, the key contact detection unit 11 includes first image information 81 stored in the first image storage unit 7, second image information 82 stored in the second image storage unit 8, and third image storage. The degree of focus is compared with the third image information 83 stored in the unit 9 (S7).

  When the key contact detection unit 11 determines that there is a difference in the degree of focus of these three image information (S8), the finger position detection unit 10 stores the second image stored in the second image storage unit 8. The degree of focus between the image information 82 and the third image information 83 stored in the third image storage unit 9 is compared (S9).

In step S9, when it is determined that there is no substantial difference in the degree of focus between the second image information 82 and the third image information 83 (S10), the finger 51 still has the predetermined distance range Δ. ₂ (position of the finger 73 or the finger 74 in FIG. 7), the finger position detection unit 10 turns off the key operation flag stored in the finger position storage unit 16 (S15). Returning to step S1 (S16), the first image information 81 is taken again. On the other hand, when the finger position detection unit 10 determines in step S9 that there is a substantial difference in the degree of focus between the second image information 82 and the third image information 83, the finger 51 is present from 50 to within a predetermined distance range delta ₂ since it is considered that (in the position of the finger 72 in FIG. 7), if the finger position detecting section 10 and the second image information 82 third image information 83 By extracting a portion having a difference in the degree of focus, it is possible to extract a region in which the finger 72 is included, and to detect the two-dimensional coordinates of the finger 72 by calculating the center of gravity or the center coordinates of the region. Yes (S11). Then, the value of the two-dimensional coordinate is temporarily stored in the finger position detection unit 10 (S12). Also, the finger position detection unit 10 turns on the key operation flag stored in the finger position storage unit 16 (S13). The trajectory generation unit 12 displays the two-dimensional coordinates of the finger 72 output from the finger position detection unit 10 as a trajectory 53 while being superimposed on the input guidance display 52 of the display unit 15.

  In step S7, when the key contact detection unit 11 determines that there is no substantial difference in the degree of focus between the first image information 81, the second image information 82, and the third image information 83 (S8). ), The key contact detection unit 11 checks the key operation flag stored in the finger position storage unit 16 (S17).

In step S17, when the key operation flag is OFF, outside fingers 51 distance range delta ₁ in FIG. 7, i.e. in the position of the finger 74 in FIG. 7, and those not too finger 51 is pressed Then, the process returns to step S1 (S18). On the other hand, when the key contact detection unit 11 determines in step S17 that the key operation flag stored in the finger position storage unit 16 is ON, the key contact detection unit 11 determines that the position of the finger 51 is the distance in FIG. It is determined that it is within the range Δ ₃ , and a signal indicating that the finger 51 has been pressed is sent to the key operation determination unit 13. In this case, the key operation determination unit 13 receives the signal indicating that the finger 51 is pressed from the key contact detection unit 11 and receives the two-dimensional coordinates of the finger 72 sent from the finger position detection unit 10 and the key arrangement. The key arrangement information 61 stored in the storage unit 14 is compared (S19), it is determined which key the user's finger 51 is pressed to operate (S20), and the information corresponding to the key Is sent to the mobile communication processing unit 60 (S22). In step S19, when the key information corresponding to the position of the finger 51 is not in the key arrangement information 61, the process returns to step S1 (S21).

  By performing such an operation, according to the mobile phone device 50 in the first embodiment of the present invention, the two-dimensional coordinates of the user's finger 51 and the presence or absence of pressing can be detected without contact. It is possible to input numbers and characters without providing an input unit such as a numeric keypad or a keyboard.

  In the first embodiment of the present invention, an example is shown in which the user performs input to the information device by moving the finger 51. However, the present invention does not limit the object that instructs the information device. Absent. For example, in addition to the finger 51, information can be input using a pen, a pointing stick, or the like.

  Also, according to the mobile phone device 50 of the first embodiment of the present invention, information can be input by taking three images and comparing the images, so that processing is performed in a short time. It is possible.

  In the first embodiment of the present invention, the mobile phone device 50 is described as an example. However, the information device of the present invention is not limited to this, and includes, for example, a personal computer device and a PDA device widely. Needless to say. As an example of such an information device, the appearance of a wristwatch-type information device 95 is shown in FIG. FIG. 10 is a diagram showing another example of the information device according to the first embodiment of the present invention. The information device 95 is the mobile phone device 50 according to the first embodiment of the present invention shown in FIG. It has the same configuration as As shown in FIG. 10, even in a wristwatch type information device 95 with a small display unit 15, various information items can be contacted by the finger 51 without contact, without providing an input unit such as a keyboard as in the mobile phone device 50 described above. Input can be made.

(Second Embodiment)
Next, an information device according to a second embodiment of the present invention will be described. Also in the second embodiment of the present invention, as in the first embodiment, a description is given using a mobile phone device 90 as an example of an information device. FIG. 11 is a block diagram showing the configuration of the mobile phone device 90 according to the second embodiment of the present invention. As shown in FIG. 11, the configuration of the mobile phone device 90 in the second embodiment of the present invention is compared with the configuration of the mobile phone device 50 in the first embodiment of the present invention shown in FIG. The finger position storage unit 16 is not provided, the finger position detection unit 91 is connected to the first image storage unit 7 and the second image storage unit 8, and the key contact detection unit 92 is the second point. The image storage unit 8 and the third image storage unit 9 are connected to each other.

In the mobile phone device 90 according to the second embodiment of the present invention, the finger position detection unit 91 compares the first image information 81 and the second image information 82 by comparing the degree of focusing. If the degree of focus is substantially different, 51 distance range Δ within ₁ finger, i.e. as in the position of the finger 73 in FIG. 7, the degree of focusing by using the center of gravity or center coordinates of the different parts The two-dimensional coordinates of the finger 73 are sent to the key operation determination unit 13 and the locus generation unit 12.

Further, in the mobile phone device 90 according to the second embodiment of the present invention, the key contact detection unit 92 compares the degree of focusing between the second image information 82 and the third image information 83, and If the degree of focus is different, within finger 51 a distance range delta _2, i.e. it determines that the key contact is as in the position of the finger 72 in FIG. 7, the key of the virtual keyboard 54 is depressed A signal to that effect is sent to the key operation determination unit 13.

  Since the functions of the other constituent elements are the same as the functions of the constituent elements described in the mobile phone device 50 of the first embodiment, detailed description thereof is omitted.

  Next, the operation of the cellular phone device 90 in the second embodiment of the present invention will be described. FIG. 12 is a flowchart showing the operation steps of the cellular phone device 90 according to the second embodiment of the present invention.

  As shown in FIG. 12, the operation steps from step S1 to step S6 are the same as the operation steps of the cellular phone device 50 in the first embodiment of the present invention shown in FIG. Omitted.

When the storage of the first image information 81, the second image information 82 and the third image information 83 in the storage unit is completed, the finger position detection unit 91 stores the first image information 81 stored in the first image storage unit 7. The degree of focus between the first image information 81 and the second image information 82 stored in the second image storage unit 8 is compared (S31). If there is a difference in the degree of focus (S8), it is considered that the finger 51 has entered the distance range Δ ₁ in FIG. 7, that is, the position of the finger 73, and the process proceeds to step S11. The process returns to S1 (S30).

After the finger position detector 91 calculates the two-dimensional coordinates of the finger 73 (S11), the key contact detector 92 compares the degree of focus between the second image information 82 and the third image information 83 ( S32). When there is a substantial difference in the degree of focus between the second image information 82 and the third image information 83 (S33), the finger 51 is in the distance range Δ ₂ in FIG. On the other hand, if it is determined that there is no difference (S33), it is assumed that the finger 73 is located, and the process returns to step S1 (S35).

  Note that the processing steps from step S19 to step S22 are the same as the processing steps shown in FIG. 9 of the first embodiment, and thus the description thereof is omitted.

As described above, in the mobile phone device 90 according to the second embodiment of the present invention, as in the mobile phone device 50 described in the first embodiment, the finger can be touched without touching the device. It becomes possible to detect the position and input the corresponding information. In the cellular phone device 90 according to the second embodiment of the present invention, the distance range Δ ₁ is a range in which the locus 53 of the finger 51 is displayed on the display unit 15 (region A ₁ in FIG. ₁ ), and the distance range Δ _2. Is a range in which it is determined that the virtual keyboard 54 is pressed (area A ₂ in FIG. 1).

  Also in the cellular phone device 90 according to the second embodiment of the present invention, information can be input by taking three images and comparing the images, so that processing can be performed in a short time. This is the same as in the first embodiment.

  According to the information apparatus, the information input method, and the program according to the present invention, it is possible to quickly detect the movement of the pointing object such as the user's finger by a relatively simple process of comparing a plurality of images having different opening degrees. Since an effect that information can be input can be achieved, the present invention is useful as an information device such as a mobile phone device, particularly an information device that can input information without contact.

The figure for demonstrating the outline | summary of the mobile telephone apparatus in the 1st Embodiment of this invention The block diagram which shows the structure of the mobile telephone apparatus in the 1st Embodiment of this invention The figure which shows the focus possible range by the difference in the opening of the aperture_diaphragm | restriction in the mobile telephone apparatus in the 1st Embodiment of this invention. The figure which shows an example of the key arrangement | positioning information of the mobile telephone apparatus in the 1st Embodiment of this invention The figure which shows the basic principle of the function of the finger position detection part of the mobile telephone apparatus in the 1st Embodiment of this invention, or a key contact detection part. The figure which shows the change of the value of the object depth with respect to the position of a focus at the time of changing the aperture_diaphragm | restriction in the mobile telephone apparatus of the 1st Embodiment of this invention. The figure for demonstrating the principle of operation of the finger position detection part and key contact detection part in the mobile telephone apparatus of the 1st Embodiment of this invention. The figure which shows an example of the image information image | photographed with the mobile telephone apparatus in the 1st Embodiment of this invention The flowchart which shows the operation | movement step of the mobile telephone apparatus in the 1st Embodiment of this invention. The figure which shows the other example of the information apparatus in the 1st Embodiment of this invention The block diagram which shows the structure of the mobile telephone apparatus in the 2nd Embodiment of this invention. The flowchart which shows the operation | movement step of the mobile telephone apparatus in the 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Aperture 2,32 Lens 3 Image pick-up part 4 Aperture adjustment part 5 Synchronization signal generation part 6 Sorting part 7 1st image memory | storage part 8 2nd image memory | storage part 9 3rd image memory | storage part 10,91 Finger position detection part 11 , 92 Key contact detection unit 12 Trajectory generation unit 13 Key operation determination unit 14 Key arrangement storage unit 15 Display unit 16 Finger position storage unit 31 Object 33 Image sensor 50, 90 Mobile phone device 51, 71, 72, 73, 74 Finger 52 Input guidance display 53 Trajectory 54 Virtual keyboard 60 Mobile communication processing unit 61 Key arrangement information 81 First image information 82 Second image information 83 Third image information 95 Information device

Claims (14)

An information device that generates information in a non-contact manner by the movement of an object,
An aperture,
An aperture adjuster for adjusting the aperture of the aperture,
A photographing unit for photographing the object;
The object is within a predetermined region by comparing a plurality of images having different opening degrees obtained by photographing the object with the photographing unit while varying the opening degree of the aperture unit by the opening degree adjusting unit. A distance determination unit that determines whether or not there is,
An information apparatus comprising: an information generation unit that generates predetermined information corresponding to a position of the object when it is determined that the object is in the predetermined area. A coordinate calculation unit that calculates two-dimensional coordinates of the object from one image selected from a plurality of images having different opening degrees of the object;
The information device according to claim 1, wherein the information generation unit generates different information according to two-dimensional coordinates of the object. An information storage unit that stores information generated in correspondence with the two-dimensional coordinates of the object;
The information device according to claim 2, wherein the information generation unit generates the different information based on information of two-dimensional coordinates of the object and information stored in the information storage unit. 4. A display unit for displaying information generated from the information generation unit in a keyboard shape corresponding to the two-dimensional coordinates of the object based on the information stored in the information storage unit. The information device described in 1. A trajectory generator for generating a movement trajectory of the object;
The distance determination unit determines whether or not the object is in a locus display area larger than the predetermined area,
The information device according to claim 4, wherein when the distance determination unit determines that the object is within the locus display area, the display unit displays a movement locus of the object. The distance determination unit determines whether or not the object is in the predetermined region based on a difference in focus degrees in a plurality of images having different opening degrees of the object. The information device according to any one of claims 1 to 5. The information device according to claim 6, wherein the distance determination unit calculates a degree of focus in a plurality of images having different opening degrees of the object based on a coefficient when DCT processing is performed. 8. The distance determination unit according to claim 7, wherein the in-focus degree in the plurality of images having different opening degrees of the object is calculated by integrating coefficient values of a predetermined frequency when DCT processing is performed. Information device. The imaging unit captures three images with different apertures,
The distance determination unit uses the first image having the largest opening degree and the second image having the smaller opening degree than the first image among the three images having different opening degrees, It is determined whether or not the object is in a locus display area, and the object is moved using the first image, the second image, and a third image having a smaller opening degree than the second image. The information device according to claim 5, wherein it is determined whether or not it is within a predetermined area. The imaging unit captures three images with different apertures,
The distance determination unit uses the first image having the smallest opening degree and the second image having the larger opening degree than the first image among the three images having different opening degrees, It is determined whether or not the object is in the locus display area, and whether the object is in the predetermined area using the second image and the third image having a larger opening degree than the second image. The information device according to any one of claims 5 to 8, wherein it is determined whether or not. The information device according to claim 1, wherein the aperture section includes a liquid crystal shutter. 12. The information apparatus according to claim 11, wherein the opening degree adjustment unit includes a light amount adjustment unit that adjusts the light amount so that a light amount of light passing through the liquid crystal shutter is constant. A first step of taking a first image of an object with a first aperture opening;
A second step of taking a second image of the object with a second aperture aperture greater than the first aperture aperture;
A third step of calculating a degree of focus of each of the first image and the second image;
A fourth step of determining whether or not the object is in a predetermined region by comparing the degrees of focus of the first image and the second image;
An information input method comprising: a fifth step of inputting predetermined information corresponding to the position of the object when it is determined that the object is in the predetermined area. On the computer,
A first step of inputting an image taken with a first aperture of the object;
A second step of inputting an image photographed at a second aperture opening degree larger than the first aperture opening degree;
A third step of calculating a degree of focus of each of the image captured with the first aperture opening degree and the image captured with the second aperture opening degree;
It is determined whether or not the object is within a predetermined region by comparing the degrees of focus of the image captured with the first aperture opening degree and the image captured with the second aperture opening degree. A fourth step to:
And a fifth step of generating predetermined information corresponding to the position of the object when it is determined that the object is within the predetermined area.