WO2022181412A1 - Input assisting mechanism and input system - Google Patents

Abstract

The present invention enables a user to intuitively understand the extent to which a finger should be brought near to a display with which hovering input is possible, in order for input to be made.　In the present invention, an input device 200 is attached to a display with which hovering input is not possible. The input device 200 is provided with a light source 11 for radiating a laser, and a spread member 12 for expanding the laser radiated from the light source 11 in parallel to the screen of the display. The laser that has passed through the spread member 12 forms a planar light plane P. The input device 200 generates position information pertaining to a user's fingertip that has reached below the light plane P. The fingertip that has crossed the light plane P shines. Therefore, when the user moves the fingertip while confirming that the fingertip is shining, it is possible for the input device 200 to reliably generate position information pertaining to the fingertip.

Description

input support mechanism, input system

TECHNICAL FIELD The present invention relates to a technique for assisting a user who performs input in an input device that is attached to a target surface so that the target surface can be hover-inputted.
A typical example of a target surface is a display that does not allow hover input, whether it is a touch panel or not.

Liquid crystal displays, organic EL displays and other displays are widely used. A display may be used simply for the purpose of displaying an image, but is very often used for the purpose of allowing a user to perform some kind of input according to the image displayed on the display.
For example, in the case of a desktop personal computer, the user is often required to perform input via an accessory that is separate from the personal computer and the display, such as a mouse and trackball. In some cases, by using a touch panel as the display provided by it, it is becoming common to make input not via accessories by touching the screen of the user's display, and in smartphones, tablets, etc. In most cases, the user makes an input by touching a display, which is a touch panel.

Input from the touch panel realized by using the display as a touch panel does not require the preparation and operation of accessories. Since it is possible to perform intuitive operation, it is convenient.
Under these circumstances, by using a touch panel as the display, input via the touch panel is widely used in public applications such as ATMs (automated/automatic teller machines), ticket vending machines, and convenience store registers. It is also widely used for the display of the equipment of

Although the input via the touch panel as described above, which is realized by using a touch panel as the display of a device for public use, is convenient, it is not well received by some users. Some users dislike having to touch a touch panel that can be touched by an unspecified number of people because it is unclean.
In particular, with the recent spread of the new coronavirus infection, users who are extremely reluctant to touch the touch panel in order to avoid infection with the new coronavirus or other viruses beyond the issue of cleanliness is appearing. Moreover, even among users who do not have such a strong sense of disgust, the number of users who wish to perform input without touching the touch panel, if possible, is certainly increasing.

Of course, there are some touch panel displays that have already been put into practical use, but there are also those that allow hover input. A hover input is an input mechanism for inputting via a touch panel by touching a space of a predetermined distance in front of the screen of the touch panel with a finger. A type of display that allows hover input that allows non-contact input, also known as a touchless touch panel, is a touch panel that has grown rapidly in recent years due to the spread of the new coronavirus infection. It is a very good match for the demand for input without having to.

A display that allows hover input, for example, has a sensor built into the display that detects a slight change in capacitance between the display and the user's finger, thereby allowing input to the display. Some detect the coordinates of the user's finger on the display. Such displays are manufactured, sold, or released by Nissha Co., Ltd. and Japan Display Co., Ltd., for example.
In addition, in Japanese Patent Application Laid-Open No. 2007-310441 and Japanese Patent Application No. 2020-145788 filed earlier by the applicant of the present application, a display that is a touch panel that has been shipped and is already in use, for example, is used as a display that can perform hover input after the fact. An apparatus for is disclosed.
In addition, "AirBar (trademark)", which is a device sold by Techwind Co., Ltd. in Japan for the exclusive purpose of making a display that is not a touch panel function as a touch panel, is also used for the purpose of enabling hover input on a display that is a touch panel. It is applicable.

As described above, there are already various techniques for making a display capable of hover input or a display that is a touch panel into a display capable of hover input afterward.
However, these techniques have common drawbacks.
When performing input by hover input, the user naturally performs input without touching the display. The point is that it is difficult to make the user intuitively understand that the system reacts.
Of course, it is conceivable to notify the user of the fact that the input has been performed, for example, by means of a display on the display or sound emitted from a speaker. However, the above-mentioned difficulty is rather that the user, who is moving his/her finger in any direction on the display while inputting, rather than after the input has been made, is exposed to this degree of motion relative to the display. The above notification is not very useful as we want them to understand that they can make an input if they bring their finger close enough.

Regardless of whether or not hover input can be performed after the fact, the present invention allows a user who performs input to a display capable of hover input to intuitively know how close the finger to the display is to perform input. The task is to provide the technology to make it understandable to everyone.

In order to solve the above problems, the inventors of the present application propose the following inventions.
The present invention receives a target surface, which is a surface on which display is performed on a display on which a user operates, and operation data, which is data about an operation performed by the user on the target surface, and performs operations based on the operation data. an information processing device that executes predetermined information processing; and an information processing device that is located on the target surface when viewed from the front of the target surface and is located at a position closer than a first distance, which is a predetermined distance from the target surface. Input support used in combination with an input device that detects the positional coordinates of a user's fingertip on the target surface and outputs positional coordinate data, which is data about the positional coordinates, to the information processing device. mechanism.
In other words, the input support mechanism of the present invention includes a display having a target surface on which display is performed, an input device that generates operation data regarding an operation performed by a user on the display, and an input device that receives operation data from the input device and provides predetermined information. It is used in combination with a device enabled for hover input, including an information processing device that executes processing. Here, the display and the information processing device are usually integrated devices. On the other hand, the input device may be integrated with the display and the information processing device, or may be used after being attached to the integrated display and information processing device.
The input support mechanism includes a light source that emits light, and directs the light from the light source to the target on a plane parallel to the target surface that is a distance equal to or smaller than the first distance from the target surface. and a diffusing member that generates a light plane, which is planar light, by expanding a range covering at least a predetermined portion of the target plane when the plane is viewed from the front. The input support mechanism enables a user who has visually recognized a fingertip illuminated by light from the light source by crossing the light surface to recognize that the user's fingertip is closer to the target surface than the first distance. is configured to
If such an input support mechanism exists, when a user who intends to perform a hover input by bringing his/her finger close to the target surface of the display brings his/her fingertip close to the display, the fingertip eventually crosses the light surface. Then, even if the light surface itself in the air is invisible to the user, the fingertip crossing the light surface reflects or scatters the light from the light source and shines. Since the light plane is parallel to the target plane and is located at a distance from the target plane that is less than or equal to the first distance, the illuminated user's fingertip is positioned at a distance from the target plane that is less than or equal to the first distance. is guaranteed to be in As described above, the input device outputs the position coordinate data of the fingertip to the information processing device when the fingertip is at a position closer than the first distance from the target surface of the display. Therefore, by moving the user's fingertip along the target surface of the display while confirming that the fingertip is shining, the position coordinates of the user's fingertip located closer to the target surface than the first distance are The input device reliably outputs to the information processing device.
In other words, if an input support mechanism such as the one described above exists, hover input is possible when the fingertip is illuminated (when the fingertip is illuminated, the fingertip approaches the target surface of the display to the extent that hover input is possible). By allowing the user to recognize that the user's fingertip is glowing, the user can see that the current fingertip position (distance from the target surface of the display) is the hover input. It becomes possible to recognize that it is in a position where In other words, the input support mechanism of the present application enables a user who performs input to a display on which hover input can be performed to intuitively understand how close the finger must be to the target surface of the display to perform hover input.

As described above, the input support mechanism of the present invention includes a light source and a diffusing member. The light source has a function of emitting light. The diffusion member has a function of generating a light plane, which is planar light, by spreading the light from the light source. As long as those functions are guaranteed, the light source and diffusion member may be configured in any way.
The light source can be a linear light source that emits linear light. Examples of linear light sources are lasers, and examples of linear light sources are laser devices. Also, the diffusion member can be a cylindrical lens or a cylindrical mirror.
The light plane generated by the combination of the linear light source and the cylindrical lens or the cylindrical mirror is in a state where the light plane exists at any moment in the time zone during which the light plane is generated. Of course, it is also possible to arrange other optical elements such as mirrors and prisms in the optical path before or after reaching the cylindrical lens or cylindrical mirror.
On the other hand, the light plane according to the present application is a case where planar light does not exist and only linear light exists when focusing on a moment in the time zone in which the light plane is generated. In the case where light on a straight line exists at a certain high speed along a certain plane, for example, by wiping or rotating, and when focusing on a certain length of time, it exists in a plane is also a light plane. To generate such a plane of light, a linear light source, typically a laser device, and a mirror that oscillates back and forth within an angle, or a mirror that rotates about an axis (both e.g. Micro Electro Mechanical Systems) technology). For example, if a straight line of light is applied to an oscillating or rotating mirror perpendicular to the axis of oscillation or rotation, a planar light surface along a plane will be generated. Of course, it is also possible to arrange other optical elements such as oscillating or rotating mirrors and mirrors, prisms, etc. in the optical path before or after reaching the cylindrical mirror.

The input device used in combination with the input support mechanism of the present invention detects not only the positional coordinates of the user's fingertip on the target surface but also the distance coordinates, which are the coordinates of the distance from the target surface. It may be. In this case, the input device approaches the target plane to a distance where the distance from the user's fingertip to the target plane is shorter than a second distance, which is a predetermined distance shorter than the distance from the light plane to the target plane. A proximity signal may be generated when it is detected based on the distance coordinates, and the generated proximity signal may be sent to the light source.
The light source in the input support mechanism used in combination with such an input device is capable of changing the state of illumination of light, and when receiving the proximity signal, the state of illumination of light is changed. may be changed.
By doing so, the user can see the change in the way the fingertip shines across the light surface caused by the change in the state of the light emitted from the light source. Therefore, it is possible to recognize that the object surface of the display is closer than that. Depending on how the second distance is set, the user can recognize that the fingertip will touch the display if the fingertip is brought closer to the target surface of the display.
Said change in the state of illumination of light carried out by the light source can be, for example, blinking of light, change of wavelength of light in the visible light range, change of intensity of light. A change in the wavelength of light in the visible light region is essentially a change in the color of the light.

The input support mechanism includes a light source and a diffusing member, as described above. A plurality of sets of the light source and the diffusion member may be provided. The second set of light source and diffusion member may be the same as the light source and diffusion member described above. In the present invention, the second set of light source and diffusion member is referred to as an auxiliary light source and auxiliary diffusion member.
An input support mechanism having an auxiliary light source and an auxiliary diffusion member includes an auxiliary light source that irradiates light, and the light from the auxiliary light source to the target surface separated from the target surface by a predetermined distance smaller than the first distance. an auxiliary diffusion member that generates an auxiliary light surface that is planar light by expanding a range covering at least a predetermined portion of the target surface when the target surface is viewed from the front on a parallel plane. By crossing the auxiliary light surface, the user who visually recognizes the fingertip illuminated by the light from the auxiliary light source can recognize that the user's fingertip is closer to the target surface than the first distance. It's like
If such an input support mechanism exists, when the user brings the fingertip closer to the target surface of the display, the fingertip will first cross the light plane and eventually cross the auxiliary light plane. When the finger crosses the light plane, the fingertip shines first, and when the finger crosses the fill light plane, the fingertip shines at two points by crossing the light plane and the fill light plane. By visually recognizing such a change in the brightness of the fingertip, the user can recognize that the fingertip is closer to the target surface of the display than the position where the hover input is possible. Depending on how the distance of the auxiliary light plane from the target plane is set, the user can recognize that the fingertip will touch the display if the fingertip is brought closer to the target plane of the display.
The wavelength of the light from the light source and the wavelength of the light from the auxiliary light source may both be wavelengths in the visible light region and may be different from each other. As a result, the color of the light reflected or scattered by the fingertip differs between when it traverses the light plane and when it traverses the auxiliary light plane. It is possible to easily determine whether the crossing crosses only the light plane or crosses both the light plane and the auxiliary light plane. Also, the light from the light source and the auxiliary light source may have the same wavelength but different intensities.
In the above example, the number of pairs of the light source and the diffusion member was two, but it is naturally possible to have three or more pairs. In other words, a plurality of auxiliary light planes may exist in parallel.

The input support mechanism according to the present invention comprises a sensor for detecting that the user has attempted to input using the input device, and only when the sensor detects that the user has attempted to input using the input device , the light source may emit light.
A light source consumes power to emit light. Especially when the light source is a laser device, the power consumption is large. If there is such a sensor as described above, the light source emits light only when the user attempts to make an input using the input device, so power consumption can be suppressed.

As described above, the input support mechanism in the present invention is used in combination with the display, the information processing device, and the input device. Here, the display and the information processing device are integrated devices, and the input device may or may not be integrated with the display and the information processing device. When the input device is not integrated with the display and the information processing device, the input device is retrofitted to the display and the information processing device.
When the input device is not integrated with the display and the information processing device, the input support mechanism can be as follows. That is, the input support mechanism in the present invention is separate from the display and the information processing device, and is integrated with the input device attached to the display and the information processing device integrated. Also good. In other words, the input support mechanism may be part of an input device attached to the display and the information processing device. According to this, when the input device is attached to the display and the information processing device, the input support mechanism of the present application is automatically attached or incorporated therein.
Where the input assisting mechanism is part of an input device, the input device may comprise a frame surrounding the target surface, and the light source and diffusion member may be attached to the frame. According to this, when the frame is correctly positioned with respect to the target surface of the display, the light source and the diffusion member in the input support mechanism are also automatically correctly positioned with respect to the target surface of the display.

The inventor of the present application also proposes, as one aspect of the present invention, an input system in which the above-described input support mechanism is combined with a device capable of hover input, including a display, an information processing device, and an input device. The effect of such an input system is equivalent to the effect of the input assist mechanism built into the input system.
An example input system accepts a target surface, which is a surface on which display is performed on a display on which a user operates, and operation data, which is data about an operation performed by the user at a position close to the target surface. an information processing device that executes predetermined information processing based on the operation data; and a first distance that is located on the target surface when viewed from the front of the target surface and that is a predetermined distance from the target surface. An input device and an input support that detect position coordinates on the target surface of a user's fingertip located closer than It is an input system with a mechanism.
The input support mechanism in the input system includes a light source that emits light, and directs the light from the light source onto a plane parallel to the target surface that is a distance equal to or smaller than the first distance from the target surface. and a diffusing member that generates a light surface that is planar light by expanding a range covering at least a predetermined portion of the target surface when the target surface is viewed from the front, wherein the light surface A user who sees the fingertip illuminated by the light from the light source by crossing can recognize that the user's fingertip is closer to the target surface than the first distance.

The perspective view which shows roughly the structure of ATM used in combination with the input device containing the input support mechanism of 1st Embodiment. 1 is a perspective view of an input device including an input support mechanism of the first embodiment; FIG. AA sectional drawing in FIG. 2 of the input device shown in FIG. FIG. 3 is a cross-sectional view taken along line BB in FIG. 2 of the input device shown in FIG. 2; 3A and 3B are views showing examples of patterns formed by light emitters on the inner surfaces of the second long side plate and the second short side plate of the input device shown in FIG. 2; FIG. 3 is a plan view of the input device shown in FIG. 2 with the front plate removed; FIG. 3 is a hardware configuration diagram of a computer included in the input device shown in FIG. 2; FIG. 3 is a functional block diagram showing functional blocks generated in a computer included in the input device shown in FIG. 2; FIG. 3 is a diagram showing paths of image light from a finger to a camera when the input device shown in FIG. 2 is used; 3 is another diagram showing the path of image light from the finger to the camera when the input device shown in FIG. 2 is used; FIG. FIG. 4 is a cross-sectional view of the input device according to modification 2 at the same position as in FIG. 3 ; FIG. 5 is a cross-sectional view of the input device according to modification 2 at the same position as in FIG. 4 ; FIG. 11 is a plan view of the input device according to modification 2 with the front panel removed; Sectional drawing of the same position as FIG. 3 of the input device in 2nd Embodiment. The perspective view which shows the state which attached the input support apparatus in 3rd Embodiment to the display. FIG. 10 is a principle diagram in the case of creating a light plane using an oscillating mirror as a diffusing member.

Hereinafter, first to third embodiments of the present invention and modifications thereof will be described with reference to the drawings.
In the description of each embodiment and modifications, common objects are denoted by common reference numerals, and overlapping descriptions may be omitted as the case may be. In addition, the contents described in each embodiment and modifications can be combined with other embodiments and modifications as long as there is no contradiction when combined.

<<First embodiment>>
In the first embodiment, an example in which an input support mechanism according to the present invention is incorporated in an input device that is retrofitted to an existing device including a display and an information processing device will be described.
Existing devices for the public are for example, but not limited to, ATMs, ticket vending machines, convenience store registers. More specifically, the device may be a device that includes a display and an information processing device, and they are integrated directly or via other parts, and does not need to be a device for public use. It doesn't even have to be an existing device. However, since all such devices may be known or well-known, the details of their construction will not be described.
The display has a function of displaying on its screen information necessary for allowing the user to input operation data. The display may or may not be a touch panel. When the display is a touch panel, the user inputs operation data by touching the display. Operation data is input from a predetermined input device such as a push button provided near the display of the device.
In this embodiment, the existing device is an ATM already installed in town (see FIG. 1), and the display is a touch panel.

The ATM has a display 110 . The display 110 is attached to a known table 131 provided in the ATM in such a manner that the screen 111 is exposed, and is connected via a signal line 132 to the information processing device 120 incorporated inside the ATM. there is The exposed portion of the screen 111 is the target surface referred to in the present application.
FIG. 1 conceptually illustrates an ATM, but in the figure, only the screen 111 of the display 110 and the table 131 are illustrated with solid lines, and the entire ATM, the information processing device 120, and the signal line 132 are illustrated with dashed lines. Illustrated. In this embodiment, the screen 111 of the display 110, which corresponds to the object surface referred to in the invention of the present application, is rectangular although it is not necessarily so. The four corners of the screen 111 may be chamfered, but since the screen 111 does not need to be rectangular in the first place, the definition of rectangle need not be strict. In this embodiment, the four corners are chamfered. However, it is safe to say that the display 110 is rectangular.
The information processing device 120 is a general computer. The information processing apparatus 120 has a function of displaying information necessary for input to a user who inputs operation data on the screen 111 of the display 110 . If the device provided with the display 110 is an ATM, the image displayed on the screen 111 may be, for example, an image prompting the user to enter the PIN number of the cash card or a procedure for the ATM to be performed by the user. It is a publicly known or well-known image for making a selection from money, deposit, transfer, and the like. The information processing device 120 generates image data for an image displayed on the screen 111 of the display 110 and sends the image data to the display 110 via the signal line 132, as is known or known. . As a result, an image corresponding to the image data is displayed on the screen 111 of the display 110 .
The information processing device 120 also executes information processing based on input operation data. For example, when the device is an ATM, the information processing executed by the information processing device 120 includes user authentication based on a personal identification number entered by the user, and processing selected by the user from withdrawal, deposit, and transfer. are three. When the display 110 of the ATM is a touch panel, the operation data is originally sent from the display 110 to the information processing device 120 via the signal line 132 . However, in this embodiment, the operation data is sent from the input device to the information processing device 120, as will be described later. Of course, it is possible to input operation data from the display 110 to the information processing apparatus 120 while basically inputting operation data from the input device to the information processing apparatus 120 (even if the input device is retrofitted, the display It is also possible to keep the function of inputting operation data from 110 as it is). In this way, even if the input device fails or has other problems, it is possible to input operation data from the display 110 as before.

The input device in this embodiment is used in combination with the ATM explained above. Specifically, the input device is used by being fixed to a table 131 to which the display 110 is attached while being properly positioned with respect to the screen 111 of the display 110 . Thereby, the user can perform hover input on the screen 111 of the display 110 .
A perspective view of the input device 200 is shown in FIG. 3 is an AA cross-sectional view of the input device 200 in FIG. 2, FIG. 4 is a BB cross-sectional view of the same, and FIG. 5 is a cut in the middle of the thickness direction of the input device 200 in FIG. It is a cross-sectional view.
The input device 200 has a frame 210 . The frame 210 in this embodiment has, but is not limited to, a flat rectangular parallelepiped shape that does not have the bottom and top surfaces in FIG. 2, or that has open bottom and top surfaces. The frame 210 is made of metal or opaque resin, for example.
When the frame 210 is viewed from above in FIG. 2, the frame 210 is rectangular. For the sake of convenience, in FIG. The vertical plate located on the left side is called a first short side plate 213 and the vertical plate facing the first short side plate 213 is called a second short side plate 214 .
The first long side plate 211, the second long side plate 212, the first short side plate 213, and the second short side plate 214 are all rectangular, and the length in the height direction in FIG. all equal. When the input device 200 in FIG. 2 is viewed in plan, the first long side plate 211, the second long side plate 212, the first short side plate 213, and the second short side plate 214 are A rectangle is created, which in this embodiment may be referred to as a "virtual rectangle".
Above the first long side plate 211, the second long side plate 212, the first short side plate 213, and the second short side plate 214 in FIG. 2, there is a front plate 215 with a hole 215A. Although the front plate 215 is a single plate in this embodiment, this is not the only option. For example, each of the first long side plate 211, the second long side plate 212, the first short side plate 213, and the second short side plate 214 is formed into a so-called angle having an L-shaped cross section, so that the first long side plate 211 , the second long side plate 212, the first short side plate 213, and the second short side plate 214 may be combined.
The hole 215A of the front plate 215 substantially matches the shape and size of the screen 111 of the display 110 used in combination with the input device 200. As shown in FIG. The size of the hole 215A does not have to match the shape and size of the screen 111 of the display 110 completely. If there is, the size of the hole 215A may be smaller than the size of the screen 111. In other words, as will be described later, the frame 210 is configured such that the screen 111 of the display 110 is formed into a virtual rectangle consisting of a first long side plate 211, a second long side plate 212, a first short side plate 213, and a second short side plate 214. It is attached to the table 131 in a surrounding manner, but at that time, it is permissible for the inner edge of the hole 215A of the front plate 215 to slightly overlap the edge of the screen 111 when viewed from the front. The front plate 215 has holes near the inner surfaces of the first long side plate 211, the second long side plate 212, the first short side plate 213, and the second short side plate 214 (the inner surface of the frame 210, the same shall apply hereinafter). This is to prevent stray light from the external environment from entering through 215A, but this is not essential.

Mirrors 221 and 222 are attached to the inner surface of the first long side plate 211 and the inner surface of the first short side plate 213, respectively.
The mirror 221 attached to the inner surface of the first long side plate 211 is rectangular, covers substantially the entire inner surface of the first long side plate 211, and has a mirror surface.
The mirror 222 attached to the inner surface of the first short side plate 213 is rectangular, covers substantially the entire inner surface of the first short side plate 213, and the inner surface is a mirror surface.
The mirror 221 may cover the entire inner surface of the first long side plate 211, but in this embodiment, it does not cover the inner surface of the first long side plate 211 on the top, bottom, left, and right. Also, the mirror 222 may cover the entire inner surface of the first short side plate 213, but in this embodiment, it does not cover the inner surface of the first short side plate 213 on the top, bottom, left, and right. That is, the mirrors 221 and 222 are present along two adjacent sides of the virtual rectangle described above, but are somewhat shorter in length than each of the two adjacent sides. In this way, the side lengths of the virtual rectangles along which the mirrors 221 and 222 are aligned may be shorter than the lengths of the sides of the virtual rectangles along which the mirror surfaces of the mirrors 221 and 222 are aligned. do not have. However, when the virtual rectangle is viewed from above in FIG. 2, both the line segment formed by the mirror surface of the mirror 221 and the line segment formed by the mirror surface of the mirror 222 are the sides of the virtual rectangle parallel to these line segments. It is preferable that the rectangular range where the two intersect when translated toward the far side of , has a shape and size that allows the screen 111 of the display 110 as the target surface to fit within the range.
2 where the mirrors 221 and 222 are present are the same in FIGS. The range in the height direction where the mirrors 221 and 222 exist may be determined according to the distance from the screen 111 where the user's finger is to be detected.

Light emitters 231 and 232 are provided on the inner surfaces of the second long side plate 212 and the second short side plate 214, respectively, although not limited thereto. The light emitters 231 and 232 have a role of increasing the contrast between the finger and its background in the captured image when the finger is imaged by a camera, which will be described later. The light emitters 231, 232 can be made of known or well-known materials such as EL wires, LEDs, and the like. These emit light by power supplied from a power source (not shown).
Both the light emitters 231 and 232 may cover the entire inner surface of the second long side plate 212 and the second short side plate 214. In this embodiment, the light emitters 231 and 232 are It covers only a part of the inner surface of the second long side plate 212 and the second short side plate 214, and more specifically, although not limited to this, a predetermined pattern continuous to those inner surfaces is formed. forming.
An example of a continuous pattern is shown in FIG.
In the example shown in FIG. 5A, the light emitters 231 and 232 form vertical stripes in FIG. 2 on the entire inner surface of the second long side plate 212 or the second short side plate 214 . In this example, the width of the light emitters 231 and 232 and the width of the portion without the light emitters 231 and 232 are the same, but this is not necessarily the case.
In the example shown in FIG. 2B, the light emitters 231 and 232 form horizontal stripes in the horizontal direction in FIG. In this example, the width of the light emitters 231 and 232 and the width of the portion without the light emitters 231 and 232 are the same, but this is not necessarily the case.
In the example shown in (C), the light emitters 231 and 232 form a checkered pattern on the entire inner surface of the second long side plate 212 or the second short side plate 214 .
Of the inner surfaces of the second long side plate 212 and the second short side plate 214, the color of the portions where the light emitters 231 and 232 do not exist is, for example, black. It is preferable to use a color with low brightness such that the contrast between the existing portion and the existing portion is high when the image is captured by a camera described later.
When the light emitters 231 and 232 are not present on the inner surfaces of the second long side plate 212 and the second short side plate 214, it is preferable to lower the brightness of the color of those inner surfaces.

A camera 240 is provided on the inner surface of the frame 210, in this embodiment, at the intersection of the second long side plate 212 and the second short side plate 214 (FIG. 6). However, when the frame 210 is attached to the table 131 having the object surface, the position where the camera 240 is arranged is outside the object surface, and both the mirror surface of the mirror 221 and the mirror surface of the mirror 222 are positioned. Any position is acceptable as long as the entire image can be captured. A dashed line in FIG. 6 indicates the angle of view of the camera 240 . The height in FIG. 2 at which the camera 240 is mounted can be selected as appropriate. For example, the camera 240 can be mounted in the middle of the frame 210 in the height direction in FIG.
The camera 240 may be any device that can capture moving images and output moving image data, which is data about the moving images. Of course, such cameras are known or known and are commercially available. A suitable camera may be selected from such cameras and used as the camera 240 . However, it is advantageous for the camera 240 to have a wide angle of view so that the entire mirror surface of the mirror 221 and the mirror surface of the mirror 222 can be captured by the camera 240. Therefore, when selecting the camera 240, Such points should also be taken into consideration.

A computer (not shown) is arranged in the frame 210 . The computer is connected to the camera 240, and receives moving image data generated by the camera 240 from the camera 240 substantially in real time by a known technique. The method by which the computer receives moving image data from the camera 240 may be wired or wireless.
Note that the computer need not be attached to the frame 210, and may be provided inside the ATM, for example. When a computer is provided inside the ATM, the information processing device 120 described above may also serve as the computer. In this embodiment, but not limited to, it is assumed that a small computer is mounted on frame 210, and more particularly in the space within frame 210 above or below camera 240 in FIG.

The computer has a function of generating operation data, which will be described later, based on moving image data and outputting it to the information processing device 120 . If so, the computer may be known or well-known and commercially available.
The computer has hardware as shown in FIG. The computer in this embodiment comprises a CPU (Central Processing Unit) 311, a ROM (Read Only Memory) 312, a RAM (Random Access Memory) 313, an interface 315, and a bus 316 connecting them.
A CPU 311 is an arithmetic device and controls the entire computer. The CPU 311 executes the processing described below by executing the computer program.
The ROM 312 is a non-rewritable memory, and stores a computer program for operating the CPU 311, data required when the computer executes the following processes, and the like.
A RAM 313 is a rewritable memory and provides a work area for the CPU 311 to execute the following processes. For example, part of moving image data and operation data may be temporarily written to the RAM 313 .
The interface 315 serves as a window for connecting the CPU 311, ROM 312, and RAM 313 to the outside of the computer hardware shown in FIG. Data can be exchanged with the outside of the computer that has the hardware. For example, if the computer receives moving image data from the camera 240 by wire, the interface 315 is connected to a connection terminal connected to a conductor (not shown) for transmitting moving image data connecting the camera 240 and the computer. It is As a result, the CPU 311 or the like in the computer can receive moving image data from the camera 240 via the cable, terminal, and interface 315 . If the computer receives moving image data wirelessly from the camera 240, a receiver for receiving moving image data from a transmitter provided in the camera 240 for transmitting the moving image data is connected. As a result, the CPU 311 or the like in the computer can receive moving image data from the camera 240 via the receiver and the interface 315 . Similarly, since the computer needs to transmit generated operation data to the information processing device 120 , components necessary for transmitting such data are connected to the interface 315 . If operation data is transmitted from the computer to the information processing apparatus 120 by wire, the interface 315 has a terminal for connecting a cable (for example, a USB cable) to the information processing apparatus 120 by wire. If the operation data is to be transmitted wirelessly to the device 120, the interface 315 includes a transmitter (for example, Wi-Fi (trademark) or Bluetooth (trademark) standard) that transmits the operation data to a receiver included in the information processing device 120. transmitter) will be connected.

When the CPU 311 executes the computer program, various functional blocks as shown in FIG. 8 are generated inside the computer.
An input unit 321, an operation data generation unit 322, a coordinate data recording unit 323, and an output unit 324 are generated inside the computer.
The input section 321 is connected to the interface 315 and receives input from the interface 315 . Data received by the input unit 321 from the interface 315 is moving image data sent from the camera 240 . Moving image data is sent from the input unit 321 to the operation data generation unit 322 .
The operation data generator 322 has a function of generating operation data based on the moving image data received from the input unit 321 . The operation data generation unit 322 reads the coordinate data recorded in the coordinate data recording unit 323 when generating operation data, and uses it. The method by which the operation data generator 322 generates operation data will be described later. After generating the operation data, the operation data generator 322 sends it to the output unit 324 .
Coordinate data is recorded in the coordinate data recording section 323 as described above. Details of the coordinate data will be described later.
The output unit 324 has a function of outputting the operation data generated by the operation data generation unit 322 from the computer to the outside. The output unit 324 outputs operation data to the interface 315 . The operation data received by the interface 315 is sent to the information processing device 120 by wire or wirelessly.

In this embodiment, the input device 200, more specifically, the frame 210 is provided with an input support mechanism according to the present invention.
When the frame 210 is fixed to the table 131 as described later, the input support mechanism is located at a first distance, which is a predetermined distance, or a shorter distance from the screen 111 of the display 110. It is for forming a light plane, which is a plane of light parallel to the screen 111 and will be described later.
The input assist mechanism in this embodiment includes a light source 11 that emits light and directs the light from the light source 11 to the screen 111 of the display 110 at a distance that is a first distance or less than the first distance from the screen 111 of the display 110 . a diffusing member 12 that generates a light plane by spreading out in a plane parallel to the . The first distance is the distance from display 110 at which input device 200 generates operation data when the user's fingertip is positioned closer to screen 111 of display 110 than that distance.

The light emitted by the light source 11 in this embodiment is light with a wavelength in the visible light region. The light source 11 in this embodiment is, but not limited to, a linear light source that emits linear light. More specifically, the light source 11 in this embodiment is a laser device that emits a laser. Laser devices that emit lasers are, of course, publicly known or well-known, and are also commercially available, so the light source 11 can be appropriately selected from such laser devices.
The light source 11 , which is a laser device in this embodiment, is positioned inside the frame 210 , more specifically, at the intersection of the first long side plate 211 and the second short side plate 213 that constitute the frame 210 . Although mounted at a height position, the mounted position of the light source 11 need not be at such a position as long as the light plane can be produced at such a position as described above. For example, the light source 11 may be attached to the middle of the first long side plate 211 or the second long side plate 212 in the length direction. Although not limited to this, the laser, which is linear light emitted from the light source 11 in this embodiment, is directed generally along the diagonals of the frame 210, that is, the second long side plate 212 and the second short side plate 214. , and is irradiated toward the portion where the Further, the light emitted from the light source 11 is emitted in a direction parallel to the screen 111 of the display 110 when the input device 200 is attached to the table 131 .
Light emitted from the light source 11 passes through the diffusion member 12 , and the diffusion member 12 is arranged slightly inside the frame 210 of the light source 11 . The light passing through the diffusing member 12 spreads parallel to the screen 111 of the display 110 with a predetermined thin thickness, for example ideally a thickness corresponding to the diameter of a straight line of light. In this embodiment, a cylindrical lens is employed as the diffusion member 12 for realizing such diffusion of light. A cylindrical mirror can be used as the diffusing member 12 instead of the cylindrical lens, but when the cylindrical mirror is used as the diffusing member 12, the positional relationship between the diffusing member 12 and the light source 11 is reversed. Since both the cylindrical lens and the cylindrical mirror are publicly known or are commercially available, the cylindrical lens and the cylindrical mirror used as the diffusing member 12 can be appropriately selected from among them. The light spread thinly after passing through the diffusion member 12 is the light plane P (see FIGS. 3 and 4). The light plane P is formed, for example, so as to cover the entire screen 111 of the display 110 when viewed from the front. Although not limited to this, in this embodiment, the light plane P extends all the way inside the frame 210 . Also, in this embodiment, although not limited to this, the distance from the light plane P to the screen 111 of the display 110 is made equal to the first distance.
The light plane P itself is invisible, but when an object that reflects or scatters light crosses the light plane P, the part of the object that crosses the light plane P glows, so the user can see the screen of the display 110 within the frame 210. By moving the finger along 111, it can be seen that there is a light plane P at a predetermined distance away from the display 110. FIG.
On the other hand, the light plane P according to the present application is the case where planar light does not exist and only linear light exists when focusing on a moment in the time zone when the light plane is generated. In the case where light on a straight line exists at high speed along a plane, for example, by wiping or rotating, so that it exists in a plane for a certain length of time. is also the light plane P. In order to generate such a light plane P, it is possible to use a linear light source, typically a laser device, a mirror that reciprocates within a certain angle, or a mirror that rotates around a certain axis. can.
For example, as shown in FIG. 15, the light X1, which is linear light (for example, laser) emitted from the light source 11, corresponds to the swing axis of the planar mirror 19 swinging between the angles α. When the light X1 is applied to a portion, the light X1 is reflected in the direction X2 when the mirror 19 is in the position indicated by the solid line, and is reflected in the direction X3 when the mirror 19 is in the position indicated by the two-dot chain line (this , the mirror 19 corresponds to the diffusion member 12). Therefore, if the light X1, which is a straight line of light, continues to be emitted, the light X1 moves the plane between the light X2 and the light X3 (shaded plane) from the position of the solid line to the position of the broken line. Fill while moving up to the position. That is, if the light source 11 that emits linear light and the mirror 19 that oscillates are used in combination, it is possible to create a planar light plane P. FIG. Of course, in the example shown in FIG. 15, α is set small for convenience of drawing. In other words, the combination of the light source 11 and the mirror 19 can generate the light plane P having the same range as the light plane P created by using the light source 11 and the diffusion member 12 which is a cylindrical lens. In this case, the swinging speed of the mirror 19 can be set to a speed at which the light moving in a straight line can be viewed with the naked eye, for example, 30 Hz (30 reciprocations/second).
Similarly, by irradiating light, which is linear light, near the axis of rotation of a rotating mirror, for example, the same light plane P as described above is created using the light source 11 and the diffusion member 12, which is a cylindrical lens. It is possible to generate a light plane P over a range. In this case, the rotation speed of the mirror can be, for example, 60 revolutions/second.
The oscillating and rotating mirror can be composed of, for example, a MEMS mirror, but is not limited to this.

The usage and operation of the ATM to which the input device 200 described above is attached will be described.
When the input device 200 is used, it is fixed to the table 131 of the ATM. The input device 200 is fixed to the table 131 with the lower portion of FIG. 2 in contact with the upper surface of the table 131 . If the positioning described later is achieved, the input device 200 can function even if it is simply placed on the table 131 and not fixed. However, when the input device 200 is used, it must be positioned at a predetermined position with respect to the display 110 as will be described later. In this embodiment, the input device 200 is fixed to the table 131 because there is a possibility that the input device 200 may deviate from the set position. The fixing of the input device 200 to the table 131 may be performed by a known or well-known technique such as screwing, adhesion, or the like.
When the input device 200 is fixed to the table 131 , the first long side plate 211 , the second long side plate 212 , the first short side plate 213 , and the second short side plate 214 are exposed on the upper surface of the table 131 . surrounding the screen 111 of the display 110 that is on. Also, the frame 210 of the input device 200 and the screen of the display 110 are relatively fixed so as to have a predetermined positional relationship (see FIG. 9).
Further, by connecting the computer of the input device 200 and the information processing device 120 of the ATM, for example, with a cable (not shown), operation data can be transmitted from the computer of the input device 200 to the information processing device 120. do. If operation data is transmitted and received wirelessly between the two, at least one of the information processing device 120 and the input device 200 is set so that the operation data can be transmitted and received wirelessly.
In this state, in the space above the screen 111 of the ATM (roughly, the space below the light plane P in the space in which the mirrors 221 and 222 exist on the sides of the screen 111), Allows hover input. It is preferable to inform the user in some way beforehand that hover input can be performed without touching the screen 111 with the fingertip by bringing the finger closer to the screen 111 of the display 110 than the position where the fingertip shines.

In this embodiment, camera 240 is constantly capturing moving images, and moving image data is constantly being sent from camera 240 to the computer. However, the camera 240 can be used only during a period in which the user may perform hover input, for example, from when the user starts a transaction (for example, after a cash card is inserted into an ATM) to when the transaction ends. A moving image may be picked up only during the interval and the moving image data may be sent to the computer. The same applies to the light source 11 as well. The light source 11 may also irradiate light only from the time the user starts trading until the end of the trading. Further, when the target for which the input support mechanism is used is not an ATM, for example, even if the light source 11 emits light only when a person approaches the device to which the hover input is performed by a well-known human sensor. Alternatively, the light source 11 that has been turned off until then may emit light when the finger is captured in the camera 240 . The control of the light source 11, in which the light source 11, which has been turned off until then, emits light when the finger is photographed in the camera 240, can of course be applied to the case where the target of the input support mechanism is an ATM. be. For example, it is possible to cause the operation data generation unit 322 to perform processing for turning on and off the light source 11 . Of course, if the operation data generator 322 performs such processing, it is necessary to connect the computer interface 315 provided in the input device 200 and the light source 11 with a cable (not shown). be.
In any case, when the light source 11 emits light, a light plane P is generated as already explained. The light plane P remains constant in this embodiment until the light source 11 is extinguished.

Moving image data generated by capturing moving images by the camera 240 is sent from the camera 240 to the computer. Although not limited to this, in this embodiment, moving image data passes from the camera 240 through a cable (not shown) and through the terminal and interface 315 to reach the input unit 321 in the computer. The moving image data is sent from the input section 321 to the operation data generation section 322 .

The imaging range of the moving image captured by the camera 240 includes, for example, all of the space within the frame 210 that is within the first distance from the screen 111, which is the space where the above-described hover input on the screen 111 can be performed. The embodiment also includes screen 111 . In addition, the moving image capturing range includes the entire mirror surfaces of the mirrors 221 and 222 .
FIG. 9 shows how an image in the moving image specified by the moving image data when the user's finger enters the space in the frame 210 through the hole 215A of the frame 210. FIG. 10 will be described as an example. 9 and 10, illustration of the input support mechanism (the light source 11 and the diffusion member 12) is omitted.
For example, the user's finger is denoted by F in FIG. be reflected. Further, the image includes the finger by the image lights F11 and F12 from the finger F reaching the camera 240 by being reflected once by the mirror surface of the mirror 221 or the mirror 222 . Furthermore, the image reflects the finger by the image lights F21 and F22 from the finger F reaching the camera 240 by being reflected twice on the mirror surfaces of the mirror 221 and the mirror 222 once each. In other words, the image captured by the camera 240 includes five fingers F, and the fingers F captured in the image are captured from different directions.
Also, when the finger F is at the position shown in FIG. 10, only the finger by the image light L0 from the finger F directly reaching the camera 240 is reflected in the image. This is because the image light reflected by the mirrors 221 and 222 is blocked by the finger F located near the camera 240 and cannot reach the camera 240 .
As is clear from the above description, it is possible to determine where the finger F is on the screen 111 and whether it is captured in each image (still image for each frame) that constitutes the moving image specified by the moving image data. There is a one-to-one correspondence between the state of the stuck finger and the finger. In other words, there is a one-to-one correspondence between the combination of the X coordinate and the Y coordinate of the finger and the state of the finger captured in each image forming the moving image.
Therefore, it is assumed that the relative positions of the screen 111 and the frame 210 are in a predetermined relationship. If an example (or a model) of an image in which the finger is captured at the coordinates is recorded in the coordinate data recording unit 323 for each combination of coordinates, the operation data generation unit 322 can store the image of the camera at a certain time. The state of the fingers reflected in the moving image specified by the moving image data received from 240 (the number of fingers, the position of each finger, and the orientation if necessary) is recorded in the coordinate data recording unit 323. By comparing with the above-mentioned data, it becomes possible to specify the coordinates on the screen 111 where the finger is located directly above at that time, that is, the X and Y coordinates of the finger. Note that the coordinate data is the above-described data that associates the coordinates with the example of the image in which the finger is reflected, which should be recorded in the coordinate data recording unit 323 .
In this way, the coordinate data for the combination of the X coordinate and the Y coordinate for the position of the user's finger is constantly generated by the operation data generator 322 while the user is performing input using the input device 200. will be

Then, in this embodiment, when the user's finger moves closer to the screen 111 of the display 110 than the first distance and leaves the screen 111 of the display 110 after staying there for a predetermined time, for example, 0.3 seconds or more, , the user's finger touches the screen 111 corresponding to the location where the user's finger was resting. When the finger moves away from the screen 111 of the display 110 as described above, the operation data generation unit 322 generates coordinate data indicating the position of the finger at which the finger remained and the location specified by the coordinate data. The operation data is generated as a set of data including information indicating . This operation data can be exactly the same as a set of coordinate data+data indicating that the user has touched the coordinate, which can be input using a general touch panel.
In order to detect that the finger has left the screen 111 of the display 110, it is necessary to detect the Z coordinate if the distance from the screen 111 is the Z coordinate. Of course, the Z coordinate is the distance from the tip of the finger image of the image light directly reaching the camera 240 from the user's finger to the screen 111 reflected in the image. It can be easily identified. In this embodiment, the operation data generator 322 detects the Z coordinate of the fingertip position all the time while the coordinate data of the fingertip position is being generated. Thereby, the operation data generator 322 can detect that the fingertip has left the screen 111 of the display 110 .

Here, when the user brings the finger close to the first distance, the fingertip traverses the light plane P, so that the fingertip shines. Therefore, by bringing the fingertip closer to the screen 111 of the display 110 while visually recognizing that the fingertip is shining, and moving the finger away from the screen 111 after keeping the finger still there for 0.3 seconds, the input can be performed reliably. However, in other words, it is possible to cause the operation data generation unit 322 to generate the operation data.

The operation data is sent from the operation data generator 322 to the output unit 324, and sent from the output unit 324 to the information processing apparatus 120 via the interface 315, terminals and cables (not shown). The information processing device 120 that receives the operation data executes necessary information processing based on the operation data.
As described above, the operation data, which is a combination of the X and Y coordinates of the screen and the data indicating that the coordinates have been touched, can be sent from the display 110 to the information processing apparatus when the display 110 is a touch panel as described above. It can be exactly the same as the data input to 120 . Therefore, the information processing device 120 can display images from the input device 200 without changing the image displayed on the display 110 at all, or rather without modifying the entire ATM other than connecting to the input device 200 . Conventional information processing can be performed based on the input operation data.
After the user finishes inputting with the input device 200, the light source 11 is turned off.

In the above example, the operation data generation unit 322 generates the image of the finger, which is reflected in the image at that time, by the image light directly reaching the camera 240 from the user's finger, and the image of the finger up to the screen 111. From the distance, the Z coordinate of the tip of the finger was specified.
However, when the light emitters 231 and 232 are as shown in FIGS. 5(B) and 5(C), the image of the finger (the image of the finger by the image light directly reaching the camera 240 from the user's finger) ), it is possible to specify the Z coordinate of the tip of the finger at that time also by which light emitters 231 and 232 are hidden.
Therefore, by adding the data of the reflection pattern of the light emitters 231 and 232 to the above-described coordinate data recorded in the coordinate data recording unit 323, the operation data generating unit 322 can generate the moving image specified by the moving image data. It is also possible to specify the Z coordinate of the tip of the user's finger based on the positions and numbers of the light emitters 231 and 232 appearing in the image of each frame in the image.

The coordinate data recorded in the coordinate data recording unit 323 is, for example, a combination of a set of the X coordinate and the Y coordinate where the finger is positioned and an image captured by the camera 240 at that point in time where the finger is captured as teacher data. It is also possible to generate as learned data generated by deep running a computer. Of course, it is assumed that the image data used for deep running is obtained in a situation where the screen 111 and the frame 210 are in a predetermined relative positional relationship.
In that case, the operation data generation unit 322 functions as artificial intelligence, and can specify the X, Y, and Z coordinates of the tip of the finger at that time from the moving image data.

<Modification 1>
Modification 1 will also be described in which the input device 200 is attached to the ATM as in the first embodiment. There is almost no difference between the ATM and the input device 200 between Modification 1 and the first embodiment. The difference between the first embodiment and Modification 1 is the function of the operation data generator 322 and the manner of lighting the light source 11 .
In the first embodiment, the light plane P is kept constant while the user is making an input, in other words, while the light source 11 is on. In addition, in the input device 200 of the first embodiment, while the user is inputting or while the light source 11 is on, the operation data generator 322 generates the X coordinate, Y coordinate, and Z coordinate of the fingertip. It was supposed to keep detecting coordinates.
Further, in the first embodiment, the user visually recognizes the fingertip that shines by crossing the light plane P generated by the light from the light source 11 that is always on, so that the fingertip receives the operation data from the input device 200. It can be confirmed that the screen 111 of the display 110 is approached to the extent that the information processing apparatus 120 can input.

By the way, while the user is performing input using the input device 200 , the fingertip may come too close to the screen 111 and touch the screen 111 . In addition, due to the fact that the input data is changed according to the distance between the fingertip and the screen 111 of the display 110, etc., the user may notice that the fingertip is closer to the screen 111 of the display 110 than the light plane P, Alternatively, there may be cases where it is desired to notify that the distance between the fingertip and the screen 111 of the display 110 has become shorter than the first distance.
To enable this, in this embodiment, the user's fingertip is closer to the screen 111 than a second distance (eg, 1/2 to 1/3 of the first distance) that is shorter than the first distance. In some cases, the state of light irradiation from the light source 11 is changed.
In Modification 1, the operation data generator 322 continues to detect the X-coordinate, Y-coordinate, and Z-coordinate of the fingertip in the same manner as in the first embodiment. Then, the operation data generation unit 322 of Modification 1 generates proximity data when detecting from the Z coordinate of the fingertip position that the distance between the fingertip and the screen 111 of the display 110 has become shorter than the second distance. It is designed to This proximity data is sent to the light source 11 via the output unit 324 , the computer interface 315 provided in the input device 200 , and a cable (not shown) connecting the interface 315 and the light source 11 . Then, the light source 11 changes the state of light irradiation. Examples of changes in the state of illumination of light from the light source 11 can be flickering of light, change of wavelength of light in the visible light region, or change of intensity of light. If the light blinks, the light plane P blinks, and the light generated at the fingertip when the fingertip crosses the light plane P blinks. If the wavelength of light is changed, the color of the light generated at the fingertip and visually recognized by the user when the fingertip crosses the light plane P changes. Further, if the intensity of light is changed, the intensity of light generated at the fingertip when the fingertip crosses the light plane P is changed.

Thus, according to Modification 1, when the user's fingertip is closer to the screen 111 of the display 110 than the first distance, the fingertip shines in a constant state, and then the fingertip is closer to the display than the second distance. When approaching the screen 111 of 110, the fingertip will glow differently. The user can intuitively recognize the distance between the fingertip and the screen 111 of the display 110 by visually recognizing such a change in the state of light occurring at the fingertip.
In the first modification, two distances, a first distance and a second distance, are set for the distance between the fingertip and the screen 111 of the display 110, and the light emission method of the light source 11 is changed to two types. Of course, it is also possible to set three or more distance settings and three or more ways of emitting light from the light source 11 .

<Modification 2>
Modification 2 will also be described in which the input device 200 is attached to the ATM as in the first embodiment. There is almost no difference between the modification 2 and the first embodiment in both the ATM and the input device 200 . The difference between the first embodiment and the modification 2 is that in the first embodiment, the input support mechanism is composed of a set of the light source 11 and the diffusion member 12, whereas in the modification 2, the light sources 11 and 13 and the diffusion member The point is that it is composed of two sets of light sources 12 and 14 and diffusion members.
The light source 13 and the diffusion member 14 generate a light plane P1 parallel to the light plane P, different from the light plane P, at a position closer to the screen 111 of the display 110 than the light plane P created by the light source 11 and the diffusion member 12. (see FIGS. 11 and 12). The light plane P1 is formed, for example, like the light plane P so as to cover the entire screen 111 of the display 110 when viewed from the front. It is made to spread all the way inside.
The light source 13 and diffuser member 14 can be designed as appropriately as they can, for example, they can be the same as the light source 11 and diffuser member 12 . The light source 13 and the diffusion member 14 in Modification 2 are the same as the light source 11 and the diffusion member 12. However, although the wavelengths of the light emitted by the light source 13 and the light source 11 belong to the visible light region, they are not necessarily so. Although not required, the wavelengths of the two are assumed to be different. Alternatively, the light sources 11 and 13 may irradiate light of the same wavelength, but the intensity of the emitted light may be different. Further, the light source 13 and the diffusion member 14 are provided directly below the light source 11 and the diffusion member 12, for example, at positions corresponding to the second distance described in the first modification.

When the user performs input with the input device 200 including the two sets of light sources 11 and 13 and the diffusion members 12 and 14, the screen 111 of the display 110 is changed while the user's fingertip approaches the screen 111 of the display 110. When the distance to is shorter than the first distance, crossing the light plane P causes the user's fingertip to shine. Further, when the distance from the user's fingertip to the screen 111 of the display 110 becomes shorter than the second distance, the user's fingertip crosses both the light plane P and the light plane P1. , is different from the case where the fingertip crosses only the light plane P.
Thus, according to Modification 2, when the user's fingertip is closer to the screen 111 of the display 110 than the first distance, and when the user's fingertip is closer to the screen 111 of the display 110 than the second distance, Then, the state of the light generated at the fingertip changes. Therefore, the user can intuitively recognize the distance between the fingertip and the screen 111 of the display 110 by visually recognizing the change in the state of light occurring at the fingertip, as in the first modification. Moreover, in Modification 2, the colors of the light generated at the fingertip when crossing the light plane P and the light plane P1 are different, so that the user can easily perform the above-described visual recognition.

<<Second embodiment>>
Also in the second embodiment, the input device 200 is combined with the ATM as in the first embodiment.
The input device 200 attached to the ATM incorporates an input support mechanism configured in the same manner as in the case of the first embodiment.
The input device 200 of the first embodiment uses the camera 240 and the two mirrors 221 and 222 to specify the X and Y coordinates of the position of the fingertip during input. The input device 200 of the second embodiment specifies the X and Y coordinates of the position of the fingertip during input in the same way as the input device 200 of the first embodiment. The principle and mechanism for specifying the Y coordinate are different from those of the first embodiment.
Similar to the input device 200 of the first embodiment, the input device 200 of the second embodiment has a frame 210 . The frame 210 in the second embodiment is configured similarly to the frame 210 in the first embodiment.
However, inside the frame 210 of the second embodiment, neither the camera 240 nor the mirrors 221, 222 nor the light emitters 231, 232 that were present in the first embodiment are present. Instead, a large number of light emitting units 291 and light receiving units 292 are provided on the inner surface of the frame 210 in the input device 200 of the second embodiment (FIGS. 13 and 14).

A large number of light emitting portions 291 are provided inside the first long side plate 211 and inside the first short side plate 213 . All of the light emitting portions 291 provided on the first long side plate 211 are positioned at the same height position when the vertical direction in FIG. Installed at regular intervals. Further, all of the light emitting portions 291 provided on the first short side plate 213 are attached at equal intervals in the length direction of the first short side plate 213 so as to be positioned at the same height position. A large number of light-emitting portions 291 provided inside the first long side plate 211 and inside the first short side plate 213 do not necessarily have to be at the same height position, but in this embodiment, they are at the same height position. It is designed to be located in
The light emitting unit 291 is a light source that emits linear light. The light emitting unit 291 is adapted to emit light, for example, infrared light, which is light with wavelengths in the invisible region. The light emitting part 291 is directed toward the facing second long side plate 212 or second short side plate 214 in a direction perpendicular to the inner surface of the first long side plate 211 or the first short side plate 213 to which it is attached. It is designed to emit a straight line of light.
A large number of light receiving portions 292 are provided inside the second long side plate 212 and inside the second short side plate 214 .
The number of light receiving units 292 provided on the second long side plate 212 is the same as the number of light emitting units 291 provided on the first long side plate 211, and the light receiving units 292 and the light emitting units 291 provided on each of the two are one to one. Yes. The light-emitting portions 291 and the light-receiving portions 292 in one-to-one correspondence are located at exactly corresponding positions when viewed from the rear surface of the first long side plate 211 . As a result, the infrared rays emitted from the light emitting portions 291 provided on the first long side plate 211 are received by the light receiving portions 292 provided on the second long side plate 212 corresponding to the light emitting portions 291 that emitted the infrared rays. I am ready to receive it.
The number of light receiving units 292 provided on the second short side plate 214 is the same as the number of light emitting units 291 provided on the first short side plate 213, and the light receiving units 292 and the light emitting units 291 provided on both sides are one-to-one. Yes. The light-emitting portions 291 and the light-receiving portions 292 in one-to-one correspondence are located at exactly corresponding positions when viewed from the rear surface of the first short side plate 213 . As a result, the infrared light emitted from the light emitting portion 291 provided on the first short side plate 213 is received by the light receiving portion 292 provided on the second short side plate 214 in a one-to-one correspondence with the light emitting portion 291 that emitted the infrared light. It is designed to be accepted.
Each light receiving unit 292 detects whether or not the light emitted by the light emitting unit 291 is received at that time, and generates data indicating the light reception state.
When the user makes an input using the input device 200 , all the light emitters 291 emit infrared rays toward the corresponding light receivers 292 . As a result, a matrix formed by infrared rays running vertically and horizontally is formed inside the frame 210 when viewed from above. In FIGS. 13 and 14, infrared rays, which are straight rays, are denoted by R, and the arrow attached to the infrared rays R indicates the direction in which the infrared rays R travel.
When the fingertip of the user who performs input using the input device 200 enters the frame 210 and cuts off the infrared rays R vertically and horizontally, for example, one by one, the light receiving unit 292 that has received the cut off infrared rays R You will not receive R. Then, the light-receiving unit 292, which has generated data indicating that light (infrared R) is being received, now generates data indicating that light is not being received.
Therefore, which of the light receiving portions 292 on the second long side plate 212 is not receiving light and which of the light receiving portions 292 on the second short side plate 214 is not receiving light. By detecting whether the state is reached, it becomes possible to detect the position of the fingertip with respect to the screen of the display 110, that is, the X and Y coordinates of the fingertip.

The input device 200 in the second embodiment is also provided with an input support mechanism. The input support mechanism in the second embodiment includes a light source 11 and a diffusion member 12 (see FIG. 14), as in the first embodiment. Note that the illustration of the input support mechanism is omitted in FIG. 13 .
The input support mechanism in the second embodiment generates a light plane P parallel to the screen 111 of the display 110 provided in the device to which the input device 200 is attached, as in the first embodiment. Although not limited to this, the light plane P in the second embodiment extends all the way inside the frame 210 like the light plane P in the first embodiment.
As described above, in the input device 200 of the second embodiment, the X coordinate and the Y coordinate of the fingertip are detected depending on which infrared rays R in the vertical and horizontal directions of the infrared rays R arranged in a matrix are blocked by the fingertip. be able to. Therefore, unless the fingertip is brought close to the screen of the display 110 at least to the position of the infrared rays R arranged in a matrix, the user cannot perform input using the input device 200 according to the second embodiment. In this case, the first distance in the invention of the present application is the distance from the position where the infrared rays R arranged in a matrix form exist to the screen 111 of the display 110 .
The light plane P is provided at a position such that the distance from the screen 111 of the display 110 is equal to or shorter than the first distance. In the example shown in FIG. 14, the light plane P is positioned slightly closer to the screen 111 of the display 110 than the first distance from the screen 111 . When the user crosses the light plane P with the fingertip, the fingertip shines as in the case of the first embodiment, so the user can visually recognize that the fingertip shines. When the fingertip glows, the user's finger always crosses the matrix-shaped infrared rays R in front of the user's finger. Therefore, if the user operates the input device 200 while confirming that his or her fingertips are glowing, an erroneous input or an input error due to the fact that the fingertips do not reach the infrared rays R arranged in a matrix can be avoided. Therefore, it is possible to realize reliable input.

<<Third Embodiment>>
In the third embodiment, an input support device 500 will be described.
The input support device 500 according to the third embodiment is manufactured and sold by Nissha Co., Ltd. and Japan Display Co., Ltd., or has been announced. It is used in conjunction with a display that has the ability to detect the coordinates on the display of a user's finger attempting to provide input to the display by detecting changes. In other words, the display using the input support mechanism 500 in this embodiment includes the input device 200 itself in the first embodiment or the function of the input device 200 .
However, even if the user tries to make an input using such a display, the user suddenly knows how close the finger must be to the screen 111 before the display can detect the coordinates of the finger. Can not.
The input support device 500 of the third embodiment is used to solve such problems.

The input support device 500 includes an irradiation section 510 and a fixing section 520 .
The irradiation unit 510 incorporates the light source 11 and the diffusion member 12 described in the first embodiment, and has a function of emitting planar light, that is, light corresponding to the light plane P in the first embodiment. there is
The fixing section 520 has a function of fixing the irradiation section 510 to the display directly or indirectly via another member while the irradiation section 510 is positioned with respect to the screen 111 of the display. The fixed part 520 is, for example, rod-shaped. Moreover, it is desirable that the fixing unit 520 be configured such that the distance from the screen 111 to the irradiation unit 510 is variable.
The input support device 500 is used in a state in which the irradiation unit 510 is positioned appropriately with respect to the screen 111 and fixed to the display using the fixing unit 520 . In this state, the irradiation unit 510 forms a light plane P at a position where the display can detect the position of the fingertip when the fingertip is brought close to the screen 111 of the display, or at a position closer to the screen 111 than that. do. The light plane P is formed, for example, in a range that covers the entire screen 111 when the screen 111 of the display is viewed from the front.
By crossing the light plane P and confirming that the fingertip is shining, if the fingertip is brought close to the screen 111 of the display and input is made to the display, the position of the fingertip can be reliably specified by the display. No typos or typos.

Claims (10)

1. A target surface, which is a surface on which display is performed on a display to be operated by a user, and operation data, which is data about an operation performed by the user at a position close to the target surface, are received, and a predetermined operation is performed based on the operation data. An information processing device that executes information processing, and a user positioned on the target surface when viewed from the front of the target surface and at a position closer than a first distance, which is a predetermined distance from the target surface. An input support mechanism used in combination with an input device configured to detect positional coordinates of a fingertip on the target surface and output operation data including data about the positional coordinates to the information processing device. hand,
   a light source that emits light;
   light from the light source on a plane parallel to the target surface at a distance equal to or less than the first distance from the target surface, when the target surface is viewed from the front. a diffusing member that generates a light plane, which is planar light, by expanding a range that covers at least a predetermined portion;
   and
   By crossing the light surface, a user who visually recognizes the fingertip illuminated by the light from the light source can recognize that the user's fingertip is closer to the target surface than the first distance.
   Input support mechanism.
2. The light source is a linear light source that emits linear light, and the diffusion member is a cylindrical lens or a cylindrical mirror,
   The input support mechanism according to claim 1.
3. The input device detects, in addition to positional coordinates of the user's fingertip on the target surface, distance coordinates, which are coordinates of a distance from the target surface, and also detects distance coordinates from the user's fingertip to the target. when it is detected based on the distance coordinates that the distance to the surface has approached the object surface to a distance shorter than a second distance, which is a predetermined distance shorter than the distance from the light surface to the object surface; generating a proximity signal and sending the generated proximity signal to the light source;
   The light source is adapted to change the state of light irradiation, and is adapted to change the state of light irradiation when the proximity signal is received.
   3. The input support mechanism according to claim 1 or 2.
4. The change in the state of light irradiation is blinking of light or a change in wavelength of light in the visible light region.
   4. The input support mechanism according to claim 3.
5. an auxiliary light source that emits light;
   The light from the auxiliary light source is projected onto a plane parallel to the target surface, which is separated from the target surface by a predetermined distance smaller than the first distance. an auxiliary diffusion member that generates an auxiliary light surface that is planar light by expanding at least a range covering the part of
   and
   By traversing the auxiliary light surface, a user who visually recognizes the fingertip illuminated by the light from the auxiliary light source can recognize that the user's fingertip is closer to the target surface than the first distance. ing,
   The input support mechanism according to claim 1.
6. both the wavelength of the light from the light source and the wavelength of the light from the auxiliary light source are wavelengths in the visible light region and are different from each other;
   6. The input support mechanism according to claim 5.
7. A sensor is provided to detect that the user has attempted to input using the input device,
   The light source emits light only when the sensor detects an attempt by the user to make an input with the input device.
   The input support mechanism according to claim 1.
8. It is separate from the display and the information processing device, and is integrated with the input device attached to the integrated display and information processing device,
   The input support mechanism according to claim 1.
9. The input device comprises a frame surrounding the target surface, and the light source and the diffusing member are attached to the frame.
   9. The input support mechanism according to claim 8.
10. A target surface, which is a surface on which display is performed on a display to be operated by a user, and operation data, which is data about an operation performed by the user at a position close to the target surface, are received, and a predetermined operation is performed based on the operation data. An information processing device that executes information processing, and a user positioned on the target surface when viewed from the front of the target surface and at a position closer than a first distance, which is a predetermined distance from the target surface. an input device adapted to detect the position coordinates of the fingertip of the user on the target surface and output operation data including data about the position coordinates to the information processing device; and an input device comprising an input support mechanism a system,
    The input support mechanism is
    a light source that emits light;
    light from the light source on a plane parallel to the target surface at a distance equal to or less than the first distance from the target surface, when the target surface is viewed from the front. a diffusing member that generates a light plane, which is planar light, by expanding a range that covers at least a predetermined portion;
    and
    By crossing the light surface, a user who visually recognizes the fingertip illuminated by the light from the light source can recognize that the user's fingertip is closer to the target surface than the first distance.
    input system.

Images