JP4175070B2 - Image input / output device, man-machine interface system and program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a man-machine interface.
[0002]
[Prior art]
The interface of the personal computer is composed of a display such as a CRT or a liquid crystal display and an input device such as a keyboard and a digitizer, and a pointing device such as a mouse is also used as an auxiliary input device. In other information processing apparatuses or processing terminals such as mobile phones and PDAs, a combination of a display and a keyboard or a digitizer is still a man-machine interface. Although voice input is possible instead of a keyboard or digitizer, it is not in a state where all personal computers can be controlled by voice input alone.
[0003]
[Patent Document 1]
JP-A-8-163476
[0004]
[Problems to be solved by the invention]
The combination of a display and a keyboard or a digitizer such as a man-machine interface is very common at present, and there is almost no intention to change the man-machine interface.
[0005]
However, this man-machine interface includes several problems. First, if information is to be displayed with a certain degree of resolution, a certain size of display is required. A display having a certain size consumes a large amount of power in order to emit light with sufficient luminance.
[0006]
Also, a keyboard or digitizer is required to enter information. The digitizer can be placed on top of the display, but in any case, something that functions as a display is required. Therefore, whatever form the display is in, it must be placed on the desk and a significant portion of the desk will be occupied by the man-machine interface.
[0007]
Therefore, a man-machine interface that combines a display and keyboard or digitizer is not an ideal interface in terms of saving energy and freeing up space on the desk. Even if an energy-saving type display is developed, the display cannot be made smaller. Rather, given the increasing amount of information, it is expected that the display will become larger except for special applications such as mobile phones. Accordingly, power consumption tends to increase.
[0008]
Therefore, the present invention has an object to provide a space-saving and energy-saving man-machine interface, unlike a conventional man-machine interface comprising a combination of a display and a keyboard. It is another object of the present invention to provide a man-machine interface that can display a large image with low power without using a display and does not occupy the desk with the display. It is another object of the present invention to provide a man-machine interface that can input images very easily without using another interface such as a scanner.
[0009]
[Means for Solving the Problems]
Therefore, according to the present invention, there is provided a man-machine interface system having an image input / output device and a pointing device having a pointing end, and the image input / output device is an output optical device capable of projecting an image on a projection surface. And a pointing device is an optical signal that outputs an optical signal that can be acquired by the input optical system near the pointing end. A man-machine interface system including an output unit is provided.
[0010]
In this man-machine interface system, for example, a desk, a floor, the surface of a bag, a wall, or the like is used as a projection surface, and information is output by projecting an image onto the projection surface. Therefore, a display as an object is unnecessary, and it is only necessary to secure one of the surfaces for projecting an image in order to view information. Further, the projector can save energy because the area of the light emitting surface can be reduced with respect to the projection surface. Furthermore, even when a large screen is required to output information, the output optical system can be compact because it can be enlarged and projected. Furthermore, by providing an input optical system, it can also be used as a space for inputting information by operating a pointing device in a space where an image is projected. In addition, if an image to be input is placed in a space where the image is projected, the image can be captured from the input optical system. Therefore, by adopting a man-machine interface system including this projector-type image input / output device and a light-emitting type pointing device, it is possible to provide a man-machine interface capable of capturing images while saving energy and space.
[0011]
One effective form of a projector for installation on a desk is one in which a first housing containing an optical system and a second housing containing electricity, particularly a power supply device, are connected by an arm or the like. It is. That is, a first housing containing at least an output optical system including a light source, a light valve that modulates light emitted from the light source, and a projection lens that projects the modulated light, and at least a power supply device An image output device having the second housing and means for connecting the first housing and the second housing at a distance is one form of a projector suitable for the man-machine interface system described above. .
[0012]
To add an image input function, a third housing containing an input optical system capable of acquiring an optical signal from the range of an image projected by the output optical system is attached adjacent to the first housing. it can. Further, an input optical system capable of acquiring an optical signal from the range of the image projected by the output optical system may be housed in the first housing. The input optical system may include an objective lens for image input set so that an optical signal output from the range of the image projected by the output optical system can be input. In addition, the projection lens of the output optical system can be used as an objective lens. By sharing the lens, both space and cost can be saved, and an optical signal can be detected from a projection lens adjusted to project an image. Therefore, there is also an advantage that it is not necessary to separately provide an objective lens for adjustment.
[0013]
The output input optical system sharing the projection lens includes an imaging device and an optical branching unit that is installed between the projection lens and the light valve and guides an optical signal input from the projection lens to the imaging device. The optical branching unit can use a wavelength separation type optical element such as a dichroic mirror if the optical signal to be input is outside the visible light region such as infrared. If the polarization direction of the projected image can be controlled, a beam splitter that branches light according to the polarization direction can be used as the light branching unit. Furthermore, if the timing for projecting an image and the timing for inputting an optical signal are different, a shutter or the like capable of time-dividing light can be used as the light branching means. Further, if a sufficient amount of light from the light source can be secured, the half mirror can be used as the light branching means.
[0014]
A projector-type image output device or image input / output device suitable for installation on a desk is a type in which a first housing is supported by an arm based on a second housing. Therefore, the output optical system only needs to be capable of projecting an image on the surface on which the second housing is placed. In addition, it is effective for the second housing to include means for fixing the second housing on the desk, for example, a clamp, a suction cup, and the like. Further, the second housing is preferably in a form in which the position of the second housing is difficult to shift, for example, in a form having a low center of gravity such as a flat box shape and the first housing being easily balanced.
[0015]
From the viewpoint of supporting the first housing via the arm by the second housing, it is desirable that the second housing is as heavy as possible and the first housing is as light as possible. Therefore, the image output device or the image input / output device can be equipped with various functions in the second housing. Conversely, it is desirable to mount functions other than the optical system as much as possible in the second housing. For example, the second housing can accommodate a control system that controls the output optical system. The second housing can accommodate a video interface for receiving a video signal output from the light valve from the outside. The video signal may be wired or wireless.
[0016]
Further, the second housing can house a television tuner that receives a television image signal output from the light valve. The second housing can accommodate a speaker, an acoustic amplifier, and an audio interface that receives an acoustic signal output from the speaker from the outside. In the image input / output device of the present invention in which the first and second housings are connected by an arm, increasing the function housed in the first housing leads to unstable installation of the device, It is preferable to store various functions in the second housing, as long as space on the desk permits, the installation status of the apparatus becomes stable.
[0017]
The image output device and the image input / output device of the present invention project an image on a desk. However, by changing the orientation of the first housing, an image can be projected on a wall or a ceiling. Therefore, it can be used as a personal man-machine interface for working on a desk and as it is as an interface for meetings and presentations. When the first housing is rotatably supported by the arm so that the orientation of the first housing can be changed, the projected image may be reversed, so the output optical system is reversed. It is desirable that an image can be projected.
[0018]
Furthermore, by providing the output optical system with a function of projecting an image in which the trapezoidal distortion is corrected, the first housing can be arranged at one corner of the desk to display an image of a predetermined size on the desk. . Therefore, the position of the first housing does not need to be directly above the desk, and the range in which the first housing is supported by the arm can be expanded. Therefore, the man-machine interface system of the present invention can be constructed even with an image input / output device of a simple form that supports the first housing on the second housing.
[0019]
In addition, since the image output device or the image input / output device of the present invention is installed on a desk, the first or second housing modulates white light output from the light source so as to function as a stand. It is desirable that a changeover switch for selecting an illumination mode to be output from the projection lens without being stored. Moreover, projecting white light is also effective when an image placed on a desk is illuminated and an image is input via an input optical system.
[0020]
As described above, since the first housing is supported by the arm, it is desirable that the first housing be as compact and lightweight as possible. One solution of such an optical system is to use a solid state light emitting device such as an LED as a light source. A light source using a solid state light emitting device not only becomes compact, but also has advantages such as low energy consumption, long life, and easy maintenance.
[0021]
The first housing and the second housing are connected to each other by wire, and an image signal and the like can be exchanged between the device housed in the second housing and the optical system housed in the first housing. Further, the optical system housed in the first housing and the control system housed in the second housing may be connected by a wireless interface such as Bluetooth or wireless LAN. Furthermore, it is possible to input and output image information wirelessly from other personal computers, AV devices, and the like, not limited to the control device housed in the second housing by the wireless interface housed in the first housing. .
[0022]
On the other hand, in a pointing device that forms a man-machine interface in combination with such an image input / output device, it is desirable that the optical signal output unit output light having a wavelength outside the visible light region. This is because it becomes easy to branch the light output from the input optical system of the image input / output device and the input light. One type of such a pointing device is a pen-type pointing device, and the tip is the pointing end, and information is input by pointing to the projected area of the image on the projection surface, such as on a desk. can do.
[0023]
The pointing device preferably includes at least one input control switch capable of modulating the optical signal output from the optical signal output unit. Thus, a signal corresponding to pressing the right button or the left button of the mouse can be input to a personal computer or the like via the input optical system of the image input / output device. For example, the input control switch may modulate a pulse of the optical signal or may change the wavelength of the optical signal. These input control switches are desirably arranged on the tip side of the pointing device so as to be easily operated when a pen-type pointing device is gripped, for example.
[0024]
The pointing device can be used to input characters and the like, and the pointing device is reflected on the image projected from the output optical system by reflecting the position or locus of the pointing device detected via the input optical system. Can input characters and images. This function can be provided as a program having instructions capable of executing the above-described processing, and can be used by being installed in a control part housed in the second housing, a personal computer serving as a host device, or the like.
[0025]
There is also a program having a command for executing processing for correcting trapezoidal distortion of the output optical system when it is detected via the input optical system that the corner of the image projected by the output optical system is dragged by the pointing device. It is effective, and trapezoidal distortion can be easily corrected with a pointing device. The program is provided by being recorded on an appropriate recording medium such as a CD-ROM, and can be installed and used in a personal computer or a control unit in the second housing.
[0026]
Furthermore, in the man-machine interface system of the present invention, the surface of the desk can be used as a screen. However, in order to display an image clearly, an appropriate paper or cloth is used as a screen on the desk. It is desirable to keep it. Furthermore, by installing a screen that can be installed on a desk and reflects the projection light of the output optical system in the direction of the eyes of the user of the desk, the light output from the output optical system is reduced. A clear image can be seen even if it is weak. As a result, a bright image can be obtained even if the intensity of the light source housed in the first housing is weakened, so that a compact and energy saving image input / output device can be used. In addition, since images hardly leak in directions other than the user, an excellent system as a personal man-machine interface can be provided.
[0027]
As such a screen, there is a screen having a minute inclined reflection structure that is inclined so as to reflect the projection light from the exit pupil of the projection lens of the output optical system toward the eye of the desk user. Furthermore, a screen is effective in which the back surface is a reflecting surface having a minute inclined reflecting structure and the front surface is a diffusing surface having fine irregularities so that a clear image is formed.
[0028]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 shows an outline of a man-machine interface system 1 of the present invention. In the man-machine interface system 1, an image 9 is displayed on a screen 7 placed on a desk 2 by a projector-like image input / output device 5, and a user 3 inputs information using a pen-type pointing device 6. First, an image input / output system or an image input / output device 5 included in the man-machine interface system 1 is a stand type, and a base unit (base) 20 installed on a desk 2 and an arm from the base 20. The head unit (head) 10 supported by 19 is provided. An optical system (output optical system) for outputting an image and an optical system (input optical system) for acquiring an optical signal are housed in a housing (first housing) 11 constituting the head 10. It is compact and lightweight.
[0029]
A housing (second housing) 21 constituting the base 20 includes a power supply device to which a household outlet 22 is connected, an interface connectable to a personal computer 8 serving as a host device, a television antenna 23, a video 24, and a CD. An interface that can be connected to peripheral devices including audio visual (AV) devices such as the player 25 is stored. Of course, the peripheral device is not limited to wired communication, and data may be exchanged wirelessly. For example, by incorporating a Bluetooth interface, data can be exchanged wirelessly with the AV device 26 incorporating the Bluetooth interface. Furthermore, a speaker 27 is attached to the base 10 so that an audio signal from the personal computer 8 and music from the AV apparatus can be output. The base 20 is provided with switches 28 for setting the timing for using the head 10 as a lamp or placing an image such as a photograph on the screen 7 for input.
[0030]
The man-machine interface system 1 outputs information by projecting an image 9 onto a projection surface on the desk 2. Therefore, unlike a man-machine interface composed of a combination of a conventional display and a keyboard, a display as an object is unnecessary, and only a space on the desk needs to be opened for viewing information when viewing information. In addition, the image input / output device 5 has a projector-like configuration and is very small and compact with respect to the size of the output image 9 in order to enlarge and project an image onto the space above the desk 2 that is the projection surface. The head 10 of the image input / output device 5 that projects the image 9 is connected to the base 20 on the desk 2 at a distance by an arm 19 and is supported by the arm 19 and floats in the air. . Therefore, the head 10 does not occupy the space of the desk 2, and the space on the desk 2 is not occupied by the device for displaying the image until it is necessary to display the image. For this reason, the space on the desk 2 can be maximally used for various jobs.
[0031]
In the interface system 1 of this example, if the space on the desk 2 is used for a job that does not require image input / output, the projected image from the head 10 is erased. Therefore, a method of using the liquid crystal display in a state in which the liquid crystal display is always turned on as in the conventional man-machine interface is not usually employed. For this reason, the man-machine interface system 1 of this example can suppress useless power consumption also in the operation. Of course, when the desk 2 is widely used for jobs that do not require image display, it is possible to always project a small screen to one corner of the desk 2, and the small screen is displayed on the base 20 of the image input / output device 2. By displaying it, it is possible to always check the presence or absence of mail.
[0032]
In the interface system 1, the optical signal 41 is output from the pen-type pointing device 6, and the head 10 of the image input / output device 5 detects the optical signal 41 to input data. Therefore, data can be input by moving the pointing device 6 within a range in which the image input / output device 5 is projecting an image, and drawing characters, pictures, figures, etc., and a keyboard and digitizer for inputting data are unnecessary. . Therefore, also in this respect, the space on the desk 2 can be utilized to the maximum for various jobs. Furthermore, since information can be input by writing characters or pictures with the pointing device 6 on the image projected on the desk 2 by the image input / output device 5, the same interval as when working toward a notebook or document. Thus, work using the personal computer 8 can be performed. Therefore, users who are not familiar with the keyboard with a natural posture can use the computer easily. Of course, a man-machine interface system in which the image input / output device 5 of the present example is combined with a keyboard and a digitizer can be provided, and such a man-machine interface is also included in the present invention.
[0033]
Furthermore, in the interface system 1 of this example, if a document or a document on which an image is drawn is set in a range where the image 9 is projected, it can be read by the image input / output device 5. Therefore, a scanner is not necessary, and the man-machine interface system is suitable for recent jobs with many opportunities to input and output images.
[0034]
FIG. 2 shows a schematic configuration inside the image input / output device 5. Inside the housing 11 of the head 10, an output optical system 12 for projecting an image 9 onto a screen 7 placed on the desk 2 and an input for inputting an optical signal output from the desk 2. An optical system 13 is built in. The output optical system 12 includes a light source 15, a light valve 16 that modulates light emitted from the light source 15, and a projection lens 17 that projects the modulated light. An LED light source is employed as the light source 15, and at least three types of LED elements 15a that repeatedly output light of the three primary colors are arranged in order. Although not limited to the LED element 15a, solid light emitting elements such as a semiconductor laser and an organic EL have high light conversion efficiency, and a large amount of light is output with respect to power consumption. Therefore, since the output optical system 12 capable of displaying a compact and bright image can be configured, the light source is suitable for the stand-type image input / output device 5 of this example.
[0035]
Furthermore, since the light source 15 employing a solid light emitting element is highly efficient, there is an advantage that heat generation is small and a heat radiating fan is unnecessary. Therefore, the power consumption of an ordinary CRT monitor is about 150 W to 200 W, and the backlight type liquid crystal panel is 15 W to 20 W, whereas the image input / output device 5 of this example uses the power consumption for image display. Can be reduced to about 5 W. Thus, the image input / output device 5 employed in the man-machine interface system 1 of this example is compact and has low power consumption.
[0036]
In this example, a tilt mirror device in which micromirrors are arranged in a matrix is used as the light valve 16 that generates display light 42 by modulating the light output from the light source 15 based on image data. The tilt mirror device 16 is a reflective device that turns on and off light in units of pixels according to the angle of the micromirror, and can display a clear image with low light absorption and high contrast. Therefore, although it is highly efficient, the output light amount is not so large, and a light valve suitable for constituting a head 10 that displays a compact, bright and clear image in combination with the light source 15 using a solid light emitting element. It can be said that. As the tilt mirror device 16, a Texas Instruments digital micromirror device (DMD) can be used.
[0037]
An evanescent device that switches using evanescent light also has a high contrast and is suitable for the image input / output device 5 of this example. Of course, some absorption loss occurs, but a reflective or transmissive liquid crystal device can also be used as the light valve.
[0038]
Since the projection lens 17 that projects the image modulated by the light valve 16 onto the screen 7 on the desk 2 is often only about 1 m from the head 10 to the surface of the desk 2, an image is clearly formed within that range. It should be possible. Further, as shown by a broken line in FIG. 1, it is desirable to project a large image 9a on a wall, ceiling, or common screen by rotating the head 10, and a projector lens or lens system 17 having a zoom function with an appropriate magnification. It is desirable to be.
[0039]
The input optical system 13 built in the housing 11 of the head 10 is an optical system that shares the projection lens 17 as an objective lens, and includes an imaging device 31 such as a CCD or CMOS sensor, the projection lens 17, the light valve 16, and the like. And an optical branching means 32 for guiding the optical signal 41 input from the projection lens 17 to the imaging device 31. By using the projection lens 17 of the output optical system 12 as the objective lens of the input optical system 13, one lens system can be omitted, so that both the space and cost of the head 10 can be saved. Further, since the optical signal 41 can be detected from the projection lens 17 adjusted to project the image 9 on the desk 2, there is no need to separately provide and adjust an objective lens. Further, since the projection lens 17 is used as an objective lens of the image input optical system 13, no parallax is generated, so that the projection image projected by the output optical system 12 and the image captured by the input optical system 13 can be accurately obtained. The input information can be interpreted with high accuracy and reflected in the projected image. Of course, it is possible to provide an input objective lens separately from the projection lens 17, and the input optical system 13 can be made independent from the output optical system 12 and optimally designed. In addition, an optical branching unit, which will be described later, is not necessary, and a low-cost image input / output device can be provided.
[0040]
The light branching means 32 employed in the input optical system 13 of this example is a dichroic mirror that can separate light according to wavelength. An optical signal is emitted from the pen-type pointing device 6 by infrared rays outside the visible light region. Although ultraviolet rays may be used, it is desirable to use infrared rays because they have a greater influence than infrared rays when they enter the user's eyes. Therefore, the dichroic mirror 32 of the present example has characteristics of transmitting visible light and reflecting infrared light, and an infrared light signal 41 input through the projection lens 17 is transmitted to the projection lens 17 and the light valve 16. Is detected by an image pickup device 31 using a CCD or CMOS sensor that can detect infrared rays.
[0041]
The light branching means 32 is not limited to a wavelength separation type optical element such as a dichroic mirror. If a filter for controlling the polarization direction of the image projected on the output optical system 12 can be inserted, the light branching means 32 has a different polarization. A beam splitter that separates light as incident light (optical signal) 41 can be employed. In addition, if the control method is to set a time space for detecting the input optical signal 41 during the frame interval for projecting the image 9, the incident in the direction of the image sensor 31 in accordance with the timing. It is also possible to employ an optical element such as a shutter or a tilt mirror that can reflect light as the light branching means. Furthermore, if the light amount of the light source 15 can be sufficiently secured, a half mirror inclined so as to reflect incident light in the direction of the image sensor 31 can be employed as the light branching means 32.
[0042]
Furthermore, the head 10 of this example incorporates a control unit 35 that receives and distributes signals to the light source 15 and the light valve 16 of the output optical system 12 from the base 20. The control unit 35 further has a function of transferring the optical signal 41 detected by the input optical system 13 or data obtained by decoding the optical signal 41 to the base 20. Further, the head 10 is provided with an orientation determining means 34 such as a gyro capable of detecting the orientation of the head 10, and the control unit 35 projects the image 9 on the desk 2. If it is determined that it is necessary to invert the image when the image 9a is projected on a wall, ceiling, or the like, a function of inverting the vertical direction of the image data supplied to the light valve 16 is provided. The function of inverting the top and bottom or the top and bottom may be supported by the base 20 or the computer 8 that is a host.
[0043]
The base 20 having a function of supporting the head 10 incorporating such an optical system via the arm 19 is desirably different from the head 10 that is most required to be light in weight. Further, although the space that occupies the desk 2 has never been small, in order to stably support the head 10 within a certain range, the shape of the housing 21 of the base 20 is not easily displaced, for example, a flat box It is desirable that the shape has a low center of gravity such as a mold. In the image input / output device 5 of this example, most of the functions except for the optical system are incorporated in the housing 21 of the base 20 so that the base 20 is as heavy as possible and the head 10 is as light as possible. The head 10 and the base 20 are connected by a cable 36 to transfer power, control signals, image data, and the like. Instead of transmitting an electrical signal through the cable 36, an optical signal may be transmitted through an optical fiber. The base 20 and the head 10 have functions as a wireless interface such as a wireless LAN and Bluetooth, and data and control are performed by the wireless 37. A signal may be transmitted.
[0044]
First, the base 20 incorporates a power device 51 having the largest weight. The power device 51 of this example also has a function as an adapter that accepts an alternating current from a household outlet 22 and converts it into a direct current, but it may have a built-in battery. Next, the base 20 houses a video interface 52 that receives a video signal (monitor signal) φ2 output from the output optical system 12 of the head 10 from the personal computer 8. The video interface 52 has a function of controlling the output optical system 12 as a whole, a function of controlling the timing at which the LEDs 15a of the respective colors of the light source 15 are turned on, a function of generating image data to be output by the light valve 16, and the like. It has. The video interface 52 also has a function of decoding an optical signal detected by the input optical system 13 and outputting it as a signal φ1 that can be interpreted by the personal computer 8. The function of controlling the input optical system 12 and the output optical system 13 can be supported by the control unit 15 of the head 10, and there is an advantage that it is not necessary to supply a large amount of data from the base 20 to the head 10. However, since the head 10 may be large and heavy, it is desirable to store it in the base 20 as much as possible.
[0045]
Also, the base 20 includes a TV tuner 53 that can receive a signal from the TV antenna 23 and convert it into an image signal, and an audio interface that can receive a signal from an external AV device and separate it into an image and a sound signal and output them. A multimedia interface 54 that also functions as an audio amplifier 55 and an audio amplifier 55 that drives the speaker 27 are incorporated. Further, a local operation panel 56 including a switch 28 capable of switching the mode of the image input / output device 5 is also provided. The image input / output device 5 projects image data provided from an application 81 such as a word processor running on the personal computer 8 onto the screen 7 on the desk, and uses the data obtained from the pointing device 6 as input to the application 81. In addition to the normal interface mode that feeds back to or provides the image input / output device 8, the image input / output device 5 of this example can be switched between the illumination mode and the scan mode. A driver program 85 is installed in the personal computer 8 so that the image input / output apparatus 5 of this example can be controlled from an application program 81 running on the personal computer 8, and the pointing device 6 operates in a normal interface mode. Feedback is made to the image data supplied from the personal computer 8, and the locus of the pointing device 6 is projected onto the desk 2.
[0046]
When the switch 28 switches to the illumination mode, the light source 15 of the head 10 simultaneously turns on the LEDs 15a of the respective colors and outputs white light, and all the light output from the LED 15a passes through the projection lens 17 without being modulated by the light valve 16. And projected onto the desk 2. Therefore, the top of the desk 2 can be brightly illuminated and functions as a table lamp. When the switch 28 is switched to the scan mode, the document or image placed on the desk 2 is read via the input optical system 13 of the head 10 and the image is operated on the OS 83 of the personal computer 8. The data is supplied to the application program 82 for processing. When capturing an image, if the image is a monochrome image or a document, each color LED 15a may be turned on simultaneously to apply white illumination to the image. On the other hand, in the case of a multi-color original, it is also possible to take in the data of each color in order by blinking the LEDs 15a for each color of RGB in a time sequential manner, and to take in a color image with a triple resolution.
[0047]
As described above, the image input / output device 5 of the present example functions as an important component of the man-machine interface 1 that displays an image on the desk 2 and acquires an optical signal from the pointing device 6. It also functions as a scanner. Furthermore, by placing the image input / output device 5 of this example that functions as a television or audiovisual terminal and occupies almost no space on the desk 2, various jobs can be performed efficiently, You can enjoy entertainment. Further, by rotating the head 10, the image 9a can be projected not only on the desk 2, but also on the wall or the ceiling. Therefore, the personal man-machine interface system 1 that performs work on the desk 2 can be constructed, and can be used as it is as an interface system for meetings and presentations.
[0048]
FIG. 3 shows a schematic configuration of a pointing device 6 suitable for the interface system of this example. The pointing device 6 is a pen type, and the tip 61a serves as an indicating end, and information can be input using the same as a pen. That is, the pen-type pointing device 6 of the present example includes a battery 62 as a power source inside a pen-shaped housing 61, and an LED 63 for outputting an infrared light signal 41 is provided at the tip 61a of the housing 61. Is arranged. The pointing device 6 is different from the LED 63 in that the switch 64 and the operation unit 65 for turning on and off the signal from the battery 62 to the LED 63, the pressure sensor 66 that detects the writing pressure at the tip 61a of the housing 61, and turns on the LED 63. Input control switches 67a and 67b for outputting periodic pulse signals, and a control unit 68 for controlling lighting of the LED 63 based on operations of the pressure sensor 66 and the input control switches 67a and 67b are provided. Therefore, in the pointing device 6 of this example, when the pressure sensor 66 is pressed against the upper surface of the screen 7 or the desk 2 to detect the writing pressure, the LED 63 is turned on, and when the first input control switch 67a is pressed, the LED 63 is the first. When the second input switch 67b is pressed, the LED 63 is lit at the second period.
[0049]
For this reason, in the man-machine interface system 1 including the image input device 5 and the pointing device 6, when characters or figures are drawn by the pointing device 6, the LED 63 is lit and lighted only when the pen pressure is detected. Since the signal 41 is output, the optical signal 41 traces the movement of the tip 61a of the pointing device 6 and outputs characters and figures. The input optical system 13 built in the head 10 of the image input / output device 5 acquires the optical signal 41, traces the movement of the tip 61a of the pointing device 6, and generates input data corresponding to the movement of the pointing device 6. To do. Then, input data corresponding to the movement of the pointing device 6 is supplied from the image input / output device 5 to the application 81 running on the personal computer 8 via the driver 85. The generation of the optical signal obtained by the input optical system 13 of the head 10 may be performed by the image input / output device 5, or may be performed by the driver 85 using the resources of the computer 8.
[0050]
When the first or second input control switch 67 a or 67 b of the pointing device 6 is pressed, a pulse signal having a frequency corresponding to the first or second input control switch 67 a or 67 b is output as the optical signal 41. Therefore, the first input control switch 67a or the second input control switch on the image input / output device 5 side is obtained by capturing the optical signals 41 having different periods with the input optical system 13 of the head 10 of the image input / output device 5. It can be determined whether or not 67b has been pressed. Therefore, by assigning the first and second input control switches 67a and 67b as right click and left click of the mouse type pointing device, the same operation environment or service as that of the mouse type pointing device can be provided. It can be provided by an example pen-type pointing device 6. The first and second input control switches 67a and 67b are preferably arranged at a position close to the front end 61a of the housing so that the pen-type pointing device 6 is easily handled.
[0051]
Further, a switch functioning as a scroll dial can be provided in the pointing device 6, and the pulse width is changed according to the scroll direction, and the movement of the scroll dial is input to the personal computer 8 wirelessly via the image input / output device 5. can do.
[0052]
A pointing device 69 shown in FIG. 4 is a different example of the pointing device according to the present invention. In the pointing device 69, two types of LEDs 63a and 63b that emit optical signals 41a and 41b having different wavelengths are attached to the front end 61a of the housing. Therefore, when the pressure sensor 66 detects the writing pressure, the two LEDs 63a and 63b are turned on simultaneously, and the image input / output device 5 can detect and trace that a character or a figure is drawn by the pointing device 6. When the first input control switch 67a is pressed, only one LED 63a is lit, and when the second input control switch 67b is pressed, only the other LED 63b is lit. Therefore, by outputting the optical signals 41a and 41b having different wavelengths as described above, it is possible to provide the same operating environment as that of the pointing device 6 described above, and it is possible to input characters and images wirelessly. A pointing device that can also be left-clicked can be provided.
[0053]
Instead of outputting the optical signal 41, data can also be input by inputting a signal from the pointing device to the base 20 of the image input / output device 5 or the personal computer 8 in a wired manner. With such a man-machine interface, the input optical system 13 is not necessary, and an interface system including such an image output apparatus is also included in the present invention. However, a non-wired pointing device is easier to operate because the cable does not get in the way. Further, the screen 7 for projecting the image 9 may be provided with a function as a digitizer, and the pen pressure may be detected on the screen 7 side to input data. Such a man-machine interface is also included in the present invention. It is. However, it is difficult to identify right click or left click through the digitizer, and the function is inferior to that of the man-machine interface system 1 described above.
[0054]
FIG. 5 is a flowchart showing a process of selecting processing based on the optical signal 41 from the pointing device 6 in the driver program 85 of the image input / output device 5. First, in step 91, when the input signal φ1 indicating that the optical signal 41 output by the writing pressure is detected is received from the image input / output device 5, in step 92, the position where the optical signal 41 is output is determined as the driver program 85. Is reflected in the image signal φ 2 supplied to the image input / output device 5. As a result, the position or trajectory of the pointing device 6 is fed back to the image 9 projected from the image input / output device 5 onto the desk 2, and it is as if a character or picture is directly written on the screen with a pen. The machine interface system 1 can be used.
[0055]
In step 93, when the input signal φ1 indicating that the optical signal 41 when the first input control switch 67a is pressed is received from the image input / output device 5, in step 94, the first input control switch is previously set. The first function assigned to 67a is executed. For example, by giving a function as an eraser, it is possible to erase the character or picture input in step 92. In step 95, when the input signal φ1 indicating that the optical signal 41 when the second input control switch 67b is pressed is received from the image input / output device 5, in step 96, the second input is input in advance. The second function assigned to the control switch 67b is executed.
[0056]
A driver program or a program product having these functions can be provided by being recorded on an appropriate recording medium such as a CD-ROM that can be read by the computer together with the image input / output device 5 or separately. It is also possible to distribute via a computer network such as the Internet. In this example, an example in which processing for capturing and decoding the optical signal 41 from the pointing device 6 is provided as a part of the function of the driver program 85 that is installed or downloaded in the personal computer 8 and other computers is executed. Although shown, it is also possible to perform equivalent processing on the image input / output device 5 side. For example, since the video interface board 52 accommodated in the base 20 of the image input / output device 5 is often equipped with a CPU of an appropriate scale, it is equipped with a program 85x having a function equivalent to the driver program. Thus, the above processing can be executed inside the image input / output device 5.
[0057]
Further, it is desirable that the driver program 85 can execute processing for correcting trapezoidal distortion. FIG. 6 shows how the trapezoidal distortion is corrected. When a base 20 of the image input / output device 5 is installed at a corner of the desk 2 and an image 9 is projected onto the surface of the desk 2 from the head 10 supported thereon, the image is projected onto the desk 2 from the projection lens 17 of the head 10. The corner 9c of the image 9 has four corners and different distances from the projection lens 17. Therefore, even if the image formed by the light valve 16 is rectangular, the image actually projected on the desk has four corners and the enlargement ratios are different, so that the shape is distorted into a trapezoidal shape and a highly accurate image cannot be displayed. Therefore, the driver program 85 of this example includes a mode for correcting trapezoidal distortion.
[0058]
FIG. 7 shows a process for correcting the trapezoidal distortion. First, when a mode for correcting trapezoidal distortion is selected in step 101, it is detected in step 102 that the corner 9c of the image 9 has been selected by the pointing device 7. For example, the driver program 85 recognizes the position of the tip of the pointing device 6 at the time when the first input control switch 67 a is pressed as the corner 9 c of the image 9. In step 103, when it is detected via the input optical system 13 that the corner 9c of the projected image 9 has been dragged by the pointing device 7, the image 9 displayed on the light valve 16 at the dragged position. And keep the setting. In the present embodiment, this dragging means that the pointing device 6 is moved while the second input control switch 67b is kept pressed. Accordingly, the user can change the setting of the driver program 85 so as to eliminate the trapezoidal distortion from the direction in which the user views while viewing the actually projected image 9. The process for eliminating the trapezoidal distortion can be performed not on the personal computer 8 side but on the image input / output device 5 side as described above. For example, the program 85x for controlling the video interface 52 can have the function.
[0059]
In any case, by providing a function that can easily correct the trapezoidal distortion, the position where the head 10 can be set becomes very flexible. Therefore, it is extremely easy to place the image input / output device 5 at one corner of the desk 2 and set the center of the desk 2 as an area for displaying the image 9 and inputting data. Of course, the image input / output device 5 can be installed at the right end or the left end of the desk 2. However, in the case of a right-handed user, if the image input / output device 5 is arranged on the right side, a shadow of a hand appears in the projected image 9, so it is desirable to install the image input / output device 5 on the left side of the desk. Even in that case, since the image 9 is projected on the back of the hand or the pointing device 6, the pointing device 6 does not absorb black light or the like, but the white device that appropriately scatters light like the screen 7. Is preferred. It is also effective to wear white gloves if you want to actively see the image projected on the back of your hand.
[0060]
In the man-machine interface system 1 of this example, the top of the desk 2 is used as an image display area. Therefore, if the surface of the desk 2 is a diffusion surface close to white in which minute irregularities are appropriately formed, the image 9 can be projected directly onto the surface of the desk 2 without installing a screen. When the surface of the desk 2 is not suitable as a screen, a clear image can be displayed by laying the screen 7 on the desk 2. Further, as shown in FIG. 8A, if the screen 7 reflects the projection light 42 output from the output optical system 12 of the image input / output device 5 in the direction of the eyes 3 a of the user 3 of the desk 2. Even when the light 42 output from the output optical system 12 is weak, a clear image 9 can be seen. Therefore, it is effective to combine with the image input / output device 5 of this example employing a solid light emitting element such as an LED as the light source 15, and further, a man-machine interface system 1 capable of displaying a bright image with energy saving can be configured. Furthermore, since the image 9 hardly leaks in directions other than the user, privacy can be easily maintained, and the system 1 excellent as a personal man-machine interface can be provided.
[0061]
As such a screen 7, as shown in an enlarged view in FIG. 8B, the projection light 42 from the exit pupil of the projection lens 17 of the output optical system 12 is directed to the eye 3 a of the user 3 of the desk 2. In some cases, the back surface 72 is provided with a minute inclined reflecting structure 73 inclined so as to be reflected. The screen 7 has a surface 71 as a diffusing surface with fine irregularities, and can form a clear color image 9 on the surface 71, and the projection light 42 is directed in the direction of the eye 3 a by the minute inclined reflection structure 73 on the back surface 72. Reflect on. Therefore, the user 3 can see the bright and clear image 9. The surface 71 of the screen 7 is desirably a diffusion sheet in which transparent beads are dispersed, for example. The surface 71 on which fine irregularities are formed is a preferable form in that respect because it is slippery when the pointing device 6 comes into contact therewith. Further, since the pointing device 6 comes into contact, it is desirable that the screen 7, particularly the surface 71, is resistant to wear and is made of a material such as hard polycarbonate. Further, the reflection angle and direction of the minute inclined reflection structure 73 on the back surface 72 are changed depending on the location of the screen 7, so that the exit pupil position of the projection lens 17 and the vicinity of the face of the user 3 are conjugated to each other in a Fresnel reflection lens shape. It is also effective to provide a man-machine interface system 1 suitable for personal use, which is brighter and difficult for others to see images.
[0062]
FIG. 9 shows a different example of the image input / output device. In the image input / output device 5 described above, the base 20 and the head 10 are connected by an arm 19 having a node or joint. However, in the image input / output device 5a, the head 10 and the base 20 have no node or joint. Connected by an arm 19, the head 19 can be rotated at the tip of the arm 19. Therefore, the rotation angle of the head 10 at the tip of the arm 19 is detected by the angle detection sensor 39 such as an encoder and projected onto the desk 2 shown in FIG. 9A, and the case shown in FIG. 9B. By appropriately inverting the image 9 as a case of projecting on the ceiling and a case of projecting on the wall shown in FIG. 9C, a normal image can be projected at any angle.
[0063]
FIG. 10 shows still another example of the image input / output device of the present invention. In this image input / output device 5c, a head 10 for a projector incorporating an output optical system and a head 49 for a digital camera incorporating an input optical system are attached to the tip of an arm 19 extending from a base 20 adjacent to each other. It has been. Accordingly, the projection lens 17 is attached to the head 10 for the projector, and the objective lens 48 is attached to the head 49 for the digital camera, both of which face the same area on the desk 2. Therefore, an image can be displayed on the desk 2 by the head 10 for the projector, and an optical signal from the area where the image is displayed can be received by the head 49 for the digital camera.
[0064]
In addition, a suction cup 45 is attached to the back surface of the base 20 in this example, and the base 20 is adsorbed on the surface of the desk 2 so that the heads 10 and 49 can be stably held. The means for attaching the base 20 to the desk 2 is not limited to the suction cup 45, and it is also possible to adopt a clamp-type attachment that is also used for a stand or the like.
[0065]
As described above, the present invention provides a new man-machine interface system 1 for projecting an image using the desk 2 or the like as a projection surface and a part thereof as an image display area. The projection surface is not limited to the surface on the desk, and if you lie down and work, the floor or the bed will function as the projection surface. Further, when traveling, it is possible to work using the surface of the bag as a projection surface. Furthermore, any surface close to the surface such as a wall or furniture can be used as a projection surface, and an interface system can be constructed regardless of time and place. Then, if a horizontal or near horizontal stable surface such as the desk 2 is used as the projection surface, the image input / output system 5 that projects an image thereon is installed on the projection surface. A self-standing interface system can be constructed by providing the base 20 and the head 10 supported in the air by the arm 19 extending therefrom. Various shapes of the base 20, the arm 19 and the head 10 can be adopted, and the image input / output device of the present invention is arranged as a part of an image-based man-machine interface system or on the desk 2. It can also be provided as one of the functional arts. Furthermore, the form of the base 20 is various, and it can be a part of a bookshelf as well as the one that can be installed on the desk 2 alone, and can be designed to be combined with a container case or a pencil stand. is there. Further, the pointing device combined with the image input apparatus is not limited to the above-described pen type, and may of course be a mouse type or a ball type. However, the pen-type pointing device is superior in that it can be operated at the same interval as in conventional handwriting.
[0066]
【The invention's effect】
As described above, the man-machine interface system of the present invention can acquire an output optical system capable of projecting an image on a desk and an optical signal from a range projected on the desk by the output optical system. An image input / output device including an input optical system, and a pen-type pointing device, and a pointing device including an optical signal output unit that outputs an optical signal that can be acquired by the input optical system near the tip; Information can be output by projecting an image on a desk, and the information can be input by pointing to the image with a pointing device or drawing a character or a figure. Therefore, there is no need for a display as an object, and the space on the desk can be used widely, and information can be obtained from a personal computer and used as if the characters and pictures were directly written on the image with a pen. You can enter information. For this reason, according to the present invention, it is possible to provide a man-machine interface that can be easily used by anyone, and that can make maximum use of the space on the desk with low power consumption.
[Brief description of the drawings]
FIG. 1 is a diagram showing an outline of a man-machine interface of the present invention.
FIG. 2 is a diagram showing an outline of the image input / output device shown in FIG. 2;
FIG. 3 is a diagram showing an outline of the pointing device shown in FIG. 2;
4 is a diagram showing an outline of a pointing device different from FIG. 3; FIG.
FIG. 5 is a flowchart illustrating an example of processing of data input by a pointing device.
FIG. 6 is a diagram showing how to correct trapezoidal distortion.
FIG. 7 is a flowchart showing processing for correcting trapezoidal distortion.
FIG. 8 is a diagram showing an outline of a screen.
FIG. 9 is a diagram showing an outline of an image input / output device different from FIG. 2;
FIG. 10 is a diagram showing an outline of still another image input / output device.
[Explanation of symbols]
1 Man-machine interface system
2 Desk
3 users
5 Image input / output device
6 pointing devices
7 screens
8 Personal computer
9 Projected image
10 Head of image input / output device
11 Head housing
12 Output optical system
13 Input optical system
19 Arm
20 Base part of image input / output device
21 base housing

Claims (13)

An output optical system having a projection lens capable of projecting an image on a projection surface;
An input optical system capable of acquiring an optical signal from a range projected by the output optical system,
An image input / output device using the projection lens as an objective lens of the input optical system ,
The output optical system includes a light source and a light valve that modulates light emitted from the light source,
The projection lens projects the modulated light;
The input optical system has an optical branching unit that is installed between the imaging device, the projection lens, and the light valve, and branches the optical signal toward the imaging device.
The image input / output apparatus, wherein the optical signal is directly guided to the imaging device. A first housing containing at least the output optical system and the input optical system;
A second housing containing at least a power supply device;
The image input / output device according to claim 1, further comprising means for connecting the first and second housings.   The image input / output device according to claim 2, wherein the connecting means is an arm capable of supporting the first housing with the second housing as a base.   4. The image input / output device according to claim 2, wherein a changeover switch for selecting an illumination mode for outputting white light from the projection lens is housed in the first or second housing. 5. An image input / output device according to any one of claims 1 to 4,
A man-machine interface system comprising: a pointing device having an optical signal output unit that outputs an optical signal that can be acquired by the input optical system in the vicinity of an instruction end. The optical signal output unit outputs light having a wavelength outside the visible light region,
6. The man-machine interface system according to claim 5 , wherein the light branching means is a wavelength separation type optical element capable of separating light according to wavelength. The output optical system has control means for controlling the polarization direction of the projected image,
6. The optical element according to claim 5 , wherein the light branching unit is an optical element that separates light having a different polarization from light polarized by the output optical system out of the optical signal input from the projection lens. The described man-machine interface system. The image input / output device includes control means for controlling a timing of projecting the image by the output optical system and a timing of inputting an optical signal input from the projection lens,
The man-machine interface system according to claim 5 , wherein the optical branching unit is an optical element that guides the optical signal to the imaging device in accordance with a timing at which the optical signal is input by the control unit. Wherein the output optical system, wherein the upper desk is capable projects the image as a projection surface, wherein the pointing device, any claims 5-8, characterized in that the pen type with the instruction end to the distal end A man-machine interface system according to one. The pointing device is man-machine interface system according to any one of the optical signal output unit according to claim the optical signal output comprises at least one input control switch capable modulation from 5-9. A distortion correction function for detecting, via the input optical system, that a corner of an image projected by the output optical system has been dragged by the pointing device, and correcting a trapezoidal distortion of the output optical system is provided. Item 11. The man-machine interface system according to any one of Items 5 to 10 . A screen which can be placed on the desk, Man described projection light of the output optical system, to any one of claims 5-11 which has a screen that reflects the direction of the eye of desk of a user Machine interface system. An image input / output device according to any one of claims 1 to 4,
A pointing device having an optical signal output unit that outputs an optical signal that can be obtained by the input optical system in the vicinity of an instruction end, and a program for controlling a man-machine interface system comprising:
A command capable of detecting, via the input optical system, that the corner of the image projected by the output optical system has been dragged by the pointing device, and correcting the trapezoidal distortion of the output optical system; ,program.

Images