JP2007272927A - Information input/output device and information input/output method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve the enhancement/enlargement of interactive input/output environments. <P>SOLUTION: This information input/output device is provided with an image pickup means for picking up an image including an ID pattern; a storage means for storing two or more pieces of ID pattern information and information relating to display processing corresponding to each ID pattern; a display means for displaying image information; and a control means for separating only the image information of the ID pattern from the image picked up by the image pickup means, and for comparing the image information of the ID pattern with the two or more pieces of ID pattern information stored by the storage means to recognize the ID pattern, and for controlling the display means to perform display processing corresponding to the recognized ID pattern. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an information input / output device and an information input / output method for outputting required information in response to, for example, input information given from the outside.

  For example, in an information input / output device such as a computer device, there is a wide variety of so-called interactive input / output modes in which a computer device side presents a predetermined response in response to a user operation under various application programs. It has been adopted. Specifically, for example, a response to a user operation performed on an input device such as a mouse, a touch panel (including a trackpad) or a keyboard is displayed as a display device such as a CRT (Cathode Ray Tube) or a liquid crystal display. Are displayed as image information.

  A projection display that functions as a computerized whiteboard is also known as an information input / output device. In such a projection display, for example, a user operates the whiteboard using a dedicated infrared light emitting pen.

  In addition, an information input / output device aiming at an interactive effect called “video place” is known. The video place is an artistic device using a video camera, for example. For example, a viewer of a video place causes a video camera to photograph a part of his / her hand or other human body as a silhouette. The viewer can freely move the hand or a part of the human body while looking at the monitor device for any image obtained by combining this captured image with another image, for example, to change the response or change of the image displayed on the monitor device. You can enjoy it.

By the way, when it is considered to realize a further expanded interactive input / output environment, the above-described conventional information input / output mode has the following limitations.
For example, when a case where a touch panel or a track pad is adopted as an information input device in a computer device is considered, the pointing operation for information input is generally limited to a finger. Further, it is not possible to perform an operation in the space on the touch panel, and it is necessary to bring a physical operation body such as a finger into contact with the operation surface on the panel surface. Further, since the touch panel is relatively expensive, it is not suitable as a large operation panel.
Considering a display device as an information output device that outputs information in response to an input operation, for example, in a computer device, a CRT or a liquid crystal display is generally used as described above. For example, when an attempt is made to increase the screen size by using such a display device, it is difficult to realize the cost because the cost increases and the installation conditions become severe.

  In the case of a projection display that functions as a computerized whiteboard, it is easy to increase the size of the operation screen and display screen. However, as described above, a special pointing device such as an infrared light emitting pen is required. Become.

In addition, in the case of a video place, since some kind of interactive operation is realized using the silhouette of the hand or human body, the interface between the input and output is indirect, and it is functional when direct operation is desired. Is insufficient.
As described above, the conventional information input / output devices have various obstacles to strengthen and expand the interactive input / output environment.

  Therefore, in order to solve the above-described problems, the present invention provides an imaging unit that captures an image including an ID pattern, a plurality of ID pattern information, a storage unit that stores information related to display processing corresponding to each ID pattern, and image information. Only the image information of the ID pattern is separated from the image captured by the image capturing means and the display means for displaying the ID pattern, and the ID pattern is recognized by comparing with the plurality of ID pattern information stored by the storage means. And control means for performing display processing corresponding to the ID pattern recognized by the display means.

Further, the display means displays an image picked up by the image pickup means, and the control means further recognizes the position of the recognized ID pattern in the image area picked up by the image pickup means, and the display means The display process is performed based on the position.
Further, the ID pattern is a two-dimensional barcode in which a predetermined pattern is drawn in black on a white background.
Further, the ID pattern is a two-dimensional barcode in which a predetermined pattern made of a plurality of colors is drawn, and the control means is configured to determine the ID based on the color difference of the pattern drawn in the two-dimensional barcode. It was decided to recognize the pattern.

An information input / output method for an information input / output device comprising: storage means for storing a plurality of ID pattern information and information relating to display processing corresponding to each ID pattern; and display means for displaying image information. Recognize an ID pattern by capturing an image including a pattern, separating only the image information of the ID pattern from the image captured by the imaging unit, and comparing it with a plurality of ID pattern information stored by the storage unit Then, the display unit is controlled to perform display processing corresponding to the recognized ID pattern.

As described above, according to the present invention, as long as a state change (the state change is reflected as detected image information) is reflected by reflecting light or electromagnetic waves in a predetermined wavelength band, this is used as input information. It becomes possible to recognize. For this reason, a special pointing device is not required for the input operation, and the input operation can be performed by using, for example, the user's own hand or finger as described in the basic configuration of the present invention. The In addition, since the detection target as input information can be recognized at a position close to the translucent surface (for example, a hollow position in front of the translucent surface), the input operation method also includes input information with respect to the translucent surface that is an operation panel. It is possible to grasp the detected object as input information by positioning it in the space in front of it without contacting it.
Furthermore, by combining a surface that reflects or absorbs light or electromagnetic waves in a predetermined wavelength band, for example, a two-dimensional barcode ID pattern can be recognized, and information output in response to the recognized ID can be performed. It becomes.

  Further, as the image input information obtained based on the detected image information, it is possible to identify the position information when the input operation is given to the translucent surface. It is possible to output the output information based on the identification information by combining the output information based on the position information. That is, for example, as described in the first embodiment, it is possible to display a menu screen corresponding to the position on the translucent surface on which the user has recognized the ID pattern.

  In addition, the translucent surface of the present invention functions as an operation panel and a display panel, so that, for example, display is performed on the translucent eye in response to input information given to the translucent surface. Therefore, a direct input operation is realized. In addition, unlike conventional touch panels and various display devices, the size of the semitransparent surface can be easily increased at a low cost.

  Furthermore, in addition to the image input information obtained based on the detected image information, if a receiving means for receiving transmission data, for example, is provided, in addition to response processing based on mere image input information, More complicated response processing based on this can be executed simultaneously. Therefore, as an example, if it is configured to display an image in response to received data as in the third and sixth embodiments described above, a display such as a portable personal computer or a game machine is displayed. The function can be expanded.

  As described above, the present invention provides the above-described degree of freedom when inputting information, and can easily provide an enlarged input panel that can also be used as a display panel with a very simple configuration. The interactive input / output environment is easily enhanced and expanded.

The information input / output device according to the embodiment of the present invention will be described below. The following description will be given in the following order.
<1. Basic Configuration of Embodiment>
(1-a. Basic Configuration of Information Input / Output Device)
(1-b. Operation for detecting input information)
<2. First Embodiment>
<3. Second Embodiment>
<4. Third Embodiment>
<5. Fourth Embodiment>
<6. Fifth embodiment>
<7. Sixth Embodiment>

<1. Basic Configuration of Embodiment>
(1-a. Basic Configuration of Information Input / Output Device)
First, the basic configuration of the embodiment of the present invention will be described with reference to FIGS. 1 and 2. In the present invention, configurations as first to sixth embodiments to be described later are realized based on a basic configuration of an information input / output device described below.

FIG. 1 conceptually shows the basic configuration of an information input / output device as an embodiment of the present invention.
An information input / output device 1 according to this embodiment includes a translucent surface 2, an infrared light emitting diode (LED) panel 3, a CCD (Charge Coupled Device) camera 4, a projector 5, and a control device 6. Configured. The infrared LED panel 3, the CCD camera 4, and the projector 5 are provided on the back side of the translucent surface 2.

The translucent surface 2 is formed by, for example, attaching a translucent film that looks like a tracing paper to a transparent glass plate, or using a translucent material such as polished glass. The operation panel is formed and has both functions of an operation panel to which input information is given by a user's operation as described later and a display panel on which various response displays according to the input information are performed.
The infrared LED panel 3 is configured by, for example, a large number of infrared LEDs collectively arranged on the panel surface, and the infrared light emitted from the infrared LED is irradiated to the entire back surface of the translucent surface. Provided. The infrared LED is driven by the control device 6 so as to constantly emit infrared light.
The infrared LED panel 3 may be provided with a sufficient number of infrared LEDs so that the emitted infrared light is emitted to the entire translucent surface 2. As will be described later, since the image information reflected from the semitransparent surface 2 side is obtained based on the difference between the current infrared image and the initial infrared image, the entire translucent surface 2 is obtained. It is not necessary that the amount of infrared light to be irradiated should be uniform. Therefore, the size of the infrared LED panel 3 can be much smaller than that of the translucent surface 2.

  The CCD camera 4 is a camera device using a CCD as an image sensor. In this case, only the infrared light component is imaged as image light reflected on the translucent surface 2, so It is provided for recognizing a broken operation as image information. For this reason, an infrared transmission filter 4 a that transmits only the wavelength band of the infrared region is provided for the optical system portion of the CCD camera 4. Further, the arrangement position is set so that the entire translucent surface 2 is included as a composition photographed by the CCD camera 4.

  The projector 5 projects and displays image light based on visible light on the back surface of the translucent surface 2 based on image information supplied from the control device 6. For example, the user can observe the image of the projector 5 projected and displayed on the translucent surface 2 from the front side of the translucent surface 2. Here, the optical system of the projector 5 is provided with an infrared blocking filter 5a that blocks the wavelength in the infrared region. However, the image light projected and displayed on the translucent surface 2 does not include infrared rays. The projection image of the projector 5 becomes invisible from the CCD camera 4.

The control device 6 includes, for example, a microcomputer, obtains image information (video data) from the imaging signal supplied from the CCD camera 4, and further obtains input information based on this image information. Based on this input information, for example, display control relating to an image displayed on the translucent surface 2 by the projector 5 is executed, and various necessary control processes are performed. Further, the infrared LED of the infrared LED panel 3 is driven to emit light.
It should be noted that the arrangement positions of the infrared LED panel 3, the CCD camera 4, and the projector 5 may be set in consideration that the roles to be played by each function sufficiently.

  FIG. 2 is a block diagram showing an example of the internal configuration of the control device 6. In the control device 6 shown in this figure, the LED drive unit 10 is a circuit part for driving the plurality of infrared LEDs provided in the infrared LED panel 3 to emit light.

  The image input unit 11 generates a video signal by performing required signal processing on the imaging signal supplied from the CCD camera 4 and supplies the video signal to the input image processing unit 12. That is, the image input unit 11 outputs infrared light incident from the translucent surface 2 side via the CCD camera 4 as video information.

  For example, the input image processing unit 12 converts the video signal supplied from the image input unit 11 into video signal data based on a digital signal. The input image processing unit 12 executes necessary analysis processing using “image information (for example, video data in units of frames)” obtained based on the video signal data, so that the translucent surface 2 is processed. The input information based on the operation performed is obtained. Here, as the input information obtained based on the image information, for example, the position (coordinates) on the image of the operating body operating the semi-transparent surface 2, the signal level of the image, and the like are used. This input information is transmitted to the database driver 14. The video signal data can also be supplied to the image composition unit 17.

  The threshold control unit 13 sets a threshold necessary for processing related to input information executed by the input image processing unit 12 and transmits the threshold to the input image processing unit 12. In the input image processing unit 12, input information that is sequentially required is identified by executing necessary processing such as analysis of image information using a threshold set in the threshold control unit 13. . Further, in the present embodiment, the current image state (detected image information) of the translucent surface 2 is obtained by calculating the frame difference of the input image data as will be described later. Information such as a reference value (reference image input level) to be used is also stored in the threshold control unit 13 as described later.

  The database driving unit 14 takes in the input information obtained by the input image processing unit 12 and appropriately executes necessary processing based on the input information. At this time, program data necessary for control processing to be executed by the database driving unit 14 is stored in the database memory 15, and the database driving unit 14 performs necessary control processing based on the program data stored in the database memory 15. Will be executed.

The image generation unit 16 generates necessary image data (video signal data based on a digital signal) under the control of the database driving unit 14 and outputs the generated image data to the image synthesis unit 17.
In the image synthesis unit 17, if necessary, the video signal data supplied from the input image processing unit 12 is synthesized with the video signal data supplied from the image generation unit 16 to the RGB signal generation unit 18. Output.
In the RGB signal generation unit 18, the video signal data supplied from the image synthesis unit 17 is converted into, for example, an analog RGB signal and output to the projector 5. As a result, the projector 5 emits and outputs image light based on an image in response to an operation performed on the translucent surface 2 to the translucent surface 2.

(1-b. Operation for detecting input information)
Next, a method for detecting input information in the information input / output device 1 of the present embodiment having the above-described configuration will be described.
As described above, the entire translucent surface 2 shown in FIG. 1 is irradiated with infrared light by the infrared LED panel 3 from the back side. Therefore, not all infrared light passes through the translucent surface 2, but some infrared light is reflected by the translucent action.
In the present embodiment, infrared light reflected by the semitransparent surface 2 is imaged by the CCD camera 4 under the condition that no operation is performed on the translucent surface 2. The initial level of the video signal data obtained in this way is stored as the “reference input image level”. The reference input image level may be determined by detecting the signal level for each pixel in one frame based on the input video signal data. This detection process is performed by the input image processing unit 12. The information on the reference input image level detected in this way is transmitted to the threshold detection unit 13 and held there.

The reference input image level detection process is, for example, as shown in the flowchart of FIG. As shown in this figure, first, in the input image processing unit 12, in step S101, based on the image data for one frame obtained from the video signal supplied from the CCD camera 4 through the image input unit 11, the above-described image data is input. In this way, the signal level is detected for each pixel, and the detection result is obtained as the reference input image level Lint. Specifically, it is conceivable that the level of the luminance signal component for each pixel is detected and used as the reference input image level Lint.
In the subsequent step S102, the input image processing unit 12 performs processing so as to transmit the reference input image level Lint to the threshold value control unit 13 for storage.

  Note that the processing (the processing operation shown in FIG. 3) for detecting the reference input image level Lint and storing it in the threshold control unit 13 is executed, for example, when the information input / output device is turned on, or somehow It is conceivable that the reference input image level Lint is updated when necessary according to a user instruction.

Image information handled as input information is obtained as follows under the condition that the information of the reference input image level Lint is held as described above.
FIG. 4 is a flowchart showing the processing operation of the input image processing unit 12 for obtaining image information (hereinafter referred to as “detected image information” in particular). In this case, the input image processing unit 12 first executes a process of detecting the current input image level Lprs in step S201. The input image level Lprs here is data in units of frames for the image of the translucent surface 2 based on infrared light currently captured by the CCD camera 4, and the signal level for each pixel in the image data in units of frames. This is information obtained by detecting.
Subsequently, in step S202, the input image processing unit 12 calculates a difference input image level L by calculating a difference between the reference input image level Lint and the current input image level Lprs (L = Lprs−Lint). Specifically, the difference input image level L is obtained by obtaining a difference between the reference input image level Lint and the data value obtained as the input image level Lprs for each pixel at the same position. Therefore, as the difference input image level L, a signal level difference between the current input image level Lprs and the reference input image level Lint is always obtained for each pixel. Then, the input image processing unit 12 proceeds to step S203 so as to generate current detected image information (video data having a level information for each pixel in units of frames) based on the difference input image level L. Is done.

The detection operation of the detected image information as described above will be described together with the movement of the actual user on the front side of the translucent surface 2. For example, the user performs an operation on the front side of the translucent surface 2 using some object that can reflect infrared rays on the front side of the translucent surface 2. The user's own fingers and body are used.
Here, for example, as shown in FIG. 1, when the user is far away from the semi-transparent surface 2 on the front side of the semi-transparent surface 2, for example, the user passes through the semi-transparent surface 2 and is reflected on the user's body. Since the amount of infrared light is small, most of the light does not return from the front surface of the translucent surface 2 through the back surface. At this time, the above-described reference input image level Lint and the current input image level Lprs are equivalent, and the input image processing unit 12 detects that the difference input image level L is substantially zero. That is, as the detected image information generated based on the difference input image level L, a state in which there is no change similar to the initial state is obtained.

For example, if the user gradually approaches the translucent surface 2 from the above state, the infrared light that passes through the translucent surface 2 and reflects on the user's body passes through the translucent surface 2. As a result, the amount of light reaching the back side gradually increases. When this state is viewed from the input image processing unit 12, it can be regarded as a state in which the current input image level Lprs gradually increases with respect to the reference input image level Lint of the image portion corresponding to the user's body. Accordingly, as the detected image information, the appearance of the user approaching the translucent surface 2 is gradually captured according to the calculated difference input image level L.
When the user's body is very close to the translucent surface 2 (for example, within 30 cm from the translucent surface 2 depending on the setting of the threshold), the infrared light reflected on the human body is almost semi-transparent. Since it passes through the transparent surface 2 and reaches the back side, detection image information with a clearer body shape is generated.

Here, it is assumed that the user places his / her body at a certain distance from the translucent surface 2 and is positioned very close to the translucent surface 2 by holding his / her finger in front.
In this case, since the user's finger close to the translucent surface 2 reflects more infrared light than other body parts, the image information obtained in the input image processing unit 12 includes an image region at a position corresponding to the user's finger. The level of the image area at the position corresponding to the body part of the user in the background portion becomes weak according to the distance from the translucent surface 2. For example, in this state, if a predetermined threshold value set by the threshold value control unit 13 is compared with the detected image information, an image of only a portion corresponding to the user's finger is easily separated from the background. Similarly, depending on the setting of the threshold value, it is also possible to obtain image information obtained by extracting only the body part of the user at a distance away from the translucent surface 2. As described above, the threshold value control unit 13 sets such a threshold value according to the condition actually required.

In this way, by adopting a configuration for detecting the state of the front surface side of the translucent surface 2, the following advantages can be obtained when the translucent surface 2 functions as an operation panel for an interactive interface, for example. can get.
First, in the present embodiment, input information is obtained based on an image obtained from the amount of reflected infrared light from the translucent surface 2 side. Therefore, as a physical object for providing input information, special pointing is particularly preferable. For example, any device that does not need a device and reflects infrared rays can be used. In other words, as described above, the entire human body or a part thereof, or some other object can be used without any problem as described above.
In addition, for example, black has the effect of absorbing infrared light without reflecting it, and as described later, for example, by giving a black specific pattern to the translucent surface 2, It is also possible to treat the pattern as input information (ID information) having a specific significance.
Furthermore, if a change in the amount of infrared light is picked up by the CCD camera 4, it can be handled as input information. Therefore, for example, a pointing device that emits infrared light is used as the input information. It is also possible to adopt a method of giving infrared light to 2.

  Further, for example, in a touch panel or the like, an operation body such as a finger needs to be brought into contact with the operation panel surface. In the case of the present embodiment, the position and movement of the operation body may be detected as reflection of infrared light. Therefore, it is not necessary to bring the operating body into contact with the translucent surface 2, and a method of operating in the space in front of the operating body can be adopted. Furthermore, in the present embodiment, in order to obtain input information based on an image obtained by infrared reflected light from the translucent surface 2 side, a plurality of operating bodies (that is, input information) are provided if image recognition is possible. It is possible to recognize at the same time.

  In addition, as described above, since the amount of reflected infrared light changes according to the distance of the operating body relative to the translucent surface 2, it is also conceivable to use, for example, the distance from the translucent surface 2 of the operating body as input information. .

  Further, as described above, the translucent surface 2 is constituted by simple means such as combining a translucent thin film such as tracing paper with a transparent glass plate or the like, or using something like polished glass. In particular, it is not necessary to provide a driving circuit for display as a panel structure, so that it can be easily enlarged at a low cost. In this respect, it is greatly different from a touch panel that is difficult to enlarge. .

Since the translucent surface 2 also has a function as an image display panel, for example, a menu screen to be operated is displayed as will be described later, and then the user touches the menu screen with a finger or the like. It is also possible to easily realize a direct operation such as enabling the operation.
As described above, the information input / output device according to the present embodiment provides many possibilities for inputting the input information, and thus an interactive input / output environment that has never existed can be easily achieved. Can be built.

  Based on the configuration described so far, the user performs a menu operation as the operation of the information input / output device 1 of the present embodiment in response to the input information obtained by the operation performed on the translucent surface 2. A case will be described as an example, and this will be described with reference to FIGS.

FIG. 5 shows a case where the user performs a menu operation using the information input / output device 1, and here shows a state in which the translucent surface 2 is viewed from the front side.
For example, as shown in this figure, if the user approaches the front surface of the translucent surface 2, first, the control device 6 of the information input / output device first translucent the user has approached based on the detected image information obtained at this time. Recognize position on surface 2. Then, display control is performed so as to display the menu screen M as shown in the figure in correspondence with the position on the semi-transparent surface 2 that the user has recognized as approaching. This menu screen M is naturally an image projected from the projector 5 onto the translucent surface 2.

  Then, under the state in which the menu screen M is displayed on the semitransparent surface 2 near the user himself / herself, for example, the user uses his / her finger to display the operation item on the menu screen M. Suppose you specify to point to an arbitrary area. At this time, the user's fingertip is set at a distance within a range of about 3 cm to 30 cm from the top of the translucent surface 2.

Thus, for example, on the menu screen M, some instruction display (for example, cursor placement display on the selected region or highlight display in a predetermined form) is performed indicating that the region of the operation item pointed to by the user has been selected. become. This display control for highlighting is realized by detecting the coordinates of the region pointed by the user's finger based on the detected image information.
Here, it is assumed that the enter operation is performed when a predetermined time (for example, about several seconds) has passed since the instruction display is started as described above. If the user performs an enter operation, that is, if a state in which a specific operation item is highlighted is maintained for a predetermined time or longer, a required control operation according to the specified operation item is executed. For example, in accordance with the designated operation item, a menu screen of another layer is displayed, or a desired operation is executed on the information input / output device. Alternatively, if the information input / output device is configured to be able to control some external device, and the menu screen is for performing operation control on the operation of the external device, an external device is designated according to the designated operation item. It will control the operation of the equipment.
When the user moves away from the front surface of the translucent surface 2 and there is a certain distance between the user and the translucent surface 2, the menu screen M that has been displayed so far is automatically deleted. It is supposed to be.

  Here, the flowchart of FIG. 6 shows the processing operation of the control device 6 executed corresponding to the usage example shown in FIG. The processing operation shown in this figure is performed mainly by the input image processing unit 12 in the control device 6 recognizing the input information based on the detected image information, and the database driving unit 14 in accordance with the program stored in the database memory 15. This is realized by appropriately performing the processing operation based on the information.

In the routine shown in this figure, first, in step S301, it is determined whether or not “approaching body” is detected as input information from the current detected image information. Here, the “approximate body” refers to some detection target (referred to as the user's own body in FIG. 5) that has approached the translucent surface 2 to a predetermined distance range.
This “approaching object” is detected by comparing, for example, the input image processing unit 12 with the detected image information and a threshold value (set by the threshold control unit 13) set for detecting the approaching object. If a value greater than or equal to this threshold is obtained in an area with detected image information, it is detected that there is an approaching object. Will be detected. The threshold for detecting the approaching body is usually set based on the image level of the human body part obtained as detected image information when the human body (user) approaches the translucent surface 2 to some extent (for example, several tens of centimeters). It only has to be done.

  If no approaching object is detected in step S301, the process proceeds to step S308, where it is determined whether or not the menu screen M is currently displayed. Here, the menu screen M is not displayed. In this case, the process returns to the original routine (i.e., shifts to the process of step S301 again). However, if the menu screen M is being displayed, the process proceeds to step S309, and the control process for deleting the menu screen M is performed. Execute. The menu screen M erasing process is realized, for example, when the database driving unit 14 stops the image data generating process of the menu screen M for the image generating unit 16.

  On the other hand, when an approaching body is detected in step S301, the process proceeds to step S302, and the position of the approaching body on the translucent surface 2 is detected. This processing can be performed, for example, by detecting the coordinates of the area occupied by the approaching body portion in the detected image information. In this case, the coordinates to be detected may be a predetermined point in the area of the approaching body or a plurality of points obtained according to a predetermined rule, and can be arbitrarily set according to the actual use environment or the like. Good.

  In the subsequent step S303, control for displaying the menu screen M on the region of the translucent surface 2 corresponding to the position of the approaching body detected in step S302 is executed. In this control processing, for example, the database drive unit 14 is controlled based on a program for menu screen display stored in the database memory 15 so that image data of a required type of menu screen is created in the image generation unit 16. Will do.

  At this time, the database drive unit 14 creates display image data, for example, by mapping the image data of the menu screen to the display area corresponding to the position of the approaching body detected in step S302. As a result, as an image finally projected from the projector 5, the menu screen M is displayed at a position on the translucent surface 2 where the user approaches.

After the process of step S303 is executed, in step S304, it is determined whether or not an “operation tool” has been detected in the display area that is the operation item of the currently displayed menu screen M. Here, the “operation body” refers to an object (detection target) at a close distance (approximately 3 cm to 30 cm depending on the setting of the threshold) on the front surface of the translucent surface 2. That is, in FIG. 5, the finger pointing to the menu screen M is a target.
Then, in the detection process of the “operation object”, first, the threshold value set in the threshold value control unit 13 for detecting the operation object is compared with the image level of the detected image information, so that the operation object is detected. The presence or absence is detected. The threshold value set at this time is set to a value larger than the threshold value for detecting the approaching object described above in order to detect an object at a close distance on the front surface of the translucent surface 2 separately from the background. Is done.
For example, if an operating tool is detected as a result of comparison with a threshold value, the coordinate position on the detected image information from which the operating tool is detected is detected, and the detected position and the menu screen M are displayed. By determining whether or not the positions on the image information are the same, the presence or absence of the operating body in the display area of the currently displayed menu screen is detected.

  The case where the operating tool is not detected in the display area which is the operation item of the menu screen M in the above step S304 means that the operating tool is not detected on the detected image information (the user moves on the translucent surface 2 at a close distance). Even if an operating tool is detected on the detected image information, the detected position (coordinates) of the operating tool is displayed in an area on the image information corresponding to the display area of the menu screen M. There is no state (a state in which the user is pointing to the semi-transparent surface 2 at a close distance is an area other than the operation item on the menu screen M). In such a case, the process returns to step S301. To be done.

  When the operating body is specified as a human hand or finger, for example, as the operating body detection process in step S304, for example, the shape of the human hand or finger appearing during operation on the database memory 15 The information on the shape of the hand or finger and the image shape obtained as the detected image information are compared, and the presence or absence of detection of the operating body is identified by checking the matching state. Is possible. That is, in the present invention, in order to detect input information from image information, it can be recognized as one of input information having some significance even based on the shape of the image obtained in the detected image information.

  If it is determined in step S304 that the operating tool has been detected within the display area that is the operation item of the menu screen M, the process proceeds to step S305, and the operation of the menu screen M corresponding to the position where the operating tool is detected. Control is performed so that instructions are displayed for the item, and the process proceeds to step S306.

The process in step S306 is an enter operation standby process. As described above, the enter operation here is defined to be determined when a predetermined time elapses from the state in which the instruction display is started. Therefore, in step S306, detection is made as to whether or not the detection state of the operating tool detected in step S304 is maintained for a predetermined time or more. This detection process is performed by monitoring the state transition of the current detected image in the input image processing unit 12.
Then, for example, the operating tool is no longer detected from the current detected image information, or the detection position of the operating tool on the current detected image information is displayed as the operation item of the menu screen M detected in step S304. If it is detected that the region has fallen out, the process returns from step S306 to step S301. (By this process, for example, when the designated position is changed so that the user points to a different operation item on the menu screen M, the operation item on the newly designated menu screen M is changed. Instruction display will be performed.)

On the other hand, if it is determined in step S306 that the detection state of the operating tool detected in the immediately preceding step S304 has been maintained for a predetermined time or longer, the process proceeds to step S307 assuming that the enter operation has been performed.
In step S307, a required control process corresponding to the operation item at the position where the operating tool is detected on the menu screen M is executed. The processing in this step is executed by the database drive unit 14 according to a program stored in the database memory 15.

<2. First Embodiment>
Next, as an embodiment based on the configuration of the information input / output device 1 described so far, the first embodiment will be described with reference to FIGS.
FIG. 7 conceptually shows a state where the user is using the information input / output device 1 as the first embodiment. The configuration of the information input / output device 1 as the first embodiment shown in FIG. 7 and the configuration inside the control device 6 may be the same as those shown in FIGS.

FIG. 7 shows a state in which the user holds the ID card 30 in the state of being positioned in front of the translucent surface 2. On the back surface of the ID card 30, for example, an ID pattern is drawn as information for specifying a user.
FIG. 8 shows an example of an ID pattern drawn on the back surface of the ID card 30. The ID pattern 30a is drawn with a so-called two-dimensional barcode, which is a specific black pattern, for example, with respect to a white ID card 30.

For example, the user shown in FIG. 7 causes the ID pattern 30 a drawn on the back surface of the ID card 30 held in his hand to approach the translucent surface 2.
For example, since black has a function of absorbing infrared light without reflecting it, an area corresponding to the ID card 30 obtained as detected image information in the above-described state has an ID just as shown in FIG. An image of the pattern 30a is obtained. Then, in the control device 6 of the information input / output device 1 of the present embodiment, the image of the ID pattern 30a obtained as described above is identified as input information for specifying a predetermined significance, that is, ID information. A required response operation is executed according to the identified ID information.

Various types of processing to be executed based on the recognized ID information can be considered according to the usage pattern. As an example, a menu screen of a type specified according to the ID information as shown in FIG. Can be displayed.
In the case of FIG. 9, the ID card 30 is a card for identifying a user, and a unique pattern is determined for each user as the ID pattern 30a drawn on the back surface thereof. Then, assuming that Mr. Kou, one of the users, presents the ID card 30 owned by himself so that the ID pattern 30a on the back side is shown on the translucent surface 2 side, for example, a menu of information contents about Mr. Kou as shown in the figure. A screen M is displayed by the projector 5. Further, the menu screen M at this time is displayed at a position on the translucent surface 2 corresponding to a position where the user is actually standing, for example, as in the case described with reference to FIG. If the display position of the menu screen M is set in this way, the user does not have to move from the previous standing position in order to operate the displayed menu screen M. The subsequent operation on the menu screen M may be performed in accordance with the method described in FIG.

Subsequently, another example as a response operation to be executed based on the recognized ID information is shown in FIG.
FIG. 10A shows a user standing in front of the translucent surface 2 while holding the document folder 31 in this case. In this case, as shown in FIG. The ID pattern 31 a is drawn on the back side of the document folder 31. The ID pattern 31a is a two-dimensional barcode drawn in black as in the case of the ID pattern 30a shown in FIG. 8, and enables identification for each folder.
In this case, for example, as shown in FIG. 10A, the user brings the back surface of the document folder 31 closer to the translucent surface 2 side. As a result, the ID pattern 31a is recognized as the ID information in the control device 6 of the information input / output device 1 as in the case of FIG. Then, based on the recognized ID information, a menu screen M for calling information on the contents related to the document folder 31 held by the user is displayed. Also at this time, the menu screen M is displayed on the translucent surface 2 so as to correspond to the user's standing position.

  FIG. 11 shows a processing operation of the control device 6 for realizing the operation as shown in FIG. 9 or FIG. Here, description will be made assuming that it corresponds to the operation of FIG. In order to be able to execute the processing operation shown in FIG. 11, ID pattern data for making it possible to identify all patterns of the ID pattern 30 a prepared in advance for the database memory 14, and each of these ID patterns It is assumed that table data indicating correspondence between data and individual users is stored, and that data having the contents of the menu screen corresponding to each user is further stored.

  In FIG. 11, first, in step S401, the process waits for the ID pattern 30a to be recognized from the detected image information. This process is performed by, for example, comparing the detected image information with a predetermined threshold value set for ID pattern recognition (set by the threshold value control unit) in the input image processing unit 12. Whether the ID pattern is present by taking in the image data of the ID pattern separated from the background by the database driving unit 14 and comparing the image data with the ID pattern data stored in the database memory 15. This is done by determining whether or not.

If any ID pattern 30a is recognized in step S401, first, in step S402, for example, the position of the ID pattern 30a detected by the input image processing unit 12 in the detected image area is identified, and the identification information Is held as the position information Pos.
In the subsequent step S403, for example, as a process of the database driving unit 14, the user name corresponding to the recognized ID pattern is specified by referring to the table data, and the menu screen data corresponding to the specified user name is determined. It is made to read.
Then, the database drive unit 14 performs control for generating an image of the menu screen M by the image generation unit 16 based on the menu screen data read out by the process of step S403 in step S404. At this time, the position information Po detected in step S403 is used to control the image processing so that the menu screen M is displayed at the position on the translucent surface 2 corresponding to the position information Pos. Will be executed.
The operation shown in FIG. 10 also changes the storage contents and processing operation of the database memory so that the folder is specified instead of the user name and the menu screen related to the folder is displayed according to the ID information. As long as it is done, it can be realized according to the processing operation shown in FIG.

<3. Second Embodiment>
In recognizing ID information as the present invention, the present invention is not limited to the configuration for detecting the detected image information as described above, and for example, a dedicated ID recognition device may be provided. Next, a configuration provided with such a dedicated ID recognition device will be described with reference to FIGS. 12 and 13 as a second embodiment of the present invention.
FIG. 12 shows a user's usage mode for the information input / output device according to the second embodiment. Here, the user with the ID card 30 stands on the translucent surface 2. It is shown.

In this case, for example, the translucent surface 2 is installed so as to be fitted to the frame wk, and is installed so that the ID recognition device 20 is hung on the frame wk. Thus, when the structure which provides the ID recognition apparatus 20 independently is taken, it becomes possible to recognize ID information by various methods.
As an example, if the ID recognition device is an imaging device or the like, the ID information can be recognized by drawing on the ID card some symbols such as the barcode used in the embodiment described on the left. If the imaging apparatus adopts a color-compatible configuration, it is possible to recognize the color difference specified by the design as ID information.

Further, after the ID information is magnetically recorded on the ID card 30, the ID recognition device 20 is configured to be able to detect the ID information based on the magnetic information, and furthermore, the light (in this case, light in a predetermined wavelength band) The ID card 30 is preferably configured to be capable of transmitting ID code data modulated by electromagnetic waves, and the ID recognition device 20 receives and demodulates the ID code data. For example, it may be configured to obtain ID information.
Furthermore, without using the ID card 30, for example, if an individual is specified, a fingerprint or a voiceprint can be recognized as ID information.

  Also in FIG. 12, the ID card 30 is assumed to have ID information as identification information for specifying the user (Mr. Ko) as a usage form, and the ID recognition device 20 according to any one of the methods described above. The state where the menu screen M dedicated to Mr. Ko is displayed on the translucent surface 2 is shown by recognizing the ID information.

Here, FIG. 13 shows the configuration of the information input / output device 1A as the second embodiment as described above. In this figure, the translucent surface 2 is omitted, but it is natural that the infrared LED panel 3, the CCD camera 4, and the projector 5 are provided on the back surface. Also, the same parts as those in FIG.
In the information input / output device 1A shown in FIG. 13, the ID recognition device 20 described in FIG. 12 is provided. The ID recognition device 20 supplies ID information obtained according to a predetermined ID recognition method to the database driving unit 14. The internal configuration of the ID recognition device 20 is appropriately changed according to the ID recognition method that is actually employed.

Note that the display control of the menu screen M based on the ID information recognized by the information input / output device 1A of the present embodiment is not shown in the flowchart, but in this case, the database driving unit 14 of the control device 6 The menu screen data specified from the ID information recognized by the information input / output device 1A is read from the database memory 15, and display control for the menu screen is performed.
In this case, for example, if the menu screen is displayed at a position on the translucent surface 2 corresponding to the position information (coordinates) of the captured image of the user detected in the detected image information, the previous implementation is performed. In the same manner as in the embodiment, the menu screen M can be displayed on the display area on the translucent surface 2 just before the user's standing position.

<4. Third Embodiment>
FIG. 14 conceptually shows the configuration of an information input / output device 1B as a third embodiment of the present invention. For example, the same parts as those in FIG.
The information input / output device 1B shown in FIG. The receiving device 21 is provided to receive a data signal spatially transmitted by, for example, infrared rays and transmit the data signal to the control device 6. The control device 6 of the present embodiment is configured such that various required control processes can be executed using the data signal supplied from the receiving device 21. In addition, the information input / output device 1A according to the present embodiment performs data transmission using infrared rays by applying intensity modulation to infrared light emitted from the infrared LED panel 3 based on required data, as will be described later. Is configured to be performed externally.
The receiving device 21 only needs to be installed at a position where the infrared signal transmitted from the front surface of the translucent surface 2 can be satisfactorily received, and an appropriate arrangement position depends on the actual installation environment of the information input / output device 1B. Since they are different, there is no particular limitation here.
FIG. 14 shows a user who is operating the personal computer device 40 in a state of being relatively close to the entire surface of the translucent surface 2. In this case, for example, the personal computer device 40 is provided with the ID pattern as described in FIG. 8 on the back surface of the housing and the like, and performs transmission / reception with the information input / output device 1B by infrared rays. It is assumed that it has an infrared communication function. An example of using the information input / output device 1B of this embodiment in combination with the personal computer device 40 will be described later.

FIG. 15 is a block diagram showing an internal configuration example of the control device 6 provided in the information input / output device 1B together with the overall configuration of the information input / output device 1B as the present embodiment, and is the same as FIG. 2 and FIG. Are denoted by the same reference numerals and description thereof is omitted.
First, in FIG. 15, as an internal configuration of the database driving unit 14 in the control device 6, for example, a configuration for dealing with data transmitted in time series from the receiving device 21 is shown.

  In this case, the database drive unit 14 includes a CPU (Central Processing Unit) 14a, a memory 14b, an address counter 14c, a clock generator 14d, and the like. The CPU 14a executes a required control process based on the input information supplied from the input image processing unit 12 (information obtained by imaging the translucent surface 2 with the CCD camera 4) and the data signal supplied from the receiving device 21. It is configured to be possible. The memory 14b is provided to temporarily hold various data such as the input information and data signals necessary for the CPU 14a to execute control processing and data necessary for various arithmetic processing. The data writing and reading control for the memory 14b is executed by the CPU 14a based on the writing address and the reading address generated by the address counter 14c. The processing operation of the CPU 14a is executed at a timing based on a clock having a predetermined frequency generated by the clock 14d. The clock 14d is configured with a crystal oscillator, for example.

In the information input / output device 1B of the present embodiment, the data signal can be intensity-modulated with infrared light and transmitted to the outside as described above. Corresponding to this, the control of the present embodiment is performed. In the device 6, an LED intensity modulation unit 19 is provided.
For example, when it is necessary to transmit a required data signal, the CPU 14 a outputs a data signal to be transmitted to the LED intensity modulator 19. The LED intensity modulation unit 19 performs a modulation process according to a predetermined method on the input data signal, thereby generating and outputting a drive signal whose level is modulated in accordance with the data signal. In the LED intensity modulation unit 19, the LEDs of the infrared LED panel 3 are driven to emit light based on the drive signal generated by the LED intensity modulation unit 19. As a result, the infrared LED panel 3 emits the light emission intensity of the LED. By functioning as a change, it functions as a transmission unit that transmits a data signal.

A usage example of the information input / output device 1B as the third embodiment configured as described above will be described with reference to FIG.
FIG. 16 shows a state in which the user is located on the front surface of the translucent surface 2 while using the personal computer device 40. The personal computer device 40 shown in this figure is the same as that described above with reference to FIG. 14, and as described above, a predetermined pattern as an ID pattern is attached to the back side. It is assumed that an infrared communication function for enabling communication between the information input / output devices 1B is provided. Note that the ID pattern attached to the personal computer device 40 here means that the object approaching the translucent surface 2 has a function corresponding to the information input / output device 1B (here, at least mutual communication is possible). As long as it indicates that the personal computer device has the above).

Here, for example, it is assumed that the user brings the personal computer device 40 currently being operated closer to the semi-transparent surface 2 with the back side facing the translucent surface 2.
Under the above-described state, the control device 6 on the information input / output device 1B side can obtain information on the ID pattern on the back of the personal computer device 40 as input information obtained based on the detected image information. However, the database drive unit 14 (CPU 14a) of the control device 6 responds to the ID pattern obtained as this input information to the personal computer device 40 with its display data (currently on the display panel of the personal computer device 40). 40b is transmitted to request transmission of the same image data as the image being displayed). This transmission is performed by driving the LED to emit light by intensity modulation from the infrared LED panel 3 as described above.
On the other hand, the personal computer device 40 having the infrared communication function receives the instruction signal transmitted as described above. In response to the received instruction signal, image data having the same content as the image currently displayed on the display panel 40b is transmitted and output as display data by an infrared signal. The format of the image data transmitted at this time is not particularly limited, and a format suitable for actual infrared transmission may be arbitrarily selected.

The display data transmitted as described above is received by the receiving device 21 provided in the information input / output device 1B and supplied to the control device 6. In the control device 6, the supplied image data is temporarily stored in the memory 14 b, for example, and appropriately supplied to the image generation unit 16, so that the image generation unit 16 displays the image data on the projector 5, for example. To generate image data in a format suitable for. Here, for example, if the display data to be transmitted is subjected to predetermined encoding processing or compression processing for adapting to transmission, it is possible to execute decoding processing and expansion processing of the received display data. In addition, the image generating unit 16 and the database driving unit 14 are configured.
In this way, the image generated by the image generating unit 16 is supplied to the projector 5 via the image synthesizing unit 17 and the RGB signal generating unit 18, so that, for example, as shown in FIG. The same monitor image GMN as that displayed on the display panel 40 b of the device 40 is displayed on the translucent surface 2.
Here, it is possible to enlarge and display an image being displayed on the display panel 40b on the large translucent surface 2, and for example, as a specific usage form, the presenter is at hand in a meeting or the like. It is possible to explain while operating the personal computer.

FIG. 17 is a flowchart showing the processing operation of the control device 6 (CPU 14a) for enlarging and displaying the image being displayed on the display panel 40b of the personal computer device 40 described above on the translucent surface 2.
As shown in this figure, first, the CPU 14a waits for an ID pattern to be recognized as input information obtained based on the detected image information in step S501.
If it is determined that the ID pattern has been recognized, the CPU 14a proceeds to step S501, and the personal computer device having a function in which the ID pattern recognized in step S501 corresponds to the information input / output device 1B. It is discriminated whether or not it is an indication. In this determination process, for example, the database memory 15 stores various data of a required ID pattern that the information input / output device recognizes and should perform a required response process. The input ID pattern and the ID pattern data stored in the database memory 15 are referred to.

If it is determined in step S502 that the input ID pattern is not a content indicating that the personal computer has a function corresponding to the information input / output device 1B, the process proceeds to step S506, and the ID pattern is actually set. The processing routine is returned to the original routine even if the processing operation according to the contents shown in FIG.
On the other hand, if it is determined that the input ID pattern indicates that the personal computer has the corresponding function of the information input / output device 1B, the process proceeds to step S503, and the personal computer is An instruction signal for requesting output of display data is transmitted, and the process proceeds to step S504.

  In step S504, it is determined whether display data to be transmitted in response to the instruction signal transmitted in step S504 has been received. If it is determined that display data has been received, the process proceeds to step S505. If it is not determined that display data has been received, the processes of steps S507 → S503 → S504 are repeated continuously. Thus, an instruction signal for requesting display data output is transmitted until a predetermined time has elapsed in step S507. Here, when a predetermined time has elapsed in step S507, the process returns to the original routine, that is, the process for displaying the monitor image GMN of the personal computer device is not particularly executed. Such a non-display process is executed when, for example, an application corresponding to the monitor image display function for the translucent surface 2 even if the personal computer apparatus is not turned on or is turned on. It is conceivable that the software has not been started or that some trouble has occurred in the personal computer device itself. In consideration of this, for example, if a positive result is obtained in step S507, an error message may be displayed on the translucent surface 2 and then the original routine may be returned. Conceivable.

  In step S505, image data in a format that can be displayed by the projector 5 is generated based on the received display data, and the generated image (monitor image GMN) is projected and displayed on the translucent surface 2 by the projector 5. Even if the control process for executing this is executed, the routine returns to the original routine. Since the operation to be executed by the CPU 14a at this time has been described in FIG. 16, the detailed description thereof is omitted here.

<5. Fourth Embodiment>
FIG. 18 shows a configuration of an information input / output device as a fourth embodiment of the present invention. The information input / output device 1C shown in this figure has the same configuration as that of the information input / output device 1B as the third embodiment shown in FIG. The description is omitted. Further, the internal configuration of the control device 6 as the present embodiment may be the same as the configuration of FIG.
In the present embodiment, the receiving device 21 receives a command signal by infrared light transmitted from the infrared remote controller 41.

  By the way, as can be understood from the above description, in the information input / output device according to the present invention, the operation information is obtained from the detected image information obtained by the infrared light from the translucent surface 2 imaged by the CCD camera 4. The In other words, in the present embodiment, as long as the amount of infrared light incident on the CCD camera 4 from the translucent surface 2 is changed, this can be recognized as operation information.

  Therefore, in the fourth embodiment, as shown in FIG. 18, an infrared remote controller 41 that transmits a command signal by infrared light is used as a pointing device and an instruction signal input device.

  For example, in the present embodiment, in a state where the user performs a predetermined key operation on the infrared remote controller 41 to transmit and output a desired command signal, the command signal is irradiated onto the translucent surface 2, Pointing is performed on a desired position on the translucent surface 2. In the CCD camera 4, this is regarded as a mere spot light that appears on the translucent surface 2. As detected image information detected at this time, the level of the position (coordinates) corresponding to the irradiated portion of infrared light changes relative to the surroundings. Therefore, the input image processing unit 12 on the control device 6 side only needs to perform processing so as to recognize the level change of the detected image information as input information having a predetermined significance. For example, the control device 6 can execute a required control process in response to the pointed position. In this manner, in the present embodiment, first, the infrared remote controller 41 can be used as a pointing device.

  Since infrared light is invisible, for example, the spot SP is displayed on the semitransparent surface 2 so that the infrared remote controller 41 can know the irradiation position of the infrared light on the semitransparent surface 2. Can be considered. In this process, the input image processing unit 12 of the control device 6 identifies the position (coordinates) where the infrared light is currently irradiated based on the detected image information, and the spot by the projector 5 with respect to the identified irradiation position. This can be realized by executing display control so that the SP is displayed.

  At the same time, the infrared light emitted from the infrared remote controller 41 is received as a command signal by the receiving device 21 and transmitted to the database driving unit 14 (CPU 14a) of the control device 6. The database drive unit 14 can be configured to execute a required control process different from the response process to the input information of the infrared spot light obtained from the detected image information in response to the command signal.

As described above, when the infrared remote controller 41 is used, it is possible to simultaneously execute a predetermined operation control by the command signal simultaneously with the pointing operation on the translucent surface 2.
The following forms can be considered as specific usage examples. For example, an infrared command signal is transmitted and output by, for example, a key operation for enlarging or reducing an image to the infrared remote controller 41 in a state where a map is displayed on the translucent surface 2. The At this time, the user points an arbitrary desired position on the map. Thereby, for example, the image of the map displayed on the translucent surface 2 is changed so as to be enlarged or reduced around the pointed position.

<6. Fifth embodiment>
FIG. 19 is a diagram showing an example of a usage form of the information input / output device as the fifth embodiment. The same parts as those in FIG. In this figure, the infrared transmission filter 4a and the infrared blocking filter 5a are not shown, but they are actually provided for the CCD camera 4 and the projector 5. In addition, the configuration in the control device 6 may be the same as that in FIG.
In this figure, the translucent surface 2 is installed as a table. That is, by being supported by the four table legs F, F, F, and F, it also functions as a table top as a normal furniture. In this case, an infrared LED panel 3, a CCD camera 4, and a projector 5 (and a control device 6) are provided on the lower surface side of the translucent surface 2 as shown in the figure. In this case, the installation position of the control device 6 does not have to be on the lower surface side of the semi-transparent surface 2 in particular. For example, considering that a space is provided on the lower surface side of the transparent surface 2 (table), It is also possible to install it at a position other than.

  In this figure, a television receiver TV is installed separately from the information input / output device 1D. The television receiver TV is configured so that predetermined operation control including, for example, volume adjustment can be performed by operating, for example, a slider operator 50 and a volume operator 51 described below. For this reason, in this Embodiment, the structure which can transmit the command signal for controlling operation | movement of the television receiver TV from the control apparatus 6 is taken. Although FIG. 19 shows a control line for supplying a command signal supplied from the control device 6 to the television receiver TV, the command signal may be transmitted wirelessly.

  In FIG. 19, the slider operator 50 and the volume operator 51 are appropriately arranged as operators. The slider operation element 30 and the volume operation element 51 are configured such that an infrared command signal corresponding to the operation of an operation element such as a slider or a volume provided in the main body is transmitted from the bottom side to the outside.

Here, for example, if the user operates the slider operator 50 on the translucent surface 2 (table surface), an infrared command signal according to the operation is transmitted from the bottom surface of the slider operator 50 halfway. The light is output so as to irradiate the transparent surface 2. This infrared command signal is received by the receiving device 21 via the translucent surface 2 and supplied to the control device 6.
Here, if it is assumed that the volume of the television receiver TV can be adjusted by the operation of the slider operator 50, the control device 6 is configured to control the television receiver TV based on the input command signal. A command signal for controlling the volume is generated and transmitted to the television receiver TV. As a result, the television receiver TV operates such that the volume changes in response to an operation performed by the user on the slider operator 50. In the same manner as described above, by operating the slider operator 50 and the volume operator 51, for example, not only the volume of the television receiver TV, but also channel switching, adjustment related to image quality, and the like are performed. Is also possible. In realizing such an operation, for example, a program for allowing the information input / output device to operate the television receiver TV is stored in the database memory 14 in the control device 6. Become.
As a matter of course, the external device whose operation can be controlled by the information input / output device 1D is not limited to the television receiver, and is configured to be able to operate other various electronic devices. It is possible.

Further, what can be controlled by operating the slider operator 50 and the volume operator 51 is not limited to an external device. Although illustration in FIG. 19 is omitted, for example, some image displayed on the translucent surface 2 by the projector 5 is moved, enlarged, or reduced according to the operation of the operation element as described above. It is also conceivable to configure so that a change can be given.

<7. Sixth Embodiment>
FIG. 20 shows an example of how the information input / output device is used as the sixth embodiment. In the embodiment shown in this figure, the same parts as those in FIG. Also, in this figure, the infrared transmission filter 4a and the infrared blocking filter 5a are not shown, but actually, they are provided for the CCD camera 4 and the projector 5, and the configuration in the control device 6 is also shown in FIG. 15 may be the same.

In this case, a state is shown in which the user holds the portable electronic pet breeding device 42 in his hand and is positioned in front of the translucent surface 2. The portable electronic pet breeding apparatus 42 is, for example, a portable game machine that grows some creature as a pet. In this case, the portable electronic pet breeding apparatus 42 has an infrared data transmission function for transmitting data by infrared rays.
Here, for example, it is assumed that the user transmits and outputs an infrared data signal by performing a required operation on the portable electronic pet breeding apparatus 42. Here, as information content transmitted as an infrared data signal, for example, in addition to information indicating that the transmission source of this transmission signal is a portable electronic pet breeding apparatus, necessary information regarding the current growth state of the electronic pet (Parameter) shall be included.

The infrared data signal is received by the receiving device 21 and supplied to the CPU 14 a of the control device 6.
In the present embodiment, for example, display data corresponding to the growth stage of the electronic pet is stored in the database memory 15. Based on the data content obtained by receiving the infrared data signal, the CPU 14a of the control device 6 first recognizes that the transmission source is a portable electronic pet breeding device. The state of predetermined parameters relating to the current pet breeding status is identified from the data contents, and necessary display data is read from the database memory 15 by referring to these parameters. This display data is data for expressing the actual growth state of the electronic pet as an image in accordance with the parameters relating to the current breeding situation.

  Then, the CPU 14 a executes display control based on the display data read from the database memory 15 so that the projector 5 performs projection display on the translucent surface 2. Thereby, the image of the electronic pet currently grown in the portable electronic pet growing apparatus 42 possessed by the user is displayed on the translucent surface 2, and the user is growing up by himself / herself. The growth state of the electronic pet can be observed from the image displayed on the translucent surface 2.

  For example, although the growth state of the electronic pet can be observed also by a display panel (for example, a liquid crystal display panel or the like) provided in the portable electronic pet growing device 42, the screen of the display panel is usually small, Also, since the number of pixels is small, an electronic pet can be expressed only by a relatively simplified state of the symbol. On the other hand, when an electronic pet is displayed on the translucent surface 2 with the above configuration, the electronic pet can be enlarged and displayed on the translucent surface 2 having a large screen. In addition to the above, since the display data can be stored in the database memory 15 independently of the portable electronic pet breeding apparatus, the image of the electronic pet to be displayed on the translucent surface 2 is It is possible to express with higher definition.

By the way, as the information input / output device of each embodiment described so far, for example, the infrared LED panel 3 can be omitted from the configuration, but this is the basic of the information input / output device of the present invention. FIG. 1 showing the configuration will be described as an example.
For example, when the information input / output device according to the present invention is used in an environment with strong external light such as outdoors, the infrared LED panel 3 as shown in FIG. In the configuration in which the infrared light to be used is a light source for detecting input information, the intensity of the infrared light emitted from the infrared LED panel 3 is relatively weak with respect to the infrared light included in the natural light. May not be detected (that is, proper detected image information capable of recognizing input information cannot be obtained).
Therefore, in such a case, the infrared LED panel 3 can be omitted, and infrared light included in natural light can be used instead as a light source for detecting input information.
In this case, the reference input image level Lint necessary for obtaining the detected image information is assumed that there is no detection target such as an approaching body and an operating body (no operation is performed on the translucent surface 2). In this state, detection is performed based on image information obtained from an imaging signal that is transmitted through the translucent surface 2 from the front surface side and captured by the CCD camera 4.

  For example, if any operation or the like is performed on the translucent surface 2, when the approaching body and the operating body on the translucent surface 2 at this time are viewed from the CCD camera 4 side, the approaching body and the operation body are operated. Since the infrared ray of natural light is blocked by the body or the like, it can be seen as a shadow of the infrared ray contained in the natural light. In such a case, in the control device 6 of the present embodiment, image information that changes so that the image level becomes lower (darker) with respect to the reference input image level Lint is handled as input information. In this case, as the internal configuration of the control device 6 shown in FIG. 2, the LED driving unit 10 is not provided in response to the omission of the infrared LED panel 3.

In each of the embodiments described so far, for example, a microwave generator may be provided in place of the infrared LED panel 3 and a microwave receiver may be provided in place of the CCD camera 4.
In this case, if the control device 6 shown in FIG. 2 is taken as an example, a microwave drive circuit for driving the microwave generator is provided instead of the LED drive unit 10. In addition, an image signal input circuit that inputs a reception microwave supplied from a microwave receiver, converts it into, for example, data of a predetermined format and outputs it, and receives microwave reception data supplied from this image signal input circuit Thus, it is necessary to provide an input data processing circuit that obtains detected image information and obtains operation information based on the detected image information, for example, by performing necessary processing. The image signal input circuit and the input data processing circuit are functional circuit units that replace the image input unit 11 and the input image processing unit 12 shown in FIG. Further, since microwaves are used as the medium for detecting operation information, the infrared transmission filter 4a provided in the CCD camera 4 and the infrared cutoff filter 5a provided in the projector 5 are not necessary.

In this way, even if a medium such as a microwave having the property of reflecting the object irradiated with it is used for detecting operation information, the embodiment described so far (infrared input information is used) The information input apparatus according to the present invention can be configured in the same manner as in the example used for
In such a configuration, for example, when an ID pattern is to be recognized according to the description of FIG. 8, for example, a material surface that reflects microwaves and a material surface that absorbs the microwaves are combined. An ID pattern equivalent to that shown in FIG. 8 may be formed.

  In addition, the input information input mode for realizing various input / output operations in the information input / output device configured according to the present invention and the control processing operation according to the input information are limited to the contents described above. There are various other information input methods and control processing methods that take advantage of the information input / output device of the present invention. In FIG. 1, the voice output system is not shown, but it is naturally conceivable that various interactive responses are made by voice in the information input / output device according to the embodiment of the present invention.

It is a conceptual diagram which shows the basic structural example of the information input / output device as embodiment of this invention. It is a figure which shows the internal structure of the control apparatus with which the information input / output device shown in FIG. 1 is equipped. It is a flowchart which shows the processing operation for detecting and holding | maintaining a reference | standard input image level. It is a flowchart which shows the processing operation for producing | generating detection image information. It is explanatory drawing which shows the example of operation with respect to a menu screen as a usage example of the information input / output device shown in FIG. It is a flowchart which shows the processing operation for implement | achieving operation with respect to the menu screen shown in FIG. It is a conceptual diagram which shows the structure of the information input / output device as 1st Embodiment. It is explanatory drawing which shows an example of ID pattern. It is explanatory drawing which shows the usage example of the information input / output device of 1st Embodiment. It is explanatory drawing which shows the other usage example of the information input / output device of 1st Embodiment. It is a flowchart which shows the processing operation as an information input / output device of 1st Embodiment. It is a conceptual diagram which shows the structural example of the information input / output apparatus as the 2nd Embodiment of this invention. It is a figure which shows the internal structure of the control apparatus with which the information input / output device of 2nd Embodiment is equipped. It is a conceptual diagram which shows the structural example of the information input / output device of 3rd Embodiment. It is a figure which shows the internal structure of the control apparatus with which the information input / output device of 3rd Embodiment is equipped. It is explanatory drawing which shows the usage example of the information input / output device of 3rd Embodiment. It is a flowchart which shows the processing operation as an information input / output device of 3rd Embodiment. It is a conceptual diagram which shows the usage example of the information input / output device as 4th Embodiment. It is a conceptual diagram which shows the usage example of the information input / output device as 5th Embodiment. It is a conceptual diagram which shows the usage example of the information input / output device as 6th Embodiment.

Explanation of symbols

  1, 1A, 1B, 1C, 1D Information input / output device, 2 translucent surface, 3 infrared LED panel, 4 CCD camera, 5 projector, 6 control device, 10 LED drive unit, 11 image input unit, 12 input image processing unit , 13 Threshold control unit, 14 Database drive unit, 15 Database memory, 16 Image generation unit, 17 Image composition unit, 18 RGB signal generation unit, 20 ID recognition device, 21 Reception device, 30 ID card, 30a, 31a ID Pattern, 31 Document folder, 40 Personal computer device, 40a Display panel, 41 Infrared remote controller, 42 Portable electronic pet breeding device, 50 Slider operator, 51 Volume operator, TV television receiver

Claims (5)

Imaging means for imaging an image including an ID pattern;
Storage means for storing a plurality of ID pattern information and information relating to display processing corresponding to each ID pattern;
Display means for displaying image information;
Only the image information of the ID pattern is separated from the image captured by the imaging unit, and the ID pattern is recognized by comparing with the plurality of ID pattern information stored by the storage unit. An information input / output apparatus comprising: control means for performing display processing corresponding to the recognized ID pattern. The display means displays an image captured by the imaging means,
The control means further recognizes the position of the recognized ID pattern in the image area imaged by the imaging means, and causes the display means to perform the display process based on the position. The information input / output device according to claim 1.   2. The information input / output device according to claim 1, wherein the ID pattern is a two-dimensional barcode in which a predetermined pattern is drawn in black on a white background. The ID pattern is a two-dimensional barcode on which a predetermined pattern composed of a plurality of colors is drawn,
2. The information input / output apparatus according to claim 1, wherein the control means recognizes the ID pattern based on a difference in color of a pattern drawn on the two-dimensional barcode. Storage means for storing a plurality of ID pattern information and information relating to display processing corresponding to each ID pattern;
An information input / output method of an information input / output device having display means for displaying image information,
Take an image containing the ID pattern,
Only the image information of the ID pattern is separated from the image captured by the imaging unit, and the ID pattern is recognized by comparing with the plurality of ID pattern information stored by the storage unit. An information input / output method, wherein control is performed to perform display processing corresponding to the recognized ID pattern.

Images