JPH0778055A - Data processor - Google Patents

Abstract

PURPOSE:To switch display data in a sense, for example, like pages of a book turned over on the basis of input from outside HMD(head mounted display). CONSTITUTION:The data processor, equipped with a display means 48 in a housing, is equipped with a data display means 49 which displays specific data on the display means 48, a specifying means 47 which virtually specifies an optional position in the display data displayed by the data display means 49, a position detecting means which detects the virtual specified position specified by the specifying means 47, a movement distance detecting means which detects whether or not the position of the specifying means 47 detected by the position detecting means moves within a specific distance, and a display data switching means which switches the display data displayed by the data display means 49 according to the detection result of the movement distance detecting means.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data processing device, and more particularly to a head-mounted display device (hereinafter referred to as H-display device).
MD: Head Mounted Display).

[0002]

2. Description of the Related Art Conventional data processing devices include, for example, engineering workstations (hereinafter, EW).
S: Engineering Work Station), personal computer (hereinafter abbreviated as personal computer), word processor (word P-rocessor; hereinafter abbreviated as word processor), and the like.

Usually, these data processing devices are provided with a keyboard as an input device and a display device as an output device, and processing corresponding to a key command input from the keyboard is performed in the data processing device. The processing result is output to the display device.

A typical display device in a data processing device is a CRT (Cathode Ray Tube).
Display device and liquid crystal display device (hereinafter, LCD: Liquid Cry
stalDisplay).

The CRT display device is generally used as a display device in, for example, a desktop type EWS, a personal computer, a word processor, etc., and a large screen display device can be obtained at a relatively low cost. When a large-screen display device is obtained, the size of the CRT display device itself becomes large, and the installation place is limited.

On the other hand, the LCD has low power consumption and is lightweight.
Since it is thin, it is widely used as a display device for laptop or notebook type personal computers, word processors, etc. Recently, it has been used as a display device for EWS etc. because of its large screen and high definition. ing.

However, it is generally difficult for an LCD to have a large screen, and when the screen size increases, the cost is higher than that size.

Therefore, the HMD has been attracting attention as a display device that can have a large screen regardless of the installation location.

The HMD is a system in which images are displayed on two liquid crystal displays placed in front of the eyes, and the vergence angle and the focal length are set to predetermined values so that both eyes can be seen like glasses or binoculars. It gives different images to each other and displays them as if they were watching a large screen display device with a small device.

Since the HMD has the above-mentioned advantages, its use in the fields of movie watching, medical treatment by pseudo 3D display (stereoscopic display), CAD, and construction is being considered.

In a data processing apparatus equipped with such a display device, since the image display area (the number of image display pixels) that can be displayed by the display device is generally fixed, the image display that can be displayed by the display device is displayed. When display data exceeding the number of pixels is displayed, the entire data to be displayed cannot be seen if it is displayed as it is.

Therefore, it is conceivable to change the display device in use to a display device having a wider image display area. In this case, data to be displayed (hereinafter,
If the display data becomes large), a new problem arises that a display device having a large image display area is required.

Therefore, when displaying display data that exceeds the image display area that can be displayed on the display device, generally, the display image data is scrolled within the image display area by operating a pointing device such as a keyboard or a mouse. A method is used in which the entire display data is displayed after being displayed.

[0014]

However, in the above-described data processing device, when viewing the entire image of the display image data larger than the image display area, the display image data is scrolled within the image display area of the display device. By doing so, the configuration is such that the display image is switched and displayed, so there are the following problems.

That is, in the data processing device having the above-described HMD, the display device is mounted on the head, so that a keyboard or a pointing device such as a mouse is not provided as an input device, and the keyboard is not used. As in a data processing device including a pointing device such as a mouse or a mouse, it is impossible to switch a display image by instructing a display switching using a pointing device such as a keyboard or a mouse.

For this reason, it is conceivable to provide a key input section on the HMD for instructing display switching.
The HMD is desired to be small and lightweight for the purpose of mounting it on the head, and it is not preferable to provide various types of keys.

By the way, in general, human beings are extremely conservative in man-machine interface, and considering the time and effort of learning, select a user-friendly one that is close to existing tools, unless it is a very excellent and groundbreaking one. There is a tendency.

This also applies to the case where display images are switched and displayed. For example, a display device is like a book, and display data for a plurality of pages is prepared in a display section having a predetermined image display area. It is considered very effective to switch the display contents so as to turn over.

The present invention has been made in view of the above problems, and it is an object of the present invention to switch display data based on an input from outside the HMD, for example, as if turning pages of a book.

[0020]

According to a first aspect of the present invention, there is provided a data processing device comprising a display means built in a housing, the data display means displaying predetermined data on the display means. A designation unit that virtually designates an arbitrary position in the display data displayed by the data display unit, a position detection unit that detects the virtual designated position designated by the designation unit, and a position detection unit. The moving distance detecting means for detecting whether or not the detected position of the designating means has moved within a predetermined distance, and the display data displayed by the data displaying means are switched based on the detection result of the moving distance detecting means. And display data switching means.

In this case, when the movement distance detecting means detects the movement of the designating means,
It is preferable that the moving direction of the designation means is also detected. Further, as described in claim 3, the display data switching means divides a predetermined display area displayed by the data display means into two left and right screens, and the division is performed. It is effective to switch the display image for each displayed screen.
As described above, it is effective that the data switching unit switches the display image based on the moving direction detected by the moving distance detecting unit.

[0022]

According to the first aspect of the present invention, the display data displayed by the data display means is switched based on the movement of the designation means detected by the movement distance detection means.

As a result, when the designating means is moved, the display data is switched and displayed, and the display data is switched based on the input from outside the HMD.

In this case, as in the second aspect of the present invention, the display data is switched in correspondence with the moving direction of the designating means by detecting the moving direction of the designating means by the moving distance detecting means. Further, according to the invention of claim 3, the display area displayed by the data display means is divided into two left and right screens, and the display is switched by the display data switching means for each of the divided screens. According to the fourth aspect of the invention, the display data is switched based on the moving direction detected by the moving distance detecting means, for example, as if turning the pages of a book.

[0025]

EXAMPLES Examples will be described below with reference to FIGS.

1 to 24 are views showing an embodiment of a data processing apparatus according to the present invention.

First, the structure will be described. 1 and 2 are external views of the data processing apparatus of the present invention, and FIG. 1 is a view of the data processing apparatus of the present embodiment as seen obliquely from the front.
[FIG. 3] is a diagram of the data processing device of the present embodiment as seen obliquely from the rear.

The data processing apparatus 1 in this embodiment is H
Since it is built in the stand-alone display device by MD and does not have a key input means such as a keyboard, by displaying a virtual keyboard on the HMD,
It enables virtual key input.

1 and 2, the data processing device 1
Includes an optical system display unit 2 that generates various data processed in the data processing as a display image, and various units that combine and display the image generated by the optical system display unit 2 and the outside scenery. Equipment case 3,
The optical system display unit 2 and the device case 3 are supported, and are composed of a retractable headband 4 to be mounted on the head.

As shown in FIG. 1, the equipment case 3 is provided with a rod antenna 5, a power ON / OFF switch 6, a designation key 7, a menu switching down instruction key 8 and a menu switching up instruction key 9. As shown in FIG. 2, viewfinders 10R and 10L are formed.
Further, as shown in FIG. 1, the headband 4 is provided with a support portion 11, earphones 12R, 12L, and
As shown in FIG. 2, a mounting detection sensor 13 is provided.

The rod antenna 5 is a telescopic antenna for converting various data processed in the data processing device 1 into radio waves and transmitting the radio waves, and the power source on / off switch 6 is a power source unit (not shown). ) Is a switch for selecting whether or not to energize the data processing device 1 with the power supply voltage supplied from (4), and the designation key 7 is used to confirm the operator who will use the data processing device 1 when the data processing device 1 is used. It is a switch to start. The antenna to be used is not limited to the retractable rod antenna 5 as in this embodiment, but may be a helical antenna, for example, and an antenna suitable for the wavelength band of the radio wave used can be used.

The menu switching down instruction key 8 and the menu switching up instruction key 9 are key switches for switching the menu contents of the menu screen displayed by the optical system display unit 2, and the menu switching down instruction key 8 is a forward switch. The menu contents are switched in the direction (the direction in which the frame cursor in the menu screen is moved downward), and the menu switch up instruction key 9 is in the opposite direction to the menu switch down instruction key 8 (the frame cursor in the menu screen is moved upward). The contents of the menu are switched in the moving direction).

The viewfinders 10R and 10L are viewing windows formed with a transparent filter, and the supporting portion 11 supports and fixes the device case 3 and the headband 4.

The earphones 12R and 12L are for listening to error sounds and guidance information, and the wearing detection sensor 13 is a sensor for detecting whether or not the headband 4 is worn on the head of the operator. .

FIG. 3 is a cross-sectional view of the data processing apparatus 1 in this embodiment as seen from the lateral direction, and FIG. 4 is a cross-sectional view of the data processing apparatus 1 as seen from the rear in this embodiment.

As shown in FIG. 3, the optical system display unit 2 includes a backlight 21 composed of a side-light type cold cathode fluorescent lamp (fluorescent lamp) and an LCD 22R which is an active matrix type color liquid crystal display unit of about 1 inch. , 2
2L and concave lenses 23R and 23L made of glass or plastic optical lenses, the image (light) that is illuminated by the backlight 21 and transmitted through the LCDs 22R and 22L is spread by the concave lenses 23R and 23L, and the device case can be efficiently used. Sent to 3.

As shown in FIGS. 3 and 4, the device case 3 includes an upper glass 31 which serves as a light receiving portion for receiving an image (light) from the optical system display unit 2, and a glass or plastic optical lens. Convex lenses 32R, 32
L, a front glass 33 that serves as a light receiving portion for receiving an outside scene (outside light), a transmission state and a light blocking state of whether or not to guide the outside light received from the front glass 33 into the device case 3. A shutter LCD 34 that serves as a liquid crystal shutter to be switched.
A reflection magic mirror 35 formed of a half mirror formed inside the device case 3 of the shutter LCD 34,
Two CCDs on the left and right side walls of the device case 3
(Charge Coupled Device) 36R, 36L
The image (light) obtained from the optical system display unit 2 is reflected by the reflection magic mirror 35 through the glass 31 and the convex lenses 32R and 32L, and the eyes ER of the operator through the viewfinders 10R and 10L. EL
Reach. Further, the image of the surrounding situation seen forward through the front glass 33 is combined with the image from the optical system display unit 2 through the reflective magic mirror 35 when the shutter LCD 34 is in the transmissive state, and the viewfinder 10R, 10L.
On the other hand, when the shutter LCD 34 is in the light-shielded state, the image of the surrounding situation seen forward through the front glass 33 does not reach the eyes ER and EL of the operator, and only the image from the optical system display unit 2 is output. Is output to the operator.

Here, the two CCDs 36R, 36L provided on the left and right side walls in the device case 3 pick up images in either direction of the front or rear viewfinders 10R, 10L, so that the direction is forward. It is rotatably provided so as to rotate 180 degrees backward and forward. When the shutter LCD 34 is in the transmissive state, two C
The front scene is imaged by the CDs 36R and 36L, and for example, it is detected whether or not there is a "hand" in the front view.
When there is a "hand", a process for obtaining the position of the "hand" and the distance from the HMD to the "hand" is performed based on the parallax of the two CCDs 36R and 36L. Also, LC for shutter
When D34 is in the light-shielded state, the two CCDs 36R,
36L rotates 180 degrees to the finder 10R, 10L side, and images the eyes ER, EL of the operator through the finder 10R, 10L. The captured eye image is
It is compared with image data of each operator's eyes stored in a data memory described later. By the way, as the eye data during the comparison process, for example, the ratio of the pigment distribution between the cornea (black eye) part and the conjunctiva (white eye) part, the shape of the peripheral part of the eye (eyebrow shape, eyelid shape, eye Overall shape),
Alternatively, when the location of the blood vessel can be specified and the blood is difficult to congest, it is conceivable to use the blood vessel pattern or the like of the eye.

FIG. 5 is a block diagram showing the overall configuration of the data processing apparatus 1 of this embodiment.

The data processing apparatus 1 according to this embodiment has a CPU 4
1, ROM 42, data memory 43, angle sensor 44,
Direction sensor 45, infrared sensor 46, input unit 47, display unit 48, display drive unit 49, page turning control circuit 50,
A wireless transmission control unit 51 is provided, and further, the mounting detection sensor 1
3, CCD 36R, 36L is provided. Note that FIG.
Middle, B is CPU41, ROM42, data memory 4
3, angle sensor 44, direction sensor 45, infrared sensor 4
6, input unit 47, display drive unit 49, page turning control circuit 50, wireless transmission control unit 51, mounting detection sensor 13, C
A bus serving as a signal transmission path connecting the CDs 36R and 36L, 101 is a printer, and 102 is an external storage device.

The CPU 41 outputs various control signals for controlling each unit in the data processing device 1 to each circuit via the bus B, and the CPU 41 is based on the input information from the input unit 47. The display drive unit 49 and the page turning control circuit 50 are controlled to perform a predetermined display on the display unit 48, and the wireless transmission control unit 51 is controlled to convert various data processed in the data processing device 1 into radio waves. And transmits necessary data to the printer 101 and the external storage device 102.

The ROM 42 is a semiconductor memory that stores programs and data used in the data processing device 1.

The data memory 43 stores various data used in the processing executed by the CPU 41, that is, display data, image data of "hand", eye data for each operator, and the work area in the data processing. The semiconductor memory has a work area that becomes The display data is display data such as various menu screens displayed by the optical system display unit 2,
The "hand" image data is virtual "hand" data displayed by the optical system display unit 2 based on an image of the "hand" detected in the forward field of view of the HMD, and eye data for each operator. Is the eyes ER, EL of the regular operator
Image, that is, image data serving as a comparison reference for recognizing the operator.

The angle sensor 44 is used in, for example, a navigation system or a gyro system,
It is a sensor for detecting how many times the HMD, which is the data processing device 1, is tilted from the horizontal state by detecting "shaking". In this embodiment, as shown in FIG. 6, a position rotated by a predetermined angle in the clockwise direction in FIG. 6 from the vertical direction is set as a reference point position (0 degree position), and slightly at the 90 degree position. It is set to face upward.

The azimuth sensor 45 is a sensor for detecting how many times the HMD, which is the data processing device 1, is rotating to the left or right from the reference position. In the present embodiment, as shown in FIG. 7, the azimuth of “North” is the reference point position (N in FIG. 7).
Is set to detect the number of rotations clockwise in FIG.

The infrared sensor 46 includes CCDs 36R, 36
In order to prevent erroneous recognition, for example, when the image of the "hand" detected by L is a toy "hand", the CCD 36R, 36L detects the "hand" in the front view, Also, after the position of the "hand" and the distance from the HMD to the "hand" are obtained, the CCD image and the "hand" are obtained.
If there is a thermal reaction at the place where the "hand" image data is located, it is checked if the thermal reaction is detected, based on the position and distance of the "hand". , The CPU 41 determines that there is a “hand”, and when the thermal reaction is not detected, it is determined that the CPU 41 has erroneously recognized the CCDs 36R and 36L.

The input section 47 is a virtual input section provided to enable key input in the data processing device 1. This is because the data processing device 1 of the present embodiment is configured only by the display unit and does not have an input unit such as a keyboard specially, so that the display unit 48 displays the keyboard as the display data in the data memory 43. By superimposing and displaying the "hands" that are transparently projected, the positional relationship between the two can be clarified and key input can be performed.

As shown in FIG. 3, the display section 48 is an optical unit (backlight 21, LCD 22R, 22L, concave lens 23R) in the optical system display unit 2 which displays two images corresponding to the left and right eyes of the operator. , 23
L etc.) and the optical unit (convex lens 3) in the equipment case 3
2R, 32L, a reflection magic mirror 35, etc.), and by fixing at a point where two images are combined, the images viewed from the left and right viewfinders 10R, 10L are combined and displayed in a natural state. .

The display drive section 49 includes LCDs 22R and 22L.
Is a display driving unit that drives the LCDs 22R and 22L to display an image by the CP.
The image data generated based on the processing of U41 is displayed on the LCD.
The analog video signals (analog RGB signals) of predetermined bits corresponding to the number of pixels of 22R and 22L are converted, and the analog video signals are output to the LCDs 22R and 22L.

The page turning control circuit 50 outputs the input data to the respective LCDs 22R and 22L at respective timings based on the control described later, and performs switching display of the display data as if turning the pages of the book. is there.

The wireless transmission control unit 51 is used for the data processing device 1.
Various data processed inside are converted into radio waves and transmitted. This is because the data processing apparatus 1 of the present embodiment is composed of only the display unit, and the data processing apparatus 1 alone cannot output the processed data. Therefore, for example, when performing the printout processing described later, it is desired to print out. Desired data is modulated by the wireless transmission control unit 51 by frequency modulation or amplitude modulation, and this data is output as radio waves to the external printer 101. Similarly, when important data is stored, it is desired to store it. Is converted into radio waves and output to the external storage device 102. In this case, it is possible to send and receive data to and from the same data processing device 1 as in the present embodiment, and further, a radio wave transmitted from one host station is transmitted by a plurality of data processing devices 1 having the same configuration as this embodiment. By receiving, it is possible to see a common image, and it is possible to use it for a conference, for example.

Next, the operation of this embodiment will be described.

FIG. 8 is a flow chart of an operator recognition process at the time of starting the data processing device 1 of this embodiment.

The operator recognition processing by the data processing apparatus 1 will be described below with reference to FIG. Note that FIG.
The program corresponding to the processing operation of the CPU 41 shown in
It is stored in the ROM 42.

First, the data processing apparatus 1 in this embodiment.
When is mounted on the operator and the power on / off switch 6 is turned on, a menu screen is displayed on the display unit 48 of the HMD (step S1), and in this state, the menu switch down instruction key 8 or the menu switch up instruction is issued. Key 9
Is checked (step S2), and when the menu switching down instruction key 8 or the menu switching up instruction key 9 is pressed, the menu contents are switched based on the switching instruction (step S3).

On the other hand, in the processing of step S2,
If the menu change down instruction key 8 or the menu change up instruction key 9 is not pressed, then the designation key 7
The pressed state of is checked (step S4), and when the designated key is not pressed, the processing from step S2 is repeated.

In the process of step S4, the designated key 7
When is pressed, the mounting detection sensor 13 checks whether or not the HMD is mounted (step S
5) If not attached, step S12 described later
Proceed to error processing in.

When the mounting of the HMD is confirmed in the process of step S5, the CCDs 36R and 36L are inverted,
The direction is changed to the finder 10R, 10L direction (step S6).

Images of the eyes ER and EL of the operator are picked up by the CCDs 36R and 36L (step S7), the images of the eyes are analyzed, and image data of the left and right eyes are generated (step S8).

Next, in step S8, it is checked whether or not the eye image data has been properly generated (step S8).
9), if not generated, step S described later
If it is generated, the CPU 41 compares the generated eye image data with the eye data for each operator in the data memory 43, and the data of the registered operator's eyes ER, EL ( For example, it is checked whether or not they match the above-mentioned black eye portion, white eye portion, pigment ratio, etc. (step S10).

As a result of the check in step S10, the registered eye data and the eye E of the operator wearing the HMD are displayed.
When the data of R and EL match, the CPU 41 determines that it is used by an authorized operator, and switches to the initial menu screen where each functional process is performed according to the designated menu (step S11), while checking in step S10. As a result, if the registered eye data does not match the eye ER and EL data of the operator wearing the HMD, C
The operator currently wearing the HMD determines that the PU 41 is illegally used by an operator other than the registrant, and a predetermined error process is executed (step S12).

When the processing of step S11 or step S12 is completed, the CCDs 36R and 36L are inverted to C
The directions of the CDs 36R and 36L are changed to the front direction of the HMD again (step S13), and the operator recognition process ends.

When it is determined in the operator recognition process that the user is a regular operator, the initial menu screen as shown in FIG. 9A is displayed by the above steps S1 to S13. By the way,
In the example shown in FIG. 9A, the selection item is “data input /
The items "edit", "data recognition input", "record", and "playback" are displayed, and the frame cursor is shown in the "data input / edit" item.
When the menu change down instruction key 8 is pressed, the frame cursor moves downward in the figure, and when the menu change up instruction key 9 is pressed, the frame cursor moves upward in the figure and the target item is highlighted.

Here, as shown in FIG. 9 (a), when the designation key 7 is pressed with the frame cursor in the item of "data input / edit", the function of "data input / edit" is selected. , A data menu screen as shown in FIG. 9B is displayed. Note that in FIG. 9B, A is a layout window for indicating or specifying the layout of data to be displayed, and B is an operation for displaying the item selected on the initial menu screen of FIG. 9A. This is an item display window, which allows you to check the current operation contents. C is
Indicates the file name of the document or table data to be edited,
A file name window for designating a file name of a document or table data to be edited, and D is a keyboard designation window for designating a virtual keyboard.

Next, the data input / editing process will be described with reference to the flow charts shown in FIGS. Note that FIG. 10 is a flowchart showing the data input / edit processing.

In the data menu screen shown in FIG. 9B, during the data input / edit processing, the data of the virtual keyboard is read from the data memory 43 and the next processing cannot proceed. Therefore, it is first checked whether or not the virtual keyboard is displayed by designation (step S
21) If the virtual keyboard is displayed, the process proceeds to the next step. On the other hand, if the virtual keyboard is not displayed, the keyboard designation process described later is executed (step S22) and the virtual keyboard is displayed.

A "read key" is displayed on this virtual keyboard.
Is prepared, and it is checked whether or not the "read key" has been selected (step S23).

When the "read key" is selected in the process of step S23, a list of storage file names is displayed in a window (step S24), and when the "cursor key" on the virtual keyboard is operated in this state ( In step S25), the cursor position is changed, and the display content is updated to the file name at the changed cursor position (step S26).

Then, it is checked whether or not the window is designated for input (step S27).
File data corresponding to the designated position is read (step S28), the previous display is erased, and the layout window is displayed again (step S29).

Here, the presence / absence of the window input designation is checked again (step S30), and if there is the input designation, file data arrangement processing is performed (step S3).
1) The arranged file data is displayed and the file name window is deleted (step S3).
2) The data input / editing process ends.

On the other hand, in the processes of steps S27 and S30, if the window input is not designated, it is checked whether or not the "clear key" on the virtual keyboard is pressed (step S33), and the "clear key". If is not pressed, the processing from step S24 is repeatedly executed, and if the "clear key" is pressed, the data input / edit processing is ended.

When the "read key" is not selected in the process of step S23, it is checked whether the "write key" prepared on the virtual keyboard is selected, like the "read key". (Step S
34).

In the process of step S34, if the "write key" is not selected, the process proceeds to another process. If the "write key" is selected, a list of storage file names is displayed in a window. (Step S35), the presence or absence of input designation is checked (step S36).

If the input is designated, it is checked whether the display data is designated (step S3).
7) When the designation of the display data is confirmed, the contents of the designated display data are written in the corresponding file (step S38), and the data input / editing process ends.

If no input is designated in the process of step S36, it is asked whether or not a new file is created (step S39). If a new created file is input, a new file is created. (Step S4
0), the process from step S37 is executed.

If it is not the input of a newly created file, it is determined that the file name is updated, and the "cursor key" is operated (step S41), the cursor position is changed and the file at the changed cursor position is changed. The display content is updated to the name (step S42), and the processing from step S35 is executed.

In the process of step S41, if the "cursor key" is not operated, it is checked whether or not the "clear key" on the virtual keyboard is pressed (step S4).
3) If the "clear key" is not pressed, the processing from step S35 is repeatedly executed, and if the "clear key" is pressed, the data input / edit processing is ended.

FIG. 11 is a flow chart showing the keyboard designation process in FIG.

As shown in FIG. 11, in the keyboard designation process, first, an input designation process described later is executed (step S51), and it is checked whether or not a keyboard window is designated in the input designation process (step S51). S52).

In the process of step S52, if the keyboard designation window is not designated, it is checked whether or not the "clear key" on the virtual keyboard is designated (step S53), and if the "clear key" is designated, it is stored. After clearing the keyboard data set to the initial state (step S54), the processing from step S51 is repeatedly executed. If the "clear key" is not specified, it is checked whether the data menu screen shown in FIG. 9B is specified (step S55), and the data menu screen is specified. The keyboard designation process ends.

When the data menu screen is not designated in the processing of step S55,
It is checked whether the fixed flag is on (step S56). Here, when the fixed flag is on, the virtual keyboard is fixedly displayed at the center position of the display unit 48 regardless of the change in the visual field due to the movement of the HMD, and the fixed flag is off. In this case, just like a real keyboard, a virtual keyboard is placed in place,
The display mode is such that the keyboard enters or does not enter due to the movement of the HMD.

That is, when the fixed flag is on,
When it is determined that the virtual keyboard is in the fixed display mode, the processing from step S51 is repeatedly executed, and when the fixed flag is off, the angle sensor 4
4, the current angle of the HMD is detected, the direction sensor 45 detects the current direction of the HMD, and the respective values (angle and direction) are displayed on the display unit 48 (step S57). When the HMD moves to enter the area where the keyboard should be displayed (step S5)
8), the virtual keyboard for the keyboard area is displayed on the display unit 4
8 is displayed (step S59), and if it is not in the keyboard area, the process from step S51 is repeatedly executed.

On the other hand, when the keyboard designation window is designated in the process of step S52, whether or not the layout window is designated is checked (step S60).

The layout window is specified prior to the specification by the keyboard specification window. If the layout window is not specified, H
In order to display the virtual keyboard at the front center position wherever the MD position is facing, keyboard data is stored in the storage position corresponding to the current angle and direction of the HMD (step S61), and the virtual keyboard display is fixed. (Step S62). At this time, the fixed flag is turned on (step S63), the layout position of the virtual keyboard is displayed (step S64), and the processing from step S51 is repeatedly executed.

When the layout window is designated, the data memory 43 corresponding to the layout position
Keyboard data is stored in (step S65), H
A mode in which the virtual keyboard is displayed only when the MD faces that position, and the process proceeds to step S64.

FIG. 12 is a flow chart showing the input designation processing in FIG.

As shown in FIG. 12, in the input designation process,
First, the contents of the previous keyboard designation window are cleared (initialized) (step S71), and then the CCD3
Images in front of the HMD are captured and captured by the 6R and 36L (step S72).

Then, it is checked whether or not the image data of the "hand" of the operator exists in the captured image data (step S73). If the image data of the "hand" does not exist, the input designation processing is performed. finish.

When the image data of the "hand" exists, the position of the "hand" of the operator with respect to the key display position is obtained (step S74), and the infrared sensor 46 causes a thermal reaction at the image position of the "hand". It is checked whether or not (step S75), and if the thermal reaction is not detected, it is determined that the recognition error has occurred, and the input designation process ends.

Next, at the first start-up, the input designation is designated by depressing the designation key 7, but in this state, since the input state is from the virtual keyboard, the input designation is at the designated display position. Whether or not it is checked (step S76), and if it is outside the designated display position, the input designation processing is ended.

In the process of step S76, when the virtual keyboard and the image data of the "hand" are overlapped and the CPU 41 determines that the input is designated at the designated display position, the image data of the "hand" on the virtual keyboard is displayed. The brightness of the key that overlaps with is changed (for example, the brightness is increased) (step S77), and it is possible to know which key the image data of the "hand" is touching. In this case, the display state of the "hand" and the virtual keyboard that are transparently projected is as shown in FIG.

Here, whether or not the tip of the "hand" is at a certain distance or more from the HMD, for example, at a position 60 cm or more away from the HMD, is the two CCDs 36R, 3C.
6L (step S78), and if the tip position of the "hand" has not reached a certain distance, the input designation process ends.

In the process of step S78,
When it is judged that the tip position of the "hand" has reached a certain distance, it becomes possible to enter keys on the virtual keyboard, and if a key on the virtual keyboard is pushed in more than a certain amount,
Corresponding key blinks for a certain period of time.

After the input designating process is completed, in order to use various process data in the input designating process during the process of calling the input designating process, the key corresponding to the position of the "hand" or the window content etc. is data. It is stored in the memory 43 (step S79).

FIG. 13 is a diagram showing an actual display example on the HMD of this embodiment.

In FIG. 13, E is a display showing the direction of the HMD detected by the direction sensor 45, F is a display showing the angle of the HMD detected by the angle sensor 44, and G is a predetermined area. A layout window showing how the specified document, table data, etc. are arranged, H is a “hand” that is transparently projected, I is a virtual keyboard, J is a file name window, and K is The currently opened document data, L, is a surrounding state that can be seen because the HMD is in a transparent state, and in this example, the table lamp is visible.

As described above, the data processing apparatus 1 of this embodiment
In (HMD), the surrounding state transmitted by the reflection magic mirror 35, which is a half mirror, is also duplicated and displayed on the display unit 48.

FIG. 14 is a flow chart showing a display process executed at regular time intervals.

In this display processing, first, the angle sensor 44
And the azimuth sensor 45 detects the angle and azimuth of the HMD (step P1), the detected angle and azimuth are displayed on the display unit 48, and the display data in the data memory 43 corresponding to the current position of the HMD is displayed. The corresponding area is searched and the pointer is moved (step P2).

Then, it is checked whether or not there is data to be displayed in this corresponding area (step P3). If there is display data in the corresponding area, the process moves to the next process and there is no display data in the corresponding area. In the case, the display processing is ended.

If there is display data in the corresponding area, the fixed flag indicating the current display mode is checked (step P3).

Here, when the fixed flag is on, it is determined that the current display mode is the keyboard fixed display, and the virtual keyboard is displayed at the center position of the display unit 48 (step P5).

Then, it is checked whether or not the display data to be displayed in the corresponding area overlaps the display of the virtual keyboard (step P6). If they overlap, only the display area for displaying the virtual keyboard is in the corresponding area. Move and display the display data.

Specifically, for example, as shown in FIG. 15A, when there is display data R0, R1, R2 between the display area pointer left position indicating the corresponding area and the display area pointer right position. As shown in FIG. 15B, the display data is moved upward in FIG. 15 by the amount of displaying the virtual keyboard, and the virtual keyboard is fixedly displayed (step P7).

On the other hand, when the fixed flag is off in the process of the step P4, the step P6 is executed again.
If there is no display data that overlaps the virtual keyboard in the display area in the processing of (1), the data in the corresponding area is displayed as is (step P8). In this case, the fixed flag is turned on in the process of step P4, and
If it does not overlap the virtual keyboard, the virtual keyboard and the data in the corresponding area must be combined and displayed.

In the data processor 1 of this embodiment, H
As shown in FIG. 16, the display data sequentially displayed based on the position of the MD is obtained by designating the displayed data by the image data of the “hand” as in the case of the input of the virtual keyboard ( In the case of this embodiment,
It is possible to move to another display position by pressing the tip of the "hand".

The display data moving process will be described below with reference to FIG.

FIG. 16 is a flow chart showing the display data moving process.

First, it is checked whether or not the input designation is at the data display position (step Q1). If it is not at the data display position, the process proceeds to another process. On the other hand, when in the data processing position, the position of the "hand" with respect to the display position is obtained,
The obtained position is stored as the reference position (step Q2).

Then, whether or not the tip of the "hand" is more than a certain distance (60 cm in this case) from the HMD is two C's.
Detected by CD36R, 36L (step Q
3) If the tip position of the "hand" has not reached a certain distance, the display data processing ends.

In the process of step Q3, "hand"
If it is determined that the tip position of has reached a certain distance,
It is checked whether or not the position of the HMD has moved (step Q4), and if not, the above step Q3.
Return to the process of, if you are moving, the moving speed is calculated,
It is checked whether or not the obtained moving speed is a certain speed or more (step Q5). When the moving speed is equal to or higher than a certain value, it is determined that the HMD attached to the operator is moving, the process returns to the step Q4, and the position movement of the HMD is checked again.

When the moving speed becomes slower than the fixed value in the processing of step Q5, the moving position is set as the reference position, and the display data is moved (step Q6).
The display data layout process described below is performed (step Q7), the layout display is changed (step Q8), and the display data movement process ends.

That is, in a certain area, the display data specified before the HMD is moved is saved, and the display data is moved and rearranged based on the position saved after the HMD is moved.

Next, the placement process in step Q7 will be described with reference to FIG.

FIG. 17 is a flow chart showing the arrangement process.

When the display data already exists at the destination and the display data overlaps with each other due to the above-mentioned movement of the display data (step Q11), the display data overlapping the display data to be moved is rearranged. (Step Q12).

Next, as to the destination of the original display data by rearrangement, it is checked whether or not the display data to be moved do not overlap (step Q13), as in the case of the above-mentioned step Q11. Returning to the processing of Q11, in the case of overlapping, the overlapping display data is rearranged in a non-overlapping position (step Q14), and the processing of the above steps Q13 and Q14 is repeatedly executed until there is no overlapping.

In this case, when there are a plurality of overlapping data, the data having a high overlapping ratio is rearranged.

Incidentally, for example, when the display data is moved when the display data is displayed in the full display area, the process between steps Q13 and Q14 is repeated because the overlapping state is not resolved forever. An infinite loop occurs. Therefore, in order to prevent the occurrence of an infinite loop, if the overlapping state is not resolved for a certain time or longer for some reason, an error is notified and the original state is restored.

FIG. 18 is a flowchart showing the virtual keyboard moving process.

In the case of moving the virtual keyboard, it is first checked whether or not the moving key 7 is selected (step Q21), and if not selected, the process proceeds to another process, and if selected, the position of the HMD is determined. It is checked whether or not it is moving (step Q22), and if it is not moving,
Returning to the processing of step Q21, if the vehicle is moving, the moving speed is obtained, and it is checked whether or not the obtained moving speed is a certain speed or more (step Q23).
When the moving speed is equal to or higher than the predetermined value, it is determined that the HMD attached to the operator is moving, the process returns to the step Q21, and the position movement of the HMD is checked again.

When the moving speed becomes slower than the fixed value in the processing of step Q23, the moving position is set as the reference position, and the virtual keyboard is moved (step Q
24) If the display data already exists at the destination along with the movement of the virtual keyboard and the display data overlaps with the display keyboard (step Q25), the display data that overlaps the display keyboard is rearranged at a position that does not overlap the display keyboard. (Step Q26).

Next, with respect to the destination of the original display data due to the rearrangement, it is checked whether or not the display data to be moved do not overlap (step Q27), as in the case of the above-mentioned step Q25. Returning to the processing of Q25, when they overlap, the overlapping display data are rearranged in the non-overlapping position (step Q28), and the processing of the above steps Q27 and Q28 is repeatedly executed until the overlap disappears. Then, in the process of step Q25, the layout display is changed when the display data of the movement destination does not overlap (step Q25).
29), the virtual keyboard movement process ends.

That is, the virtual keyboard can be moved in the same manner as the display data described above. In this case, except that the arrangement position of the virtual keyboard is prioritized.
The processing is almost the same as the movement of the display data.

FIG. 19 is a flowchart of the printout processing of the data processing apparatus 1 of this embodiment.

As described above, the data processing device 1 of this embodiment is composed of only the HMD (display unit), and the data processing device 1 alone cannot output the processed data. Various data processed in the data processing device 1 are converted into radio waves and transmitted to the printer 101.

Specifically, as shown in FIG. 19, first,
It is checked whether or not the "print key" in the virtual keyboard is selected (step R1). If it is not selected, the process proceeds to another process. If it is selected, it is checked whether or not the display area is designated. If not specified (step R2), it is further checked whether or not the "clear key" on the virtual keyboard is specified (step R2).
3) If the "clear key" is specified, the stored keyboard data is cleared, the printout process is terminated after the initial state is set. Where "clear key"
If is not specified, repeat step R above.
Process 2 is executed.

On the other hand, in the processing of step R2,
When the display area is designated, the data in the designated display area is output to the wireless transmission control unit 51, and the data in the designated display area to be printed is modulated by the wireless transmission control unit 51 by frequency modulation or amplitude modulation. The data is transmitted as radio waves to the external printer 101.

The page turning control process will be described in detail below with reference to FIGS.

FIG. 20 is a block diagram showing a main configuration of the page turning control circuit 50.

As shown in FIG. 20, the page turning control circuit 50 includes a page turning display data conversion section 71, a previous display data first storage section 72, a previous display data second storage section 73,
The current display data first storage unit 74, the current display data second storage unit 75, the rear display data first storage unit 76, and the rear display data second storage unit 77 are configured.

The page-turning display data conversion unit 71 uses C
Based on the conversion control command from the PU 41 and the display position data specified by the CPU 41, the previous display data first storage unit 72, the previous display data second storage unit 73, the current display data first storage unit 74, the current display data Second storage unit 75, rear display data first storage unit 76, rear display data second storage unit 77
It outputs predetermined display data among the display data stored in each of, and creates a state of turning the page based on the display position data corresponding to the position of the "hand".

As the display area of the display data in the present embodiment, a display area consisting of two left and right screen areas is secured, so that the display data storage unit consisting of the video RAM is displayed before the current display. A previous display data first storage unit 72 and a previous display data second storage unit 73 which store display data, and a current display data first storage unit 74 and a current display data second storage unit 75 which store current display data.
And a rear display data first storage unit 76 and a rear display data second storage unit 7 which store the subsequent display data currently displayed.
7 and a storage unit for two screens, respectively.

The page-turning control processing of this embodiment will be described below by taking the case of displaying the page-turning data as shown in FIG. 21 as an example.

Here, FIG. 21 is a diagram for explaining the page turning control processing of the present embodiment, in which the data in the hatched portion indicated by C in FIG. 21 and the data indicated by D are displayed. Then, the data C is displayed as a page-turning display. Further, a "page-turning key" for performing page-turning display is provided on the virtual keyboard, and by manually designating the display area of the data C, the display data of the previous and next pages can be displayed as shown in FIG. Shall be displayed as.

FIG. 22 is a flow chart showing the page turning control processing.

First, it is checked whether or not an input is designated by the "page turning key" (step T1),
If the "page-turning key" is not pressed, the process proceeds to another process. If the "page-turning key" is pressed, it is checked whether or not the display data is designated (step T).
2). If the display data is not specified in the process of step T2, it is checked whether or not the "clear key" on the virtual keyboard is specified (step T3). If the "clear key" is specified, the stored keyboard data is checked. Is cleared and the initial state is set, and then the page turning process is ended. Here, if the "clear key" is not designated, the process of step T2 is executed again.

On the other hand, in the processing of step T2,
If the display data is designated, the display data corresponding to each page is stored, and the data of the input designated display area is displayed for each page (step T4).

Then, in order to turn the page, "hand"
It is checked whether or not the tip of has moved a certain distance or more (step T5).
Whether or not the "clear key" is specified on the virtual keyboard is checked (step T6), and if the "clear key" is specified, the stored keyboard data is cleared,
The page turning process is terminated after the initial state. Here, if the "clear key" is not designated, the process of step T5 is executed again.

In the processing of the above step T5, "hand"
If the tip of has moved more than a certain distance,
It is checked whether or not the tip of the "hand" is within a certain distance (step T7), and if it is not within the certain distance, it is checked whether or not the "clear key" on the virtual keyboard is designated (step T8). If the "clear key" is specified, the stored keyboard data is cleared, the page is turned over after the initial state is set.

On the other hand, in the processing of step T7,
If the tip of the "hand" approaches within a certain distance, the position of the HMD is detected (step T9). Incidentally, in the processing of steps T5 and T6, as shown in FIG. 24, it is checked whether or not the "hand" has performed an action of turning over the pages of the book. In addition, FIG.
It is a figure for demonstrating a page turning operation.

In the processing of step T9, when the position of the HMD is detected, the display position is calculated based on the HMD position and the position of the "hand" (step T10), and the corresponding page is calculated based on the calculated display position. Is displayed on the display unit 48 by the page turning control circuit 50 (step T11).

Next, it is checked whether or not the tip of the "hand" has moved substantially horizontally in the opposite direction to the previous movement direction (step T12). If it has not moved horizontally in the opposite direction, the "hand" is checked. It is checked whether or not the tip position of has moved for a certain time or longer (step T13). If no movement has been detected for a certain time, the processing is ended. If movement is detected, the processing from step T9 is repeatedly executed. It

In the processing of step T12,
If the tip of the “hand” moves horizontally in the opposite direction,
The movement stop of the "hand" is checked (step T14),
The process is repeated from step T9 until the movement is stopped. When the movement is stopped, the display data of each page is replaced according to the moving direction to display the current display data (step T15). The process of is repeated.

Therefore, in the page turning control processing in this embodiment, when the forward page turning switching display as shown on the left side of FIG.
4 (a), the operation shown in FIG.
When performing page turning switching display in the opposite direction as shown on the right side of FIG. 24, by performing an operation as shown in FIG. 24B, if the “hand” is separated by a certain distance from the HMD, The data is regarded as a pinch, and if the "hand" is moved within a certain distance, it is regarded as turning the page and the displayed data is converted to the page turning display. The data is transferred to the data converter 71.

When the "hand" moves substantially horizontally in the opposite direction, it is considered that the data of the next page is in the knob for turning the next page, and nothing is done until the movement is stopped. Then, when the movement is stopped, the data in each display data storage unit in which the display data is stored corresponding to each page is shifted based on the current display data in order to turn the next page. This is because the turned-over display data becomes the current display data when the page turning is completed.

As described above, in this embodiment, the display data displayed by the display unit 48 can be switched based on the movement of the "hand" detected by the movement distance detection.

Therefore, the display data can be switched based on the movement of the "hand", which allows the HM
The display data can be switched based on the input from the outside.

In the above embodiment, when the display data to be moved or the virtual keyboard overlap, the display data at the original position is moved by redrawing at a predetermined distance (offset). , But not limited to
The display data to be moved may be moved by shifting, and in this case, both the display data to be moved and other display data that overlap may be shifted and moved.

In the above embodiment, the eye data is
For example, if the ratio of pigment distribution between the black and white parts of the eye, the shape of the peripheral part of the eye (eyebrow shape, eyelid shape, shape of the whole eye, etc.), or the location of blood vessels can be specified and it is difficult to hyperemia A blood vessel pattern or the like of the eye may be considered, but the present invention is not limited to this. Further, after the shutter LCD 34 is in a light-shielding state and the inside of the device case 3 is darkened, light of a constant illuminance (for example, a fluorescent lamp) is applied. The operator may be recognized by detecting the respective proportions of the pupil sphincter muscle, pupil dilated muscle, and lens body in the operator's pupil by irradiation.

Further, in the above embodiment, the front position of the HMD is made transparent so that the outside scenery can be directly seen. However, the front scenery can be seen directly when the HMD is mounted. Alternatively, the HMD may be of a type that grasps the scenery in front by incorporating various units that combine and display the image generated by the display unit and the scenery outside.

[0152]

According to the present invention, in the invention described in claim 1, the display data displayed by the data display means can be switched based on the movement of the designating means detected by the movement distance detecting means.

In this case, as in the second aspect of the invention, the display data is switched according to the moving direction of the designating means by detecting the moving direction of the designating means by the moving distance detecting means. And claim 3
The invention according to claim 4, wherein the display area displayed by the data display means is divided into two left and right screens, and the display is switched by the display data switching means for each of the divided screens. As in the invention, it is possible to switch the display data based on the moving direction detected by the moving distance detecting means, for example, as if turning the pages of a book.

Therefore, based on the movement of the designating means,
The display data can be switched, which allows the H
Display data can be switched based on an input from outside the MD.

[Brief description of drawings]

FIG. 1 is a diagram of a data processing device according to an embodiment of the present invention viewed obliquely from the front.

FIG. 2 is a diagram of the data processing apparatus according to the present exemplary embodiment as viewed obliquely from the rear.

FIG. 3 is a cross-sectional view of the data processing device according to the present embodiment as viewed from the lateral direction.

FIG. 4 is a cross-sectional view of the data processing device according to the present embodiment as viewed from the rear.

FIG. 5 is a block diagram showing the overall configuration of a data processing apparatus of this embodiment.

FIG. 6 is a diagram for explaining angle detection of the data processing device of this embodiment.

FIG. 7 is a diagram for explaining direction detection of the data processing device according to the present embodiment.

FIG. 8 is a flowchart of an operator recognition process when the data processing device according to the present embodiment is activated.

FIG. 9 is a diagram showing an initial menu screen and a data menu screen.

FIG. 10 is a flowchart showing a data input / edit process.

FIG. 11 is a flowchart showing a keyboard designation process in FIG.

FIG. 12 is a flowchart showing an input designation process in FIG.

FIG. 13 is a diagram showing an actual display example on the HMD according to the present embodiment.

FIG. 14 is a flowchart showing a display process executed at regular time intervals.

FIG. 15 is a diagram showing a display example of a virtual keyboard and display data.

FIG. 16 is a flowchart showing a display data moving process.

FIG. 17 is a flowchart showing a placement process.

FIG. 18 is a flowchart showing a virtual keyboard moving process.

FIG. 19 is a flow chart of printout processing of the data processing apparatus of this embodiment.

FIG. 20 is a block diagram showing a main configuration of a page turning control circuit.

FIG. 21 is a diagram for explaining the page turning control process of the present embodiment.

FIG. 22 is a flowchart showing a page turning control process.

FIG. 23 is a diagram showing a display example of display data.

FIG. 24 is a diagram for explaining a page turning operation.

[Explanation of symbols]

 1 Data Processing Device 2 Optical System Display Unit 3 Equipment Case 4 Headband 5 Rod Antenna 6 Power On / Off Switch 7 Designation Key 8 Menu Change Down Instruction Key 9 Menu Change Up Instruction Key 10R Finder 10L Finder 11 Support 12R Earphone 12L Earphone 13 Mounting Detection Sensor 21 Backlight 22R LCD 22L LCD 23R Concave Lens 23L Concave Lens 31 Upper Glass 32R Convex Lens 32L Convex Lens 33 Front Glass 34 Shutter LCD 35 Reflective Magic Mirror 36R CCD 36L CCD 41 CPU 42 ROM 43 Data Memory 44 Angle Sensor 45 Direction Sensor 45 46 infrared sensor 47 input unit 48 display unit 49 display drive unit 50 page turning control circuit 51 wireless transmission control unit 61 L / R Separated circuit 62R R display controller 62L L display controller 63R R memory 63L L memory 71 Page turning display data conversion unit 72 Previous display data first storage unit 73 Previous display data second storage unit 74 Current display data first storage Part 75 Current display data second storage unit 76 Rear display data first storage unit 77 Rear display data second storage unit 101 Printer 102 External storage device

Continuation of front page (51) Int.Cl. ⁶ Identification code Office reference number FI Technical display area G06T 1/00

Claims (4)

[Claims] 1. A data processing device comprising a display means built in a housing, wherein data display means for displaying predetermined data on the display means, and display data displayed by the data display means. Of the designated position virtually, a position detecting unit that detects the virtual designated position designated by the designating unit, and a position of the designating unit detected by the position detecting unit within a predetermined distance. And a display data switching unit that switches display data displayed by the data display unit based on a detection result of the moving distance detecting unit. And data processing device. 2. The data processing apparatus according to claim 1, wherein the moving distance detecting means also detects the moving direction of the designating means when detecting the movement of the designating means. 3. The display data switching means divides a predetermined display area displayed by the data display means into two left and right screens, and switches a display image for each of the divided screens. 2. The data processing device according to 2. 4. The data processing apparatus according to claim 2, wherein the data switching means switches the display image based on the moving direction detected by the moving distance detecting means.