JPH0713690A - Data processor - Google Patents

Abstract

PURPOSE:To display desired display data when an HMD, etc., is used. CONSTITUTION:This processor is a data processor equipped with a builtin display means 47 in a cabinet, and it is provided with a data display means 48 which displays prescribed data by the display means 47, a designation means 32 which designates the arbitrary position of display data displayed by the data display means 48 virtually. and position detecting means 26R, 26L, 27R, and 27L which detect a virtual designation position designated by the designation means 32. Furthermore, it is equipped with a data specifying means 41 which calculates the designated position of the display data in accordance with the position detected by the position detecting means 26R, 26L, 27R, and 27L, and specifies the prescribed data in the display data.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data processing device, and more particularly to a head-mounted display device (hereinafter referred to as H-display device).
MD: Head Mounted Display).

[0002]

2. Description of the Related Art Conventional data processing devices include, for example, engineering workstations (hereinafter, EW).
S: Engineering Work Station), personal computer (hereinafter abbreviated as personal computer), word processor (word processor; hereinafter abbreviated as word processor), and the like.

Usually, these data processing devices are provided with a keyboard as an input device and a display device as an output device, and processing corresponding to a key command input from the keyboard is performed in the data processing device. The processing result is output to the display device.

A typical display device in a data processing device is a CRT (Cathode Ray Tube).
Display device and liquid crystal display device (hereinafter, LCD: Liquid Cry
stalDisplay).

The CRT display device is generally used as a display device in, for example, a desktop type EWS, a personal computer, a word processor, etc., and a large screen display device can be obtained at a relatively low cost. When a large-screen display device is obtained, the size of the CRT display device itself becomes large, and the installation place is limited.

On the other hand, the LCD has low power consumption and is lightweight.
Since it is thin, it is widely used as a display device for laptop or notebook type personal computers, word processors, etc. Recently, it has been used as a display device for EWS etc. because of its large screen and high definition. ing.

However, it is generally difficult for an LCD to have a large screen, and when the screen size increases, the cost is higher than that size.

Therefore, the HMD has been attracting attention as a display device that can have a large screen regardless of the installation location.

The HMD is a system in which images are displayed on two liquid crystal displays placed in front of the eyes, and the vergence angle and the focal length are set to predetermined values so that both eyes can be seen like glasses or binoculars. It gives separate images and displays them as if they were looking at a large-screen display device with a small device. For example, the left and right liquid crystal displays are tilted 1.5 degrees with respect to the image forming surface, and are displayed at 3 degrees. Given that a person with a distance between their eyes of 65 mm (average value for Japanese people) gazes at about 1 m ahead when a convergence angle of 2 is given, a 2 inch type liquid crystal display and a lens with a focal length of about 20 mm are used. If an angle of view of 30 degrees is realized by combining the above, an optical system having a display screen of about 20 inches 1 m ahead can be obtained.

Since the HMD has such advantages, it can be used for movie watching, medical treatment by pseudo 3D display (stereoscopic display),
It is expected to be used in the fields of CAD and architecture.

[0011]

CRT as an output device
In a conventional data processing device that includes a display device such as an LCD or an LCD and a keyboard as an input device, key input required for data processing can be performed from the keyboard, but a data processing device that uses an HMD as a display device is used. Considering this, since the HMD is configured to be mounted on the operator's head and used, there is a possibility that the following problems may occur.

That is, in the above-mentioned HMD, due to the nature of mounting the display device on the head, an operator other than a person who is capable of complete blind touch input,
Since it is not possible to use a keyboard or the like which is a general input device, when the HMD is mounted, for example, only a pointing device that can be used by a simple operation such as a joystick can be operated.

Although it is conceivable to provide a key input section on the HMD itself, it is desirable that the HMD be small and lightweight for the purpose of mounting it on the head, and it is not preferable to provide various types of keys.

An object of the present invention is to enable desired display data to be specified when using an HMD or the like.

[0015]

According to a first aspect of the present invention, there is provided a data processing device comprising a display means built in a housing, the data display means displaying predetermined data on the display means. A designation unit that virtually designates an arbitrary position in the display data displayed by the data display unit, a position detection unit that detects the virtual designated position designated by the designation unit, and a position detection unit. Data specifying means for calculating a specified position of the display data corresponding to the detected position and specifying predetermined data in the display data is provided.

In this case, as described in claim 2, the designating means includes a position designating part provided outside the housing, and a relative position detecting part for detecting a relative position of the position designating part with respect to the housing. It is effective to have a pointer display unit that displays a predetermined pointer image in the display data displayed by the data display unit based on the relative position detected by the relative position detection unit.

[0017]

According to the first aspect of the invention, with respect to the display data displayed by the data display means, an arbitrary position is virtually designated by the designating means, and the virtual designated position is detected by the position detecting means. Then, the predetermined data in the display data is specified based on the specified position detected by the data specifying means.

In this case, as in the second aspect of the invention, the relative position of the position designating unit with respect to the housing is detected,
By displaying the predetermined pointer image in the display data displayed by the data display means based on the detected relative position, the predetermined data in the display data can be easily specified.

[0019]

EXAMPLES Examples will be described below with reference to FIGS.

1 to 18 are views showing an embodiment of a data processing apparatus according to the present invention.

First, the structure will be described. 1 and 2 are external views of the data processing apparatus of the present invention, and FIG. 1 is a view of the data processing apparatus of the present embodiment as seen obliquely from the front.
[FIG. 3] is a diagram of the data processing device of the present embodiment as seen obliquely from the rear.

The data processing device 1 in this embodiment is H
HMD built in a single display device by MD
Since the data processing apparatus 1 is an integrated type and does not include a key input means such as a keyboard, a virtual keyboard can be input by displaying a virtual keyboard on the HMD.

1 and 2, the data processing device 1
Is a device case (housing) 2 that incorporates various units such as a display unit that generates various data processed in the data processing as a display image; Headband 3
A backlight unit 4 that generates backlight light for the display unit is detachably provided on the upper part of the device case 2, and the data processing device 1 can obtain sufficient illuminance of external light. When used in a place, it can be used even with the backlight unit 4 removed.

As shown in FIG. 1, the equipment case 2 is provided with a rod antenna 5, a power ON / OFF switch 6, a designation key 7, a menu switching down instruction key 8D, and a menu switching up instruction key 8U. As shown in FIG. 2, viewfinders 9R and 9L are formed.
Further, as shown in FIG. 1, the headband 3 is provided with a support portion 11, earphones 12R and 12L, and
As shown in FIG. 2, a mounting detection sensor 13 is provided.

The rod antenna 5 is an expandable antenna for converting various data processed in the data processing device 1 into a radio wave and transmitting the same, and the power source on / off switch 6 is a power source unit (not shown). ) Is a switch for selecting whether or not to energize the data processing device 1 with the power supply voltage supplied from (4), and the designation key 7 is used to confirm the operator who will use the data processing device 1 It is a switch to start. The antenna to be used is not limited to the retractable rod antenna 5 as in this embodiment, but may be a helical antenna, for example, and an antenna suitable for the wavelength band of the radio wave used can be used.

The menu switching down instruction key 8D and the menu switching up instruction key 8U are key switches for switching the menu contents of the menu screen displayed by the display unit.
D switches the menu contents in the forward direction (for example, the direction in which the frame cursor in the menu screen is moved downward),
The menu switching up instruction key 8U switches the menu contents in the opposite direction to the menu switching down instruction key 8D (for example, the direction in which the frame cursor in the menu screen is moved upward).

The viewfinders 9R and 9L are magnifying lenses 10R and 1 made of glass or plastic optical lenses.
A support window 11 supports and fixes the equipment case 2 and the headband 3 to each other.

The earphones 12R and 12L are for listening to error sounds and guidance information, and the wearing detection sensor 13 is a sensor for detecting whether or not the headband 3 is worn on the operator's head. .

FIG. 3 is a cross-sectional view of the data processing apparatus 1 according to the present embodiment as viewed from the lateral direction, and FIG. 4 is a cross-sectional view of the data processing apparatus 1 according to the present embodiment as viewed from above.

As shown in FIGS. 3 and 4, the device case 2 includes a diffuser plate 21 composed of a white acrylic plate or the like for diffusing light from the backlight unit 4 to balance the brightness, and a display. LC which is the active matrix color liquid crystal display unit of about 1 inch size which is the center of the unit
D22R and 22L, the reflection mirror 23 that reflects the display images from the LCDs 22R and 22L and guides them to the magnifying lenses 10R and 10L, and the driver IC (In
a circuit board 24 for mounting a control circuit in the device case 2 such as an integrated circuit), two CCDs (Charge Coupled Devices) 25R and 25L provided on the left and right sidewalls of the device case 2, and the device case An infrared light detection sensor 26 provided at the lower left and right side wall positions in 2 for detecting the position of an infrared light emitting pen (position designation unit) 32 described later.
R and 26L, and infrared light detection sensors 27R and 27L that are provided inside the infrared light detection sensors 26R and 26L and that detect a position designation input signal from the infrared light emitting pen 32.

As shown in FIG. 3, the backlight unit 4 is equipped with a backlight 31 composed of a direct type cold cathode fluorescent lamp (fluorescent lamp). When the backlight unit 4 is mounted in the device case 2, the power source is supplied from the device case 2 side. It emits light in response to the supply of voltage, and LCDs 22R, 22L are provided via the diffusion plate 21
It illuminates the back light. If the surroundings are sufficiently bright, the image can be viewed by introducing external light from the diffusion plate 21 without mounting the backlight unit 4.

The LCD 22 is illuminated by the backlight 31.
The image (light) obtained through R and 22L is reflected by the reflection mirror 23, and passes through the magnifying lenses 10R and 10L and the viewfinders 9R and 9L to the eyes ER and EL of the operator.
Reach. Furthermore, the image of the surrounding situation seen ahead is CC
Photographed by D25R, 25L, LCD as required
It is displayed on 22R and 22L and can be referred to by the operator.

FIG. 5 is a perspective view of the infrared light emitting pen 32.

The infrared light emitting pen 32 is an infrared light emitting pen 3.
2, the power switch 33 for starting the operation, the light emitting switch 34 for inputting the position designation, the light emitting portion 35 for confirming the position that starts continuous light emission when the power switch 33 is turned on, and the light emitting switch 34 is turned on. And a light emitting section 36 for inputting to emit light by operation.

That is, by the infrared light detection sensors 26R and 26L provided on the left and right side walls of the equipment case 2,
The position of the infrared light emitting pen 32 (relative position from the device case 2) is detected by infrared light continuously emitted from the light emitting unit 35 of the infrared light emitting pen 32 when the power switch 33 is turned on, and infrared light is detected. Sensor 26R, 26L
Infrared light (position designation input signal) emitted from the light emitting portion 36 is detected by turning on the light emitting switch 34 provided in the infrared light emitting pen 32 by the infrared light detecting sensors 27R and 27L provided inside the. As a result, it becomes possible to specify the position with the infrared light emitting pen 32.

FIG. 6 is a block diagram showing the overall configuration of the data processing device 1 of this embodiment.

The data processing apparatus 1 according to this embodiment includes a CPU 4
1, ROM 42, data memory 43, angle sensor 44,
An orientation sensor 45, an input unit 46, a display unit 47, a display drive unit 48, a wireless transmission control unit 49, and a CCD 25
R, 25L, infrared light detection sensors 26R, 26L, and infrared light detection sensors 27R, 27L are provided. Note that FIG.
Middle, B is CPU41, ROM42, data memory 4
3, angle sensor 44, direction sensor 45, input unit 46, display drive unit 48, wireless transmission control unit 49, CCD 25R, 2
5L, the infrared light detection sensors 26R and 26L, and the infrared light detection sensors 27R and 27L are buses that serve as signal transmission paths, respectively, 101 is a printer, and 102 is an external storage device.

The CPU 41 outputs various control signals for controlling each unit in the data processing apparatus 1 to each circuit via the bus B, and the CPU 41 is based on the input information from the input unit 46. The display drive unit 48 is controlled to perform a predetermined display on the display unit 47, and the wireless transmission control unit 4
9 to control various data processed in the data processing device 1 into radio waves to convert the data into a printer 101 or an external storage device 10.
The data required for 2 is transmitted.

The ROM 42 is a semiconductor memory that stores programs and data used in the data processing device 1.

The data memory 43 stores various data used in the processing executed by the CPU 41, that is, display data, image data of a "pen" which is a pointer image (hereinafter referred to as a pen cursor), and the data. It is a semiconductor memory having a work area serving as a work area in processing. The display data is display data of various menu screens displayed by the display unit 47, and the pen cursor is a virtual “pen” displayed by the display unit 47 based on the position detection of the infrared light emitting pen 32. Image data.

The angle sensor 44 is used in, for example, a navigation system or a gyro system,
It is a sensor for detecting how many times the HMD, which is the data processing device 1, is tilted from the horizontal state by detecting "shaking". In this embodiment, as shown in FIG. 7, a position rotated clockwise by a predetermined angle in FIG. 7 from the vertical direction is set as a reference point position (0 degree position), and at the 90 degree position, it is slightly It is set to face upward.

The azimuth sensor 45 is a sensor for detecting how many times the HMD, which is the data processing device 1, is rotating to the left or right from the reference position. In this embodiment, as shown in FIG. 8, the azimuth of “North” is the reference point position (N in FIG. 8).
Is set to detect the number of rotations clockwise in FIG.

The input section 46 is a virtual input section provided to enable key input in the data processing device 1. This is because the data processing apparatus 1 of the present embodiment is configured only by the display unit and does not have an input unit such as a keyboard specially, so that the display unit 47 displays the keyboard as the display data in the data memory 43. By displaying the pen cursors stored in the data memory 43 in an overlapping manner, the positional relationship is clarified to enable key input.

As shown in FIG. 3, the display unit 47 is an optical unit (diffusion plate 21, LCD) in the equipment case 2 for displaying two images corresponding to the left and right eyes of the operator.
22R, 22L, reflection mirror 23, magnifying lens 10R,
10 L, etc.) and an optical unit (backlight 31, etc.) in the backlight unit 4 and fixed to a point where two images are combined, so that the left and right viewfinders 9 can be fixed.
The images viewed from R and 9L are combined and displayed in a natural state.

The display drive section 48 includes LCDs 22R and 22L.
Is a display driving unit that drives the LCDs 22R and 22L to display an image by the CP.
Image data generated based on the processing of U41 is displayed on the LCD.
The analog video signals (analog RGB signals) of predetermined bits corresponding to the number of pixels of 22R and 22L are converted, and the analog video signals are output to the LCDs 22R and 22L.

The wireless transmission control unit 49 is used for the data processing device 1.
Various data processed inside are converted into radio waves and transmitted. This is because the data processing apparatus 1 of the present embodiment is composed of only the display unit and the processed data cannot be output by the data processing apparatus 1 alone. Therefore, for example, when printing is performed, desired data to be printed out is desired. Is modulated by the radio transmission control unit 49 by frequency modulation or amplitude modulation, and this data is output as radio waves to the external printer 101. Similarly, when important data is stored, desired data to be stored is stored. It is converted into radio waves and output to the external storage device 102. In this case, it is possible to send and receive data to and from the same data processing device 1 as in the present embodiment, and further, a radio wave transmitted from one host station is transmitted by a plurality of data processing devices 1 having the same configuration as this embodiment. By receiving, it is possible to see a common image, and it is possible to use it for a conference, for example.

Next, the operation of this embodiment will be described.

FIG. 9 is a flow chart of operator recognition processing when the data processing apparatus of this embodiment is started.

The operator recognition processing by the data processing apparatus 1 will be described below with reference to FIG. Note that FIG.
The program corresponding to the processing operation of the CPU 41 shown in
It is stored in the ROM 42.

First, the data processing device 1 in this embodiment.
Is attached to the operator and the power on / off switch 6 is turned on, a menu screen is displayed on the display unit 47 of the HMD (step S1).

Then, in this state, the pressed state of the menu switching down instruction key 8D or the menu switching up instruction key 8U is checked (step S2), and the menu switching down instruction key 8D or the menu switching up instruction key 8U is pressed. In this case, the menu contents are switched based on the switching instruction (step S3), while in the process of step S2, the menu switching down instruction key 8
If D or the menu switching-up instruction key 8U is not pressed, then the pressed state of the designation key 7 is checked (step S4). If the designation key is not pressed, the processing from step S2 is repeated.

In the processing of step S4, the designated key 7
When is pressed, the screen is switched to the initial menu screen where each functional process is performed according to the designated menu (step S5).

As described above, when the use of the HMD is confirmed by the operator by the processing of steps S1 to S5, the initial menu screen as shown in FIG. 10A is displayed. By the way, in the example shown in FIG. 10A, the selection items are “data input / edit”, “data recognition input”,
The items “record” and “play” are displayed, and the frame cursor is in the item “data input / edit”. In this state, the menu switching down instruction key 8
When D is pressed, the frame cursor moves downward in the figure, and when the menu switching up instruction key 8U is pressed, the frame cursor moves upward in the figure and the target item is highlighted.

Here, as shown in FIG. 10A, when the designation key 7 is pressed with the frame cursor on the item of "data input / edit", the function of "data input / edit" is selected. A data menu screen as shown in FIG. 10B is displayed. In FIG. 10B, A is a layout window for indicating or specifying the layout of data to be displayed, and B is an operation for displaying the item selected on the initial menu screen of FIG. 10A. This is an item display window, which allows you to check the current operation contents.
C is a file name window for indicating the file name of the document or table data to be edited, and specifying the file name of the document or table data to be edited, and D is a keyboard specification window for specifying the virtual keyboard. is there.

In addition, "data recognition input" in FIG.
When the designation key 7 is pressed while the frame cursor is on the item of, the function of "data recognition input" is selected, and
A menu screen as shown in (a) is displayed. Similarly, when the designation key 7 is pressed in a state where the frame cursor is on the item of “recording” in FIG. 10A, the function of “recording” is selected and a screen as shown in FIG. 11B is displayed. Is displayed.

Next, the data input / editing process will be described with reference to the flow charts shown in FIGS. Note that FIG. 12 is a flowchart showing the data input / edit processing.

In the data menu screen shown in FIG. 10B, during the data input / edit processing, the virtual keyboard data is read from the data memory 43 and the next processing cannot be performed. Therefore, it is first checked whether or not the virtual keyboard is displayed by designation (step S11). If the virtual keyboard is displayed, the process proceeds to the next step. On the other hand, if the virtual keyboard is not displayed, it will be described later. The keyboard designation process is performed (step S12), and the virtual keyboard is displayed.

A "read key" is displayed on this virtual keyboard.
Is prepared, and it is checked whether or not the "read key" has been selected (step S13).

When the "read key" is selected in the process of step S13, a list of storage file names is displayed in a window (step S14), and when the "cursor key" on the virtual keyboard is operated in this state ( In step S15), the cursor position is changed, and the display content is updated to the file name at the changed cursor position (step S16).

Then, it is checked whether or not the window is designated for input (step S17).
The file data corresponding to the designated position is read (step S18), the previous display is erased, and the layout window is displayed again (step S19).

Here, the presence / absence of the window input designation is checked again (step S20), and if there is the input designation, file data arrangement processing is performed (step S2).
1) The arranged file data is displayed and the file name window is deleted (step S2).
2) The data input / editing process ends.

On the other hand, in the processes of steps S17 and S20, if no window input is designated, it is checked whether or not the "clear key" on the virtual keyboard is pressed (step S23), and the "clear key" is pressed. If is not pressed, the processing from step S14 is repeatedly executed, and if the "clear key" is pressed, the data input / edit processing is ended.

When the "read key" is not selected in the process of step S13, it is checked whether the "write key" prepared on the virtual keyboard is selected as with the "read key". (Step S
24).

In the process of step S24, if the "write key" is not selected, the process proceeds to another process, and if the "write key" is selected, a list of storage file names is displayed in a window. (Step S25), the presence or absence of input designation is checked (step S26).

If the input is designated, it is checked whether the display data is designated (step S2).
7) When the designation of the display data is confirmed, the contents of the designated display data are written in the corresponding file (step S28), and the data input / editing process ends.

If no input is designated in the process of step S26, it is asked whether or not a new file is created (step S29). If a new created file is input, a new file is created. (Step S3
0), the processing from step S27 is executed.

If it is not the input of a newly created file, it is determined that the file name is updated, and if the "cursor key" is operated (step S31), the cursor position is changed and the file at the changed cursor position is changed. The display content is updated to the name (step S32), and the processing from step S25 is executed.

In the process of step S41, if the "cursor key" is not operated, it is checked whether or not the "clear key" on the virtual keyboard is pressed (step S3).
3) If the "clear key" is not pressed, the process from step S25 is repeatedly executed, and if the "clear key" is pressed, the data input / edit process is ended.

FIG. 13 is a flow chart showing the keyboard designation process in FIG.

As shown in FIG. 13, in the keyboard designating process, first, an input designating process which will be described later is executed (step S41), and it is checked whether or not a keyboard window is designated in the input designating process (step S41). S42).

In the process of step S52, if the keyboard designation window is not designated, the presence / absence of designation of the "clear key" on the virtual keyboard is checked (step S43), and if the "clear key" is designated, it is stored. After clearing the keyboard data that has been set to the initial state (step S44), the processing from step S41 is repeatedly executed. Here, if the "clear key" is not designated, it is checked whether or not the data menu screen shown in FIG. 10B is designated (step S45), and the data menu screen is designated. The keyboard designation process ends.

When the data menu screen is not designated in the process of step S45,
It is checked whether the fixed flag is on (step S46). Here, when the fixed flag is on, the virtual keyboard is fixedly displayed at the center position of the display unit 47 regardless of the change in the visual field due to the movement of the HMD, and the fixed flag is off. In this case, just like a real keyboard, a virtual keyboard is placed in place,
The display mode is such that the keyboard enters or does not enter due to the movement of the HMD.

That is, when the fixed flag is on,
When it is determined that the virtual keyboard is in the fixed display mode, the processing from step S41 is repeatedly executed, and when the fixed flag is off, the angle sensor 4
The current angle of the HMD is detected by 4, and the current direction of the HMD is detected by the azimuth sensor 45, and the respective values (angle and azimuth) are displayed on the display unit 47 (step S47). When the HMD moves to enter the area where the keyboard should be displayed (step S4)
8), the virtual keyboard for the keyboard area is displayed on the display unit 4
7 (step S49), and if the keyboard area is not entered, the processing from step S41 is repeatedly executed.

On the other hand, when the keyboard designation window is designated in the process of step S42, whether or not the layout window is designated is checked (step S50).

The layout window is specified prior to the specification by the keyboard specification window. If the layout window is not specified, H
In order to display the virtual keyboard at the front center position wherever the MD position is facing, the keyboard data is stored in the storage position corresponding to the angle and direction of the current HMD (step S51), and the virtual keyboard display is fixed. Then, the mode is executed (step S52). At this time, the fixed flag is turned on (step S53), the layout position of the virtual keyboard is displayed (step S54), and the processing from step S41 is repeatedly executed.

When the layout window is designated, the data memory 43 corresponding to the layout position
Keyboard data is stored in (step S55), H
The mode in which the virtual keyboard is displayed only when the MD faces that position, and the process proceeds to step S54.

FIG. 14 is a flow chart showing the input designation processing in FIG.

As shown in FIG. 14, in the input designation processing,
First, it is checked whether the infrared light emitting pen 32 is powered on and continuously emitting light (step S61),
If no light emission is detected, the infrared light emitting pen 32 determines that the power is off and ends the process.

On the other hand, when continuous light emission from the infrared light emitting pen 32 is detected, the position of the infrared light emitting pen 32 (relative position from the device case 2) is detected based on the detected infrared light (step S62). The CPU 41 calculates the display position corresponding to the position of the infrared light emitting pen 32 (step S63).

Then, the pen cursor (pointer image) is displayed at the obtained display position (step S64).

Next, it is checked whether or not the infrared light is emitted by pressing the light emitting switch 34 of the infrared light emitting pen 32 (step S65). If no light emission is detected, the light emitting switch 34 of the infrared light emitting pen 32 is detected. Judges that it has not been pressed, and terminates the processing.

On the other hand, when the light emission from the infrared light emitting pen 32 is detected, the position of the infrared light emitting pen 32 is detected based on the detected infrared light, and the detected position and the position detected last time are detected. It is determined whether or not the difference is within a certain distance (step S66), and if it is within the certain distance, it is determined that there is no movement of the position designation by the infrared light emitting pen 32, and the process ends.

In the process of step S66, when the position of the infrared light emitting pen 32 moves by a certain distance or more from the position detected last time and the virtual keyboard and the pen cursor overlap, the key that overlaps the pen cursor on the virtual keyboard. Is changed (for example, the brightness is increased) (step S67) so that it is possible to know which key the pen cursor is touching. In this case, the display states of the virtual keyboard and the pen cursor are as shown in FIG.

When the input designation is completed, the key on the virtual keyboard corresponding to the position of the pen cursor, the window contents, etc. are stored in the data memory 43 in order to utilize various processing data in the input designation processing (step S68).

FIG. 15 is a diagram showing an actual display example on the HMD of this embodiment.

In FIG. 15, E is a display showing the direction of the HMD detected by the direction sensor 45, F is a display showing the angle of the HMD detected by the angle sensor 44, and G is a predetermined area. A layout window showing how specified documents and table data are arranged, H is a pen cursor, I is a virtual keyboard, J
Is a file name window, and K is currently open document data.

FIG. 16 is a flow chart showing a display process executed at regular time intervals.

In this display processing, first, the angle sensor 44
The angle and azimuth of the HMD are detected by the azimuth sensor 45 and the azimuth sensor 45 (step P1), the detected angle and azimuth are displayed on the display unit 47, and the display data in the data memory 43 corresponding to the current position of the HMD is displayed. The corresponding area is searched and the pointer is moved (step P2).

Then, it is checked whether or not there is data to be displayed in this corresponding area (step P3). If there is display data in the corresponding area, the process moves to the next process, and there is no display data in the corresponding area. In the case, the display processing is ended.

When there is display data in the corresponding area, the fixed flag indicating the current display mode is checked (step P3).

Here, when the fixed flag is on, it is determined that the current display mode is the keyboard fixed display, and the virtual keyboard is displayed at the center position of the display unit 47 (step P5).

Then, it is checked whether or not the display data to be displayed in the corresponding area overlaps the display of the virtual keyboard (step P6). If they overlap, only the display area for displaying the virtual keyboard is in the corresponding area. Move and display the display data.

Specifically, for example, as shown in FIG. 17A, when there is display data R0, R1, R2 between the display area pointer left position indicating the corresponding area and the display area pointer right position. As shown in FIG. 17B, the display data is moved upward in FIG. 17 by the amount of displaying the virtual keyboard, and the virtual keyboard is fixedly displayed (step P7).

On the other hand, when the fixed flag is off in the process of the step P4, the step P6 is executed again.
If there is no display data that overlaps the virtual keyboard in the display area in the processing of (1), the data in the corresponding area is displayed as it is (step P8). In this case, the fixed flag is turned on in the process of step P4, and
If it does not overlap the virtual keyboard, the virtual keyboard and the data in the corresponding area must be combined and displayed.

In the data processor 1 of this embodiment, H
The display data sequentially displayed based on the position of the MD is specified by the pen cursor as in the case of inputting the displayed data (in the case of the present embodiment, the display data is displayed at the tip of the pen cursor. It can be moved to another display position by pressing.

FIG. 18 is a flowchart showing the arrangement process.

When the display data already exists at the destination due to the above-mentioned movement of the display data and the display data overlap each other (step Q1), the display data overlapping the display data to be moved are rearranged. (Step Q2).

Next, as to the destination of the original display data by rearrangement, it is checked whether or not the display data to be moved do not overlap (step Q3), as in the case of the above-mentioned step Q1. Q
Returning to the processing of step 1, when they overlap, the overlapping display data are rearranged in a position where they do not overlap (step Q4), and the processing of the above steps Q3 and Q4 is repeatedly executed until there is no overlap.

In this case, when there are a plurality of overlapping data, the rearrangement is performed from the data having a high overlapping ratio.

By the way, for example, when the display data is moved when the display data is displayed in the full display area, the process between steps Q3 and Q4 is repeated because the overlapping state is not resolved forever. An infinite loop occurs. Therefore, in order to prevent the occurrence of an infinite loop, if the overlapping state is not resolved for a certain time or longer for some reason, an error is notified and the original state is restored.

As described above, in this embodiment, an arbitrary position can be virtually designated by the infrared light emitting pen 32 with respect to the display data displayed by the display unit 47, and the infrared light detecting sensors 26R and 26L can be further specified. And the infrared light detection sensors 27R and 27L to create a virtual infrared light emitting pen 3
The designated position by 2 is detected, and the predetermined data in the display data is specified based on the detected designated position.

Therefore, even when the HMD is used, it is possible to virtually input a keyboard, and desired data can be input without lowering the input capability.

In the above-mentioned embodiment, various units for synthesizing and displaying the image generated by the display unit and the outside scenery are built in, and it is possible to directly see the scenery in front with the HMD attached. However, the front position of the HMD may be a transmissive type so that the outside scenery can be directly seen.

[0104]

According to the present invention, in the invention described in claim 1, an arbitrary position can be virtually designated by the designating means with respect to the display data displayed by the data displaying means, and the virtual position can be virtually designated by the position detecting means. It is possible to detect the specified position and specify the predetermined data in the display data based on the specified position detected by the data specifying means.

In this case, as in the second aspect of the invention, the relative position of the position designating unit with respect to the housing is detected, and the display data displayed by the data display means is based on the detected relative position. By displaying the predetermined pointer image, it is possible to easily specify the predetermined data in the display data.

Therefore, when the HMD or the like is used, desired data can be input without lowering the input capability.

[Brief description of drawings]

FIG. 1 is a diagram of a data processing device according to an embodiment of the present invention viewed obliquely from the front.

FIG. 2 is a diagram of the data processing apparatus according to the present exemplary embodiment as viewed obliquely from the rear.

FIG. 3 is a cross-sectional view of the data processing device according to the present embodiment as viewed from the side.

FIG. 4 is a cross-sectional view of the data processing device according to the present embodiment as viewed from above.

FIG. 5 is a perspective view of an infrared light emitting pen.

FIG. 6 is a block diagram showing an overall configuration of a data processing device of this embodiment.

FIG. 7 is a diagram for explaining angle detection of the data processing device of the present embodiment.

FIG. 8 is a diagram for explaining direction detection of the data processing device according to the present embodiment.

FIG. 9 is a flowchart of operator recognition processing when the data processing apparatus according to the present embodiment is activated.

FIG. 10 is a diagram showing an initial menu screen and a data menu screen.

FIG. 11 is a diagram showing a data recognition input screen and a recording screen.

FIG. 12 is a flowchart showing a data input / edit process.

FIG. 13 is a flowchart showing a keyboard designation process in FIG.

FIG. 14 is a flowchart showing an input designation process in FIG.

FIG. 15 is a diagram showing an actual display example on the HMD of the present embodiment.

FIG. 16 is a flowchart showing a display process executed at regular time intervals.

FIG. 17 is a diagram showing a display example of a virtual keyboard and display data.

FIG. 18 is a flowchart showing a placement process.

[Explanation of symbols]

 1 Data processing device 2 Equipment case 3 Headband 4 Backlight unit 5 Rod antenna 6 Power on / off switch 7 Designation key 8D Menu switching down instruction key 8U Menu switching up instruction key 9R, 9L Finder 10R, 10L Magnifying lens 11 Support part 12R, 12L earphone 13 wearing detection sensor 21 diffusion plate 22R, 22L LCD 23 reflection mirror 24 circuit board 25R, 25L CCD 26R, 26L infrared light detection sensor 27R, 27L infrared light detection sensor 31 backlight 32 infrared light emitting pen 33 power supply Switch 34 Light emitting switch 35 Light emitting unit 36 Light emitting unit 41 CPU 42 ROM 43 Data memory 44 Angle sensor 45 Direction sensor 46 Input unit 47 Display unit 48 Display drive unit 49 Wireless transmission control unit 101 Print External storage device

Claims (2)

[Claims] 1. A data processing device comprising a display means incorporated in a housing, the data display means displaying predetermined data on the display means, and the display data displayed by the data display means. Specifying means for virtually specifying an arbitrary position, position detecting means for detecting a virtual specified position specified by the specifying means, and specification of the display data corresponding to the position detected by the position detecting means A data processing device comprising: a data specifying unit that calculates a position and specifies predetermined data in the display data. 2. The specifying means is a position specifying part provided outside the housing, a relative position detecting part for detecting a relative position of the position specifying part with respect to the housing, and the relative position detecting part. 2. The data processing device according to claim 1, further comprising: a pointer display unit that displays a predetermined pointer image in the display data displayed by the data display means based on the relative position.