JP5418480B2 - Image display device and image display method - Google Patents

Description

  The present invention relates to a technique for displaying the image by projecting image light representing an image to be displayed on the eyes of the observer, and in particular, when the observer visually recognizes the display image, The present invention relates to an improvement in technology for improving comfort felt from a display image.

  A direct-view image display device that displays the image by projecting image light representing the image to be displayed onto the eyes of the observer is already known (for example, see Patent Document 1). According to this image display device, an observer can directly observe an image without using a screen on which the image is projected.

  When direct-view image display devices are classified according to image light forming methods, they are spatially modulated, that is, the image light forming unit uses a light source and liquid crystal or organic material that forms the image light using incident light from the light source. An EL that is configured to include an element that operates in accordance with the image signal, and a scanning type, that is, a light source that emits light having an intensity corresponding to the image signal by the image light forming unit. And an optical scanning unit that forms the image light by scanning incident light from the light source.

  Further, when direct-view type image display devices are classified according to the display image observation method, see-through type, that is, an observer can observe the actual outside world superimposed on the display image by the image display device. And the sealed type, that is, the type that blocks the incident light from the actual outside world and allows the observer to observe only the display image.

  In addition, the direct-view type image display device is classified into a head mount type, that is, a device that is mounted on the head of the observer and moves integrally with the head, and a peep type, that is, an observation type, when classified according to the installation method. That are installed independently of the user and represented by electronic viewfinders for imaging devices such as digital still cameras and digital video camcorders, as well as stationary installations, that is, fixed members (floors, walls, tables, etc.) Are classified as those in which image display devices are installed.

  Further, when the direct-view image display device is classified according to the number of eyes used by the observer, a one-eye type that projects image light representing the image onto one eye of the observer and displays the image to the observer; There is a binocular system in which the image is projected on both eyes of the observer and the image is displayed to the observer.

  In any variation, in an application in which an image is displayed using a direct-view image display device, the comfort that the observer feels from the display image when the viewer visually recognizes the display image by the image display device is improved. There is a problem of wanting. An example of this problem is disclosed in Patent Document 1. Specifically, when a display image includes a character, the character is desired to be displayed in an easy-to-read manner.

JP 2007-142542 A

  In Patent Document 1, the reason why the displayed characters are difficult to read is the MTF (Modulation Transfer Function) characteristic of the image display device, that is, the position where the characters are displayed in the image display area of the image display device. It states that the image quality changes depending on whether it is located at the center or the periphery of the display area. Patent Document 1 further describes an image display area of characters to be displayed in order to solve the problem of displaying characters in an easy-to-read manner, that is, improving the comfort that the observer feels from the display image. And changing the position of the character and its character size so that the MTF characteristics are compensated.

  On the other hand, the present inventors conducted experiments with the details described later using an image display device, and the comfort that the observer feels from the display image when observing the display image is I noticed that it also depends on the characteristics. That is, it has been found that the comfort felt from the display image depends not only on the display characteristics of the display device but also on the human perceptual characteristics.

  Furthermore, when the display image includes a display object (for example, an information item to be described later), the inventors of the present invention, due to human perceptual characteristics, depending on the position where the display object is displayed in the image display area, It was also noticed that the comfort (for example, ease of viewing) felt by the observer observing the same display object is different. I realized that even though the image quality of the display object itself is the same, the comfort felt by the observer differs depending on the position of the display object.

  Based on such knowledge, the present invention is a technique for displaying the image by projecting image light representing the image to be displayed on the eye of the observer, and when the viewer visually recognizes the display image. Another object of the present invention is to provide a device that can easily improve the comfort that an observer feels from a display image.

  The following aspects are obtained by the present invention. Each aspect is divided into sections, each section is numbered, and is described in the form of quoting the section numbers of other sections as necessary. In this way, each section is referred to the section number of the other section. By describing in the format, the technical features described in each section can be appropriately made independent according to the properties.

(1) An image display device that displays an image by projecting image light representing an image including one or more display objects onto an observer's eye,
A plurality of sub-areas in which the image display area observed from the observer side is divided so as to be positioned in front of a position biased toward the left eye side or right eye side of the observer from the center line of the observer's body When each display object is displayed, each display object has the same subjective evaluation value of visual comfort that the viewer feels as the viewer visually recognizes each displayed object. A sub-area selection unit that selects a display sub-area higher than that displayed in the area,
An image display device comprising: an image light generation unit that generates the image light so that each display object is displayed in the selected display subarea.

  In this section, “the image display area that is observed from the viewer's side so as to be located in front of a position biased toward the left eye side or right eye side of the viewer's body from the center line of the viewer” For example, when the image display device is the above-described one-eye type, the image display device is positioned in front of one eye of the observer, while when the image display device is the above-described binocular type, the observer Although it is not located in front of the left eye or right eye, it is located in front of a position biased toward the left eye side or right eye side of the observer from the center line of the observer's body.

  Further, in this section, the “sub-area selection unit” is, for example, always fixedly selecting the same sub-area as a display sub-area, or is higher as the importance is higher according to the importance of the display object to be displayed. The display subarea may be variably selected so that the display subarea having the comfort level is selected.

(2) The sub-area selection unit selects, as the display sub-area, the one close to the observer's nose with respect to the center position of the image display area among the plurality of sub-areas. Image display device.

(3) The plurality of sub-areas form a matrix of 3 rows and 3 columns by dividing the image display area into three vertically and horizontally, whereby the number of the plurality of sub-areas is nine. Yes,
The position of each sub-area is an integer i representing a row number (i is 1 or more and 3 or less), and increases from the upper side to the lower side, and an integer j (j is 1 or more and 3 or less) representing a column number. ), Which is expressed as (i, j) using the one that increases from the side closer to the observer's nose to the side closer to one eye of the observer's both ears. ,
The sub-area selection unit selects a predetermined one of a plurality of sub-areas (1, 2), (2, 1), (2, 2), (3, 1) and (3, 2). The image display device according to item (2), which is selected as the display subarea.

  In this section, the position of each sub-area is defined using the notation (i, j), but this definition is merely a definition for convenience of explanation. If an image display device uses a different notation method, for example, the numbers 1, 2, 3, 4, 5, 6, 7, 8, and 9 as illustrated later in the column of the embodiment, When implemented in a manner that defines the location of an area, it should not be construed that the image display device departs from the scope of the present invention solely because of the difference in notation.

(4) The sub-area selection unit selects a predetermined one of the plurality of sub-areas (2, 2) and (3, 2) as the display sub-area. Image display device.

(5) The image display device according to (4), wherein the sub-area selection unit selects the sub-area (3, 2) as the display sub-area.

(6) The image display device displays the image by projecting the image light onto one eye of an observer.
The image display area is disposed in front of the one eye,
The image display device further includes an identification unit that identifies whether the one eye is an observer's left eye or right eye,
The sub-area selection unit is configured such that the relative position of the display sub-area with respect to the image display area is determined based on whether the one eye is identified as the left eye or the right eye by the identification unit. The image display device according to any one of (1) to (5), wherein the image display device is selected so as to change so as to be symmetrical with respect to a center line of the observer's body between cases identified as being present.

(7) An image display method for displaying the image by projecting image light representing an image including one or a plurality of display objects onto an observer's eye,
A plurality of sub-areas in which the image display area observed from the observer side is divided so as to be positioned in front of a position biased toward the left eye side or right eye side of the observer from the center line of the observer's body When each display object is displayed, each display object has the same subjective evaluation value of visual comfort that the viewer feels as the viewer visually recognizes each displayed object. A sub-area selection step for selecting a display sub-area that is higher than that displayed in the area;
An image light generation method comprising: an image light generation step of generating the image light so that each display object is displayed in the selected display subarea.

(8) A program executed by a computer to implement the image display method according to (7).

  The inventors of the present invention (described in detail later), based on the observer's body centerline, the observer's left eye or the right eye side of the observer When an image display device of a type having an image display area that is observed from the side displays a display object in the image display area, the viewer feels that the displayed display object is visually recognized by the observer It has been found that the subjective subjective evaluation value of comfort (hereinafter referred to as “comfort level”) is not uniformly distributed throughout the image display area, but varies depending on the display position of the display object. In the image display area, the fact that there is a display position with a high degree of comfort and a display position with a low degree of comfort, that is, the property that the degree of comfort depends on the display position of the display object has been found.

  Based on this knowledge, according to the present invention, a plurality of sub-divided image display areas located in front of a position biased toward the left eye side or the right eye side of the observer from the center line of the observer's body Among the areas, those having higher comfort levels than the other sub-areas are selected as display sub-areas, and the image light is generated and displayed so that each display object is displayed in the selected display sub-area. Is formed.

  Therefore, according to the present invention, it is easy to improve the comfort that the observer feels from the display image when the observer visually recognizes the display image.

FIG. 1 is a plan view showing an appearance of a see-through type head mounted display device (hereinafter abbreviated as “HMD”) according to an embodiment of the present invention. FIG. 2 is a system diagram showing an internal configuration of the HMD shown in FIG. FIG. 3 is a functional block diagram showing the control unit shown in FIG. FIG. 4 shows a display mode of information items that enables the user to properly view the information items when the user observes the information items as objects displayed by the HMD shown in FIG. It is a figure which shows the display result of the information item used in the experiment which the present inventors performed in order to acquire. FIG. 5 is a diagram showing display results of five types of external patterns used for simulating an image of the real external environment in the experiment. FIG. 6A is a graph showing a result of an experiment in the above-described experiment, and is a graph showing the relationship between the type of font size of the information item and the comfort level that the subject feels while observing the information item. These are the other experimental results in the said experiment, Comprising: It is a graph showing the relationship between each position where the said information item is displayed in the image display area of HMD, and the comfort level which a test subject feels while observing an information item. FIG. 7A is still another experimental result in the experiment, and is a graph showing the relationship between the type of the external pattern and the comfort level that the subject feels while observing the information item, and FIG. FIG. 7 is a graph showing the relationship between the type of external pattern, the number of information items / type of information items, and the degree of comfort that the subject feels while observing the information items. c) is a graph showing the relationship between the type of external pattern, the number of colors of information items, and the comfort level that the subject feels while observing the information items. FIG. 8 is a flowchart conceptually showing a subarea selection program stored in the program ROM shown in FIG. FIG. 9A is a diagram showing the contents of the left-eye map in step S2 shown in FIG. 8 in a geometrically associated manner with the HMD display space, and FIG. 9B is a step shown in FIG. It is a figure which shows the content of the map for right eyes in S3 geometrically linked | related with the display space of HMD. FIG. 10 is a flowchart conceptually showing the mode response program stored in the program ROM shown in FIG. FIG. 11 is a flowchart conceptually showing the color classification number determining program stored in the program ROM shown in FIG. FIG. 12A is a diagram for explaining an example of the divided display in steps S112 and S113 shown in FIG. 11, and FIG. 12B is an example of the divided display in steps S125 and S126 shown in FIG. It is a figure for demonstrating. FIG. 13 is a flowchart conceptually showing an external response program stored in the program ROM shown in FIG. FIG. 14 is a flowchart conceptually and temporally showing an image display method according to this embodiment, which displays an image using the HMD shown in FIG.

  FIG. 1 is a plan view of a see-through type head mounted display device (hereinafter abbreviated as “HMD”) 10 according to an exemplary embodiment of the present invention. The HMD 10 includes a projection unit 12 that projects image light representing an image onto the eyes of an observer. The projection unit 12 is attached to the head of the observer by the head mount device 14 while the observer is wearing spectacles (not shown).

  The head mount device 14 includes a frame 16 and an attachment device 18. The frame 16 is attached to the observer's head in a state where the observer is wearing eyeglasses (not shown) and is put on both ears of the observer. The projection unit 12 is mounted on a part of the frame 16 via an attachment device 18.

  Next, the projection unit 12 and the control unit 20 for controlling the projection unit 12 will be described with reference to FIGS. 1 to 3.

  First, the outline of the projection unit 12 will be described. FIG. 1 shows the projection unit 12 in a plan view. The projection unit 12 is monocular, and projects image light representing an image onto one eye of the observer to display the image to the observer. Furthermore, the projection unit 12 is of a retinal scanning type, and projects the light flux from the light source onto the observer's retina and scans the projected light flux on the retina so that the observer observes the image as a virtual image. Make it possible. Furthermore, the projection unit 12 is a see-through type, and allows the observer to observe the display image superimposed on the actual outside world.

  In addition, in the present embodiment, the projection unit 12 is a monocular type, a retinal scanning type, and a see-through type. Using a spatial modulation element, etc., each pixel is spatially modulated, and the modulated light is changed to a type that projects onto the retina of the observer, or the real external environment is observed in parallel with the display image observation It is possible to change to a closed type that cannot be done.

  In addition, this embodiment is an example when the present invention is applied to a head-mounted image display device. However, the present invention can be applied to a peep-type image display device. The present invention can also be applied to an image display device.

  Next, the control unit 20 will be described. A control unit 20 is connected to the projection unit 12 via a cable 22 as shown in FIGS. The cable 22 includes a control line that supplies a control signal, a power line that supplies electric power, and an optical fiber 82 (described in detail later) that transmits light. While the projection unit 12 is mounted on the observer's head, the control unit 20 is mounted on a portion of the observer other than the head (for example, the waist).

  As shown in FIG. 2, the control unit 20 includes a light source unit 24 that generates and emits linear image light (for example, RGB color laser beam). The configuration of the light source unit 24 will be described in detail later. The control unit 20 further includes a signal processing circuit 25 mainly composed of a computer.

As conceptually shown in the functional block diagram of FIG. 3, the signal processing circuit 25 includes a CPU (Central Processing Unit) 26 as a processor, a program ROM (Read Only Memory) 27 and a flash ROM 28 as nonvolatile memories, A RAM ( Random Access Memory) 29 as a volatile memory, an operation unit (for example, key, button, touch panel) 30, an input / output interface (indicated simply as “I / F” in FIG. 3) 31, An external input / output terminal 32 and a bus 33 for connecting these components to each other are provided.

  An external device (not shown) such as a personal computer is connected to the external input / output terminal 32, and a video signal is input to the signal processing circuit 25 from the external device via the external input / output terminal 32. The video signal represents display content (for example, still image content or moving image content) to be reproduced by the projection unit 12. The input display content is stored in the flash ROM 28.

  As will be described in detail later, the projection unit 12 is connected to the external input / output terminal 32. The HMD 10 has a camera (for example, a CCD (Charge-Coupled Device) camera) 23 mounted on the upper surface (or other position) of the frame 16 shown in FIG. The camera 23 photographs the actual outside world that the observer observes together with the display image (display content). The camera 23 is also connected to the external input / output terminal 32 so that the signal processing circuit 25 captures a signal representing the imaging result of the camera 23.

  In order to intensity-modulate the image light for each component light (RGB), the signal processing circuit 25 generates an R luminance signal representing the luminance of the red (R) laser beam (component image light) from the input video signal, and green (G) A G luminance signal representing the luminance of the laser beam (component image light) and a B luminance signal representing the luminance of the blue (B) laser beam (component image light) are generated. The signal processing circuit 25 also generates a horizontal synchronization signal and a vertical synchronization signal that are used as a reference for horizontal scanning and vertical scanning described later.

  Next, the light source unit 24 will be described in detail with reference to FIG. The light source unit 24 includes three lasers 34, 36, 38, three collimator lenses 40, 42, 44, three dichroic mirrors 50, 52, 54, and a coupling optical system 56.

  The three lasers 34, 36, and 38 are an R laser 34 that generates a red laser beam, a G laser 36 that generates a green laser beam, and a B laser 38 that generates a blue laser beam.

  Any of the lasers 34, 36, and 38 can be configured as, for example, a semiconductor laser or a solid-state laser. However, since the semiconductor laser can modulate the intensity by itself, the solid-state laser cannot. Therefore, when each of the lasers 34, 36, and 38 is configured as a solid-state laser, the intensity modulator is used. It is necessary to add.

  The three collimator lenses 40, 42, and 44 are lenses that collimate the three color laser beams emitted from the three lasers 34, 36, and 38, respectively. The three dichroic mirrors 50, 52, and 54 are wavelength selective to the three color laser beams in order to combine the three color laser beams emitted from the three collimator lenses 40, 42, and 44 with each other. Reflect and transmit.

  These three color laser beams are combined with each other in one representative dichroic mirror representing the dichroic mirrors 50, 52, and 54. In the present embodiment, the dichroic mirror 50 is selected as the representative dichroic mirror. The laser beam combined in the dichroic mirror 50 is incident on the combining optical system 56 as a combined laser beam (combined image light) and is condensed.

  As shown in FIG. 2, the three lasers 34, 36, and 38 are electrically connected to the signal processing circuit 25 through three laser drivers 70, 72, and 74, respectively. The signal processing circuit 25 modulates the intensity of the laser beam emitted from each laser 34, 36, 38 via the corresponding laser driver 70, 72, 74 based on the R luminance signal, G luminance signal, and B luminance signal. To do.

  As shown in FIG. 2, a laser beam (combined image light; hereinafter simply referred to as “laser beam”) emitted from the coupling optical system 56 is collimated in the projection unit 12 via an optical fiber 82 as an optical transmission medium. It is transmitted to the lens 84. The laser beam collimated and emitted from the collimator lens 84 enters the scanning unit 88 in the projection unit 12.

  Next, the projection unit 12 will be described with reference to FIG. The projection unit 12 includes a scanning unit 88, and the scanning unit 88 includes a horizontal scanning device 90 and a vertical scanning device 92.

  The horizontal scanning device 90 deflects an incident laser beam and has a resonance-type deflecting element 96 having a deflecting surface (for example, a reflecting surface) 94 that is reciprocally swung in order to scan the deflected light in the horizontal direction. A horizontal scanning drive circuit 98 that drives the deflection element 96 based on the horizontal synchronization signal supplied from the signal processing circuit 25 is provided.

  Similarly, the vertical scanning device 92 deflects an incident laser beam and includes a deflecting surface (for example, a reflecting surface) 100 that is reciprocally swung to scan the deflected light in the vertical direction. An element 102 and a vertical scanning drive circuit 104 that forcibly drives the deflection element 102 by a sawtooth drive signal based on the vertical synchronization signal supplied from the signal processing circuit 25 are provided.

  As shown in FIG. 2, the laser beam emitted from the horizontal scanning device 90 is incident on the vertical scanning device 92 through the first relay optical system 106 and after being converged thereby.

  The laser beam scanned by the scanning unit 88 is emitted from the emission port of the projection unit 12 through the second relay optical system 108 and after being converged thereby. As shown in FIG. 1, a half mirror 112 is attached to the housing 110 of the projection unit 12.

  The laser beam emitted from the projection unit 12 is incident on the half mirror 112 as shown in FIGS. The incident laser beam is reflected by the half mirror 112, passes through the pupil 122 in the eyeball 120 of one eye of the observer, and finally enters the retina 124.

  The laser beam incident on the retina 124 is scanned on the retina 124, and as a result, the laser beam is converted into planar image light. Thereby, the observer can observe the two-dimensional image as a virtual image. Not only the image light reflected by the half mirror 112 but also light from the external environment (external light) is transmitted through the half mirror 112 and incident on one eye of the observer. As a result, the observer can observe the actual outside world in parallel with the observation of the image displayed by the image light.

  The HMD 10 displays display content (for example, video content and image content) in a rectangular image display area based on a video signal input from the outside. The display content includes at least one display object. An example of each display object is an information item composed of a plurality of characters (including numbers, symbols, and icons) having a unique meaning, but is not limited thereto, and may be composed of an image, for example.

  The information item is text data, and unlike the video content for viewing, the information item does not have a specific attribute (for example, displayed line thickness, color, position, etc.), and the attribute can be arbitrarily set on the user side. It is usually editable. In other words, as long as the content of the text of this type of information item is the same, even if the unique attribute is changed, the information amount of the information item does not deteriorate.

  Therefore, it is important to design and operate the HMD 10 so that the attributes of the information items, that is, the display conditions are optimized, in order to improve the visibility of the displayed information items, that is, the visibility.

  However, the degree of visibility of an information item is not always determined only by the attribute of the information item. The HMD 10 allows an observer to visually recognize the displayed information item by overlaying it on the outside world. In this case, the degree of visibility of the displayed information item depends on the attribute of the image of the outside world. This is because there is a possibility of change.

  Against this backdrop, the present inventors conducted experiments using the prototype of this HMD10 to optimize the attributes of information items, and based on the results, final version of this HMD10 I decided to design. Hereinafter, the contents and results of the experiments conducted by the present inventors and the considerations on the experimental results will be described in detail.

1. Purpose of Experiment While observing a work object in the actual outside world, a scene of referring to work auxiliary information (information item) in the image display area of the HMD 10 is simulated as necessary.

2. Experimental environment (1) Large monitor A large monitor having a 42-inch screen simulated the outside world (the display pattern has five variations).

(2) Observation position of test subjects A plurality of test subjects were seated and observed at a position 75 cm away from the large monitor so that they were alternately replaced.

(3) HMD10
HMD10 was mounted on the head of each subject. The HMD 10 was attached to the head of each subject so as to cover the non-dominant eyes (in this case, all the observers were left eyes) of both eyes of each subject. On the image display area of the HMD 10, information items (each of the six attributes has a plurality of variations, and as a combination thereof, the display mode of the information items also has a plurality of variations) are displayed in a display mode that changes sequentially. It was displayed.

(4) A keyboard for the subject to input the experiment result As described later, the subject copies the information displayed on the HMD 10 for each task and whether the display mode is comfortable or not. Subjective evaluation results were entered on the keyboard. The former information is used to calculate the percentage of correct answers that indicate the degree to which the subject correctly recognized the information item, while the latter information is the number of all subjects who were subjectively evaluated as comfortable. Used to calculate subject comfort as a percentage of

3. Information Item Configuration 3.1 Information Item Display Mode As shown in FIG. 4, the information item to be targeted this time is composed of a plurality of characters (including numbers, symbols, underlines, and frame lines).

3.2 Six attributes of information items (main design elements)
(1) Font size (item size)
The font size variations for displaying information items are 18 points, 26 points, 34 points, 42 points, and 50 points.

(2) Information Item Display Position Among the image display area (screen) of the HMD 10, the variation of the position where the information item is displayed is the case where the image display area is equally divided into nine parts in total, vertically and horizontally. This is the position of nine sub areas (see FIG. 6B).

  These nine sub-areas form a 3 × 3 matrix. The position of each sub-area is an integer i representing a row number (i is 1 or more and 3 or less), and increases from the upper side to the lower side, and an integer j (j is 1 or more and 3 or less) representing a column number. ), And is expressed as (i, j) using the one that increases from the side closer to the observer's nose to the side closer to the one eye of the observer's both ears. .

  In the present embodiment, as shown in FIG. 6B, subarea (1,1) is represented as subarea A, subarea (1,2) is represented as subarea B, and subarea (1 , 3) is denoted as subarea C, subarea (2,1) is denoted as subarea D, subarea (2,2) is denoted as subarea E, and subarea (2,3) is denoted as subarea Described as area F, subarea (3, 1) is denoted as subarea G, subarea (3, 2) is denoted as subarea H, and subarea (3, 3) is denoted as subarea I. ing.

(3) Background color of information item (ground color of image display area)
The variations of the background color of the information item are as follows.
BB: The entire background is black (all black)
WW: The whole background is white (all white)
BW: The entire background is white only around the information item

(4) Font color (item color)
A plurality of variations of colors for displaying information items are 14 colors that cover all colors that can be perceived by humans.

(5) Number of items / arrangement The number of the same information items displayed and a plurality of variations of the arrangement are as follows.
AL: Information item is displayed in all nine display positions (sub-areas) HL: Three information items are displayed for each row (three display positions) VL: For each column A state where information items are displayed three by three (in three display positions)

(6) Number of Item Color Classifications The following are a plurality of variations of the number of color classifications of nine information items displayed in all nine display positions (sub-areas).
・ Nine information items have the same color ・ Nine information items have different colors ・ Three information items have different colors

4). As shown in FIG. 5, the image displayed on the large monitor has the following five patterns in order to reproduce the actual outside world mainly from the viewpoint of color saturation.

-External pattern A: The whole is black-External pattern B: The whole is white-External pattern C: Monochrome mosaic pattern-External pattern D: Mosaic pattern colored in black and white and other two colors-External pattern E: Full color Colored mosaic pattern

  Here, the definition of the term “color saturation” will be described. This term is the number of different colors (hereinafter simply referred to as “the number of external colors”) used in an external image (external field range). A low color saturation means that the number of external colors is small, while a high color saturation means that the number of external colors is large. Further, the term “color saturation” can also be interpreted as a term meaning at least one of hue and saturation.

5. Experiment contents

5.1 One task performed by the subject (1) With the external environment being simulated on the screen of a large monitor, an arbitrary alphabet appears on the screen at random positions and at random timing. When appearing, each subject presses the enter key on the keyboard at hand when the appearance is recognized. Then, the information item is displayed on the image display area of the HMD 10 with the currently selected information item among a plurality of display modes that change sequentially.

(2) When the information item is displayed, each subject inputs the display contents that each subject can visually recognize using the keyboard at hand. As a result, the accuracy of recognition was measured.

(3) Each time an information item is displayed, each subject further determines whether or not the display mode is comfortable as a subjective evaluation of each subject with respect to the display mode of the information item (for example, it is easy to see or the view is natural) Or whether it is impossible to move the eyeball).

(4) Each test subject subjectively evaluated task comfort using the Visual Analogue Scale (VAS) method after completing one task. The VAS method is an example of a method for sensibly quantifying the degree of intensity of a stimulus received by a human.

5.2 Process of Experiment (1) Stage 1: Optimization of Font Size and Display Position Each subject has 135 cases with a combination of 3 background colors, 5 font sizes and 9 display positions. The task was executed every time. In this first stage, other attributes (design elements) were fixed to any of the variations. There were 12 subjects in total.

(2) Second stage: Optimization of background color and display color Each subject performs the above task for each case in 210 cases that are a combination of 5 types of external patterns, 3 types of background colors, and 14 types of display colors. Executed. In this second stage, the font size and the display position are fixed to the variations evaluated to be optimal in the experimental results of the first stage, and any other attribute (design element) is selected from any variation. Fixed to. There were 10 subjects in total.

(3) Third stage: Optimization of the number of items / arrangement and the number of color coding Each subject takes all nine information items displayed in the nine sub-areas as one task, and uses five types of external patterns, The task was executed in each case for 60 cases in which the number of items / arrangement 4 types (AL, SG, HL, and VL) and the color classification number 3 types were combined. When the types of item number / arrangement variation are SG, HL, and VL, a key operation by each subject was required to switch the displayed information items. There were 11 subjects in total.

  For example, when the number of items / arrangement variation type is HL, first, the information items shown in FIG. 4 are simultaneously placed in the three sub-areas A, B, and C arranged horizontally in the uppermost row. Next, when the subject performs a key operation, they are displayed simultaneously in three sub-areas D, E, and F arranged side by side in the center row, and when the subject performs a key operation, Are simultaneously displayed in the three sub-areas G, H and I arranged side by side in the row.

  In this third stage, the font size and the display position are fixed to the variations evaluated to be optimum in the experimental results of the first stage, and the external pattern, background color and display color are set in the second stage. It was fixed to the variation evaluated as the optimum in the experimental results.

6). Experimental Results 6.1 Results of First Stage Experiment (1) Optimal Range of Font Size FIG. 6A shows the comfort level of the subject with respect to the font size. The comfort level is the percentage of cases in which each subject subjectively evaluated that they were comfortable among all cases tested. As shown in FIG. 6A, when the font size is 34 points or more, the comfort level is 80% or more. Therefore, in consideration, it is desirable that the optimum range of the font size is a range of 34 points or more.

(2) Optimal range of display position In FIG. 6B, when the font size is 34 points, the display position of the information item (9 positions) and the comfort level of the subject with respect to the information item displayed at each position The relationship is shown geometrically related to the display position. Although not shown, the correct answer rate of subjects was 60% or more regardless of the display position. From this, it may be reasonable to conclude that any display position is optimal.

  However, paying attention to the comfort level which is the subjective evaluation value shown in FIG. 6B, the comfort level is relatively examined among the nine display positions. It was as follows when comfort level was arranged in descending order.

1. Subarea H (Sub area (3, 2) located at the bottom of the center row)
2. Sub-area E (sub-area located in the center of the center row (2, 2))
3. Sub-area D (sub-area (2, 1) located in the center of the rows closest to the nose), sub-area G (sub-area (3, 1) located at the bottom of the rows closest to the nose) And subarea B (the top subarea (1, 2) in the center row)
4). Sub-area A (sub-area (1, 1) located at the top of the rows closest to the nose), sub-area F (sub-area (2, 3) located at the center of the rows closest to the left ear) ), Sub-area I (the sub-area (3, 3) located at the bottom of the row closest to the nose))
5. Sub-area C (the sub-area (1, 3) located at the top of the row closest to the left ear)

  Thus, for consideration, the higher the priority of an information item to be displayed (for example, the more important the content of the information item is to the worker who should refer to that information item, or the information item is referred to by the worker). It is important to display the corresponding information item preferentially in a sub-area having a high degree of comfort so that the content of the information item is accurately recognized by the operator. Therefore, the order of the subareas related to the comfort level described above corresponds to the order of the subareas related to the priority in which the information items are displayed.

  The characteristics shown in FIG. 6B are those when the subject observes the image with the left eye. On the other hand, when the subject observes the image with the right eye, the characteristics shown in FIGS. ), Nine subareas are assigned to be symmetrical with respect to the nine subareas for the left eye with respect to the subject's body centerline (eg, a vertical line passing through the subject's nose) .

  Therefore, when the same HMD 10 is rotated in the horizontal plane or moved in the horizontal direction to switch the eye for observing the image between the left eye and the right eye, the same information item is used. However, the relative position of the display position (the position where the information item is displayed among the nine sub-areas) with respect to the HMD 10 is changed (for example, when the same HMD 10 is rotated in the horizontal plane, The sub-area is moved symmetrically in the vertical direction with respect to the center line, and when the same HMD 10 is moved in the horizontal direction, the nine sub-areas are moved symmetrically in the horizontal direction with respect to the center line. May be required.

6.2 Results of Experiment of Second Stage (1) Optimum Range of Background Color FIG. 7A shows the comfort level of the subject with respect to the combination of the type of external pattern and the type of background color. As shown in FIG. 7A, the highest comfort level was obtained for any type of external pattern when the background color was BB (all black). Thus, for consideration, the optimum background color is BB. The optimal background color does not depend on the type of external pattern.

(2) Optimal range of item color An item color can be expressed by a combination of brightness and absolute luminance. Although the experimental results for the optimal range of item colors are not shown in the figure, when the background color is BB (all black) and the subject's comfort level is analyzed, the optimal range of item colors is determined by the type of external pattern. Specifically, when the external pattern is A (all black), the absolute luminance is 0.04 or more and the brightness is 35 or more, whereas the external pattern is in another state. Is a range in which the absolute luminance is 0.16 or more and the brightness is 70 or more. Moreover, as an item color, a subject's subjective evaluation value was high about red, green, and yellow, for example. Thus, for consideration, it is desirable to select an item color to be used to display an information item within this range.

6.3 Results of Experiment in Stage 3 (1) Optimal Range of Item Number / Arrangement FIG. 7 (b) shows the correct answer rate of the subject with respect to the combination of the type of external pattern and the number of items / arrangement type. As shown in FIG. 7 (b), the correct answer rate was the highest when the number of items / arrangement was AL, regardless of the type of external pattern. In addition, in the case of HL or VL that displays nine information items partially rather than all at once, the correct answer rate is the external patterns D and E with relatively complex images and high color saturation. There was also a tendency to increase compared to other types of external patterns.

  Therefore, to consider, basically, it is desirable to adopt a display mode called AL that displays an information item without switching the display of the information item in order to improve the correct answer rate.

(2) Optimal range of number of item color classifications FIG. 7C shows the comfort level of the subject with respect to the combination of the type of external pattern and the number of item color classification numbers. As shown in FIG. 7 (c), the comfort level is different from the display mode (in the case of displaying in 9 colors) for all nine information items regardless of the type of the external pattern. It was lower than the aspect.

  In addition, when the external pattern is A, B, or C (when the number of colors of the external environment is 2 or less), the comfort level is higher than that of other display modes for the display mode that unifies the colors of the nine information items. it was high.

  When the external pattern is D or E (when the number of colors in the external world is 3 or more, the external image is relatively complex, and the color saturation is high), the comfort level is nine. The display mode for color-coding information items in three colors was higher than other display modes.

  In addition, regardless of the type of the external pattern, generally, the comfort level is higher than the other display modes for the display mode in which nine information items are color-coded in three colors.

  Therefore, for consideration, it is desirable to color-code the display image with three colors. However, when it is necessary to reduce the number of colors used by the display image, when the external pattern is A, B, or C ( When the color saturation is low), the display image is displayed in one color, while when the external pattern is D or E (when the color saturation is high), the display image is displayed in three colors. It is desirable to change the number of colors used by the image.

  In addition, it can be easily inferred from the graph shown in FIG. 7C that when the number of colors, which is the total number of colors used simultaneously for image display, is 3, the comfort level is the highest value. Whether the number of colors is smaller than 3 (for example, 1) or larger than 3 (for example, 9), the comfort level is lower than the maximum value.

  The HMD 10 stores at least one information item (for example, information for assisting or supporting the work of a worker as a user) in the nine sub-areas in the image display area based on a video signal input from the outside. The selected one is displayed. The HMD 10 determines the above-described six attributes (design elements) when displaying each information item as follows based on the above-described experimental results.

(1) Font size (item size)
The font size for displaying the information item is a predetermined point of 34 points or more.

(2) Information Item Display Position Among the above nine sub-areas, the position where the information item is displayed is the importance of the information item (whether the information item is important to the user, The frequency is selected according to the frequency of reference by the user.

  In this way, a sub-area selection program is stored in the program ROM 27 in order to individually determine the display position of the information item. The position of the subarea (display subarea) where the information item is to be displayed depends on the importance of the information item, and whether the user's eyes observing the information item are the left eye or the right eye That is, it also depends on whether it is the left eye observation mode or the right eye observation mode.

(3) Background color of information item (ground color of image display area)
The background color of the information item is BB (full black).

(4) Font color (item color)
The color for displaying each information item is at least one color selected from the aforementioned 14 colors.

(5) Number of Items / Arrangement Only one same information item is displayed in the image display area, and is displayed in one sub-area selected as described above. However, a plurality of different information items may be displayed simultaneously in the image display area and in different sub-areas.

(6) Color coding number of information item The color coding number of at least one information item to be displayed is when the color saturation of the outside world is a predetermined value or less (when the outside world pattern is A, B or C, for example, 1 when the number of colors in the outside world is 2 or less), and when the color saturation of the outside world is higher than the predetermined value (when the outside pattern is D or E, that is, for example, the number of colors in the outside world is In the case of 3 or more), the number is determined to be 3. The number of colors of information items depends on the color saturation of the outside world. Further, when the color saturation of the outside world is higher than the predetermined value, even if the number of information items displayed simultaneously is 4 or more, the total number of colors used for displaying the information items is 3 Maintained.

  In order to determine the color classification number of the information item in this way, a color classification number determination program is stored in the program ROM 27. The number of colors of information items depends on the color saturation of the outside world observed by the user together with the information items. In the present embodiment, the color saturation of the outside world is automatically detected using the imaging result of the camera 23. For example, the user operates the operation unit 30 to input the color saturation of the outside world. Also good.

  FIG. 8 conceptually shows the subarea selection program in a flowchart. The sub area selection program is read from the program ROM 27 and executed by the CPU 26 as necessary.

  When the user turns on the power of the HMD 10, the execution of the subarea selection program is started. When this subarea selection program is executed, first, in step S1, it is determined whether or not the user is in the left eye observation mode in which the user observes the display image on the HMD 10 with the left eye.

  In one example, the determination in step S1 is performed by the user operating the operation unit 30 to directly input information that specifies the position of the eye to be used (the type of observation mode). The HMD 10 automatically detects the orientation of the HMD 10 with respect to the absolute space to estimate whether the eye covered by the HMD 10 is the left eye or the right eye. Is called. In an example in which the orientation of the HMD 10 is automatically detected, the orientation of the HMD 10 with respect to gravitational acceleration (that is, the vertical direction) is measured using an acceleration sensor, a weight, a pendulum, or the like mounted on the HMD 10.

  This time, assuming that the user is in the left eye observation mode in which the user observes the HMD 10 with the left eye, the determination in step S1 is YES, and in step S2, the left eye map shown in FIG. 9B is selected. On the other hand, this time, assuming that the user is in the right-eye observation mode in which the user observes the HMD 10 with the right eye, the determination in step S1 is NO, and in step S3, the right-eye map shown in FIG. Selected.

  When the user observes the display image by the HMD 10 with the left eye, the left-eye map is on the two-dimensional coordinate system of the sub-area number u and the left-eye observation image display area of each sub-area u. The relationship with the position v is defined (v = gL (u); gL (u) is a function for the left eye observation mode). Similarly, the right-eye map is a two-dimensional coordinate system of the number u of each sub-area and the image display area for right-eye observation of each sub-area when the user observes the display image by the HMD 10 with the right eye. The relationship with the upper position v (v = gR (u); gR (u) is a function for the right eye observation mode) is defined.

  The two-dimensional array of sub-areas on the left-eye map and the two-dimensional array of sub-areas on the right-eye map are shown geometrically associated with each other in FIGS. 9 (a) and 9 (b). In addition, when it is assumed that these maps are superimposed on both eyes of the observer, the vertical plane passing through the nose is in a mirror image relationship (symmetrical relationship).

  FIG. 9 (c) also shows the relationship among the alphabet notation (AI) of the sub area, the position u (1-9), and the above-described two-dimensional coordinates (i, j). The two-dimensional coordinates (i, j) are defined by a coordinate system CS1 having the sub-area A as the origin (1, 1) in each map. On the other hand, the two-dimensional coordinate position v (v1, v2) on the image display area is defined by a coordinate system CS2 having the origin at the lower left corner of the image display area.

  The two-dimensional coordinates (i, j) are in a symmetrical relationship between the left-eye observation mode and the right-eye observation mode, whereas the two-dimensional coordinate position v (v1, v2) is the left-eye observation mode. And the right-eye observation mode are translated in the horizontal direction. When the HMD 10 displays an image, since the coordinate system CS2 is used, conversion from the coordinate system CS1 to CS2 is necessary. For this reason, the above-described two functions gL and gR are selectively used. .

  Regardless of which map is selected, it is then determined in step S4 whether or not a new information item has been input based on the video signal input from the outside. If a new information item has been input, the determination in step S4 becomes YES, and in step S5, the importance (for example, indicated by “X” representing a unique number) of the current information item (for example, is described below). The importance of the content of the information item X and the frequency of reference frequency (Y) of the information item X by the user are determined.

  Specifically, the relationship between the information item X and the importance level Y is defined as Y = f (X) using the function f, and the function f (X) is stored in the flash ROM 28 in advance. . The importance level Y corresponding to the current information item X is derived using the function f (X). The number of importance levels Y is the same as the number of sub-areas described above, and specifically, nine levels from 8 to 0 are prepared. The importance level Y is defined to mean that the greater the number, the higher the importance level of the corresponding information item X.

  Subsequently, in step S6, among the nine sub-areas (AI), the information item X to be displayed this time, that is, the sub-area Z is determined. When the sub-area Z is determined, if the current time is the left-eye observation mode, the position of the sub-area Z on the two-dimensional coordinate system is determined using the left-eye map. Is the right-eye observation mode, the position of the sub-area Z on the two-dimensional coordinate system is determined using the right-eye map.

  Specifically, the relationship between the importance level Y and the subarea Z is defined as Z = F (Y) using the function F, and the function F (Y) is stored in the flash ROM 28 in advance. . FIG. 9C shows the characteristics of the function F (Y) in a table. The sub-area Z corresponding to the current information item X is derived using the importance Y corresponding to the current information item X as a parameter. The same number of subareas Z as the number of subareas described above are prepared. Specifically, nine subareas Z from A to I are prepared.

For example, if the importance level Y of the information item X at this time is the highest level, that is, “8”, the subarea Z is “H”. , Represents the lowest position. On the other hand, if the current information item Y is at the lowest level, that is, “0”, the subarea Z is “C”, which is the left of the nine subareas described above in the left-eye observation mode. In the ear / right-eye observation mode, it represents the uppermost one of the central rows closest to the right ear.

  Thereafter, in step S7, it is determined whether or not the current information item X can be displayed in the determined subarea Z. In the present embodiment, only a predetermined number of information items (for example, one information item) are displayed in one subarea. Therefore, if there is already a predetermined number of other information items at the same position as the current sub-area Z, the current information item X cannot be displayed in the current sub-area Z. The flash ROM 28 stores the information item currently displayed in association with the subarea in which the information item is displayed. This step S7 is executed using this information.

  If it is determined in step S7 that the current information item X can be displayed in the current sub-area Z, in step S8, the function gL (u) is used in the left-eye observation mode, and the function is displayed in the right-eye observation mode. Using gR (u), the current position u of the sub-area Z is converted to the coordinate position v of the image display area of the HMD 10. Subsequently, in step S9, based on the converted coordinate position v, the current information item X is arranged in the current sub-area Z. Thereafter, the process proceeds to step S10.

  On the other hand, if it is determined in step S7 that the current information item X cannot be displayed in the current sub-area Z, the current information item X can be displayed in step S11. In order to search for another sub-area that is associated with an importance level lower than the current value of importance Y (this time, the initial value is this time), the current value of importance Y of information item X is 1 Only subtracted.

  Thereafter, in step S12, whether or not the importance Y thus subtracted is lower than “0”, that is, even if the importance Y is subtracted to “0”, the current information item X is displayed. It is determined whether or not a sub-area that is capable of being found has been found (a predetermined number of information items are already displayed in all sub-areas). If it is assumed that the importance level Y is “0” or more this time, the determination in step S12 is YES, and the process returns to step S6 to display the current information item X based on the new importance level Y. Possible subareas are searched.

  On the other hand, this time, assuming that the updated importance Y is smaller than “0”, the determination in step S12 is YES, and in step S13, a predetermined condition is selected from the nine sub-areas described above. One subarea that is satisfied, that is, in the present embodiment, the subarea having the lowest importance Y, that is, the subarea having the importance Y of “0” is selected. Further, the display content of the selected subarea is made blank. That is, all the information items displayed in the subarea are deleted.

  Thereafter, in step S14, the position u of the current sub-area Z (= 0) is converted into the coordinate position v of the image display area of the HMD 10 in the same manner as in step S8. Subsequently, in step S15, the information item X of this time is arranged in the selected sub-area (the sub-area where the importance level Y is “0”). Thereafter, the process proceeds to step S10.

  In any case, in step S10, the contents are updated as necessary for each of all the information items arranged in the image display area. That is, the contents of the flash ROM 28 are updated so that the correspondence between the information items and the sub-areas reflects the latest state, and the contents of the information items reflect the latest contents. Then, it returns to step S4.

  In addition, in the subarea selection program shown in FIG. 8, when the user turns on the power of the HMD 10 and then designates the image observation mode of the HMD 10 through the operation unit 30, the user selects the HMD 10. It is assumed that the user does not change the image observation mode of the HMD 10 until the power is turned off.

  On the other hand, FIG. 10 shows a mode response that allows the user to change the image observation mode of the HMD 10 as many times as necessary through the operation unit 30 while the HMD 10 is powered on. The program is conceptually represented by a flowchart. This mode response program is additionally executed instead of replacing the subarea selection program.

  In this mode response program, first, in step S31, it is determined whether or not the user has input the current image observation mode via the operation unit 30. If there is an input, it is determined in step S32 whether or not the input image observation mode is the left eye observation mode. If the current mode is the left-eye observation mode, the left-eye map (function gL (u)) is selected in step S33, whereas if the current mode is the right-eye observation mode, in step S34. The right eye map (function gR (u)) is selected.

  In any case, thereafter, in step S35, the number u for sequentially specifying all the sub-areas is set to “1”. Subsequently, in step S36, the position u of the current sub-area is converted into the coordinate position v of the image display area of the HMD 10 using the selected function. Thereafter, in step S37, at least one information item to be displayed in the current sub-area is displayed at the position represented by the converted coordinate position v.

  Subsequently, at step S38, it is determined whether or not the current value of u is equal to or greater than the total number of all subareas (currently “9”). If the current value of u is equal to or greater than the total number uto, the process returns to step S31. However, if the current value of u is smaller than the total number uto, u is incremented by 1 in step S39, and then step S36. Return to.

  FIG. 11 conceptually shows the color classification number determination program in a flowchart. The color classification number determination program is read from the program ROM 27 and executed by the CPU 26 as necessary.

  When the power of the HMD 10 is turned on by the user, execution of this color classification number determination program is started. When the color classification number determination program is executed, first, in step S101, the HMD 10 is a flag fmc indicating whether or not an image is displayed in multicolor (multiple colors) instead of monochrome, and “0” indicates monochrome. The display mode is set, and “1” indicating the multi-color display mode is set to “1”. This is based on the premise that the HMD 10 is in the default mode of the multi-color display mode.

  In step S102, it is determined whether or not a new information item has been input based on the video signal input from the outside. If a new information item (current information item) is input, the determination in step S102 is YES, and in step S103, the color saturation of the actual outside world that the user is currently observing (the number of colors used by the image of the actual outside world). ) Α is obtained.

  In the present embodiment, the external color saturation α is acquired based on the imaging result of the camera 23 without requiring user intervention. In the above-mentioned “Summary of the Invention” column, the term “color saturation” has been described as a term that can mean at least one of hue and saturation. However, in the present embodiment, external color The degree α is used as a term meaning saturation. That is, “color saturation” in the “Summary of Invention” column is a superordinate concept of “saturation α” in the present embodiment.

  The saturation α is a physical quantity that is 0 for achromatic colors (white, black, and gray) and is the maximum for a pure color. Saturation α is saturation, and is expressed in a range from 0% to 100%.

  In one example, the saturation α of the HSV color space is detected from the RGB values of the RGB color space (including the luminance value R of the red light component, the luminance value G of the green light component, and the luminance value B of the blue light component). In this case, the maximum value and the minimum value of the R value, the G value, and the B value of each pixel in the imaging area of the camera 23 are acquired from the imaging result of the camera 23, and the acquired maximum value and minimum value are acquired. The saturation α is detected as a value obtained by dividing the difference by the maximum value.

  However, since the saturation α may be different for each pixel, in one example, in order to obtain one representative saturation α for the imaging area of the camera 23, the imaging area is divided into a plurality of blocks. For each block, one individual saturation (for example, an average value or median value) that represents the block is detected, and one value (for example, the maximum value) that represents a plurality of individual saturations for the blocks. Is detected as one overall saturation representing the imaging area.

  In order to detect the saturation α, it is not essential to use the camera 23. For example, a CMOS sensor or another light receiving element can be used.

  Subsequently, in step S104, it is determined whether or not the acquired saturation α is equal to or less than a threshold value. In the present embodiment, the threshold value is set to 20%. A saturation α of 20% or less means that the external pattern is A, B or C (for example, the number of colors used by the external world is 2 or less), while the saturation α is 20 Higher than% means that the external pattern is D or E (for example, the number of colors used by the external environment is 3 or less). Therefore, in this step S104, it is determined whether or not the external pattern is A, B or C. Based on the above experimental results, when the saturation α is equal to or less than the threshold value, it is desirable to display all information items including the current information item in one color.

  This time, assuming that the saturation α is equal to or less than the threshold value, the determination in step S104 is YES, and in step S105, it is determined whether or not the flag fmc is “1”. If it is assumed that the flag fmc is “1” this time (however, if the current execution is the first time after turning on the power, the flag fmc is “1”), the determination in step S105 is YES, and the step The process proceeds to S106.

  In step S106, all information items are displayed in one color. For all information items, if there is no existing information item (hereinafter simply referred to as “existing item”), it means only the current information item, while if there is an existing item, the existing item And the information item this time. One display color common to all information items is selected in advance, and is, for example, a color with a high subjective evaluation value of the user, for example, one of red, green, and yellow.

  Thereafter, in step S107, the flag fmc is set to “0”, thereby recording that the display mode of the HMD 10 has shifted from the multi-color display mode to the monochrome display mode. Subsequently, in step S108, the contents of all information items are updated. Data of all information items is stored in the flash ROM 28.

  The case where the flag fmc is “1” has been described above. However, when the flag fmc is “0”, the determination in step S105 is NO, and steps S106 and S107 are skipped. Thereby, the monochrome display mode in which all information items are displayed in one color is continued. Also in this case, the process proceeds to step S108.

  The case where the saturation α is equal to or less than the threshold value has been described above. However, when the saturation α is larger than the threshold value, that is, the external pattern is D or E, all information items are displayed in three colors. If this is desirable, the determination in step S104 is NO. In this case, subsequently, the number N of existing items is acquired in step S109. Since the data of the existing item is stored in the flash ROM 28, this step S109 is executed by referring to the contents of the flash ROM 28.

  Thereafter, in step S110, it is determined whether the number N of existing items is 2 or less. If it is assumed that the number is 2 or less this time, the determination is YES, and it is determined whether or not the number N of existing items is 0 in step S111. This time, assuming that the number N of existing items is 0 (there is no existing item), the determination in step S111 is YES, and the process proceeds to step S112.

  In this step S112, in order to display the current information item in three colors, one sub-area where the current information item is to be displayed is divided into three. In one example, as shown in FIG. 12A, one corresponding subarea (for example, subarea H) is divided into three in the vertical direction (each divided into three divided areas extending in the horizontal direction). )

  Subsequently, in step S113, three different colors are designated for the three divided areas. In one example, as shown in FIG. 12A, red, green, and yellow are designated for the three divided areas, respectively. Thereby, a divided display (a kind of multi-color display) for displaying the current information item in three colors is performed.

  Thereafter, in step S114, the flag fmc is set to “1”. Subsequently, the process proceeds to step S108.

  As described above, the case where the existing item number N is 0 has been described. When the existing item number N is not 0, the determination in step S111 is NO, and in step S115, it is determined whether the existing item number N is 1. . If it is assumed that the current time is 1 (one existing item exists), the determination in step S115 is YES, and the process proceeds to step S116.

  In step S116, it is determined whether or not the flag fmc is “1”. This time, there is only one existing item, and in this state, assuming that the flag fmc is “1”, that one existing item is divided and displayed by executing step S113 described above. It will be. In this case, the determination in step S116 is YES, and in step S119, the current information item is displayed in one color in the same color as any of the existing display colors (this time, the three colors described above). As a result, the total number of colors used by all information items does not need to exceed three. Thereafter, the process proceeds to step S114.

  On the other hand, assuming that the flag fmc is “0”, one existing item is displayed in monochrome. In this case, the determination in step S116 is NO, and in step S117, one subarea in which one existing item is displayed is divided into three as in step S112.

  Subsequently, in step S118, as in step S113, different colors are designated for the three divided areas generated by the three divisions. Thereby, divided display (a kind of multi-color display) is performed for displaying one existing item in three colors. Thereafter, the process proceeds to step S119, whereby the total number of colors used by all information items does not need to exceed 3. Thereafter, the process proceeds to step S114.

  The case where the number N of existing items is 1 has been described above, but when it is not 1, that is, when it is 2 in this context, the determination in step S115 is NO. Subsequently, in step S120, it is determined whether or not the flag fmc is “1”.

  This time, there are two existing items, and in this state, assuming that the existing flag fmc is “1”, one of these two existing items has the above-described split display. It will be. On the other hand, since the number of all information items (combination of two existing items and the current information item) is three this time, even if each information item is displayed in a different color and a single color, The total number of colors used by all information items is 3. Therefore, the divided display for any existing item is not necessary.

  If the existing flag fmc is “1”, the determination in step S120 is YES, and in step S121, the divided display for the existing item is canceled. Subsequently, in step S122, different colors (for example, red and green) are designated for the two existing items.

  Thereafter, in step S123, the current information item is displayed in one color (for example, yellow) different from the two colors (existing display colors) specified for the two existing items. Subsequently, the process proceeds to step S114.

  On the other hand, this time, assuming that the flag fmc is “0”, two existing items are displayed in the same color and in one color. In this case, after the determination in step S120 is NO and step S121 is skipped, steps S122 and S123 are executed in the same manner as described above.

  As described above, the case where the number N of existing items is 0, the case where it is 1, and the case where it is 2 have been described. If the number N of existing items is 3, the determination in step S110 is NO. Subsequently, in step S124, it is determined whether or not the flag fmc is “1”.

  This time, assuming that there are three existing items and the flag fmc is “1”, the three existing items are displayed in one color and a total of three colors. In this case, after that, the process proceeds to step S119, and the current information item is displayed in one color in the same color as any one of the existing display colors (the above-mentioned three colors) (for example, red). As a result, the total number of colors used by the four information items (the combination of the three existing items and the current information item) does not need to exceed three. Subsequently, the process proceeds to step S114.

  On the other hand, this time, assuming that the flag fmc is “0”, three existing items are displayed in the same color and in one color. In this case, after that, in step S125, the image display area is divided into three in order to display the three existing items in different colors from each other in a total of three colors. In one example, as shown in FIG. 12B, the image display area is divided into three in the horizontal direction (each divided into three divided areas extending in the vertical direction).

  Subsequently, in step S126, three different colors are designated for the three divided areas. In one example, as shown in FIG. 12B, red, green, and yellow are designated for the three divided areas, respectively. Thus, multi-color display is performed in which each of the three information items is displayed in one color and in all three colors. Thereafter, the process proceeds to step S119.

  If the existing item number N is 4, the determination in step S110 is NO. Subsequently, in step S124, it is determined whether or not the flag fmc is “1”. This time, assuming that there are four existing items and the flag fmc is “1”, the four existing items are displayed in one color and a total of three colors. In this case, after that, the process proceeds to step S119, and the current information item is displayed in the same color as one of the existing display colors (the aforementioned three colors) (for example, red). As a result, the total number of colors used by the five information items (the combination of the four existing items and the current information item) does not exceed three. Subsequently, the process proceeds to step S114.

  On the other hand, this time, assuming that the flag fmc is “0”, four existing items are displayed in the same color. In this case, thereafter, in step S125, the image display area is divided into three to display the four information items in three colors. Subsequently, in step S126, three different colors are designated for the three divided areas. Thus, multi-color display is performed in which each of the four information items is displayed in one color and in all three colors. Thereafter, the process proceeds to step S119.

  When the number N of existing items is 5 or more, the color classification number determination program is executed in accordance with the case where the number N of existing items is 4, and thus a duplicate description is omitted.

  That is, in this embodiment, each information item (an example of the “display object”) is displayed in a single color, and the total number of colors used for displaying all the information items is 2 or more. Even when the number of information items simultaneously present in the display image exceeds the same number as the upper limit value, the upper limit value (3 in the present embodiment) is preset. It is maintained at the value.

  In addition, the color classification number determination program shown in FIG. 11 uses the input of a new information item as a trigger, and switches the display mode of the information item between one color display and three color display based on the external saturation α at that time. . Therefore, in the period between the input timing of a certain information item and the input timing of the next information item, even if the saturation α of the outside world changes, the change of the information item is reflected. The display mode does not change. This is because it is not designed so that the number of color classifications can be changed using a change in saturation α as a trigger.

  On the other hand, FIG. 13 shows an external response program that changes the display mode of information items (this time, the number of colors) in response to changes in the saturation α of the external environment after the latest information item is input. It is conceptually represented in the flowchart. This external response program is additionally executed instead of replacing the color classification number determination program.

  In this external response program, first, in step S151, it is waited for a predetermined time (for example, 10 seconds) to elapse. Here, the length of the predetermined time means the length of a period in which the camera 23 intermittently captures the outside world and acquires the saturation α using the imaging result. If the predetermined time has elapsed, since a new information item has been input in step S152, whether the color classification number using the saturation α of the outside world is being determined in the color classification number determination program. It is determined whether or not. If it is assumed that it is in the middle of this time, steps S153 to S157 are skipped and the process returns to step S151.

  On the other hand, in this case, assuming that the color classification number using the saturation α of the external environment is not currently being determined in the color classification number determination program, in step S153, the camera 23 uses the current external environment. Is imaged. Subsequently, in step S154, the saturation α is acquired based on the imaging result.

  Thereafter, in step S155, it is determined whether or not the acquired saturation α is equal to or less than the threshold value th. If the saturation α is equal to or less than the threshold value th, all information items are displayed in one color in step S156. Thereafter, the process returns to step S151. On the other hand, this time, assuming that the saturation α is larger than the threshold value th, in step S157, all the information items are 3 by the same algorithm as that used in the color classification number program. Displayed in color. Also in this case, the process returns to step S151.

  In FIG. 14, the image display method according to the present embodiment that displays an image using the HMD 10 is represented conceptually over time in a flowchart.

  First, in step S201, a new information item is input. Next, in step S202, among the nine sub-areas, the input information item to be displayed is the human perceptual characteristic and the left-eye observation mode or the right-eye observation mode. Is selected on the basis of the distinction. This step S202 is executed by the CPU 26 executing the subarea selection program shown in FIG. 8 while referring to the flash ROM 28.

  Subsequently, in step S203, by using the camera 23, the color saturation of the outside world observed by the observer is acquired together with the information item. Thereafter, in step S204, based on the acquired color saturation, the number of colors to be used for displaying all the information items, that is, whether to display one color or display in three colors is determined. . These steps S203 and S204 are executed by the CPU 26 executing the color classification number determination program shown in FIG. 11 while referring to the flash ROM 28.

  Subsequently, in step S205, the component laser beams emitted from the lasers 34, 36, and 38, and the combined optical system are reflected so that the position of the selected sub-area and the determined number of colors are reflected. A combined laser beam generated by combining at 56 is generated. The generated combined laser beam is scanned two-dimensionally by the horizontal scanning device 90 and the vertical scanning device 92 of the scanning unit 88. The scanned combined laser beam is final image light (the combined laser beam is also image light).

  Thereafter, in step S206, the generated image light is reflected by the half mirror 112, passes through the pupil 122, and is eventually projected onto the retina 124, whereby all information items are displayed on the image display area. The An observer who is a user observes all the displayed information items superimposed on the actual outside world located in front of the left or right eye of the observer.

  In addition, in this embodiment, the sub-area Z in which each information item is displayed is variably determined according to the importance Y of each information item to be displayed. However, among the nine sub-areas, one sub-area selected in advance as the comfort level is sufficiently high or a plurality of sub-areas adjacent to each other as a fixed display area, It is possible to implement the present invention in such a manner that information items are displayed.

  In this aspect, such a display area is defined as a sub-area H that is one sub-area having the maximum comfort level, or one block having the sub-area H (for example, four sub-areas D and E). , G, and H). That is, it is not indispensable to implement the present invention that the display position of each information item is variably determined according to the importance of each information item to be displayed.

  As is clear from the above description, in the present embodiment, for the sake of convenience of explanation, in the signal processing circuit 25, the part for executing step S1 shown in FIG. It can be considered that it constitutes an example of the “identification unit” in the section 5), and the portion of the signal processing circuit 25 that executes steps S2, S3, S8 and S14 shown in FIG. It can be considered that it constitutes an example of a part of the “sub-area selection unit” in the same section.

  As mentioned above, although one embodiment of the present invention was described in detail based on a drawing, this is an illustration, and it is various based on the knowledge of those skilled in the art including the aspect described in the section of the [Disclosure of the Invention]. The present invention can be implemented in other forms that have been modified or improved.

Claims (4)

An image display device that displays an image by projecting image light representing an image including one or a plurality of display objects onto an observer's eye,
A plurality of sub-areas in which the image display area observed from the observer side is divided so as to be positioned in front of a position biased toward the left eye side or right eye side of the observer from the center line of the observer's body When each display object is displayed, each display object has the same subjective evaluation value of visual comfort that the viewer feels as the viewer visually recognizes each displayed object. A sub-area selection unit that selects a display sub-area higher than that displayed in the area,
An image display device comprising: an image light generation unit that generates the image light so that each display object is displayed in the selected display subarea. The image display device displays the image by projecting the image light onto one eye of an observer,
  The image display area is disposed in front of the one eye,
  The image display device further includes an identification unit that identifies whether the one eye is an observer's left eye or right eye,
  The sub-area selection unit is configured such that the relative position of the display sub-area with respect to the image display area is determined based on whether the one eye is identified as the left eye or the right eye by the identification unit. The image display device according to claim 1, wherein the image display device is selected so as to change so as to be symmetrical with respect to a center line of the observer's body between cases identified as being present. An image display method for displaying an image by projecting image light representing an image including one or more display objects onto an observer's eye,
  A plurality of sub-areas in which the image display area observed from the observer side is divided so as to be positioned in front of a position biased toward the left eye side or right eye side of the observer from the center line of the observer's body When each display object is displayed, each display object has the same subjective evaluation value of visual comfort that the viewer feels as the viewer visually recognizes each displayed object. A sub-area selection step for selecting a display sub-area that is higher than that displayed in the area;
  An image light generation step for generating the image light so that each display object is displayed in the selected display subarea;
  An image display method including: A program executed by a computer to implement the image display method according to claim 3.

Images