JP5870778B2 - Head mounted display, display method, and display program - Google Patents

Description

  The present invention relates to a head mounted display, a display method, and a display program that are arranged on a user's head and display an image superimposed on an external landscape.

  2. Description of the Related Art Conventionally, a head-mounted display that is disposed on a user's head and displays an image superimposed on an external landscape is known. For example, in the video display device described in Patent Document 1, display colors of characters and lines in an image and the background thereof are determined. If the background color is white, it is determined whether the characters and lines are monochrome images. If it is not a monochrome image (ie, if it is a color image), characters and lines are converted to a monochrome image. After conversion to a monochrome image, the characters and lines and the background are reversed in black and white. By black and white reversal, white letters and lines are displayed against a black background. This enhances the visibility of characters and lines.

JP 2008-116704 A

  However, in the conventional video display device, when the background color is white, processing such as black and white reversal is performed, but when the background is other than white, black and white reversal is not performed. For this reason, when the background is other than white, the background is displayed as it is. Therefore, there is a problem that visibility (see-through property) of the scenery in the outside world is impaired depending on the background.

  An object of the present invention is to provide a head mounted display, a display method, and a display program capable of improving see-through performance.

A head-mounted display according to a first aspect of the present invention is a head-mounted display that is arranged on a user's head and displays a display image superimposed on an external landscape, in order to make the user visually recognize the display The emitting means for emitting the image light generated based on the image data corresponding to the image, the background specifying means for specifying the background portion in the display image, and the background portion specified by the background specifying means are the same. A background determination unit that determines whether the pixel of color information is a solid image that is an image of an area in which pixels are continuously distributed, and the background determination unit determines that the background portion is not the solid image. when the, the brightness of the background portion with emit the image light from said emitting means is lowered as compared with the state shown in the image data, the background portion other than Wherein the luminance of the part from the emitting means without reducing as compared with the state shown in the image data that a brightness reducing means for emitting the image light. In this case, the brightness of the background portion decreases and the image of the background portion becomes light. For this reason, it becomes easy for the user to visually recognize the scenery of the outside world overlapping the background portion. That is, see-through performance is improved. In addition, since the luminance of the portion other than the background portion does not decrease, the image of the portion other than the background portion is easily visible. For this reason, it is possible to make it easy to visually recognize parts other than the background part while improving see-through performance.

If the image is not a solid image, that is, if there is a pattern or the like in the background portion, the image in the background portion becomes thin. For this reason, see-through property can be improved while making a user visually recognize a pattern or the like in the background.

In the head mounted display according to the first aspect, when the background determination unit determines that the background portion is the solid image, the image light is emitted from the background portion emitted from the emission unit. There may be provided background light stopping means for stopping the light. If the background portion is a solid image, there is little information that the user needs to visually recognize in the background portion. When there is little information that the user needs to visually recognize, the user visually recognizes that the image of the background portion is not displayed at all by stopping the emission of the image light of the background portion. This makes it easier for the user to see the scenery of the outside world.

In the head-mounted display according to the first aspect, wherein the said background portion identified by the background specifying unit, contains the solid-like images of multiple, pre Symbol background light stopping means, before Symbol plurality of solid The emission of the image light at a part of the solid image in the shape image may be stopped. In this case, among the solid images in the background portion, some solid images are not visually recognized by the user because emission of image light is stopped. Further, the other solid image is visually recognized by the user because the emission of the image light is not stopped. For this reason, for example, the user can visually recognize a solid image that needs to be visually recognized while improving the see-through property so that only the less important solid image is not visually recognized by the user.
In the head-mounted display according to the first aspect, the head-mounted display includes designation means for designating a part of the solid images among the plurality of solid images, and the background light stopping means is designated by the designation means The emission of the image light in a part of the solid image may be stopped.

In the head-mounted display according to the first aspect, the background light stopping unit may stop the emission of the image light in the portion of the solid image having the largest area among the plurality of solid images. . In this case, it is possible to prevent the user from visually recognizing only the solid image having a large area that has the largest influence on the see-through property. Therefore, the see-through property can be improved while leaving some information in the display image.

In the head mounted display according to the first aspect, the display image is composed of a plurality of slides that change in a temporal order, and the background specifying means does not change between the slides of the display image. A portion may be specified as the background portion. In this case, since the background portion is automatically specified, the efficiency of the operation of specifying the background portion can be improved as compared with the case where the user specifies the background portion from the beginning.

The head mounted display according to the first aspect includes character extraction means for extracting a description portion of the character in the display image, and the background specifying means does not include the character extracted by the character extraction means. The background portion may be specified. In this case, the brightness of the character can be maintained. Therefore, it becomes easy for the user to visually recognize the character.

The head-mounted display according to the second aspect of the present invention is a head-mounted display that is arranged on a user's head and displays a display image overlaid on an external landscape, An emitting means for emitting image light generated based on image data corresponding to the display image; a background specifying means for specifying a background portion in the display image; and a luminance of the background portion specified by the background specifying means. The image light is emitted from the emission unit with a lower level than the state indicated in the image data, and the luminance of a portion other than the background portion is reduced from the level shown in the image data without lowering the luminance. a brightness reduction means to emit the image light, an imaging unit for imaging the external image is an image of a scene of the outside, the imaging of the image pickup means A color gradient acquisition unit that acquires a color gradient of the external image, and whether or not there are a predetermined number or more of the portions in the external image where the color gradient acquired by the color gradient acquisition unit is a predetermined value or more. A color gradient judging means for judging, wherein the brightness lowering means determines that the color gradient judging means does not judge that there are a predetermined number or more of the parts in the external image where the color gradient is a predetermined value or more. only, said lowering the brightness of the background portion identified by the background specifying unit, characterized by Rukoto to emit the image light from said emitting means. In this case, the see-through property can be improved by reducing the brightness of the background portion only when the scenery of the outside world is not complicated. In addition, when the scenery of the outside world is complicated, it is possible to make the display image information easier to see without reducing the luminance.

In the head-mounted display according to the first or second aspect, the brightness lowering unit may make the user visually recognize the color of the background portion lightly while maintaining the RGB value ratio of the pixels of the background portion. You may reduce the brightness | luminance of the said background part. In this case, since the ratio of the RGB values is maintained, the see-through property can be improved without giving the user a sense of incongruity.

In the head-mounted display according to the first or second aspect, the luminance reduction unit may reduce the luminance of the background portion by shifting the bit of the RGB value of the pixel of the background portion by at least a right bit. Good. In this case, by performing the right bit shift, the luminance of the background portion can be reduced to about half, and the color of the background portion can be thinly made visible to the user. Also, since all RGB values are right bit shifted, the ratio of RGB values can be maintained.

The display method according to the third aspect of the present invention is based on the image data corresponding to the display image that is arranged on the user's head, displays the display image over the landscape of the outside world, and allows the user to visually recognize the display image. A display method executed in a head-mounted display including an emitting unit that emits image light generated in the above-described manner, the background specifying step for specifying a background portion in the display image, and the background specifying step specified by the background specifying step The luminance of the background portion is lowered from the state shown in the image data and the image light is emitted from the emitting means, and the luminance of the portion other than the background portion is not lowered than the state shown in the image data. wherein comprises a from the emitting means and brightness reduction step of emitting the image light, the display image, a plurality of varying temporal order slides Thus configured, the background specifying step, that identifies a portion that does not change between multiple slides of the display image as the background portion. In this case, it is possible to easily see a portion other than the background portion while improving the see-through property. In addition, since the background portion is automatically specified, the efficiency of the operation of specifying the background portion can be improved as compared with the case where the user specifies the background portion from the beginning.

A display program according to a fourth aspect of the present invention is arranged on the user's head, displays a display image superimposed on an external landscape, and is based on image data corresponding to the display image for visual recognition by the user. emits image light generated Te a display program to be executed in the head-mounted display comprising an emission means, the controller of the head-mounted display, a background specifying step of specifying a background portion in the displayed image, the A background determination step for determining whether or not the background portion specified by the background specifying step is a solid image that is an image of an area in which pixels of the same color information are continuously distributed; and the background determination If it is determined that the background portion is not the continuous solid image by step, it shows the brightness of the background portion in the image data The image light is emitted from the emission means with a lower brightness than the state indicated by the image data, and the image light is emitted from the emission means without lowering the luminance of the portion other than the background portion than the state indicated in the image data. And a luminance reduction step to be executed. In this case, it is possible to easily see a portion other than the background portion while improving the see-through property. If the image is not a solid image, that is, if there is a pattern or the like in the background portion, the image in the background portion becomes thin. For this reason, see-through property can be improved while making a user visually recognize a pattern or the like in the background.

It is a perspective view of the spectacles 91 which attached HMD1. 2 is a plan view of the projection apparatus 100. FIG. It is the II sectional view taken on the line in FIG. It is a block diagram which shows the electric constitution of HMD1. It is a flowchart of an image display process. It is a flowchart of a background specific process. It is a figure which shows the image. It is a figure which shows the image 70 of the state in which the brightness | luminance of the background color part 74 and the strip | belt part 72 was reduced. It is a figure which shows the image 701 which has the background color part 74 without the flower pattern 741 shown in FIG. It is a figure which shows the image 701 of the state which the emission of the image light of the background color part 74 and the strip | belt part 72 was stopped. It is a figure which shows the image 701 of the state which the emission of the image light of the background color part 74 stopped.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. A head mounted display 1 (hereinafter referred to as an HMD 1) shown in FIG. 1 displays an image so as to be visually recognized as an image in front of the user's eyes. In the following description, the upward direction, the downward direction, the downward diagonal direction to the right, the upward diagonal direction to the left, the upward diagonal direction to the right, and the downward diagonal direction to the left are respectively the upward direction, downward direction, forward direction, and backward direction of HMD1. Right direction, left direction. The downward direction, upward direction, right direction, and left direction in FIG. 2 are the front direction, rear direction, right direction, and left direction of the projection apparatus 100, respectively. The upward direction, downward direction, right direction, and left direction in FIG. 3 are the upward direction, downward direction, right direction, and left direction of the projection apparatus 100, respectively.

  First, the configuration of the HMD 1 will be described with reference to FIGS. As shown in FIG. 1, the HMD 1 includes a projection device 100 and a control device 200. The projection apparatus 100 is used by being mounted on spectacles 91 that are dedicated mounting tools. However, the projection apparatus 100 may be attached to other wearing tools such as glasses, helmets, and headphones that the user uses on a daily basis. The projection apparatus 100 irradiates the user's eyes with image light. The projection device 100 is detachably connected to the control device 200 via a harness 150. The control device 200 is used by being worn on a user's waist belt, for example. The control device 200 controls the projection device 100.

  Next, the configuration of the projection apparatus 100 will be described. As shown in FIGS. 1 and 2, the projection apparatus 100 includes a housing 2. The housing | casing 2 is a square cylindrical resin member, and is L shape in planar view (refer FIG. 2). As shown in FIG. 3, the housing 2 contains a projection unit 10. The projection unit 10 generates image light and emits it in the left direction through the opening 7 on the left end side of the housing 2. A half mirror holder 5 (hereinafter referred to as HM holder 5) is fixed to the opening 7 on the left side of the housing 2. The HM holder 5 is a resin. As shown in FIGS. 1 to 3, the HM holder 5 holds a half mirror 8 (hereinafter referred to as HM8).

  HM8 is a resin. The HM 8 reflects at least a part (for example, half) of the light emitted from the projection unit 10 and enters the left eyeball (not shown) of the user. The user can visually recognize the image superimposed on the real image (outside scene) in his field of view. As shown in FIGS. 1 and 2, a slit 9 </ b> A is provided at the center in the left-right direction on the front surface of the housing 2. A part of the adjuster 16 is exposed in the slit 9A. The user can adjust the focus of an image that can be viewed with the HM 8 by rotating the adjuster 16 in the vertical direction with a finger. A circular small hole 40 is provided on the right side of the HM holder 5 on the front surface of the housing 2. A camera 39 is exposed forward in the small hole portion 40. The camera 39 captures an image of an external landscape in front of the user's visual field.

  Next, the configuration of the projection unit 10 will be described. As shown in FIG. 3, the projection unit 10 includes a lens holder 15, an adjuster 16, and a display holder 17. The lens holder 15 holds the eyepiece optical system 120. The display holder 17 holds the display 12. In the present embodiment, the display 12 is a liquid crystal display, but may be another display (for example, an organic EL display). The display 12 emits image light. The eyepiece optical system 120 accommodates a plurality of lenses (not shown), collects image light emitted from the display 12, and guides it to the opening 7 of the housing 2. “Condensing” means an optical action that reduces the spread of the luminous flux of image light. That is, “collection” is not limited to a configuration in which image light is converted into convergent light or parallel light by the eyepiece optical system 120.

  The adjuster 16 has a ring shape. The adjuster 16 is attached to the display holder 17 by extrapolation. A spiral groove cam 17 </ b> A is provided on the outer peripheral surface of the display holder 17. An engaging portion (not shown) is provided on the inner peripheral surface of the adjuster 16. The engaging portion engages with the groove cam 17A of the display holder 17 and moves along the groove cam 17A. The adjuster 16 is positioned in the vertical and horizontal directions within the housing 2. Therefore, when the adjuster 16 is rotated, the display holder 17 moves in the left-right direction by the action of the groove cam 17A. As a result, the distance between the display 12 and the eyepiece optical system 120 changes, so that the focus of an image that can be visually recognized by the HM 8 can be adjusted.

  Next, the glasses 91 will be described. As shown in FIG. 1, the glasses 91 include a left frame portion 92, a right frame portion 93, a central frame portion 94, and a support portion 96. The left frame portion 92 is a portion that is hung on the user's left ear. The right frame portion 93 is a portion that is hung on the user's right ear. The center frame portion 94 is provided between the front end portion of the left frame portion 92 and the front end portion of the right frame portion 93. The center frame portion 94 includes a pair of nose pads 95 (only one is shown in FIG. 1) at the center in the longitudinal direction. The support portion 96 is provided on the upper left end side (upper right end in FIG. 1) of the central frame portion 94 as viewed from the user side. The support part 96 includes a downward extension part 98. The downward extending portion 98 extends in the up and down direction at the left front of the user's face. A tooth portion (not shown) having a saw-toothed unevenness is engraved in the longitudinal direction on the front surface of the downward extending portion 98.

  As shown in FIGS. 1 and 2, a mounting portion 49 is provided in a portion of the housing 2 that faces the glasses 91. The mounting portion 49 includes a U-shaped groove 49A (see FIG. 2) along the vertical direction. A leaf spring 49B is provided at the bottom of the U-shaped groove 49A. A downward extending portion 98 provided in the support portion 96 of the glasses 91 is inserted into the U-shaped groove 49A. The leaf spring 49B is locked to any peak of the tooth portion provided in the downward extending portion 98. Therefore, the projection device 100 can be positioned within a predetermined range in the vertical direction of the user's head.

  Next, the configuration of the control device 200 will be described. As shown in FIG. 1, the control device 200 is a substantially rectangular parallelepiped system box, for example. The control device 200 includes, for example, a power switch 62 and an operation unit 64 (see FIG. 4, not shown in FIG. 1). The power switch 62 turns on / off the power of the HMD 1. The operation unit 64 can perform various operations such as selection of an image in the projection apparatus 100 and selection of a background portion in the image.

  Next, the electrical configuration of the control device 200 will be described with reference to FIG. The control device 200 includes a CPU 51, a ROM 52, a RAM 53, a flash memory 57, a power supply IC 55, an output unit 56, a power switch 62, an operation unit 64, and the like. The ROM 52, RAM 53, flash memory 57, output unit 56, power switch 62, operation unit 64, and the like are connected to the CPU 51. The CPU 51 controls the control device 200. The ROM 52 stores various programs. The RAM 53 temporarily stores various data. The flash memory 57 stores image data and various data. The CPU 51 transmits the image data read from the flash memory 57 to the projection device 100 via the output unit 56 and the harness 150. The transmitted image data is input to the CPU 31 of the projection apparatus 100.

  The power supply IC 55 is connected to the output unit 56. The power supply IC 55 converts an unstable voltage supplied from the battery 61 into a stable voltage. Stable voltage power is supplied to the projection device 100 via the output unit 56 and the harness 150 in addition to the various electronic components of the control device 200.

  Next, the electrical configuration of the projection apparatus 100 will be described with reference to FIG. The projection apparatus 100 includes a controller 11 and a display 12. The controller 11 includes, for example, a CPU 31, a ROM 32, a RAM 33, a display control unit 36, an input unit 37, a power supply IC 38, a camera 39, and the like. The ROM 32, RAM 33, display control unit 36, input unit 37, camera 39, and the like are connected to the CPU 31. A power supply IC 38 and the like are connected to the input unit 37. The CPU 31 controls the projection device 100. The ROM 32 stores various programs. The RAM 33 temporarily stores various data. The CPU 31 controls the display control unit 36 and drives the display 12. The power supply IC 38 supplies power supplied from the power supply IC 55 of the control device 200 via the harness 150 to various electronic components of the projection device 100.

  With reference to FIG. 5, the image display processing by the CPU 31 will be described. The user operates the operation unit 64 to select an image stored in the flash memory 57 and inputs an instruction to display the image. When an instruction to display an image is input, the CPU 31 calls a display program stored in the ROM 32 and executes this process. In the following description, an image selected by the user is referred to as a selected image. The selected image may be composed of a plurality of images (for example, images of a plurality of slides for presentation) or may be composed of a single image. The display program is written in the ROM 32 when the HMD 1 is shipped from the factory. However, the display program may be stored in a storage medium such as an external server. In this case, the display program is downloaded from an external server or the like via a communication unit (not shown) and stored in the flash memory 57. Further, the projection apparatus 100 may be provided with a flash memory, and a display program downloaded from an external server or the like may be stored in the flash memory.

  As shown in FIG. 5, in the image display process, first, selected image data (hereinafter referred to as selected image data) is acquired (S11). The selected image data is read from the flash memory 57 by the CPU 51 of the control device 200 and transmitted to the CPU 31 of the projection device 100. CPU31 acquires the selection image data transmitted from CPU51 of the control apparatus 200 (S11). Next, a background specifying process is executed (S12).

  The background specifying process executed by the CPU 31 will be described with reference to FIG. The background specifying process is a process for specifying a background portion in a selected image. The background part is defined by certain conditions. For example, the background portion is a portion other than a main portion (for example, a setting screen 71 in the image 70 described later) including important information in the selected image. As another example, the background portion may be a portion in a selected image excluding predetermined information such as character information. In addition, an information portion (for example, a setting screen 71 and characters 73, 75, 76, which will be described later) to be transmitted to the user is arranged behind the information portion in order to improve the visibility and decoration of the information portion. It may be a part. In the background specifying process, first, characters (characters, numbers, symbols, etc.) in the selected image are extracted (S31). For example, the character is extracted by recognizing a character code included in the selected image data.

  Next, the number of images included in the selected image is detected (S32). Next, as a result of S32, it is determined whether or not there is one image included in the selected image (S33). When there is one image (S33: YES), the image light of the selected image based on the selected image data is emitted from the display 12 (S34). That is, the selected image is displayed on the display 12 and visually recognized by the user.

  Next, the background portion is specified by accepting the specification of the background portion by the user (S35). Information about the background portion designated by the user is stored in the RAM 33. In S35, control is performed so that the character extracted in S31 cannot be designated as the background portion. Next, it is determined whether or not the specification of the background portion has been completed (S36). If the background portion has not been designated (S36: NO), the process returns to S35. That is, the reception of designation of the background portion is continued.

  For example, assume that an image 70 shown in FIG. 7 is selected as the selected image. The selected image is assumed to be composed of one image 70. The image 70 is an image for explaining an operation method of a print resolution setting screen of a printer (not shown). A print resolution setting screen 71 is displayed in the center of the image 70. A blue strip 72 that is long in the left-right direction is provided at the top of the image 70. On the right side of the belt portion 72, a character 73 “ABCD COMPANY” indicating a company name is described.

  A background color portion 74 is provided below the band portion 72 and around the setting screen 71. A background color portion 74 is provided below the band portion 72 and around the setting screen 71. The background color portion 74 is a portion of an orange image in which a plurality of pink flower patterns 741 are scattered. A character 75 “Confidential” is displayed in the lower left part of the image 70. On the left side of the setting screen 71, a character 76 “CLICK” is displayed. An arrow 77 extends from the character 76 toward the PRINT button 711 on the setting screen 71. A cursor 78 is displayed on the image 70. The user can freely move the cursor 78 on the image 70 by operating the operation unit 64.

  In S31 shown in FIG. 6, a character 73 “ABCD COMPANY”, a character 75 “CLICK”, and a character 76 “Confidential” are extracted from the image 70. Note that characters in the setting screen 71 (for example, “PRINT”, “400 dpi”, etc.) are not extracted because they are data pasted on the image 70 as a pattern. However, the CPU 31 may extract characters in the setting screen 71. In this case, for example, a well-known OCR (Optical Character Recognition) technique is used instead of character code recognition.

  For example, when the user sets the background color portion 74 as the background, a part of the background color portion 74 is designated by the cursor 78. As a result, the background color portion 74 is designated as the background portion (S35). Similarly, the user can designate an arbitrary portion (for example, the belt portion 72) as the background portion. When all the background parts have been specified, the user operates the operation unit 64 to input an instruction to end the background part specification. When the user inputs an instruction to end the specification of the background portion, the CPU 31 determines that the specification of the background portion has ended (S36: YES), and performs the process of S41 described later. As described above, in S35, the character extracted in S31 is controlled so that it cannot be designated as the background portion. For this reason, the character 73 “ABCD COMPANY”, the character 76 “CLICK”, and the character 75 “Confidential” extracted in S31 cannot be specified as the background portion.

  In S33, when the number of images included in the selected image is not one (S33: NO), a portion that does not change between the plurality of images included in the selected image is specified as the background portion (S37). At this time, the CPU 31 specifies the background portion so as not to include the character extracted in S31. For example, it is assumed that the image selected by the user includes a plurality of images including the image 70 (see FIG. 7), and the display of the setting screen 71 and the portion of the arrow 77 is different among the plurality of images. In this case, in S37, the setting screen 71 and the arrow 77 are not specified as the background. Further, the characters 73, 75, and 76 extracted in S31 are not specified as the background portion. Then, the background color part 74 and the band part 72 are specified as the background part. An example of this specifying method will be described. CPU31 acquires the RGB value for every pixel between several images from selection image data. Then, the CPU 31 specifies, as the background portion, a pixel whose RGB value does not change between a plurality of images and a pixel whose RGB value changes between a plurality of images is smaller than a predetermined number. In the present embodiment, a pixel portion in which the number of times the RGB value changes between a plurality of images is smaller than a predetermined number is also included in the “portion that does not change between the plurality of images”. Further, the background portion specifying method is an example, and the background portion may be specified by another specifying method.

  Next, the background portion specified in S37 is notified to the user (S38). This notification is performed, for example, by displaying the image 70 (the image light of the image 70 is emitted) while the background portion specified in S37 is blinking. Next, the specification by the user of the portion determined as the background portion among the background portions identified in S37 is accepted (S39). The user operates the operation unit 64 to move the cursor 78, and designates a portion to be confirmed as the background portion among the background portions notified in S38. For example, when the setting screen 71 is included in the background portion automatically specified in S37, if a portion other than the setting screen 71 is specified, the setting screen 71 is not set as the background portion.

  Next, as in S36, it is determined whether or not the designation of the background portion by the user has been completed (S40). When the designation of the background portion has not ended (S40: NO), the process returns to S39. When it is determined that the user has specified the background portion (S40: YES), the process of S41 is performed. In S41, the setting of the importance of the background part specified in S37 and S39 or S35 is accepted (S41). The user erases the display when the background part that does not want to be completely erased, such as when the important part is included in the background part, is set as “high importance” and the important part is not included at all. The background part that can be used is set as “low importance”. The importance level is set by the user operating the operation unit 64. The set importance is stored in the RAM 33. Next, it is determined whether or not the importance setting by the user has been completed (S42). If the importance level has not been set (S42: NO), the process returns to S41. When the user operates the operation unit 64 and performs an operation for finishing the setting of the importance level, it is determined that the setting of the importance level is finished (S42: YES), and the background specifying process is finished.

  The process returns to the image display process (see FIG. 5), the camera 39 is controlled, and an image of an external landscape (hereinafter referred to as an external image) is captured (S13). Next, data of the external image captured in S13 (hereinafter referred to as external image data) is analyzed, and a color gradient of the external image is acquired (S14). For obtaining (calculating) the color gradient, a known method (for example, differentiating the RGB value of each pixel with respect to the X-axis direction and the Y-axis direction of the image, dividing the R value by the G value) A value obtained by dividing the value and the B value by the G value, etc.).

  Next, it is determined whether or not there are a predetermined number N or more portions in the external image where the color gradient value acquired in S14 is α (predetermined value) or more (S15). The portion where the value of the color gradient is α or more is an edge portion such as an object or a pattern in the external image. The case where the number of edge portions is N or more is a case where the outside landscape is complicated, for example, when the outside landscape includes a complicated pattern (for example, stained glass).

  When it is not determined that there are a predetermined number N or more in the external image where the value of the color gradient is α or more (S15: NO), the color gradient of the background portion of the selected image based on the selected image data is acquired. (S16). Next, it is determined whether or not the background portion is an image of an area where pixels having the same color information are continuously distributed (hereinafter referred to as a solid image) (S17). The “region in which pixels having the same color information are continuously distributed” may not be a region in which the RGB values are completely the same. For example, pixels having approximate RGB values are continuously distributed. It may be a region. In S17, it is determined whether or not the background portion is a solid image by determining whether or not there is a portion where the color gradient is greater than or equal to α among the color gradients of the background portion acquired in S16. (S17). In S17, “α” that is the same as the threshold in the case of performing the process in S15 is used, but the present invention is not limited to this, and a different threshold may be used for determination.

  If the background portion is not a solid image (S17: NO), the bits of the RGB values of the pixels of the background portion of the selected image are right-bit shifted (S18). For example, the RGB value is represented by 8 bits for each color. That is, it is assumed that each RGB value is 256 gradations. In this case, if the bit of the R value (red) is “10011111”, the bit is shifted to the right and becomes “01001111”. In this case, the gradation value of R is reduced from “159” to “79”. That is, the gradation value is about half. Similarly, the G value (green) and the B value (blue) are also right-bit shifted.

  Next, image light is emitted from the display 12 (S19). That is, an image is displayed on the display 12. At this time, the gradation value of the background portion is reduced in S18. For this reason, in S19, image light is emitted from the display 12 so that the luminance of the background portion is lower than the state indicated by the selected image data acquired in S11. Therefore, the brightness of the background portion is reduced. On the other hand, the gradation values of the portions other than the background portion are not changed. For this reason, in S19, the image light is emitted so that the brightness of the portion other than the background portion is the same as the state shown in the selected image data acquired in S11. Therefore, the brightness of portions other than the background portion does not decrease. Next, the image display process is terminated. After the image display process is completed, for example, when there are a plurality of images included in the selected image and the image is changed (page is changed), the brightness of the background portion is also changed in the image after the change as in S19. The image light is emitted after being lowered.

  For example, it is assumed that the background color portion 74 and the band portion 72 are specified as the background portion in S37 and S39 or S35. The background color portion 74 includes a flower pattern 741. For this reason, the background color portion 74 is not a solid image. For this reason, it is determined that the background portion is not a solid image (S17: NO). Note that the band portion 72 is a solid image, but the background portion (background color portion 74 and band portion 72) includes a background color portion 74 that is not a solid image, so the background portion as a whole is not a solid image. It is judged. Then, the background color portion 74 and the band portion 72, which are background portions, are right bit shifted (S18).

  Then, as shown in FIG. 8, the image light of the image 70 is emitted from the display 12 (S19). In the image 70 shown in FIG. 8, the brightness of the background color portion 74 and the band portion 72, which are background portions, is lower than that in FIG. 7, and approaches black. The closer the color is to black, the smaller the luminance of the image light emitted from the display 12 is, so it is difficult for the user to visually recognize it. For this reason, the user visually recognizes the background color part 74 and the band part 72, which are background parts, in a lighter color than in the case of FIG. Therefore, it becomes easier to visually recognize the scenery of the outside world that is visible beyond the image 70. That is, see-through performance is improved. In addition, since luminance other than the background portion does not decrease, display of important information other than the background portion (information on the setting screen 71, characters 73, 75, 76, etc.) is easy to visually recognize. Thus, in this embodiment, it is possible to make it easy to visually recognize important information included in a portion other than the background portion while improving see-through performance.

  Further, in the present embodiment, when the image is not a solid image (S17: NO), that is, when the background portion has a pattern or the like (the flower pattern 741 in the present embodiment), the image of the background portion becomes light. For this reason, see-through property can be improved, making a user visually recognize the pattern etc. in a background part.

  Also, by performing the right bit shift (S18), the luminance of the background portion can be suppressed to about half compared to the case of not performing the right bit shift, and the user can visually recognize the color of the background portion lightly. . Also, since all RGB values are right bit shifted, the color type can be maintained (for example, from dark blue to light blue). Since the color type is maintained, the see-through property can be improved without giving the user a sense of incongruity.

  Next, the image display process is terminated. After the image display process is completed, for example, when there are a plurality of images included in the selected image and the image is changed (page is changed), the CPU 31 also uses the background portion of the changed image as in S19. The image light is emitted in a state where the luminance of the light is lowered.

  As shown in FIG. 5, when the background portion is a solid image (S17: YES), it is determined whether or not the background portion solid image includes a portion of high importance (S20). The importance level is the importance level set in S41 (see FIG. 6). If the portion of high importance is not included (S20: NO), the gradation value of the background portion is set to “0” (S21). That is, the RGB bits are set to “00000000”. Next, image light is emitted from the display 12 (S22). At this time, the gradation value of the background portion is set to “0” in S21. Accordingly, the emission of the image light in the background portion is stopped.

  For example, it is assumed that the selected image selected by the user is an image 701 shown in FIG. The background color portion 74 of the image 701 is a solid image without the flower pattern 741 of the background color portion 74 of the image 70 shown in FIG. The portion of the image 701 other than the background color portion 74 is the same as the image 70. Here, it is assumed that the background color portion 74 and the band portion 72 of the image 701 are specified as the background portion in S37 and S39, or S35 (see FIG. 6). The importance levels of the background color portion 74 and the band portion 72 are assumed to be small. In this case, both the background color part 74 and the band part 72, which are background parts, are solid images and have a low importance. Therefore, it is determined that the background portion is a solid image (S17: YES), and it is determined that the background portion does not include a portion of high importance (S20: NO). Then, the gradation values of the background color portion 74 and the band portion 72 are set to “0” (S21), and image light is emitted from the display 12 (S22).

  In this case, image light of an image 701 shown in FIG. 10 is emitted. In the image 701 shown in FIG. 10, the emission of the image light from the background color portion 74 and the band portion 72 that are solid images is stopped. For this reason, the background color part 74 and the band part 72 are not visually recognized by the user. If the background color portion 74 and the band portion 72 that are background portions are solid images, the background portion has little information that the user needs to visually recognize. That is, when there is little information that the user needs to visually recognize, the user visually recognizes that the image of the background portion is not displayed by stopping the emission of the image light of the background portion. For this reason, the scenery of the outside world can be made easier to visually recognize.

  Next, the image display process is terminated. After the image display process is completed, for example, when there are a plurality of images included in the selected image and the image is changed (page is changed), the image light of the background portion is also displayed in the changed image as in S22. The image light of the other part is emitted in a state where the emission of is stopped.

  As shown in FIG. 5, when the solid image of the background portion includes a portion with a high importance level (S20: YES), the gradation value of the solid image with a low importance level is set to “0”. (S23). Next, image light is emitted from the display 12 (S24). At this time, the gradation value is set to “0” in the portion of the background portion with a low importance level in S23. In addition, the gradation value is not changed in the portion of the background portion where the importance is high. Therefore, in S22, the image light of the portion with the higher importance is emitted from the background portion in the selected image to be emitted, and the emission of the image light of the portion with the lower importance is stopped. That is, the CPU 31 stops the emission of image light of a part of the solid image (in this embodiment, a solid part with low importance) among the plurality of solid images.

  For example, of the background color portion 74 and the band portion 72 (see FIG. 9) specified as the background portion, the background color portion 74 is set to a low importance level and the band portion 72 is set to a high importance level (S41). ). In this case, the image light of the image 701 shown in FIG. 11 is emitted by the processing of S23 and S24. As shown in FIG. 11, among the background color portion 74 and the band portion 72 that are background portions, the band portion 72 having a high degree of importance emits image light. On the other hand, the image light of the background color portion 74 having a low importance level is not emitted. As described above, in the present embodiment, a solid image having a high importance level that needs to be visually recognized by the user while improving the see-through property so that only the background color portion 74 that is a solid image having a low importance level is not visually recognized by the user. The user can visually recognize the band portion 72 that is a shape image.

  Next, the image display process is terminated. After the image display process is completed, for example, when there are a plurality of images included in the selected image and the image is changed (page change), the background after the change is less important as in S24. In a state where the emission of the image light of the part is stopped, the image light of the other part is emitted.

  As shown in FIG. 5, for example, when the scenery of the outside world is complicated, it is determined that there are a predetermined number N or more in the outside image where the value of the color gradient is α or more (S15: YES). Image light is emitted (S25). In S25, unlike S18, S21, S23, etc., the gradation value of the background portion is not changed. For this reason, for example, an image in which the luminance of the background portion is not reduced is displayed, such as an image 70 shown in FIG. 7 or an image 701 shown in FIG.

  For example, if the scenery of the outside world contains complex patterns (for example, stained glass) and the see-through property is enhanced when the scenery of the outside world is complex, the selected image overlaps the complicated scenery, and the selected image May be difficult to see. For this reason, in this embodiment, when the scenery of the outside world is complicated, the brightness of the background portion is not lowered, and the information of the selected image is easily viewed. Further, only when the scenery of the outside world is not complicated, the brightness of the background portion is reduced or the emission of the image light is stopped (S19, S22, S24). For this reason, the brightness of the background portion is reduced or the emission of the image light is stopped only when the scenery of the outside world is not complicated and the information of the selected image is easy to see even if the luminance of the background portion is reduced or the emission of the image light is stopped. And see-through performance can be improved.

  Next, the image display process is terminated. After the image display process is finished, for example, when there are a plurality of images included in the selected image and the image is changed (page is changed), the luminance is reduced or the emission of the image light is stopped as in S25. The image light is emitted without being performed.

  As described above, the image display process is performed. In this embodiment, a character is extracted (S31), and a background portion is specified so as not to include the extracted character (S35, S37). Therefore, in FIGS. 8, 10, and 11, even when the brightness is adjusted, the brightness of the character portion can be maintained. Therefore, it becomes easy for the user to visually recognize the character.

  In this embodiment, when there are a plurality of selected images, a portion that does not change between the plurality of images can be specified as a background portion (S37). Since the background portion is automatically specified, the efficiency of the operation of specifying the background portion can be improved as compared with the case where the user specifies the background portion from the beginning.

  In the above embodiment, the display 12 corresponds to the “emitter” of the present invention, and the selected image corresponds to the “display image” of the present invention. The CPU 31 that performs the processes of S35, S37, and S39 corresponds to the “background specifying unit” of the present invention. The CPU 31 that performs the processing of S18 and S19 corresponds to the “luminance reduction means” of the present invention, and the CPU 31 that performs the processing of S17 corresponds to the “background determination means” of the present invention, and the processing of S21, S22, S23, and S24 The CPU 31 that performs the process corresponds to the “background light stopping means” of the present invention. The CPU 31 that performs the process of S31 corresponds to the “character extracting unit” of the present invention, and the camera 39 corresponds to the “imaging unit” of the present invention. The CPU 31 that performs the process of S14 corresponds to the “color gradient acquisition unit” of the present invention, and the CPU 31 that performs the process of S15 corresponds to the “color gradient determination unit” of the present invention. The processes of S35, S37, and S39 correspond to the “background specifying step” of the present invention, and the processes of S18 and S19 correspond to the “luminance reduction step” of the present invention. The CPU 31 corresponds to the “controller” of the present invention.

  In addition, this invention is not limited to said embodiment, A various change is possible. For example, the emission of the image light of the portion of the solid image with a low importance among the plurality of solid images has been stopped (S23 and S24), but is not limited thereto. For example, the emission of the image light of the portion of the solid image having the largest area among the plurality of solid images identified as the background portion may be stopped. In this case, it is possible to prevent the user from visually recognizing the background color portion 74 (see FIG. 8) having the largest area and having the largest influence on the see-through property (see FIG. 11). Also, the image light of the band portion 72 having a smaller area than the background color portion 74 is emitted from the background portion. In this way, the see-through property can be improved while leaving some information in the selected image.

  In S18, the right bit shift is performed to reduce the luminance of the background portion, but the method of reducing the luminance is not limited. For example, instead of performing the right bit shift, the gradation value may be decreased by a predetermined value (for example, 50). Further, fine adjustment (for example, adding 5 to the gradation value after the right bit shift) may be performed after the right bit shift.

  Moreover, although the color gradient was acquired and it was judged whether the background part was a solid image (S16, S17), it is not limited to this. For example, a solid image may be determined when a range in which pixels of the same color (same or approximate RGB values) are continuous is equal to or greater than a predetermined range.

  Further, when stopping the emission of the image light, it is not necessary to stop the leakage of light completely. For example, when the display 12 is a liquid crystal display, part of the backlight light may leak from the background portion. The “stop emission of image light” in the present embodiment includes a case where part of the light of the backlight leaks from the background portion.

  In S18, S21, and S23, the gradation value of the background portion is changed to be lowered. However, the present invention is not limited to this. For example, the tone value of at least a part of the portion other than the background portion may be raised while the tone value of the background portion is lowered. In this case, the luminance of the portion where the gradation value is increased among the portions other than the background portion is increased, which makes it easier for the user to visually recognize the portion. That is, when image light is emitted in S19, S22, and S24, the image light is emitted so that the luminance of the portion other than the background portion is not lower than the state shown in the selected image data acquired in S11. Also good.

  In addition, in the conventional video display device, when the background is other than white, the background is displayed as it is and the see-through property is impaired. However, in the above embodiment, the see-through property is improved regardless of the background color. Can do.

1 Head mounted display 12 Display 31 CPU
39 Camera 70 Image 71 Setting screen 72 Band part 73, 75, 76 Character 74 Background color part

Claims (12)

A head-mounted display that is placed on the user's head and displays a display image superimposed on the scenery of the outside world,
Emitting means for emitting image light generated based on image data corresponding to the display image in order to make the user visually recognize;
Background specifying means for specifying a background portion in the display image;
Background determination means for determining whether the background portion specified by the background specifying means is a solid image that is an image of an area in which pixels of the same color information are continuously distributed;
When the background determination unit determines that the background portion is not the solid image, the luminance of the background portion is lowered from the state indicated in the image data and the image light is emitted from the emission unit. A head-mounted display comprising: a luminance lowering unit that emits the image light from the emitting unit without lowering the luminance of the portion other than the background portion than the state indicated by the image data. When the background determining means determines that the background portion is the solid image, a background light stopping means is provided for stopping emission of the image light of the background portion emitted from the emitting means. The head-mounted display according to claim 1 . The said background portion identified by the background specifying unit contains the solid form images of multiple,
Before SL background light stopping means, a head mounted display according to claim 2, characterized in that to stop the emission of the image light of a portion of a portion of the solid-like image of the previous SL plurality of continuous solid image .   A designating unit for designating a part of the solid images among the solid images;
  4. The head mounted display according to claim 3, wherein the background light stopping unit stops emission of the image light of the part of the solid image designated by the designation unit. 4. The head mounted display according to claim 3 , wherein the background light stopping unit stops emission of the image light at a portion of the solid image having the largest area among the plurality of solid images. The display image is composed of a plurality of slides that change in a temporal order ,
6. The head mounted display according to claim 1, wherein the background specifying unit specifies a portion that does not change between a plurality of slides of the display image as the background portion. Character extraction means for extracting a character description portion in the display image,
The head-mounted display according to claim 1, wherein the background specifying unit specifies the background portion so as not to include the character extracted by the character extracting unit. A head-mounted display that is placed on the user's head and displays a display image superimposed on the scenery of the outside world,
Emitting means for emitting image light generated based on image data corresponding to the display image in order to make the user visually recognize;
Background specifying means for specifying a background portion in the display image;
The brightness of the background portion specified by the background specifying means is lowered from the state shown in the image data and the image light is emitted from the emitting means, and the luminance of the portion other than the background portion is set to the image data. Brightness reducing means for emitting the image light from the emitting means without lowering than the state shown in
An image pickup means for picking up an image of the outside world, which is an image of an external landscape;
A color gradient acquisition unit for acquiring a color gradient of the external image captured by the imaging unit;
A color gradient determining unit that determines whether or not there are a predetermined number or more of portions in the external image where the color gradient acquired by the color gradient acquiring unit is equal to or greater than a predetermined value;
The brightness lowering means is specified by the background specifying means only when the color gradient determining means does not determine that there are a predetermined number or more of portions in the external image where the color gradient is a predetermined value or more. wherein the luminance of the background portion is lowered from the state shown in the image data, features and to Ruhe head-mounted display that emit the image light from said emitting means. The luminance reduction means reduces the luminance of the background portion so that the user can visually recognize the color of the background portion lightly while maintaining the RGB value ratio of the pixels of the background portion. The head mounted display in any one of 1-8.   10. The head mounted display according to claim 9, wherein the luminance lowering unit lowers the luminance of the background portion by shifting at least a right bit of the RGB value of the pixel of the background portion. Disposed on the user's head, is provided with an emitting means for displaying a display image superimposed on an external landscape and emitting image light generated based on image data corresponding to the display image for visual recognition by the user A display method executed in a head-mounted display,
A background specifying step of specifying a background portion in the display image;
The brightness of the background part specified by the background specifying step is made lower than the state shown in the image data so that the image light is emitted from the emitting means, and the brightness of the part other than the background part is made the image data And a luminance reduction step for emitting the image light from the emission means without lowering than the state shown in
The display image is composed of a plurality of slides that change in a temporal order,
The background specifying step specifies a portion that does not change between a plurality of slides of the display image as the background portion . Disposed on the user's head, is provided with an emitting means for displaying a display image superimposed on an external landscape and emitting image light generated based on image data corresponding to the display image for visual recognition by the user A display program executed on the head-mounted display,
To the controller of the head mounted display,
A background specifying step of specifying a background portion in the display image;
A background determination step for determining whether or not the background portion specified by the background specifying step is a solid image that is an image of an area in which pixels of the same color information are continuously distributed;
When the background determination step determines that the background portion is not the solid image, the luminance of the background portion is lowered from the state indicated in the image data and the image light is emitted from the emission means. And a luminance reduction step of emitting the image light from the emission means without lowering the luminance of the portion other than the background portion from the state indicated by the image data.