JP2010200209A - Head mount display - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a head mount display capable of easily switching an actual object image and a related image as a user intends. <P>SOLUTION: A CCD sensor takes a photograph in the visual field direction of the user of the head mount display (S11). A barcode is extracted from the picked-up image (S12). An automobile is specified by specifying it with the barcode (S13). Then when an imaging magnification is equal to or less than a threshold magnification (S14:NO), the actual object image is displayed (S17), but when the imaging magnification is larger than the threshold magnification (S14:YES), an internal image corresponding to the actual object image is displayed (S15). The user can set the imaging magnification and the actual object image or internal image is displayed on the head mount display based on the imaging magnification. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a head mounted display that displays an actual image showing an actual object or a related image related to the actual object.

  2. Description of the Related Art Conventionally, a head-mounted display that displays an actual image showing an actual image or a related image related to the actual image has been proposed (see, for example, Patent Document 1). In the head-mounted display described in Patent Document 1, when the user wearing the head-mounted display faces the direction of a specific real object, the real image showing the real object is displayed, and the user is fixed. When the real image has been viewed for more than a certain time, it is assumed that the user is interested in the real image, and a related image related to the real image is also displayed. In this way, the user's interest is reflected in the image display on the head mounted display.

JP 2005-38008 A

  However, just because a user sees the same real image for a certain period of time or more does not always mean that he / she is interested in the real image shown in the real image. Sometimes I look at one point without thinking. Therefore, the head-mounted display described in Patent Document 1 has a problem in that related images may be displayed even if the user does not want them, which is troublesome. In addition, when there are a plurality of objects intersecting in the line-of-sight direction, there is a problem that the object to be identified cannot be selected and a desired image cannot be viewed.

  The present invention has been made in view of the above problems, and in a head-mounted display that displays a real image showing a real thing or a related image related to the real thing, the real image and the related image are made to the user's intention. An object of the present invention is to provide a head mounted display that can be easily switched based on the above.

  In order to achieve the above object, a head mounted display according to claim 1 is mounted on a user's head and includes a display means for displaying an image on the user's eyes, and the head mounted display. A real image corresponding to the field of view of the user wearing the image is captured as a real image, and an imaging unit that allows the user to set an imaging condition when the real image is captured, and a real image captured by the imaging unit And a related image storage means for storing a related image corresponding to the real image, the head mounted display used in the head mounted display system, and based on the imaging conditions, the real image and the Selection means for selecting one of the related images, and the real image is selected by the selection means. In this case, the real image is displayed on the display means, and when the related image is selected, the related image stored in the related image storage means corresponding to the real object is displayed on the display means. Display control means for displaying.

  According to a second aspect of the present invention, in addition to the configuration of the first aspect of the present invention, the head mounted display includes a real identification unit that identifies a real object included in the real image captured by the imaging unit, and the display control. The means displays the related image stored in the related image storage means in correspondence with the real thing specified by the real thing specifying means.

  According to a third aspect of the present invention, in addition to the configuration of the invention according to the second aspect, the related image storage means includes, as the related image, an internal image corresponding to a position where the user views the actual object. , Storing corresponding to each position, the real object specifying means also specifies a position where the specified real object is imaged, and the display control means, when displaying the related image, the position specified by the real object specifying means The internal image stored in the related image storage means is displayed on the display means corresponding to the above.

  According to a fourth aspect of the present invention, in addition to the configuration of the invention according to any one of the first to third aspects, the imaging magnification can be set as the imaging condition. The real image is selected when it is equal to or smaller than a predetermined threshold, and the related image is selected when the imaging magnification is larger than the predetermined threshold.

  Further, in addition to the configuration of the invention according to any one of claims 1 to 4, the head-mounted display according to claim 5 is provided with identification information for identifying itself in an actual image picked up by the image pickup means. The real object specifying means specifies the real object imaged by the imaging means based on the identification information included in the real image.

  According to the head-mounted display of the first aspect, the real object specifying unit specifies the real object indicated by the real image captured by the imaging unit. The selection means can select one of the real image and the related image based on the imaging condition. The display control means displays the real image when the selection means selects the real image, while when the related image is selected, the display control means stores the real image corresponding to the real image in the related image storage means. Related images can be displayed. As described above, the imaging condition can be set by the user, and the real image or the related image is displayed on the head mounted display based on the imaging condition. Therefore, the real image and the related image can be easily switched based on the user's intention.

  According to the head-mounted display according to claim 2, the real object specifying unit specifies which real image is an image showing the real object included in the captured image captured by the image capturing unit. The display control means can display the related image stored in the related image storage means corresponding to the real thing specified by the real specifying means.

  According to the head mounted display of the third aspect, the user can visually recognize the actual internal image. The internal image is an image corresponding to the position where the user sees the actual object. Therefore, the user can grasp the inside of the real object from multiple directions.

  According to the head mounted display of the fourth aspect, since the image to be displayed is determined as the real image or the related image according to the imaging magnification, the user can easily switch the image to be displayed. In general, it is considered that the user intuitively sets a larger imaging magnification for an actual object that is interested, that is, an actual object for which a related image is to be displayed. Therefore, the selection unit selects the real image when the imaging magnification is smaller than the predetermined value, and selects the related image when the imaging magnification is equal to or larger than the predetermined value. As a result, the user can easily perform an operation for determining which image to display.

  According to the head-mounted display of the fifth aspect, since the actual object is specified based on the identification information, the actual object captured by the imaging unit can be specified with certainty.

It is the figure which showed schematic structure of the head mounted display system 5 to which the head mounted display 1 is applied. It is explanatory drawing which shows the electrical and optical structure of the head mounted display. 3 is a conceptual diagram of a table 301 stored in a storage device 300. FIG. It is the flowchart which showed the process which the control part 110 performs. FIG. 6 is a diagram showing a real image 404 displayed on the head mounted display 1 when the user 50 is in front of the automobile 401. FIG. 6 is a diagram showing a real image 404 displayed on the head mounted display 1 when the user 50 is on the side of the automobile 401. It is the figure which displayed the internal image 403a1 in case the user 50 is ahead of the motor vehicle 401 on the head mounted display 1. FIG. It is the figure which displayed on the head mounted display 1 the internal image 403a2 in case the user 50 exists in the side of the motor vehicle 401. FIG. It is a figure for demonstrating the method of determining in which position the user 50 is with respect to the motor vehicle 401. FIG.

  Hereinafter, embodiments of a head mounted display according to the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing a schematic configuration of a head mounted display system 5 using the head mounted display 1 of the present embodiment. As shown in FIG. 1, the head mounted display system 5 is configured by connecting a head mounted display 1 and a storage device 300 via a network 200. The head mounted display 1 is worn by the user 50.

  As an example, it is assumed that the user 50 visually recognizes automobiles 401a, 401b, and 401c (hereinafter collectively referred to as “automobile 401”). In addition, it is assumed that the automobiles 401a, 401b, and 401c of the present embodiment have different vehicle types. A barcode 402 (identification information) is attached to the front and side surfaces of each automobile 401. Specifically, a barcode 402a1 is attached to the front surface of the automobile 401a, and a barcode 402a2 is attached to the side surface. Further, a barcode 402b1 is attached to the front surface of the automobile 401b, and a barcode 402b2 is attached to the side surface. Further, a barcode 402c1 is attached to the front surface of the automobile 401c, and a barcode 402c2 is attached to the side surface. In addition, other automobiles not shown in the figure also have unique barcodes on the front and side surfaces. Therefore, the barcodes 402a1 to 402c2 are information for specifying the automobiles 401a to 401c. Further, the barcodes 402a1 to 402c2 specify an imaging position when the CCD sensor 2 described later captures an image, that is, whether the user 50 is at a position for imaging the front surface of the automobile 401 or a side image. It is also information to do.

  Next, the structure of the head mounted display 1 will be described with reference to FIGS. As shown in FIGS. 1 and 2, the head mounted display 1 converts various kinds of information stored inside or outside into an image signal, and generates image light (hereinafter referred to as “image light”) based on the image signal. The optical scanning unit 10 (see FIG. 2) that scans the user E with the eye E of the user 50 is provided. In the head mounted display 1, when the user 50 turns on the power of the head mounted display 1 with the head mounted display 1 mounted on the head of the user 50, the optical scanning unit 10 starts operating. By the operation of the optical scanning unit 10, the image light is scanned in a two-dimensional direction on the retina of the user 50, so that the user 50 can visually recognize an image corresponding to the information. The head mounted display 1 is configured such that the user 50 cannot visually recognize the outside world.

  In addition, the head mounted display 1 of the present embodiment includes a CCD sensor 2 as an imaging unit that images the automobile 401 corresponding to the visual field of the user 50. The head mounted display 1 includes a luminance sensor 8 that detects the brightness of the outside world and an LED 3 that illuminates the imaging range of the CCD sensor 2 (see FIG. 2). The LED 3 illuminates the imaging range of the CCD sensor 2 when it is detected that the brightness is below the brightness of the image sensor.

  As shown in FIG. 2, the head mounted display 1 includes a control unit 110 that performs overall control of the operation of the entire head mounted display 1, and 2 image lights generated based on an image signal supplied from the control unit 110. An optical scanning unit 10 that visually displays an image according to an image signal by displaying an image by scanning in dimension.

  The optical scanning unit 10 includes an image light generation unit 20 that reads out the image signal supplied from the control unit 110 for each dot clock, generates image light that is intensity-modulated according to the read video signal, and emits the image light. Further, a laser beam (image light) generated by the image light generation unit 20 and emitted through the optical fiber 100 is converted into parallel light between the image light generation unit 20 and the eye E of the user 50. A collimating optical system 61, and a horizontal scanning unit 70 functioning as a first light scanning unit that reciprocally scans the image light collimated by the collimating optical system 61 in the horizontal direction (first direction) for image display; A vertical scanning unit 80 functioning as a second optical scanning unit that reciprocally scans the image light scanned in the horizontal direction by the horizontal scanning unit 70 in the vertical direction (second direction substantially orthogonal to the first direction); And vertical scanning section 80 And a relay optical system 90 for emitting the image light thus scanned in the horizontal direction and the vertical direction (two-dimensionally scanned) to the pupil Ea. . Moreover, in order to prevent the user 50 from visually recognizing the outside world, a closing member (not shown) that closes around the eyes of the user 50 is provided.

  The image light generation unit 20 receives an image signal supplied from an external device such as a personal computer (not shown) via the interface 104 and the control unit 110, and synthesizes an image based on the image signal. There is provided a signal processing circuit 21 for generating each signal or the like as an element for the purpose. In the signal processing circuit 21, blue (B), green (G), and red (R) image signals 22 a to 22 c are generated and output to the light source unit 30 described later. The signal processing circuit 21 outputs a horizontal drive signal 23 used in the horizontal scanning unit 70 and a vertical drive signal 24 used in the vertical scanning unit 80, respectively.

  Further, the image light generation unit 20 outputs an image light of blue, green, and red corresponding to the three image signals (B, G, R) 22a to 22c output from the signal processing circuit 21 for each dot clock. A light source unit 30 that functions as a light output unit, and a light combining unit 40 for combining these three image lights into one image light to generate arbitrary image light are provided.

  The light source unit 30 includes a B laser 34 that generates blue image light and a B laser driver 31 that drives the B laser 34, a G laser 35 that generates green image light, and a G laser driver 32 that drives the G laser 35. And an R laser 36 for generating red image light and an R laser driver 33 for driving the R laser 36. Each of the lasers 34, 35, and 36 can be configured as, for example, a semiconductor laser or a solid-state laser with a harmonic generation mechanism. If a semiconductor laser is used, the drive current can be directly modulated to modulate the intensity of the image light. However, if a solid-state laser is used, each laser is equipped with an external modulator to modulate the intensity of the image light. I do.

  The light combining unit 40 is provided with collimating optical systems 41, 42, and 43 provided to collimate image light incident from the light source unit 30 into parallel light, and dichroic mirrors 44 and 45 for combining the collimated image light. , 46 and a coupling optical system 47 for guiding the combined image light to the optical fiber 100.

  Laser beams emitted from the lasers 34, 35, and 36 are collimated by collimating optical systems 41, 42, and 43, and then enter the dichroic mirrors 44, 45, and 46. Thereafter, the image light is selectively reflected and transmitted with respect to the wavelength by these dichroic mirrors 44, 45 and 46.

  Specifically, the blue image light emitted from the B laser 34 is collimated by the collimating optical system 41 and then enters the dichroic mirror 44. The green image light emitted from the G laser 35 enters the dichroic mirror 45 through the collimating optical system 42. The red image light emitted from the R laser 36 enters the dichroic mirror 46 through the collimating optical system 43.

  The three primary color image lights respectively incident on the three dichroic mirrors 44, 45, and 46 are reflected or transmitted in a wavelength selective manner, reach the coupling optical system 47, and are collected and output to the optical fiber 100.

  The horizontal scanning unit 70 and the vertical scanning unit 80 scan the image light incident from the optical fiber 100 in a state where it can be projected as an image, and scan it in the horizontal direction and the vertical direction to obtain scanning image light.

  The horizontal scanning unit 70 has a resonance type deflection element 71 having a reflection surface for scanning image light in the horizontal direction, and a drive that resonates the resonance type deflection element 71 and swings the reflection surface of the resonance type deflection element 71. Based on a horizontal scanning drive circuit 72 as a drive signal generator for generating a signal and a displacement signal output from the resonance type deflection element 71, the oscillation range and oscillation frequency of the reflection surface of the resonance type deflection element 71, etc. And a horizontal scanning angle detection circuit 73 for detecting the swinging state.

  In the present embodiment, the horizontal scanning angle detection circuit 73 inputs a signal indicating the detected oscillation state of the resonance type deflection element 71 to the control unit 110.

  The vertical scanning unit 80 includes a deflection element 81 for scanning the image light in the vertical direction, a vertical scanning control circuit 82 for driving the deflection element 81, and a swing range and fluctuation of the reflection surface by the vertical scanning control circuit 82. And a vertical scanning angle detection circuit 83 for detecting a swinging state such as a dynamic frequency.

  The horizontal scanning drive circuit 72 and the vertical scanning control circuit 82 are driven based on the horizontal driving signal 23 and the vertical driving signal 24 output from the signal processing circuit 21, respectively, and the vertical scanning angle detection circuit 83 detects the detected deflection. A signal indicating the swinging state of the element 81 is input to the control unit 110.

  The control unit 110, which will be described in detail later, controls the operation of the signal processing circuit 21 and adjusts the horizontal drive signal 23 and the vertical drive signal 24 to thereby adjust the horizontal scanning unit 70 and the vertical scanning unit 80. An image to be displayed is formed by changing the scanning angle of the image light.

  The scanning angle thus changed is detected by the control unit 110 based on detection signals from the horizontal scanning angle detection circuit 73 and the vertical scanning angle detection circuit 83, and is horizontally driven via the signal processing circuit 21 and the horizontal scanning drive circuit 72. It is fed back to the signal 23 and also fed back to the vertical drive signal 24 via the signal processing circuit 21 and the vertical scanning control circuit 82.

  The relay optical system 75 that relays the image light between the horizontal scanning unit 70 and the vertical scanning unit 80 causes the image light scanned in the horizontal direction by the resonance type deflection element 71 to be reflected on the reflection surface of the deflection element 81. Make it incident. The image light incident on the deflecting element 81 is scanned in the vertical direction by the deflecting element 81 and is emitted to the relay optical system 90 as scanned image light scanned two-dimensionally.

  The relay optical system 90 includes lens systems 91 and 94 having a positive refractive power. The display scanning image light emitted from the vertical scanning unit 80 is converted into convergent image light by the lens system 91 so that the respective image lights have their center lines substantially parallel to each other. The scanned image light is converted into substantially parallel image light by the lens system 94 and is converted so that the center line of the image light converges on the pupil Ea of the user 50.

  In the present embodiment, the image light emitted from the optical fiber 100 is scanned in the horizontal direction by the horizontal scanning unit 70 and then scanned in the vertical direction by the vertical scanning unit 80. However, the horizontal scanning unit 70 and the vertical scanning unit 80 may be interchanged so that the image light is scanned in the vertical direction by the vertical scanning unit 80 and then scanned in the horizontal direction by the horizontal scanning unit 70.

  The control unit 110 includes a CPU 101, a flash memory 102 that is a nonvolatile memory, and a RAM 103. The CPU 101, the flash memory 102, and the RAM 103 are respectively connected to a data communication bus, and various information is transmitted / received via the data communication bus.

  In addition, the control unit 110 controls the brightness of the outside world by the power switch SW of the head mounted display 1, the CCD sensor 2 that images the automobile 401, the brightness sensor 8 that detects the brightness (brightness) of the outside world, and the brightness sensor 8. When it is detected that the brightness falls below a predetermined brightness, the LED 3 that illuminates the imaging range of the CCD sensor 2, the operation switch 7 that can be operated by the user 50, and a communication control circuit for controlling communication with other devices 9. An imaging magnification setting unit 11 for setting an imaging magnification by the CCD sensor 2 and an interface 104 that can be connected to an external device such as a personal computer are also connected. Note that the user 50 can set the imaging magnification in the imaging magnification setting unit 11. Here, the imaging magnification set by the imaging magnification setting unit 11 is a predetermined zoom range, for example, 1 to 5 times when performing digital zoom. Further, when an electric zoom lens (not shown) is provided in the CCD sensor 2 and optical zoom is performed, the magnification of the electric zoom lens may be set. For example, it is 1 to 3 times. Note that the imaging magnification set by the imaging magnification setting unit 11 is stored in the flash memory 102.

  The CPU 101 executes various information processing programs stored in the flash memory 102 to operate various circuits (not shown) constituting the head mounted display 1 to execute various functions provided in the head mounted display 1. Device.

  The flash memory 102 includes an information processing program for operating the image light generation unit 20, the horizontal scanning unit 70, the vertical scanning unit 80, and the like when the display control of an image displayed by the head mounted display 1 is performed. Various information processing programs executed by the CPU 101 for overall control of the operation of the entire head mounted display 1 are stored.

  Next, the storage device 300 will be described with reference to FIGS. 1 and 3. This storage device 300 corresponds to the “related image storage means” of the present invention, and the storage device 300 includes a table including related images that are images related to each automobile 401 that may be imaged by the CCD sensor 2. 301 is stored. As shown in FIG. 3, the table 301 includes an automobile classification field 302, an imaging position field 303, a barcode storage field 304, and a related image storage field 305.

  In the automobile classification column 302, information indicating each automobile 401a, 401b, 401c is stored. In the imaging position column 303, information indicating from which direction of the automobile 401 the image is taken is stored. In the bar code storage column 304, numbers indicated by bar codes 402a1, 402a2, 402b1, 402b2, 402c1, and 402c2 previously attached to the respective automobiles 401 are stored. As an example, in the case of any of Japanese JIS standards, European EAN standards, and American UPC standards, 13-digit or 8-digit numbers are stored. Since there are currently about 100 types of barcodes used in the world, the barcode storage column 304 may store a numerical value corresponding to the barcode to be used. Note that an image of the barcode itself may be stored, or both the barcode and a numerical value may be stored. In the related image storage field 305, internal images 403a1, 403a2, 403b1, 403b2, 403c1, 403c2,... Has been. Specifically, an internal image 403a1 of the automobile 401a when the automobile 401a is viewed from the front is stored in association with the barcode 402a1. Further, an internal image 403a2 of the automobile 401a when the automobile 401a is viewed from the side is stored in association with the barcode 402a2. Further, an internal image 403b1 of the automobile 401b when the automobile 401b is viewed from the front is stored in association with the barcode 402b1. Further, an internal image 403b2 of the automobile 401b when the automobile 401b is viewed from the side is stored in association with the barcode 402b2. Further, an internal image 403c1 of the automobile 401c when the automobile 401c is viewed from the front is stored in association with the barcode 402c1. Further, an internal image 403c2 of the automobile 401c when the automobile 401c is viewed from the side is stored in association with the barcode 402c2. Although not described here, the internal image 403 is also stored in association with the barcode 402 for other automobiles 401.

  Next, processing executed by the control unit 110 will be described with reference to the flowchart of FIG. This process starts when the start button included in the operation switch 7 is operated by the user 50. Hereinafter, “display” means that the user 50 visually recognizes an image corresponding to information by scanning image light in a two-dimensional direction on the retina of the user 50.

  First, under the control of the control unit 110, the CCD sensor 2 captures an automobile 401 corresponding to the field of view of the user 50 as a real image 404 (see FIGS. 5 and 6) (S11). Then, the barcode 402 is extracted from the actual image 404 (S12). In this extraction method, an image of a portion indicating the characteristic of the barcode is cut out from the actual image 404 by a known image recognition method, a process such as noise removal is performed, and then the image of the barcode 402 is converted into a numerical value (S12). ). It is specified to which of the barcodes 402a1, 402a2, 402b1, 402b2, 402c1, 402c2,... In the barcode storage column 304 of the table 301 stored in the storage device 300 (S13).

  If there is a matching barcode (S13: YES), then whether or not the imaging magnification set by the user 50 in the imaging magnification setting unit 11 is greater than a predetermined threshold magnification (for example, 2 times). Is determined (S14). When the imaging magnification is equal to or less than the threshold magnification (S14: NO), as shown in FIGS. 5 and 6, the real image 404 captured in S11 is displayed (S17). At this time, the real image 404 is displayed with a size corresponding to the imaging magnification (S17). Thus, the user 50 cannot visually recognize the outside world and cannot directly recognize the automobile 401, but can visually recognize the automobile 401 as the real image 404.

  On the other hand, when the imaging magnification is larger than the threshold magnification (S14: YES), the internal image 403 stored in the storage device 300 is displayed in association with the barcode 402 specified in S13 (S15). At this time, the internal image is displayed in a size corresponding to the imaging magnification (S15). For example, when the barcode 402 specified in S13 is the barcode 402a1, as shown in FIG. 7, an internal image 403a1 of the automobile 401a when the automobile 401a is viewed from the front is displayed. When the barcode 402 specified in S13 is the barcode 402a2, as shown in FIG. 8, an internal image 403a2 of the automobile 401a when the automobile 401a is viewed from the lateral direction is displayed. Thus, the user 50 can visually recognize the inside of the automobile 401 as the internal image 403. If the barcode is not specified in the determination process of S13 (S13: NO), the actual image is displayed (S17).

  Thereafter, whether or not to end the process shown in the flowchart of FIG. 4 is determined based on whether or not the end button included in the operation switch 7 is operated by the user 50 (S16). If the end button is operated (S16: YES), the process shown in the flowchart of FIG. 4 is ended. On the other hand, when the end button is not operated (S16: NO), the process returns to S11. Accordingly, the display is switched between the internal image 403 and the real image 404 of the automobile 401 when the user 50 keeps the position where the user 50 is present and the user 50 increases or decreases the imaging magnification to a predetermined value or less. (S11 to S17). When the line of sight of the user 50 moves and the CCD sensor 2 images another car 401 (S11), the bar code 402 attached to the other car 401 is extracted (S12). The bar code 402 is specified by the characteristic (S12), and the real image 404 or the internal image 403 showing another car 401 is displayed (S11 to S17).

  As described above, according to the head mounted display 1 of the present embodiment, the automobile 401 indicated by the real image 404 captured by the CCD sensor 2 is specified by the barcode 402. When the imaging magnification is smaller than the predetermined value, the real image 404 is displayed. On the other hand, when the imaging magnification is higher than the predetermined value, the internal image 403 corresponding to the actual image 404 is displayed.

  Thus, the user 50 can set the imaging magnification, and the real image 404 or the internal image 403 is displayed on the head mounted display 1 based on the imaging magnification. Therefore, the real image 404 and the internal image 403 can be easily switched based on the intention of the user 50.

  Further, according to the head mounted display 1, the user 50 can visually recognize the internal image 403 of the automobile 401. The internal image 403 is an image corresponding to the position where the user 50 views the automobile 401. Therefore, the user 50 can grasp the inside of the automobile 401 from multiple directions.

  Further, according to the head mounted display 1, since the image to be displayed is determined as the real image 404 or the internal image 403 according to the imaging magnification, the user 50 can easily switch the image to be displayed. Further, it is generally considered that the user 50 intuitively sets the imaging magnification larger for the automobile 401 that is interested, that is, the automobile 401 that wants to display the internal image 403. Therefore, the real image 404 is displayed when the imaging magnification is smaller than the predetermined value, while the internal image 403 is displayed when the imaging magnification is greater than or equal to the predetermined value. As a result, it is considered that the user 50 can easily perform an operation for determining which image to display.

  Moreover, according to the head mounted display 1, since the automobile 401 is specified based on the barcode (identification information), the automobile 401 captured by the CCD sensor 2 can be specified reliably.

  The head mounted display 1 according to the present invention is not limited to the above embodiment, and various modifications may be made without departing from the spirit of the head mounted display 1. For example, in the above embodiment, the barcode 402 is used to specify the automobile 401 and the imaging position. However, other identification information such as a QR code (registered trademark) or an IC tag is attached to the automobile 401, and the barcode 401 is used. The automobile 401 and the imaging position may be specified based on the identification information.

  Further, in the above embodiment, the imaging positions are limited to the front and side of the automobile 401, but the internal image 403 is displayed from multiple directions by adopting more imaging positions such as rearward and diagonally forward. You may make it do. In this case, it is necessary to prepare an internal image 403 corresponding to each direction. In addition, as a method for determining where the user 50 is located with respect to the automobile 401, the position can also be determined based on, for example, the size or inclination of the barcode 402 included in the real image 404. .

  As an example, as shown in FIG. 9, when the user is positioned in front of the barcode 402 attached to the automobile 401, the barcode is photographed without distortion. On the other hand, when the user is at an oblique position with respect to the barcode 402 attached to the automobile 401, the barcode is photographed with distortion. Therefore, by detecting this distortion, it is possible to determine the positional relationship of the user with respect to the automobile 401 (see FIG. 9).

  In addition, the position of each car 401 is stored in advance, the position of the user 50 is detected by a GPS receiver, and based on the position of the car 401 and the position of the user 50, the user 50 It may be determined at which position. Further, as described in Japanese Patent Publication No. 3422383, light emitters are attached to the four corners of the automobile 401, and in the real image 404, the user 50 moves to the automobile 401 based on the relative positions between the light emitters. In contrast, it may be determined at which position.

  In the above embodiment, the internal image 403 showing the inside of the automobile 401 is used as the related image. However, as the related image, for example, a 3D CAD line of the automobile 401, a model name, an image showing fuel consumption, and the like are related to the automobile 401. If it is an image to be displayed, its content does not matter. In the above-described embodiment, the imaging target is the automobile 401, but another real object may be the imaging target.

  In the above embodiment, the real image 404 and the internal image 403 are switched based on the imaging magnification. However, the real image 404 is based on other imaging conditions such as the distance between the user 50 and the automobile 401 to be imaged. And the internal image 403 may be switched. When switching between the real image 404 and the internal image 403 based on the distance between the user 50 and the car 401 to be imaged, for example, the ratio of the image corresponding to the car 401 to the entire area of the real image 404 or the barcode 402 The distance may be determined by the size of or by focus adjustment.

  In the above embodiment, the storage device 300 is provided separately from the head mounted display 1 and connected via the network 200. However, the storage device 300 is configured by a flash memory or the like and integrated with the head mounted display 1. May be.

1 Head mounted display 2 CCD sensor (imaging means)
5 Head Mounted Display System 11 Imaging Magnification Setting Unit 110 Control Unit 300 Storage Device (Related Image Storage Unit)
401 Car (actual)
402 Bar code (identification information)
403 Internal image (Related image)
404 real image S13 real identification means S14 selection means

Claims (5)

A head mounted display mounted on the user's head and provided with a display means for displaying an image in the user's eyes;
An imaging unit capable of imaging a real object corresponding to the visual field of the user wearing the head mounted display as a real image, and allowing the user to set an imaging condition when imaging the real object,
A head-mounted display used in a head-mounted display system including a related image storage unit that stores a related image, which is an image related to the real image captured by the imaging unit, in association with the real object,
Selection means for selecting one of the real image and the related image based on the imaging condition;
When the real image is selected by the selection means, the real image is displayed on the display means, and when the related image is selected, the real image is stored in the related image storage means corresponding to the real object. And a display control means for displaying the related image being displayed on the display means. A real object specifying unit for specifying an actual object included in the real image captured by the image capturing unit;
2. The head mounted display according to claim 1, wherein the display control unit displays the related image stored in the related image storage unit in correspondence with the real object specified by the real object specifying unit. The related image storage means stores, as the related image, an internal image corresponding to a position where the user sees the actual object, corresponding to each position,
The real object specifying means also specifies a position where the specified real object is imaged,
The display control means displays the internal image stored in the related image storage means on the display means corresponding to the position specified by the real object specifying means when displaying the related image. The head mounted display according to claim 2. An imaging magnification can be set as the imaging condition,
The selection means selects the actual image when the imaging magnification is equal to or smaller than a predetermined threshold value, and selects the related image when the imaging magnification is larger than the predetermined threshold value. The head mounted display according to any one of 1 to 3. The real thing imaged by the imaging means is attached with identification information for identifying itself,
The head mounted display according to any one of claims 1 to 4, wherein the real object specifying unit specifies the real object imaged by the imaging unit based on the identification information included in the real image. .

Images