JP6648050B2 - Image display device - Google Patents

Description

  The present invention relates to an image display device that displays an image in front of a user.

  2. Description of the Related Art In recent years, a so-called head-mounted display for experiencing mixed reality (MR) in which a virtual object exists in a real space has been known.

  2. Description of the Related Art In an image display device such as a head-mounted display, an imaging unit that captures an image in front of a user is provided, and a display unit that displays an image in front of a user's eyes is provided. Then, a combined image is generated by aligning a captured image obtained by capturing an image in front of the user with the imaging unit and a computer graphics (CG) image representing the virtual object, and the combined image is displayed on the display unit. (For example, Patent Document 1).

JP 2014-112887 A

  In the image display device as disclosed in Patent Document 1, it is necessary to take an image of the front of the user with an imaging unit and to synthesize a CG image with the taken image, which complicates the configuration and reduces the processing load. There was a problem that it became large.

  In view of the above, an object of the present invention is to provide an image display device that can display a composite image with a simple configuration while reducing the processing load.

In order to solve the above problem, an image display device according to the present invention is an image display device including a display unit that displays an image in front of a user, and a virtual image incorporating a device related to a near-future technology is a captured image. A storage unit in which image data of a combined image combined with the detection unit that detects in advance the gaze direction of the user, and a gaze direction detected by the detection unit. A control unit that controls a display image corresponding to the line-of-sight direction to be displayed on the display unit, wherein the display unit is movable integrally with the user, and the captured image is At a predetermined position where the image display device is used, a space which the user will see before using the image display device is imaged in advance.

  The captured image may be captured at the predetermined position with reference to a predetermined reference direction.

  The captured image may be captured at the predetermined position in all directions in a horizontal direction and a vertical direction based on the reference direction.

  According to the present invention, it is possible to display a composite image with a simple configuration while reducing the processing load.

FIG. 2 is a diagram illustrating an external configuration of an image display device. FIG. 2 is a diagram illustrating a configuration of an image display device. FIG. 3 is a diagram illustrating a space in which the image display device is used. FIG. 4A is a diagram illustrating a captured image. FIG. 4B is a diagram illustrating a CG image. FIG. 4 is a diagram illustrating a composite image. It is a figure explaining a display image.

  Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, other specific numerical values, and the like shown in the embodiments are merely examples for facilitating the understanding of the invention, and do not limit the present invention unless otherwise specified. In the specification and the drawings, elements having substantially the same function and configuration will be denoted by the same reference numerals, and redundant description will be omitted. Elements not directly related to the present invention will be omitted. I do.

  FIG. 1 is a diagram illustrating an external configuration of the image display device 1. As shown in FIG. 1, the image display device 1 is a head-mounted display including a main body 10 and a band 12.

  The display unit 20 is provided in the main body unit 10. The display unit 20 is provided so as to be located in front of the eyes of the user when the image display device 1 is mounted on the head of the user. The band portion 12 is made of, for example, a rubber material or a plastic material, and is connected to both ends in the longitudinal direction of the main body portion 10. The image display device 1 is mounted on the head of the user by the band unit 12 so that the display unit 20 is located in front of the user's eyes. Therefore, the display unit 20 is always located in front of the eyes of the user, and can be moved integrally with the user as the head of the user moves.

  FIG. 2 is a diagram illustrating a configuration of the image display device 1. As shown in FIG. 2, the main unit 10 includes a display unit 20, a control unit 22, a detection unit 24, and a storage unit 26. The display unit 20 includes a liquid crystal display or an organic EL (Electroluminescence) display, and can separately display different images to both eyes of the user.

  The control unit 22 is a microcomputer including a central processing unit (CPU), a ROM storing programs and the like, a RAM as a work area, and the like, and controls the image display device 1 overall.

  The detection unit 24 is, for example, a gyro sensor, and detects the direction of the main body 10 by detecting the rotational angular velocities of the main body 10 in three axial directions and the accelerations of each axis, and controls a signal indicating the detected direction. To the unit 22. Therefore, when the image display device 1 is worn on the head of the user, the detection unit 24 detects the direction in which the user is facing in any space, that is, the direction of the user's line of sight.

  The storage unit 26 is a hard disk or a non-volatile memory, and stores image data 28 in advance. As will be described in detail later, the image data 28 is data of a composite image in which a computer graphic image (hereinafter, referred to as a CG image) is composited with a captured image captured in advance.

  The control unit 22 reads out the image data 28 stored in the storage unit 26, and among the composite images shown in the image data 28, displays the display image corresponding to the user's line of sight indicated by the signal transmitted from the detection unit 24. The display is controlled to be displayed on the display unit 20.

  FIG. 3 is a diagram illustrating a space in which the image display device 1 is used. As shown in FIG. 3, the image display device 1 of the present embodiment is provided with furniture such as a table 130 and a chair 132, a kitchen 134, and the like, and is a building for introducing a technology that will be possible in the near future. It is used at a predetermined position (hereinafter, referred to as a use position) 110 in a room 100 such as a (concept house) or a real house.

  The use position 110 is set, for example, substantially in the center of the room 100, and the upper direction in FIG. Therefore, the user faces the reference direction 120 at the use position 110, and wears the image display device 1 in that state.

  FIG. 4A is a diagram illustrating a captured image 200. FIG. 4B is a diagram illustrating the CG image 210. A captured image 200 based on the image data 28 (composite image) stored in the storage unit 26 is obtained by a camera capable of capturing images in all directions (360 °) in the horizontal and vertical directions in the room 100 shown in FIG. An image is taken from the use position 110. Note that the captured image 200 may be an image of a predetermined range in the horizontal and vertical directions.

  Here, the captured image 200 is a moving image embodied by a model in the room 100 in a daily life. FIG. 4A shows a part (still image) of the captured image 200. As shown in FIG. 4A, the captured image 200 has a reference direction 120 at the center in the horizontal direction. Become. Further, the captured image 200 is one in which both left and right ends are connected.

  When the captured image 200 is captured, a CG image 210 (virtual image) to be synthesized with the captured image 200 is created.

  The CG image 210 is, for example, an image for introducing a technology that will be possible in the near future. For example, as shown in FIG. 4B, the CG image 210 displays a three-dimensional map 210a displayed on the table 130 or the current weather and temperature in front of the model in the room 100. It is composed of a virtual panel 210b and the like. These CG images 210 are also created as moving images, similarly to the captured image 200.

  FIG. 5 is a diagram illustrating the composite image 220. As described above, when the CG image 210 is created, a combined image 220 in which the CG image 210 is combined with the captured image 200 is generated. As illustrated in FIG. 5, the combined image 220 is a combined image of the captured image 200 and the CG image 210, and becomes an image (moving image) as if a daily life in the near future were captured. Then, the image data 28 of the composite image 220 is stored in the storage unit 26.

  FIG. 6 is a diagram illustrating the display image 230. In FIG. 6, a region surrounded by a broken line in the composite image 220 is a display image 230 (230a to 230c).

  As described above, the captured image 200 is captured in all directions in the horizontal and vertical directions at the use position 110, and the CG image 210 is also created in all horizontal and vertical directions in accordance with the captured image. ing. Therefore, the composite image 220 is also an image in all directions in the horizontal and vertical directions.

  Then, as shown in FIG. 3, the image display device 1 is worn by a user who is facing the reference direction 120 at the use position 110 in the room 100. At this time, the control unit 22 displays, on the display unit 20, an image portion (display image 230a) corresponding to the reference direction 120, which is the central portion of the composite image 220, as shown in FIG.

  Thus, the user can view, through the display unit 20, a display image 230a that is the same video as the space seen before mounting the image display device 1. In this way, the user has an illusion as if he / she is looking directly at the room 100 where the user is.

  Then, the control unit 22 reproduces and displays the composite image 220 according to an operation via an operation unit (not shown). At this time, the user moves the gaze direction while wearing the image display device 1. Then, the detection unit 24 detects the gaze direction according to the movement of the user's head. Further, based on the signal indicating the line-of-sight direction detected by the detection unit 24, the control unit 22 may, for example, display the display image 230b or the display image 230c corresponding to the line-of-sight direction in the composite image 220 as illustrated in FIG. Is displayed on the display unit 20.

  Accordingly, the user changes the display image 230 in accordance with the movement of the user's line of sight, and thus has the illusion that the room 100 in which the user is located has changed in the near future. In addition, it is possible to experience a sense of reality as if the room 100 in which the user is has changed into a near-future space.

  In particular, in a concept house, in the past, it was necessary to incorporate a device related to near-future technology into the room 100 in order to allow a user to experience the near-future experience. However, there is a limit in the production of the device when incorporating a device relating to the near-future technology, and a large-scale modification is required when trying to change the device.

  On the other hand, in the image display device 1, when the device related to the near-future technology is changed, it is only necessary to change the content of the CG image 210. Becomes

  Here, even if all of the images displayed on the display unit 20 are converted into CG images, it is difficult to generate all of the room 100 where the user is located by a computer. Will be.

  As described above, the image display device 1 displays, on the display unit 20, the composite image 220 in which the CG image 210 is composited with the captured image 200 previously captured at the use position 110 used by the user in the room 100. . As a result, the image display device 1 displays the display image 230 on the display unit 20 with a simple configuration that does not require an imaging unit, reduces the processing load without combining CG images in real time, and reduces the processing load. be able to.

  At this time, since the captured image 200 is captured at the use position 110 used by the user, when the image display device 1 is mounted at the use position 110, the space viewed before mounting the image display device 1 is The display image 230a that is the same video can be provided to the user, and a sense of realism can be given.

  As described above, the preferred embodiments of the present invention have been described with reference to the accompanying drawings, but it goes without saying that the present invention is not limited to such embodiments. It is obvious to those skilled in the art that various changes or modifications can be conceived within the scope of the claims, and it is understood that these also naturally belong to the technical scope of the present invention. Is done.

  For example, in the above embodiment, the case where the captured image 200, the CG image 210, and the composite image 220 are moving images has been described. However, the captured image 200, the CG image 210, and the composite image 220 may be still images.

  In the above embodiment, the captured image 200, the CG image 210, and the composite image 220 described in the above embodiment are merely examples, and the contents of the captured image 200, the CG image 210, and the composite image 220 are not limited.

  Further, in the above embodiment, the case where the image display device 1 is used in the room 100 such as a concept house or a real house has been described. However, the image display device 1 may be used in various rooms, such as a model house and a model room, and outdoors. However, in the image display device 1, a predetermined use position 110 is set, and the captured image 200 needs to be captured at the use position 110. For example, when the furniture is used in a model house or a model room, a state when the arrangement of the furniture is changed, a flow line of a person in the room, and the like are created in the CG image 210 and combined with the captured image 200 to obtain the furniture. It is possible to provide the user with the state when the arrangement is changed and the confirmation of the flow line. Further, in a real house, the arrangement of newly purchased furniture, the state of the interior of the room after renovation, and the like are generated as a CG image 210 and combined with the captured image 200, so that an image when newly purchasing furniture and a renovation are obtained. Later images can be provided to the user.

  Further, in the above embodiment, the case has been described where the display unit 20 can display different images for both eyes of the user. However, the same image may be displayed for both eyes of the user. Good.

  In the above-described embodiment, the image display device 1 captures the captured image 200 as one moving image, and stores the image data 28 of the composite image 220 obtained by combining the captured image 200 with the CG image 210 in the storage unit 26. I remember it. However, the present invention is not limited to this. In addition to capturing captured images corresponding to the positions of both eyes of the user, a CG image is created for these captured images, and a CG image is synthesized for each captured image. Data may be stored in the storage unit.

  In the above-described embodiment, the main body unit 10 is configured to include the display unit 20, the control unit 22, the detection unit 24, and the storage unit 26. However, the main unit 10 may be provided so that at least the display unit 20 is located in front of the user's eyes, and the control unit 22, the detection unit 24, and the storage unit 26 are separately provided from the main unit 10. It may be provided.

  INDUSTRIAL APPLICATION This invention can be utilized for the image display apparatus which displays an image in front of a user.

1 image display device 20 display unit 22 control unit 24 detection unit 26 storage unit 28 image data 200 captured image 210 CG image (virtual image)
220 Composite image 230 Display image

Claims (3)

An image display device including a display unit that displays an image in front of a user,
A storage unit in which image data of a combined image in which a virtual image incorporating a device related to the near future technology is combined with the captured image is stored in advance,
A detection unit configured to detect a gaze direction of the user;
A control unit that controls the display unit to display a display image corresponding to the line-of-sight direction in the composite image based on the line-of-sight direction detected by the detection unit,
With
The display unit is movable integrally with the user,
The image display device, wherein the captured image is an image in which a space to be viewed before the user uses the image display device is captured in advance at a predetermined position where the user uses the image display device. . The captured image is
The image display device according to claim 1, wherein an image is captured at the predetermined position with reference to a predetermined reference direction. The captured image is
The image display device according to claim 2, wherein an image is taken at all the horizontal and vertical directions with respect to the reference direction at the predetermined position.

Images