JP6629528B2 - Virtual reality display system, virtual reality display method, and computer program - Google Patents

Description

  The present invention relates to a system for displaying a three-dimensional image of a virtual reality world, and more particularly to a virtual reality display system, a virtual reality display method, and a computer program which can reduce discomfort and can be easily used.

  Conventionally, a head mounted display (hereinafter, referred to as an HMD (Head Mounted Display)) has been used as a viewer of virtual reality. In general, the HMD is often used as a display for displaying an image that requires an immersive feeling, such as a shooting game and an adventure game.

  HMDs with a wide viewing angle have been widely used due to advances in miniaturization and miniaturization of displays, and advances in sensor technology and cost reduction. Accordingly, in recent years, many contents using the HMD have been proposed.

  Even in the field of home sales, HMD can be used to display CG images showing the state viewed from indoor and outdoor viewpoints based on three-dimensional CAD data, so that end users can experience life in houses to be built. A display method of a CG image that realizes a possible walk-through has been proposed (Patent Documents 1 and 2, etc.).

JP-A-11-316855 JP-A-09-106409

  However, the display methods disclosed in Patent Literatures 1 and 2 aim at realizing walk-through. Therefore, a part of the body movement of the wearer of the HMD, the movement of the hand, and the direction of the line of sight are captured by the sensor to change the image, but the displayed moving image, switching of the image, and the like are performed by the operator. Thus, the end user who is the wearer was passive for operations such as switching. Since it is difficult for the wearer to actively switch and change the image, when using the HMD, discomfort such as three-dimensional sickness (3D sickness) may occur.

  Patent Literature 2 proposes a method of actively operating a hand-operated controller such as a joystick while wearing an HMD. However, in order to operate the controller while the HMD deprives the real world of the field of view, technology or familiarity is required, and it has been difficult for the home sales customer group to use the controller.

  Further, the display method disclosed in Patent Literature 1 is intended to realize a walk-through, and it is necessary to capture the movement of the whole body of an end user who is a wearer, so that the necessary devices are large-scale. . Furthermore, in order to capture the movement of the whole body of the wearer who is immersed in the HMD, a space that can safely and freely move is required. Thus, it was not easy to use for the end user.

  The present invention has been made in view of such circumstances, and it is an object of the present invention to provide a virtual reality display system, a virtual reality display method, and a computer program which can reduce discomfort to a wearer and can be easily used. Aim.

  A virtual reality display system according to the present invention includes a head-mounted display device that displays an image, and an image output device that outputs a three-dimensional image to the display device, and outputs a three-dimensional image according to virtual reality to the display device. The virtual reality display system further includes a detecting unit that detects an operation of a wearer of the display device, wherein the image output device displays the three-dimensional image while the operation is detected by the detecting unit. Means for creating and outputting an image in which an operation object image drawn in three dimensions is superimposed on the image, an operation detected by the detection unit, and a display position where the operation object image is displayed And changing means for changing the three-dimensional image being displayed based on the above.

  In the virtual reality display system according to the present invention, the three-dimensional image is an indoor or outdoor image of a building, includes an image of equipment installed indoor or outdoor, and the object image includes an image of a button. When the operation detected by the detection unit is an operation of pressing the button, the changing unit switches the image of the facility to another image and changes the image.

  In the virtual reality display system according to the present invention, the three-dimensional image is an indoor or outdoor image of a building, the object image includes a slide bar image, and the changing unit detects an operation detected by the detection unit. Is an operation of tracing the slide bar, wherein the indoor or outdoor image is continuously changed.

  A virtual reality display method according to the present invention includes a head-mounted display device that displays an image, a detection device that detects an operation of a wearer of the display device, and an image output device that outputs a three-dimensional image to the display device And a virtual reality display method for displaying a three-dimensional image related to virtual reality on the display device, wherein the image output device displays the three-dimensional image while the detection device detects an operation. In the image, an image in which an operation object image drawn in three dimensions is superimposed is created and output, and the operation movement detected by the detection device and the display position where the operation object image is displayed The three-dimensional image being displayed is changed based on the above.

  A computer program according to the present invention causes a processor connected to a display unit that displays an image to receive a detection result from a detection unit that detects an operation of a user who visually recognizes the display unit, and performs a virtual process according to the detection result. In a computer program for outputting a three-dimensional image according to reality to the display unit, the processor is configured to perform an operation for three-dimensionally drawing in a three-dimensional image being displayed while detecting an operation by the detection unit. A three-dimensional image being displayed based on the movement of the operation detected by the detection unit and the display position where the object image for operation is displayed, by creating and outputting an image in which the object image of the object is superimposed. Is changed.

  In the present invention, a detection unit that detects an operation of a wearer who wears a head-mounted display device that displays a three-dimensional image according to virtual reality is further provided, and an operation object is detected while the operation is detected. The image is displayed three-dimensionally, so that the wearer can actively change the image at his / her desired timing by his / her own operation.

  In the present invention, the object image includes the image of the button, and the image is switched according to the operation of pressing the button. When a button is displayed in front of the eyes in the virtual reality space, the operation can be easily performed by a natural movement of trying to press the button.

  In the present invention, the object image includes the image of the slide bar, and the image continuously changes according to the operation of tracing the slide bar. When displaying a three-dimensional image on an HMD with a wide viewing angle, continuous changes may cause discomfort, but discomfort is reduced by the wearer's active operation.

  According to the present invention, the wearer of the HMD can actively change the displayed image by his / her own operation, thereby avoiding being passive and reducing discomfort. In addition, since a controller is not provided separately, familiarity or technology for operating the controller is not required, so that the operation can be performed intuitively and can be easily used.

FIG. 1 is a schematic diagram illustrating a virtual reality display system according to the present embodiment. FIG. 1 is a block diagram illustrating a configuration of a virtual reality display system according to the present embodiment. 13 is a flowchart illustrating an example of a processing procedure executed by a control unit of the information processing device. FIG. 5 is an explanatory diagram two-dimensionally expressing an image that can be visually recognized by a wearer through the HMD according to the present embodiment. FIG. 5 is an explanatory diagram two-dimensionally expressing an image that can be visually recognized by a wearer through the HMD according to the present embodiment. FIG. 5 is an explanatory diagram two-dimensionally expressing an image that can be visually recognized by a wearer through the HMD according to the present embodiment. FIG. 5 is an explanatory diagram two-dimensionally expressing an image that can be visually recognized by a wearer through the HMD according to the present embodiment. FIG. 5 is an explanatory diagram two-dimensionally expressing an image that can be visually recognized by a wearer through the HMD according to the present embodiment.

  Hereinafter, the present invention will be specifically described with reference to the drawings showing the embodiments. The embodiment described below is an example, and the present invention is not limited to the following configuration.

  FIG. 1 is a schematic diagram illustrating a virtual reality display system according to the present embodiment, and FIG. 2 is a block diagram illustrating a configuration of the virtual reality display system according to the present embodiment. The virtual reality display system includes an HMD 1 that is a head-mounted display device, a headphone-type speaker 2, and an information processing device 3 that is an image output device.

  The HMD 1 includes a display unit 11, a motion detection unit 12, a position detection unit 13, an operation detection unit 14, and a connection unit 15. The display unit 11, the motion detection unit 12, a part of the position detection unit 13, and the operation detection unit 14 are provided on the main body of the HMD 1, and the other part of the position detection unit 13 and the connection unit 15 are separately provided. It is connected to the main unit via a cable.

  The display unit 11 includes, for example, a small liquid crystal display and an optical lens, and can display a three-dimensional image with a viewing angle of 110 ° or more. The display unit 11 receives and displays an image signal (including a video signal) output from the information processing device 3.

  The motion detection unit 12 collects signals from a plurality of three-axis acceleration sensors and gyro sensors provided in various directions on the main body (the cover and the mounting belt of the display unit 11) and signals from these sensor groups. And a control circuit for outputting. The motion detector 12 detects the motion of the head of the wearer.

  The position detection unit 13 uses a mounting type infrared camera 131 as a separate body, and directly emits light emitted from infrared LEDs 132 that are provided in various parts of the main body and emits infrared rays outward and emits infrared rays. Detect. The infrared camera 131 is mounted at an appropriate position where the infrared camera 131 can capture the main body. The position detection unit 13 detects the intensity of the irradiation light from the infrared LED 132 provided on the surface of the main body by the infrared camera 131, thereby calculating and outputting the distance of each location of the main body of the HMD 1 with respect to the mounting location. The circuit further includes a circuit.

  The operation detection unit 14 uses two infrared cameras arranged side by side on the outer surface of the main body, and an infrared LED provided to radiate infrared rays similarly to an intermediate position between the infrared cameras. Specifically, as shown in FIG. 1, the operation detection unit 14 is provided outward at a location located beyond the line of sight on the outer surface of the main body with reference to the position where the eye is applied on the main body. The operation detection unit 14 is configured to detect the reflection of the light emitted from the infrared LED by the two infrared cameras based on a result of the user's finger in a real space within a predetermined range based on the position of the infrared camera or the infrared LED. It is a device that detects the position. The operation detection unit 14 calculates the coordinates of each part (for example, the position of the back of the hand and the joint of each finger) on the finger based on the image of the finger captured by the two infrared cameras, and outputs information of the calculated coordinates. Including circuits.

  The connection unit 15 is provided as a separate body connected to the main body of the HMD 1 by a cable. The connection unit 15 synchronizes and outputs a signal output from the motion detection unit 12 and a signal related to a distance output based on the infrared camera 131 of the position detection unit 13.

  The speaker 2 receives and outputs an audio signal output from the information processing device. Note that the speaker 2 may be configured integrally with the HMD 1. Note that the speaker 2 is not essential.

  The information processing apparatus 3 uses a notebook (laptop) PC (Personal Computer) as shown in FIG. 1, and includes a control unit 30, a temporary storage unit 31, a storage unit 32, a display unit 33, an operation unit 34, An output unit 35, an image output unit 36, and an audio output unit 37 are provided. Note that the information processing apparatus 3 is not limited to a notebook PC, and may be, for example, a desktop PC or a server computer. A tablet-type computer may be used as long as it can draw an image at a sufficient speed and transmit and receive an image signal and a control signal simultaneously and in parallel. The audio output unit 37 is not essential.

  The control unit 30 uses a CPU (Central Processing Unit). The control unit 30 causes the information processing device 3 to perform various functions by reading and executing various programs stored in the storage unit 32. The temporary storage unit 31 uses a RAM such as a dynamic random access memory (DRAM). The temporary storage unit 31 temporarily stores information generated by the processing of the control unit 30.

  The storage unit 32 uses a nonvolatile storage medium such as a hard disk or a flash memory. The storage unit 32 stores a display control program 3P for realizing image creation and output for displaying a three-dimensional image on the HMD 1. The storage unit 32 further stores image data 3I including information on a three-dimensional object image serving as a material of an image signal to be created.

  The display unit 33 uses, for example, a liquid crystal display. Of course, the display unit 33 may use another type of display such as an organic EL (Electro Luminescence). The control unit 30 causes the display unit 33 to display various screens including characters or images. The display unit 33 is also used as a monitor of an image based on the image signal output to the HMD 1, and allows the operator and a plurality of other persons to recognize the contents of the image visually recognized by the wearer in real time. The operation unit 34 uses an input interface such as a keyboard and a pointing device. The control unit 30 controls each component based on the operation information input through the input interface. The display unit 33 and the operation unit 34 may be integrally configured as a display with a built-in touch panel.

  The input / output unit 35 uses a USB (Universal Serial Bus) interface. The control unit 30 inputs and outputs signals related to control with an external device via the input / output unit 35. In the present embodiment, the control unit 30 inputs a signal output from the motion detection unit 12 and the position detection unit 13 of the HMD 1 and a signal output from the operation detection unit 14 via the input / output unit 35.

  The image output unit 36 includes HDMI (High Definition Multimedia Interface: registered trademark) and a graphic board for rendering a three-dimensional image at high speed. Based on the image data 3I stored in the storage unit 32, the image output unit 36 controls the right-eye image and the left-eye image to be visually recognized as a three-dimensional image based on the image data 3I stored in the storage unit 32. Create and output an image signal.

  The audio output unit 37 outputs an audio signal under the control of the control unit 30 based on the audio data stored in the storage unit 32.

  In the virtual reality display system having the above-described hardware configuration, the motion detection unit 12 and the position detection unit 13 detect the movement and position of the head of the wearer of the HMD 1 and output the signal to the information processing device 3. The image signal of the image created by the information processing device 3 based on the movement and the position is input and displayed on the display unit 11. This allows the wearer to display an image relating to virtual reality according to his / her own motion. In the virtual reality display system according to the present embodiment, in particular, a virtual reality image from the viewpoint of a building indoors or outdoors is displayed. For example, it is used in a house exhibition hall, a model room exhibition hall, a meeting place before construction work or renovation work, or the like so that the owner can easily grasp the inside and outside images of the building to be constructed.

  FIG. 3 is a flowchart illustrating an example of a processing procedure executed by the control unit 30 of the information processing device 3. When executing the display control program 3P stored in the storage unit 32 as an application program, the control unit 30 starts the following processing.

  First, the control unit 30 displays an operator screen on the display unit 33 and accepts selection of the type of the wearer of the HMD 1 (step S1). The types are, for example, gender and height. The content of the displayed image is changed depending on the gender, and the height of the initial line of sight is set according to the height. The change of the content of the image by gender is, for example, as follows. If the wearer is a woman, the image is an indoor image of the kitchen, and the user can experience the height of the kitchen equipment, the height of the cabinet, the conductive lines in the kitchen or between the kitchen and the washroom, and the colors of the kitchen equipment. Image for When the wearer is a male, the image is a skeleton display of the structure of the building, and an image that allows the user to experience the process of deterioration over time.

  Next, the control unit 30 inputs the image signal of each image for the left eye and the right eye through the input / output unit 35 based on the image data 321 stored in the storage unit 32 according to the type selected in step S1. By making a change based on the signals from the motion detection unit 12 and the position detection unit 13, the image output unit 36 starts creating and outputting an image signal of a three-dimensional image that follows the head direction of the wearer (step). S2). At this time, the image signals of the left-eye image and the right-eye image are created such that the three-dimensional model image of the nose can be visually recognized in the image. Further, the image signals of the left-eye image and the right-eye image when the wearer faces downward are created so that the images include images corresponding to the steps from the chest to the feet. In the three-dimensional image displayed in this way, an image signal is created so that a part of the body to be seen is displayed. Thereby, three-dimensional sickness in the HMD 1 is reduced.

  The control unit 30 determines whether or not the finger of the wearer is detected by the operation detection unit 14 within a predetermined range beyond the line of sight of the wearer outside the HMD 1 (step S3).

  If it is determined in step S3 that the operation detection unit 14 has detected a finger (S3: YES), the control unit 30 converts the three-dimensional object image for operation into a three-dimensional object image that follows the movement of the wearer's head. An image in which the image is superimposed on the image and further the image of the finger is generated and output (step S4). The three-dimensional object image is, for example, a three-dimensional image such as a sphere (scroll), a push button, a rocker switch, and a slide bar. A plurality of types of object images may be displayed side by side. As a result, while the finger is detected within the predetermined range, the operation object image is displayed so as to float in the field of view, and further based on the positions of the back of the hand and various positions of the finger detected by the operation detection unit 14, The image of the finger moves and is displayed in real time in accordance with the movement of the finger (see FIGS. 5 to 8).

  The control unit 30 determines a three-dimensional object image based on the display position in the displayed three-dimensional image, the position of the finger detected by the operation detection unit 14, and the detected movement of the finger. It is determined whether an operation has been performed on the displayed object image (step S5). For example, the control unit 30 determines that an operation has been performed when the operation detection unit 14 detects a movement of pushing with a finger at a display position of an object image that is a push button. The control unit 30 determines that an operation has been performed when the operation detection unit 14 detects a movement of tracing a finger in the left-right direction at the display position of the object image as a slide bar.

  If it is determined in step S5 that the operation has been performed (S5: YES), the control unit 30 determines the type of the object image to be operated (one of scroll, push button, rocker switch, slide bar, etc.) and the motion The content of the operation is identified according to the position and the type of (Step S6), and the displayed three-dimensional image is changed according to the identified content of the operation (Step S7). For the operation of “pressing a button” and the operation of “tracing a slide bar”, the signal patterns indicating the position of the back of the hand and the position of the finger joint are learned, and the identification is performed based on the pattern matching. It is possible.

  In step S7, for example, every time an operation of pressing a push button is detected, the control unit 30 checks the facilities, furniture, or home appliances (for example, the type or color of flooring, the type or color of wallpaper, or the type of baseboard) in the indoor image. Or color, door type or color, window frame type or color, kitchen panel type or color, closet type or color, curtain type or color, table and chair type or color, bed type or color, light Of the home appliance, the type or color of the home appliance, etc.). In the case of an outdoor image, exterior (type or color of roof, type or color of exterior wall panel, type or color of exterior wall painting, etc.), equipment (type or color of door and window, rooftop equipment), or external equipment (gate) , Fence, approach, garage, parking space, garden, deck or planting type, color, etc.). The control unit 30 may change the type (drawing, falling down, turning, fitting, attaching a grid, etc.) or the size (lumbar window, sweep window, etc.) of the window, equipment or home appliance (kitchen counter). , Cabinets, tables, home appliances) and the height of the ceiling may be changed. In addition, for example, when an operation of switching the rocker switch is detected, the control unit 30 executes switching between indoor and outdoor day and night, switching on and off the light, and changing the type of light (color of light). Further, for example, when an operation of tracing a scroll or a slide bar is detected, the control unit 30 adjusts the brightness of the indoor / outdoor image according to the elapse of daytime in accordance with the scroll position or the position of the knob portion of the slide bar. , The solar radiation angle in the indoor and outdoor images is continuously changed. In addition, the control unit 30 continuously changes the simulation image of the aging deterioration inside and outside the building according to the position of the knob portion of the slide bar.

  In step S7, the control unit 30 may switch on / off the output of the sound according to the content of the identified operation as well as the change of the three-dimensional image. For example, when an operation of turning on the rocker switch is detected, the control unit 30 outputs a life sound from the audio output unit 37. The living sound is, for example, an outdoor sound, a running water sound, a talking voice, a door opening / closing sound, a collision sound when an object falls on the floor, a sound heard downstairs, and the like. In combination with the above-described change in the type of equipment, furniture, or home appliances, the wearer can also experience the difference in living sounds depending on the type of flooring, the type of door, the type of window, and the like by his / her own operation. .

  If it is determined in step S5 that no operation has been performed (S5: NO), control unit 30 returns the process to step S3.

  After step S7, control unit 30 determines whether or not an operation of terminating display has been performed by the operator or the wearer (step S8). If it is determined in step S3 that the operation detection unit 14 has not detected a finger in the predetermined range (S3: NO), the process proceeds to step S8. When it is determined in step S8 that the operation of terminating the display has been performed (S8: YES), the control unit 30 stops the creation and output of the three-dimensional image by the image output unit 36 (step S9), and ends the processing. . If it is determined in step S8 that the operation of ending the display has not been performed (S8: NO), control unit 30 returns the process to step S3.

  In addition, when equipment, furniture, or home appliances in the displayed three-dimensional image are included, the control unit 30 further causes the operation detection unit 14 to set the position of the finger to a position corresponding to the location of the equipment, furniture, or home appliances. If a movement that detects the movement and the operation thereof is detected, the equipment, furniture, or home appliance to be operated may be changed according to the movement of the finger. The displayed three-dimensional image is drawn from three-dimensional object information of each of a wall (outer wall), a roof, a floor, and furniture, and it is determined whether or not any object is being operated by a finger. It is possible to detect according to the position. For example, if an image of the kitchen counter is included, an image of the kitchen equipment placed on the counter is displayed on the kitchen counter, and the kitchen equipment moves in the three-dimensional image according to the movement of the wearer's finger. To do. This allows the wearer to experience the height of the counter. In the same manner, it is also possible to experience the lifting and lowering of luggage to a storage such as a shelf or a closet in a three-dimensional image. For example, when an operation of touching a wall surface such as an inner wall, an outer wall, or a kitchen is detected by the operation detection unit 14, the color or type of the wall to be operated (the type of finishing such as cloth, painting, or tile). May be switched. Furthermore, regarding the electric wiring, when an image of a switch of indoor / outdoor lighting is drawn on a wall surface or the like, and the control unit 30 detects the position of the finger at the position of the switch by the operation detection unit 14, the three-dimensional image being displayed is displayed. The image may be switched between a light-on image and a light-off image. As described above, any of the objects (equipment and the like) included in the indoor and outdoor images being displayed may be used as object images for operation in the virtual reality space.

  When the operation detection unit 14 detects the position of the finger at the position corresponding to the location of the specific equipment, furniture, or home appliance, a predetermined animation image may be displayed. At this time, the sound output unit 37 may output a sound accompanying opening and closing. For example, when a specific piece of furniture is a door, an animation image for opening and closing the door is displayed when a finger points at the door. Thus, the wearer can enjoy a sense of reality in the virtual reality world.

  Hereinafter, the virtual reality image displayed by the above-described processing will be described in detail using an example. 4 to 8 are explanatory diagrams two-dimensionally expressing images that can be visually recognized by the wearer through the HMD 1 in the present embodiment. FIG. 4 shows an image that can be visually recognized when the wearer is standing in the kitchen. In the displayed image, the height of the line of sight is set according to the height information of the wearer (S2). FIG. 5 is an example of an image when the operation object images O1 to O3 are superimposed on the image of FIG. The image H of the finger in FIG. 5 is displayed in accordance with the positions of the back of the hand and the positions of the fingers detected by the operation detection unit 14. According to the type of the wearer set in advance (S1), the displayed finger image H may be, for example, a female finger or a male finger. As shown in FIG. 5, the operation object images O1 to O3 are displayed as floating in the virtual reality space, and the image of the kitchen to be visually recognized behind the object images O1 to O3 is translucent. It is made visible. In FIG. 5, a plurality of push buttons O1, a slide bar O2, and a slide type button O3 are displayed side by side as operation object images O1 to O3.

  As shown in FIG. 5, for example, a color sample is displayed below each of the push buttons O1. As shown in FIG. 6, when an operation of tapping at a position corresponding to the push button O1 is detected by the operation detection unit 14, the control unit 30 identifies that the operation of pressing the push button O1 has been performed (S6). The control unit 30 changes the color of the kitchen panel, for example, to the color of the color sample below the pushed push button O1 (S7). This allows the wearer to experience the color atmosphere of the entire kitchen facility by changing the color. The wearer can actively execute the operation of changing the display without receiving explanation by a natural operation of pressing the visually recognized button.

  5 and 6 show windows in the kitchen. As shown in FIG. 5, when the wearer performs an operation of sliding the button O3 up or down at a position corresponding to the button O3 for switching the type of window, and the operation detection unit 14 detects this, the control unit 30 Identifies that the operation of switching the button O3 has been performed (S6), and the control unit 30 changes the size of the window so as to switch between the “lumbar window” and the “sweep window” (S7). This allows the wearer to experience the difference in the atmosphere of light entering the kitchen depending on the size of the kitchen window.

  In this manner, the wearer of the HMD 1 can experience a change in the impression received from the difference in the color or type of the equipment. Since the changed image is also displayed on the display unit 33 of the information processing device 3, it is possible for the operator (sales rep) and a plurality of other persons to grasp what color the reaction of the wearer was good. Yes, and because multiple people can share information, for example, in specification specification, bridge the recognition gap between the provider and the owner, and between the owner's family, and make the selection of equipment and specification determination more efficient. Can be.

  7 and 8 show images that can be visually recognized when the wearer is standing in a certain room. In the image displayed also in FIG. 7, the eyes are set according to the height information of the wearer. 7 and 8, only the slide bar O2 is superimposed and displayed as the operation object images O1 to O3. Also in FIGS. 7 and 8, the object images O1 to O3 are displayed as floating in the space, and the images in the room to be visually recognized behind the object images O1 to O3 are made translucent. It is.

  As illustrated in FIG. 7, when an operation that traces the knob of the slide bar O2 is detected by the operation detection unit 14, the control unit 30 identifies that the operation that traces the knob of the slide bar O2 is performed (S6). ), The control unit 30 changes the solar radiation angle in the displayed three-dimensional indoor image corresponding to the time corresponding to the position of the knob portion (S7). FIGS. 7 and 8 show a state in which the shadow of the window according to the solar radiation angle is projected into the room. 7 and 8, FIG. 8 shows a state in which time has advanced from that in FIG. 7, FIG. 7 shows a solar radiation angle around morning, and FIG. 8 shows a solar radiation angle around noon. In this way, it is possible to continuously grasp the image of the difference in the solar radiation angle due to the difference in season and time, and for example, the owner can easily grasp the image of the change in the solar radiation angle due to the location conditions, Or the type or size of the window.

  As described above, in the virtual reality display system according to the present embodiment, the wearer can change the image actively at his / her desired timing by his / her own operation, and avoid being passive to the display of the virtual reality. Discomfort is reduced. In the present embodiment, since the information detected by the operation detection unit is limited to the line of sight, that is, the movement of the head and the movement of the finger, not the movement of the whole body, the walking is captured by the sensor and It can be used more easily than a configuration in which walk-through is realized by moving the viewpoint position in a three-dimensional image. Since large-scale equipment is not necessary, for example, in the home sales industry, it is necessary to install the equipment so that it can be easily experienced at the home exhibition hall, and for any customer to try various equipment changes etc. Make it possible. In addition, it can be easily used at the meeting of specification determination performed at each sales office.

  In addition, by creating a three-dimensional image based on the three-dimensional CAD data of the design drawing for each owner and storing it in the storage unit 32 as image data 3I, a specific image can be obtained at the time of specification determination at the design stage. It is also possible to make them feel individually.

  As shown in FIGS. 4 to 8, the three-dimensional image displayed in the present embodiment has a configuration in which the image is displayed at the full viewing angle. This increases the immersion sensation, but may cause discomfort for some wearers. Therefore, a configuration in which switching to a mode in which display is performed so as to narrow the viewing angle may be possible. Similarly, a frame may be displayed in the field of view in order to display an image that is visible from the forehead. Also, there is a case where three-dimensional sickness may occur for the display following the movement of the head, so fix the viewpoint for a predetermined period after wearing it, fix the viewpoint by the wearer's own operation, and fix the head movement. May be switched between resuming display and following. In addition, in order to suppress three-dimensional sickness, when changing the height of the viewpoint, or when changing the room where the viewpoint is placed, the image is created and output so as to darken the field of view and then switch. It may be.

  Of course, the HMD 1 is not limited to the one shown in the present embodiment. For example, the HMD 1 may be of an eyeglass type. The communication between the HMD 1 and the information processing device 3 may be wireless.

  The virtual reality display system may have a configuration in which the HMD 1 and the information processing device 3 are integrated. Specifically, the HMD 1 is realized by combining a display unit provided in an information terminal device called a tablet terminal device or a so-called smartphone with an optical lens. In this case, the tablet terminal device or the smartphone is incorporated by being inserted into a head-mounted cover body. The function of the information processing device 3 is exerted by causing a processor included in the terminal device to execute the display control program 3 that realizes the above-described processing.

  In addition, the operation detecting unit 14 is not limited to the one using the infrared rays described in the present embodiment. A so-called motion capture using a visible light camera, a depth sensor, or the like may be used, and the function of the position detection unit 13 may be realized by the motion capture. For example, the operation detection unit 14 is a device that detects the user's line of sight, and when the line of sight is substantially stationary in any of the images, it is assumed that the object image displayed at that position is selected. The operation may be detected by the line of sight. The operation detection unit 14 is a device that detects a user's utterance, and may be a device that receives a command related to an operation by voice recognition, and recognizes the utterance of a color name to switch the color of the facility. Is also good.

  The embodiment disclosed this time is an example in all points, and should be considered as not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

1 HMD (display device)
12 Motion detection unit 14 Operation detection unit (detection device)
3 Information processing device (image output device)
30 control part (change means)
32 storage unit 36 image output unit 3P display control program

Claims (6)

In a virtual reality display system that includes a head-mounted display device that displays an image and an image output device that outputs a three-dimensional image to the display device, and that displays a three-dimensional image related to virtual reality on the display device,
The display device further includes a detection unit that detects an operation of a wearer of the display device,
The image output device,
Means for creating and outputting an image in which a three-dimensionally drawn object image for operation is superimposed on a three-dimensional image being displayed while the operation is being detected by the detection unit,
Changing means for changing the displayed three-dimensional image based on the operation detected by the detection unit and a display position at which the operation object image is displayed ,
The three-dimensional image is an image of a building indoor or outdoor, including an image of equipment installed indoors or outdoors,
The object image includes a button image,
The virtual reality display system according to claim 1, wherein the change unit changes the three-dimensional image of the facility by switching to another image when the operation detected by the detection unit is an operation of pressing the button . A virtual reality display system that includes a head-mounted display device that displays an image and an image output device that outputs a three-dimensional image to the display device, and that displays a three-dimensional image related to virtual reality on the display device.
The display device further includes a detection unit that detects an operation of a wearer of the display device,
The image output device,
Means for creating and outputting an image in which a three-dimensionally drawn operation object image is superimposed on the displayed three-dimensional image while detecting the operation by the detection unit,
Changing means for changing the three-dimensional image being displayed based on the operation detected by the detection unit and the display position at which the operation object image is displayed;
With
The three-dimensional image is an indoor or outdoor image of a building,
The object image includes a slide bar image,
The changing means includes:
Wherein when the detected operation by detection unit is an operation of tracing the slide bar, the virtual reality display system that characterized by continuously changing the indoor or outdoor images. A head mounted type display device for displaying an image, a detection device for detecting the operation of a wearer of the display device, and an image output device for outputting a three-dimensional image to the display device at including systems, virtual reality A virtual reality display method for displaying the three-dimensional image according to the display device,
The image output device is an indoor or outdoor image of the building being displayed while detecting operation by the detection device, and is a three-dimensional image including an image of equipment installed indoors or outdoors. Inside, a step of creating and outputting an image in which an object image including an image of an operation button drawn in three dimensions is superimposed , and
The operation detected by the detection device presses the button based on the movement of the operation detected by the image output device and the display position at which the operation object image is displayed. If it identified as an operation, a three-dimensional image containing an image of the equipment in the display, and step of changing by switching to another image
Virtual reality display method, which comprises a. A head mounted type display device for displaying an image, a detection device for detecting the operation of a wearer of the display device, and an image output device for outputting a three-dimensional image to the display device at including systems, virtual reality A virtual reality display method for displaying the three-dimensional image according to the display device,
While the image output device is detecting an operation by the detection device, in a three-dimensional image that is an indoor or outdoor image of the building being displayed, an image of an operation slide bar drawn in three dimensions. Creating and outputting an image in which object images including
The operation detected by the image output device based on the movement of the operation detected by the detection device and the display position at which the operation object image is displayed traces the slide bar. If the operation is identified, continuously changing the indoor or outdoor three-dimensional image being displayed ;
Virtual reality display method, which comprises a. A processor connected to a display unit that displays an image causes a processor to receive a detection result from a detection unit that detects an operation of a user who visually recognizes the display unit, and outputs a three-dimensional image related to virtual reality according to the detection result. In a computer program to be output to a display unit,
To the processor,
While the operation is detected by the detection unit, it is an indoor or outdoor image of the building being displayed, and in a three-dimensional image including an image of equipment installed indoor or outdoor , Create and output an image on which the object image including the drawn operation button image is superimposed,
Based on the movement of the operation detected by the detection unit and the display position where the object image for operation is displayed, the operation detected by the detection unit is identified as the operation of pressing the button. that case, the computer program characterized by the image of the facilities included in the three-dimensional image being displayed, changing by switching to another image. A processor connected to a display unit that displays an image causes a processor to receive a detection result from a detection unit that detects an operation of a user who visually recognizes the display unit, and outputs a three-dimensional image related to virtual reality according to the detection result. In a computer program to be output to a display unit,
To the processor,
While the operation is detected by the detection unit, an object image including an operation slide bar image drawn in three dimensions is superimposed on a three-dimensional image that is an indoor or outdoor image of the building being displayed. Create and output the image
If the operation detected by the detection unit is an operation of tracing the slide bar based on the movement of the operation detected by the detection unit and the display position where the object image for operation is displayed, Continuously changing the indoor or outdoor three-dimensional image of
A computer program characterized by the above-mentioned.

Images