WO2015125508A1 - 頭部装着ディスプレイ、制御装置および制御方法 - Google Patents
頭部装着ディスプレイ、制御装置および制御方法 Download PDFInfo
- Publication number
- WO2015125508A1 WO2015125508A1 PCT/JP2015/050318 JP2015050318W WO2015125508A1 WO 2015125508 A1 WO2015125508 A1 WO 2015125508A1 JP 2015050318 W JP2015050318 W JP 2015050318W WO 2015125508 A1 WO2015125508 A1 WO 2015125508A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- wearer
- information
- head
- unit
- display
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00043—Operational features of endoscopes provided with output arrangements
- A61B1/00045—Display arrangement
- A61B1/00048—Constructional features of the display
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0179—Display position adjusting means not related to the information to be displayed
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/02—Viewing or reading apparatus
- G02B27/022—Viewing apparatus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/74—Projection arrangements for image reproduction, e.g. using eidophor
- H04N5/7475—Constructional details of television projection apparatus
- H04N5/7491—Constructional details of television projection apparatus of head mounted projectors
Definitions
- the present disclosure relates to a head-mounted display used by a user wearing on the head, a control device that controls the head-mounted display, and a control method thereof.
- HMD head mounted display
- the HMD is a display device that is worn on the user's head and used in recent years as a display device for a user to check information while working in a work environment in addition to a display device such as an AV device or a computer game. in use.
- an HMD is used as a display device for projecting an endoscope image (for example, Patent Documents 1 and 2).
- the surgeon wears the HMD and performs an operation while viewing the video displayed on the HMD.
- an endoscope image it has been common for an endoscope image to be displayed on a monitor installed near the operator, so the operator has to move his / her line of sight frequently between the monitor and the affected area.
- the surgeon can confirm the endoscope image and the affected area displayed on the display unit of the HMD without greatly moving the line of sight.
- a display unit for presenting an image to the wearer, a wearer to be worn on the wearer's head, and a notification for notifying a person other than the wearer of information that the wearer is viewing.
- a head-mounted display is provided.
- the visual information acquisition unit that acquires information visually recognized by the wearer of the head-mounted display, and the mounting of the head-mounted display based on the information visually recognized by the wearer
- a control device includes a notification control unit that controls a notification unit that notifies a person other than the user of information that the wearer is viewing.
- the information that the wearer of the head-mounted display is viewing is acquired, and the person other than the wearer that the head-mounted display has based on the information that the wearer is viewing is obtained.
- a control method for controlling a notification unit for notifying information visually recognized by the wearer is provided.
- information that is visually recognized by the wearer of the head-mounted display is notified to a person other than the wearer by the notification unit.
- a person other than the wearer can recognize what the wearer is looking at.
- FIG. 1 is a system configuration diagram illustrating a configuration example of an endoscope system according to a first embodiment of the present disclosure. It is a schematic side view which shows the state which the user has mounted
- FIG. 1 is a system configuration diagram illustrating a configuration example of the endoscope system 1 according to the present embodiment.
- FIG. 2 is a schematic side view showing a state in which the user is wearing the HMD 100 according to the present embodiment.
- FIG. 3 is a schematic diagram showing the display surfaces 152 and 154 of the HMD 100.
- the user who uses the endoscope system 1 is defined as follows. “Wearer”: person wearing HMD “non-wearer”: person not wearing HMD Also, when there are multiple “wearers”, The wearer of the HMD is also referred to as “other wearer”.
- the endoscope system 1 is a system used in endoscopic surgery, and an operator operates while visually recognizing the state of an affected part imaged by an endoscopic apparatus with an HMD.
- the endoscope system 1 includes an HMD 100 (100A, 100B), a display 300, and an external device 400 (400A, 400B) connected to a processor unit 200.
- the HMD 100 is a display device that displays information such as an input video from the external device 400. As shown in FIG. 2, the HMD 100 is a non-transparent HMD having a goggle shape, for example, and is used in a state where the user wears the head.
- the HMD 100 includes a main body unit 110 including display units 152 and 154 that present information to the wearer of the HMD 100, and an upper fixing unit 120 and a rear fixing unit 130 for fixing the main body unit 110 to the head. When fixed to the wearer's head by the fixing unit, the display units 152 and 154 of the main body 110 are positioned in front of the left and right eyes of the wearer.
- the main body 110 is a part that covers both eyes of the wearer.
- the main body 110 may be configured to cover, for example, the vicinity of the left and right temples of the wearer.
- an imaging unit 112 may be provided on the outer surface of the main body 110 to capture the surrounding environment.
- the wearer of the HMD 100 can recognize information on the surrounding environment that is visible when the HMD 100 is not worn, in addition to information provided from the external device 400 or the like via the processor unit 200 (video see-through). .
- a first display unit 152 for the left eye and a second display for the right eye so as to correspond to the positions of the left and right eyes of the wearer.
- a notch for positioning the wearer's nose may be formed between the first display unit 152 and the second display unit 154.
- a gap may be provided between the eyepiece surface 110a of the main body 110 and the eye of the wearer.
- a first display element (reference numeral 165 in FIG. 8) that presents a left-eye image on the first display unit 152, and a right-eye image on the second display unit 154.
- a second display element (reference numeral 166 in FIG. 8) is provided.
- Each display element presents, for example, an image of an endoscope apparatus provided by the processor unit 200, an image captured by the imaging unit 112 of the main body unit 110, and the like.
- the display control process for images displayed on the display unit of the HMD 100 will be described later.
- the main body 110 is provided with a cable 140 connected to the processor unit 200 in order to transmit and receive information to and from the processor unit 200.
- the HMD 100 and the processor unit 200 are connected by wire, but the present disclosure is not limited to this example, and information communication between devices may be performed by wireless communication.
- the upper fixing unit 120 supports the main body 110 from above the head of the wearer in order to fix the main body 110 to the position of the wearer's eyes.
- the upper fixing portion 120 includes support portions 121 and 124 located at the top of the head, a first adjustment portion 122 that adjusts the position on the front side of the wearer, and a first position that adjusts the height position on the left and right sides of the wearer. 2 adjustment units 125.
- the first adjustment unit 122 is a member that connects the upper portion of the main body 110 and the support unit 121, and the length thereof is configured to be adjustable. By adjusting the length of the first adjustment unit 122, the degree to which the forehead pad 127 installed above the eyepiece surface 110a of the main body 110 is pressed against the forehead of the wearer is changed. The position can be adjusted.
- the second adjustment portion 125 is a member that connects the both side portions of the main body portion 110 extending toward the upper portions of the left and right ears of the wearer and the support portion 124 when worn.
- the height is adjustable.
- the second adjustment unit 125 is provided on each of the left and right sides. By adjusting the length of the second adjustment unit 125, the head pad 122a installed between the first adjustment unit 122 and the head is pressed against the head of the wearer, The height position of both sides is adjusted.
- the rear fixing unit 130 supports the main body 110 from the back of the wearer in order to fix the main body 110 to the position of the wearer's eyes.
- the rear fixing portion 130 is a member that connects both side portions of the main body portion 110 on the rear head side, and is configured to be adjustable in length. By adjusting the length of the rear fixing part 130, the degree to which the side pads 126 provided on both sides of the main body part 110 are pressed against the head is changed, and the degree of support at the side head is adjusted. Can do.
- the information displayed on the display units 152 and 154 of the HMD 100 may be switched by the remote controller 102 (102A and 102B).
- the remote controller 102 is provided as a pair with one HMD 100.
- the remote controller may be a foot switch on which the wearer performs a stepping input operation with a foot.
- Input information from the remote controller 102 is output to the processor unit 200.
- the processor unit 200 is a control device that controls connected devices.
- the processor unit 200 controls the HMD 100 (100A, 100B), the display 300, and the external device 400 (400A, 400B) as shown in FIG.
- the processor unit 200 processes information input from the external device 400 into information that can be displayed on the display device of the HMD 100 or the display 300, and outputs the information to each display device.
- the processor unit 200 switches information displayed on the display units 152 and 154 of the HMD 100 based on an operation input from the remote controller 102 of each HMD 100.
- the display 300 is a display device for an unspecified user to view information.
- the display 300 is mainly used by a non-HMD 100 wearer who works with the HMD 100 wearer to view information.
- Input information from the external device 400 and other information can be displayed on the display 300.
- Information displayed on the display 300 is set by the wearer, the non-wearer, or the processor unit 200.
- External device 400 is a device that outputs information to be displayed on a display device such as HMD 100 or display 300.
- the external device A 400A is an endoscope apparatus, and an image captured by the camera of the endoscope apparatus is output to the processor unit 200.
- information input from the external device 400 is processed by the processor unit 200 and displayed on a display device such as the HMD 100 or the display 300.
- the HMD 100 when a plurality of pieces of information displayed on the display units 152 and 154 can be switched, since the wearer's eyes are covered with the main body unit 110, what the wearer of the HMD 100 sees It is impossible for anyone other than the person to recognize.
- the HMD 100 according to the present embodiment is provided with a notification unit 170 that represents information visually recognized by the wearer of the HMD 100 on the outer surface of the main body unit 110.
- the notification unit 170 only needs to be provided at a position where a person other than the person wearing the HMD 100 can visually recognize.
- forms shown in FIGS. 4 to 7 are conceivable.
- the lamp 4 includes a single lamp provided near the center of the outer surface of the main body 110.
- the lamp may be, for example, an LED lamp.
- the lighting state of the lamp is controlled in accordance with information that the wearer of the HMD 100 is viewing. Examples of the lighting state include continuous lighting, extinguishing, and blinking. For example, the lamp is turned on when the wearer is viewing information on the external device A 400A, blinks when viewing information on the external device B 400B, and is turned off when viewing the surrounding environment by the imaging unit 112.
- the lamp is controlled. Or you may make it change the light quantity of a lamp
- the 5 includes three lamps 171, 172, and 173 provided in the vicinity of the center of the outer surface of the main body 110.
- the lamps 171, 172, 173 may also be LED lamps, for example. When these are the same color, for example, information that can be displayed on the display units 152 and 154 (or an input terminal of the processor unit 200) is associated with a lamp that is lit, and according to information that the wearer of the HMD 100 is viewing. The corresponding lamp may be turned on.
- the number of lamps may be smaller than the number of information that can be displayed.
- the difference in information can be expressed by changing the lighting state of each lamp.
- the difference in information can be expressed by changing the number of lamps lit. For example, one lamp 171 may be lit when the wearer is viewing information on the external device A 400A, and two lamps 171 and 172 may be lit when viewing information on the external device B 400B.
- the notification unit 170C shown in FIG. 6 includes three lamps 174, 175 and 176 of different colors provided near the center of the outer surface of the main body 110.
- the lamps 174, 175, 176 may also be LED lamps, for example. As described above, by using the lamps 174, 175, and 176 of different colors, it is possible to notify information to a non-wearer more easily. Note that the notification unit 170C shown in FIG. 6 does not have to have a one-to-one correspondence between information and lamps as in the notification unit 170B of FIG. 5, and the difference in information by changing the lighting state and the number of lightings. Can also be represented.
- a display device that displays information visually recognized by the wearer of the HMD 100.
- a display device for example, a light-weight display device such as a liquid crystal display, an organic EL display, or electronic paper, a display device that can be curved so as to be installed on a curved surface, or the like may be used.
- the information displayed on the display device may be the information that is actually viewed by the wearer of the HMD 100, or may be correspondence information representing the information. Examples of correspondence information include thumbnails, text information, charts, and the like. Thereby, a non-wearing person can recognize easily the information which the wearer of HMD100 has visually recognized by looking at notice part 170D.
- the form of the notification unit 170 of the HMD 100 according to the present embodiment is not limited to the above example.
- the above-described forms of the notification units 170A to 170D can be appropriately combined, for example, by providing a combination of a lamp and a display device.
- the wearer of the HMD 100 performs an operation while switching between a medical image such as an endoscope apparatus presented from the processor unit 200 and a video imaged by the imaging unit 112 of the HMD 100. At this time, information on the external device 400 and the like is displayed on the display units 152 and 154 on the HMD 100, and information that the wearer of the HMD 100 is viewing is notified by the notification unit 170 of the main body unit 110.
- a medical image such as an endoscope apparatus presented from the processor unit 200 and a video imaged by the imaging unit 112 of the HMD 100.
- FIG. 8 shows a functional configuration of the HMD 100 and the processor unit 200 constituting the endoscope system 1 according to the present embodiment.
- FIG. 8 shows a functional unit that functions when performing display control of the display units 152 and 154 of the HMD 100 and a notification control function of the HMD 100, and actually includes other functional units.
- the processor unit 200 functions as a display control device that performs display control of the two HMDs 100 ⁇ / b> A and 100 ⁇ / b> B, and switches information to be presented to each based on the display switching instruction of each HMD 100 ⁇ / b> A and 100 ⁇ / b> B.
- the HMD 100 includes a display port 162, an image generation unit 164, display elements 165 and 166, and a notification control unit 168.
- the display port 162 is an interface that receives input information from the processor unit 200.
- a cable 140 that enables information communication with the processor unit 200 is connected to the display port 162.
- image signals to be output to the display elements 165 and 166 and information visually recognized by the wearer of the HMD 100 are input to the display port 162, for example.
- Information input from the display port 162 is output to the image generation unit 164.
- the image generation unit 164 generates image signals to be output to the display elements 165 and 166 based on the information acquired via the processor unit 200.
- the image generation unit 164 outputs the left-eye image signal output to the first display element 165 and the right-eye image output to the second display element 166.
- a shift process that causes a shift in the image signal is performed.
- the shift amount between the left-eye image signal and the right-eye image signal is determined according to, for example, the distance between the display elements 165 and 166 and the wearer's eyes, the interval between the wearer's eyes, the virtual image position, and the like. Is done.
- the image generation unit 164 outputs the generated image signal to the first display element 165 and the second display element 166.
- the display elements 165 and 166 emit the image light toward the display units 152 and 154 based on the image signal input from the image generation unit 164.
- the display elements 165 and 166 are disposed so as to face the display units 152 and 154 in the front-rear direction of the wearer's face when the HMD 100 is worn, for example. Thereby, the optical axis of the image light emitted from the display elements 165 and 166 and the display units 152 and 154 is substantially parallel to the direction of the line of sight when the wearer faces the front.
- the display elements 165 and 166 are composed of, for example, organic EL (Electroluminescence) elements.
- organic EL elements Electrode Electrode elements
- the display elements 165 and 166 have a configuration in which, for example, a plurality of red organic EL elements, green organic EL elements, blue organic EL elements, and the like are arranged in a matrix. Each of these elements is driven by an active matrix type or passive matrix type drive circuit to emit light at a predetermined timing, brightness, or the like.
- a predetermined image is displayed as a whole on the display elements 165 and 166, and is displayed to the wearer via the display units 152 and 154. Provided.
- a plurality of eyepieces may be disposed between the display elements 165 and 166 and the display units 152 and 154 as an optical system.
- these eyepieces and the wearer's eyes face each other at a predetermined distance, it becomes possible for the wearer to observe a virtual image that appears to display an image at a predetermined position (virtual image position).
- a 3D image can be provided.
- the virtual image position and the size of the virtual image are set according to the configuration of the display elements 165 and 166 and the optical system.
- the main body 110 is worn by the user so that image light emitted from the display elements 165 and 166 in the optical axis direction forms images on the retinas of the left and right eyes, respectively.
- the main body 110 is not properly worn, the image becomes out of focus or the 3D image is blurred, and the wearer cannot see the desired image. Therefore, when the HMD 100 is mounted, the user adjusts the main body 110 to an appropriate position, and then fixes the main body 110 to the head by the upper fixing portion 120 and the rear fixing portion 130 so that the position does not shift.
- the notification control unit 168 controls the notification unit 170 based on the input information from the processor unit 200 input to the display port 162. For example, the notification control unit 168 controls the notification unit 170 so as to represent newly displayed information at the timing when the display information on the display units 152 and 154 is switched. For example, when the notification unit 170 is a lamp, the notification control unit 168 controls the lighting state, the number of lights, the amount of light, and the like of the lamp. For example, when the notification unit 170 is a display device, the notification control unit 168 displays information displayed on the display device or corresponding information such as thumbnails, text information, and charts corresponding to the information.
- the notification control unit 168 may notify the notification unit 170 that the display of the HMD 100 has been switched. For example, when the notification unit 170 is a lamp, the notification control unit 168 turns on the lamp with a lighting pattern that notifies the switching of the display, and then shifts to a lighting state corresponding to the information displayed after the switching. You may control. Further, for example, when the notification unit 170 is a display device, the notification control unit 168 temporarily blinks a thumbnail displayed on the display device or displays an alarm notifying the switching of the display. Also good.
- the processor unit 200 includes an image input unit 211, an image processing unit 212, a display control unit 213, an output unit 214, and an operation input. Part 215.
- the image input unit 211 is an interface that receives an image input from the external device 400 to the processor unit 200.
- the endoscope apparatus 10 is shown as the external device 400, and an image captured by a camera (not shown) of the endoscope apparatus 10 is input to the image input unit 211 at this time. Is done.
- the image input unit 211 outputs the input image to the image processing unit 212.
- the image processing unit 212 processes the image input to the processor unit 200 into an image to be displayed on the HMD 100.
- the image processing unit 212 for example, a left-eye image displayed on the first display unit 152 of the HMD 100 and a right-eye image displayed on the second display unit 154 from an image captured by the camera of the endoscope apparatus 10. Generate an image.
- the image processed by the image processing unit 212 is output to the display control unit 213.
- the display control unit 213 controls information displayed on the display units 152 and 154 of the HMD 100.
- the display control unit 213 controls to display the instructed information based on the display switching instruction from the remote controller 102.
- the display control unit 213 determines information to be displayed on each HMD 100, the display control unit 213 outputs the information to each HMD 100 via the output unit 214.
- the operation input unit 215 is an input unit that receives an operation input from the wearer of the HMD 100.
- information displayed on the display units 152 and 154 of the HMD 120 can be switched by the remote controller 102.
- An operation input to the remote controller 102 is output to the operation input unit 215, and the operation input unit 215 outputs this operation input information to the display control unit 213.
- the display control unit 213 outputs the instructed information to the HMD 100 via the output unit 214 based on the display switching instruction from the remote controller 102.
- the notification unit 170 is controlled in accordance with the information visually recognized by the wearer of the HMD 100, and the information visually recognized by the wearer of the HMD 100 is notified to the non-wearer.
- the non-wearer can grasp what the wearer of the HMD 100 working together is currently working on. Since conversations and instructions are performed thereafter, communication between users can be facilitated, and the occurrence of linkage loss and the like can be avoided.
- FIG. 9 is a functional block diagram showing a functional configuration of the HMD 100 and the processor unit 200 configuring the endoscope system 1 according to the present embodiment.
- FIG. 9 shows a functional unit that functions when performing display control of the display units 152 and 154 of the HMD 100 and a notification control function of the HMD 100, and actually includes other functional units. Shall.
- the processor unit 200 functions as a display control device that performs display control of the two HMDs 100 ⁇ / b> A and 100 ⁇ / b> B similarly to FIG. 8 of the first embodiment, and based on the display switching instruction of each of the HMDs 100 ⁇ / b> A and 100 ⁇ / b> B. , Switch the information presented to each.
- the endoscope system 1 according to the present embodiment is different from the configuration of the first embodiment illustrated in FIG. 8 in that the HMD 100 includes a line-of-sight detection unit 167. Since the other functional units are the same as those in the first embodiment, description of these functional units will be omitted.
- the line-of-sight detection unit 167 provided in the HMD 100 detects the line of sight of the wearer of the HMD 100.
- the line-of-sight detection unit 167 images the eyes of the wearer of the HMD 100 and detects the line of sight from the positional relationship between the reference point and the moving point in the eye.
- the line of sight of the wearer detected by the line-of-sight detection unit 167 is output to the notification control unit 168.
- the notification control unit 168 determines whether the wearer is viewing the display units 152 and 154 or other parts such as the hand. Then, the notification control unit 168 controls the notification unit 170 based on the determination result to notify the non-wearer of information that the wearer is viewing.
- the line-of-sight detection unit 167 by detecting the wearer's line of sight by the line-of-sight detection unit 167, for example, it is possible to grasp whether or not the wearer is viewing the display units 152 and 154 of the HMD 100.
- the wearer of the HMD 100 is displayed on the display units 152 and 154 when there is a gap between the main body 110 and the eyes of the wearer and the lower part is opened. Sometimes you can see things other than information. At this time, if the information displayed on the display units 152 and 154 is notified to the non-wearing person, recognition between users may be different, and communication may not be successfully performed.
- the information visually recognized by the wearer is not necessarily the information displayed on the display units 152 and 154 of the HMD 100.
- the display units 152 and 154 include a plurality of display areas. If there is, it may be information on one of the display areas. Therefore, the line-of-sight detection unit 167 is provided in the HMD 100 as in the present embodiment, and the information that the wearer is viewing from the line of sight of the wearer of the HMD 100 is specified. As a result, information that the wearer sees can be transmitted to the non-wearer more accurately, and communication between users can be improved.
- the notification unit 170 is controlled in accordance with the information visually recognized by the wearer of the HMD 100, and the information visually recognized by the wearer of the HMD 100 is notified to the non-wearer.
- the non-wearer can grasp what the wearer of the HMD 100 working together is currently working on. Since conversations and instructions are performed thereafter, communication between users can be facilitated, and the occurrence of linkage loss and the like can be avoided.
- the information that the wearer of the HMD 100 is viewing is notified by the notification unit 170 provided in the HMD 100, but the present disclosure is not limited to such an example.
- the notification unit 170 that notifies information that each wearer visually recognizes may be provided in the processor unit 200.
- the processor unit 200 is provided with a notification unit such as a lamp or a display device, and represents information that the wearer of the HMD 100 connected by the notification unit is viewing. Thereby, when the processor unit 200 is seen, it becomes possible to grasp
- a notification unit such as a lamp or a display device
- the notification control unit 168 and the line-of-sight detection unit 167 that control the notification unit 170 of the HMD 100 are provided in each HMD 100, but the present technology is not limited to such an example.
- the processor unit 200 may be provided with a function of a notification control unit or a line-of-sight detection unit.
- an organic EL element is used as a display element, but the present technology is not limited to this example.
- a liquid crystal display element (LCD) or the like may be used as the display element.
- the present technology is not limited to such an example.
- the wearer can visually recognize the notification unit 170 of the other wearer's HMD 100.
- the wearer can see the other wearer's notification unit 170 through the imaging unit 112 and can recognize information that the other wearer is viewing.
- communication between the wearer of the HMD 100 and a person other than the wearer can be enhanced.
- the notification unit includes one or a plurality of lamps.
- the notification unit notifies information that the wearer is viewing based on at least one of the color, lighting state, lighting number, or light emission amount of the one or more lamps.
- the notification unit is a display device that displays information that the wearer is viewing.
- the head mounted display according to (1), wherein the notification unit is a display device that displays correspondence information representing information that the wearer is viewing. (6) The head mounted display according to (5), wherein the display device is bendable. (7) The head mounted display according to any one of (1) to (6), wherein the information visually recognized by the wearer includes at least an image captured by an endoscope apparatus. (8) The head-mounted display according to any one of (1) to (7), further including a notification control unit that controls the notification unit based on information visually recognized by the wearer. (9) The notification unit includes the one or more lamps, The notification control unit controls at least one of a color, a lighting state, a lighting number, and a light emission amount of the one or more lamps based on information visually recognized by the wearer.
- a gaze detection unit that detects the gaze of the wearer is provided, The head mounted display according to (8) or (9), wherein the notification control unit determines information visually recognized by a wearer based on a detection result of the line-of-sight detection unit.
- a visual information acquisition unit that acquires information that the wearer of the head-mounted display visually recognizes;
- a notification control unit configured to control a notification unit of the head-mounted display that notifies the information other than the wearer of the information viewed by the wearer based on the information viewed by the wearer; ,Control device.
Landscapes
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Optics & Photonics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Surgery (AREA)
- Veterinary Medicine (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Biophysics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Pathology (AREA)
- Radiology & Medical Imaging (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Computer Graphics (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Controls And Circuits For Display Device (AREA)
- Endoscopes (AREA)
- Control Of Indicators Other Than Cathode Ray Tubes (AREA)
- Devices For Indicating Variable Information By Combining Individual Elements (AREA)
- User Interface Of Digital Computer (AREA)
- Instruments For Viewing The Inside Of Hollow Bodies (AREA)
Abstract
Description
1.第1の実施形態
1.1.システム構成
1.2.通知部
1.3.制御処理
1.4.まとめ
2.第2の実施形態(視線検出部を備える例)
3.変形例
[1.1.システム構成]
まず、図1~図3を参照して、本開示の第1の実施形態に係るHMDを使用したシステムの一例として、内視鏡システムについて説明する。なお、図1は、本実施形態に係る内視鏡システム1の一構成例を示すシステム構成図である。図2は、本実施形態に係るHMD100をユーザが装着している状態を示す概略側面図である。図3は、HMD100の表示面152、154を示す概略図である。
「装着者」:HMDを装着している人
「非装着者」:HMDを装着していない人
また、「装着者」が複数存在するとき、1人の「装着者」に対し、それ以外のHMDの装着者を「他の装着者」ともいう。
本実施形態に係るHMD100において、表示部152、154に表示される情報を複数切り替え可能な場合、装着者の眼が本体部110で覆われているために、HMD100の装着者が何を見ているのか、本人以外は認識することができない。そこで、本実施形態に係るHMD100には、本体部110の外表面に、HMD100の装着者が視認している情報を表す通知部170が設けられている。通知部170は、HMD100を装着している本人以外が視認できる位置に設けられていればよい。通知部170の構成として、例えば図4~図7に示すような形態が考えられる。
HMD100の装着者は、プロセッサユニット200から提示される内視鏡装置等の医療映像や、HMD100の撮像部112により撮影された映像を切り替えながら、手術を行う。このとき、HMD100には、表示部152、154に外部機器400等の情報が表示されるとともに、本体部110の通知部170により、HMD100の装着者が視認している情報が通知される。以下、図8に基づき、本実施形態にかかる内視鏡システム1におけるHMD100への情報表示制御および通知制御について説明する。
以上、本実施形態に係る内視鏡システム1において、HMD100に表示する情報の提示方法について説明した。本実施形態によれば、HMD100の装着者が視認している情報に応じて通知部170を制御し、HMD100の装着者が視認している情報を非装着者に通知する。これにより、非装着者は、共同して作業を行っているHMD100の装着者が現在何を見て作業を行っているのか把握することが可能となる。その上で会話や指示を行うので、ユーザ間のコミュニケーションを容易に図ることが可能となり、連携ロスの発生等を回避することができる。
次に、図9に基づいて、本開示の第2の実施形態に係るHMDを使用したシステムにおける通知制御について説明する。本実施形態では、第1の実施形態と同様、内視鏡システムを例として説明する。なお、図9は、本実施形態に係る内視鏡システム1を構成するHMD100およびプロセッサユニット200の機能構成を示す機能ブロック図である。なお、図9には、HMD100の表示部152、154の表示制御を行う際に機能する機能部と、HMD100の通知制御機能とを示しており、実際には、他の機能部を備えているものとする。
上記実施形態では、HMD100の装着者が視認している情報をHMD100に設けられた通知部170により通知したが、本開示はかかる例に限定されない。例えば、各装着者が視認している情報を通知する通知部170は、プロセッサユニット200に設けてもよい。
(1)装着者に画像を提示するための表示部と、
前記装着者の頭部へ装着される装着部と、
前記装着者が視認している情報を前記装着者以外の者に通知する通知部を備える、頭部装着ディスプレイ。
(2)前記通知部は、1または複数のランプからなる、前記(1)に記載の頭部装着ディスプレイ。
(3)前記通知部は、前記1または複数のランプの色、点灯状態、点灯数、または発光量のうち少なくともいずれか1つに基づいて、前記装着者が視認している情報を通知する、前記(2)に記載の頭部装着ディスプレイ。
(4)前記通知部は、前記装着者が視認している情報を表示する表示デバイスである、前記(1)に記載の頭部装着ディスプレイ。
(5)前記通知部は、前記装着者が視認している情報を表す対応情報を表示する表示デバイスである、前記(1)に記載の頭部装着ディスプレイ。
(6)前記表示デバイスは湾曲可能である、前記(5)に記載の頭部装着ディスプレイ。
(7)前記装着者が視認している情報は、内視鏡装置により撮像された画像を少なくとも含む、前記(1)~(6)のいずれか1項に記載の頭部装着ディスプレイ。
(8)前記装着者が視認している情報に基づいて、前記通知部を制御する通知制御部を備える、前記(1)~(7)のいずれか1項に記載の頭部装着ディスプレイ。
(9)前記通知部は、前記1または複数のランプからなり、
前記通知制御部は前記装着者が視認している情報に基づいて、前記1または複数のランプの色、点灯状態、点灯数、または発光量のうち少なくともいずれか1つを制御する、前記(8)に記載の頭部装着ディスプレイ。
(10)装着者の視線を検出する視線検出部を備え、
前記通知制御部は、前記視線検出部の検出結果に基づいて、装着者が視認している情報を判定する、前記(8)または(9)に記載の頭部装着ディスプレイ。
(11)頭部装着ディスプレイの装着者が視認している情報を取得する視認情報取得部と、
前記装着者が視認している情報に基づいて、前記頭部装着ディスプレイが有する、前記装着者以外の者に前記装着者が視認している情報を通知する通知部を制御する通知制御部を備える、制御装置。
(12)頭部装着ディスプレイの装着者が視認している情報を取得し、
前記装着者が視認している情報に基づいて、前記頭部装着ディスプレイが有する、前記装着者以外の者に装着者が視認している情報を通知する通知部を制御する、制御方法。
100 HMD
102 リモートコントローラ
110 本体部
152 第1の表示部
154 第2の表示部
162 ディスプレイポート
164 画像生成部
165 第1の表示素子
166 第2の表示素子
167 視線検出部
168 通知制御部
170 通知部
200 プロセッサユニット
211 画像入力部
212 画像処理部
213 表示制御部
214 出力部
215 操作入力部
300 ディスプレイ
400 外部機器
Claims (12)
- 装着者に画像を提示するための表示部と、
前記装着者の頭部へ装着される装着部と、
前記装着者が視認している情報を前記装着者以外の者に通知する通知部を備える、頭部装着ディスプレイ。 - 前記通知部は、1または複数のランプからなる、請求項1に記載の頭部装着ディスプレイ。
- 前記通知部は、前記1または複数のランプの色、点灯状態、点灯数、または発光量のうち少なくともいずれか1つに基づいて、前記装着者が視認している情報を通知する、請求項2に記載の頭部装着ディスプレイ。
- 前記通知部は、前記装着者が視認している情報を表示する表示デバイスである、請求項1に記載の頭部装着ディスプレイ。
- 前記通知部は、前記装着者が視認している情報を表す対応情報を表示する表示デバイスである、請求項1に記載の頭部装着ディスプレイ。
- 前記表示デバイスは湾曲可能である、請求項5に記載の頭部装着ディスプレイ。
- 前記装着者が視認している情報は、内視鏡装置により撮像された画像を少なくとも含む、請求項1に記載の頭部装着ディスプレイ。
- 前記装着者が視認している情報に基づいて、前記通知部を制御する通知制御部を備える、請求項1に記載の頭部装着ディスプレイ。
- 前記通知部は、前記1または複数のランプからなり、
前記通知制御部は前記装着者が視認している情報に基づいて、前記1または複数のランプの色、点灯状態、点灯数、または発光量のうち少なくともいずれか1つを制御する、請求項8に記載の頭部装着ディスプレイ。 - 装着者の視線を検出する視線検出部を備え、
前記通知制御部は、前記視線検出部の検出結果に基づいて、装着者が視認している情報を判定する、請求項8に記載の頭部装着ディスプレイ。 - 頭部装着ディスプレイの装着者が視認している情報を取得する視認情報取得部と、
前記装着者が視認している情報に基づいて、前記頭部装着ディスプレイが有する、前記装着者以外の者に前記装着者が視認している情報を通知する通知部を制御する通知制御部を備える、制御装置。 - 頭部装着ディスプレイの装着者が視認している情報を取得し、
前記装着者が視認している情報に基づいて、前記頭部装着ディスプレイが有する、前記装着者以外の者に装着者が視認している情報を通知する通知部を制御する、制御方法。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/118,367 US10139624B2 (en) | 2014-02-21 | 2015-01-08 | Head mounted display, control device, and control method |
EP15752719.3A EP3108801A4 (en) | 2014-02-21 | 2015-01-08 | Head-mounted display, control device, and control method |
JP2016503991A JP6589855B2 (ja) | 2014-02-21 | 2015-01-08 | 頭部装着ディスプレイ、制御装置および制御方法 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2014-032009 | 2014-02-21 | ||
JP2014032009 | 2014-02-21 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015125508A1 true WO2015125508A1 (ja) | 2015-08-27 |
Family
ID=53878024
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2015/050318 WO2015125508A1 (ja) | 2014-02-21 | 2015-01-08 | 頭部装着ディスプレイ、制御装置および制御方法 |
Country Status (4)
Country | Link |
---|---|
US (1) | US10139624B2 (ja) |
EP (1) | EP3108801A4 (ja) |
JP (1) | JP6589855B2 (ja) |
WO (1) | WO2015125508A1 (ja) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2019102062A (ja) * | 2017-12-07 | 2019-06-24 | パナソニックIpマネジメント株式会社 | ヘッドマウントディスプレイ及びその制御方法 |
WO2019208005A1 (ja) | 2018-04-23 | 2019-10-31 | ソニー株式会社 | 接眼光学系、医療用ビューア及び医療用ビューアシステム |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111295127B (zh) * | 2017-10-31 | 2022-10-25 | 富士胶片株式会社 | 检查支持装置、内窥镜装置及记录介质 |
US20230221566A1 (en) * | 2022-01-08 | 2023-07-13 | Sony Interactive Entertainment Inc. | Vr headset with integrated thermal/motion sensors |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH10201700A (ja) * | 1997-01-20 | 1998-08-04 | Olympus Optical Co Ltd | 蛍光観察内視鏡装置 |
JP2001104331A (ja) * | 1999-10-06 | 2001-04-17 | Olympus Optical Co Ltd | 医療用顔面装着型映像表示装置 |
JP2001290103A (ja) * | 2001-02-13 | 2001-10-19 | Olympus Optical Co Ltd | 観察システム |
JP2009229859A (ja) * | 2008-03-24 | 2009-10-08 | Nikon Corp | ヘッドマウントディスプレイ装置 |
Family Cites Families (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3262849B2 (ja) | 1992-08-14 | 2002-03-04 | オリンパス光学工業株式会社 | 立体像観察システムおよび内視鏡画像観察システム |
US5949388A (en) * | 1995-03-03 | 1999-09-07 | Olympus Optical Co., Ltd. | Head-mounted video display |
US6847336B1 (en) * | 1996-10-02 | 2005-01-25 | Jerome H. Lemelson | Selectively controllable heads-up display system |
JPH1184308A (ja) * | 1997-09-02 | 1999-03-26 | Minolta Co Ltd | 頭部装着装置及び映像観察装置 |
JP2002108513A (ja) * | 2000-07-27 | 2002-04-12 | Olympus Optical Co Ltd | 表示システムおよびディスプレイ装置 |
US7474276B2 (en) * | 2000-06-20 | 2009-01-06 | Olympus Optical Co., Ltd. | Display system and microdisplay apparatus |
ATE447205T1 (de) * | 2003-05-12 | 2009-11-15 | Elbit Systems Ltd | Verfahren und system zur audiovisuellen kommunikation |
EP2046026A4 (en) * | 2006-07-25 | 2010-05-05 | Nikon Corp | OUTPUT APPARATUS AND IMAGE DISPLAY APPARATUS |
WO2009005901A2 (en) * | 2007-05-18 | 2009-01-08 | The Uab Research Foundation | Virtual interactive presence systems and methods |
JP5481890B2 (ja) * | 2009-03-12 | 2014-04-23 | ブラザー工業株式会社 | ヘッドマウントディスプレイ装置、画像制御方法および画像制御プログラム |
WO2012062872A1 (en) * | 2010-11-11 | 2012-05-18 | Bae Systems Plc | Image presentation method, and apparatus therefor |
JP5730177B2 (ja) | 2011-11-21 | 2015-06-03 | 独立行政法人国立がん研究センター | 電子内視鏡システム |
US20130241805A1 (en) | 2012-03-15 | 2013-09-19 | Google Inc. | Using Convergence Angle to Select Among Different UI Elements |
JP2016032485A (ja) | 2012-12-27 | 2016-03-10 | 国立大学法人 東京医科歯科大学 | 内視鏡下手術支援システム及び画像制御方法 |
FR3021134B1 (fr) * | 2014-05-14 | 2023-01-06 | Lg Electronics Inc | Terminal mobile |
-
2015
- 2015-01-08 JP JP2016503991A patent/JP6589855B2/ja not_active Expired - Fee Related
- 2015-01-08 US US15/118,367 patent/US10139624B2/en active Active
- 2015-01-08 EP EP15752719.3A patent/EP3108801A4/en not_active Withdrawn
- 2015-01-08 WO PCT/JP2015/050318 patent/WO2015125508A1/ja active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH10201700A (ja) * | 1997-01-20 | 1998-08-04 | Olympus Optical Co Ltd | 蛍光観察内視鏡装置 |
JP2001104331A (ja) * | 1999-10-06 | 2001-04-17 | Olympus Optical Co Ltd | 医療用顔面装着型映像表示装置 |
JP2001290103A (ja) * | 2001-02-13 | 2001-10-19 | Olympus Optical Co Ltd | 観察システム |
JP2009229859A (ja) * | 2008-03-24 | 2009-10-08 | Nikon Corp | ヘッドマウントディスプレイ装置 |
Non-Patent Citations (1)
Title |
---|
See also references of EP3108801A4 * |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2019102062A (ja) * | 2017-12-07 | 2019-06-24 | パナソニックIpマネジメント株式会社 | ヘッドマウントディスプレイ及びその制御方法 |
JP7065353B2 (ja) | 2017-12-07 | 2022-05-12 | パナソニックIpマネジメント株式会社 | ヘッドマウントディスプレイ及びその制御方法 |
WO2019208005A1 (ja) | 2018-04-23 | 2019-10-31 | ソニー株式会社 | 接眼光学系、医療用ビューア及び医療用ビューアシステム |
US11733517B2 (en) | 2018-04-23 | 2023-08-22 | Sony Corporation | Ocular optical system, medical viewer, and medical viewer system |
Also Published As
Publication number | Publication date |
---|---|
US20170168297A1 (en) | 2017-06-15 |
JPWO2015125508A1 (ja) | 2017-03-30 |
US10139624B2 (en) | 2018-11-27 |
EP3108801A1 (en) | 2016-12-28 |
JP6589855B2 (ja) | 2019-10-16 |
EP3108801A4 (en) | 2017-10-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6264087B2 (ja) | 表示制御装置、表示装置および表示制御システム | |
US10725535B2 (en) | Head-worn image display apparatus for stereoscopic microsurgery | |
JP6574939B2 (ja) | 表示制御装置、表示制御方法、表示制御システムおよび頭部装着ディスプレイ | |
JP6693507B2 (ja) | 情報処理装置、情報処理方法及び情報処理システム | |
US9507155B2 (en) | Head-mounted display | |
US9898662B2 (en) | Information processing apparatus, information processing method, and information processing system | |
EP3204816B1 (en) | Head-mounted displaying of magnified images locked on an object of interest | |
US9581822B2 (en) | Head-mounted display | |
US10302944B2 (en) | Head-mount type display device and method of controlling head-mount type display device | |
JP2004537227A (ja) | イメージディスプレイ方法およびイメージディスプレイ装置 | |
JP2013195924A (ja) | ヘッドマウントディスプレイ | |
US20210335483A1 (en) | Surgery visualization theatre | |
JP6589855B2 (ja) | 頭部装着ディスプレイ、制御装置および制御方法 | |
JP2014130204A (ja) | 表示装置、表示システム、および、表示装置の制御方法 | |
US11081015B2 (en) | Training device, training method, and program | |
JP6617766B2 (ja) | 医療用観察システム、表示制御システムおよび表示制御装置 | |
Cuschieri | Visual display technology for endoscopic surgery | |
US20170242482A1 (en) | Training device, corresponding area specifying method, and program | |
JP2009086006A (ja) | 頭部装着型表示装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15752719 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2016503991 Country of ref document: JP Kind code of ref document: A |
|
REEP | Request for entry into the european phase |
Ref document number: 2015752719 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2015752719 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15118367 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |