WO2011070845A1

WO2011070845A1 - Head-mounted display and head-mounted display system

Info

Publication number: WO2011070845A1
Application number: PCT/JP2010/067461
Authority: WO
Inventors: 悟菊池; 裕昌小林; 浩池田
Original assignee: オリンパス株式会社
Priority date: 2009-12-11
Filing date: 2010-10-05
Publication date: 2011-06-16
Also published as: JP2011120790A

Abstract

Provided is a head-mounted display capable of switching an image unnecessary to a wearer. Specifically provided is a head-mounted display provided with: an image input unit (4) into which an image is inputted; an image display unit (3) which is mounted onto the head of a wearer and supported so as to be disposed in front of the eyes of the wearer, and displays the input image inputted into the image input unit (4); an image determination unit (9) which determines whether or not the input image is an image of an observation region in the body of the wearer; and a control unit (10) which causes the image display unit (3) to display the input image when the image determination unit (9) determines that the input image is the image of the observation region in the body of the wearer.

Description

Head-mounted display device and head-mounted display system

The present invention relates to a head-mounted display device and a head-mounted display system.

Conventionally, a spectacle-type or goggles-type head mounted display (HMD) may be used in endoscopic surgery (see, for example, Patent Document 1). The HMD includes a frame that is worn on the head and a display element that receives an image from the endoscope. A wearer wearing the HMD can observe an endoscopic image inside the body without changing the posture while operating the instrument or the like at the operation site by moving the line of sight to the display element arranged in front of the eyes. .

JP-A-9-98985

However, since blood or the like adheres to the endoscope and the screen becomes dirty, it is necessary to frequently remove and clean the endoscope during surgery. Also, when an endoscope with different specifications such as an angle of view is required, it is necessary to remove the endoscope and insert it again. At this time, the display element arranged in front of the wearer's eyes also displays an image observed by the endoscope while the endoscope is being inserted / removed inside / outside the body or washed. As described above, there is a problem that an image unnecessary for the wearer is displayed during the operation.

The present invention has been made in view of the above-described circumstances, and an object thereof is to provide a head-mounted display device and a head-mounted display system that can switch an image unnecessary for the wearer.

In order to achieve the above object, the present invention provides the following means.
According to a first aspect of the present invention, a video input unit to which video is input and a head mounted on the wearer's head and supported so as to be placed in front of the wearer's eyes are input to the video input unit. A video display unit that displays the input video, a video determination unit that determines whether or not the input video is a video of an observation region of a subject, and the video determination unit allows the input video to be observed in the body. The head-mounted display device includes a control unit that displays the input video on the video display unit when it is determined that the video is a video.

According to the first aspect of the present invention, when the wearer wears the wearing frame, the wearer inputs the video imaged by the imaging device or the like to the video input unit, so that the wearer can use the imaging device without changing his / her posture. You can see the shot video in front of you.
In this case, the control unit causes the video display unit to display the input image when the video determination unit determines that the input image input from the imaging device or the like is a video of the observation site of the subject. That is, an unnecessary input image can be switched while displaying an input image necessary for the wearer.

In the first aspect, the first video storage unit that stores the input video, the second video storage unit that stores a predetermined reference video, and the input video stored in the first video storage unit A similarity determination unit that determines whether or not the reference image stored in the second image storage unit is similar to the reference image, and the image determination unit uses the similarity determination unit to convert the input image from the reference image. When it is determined that they are similar, it is determined whether the input image is an image of a non-observation part other than the observation part or an image of the observation part, and the control unit When the video determination unit determines that the input video is an image of the non-observation site, the input video is not displayed on the video display unit, and the video determination unit displays the input video as the video of the observation site. When it is determined that Video may be configured to be displayed on the video display unit.

With this configuration, the input image is moved between the observation region and the non-observation region by using the image captured in the process of moving the field of view of the imaging device from the observation region to the non-observation region as a reference image. This can be easily determined.

In the above configuration, the reference image may be an image obtained by photographing an opening of a cylindrical trocar from which an endoscope can be inserted and removed.
In this way, when the endoscope is inserted into or removed from the inside or outside of the body through the trocar previously inserted into the body, the endoscope tip is opened at the opening of the trocar when the endoscope is inserted into the body. It is determined that the input video has become an observation site before exiting from the camera, while when the endoscope is removed from the body, it is determined that the endoscope has become a non-observation site after being stored in the trocar. . Thereby, it is possible to quickly and reliably determine the movement between the observation site and the non-observation site of the input video.

In the first aspect, a video storage unit that stores the input video, a feature extraction unit that extracts a predetermined feature portion from the input video stored in the video storage unit, and extraction by the feature extraction unit A motion detection unit that detects a motion of the feature portion that has been detected, and the video determination unit detects when the motion of the feature portion detected by the motion detection unit continues in a substantially same direction for a predetermined time. Determining whether the input image is an image of a non-observation region other than the observation region or an image of the observation region, and the control unit determines whether the input image is the non-observation by the image determination unit. When it is determined that the image is a part image, the input image is not displayed on the image display unit, and when the input image is determined by the image determination unit to be an image of the observation region, the input image is displayed. Video table It may be configured to display the part.

By configuring in this way, when the imaging device is operated in one direction to approach and separate the observation site, by detecting the movement of the imaging device or the like at that time from the movement of the characteristic part in the input video, It is possible to determine whether the input image is an image of an observation site or an image of a non-observation site.

In the above-described configuration, the characteristic portion may be a contour when an opening of a cylindrical trocar into which an endoscope can be inserted and removed is photographed from the inside.
By doing in this way, when the endoscope is inserted / removed inside / outside the body through the trocar inserted in the body in advance, the opening of the trocar is reflected in the input image in the process. Therefore, it is possible to detect an operation in which the endoscope is inserted / removed inside / outside the body from the movement of the contour of the opening.

In the first aspect, a video storage unit that stores the input video, a luminance distribution detection unit that detects a luminance distribution of a predetermined line of the input video stored in the video storage unit, and the luminance distribution detection unit It is determined whether or not the luminance distribution detected by the method is bipolarized into a low luminance region having a luminance value lower than a predetermined threshold value and a high luminance region having a luminance value higher than the predetermined threshold value. And a video processing unit that detects a line width of the low luminance area, and the video determination unit increases the line width of the low luminance region detected by the video processing unit over time. When it is determined that the input image is an image of a non-observation region other than the observation region, and the line width of the low luminance region detected by the image processing unit decreases with time, the input image Is the image of the observation site It may be determined to that.

In this way, when the endoscope is inserted / removed through the trocar inserted in the body, when the opening is projected from the inside of the trocar, the inner surface of the trocar having a relatively high brightness and the trocar A bipolar luminance distribution is detected by the difference in luminance from the body having a relatively low luminance viewed through the opening. When the endoscope is inserted, the line width of the low luminance area is enlarged, and when the endoscope is removed, the line width of the low luminance area is reduced. As a result, an operation of inserting / removing the endoscope into / from the body can be detected.

In the first aspect, a video switching unit that switches a video to be input to the video display unit between the input video and another arbitrary video is provided, and the control unit is configured so that the video is input by the video determination unit. When it is determined that the image is an observation site image, the video switching unit switches to the input image, and the video determination unit determines that the input image is a non-observation site other than the observation site The video switching unit may switch to the arbitrary video.
In this way, when the input video is a non-observation site, an arbitrary video can be displayed on the video display unit.

In the first aspect, the control unit turns on the display of the video display unit when the video determination unit determines that the input video is a video of the observation site, and the video determination unit The display of the video display unit may be turned off when it is determined that the input video is a non-observation site other than the observation site.
In this way, when the input video is a non-observation site, the video display unit can be turned off.
In the first aspect, a video acquisition unit may be provided that acquires a video of the subject and inputs the acquired video to the video input unit.

According to a second aspect of the present invention, an endoscope is provided with a motion detection unit that detects a motion in the insertion direction at the distal end portion, and an endoscope video imaged by the endoscope is the video input unit. A head-mounted display system comprising any one of the head-mounted display devices described above.
According to the second aspect of the present invention, when the endoscope is inserted / removed inside / outside the body, the movement is detected by the motion detector, so that the endoscope image is an image of the observation site. It is determined whether the image is an observation site image. And when it determines with it being the image | video of an observation site | part, an endoscopic image | video is displayed on an image | video display part. That is, the video can be switched so that it is not necessary to view an endoscopic video unnecessary for the wearer.

According to the present invention, there is an effect that an image unnecessary for the wearer can be switched.

1 is an overall configuration diagram of a head-mounted display device according to an embodiment of the present invention. FIG. 2 is a functional block diagram of the head-mounted display device of FIG. 1. It is a figure which shows the structure of a video display part. It is a figure which shows an example of a reference | standard image | video. It is a functional block diagram which shows the modification of the head mounted display apparatus of FIG. It is a functional block diagram which shows another modification of the head mounted display apparatus of FIG. It is a functional block diagram which shows another modification of the head mounted display apparatus of FIG. It is a figure explaining the determination method of the endoscopic image | video based on the luminance distribution of an endoscope, and is an endoscopic image | video which showed the opening of the trocar. It is a figure explaining the determination method of the endoscope image | video based on the luminance distribution of an endoscope, and is the luminance graph in the scanning line A of FIG. 8A. It is a figure explaining the determination method of the endoscopic image | video based on the luminance distribution of an endoscope, and is the endoscopic image moved from FIG. 8A. It is a figure explaining the determination method of the endoscopic image | video based on the luminance distribution of an endoscope, and is the luminance graph in the scanning line A of FIG. 9A. It is a modification of the head mounted display apparatus of FIG. 1, and has shown the block diagram provided with the imaging device. FIG. 11 is a functional block diagram of the head-mounted display device of FIG. 10. 1 is an overall configuration diagram of a head-mounted display system according to an embodiment of the present invention. It is a figure which shows the structure of the endoscope used with the head mounted display system of FIG. It is a functional block diagram of the head mounted display system of FIG.

A head-mounted display device (HMD) 1 according to an embodiment of the present invention will be described below with reference to the drawings.
As shown in FIG. 1, the HMD 1 according to the present embodiment is for observing an image of the image display unit 3 placed in front of the user with the mounting frame 2 mounted on the head. As shown in FIG. 2, the HMD 1 includes a video input unit 4 to which video (endoscopic video) observed using an endoscope is input, and an endoscopic video input to the video input unit 4 ( A signal processing unit 5 that converts the format of the input video), a frame memory (first storage unit) 6 that stores the endoscopic video, and a reference memory (second storage) that stores a preselected reference video 7), a similarity determination unit 8 that determines whether or not the endoscopic video is similar to the reference video, and a video determination unit that determines whether or not the endoscopic video is an image of an observation site in the body. 9 and a control unit 10 for controlling the display of the video display unit 3. In the present embodiment, “internal body” means the inside of the body, and is a concept including not only a body wall and internal organs but also a body cavity and an oral cavity.

The mounting frame 2 is formed in an annular shape so as to surround the head of the wearer wearing the HMD 1. The mounting frame 2 may include a belt (not shown) or the like so that the length can be adjusted according to the size of the head of the wearer. The mounting frame 2 is provided with a head contact portion 2a made of an elastic material at a position arranged on the frontal head side so as to exactly fit the shape of the head of the wearer.

The video display unit 3 is supported by the mounting frame 2 so as to be disposed at a position that blocks a part of the field of view of the wearer when the wearer wears the mounting frame 2. Accordingly, the wearer can view the video on the video display unit 3 by moving the line of sight while directly viewing the peripheral portion of the video display unit 3. Here, “viewing directly” means that the wearer directly views the object without using the video display unit 3. As shown in FIG. 3, the video display unit 3 includes a display element 3a that displays the video input to the video input unit 4, and a prism 3b that guides the video displayed on the display element 3a to the eyes of the wearer. I have. As the display element 3a, a display using LCD (liquid crystal), OLED (organic EL), LCOS (reflection liquid crystal), or the like is used. The video display unit 3 can be switched between a state in which the display element 3a is turned on and video display is turned on and a state in which the display element 3a is turned off and video display is turned off by a switch (not shown). Yes.

The video input unit 4 receives an endoscope video imaged by the endoscope. The endoscopic video is also input to a monitor (not shown) so that a person other than the wearer of the HMD 1 can observe the endoscopic video on the monitor. The video input unit 4 converts the endoscopic video into a signal that can be processed by the signal processing unit 5.

The signal processing unit 5 appropriately changes the format of the endoscope video input to the video input unit 4. For example, an interlaced video signal is changed to a progressive video signal.
The frame memory 6 stores the endoscope video whose format has been converted by the signal processing unit 5 in time series in units of frames.

The reference memory 7 records the reference video A. Here, the reference image A is an image obtained by photographing the opening 11a of the trocar 11 from the inside of the trocar 11 inserted in the patient's body in the past endoscopic surgery, as shown in FIG. . That is, the reference image A is an image when the distal end of the endoscope is disposed in the vicinity of the opening 11a while the endoscope is passing through the trocar 11. In the reference image A and the endoscope image, it is preferable that the contrast between the inner surface of the trocar 11 and the dark body viewed through the opening 11a is clear. Therefore, the trocar 11 has an inner surface covered with a material having a high brightness or a material such as a metal that reflects the illumination light of the endoscope.

The similarity determination unit 8 sequentially determines whether or not the frame image of the endoscopic image stored in the frame memory 6 is similar to the reference image A. When the similarity determination unit 8 determines that the frame image is similar to the reference image A, the similarity determination unit 8 outputs a signal to that effect to the image determination unit 9. Thereby, when the endoscope is inserted into the trocar 11 and advanced to the vicinity of the opening of the trocar 11, or when the endoscope is pulled out from the trocar 11 and stored in the trocar 11, the reference image A An endoscope image similar to the above is taken, and the insertion / removal operation of the endoscope is quickly detected.

The method for determining whether or not the frame image and the reference image A are similar is not particularly limited in the present embodiment. For example, a known method such as a method using the correlation between these two images or a method using the difference between the pixels is used to determine whether these images are similar.

The image determination unit 9 determines whether the endoscopic image is an image of an observation site in the patient (subject) body or an image of a non-observation site other than that. That is, when the video determination unit 9 receives a signal from the similarity determination unit 8 that the frame video is similar to the reference video A, the video determination unit 9 switches the determination to the other. More specifically, when the video determination unit 9 determines that the endoscopic video is “the image of the observation site” until receiving the signal from the similarity determination unit 8, When the above signal is received, the determination is switched to “the image of the non-observed part”. Conversely, if the video determination unit 9 determines that the endoscopic video is “a video of a non-observed part” until the signal from the similarity determination unit 8 is received, the signal from the similarity determination unit 8 Is switched to a determination that the image is an observation site image. Here, in the initial state when the power is turned on to the HMD 1, the video determination unit 9 is set so that the endoscopic video is a video of a non-observation site.

As a result, when the video determination unit 9 first receives a signal from the similarity determination unit 8, the endoscope video is set as the video of the non-observation site as an initial state. The determination is switched by determining that the image has been changed to the image of the observation site. When the video determination unit 9 receives a signal from the similarity determination unit 8 next time, the video determination unit 9 switches from the determination that the endoscopic image is the image of the observation region to the determination that the image is the image of the non-observation region. Hereinafter, every time a signal is received from the similarity determination unit 8, the video determination unit 9 switches alternately between determination that the endoscopic video is an image of an observation site and determination that the video is an image of a non-observation site.

When the video determination unit 9 determines that the endoscopic video is an image of the observation site, the control unit 10 turns on the display of the endoscopic video on the video display unit 3 using a switch. In addition, when the video determination unit 9 determines that the endoscopic video is a non-observation site video, the control unit 10 turns off the display of the endoscopic video on the video display unit 3 using a switch. It has become.

The operation of the HMD 1 configured as described above will be described below.
The surgeon (wearer) wears the HMD 1 according to the present embodiment on the head and starts the operation while viewing the patient directly. In addition to the surgeon, there will be a scopist who operates the endoscope and photographs the surgical part inside the body. When the surgeon inserts the trocar 11 into the patient's body and turns on the power of the endoscope, an endoscopic image outside the body is displayed on the monitor. At this time, the video display unit 3 keeps the video disappeared.

When the endoscope is inserted into the trocar 11 while observing the monitor, the endoscope image is displayed on the video display unit 3 immediately before the endoscope exits the trocar 11 from the trocar 11. The The surgeon performs an operation by operating an instrument or the like while viewing the hand directly while referring to the endoscope image of the image display unit 3 while appropriately shifting the line of sight. When the scoopist removes the endoscope from the trocar 11 in order to clean and replace the endoscope during the operation, an endoscope image is displayed from the front of the surgeon immediately after the endoscope is stored in the trocar 11. Disappear. Thereafter, the video on the video display unit 3 disappears until the scopist inserts the endoscope into the trocar 11 again.

Thus, according to the present embodiment, the display of the video display unit 3 is turned on or off at the timing when the endoscope is inserted into or removed from the trocar 11. This eliminates the need for the operator to view unnecessary endoscope images in front of the eyes, such as when the endoscope is passing through the trocar 11 or while the endoscope is being cleaned. Moreover, such an unnecessary endoscopic image may be an unpleasant image depending on the surgeon, which may hinder the surgeon's concentration or give unnecessary stress to the surgeon. According to this embodiment, there is an advantage that it is possible to prevent such stress on the operator. In addition, since the display on the video display unit 3 can be switched at an appropriate timing, there is no need for an operator to operate the video display unit 3 or to indicate the timing for switching the display, and the operator can be saved without burden. There is.

In the above embodiment, the display of the video display unit 3 is turned off when the endoscopic video becomes a video of a non-observation site. Instead, another video is displayed on the video display unit 3. It may be displayed. In this case, for example, as shown in FIG. 5, the HMD 1 includes a sub-memory or another video input unit 12 and a video switching unit 13 that switches a video to be input to the video display unit 3. If it is necessary to change the format of the endoscopic video input to the video input unit 12, a signal processing unit (not shown) may be provided between the video input unit 12 and the video switching unit 13.

When the sub memory 12 is provided, videos that do not cause discomfort to the surgeon, such as a blue-colored blue background video and videos according to the operator's preference, are stored in the sub memory 12 in advance. When another video input unit 12 is provided, an external device video such as a patient's vitals or an electrocardiogram is input to the other video input unit 12. While the video determination unit 9 determines that the endoscopic video is a non-observation site video, the control unit 10 inputs the video display unit 3 from the video input unit 4 to the sub memory or another video input. The video switching unit 13 is controlled to switch to the unit 12. The video display unit 3 displays an endoscopic video or an arbitrary video other than the endoscopic video in accordance with the switching operation of the video switching unit 13. Even if it does in this way, when an endoscopic image | video is unnecessary for an operator, it will not be displayed (it will not display), and an operator's stress can be reduced.

In the embodiment described above, whether the endoscopic image is an image of an observation site or an image of a non-observation site is determined based on whether or not the endoscopic image and the reference image A are similar. However, instead of this, the determination may be made based on the movement of the endoscope video. In this case, as shown in FIG. 6, the HMD 1 includes a feature portion extraction unit 14 that extracts a feature portion from an endoscopic video, and a motion detection unit 15 that detects the motion of the feature portion.

The feature portion extraction unit 14 extracts the outline of the opening of the trocar 11 as a feature portion from the frame image stored in the frame memory (image storage unit) 6. There is no particular limitation on the method for extracting the feature portion. For example, the contour of the opening of the trocar 11 can be extracted by spatially differentiating the luminance at each position using the difference in luminance between the bright inner surface of the trocar 11 and the dark body viewed through the opening.
Alternatively, as in the above embodiment, a reference memory is provided, and an image of the opening portion of the trocar 11 is stored as a template image in the reference memory. Then, the feature portion extraction unit 14 may extract the outline of the trocar 11 by searching a region similar to the template video from the frame video.

The motion detection unit 15 detects the motion vector of the feature portion from the change in the position of the feature portion extracted by the feature portion extraction unit 14 in each frame image for the continuous frame images. The video determination unit 9 determines that the endoscopic video is an image of a non-observation site and the observation site when the direction of the motion vector detected by the motion detection unit 15 is substantially the same over a predetermined time. The judgment that it is a video is switched alternately. Even in this way, it is possible to reduce the surgeon's stress by appropriately switching the display of the video display unit 3 while determining the endoscopic video necessary for the surgeon and the unnecessary endoscopic video.

In the above embodiment, the determination as to whether the endoscopic image is an image of an observation site or a non-observation site may be made based on the luminance distribution of the endoscopic image. . In this case, as shown in FIG. 7, a luminance distribution detection unit 16 that detects the luminance distribution of a predetermined scanning line of the endoscopic video and a video processing unit 17 that determines and processes the shape of the luminance distribution are provided. .

The luminance distribution detection unit 16 sequentially creates a luminance graph (luminance distribution) of the scanning line at a predetermined position for each frame image stored in the frame memory 6. At this time, as shown in FIG. 8A, when the opening 11a of the trocar 11 is reflected in the frame image C, the luminance graph in the scanning line a has a luminance value at the middle position as shown in FIG. 8B. Bipolarization into a high luminance region with a high brightness and a low luminance region with a low luminance value. Then, when the endoscope moves in the trocar 11, the size of the opening 11a in the frame image changes as shown in FIG. 9A, and on the scanning line a occupied by the low luminance area as shown in FIG. 9B. The line widths W1 and W2 change.

Note that the luminance distribution detection unit 16 sets 2 for each frame image before creating a luminance graph by setting a pixel having a luminance equal to or higher than a predetermined threshold to white and a pixel having a luminance lower than the predetermined threshold to 2 It may be converted into a value.

The video processing unit 17 determines whether or not the luminance graph is bipolar in a region where the luminance value detected by the luminance distribution detection unit 16 is higher than a predetermined threshold and a region lower than the predetermined threshold. If the video processing unit 17 determines that the luminance graph is bipolar, the video processing unit 17 calculates the line width W1 of the low-luminance region, and similarly calculates the line width W2 for the next frame video. W1 is compared with the line width W2.

Then, if the line widths W1 and W2 of the low-luminance area increase with time, the video determination unit 9 determines that the endoscope has been inserted into the body close to the opening 11a, and determines the video of the observation site. Switch to. On the other hand, if the line widths W1 and W2 of the low-luminance area have decreased over time, the video determination unit 9 determines that the endoscope has been removed from the body away from the opening, and the determination is made for the non-observation site. Switch to video. Even in this way, the endoscope image can be determined based on the operation of inserting and removing the endoscope into the trocar 11.

In the above embodiment, an endoscopic video is displayed on the video display unit 3, but instead, as shown in FIG. 10, a video acquisition unit such as a camera 18 is provided, and the video acquisition unit The video acquired by the above may be displayed on the video display unit 3. For example, as shown in FIG. 10, two cameras 18 may be supported by the mounting frame 2 so as to be arranged symmetrically on the frontal side of the surgeon. Since the camera 18 can magnify the field of view in front of the surgeon, the image display unit 3 can observe an enlarged image of the surgeon in a comfortable posture. An illumination unit 19 that illuminates the field of view of the camera 18 may be provided.

In this case, the HMD 1 includes a hue detection unit 20 and a video processing unit 21, for example, as shown in FIG. The hue detection unit 20 detects the degree occupied by the red color component by decomposing the color of each pixel of the frame image into the red, green, and blue color components. The video processing unit 21 calculates an average value of all the pixels of the frame video for the degree of the red color component detected by the hue detection unit 20. If the average value of the red color component calculated by the video processing unit 21 is equal to or greater than a predetermined threshold, the video determination unit 9 determines that the camera video is an image of the observation region, and if the average value is smaller than the predetermined threshold. Then, it is determined that the camera image is an image of a non-observation site. In this way, when the camera image is an image of the surgical site, the color of the image is biased to skin color or red, and when the camera image is an image other than the surgical site, the color bias is different from that of the surgical site. Is used to determine whether or not it is an observation site from the difference in red hue of the video. Even in such a case, since the camera image that does not require magnified observation is not displayed on the image display unit 3, the surgeon's stress can be reduced.

In the above embodiment, the configuration of the HMD 1 alone is used, but instead of this, as shown in FIG. 12, a head-mounted display system 100 including the HMD 1 and the endoscope 22 may be configured. Good. Reference numeral 23 denotes a processing device for processing an endoscopic video, and reference numeral 24 denotes a monitor for displaying the endoscopic video. The endoscopic video is input to the video input unit 4 via the processing device 23.

In this case, as shown in FIG. 13, the endoscope 22 includes a motion detection unit that detects the insertion / removal operation, for example, an acceleration sensor 25 that detects acceleration in the front-rear direction of the distal end portion of the endoscope 22. You may have. As the acceleration sensor 25, for example, a MEMS acceleration sensor can be used, but is not limited thereto. Reference numeral 26 denotes an optical member such as a lens or a cover glass, reference numeral 27 denotes an imaging device such as a CCD camera, reference numeral 28 denotes an illumination device such as a light emitting diode, and reference numeral 29 denotes a substrate provided with an integrated circuit.

Acceleration information detected by the acceleration sensor 25 is transmitted to the HMD 1 via a wiring (not shown), and the acceleration information is input to the motion detector 15 as shown in FIG. In this case, the motion detection unit 15 detects the motion vector of the distal end portion of the endoscope 22 from the input acceleration. Based on the magnitude and direction of the motion vector detected by the motion detection unit 15, the video determination unit 9 determines whether the endoscopic video is an image of an observation site or an image of a non-observation site. Even in this case, by detecting the insertion / removal operation of the endoscope 22 based on the acceleration, it is determined whether or not the endoscope image is an observation site in the body, and unnecessary based on the determination. By not displaying (not displaying) the endoscope image on the image display unit 3, it is possible to reduce the stress on the operator.

1 Head-mounted display device (HMD)
DESCRIPTION OF SYMBOLS 2 Mounting frame 2a Head contact part 3 Image | video display part 3a Display element 3b Prism 4 Image | video input part 5 Signal processing part 6 Frame memory (1st image | video storage part, image | video storage part)
7 Reference memory (second video storage unit)
DESCRIPTION OF SYMBOLS 8 Similarity determination part 9 Image | video determination part 10 Control part 11 Trocar 11a Aperture 12 Sub memory or image | video input part 13 Image | video switching part 14 Feature part extraction part 15 Motion detection part 16 Luminance distribution detection part 17 Video | video process part 18 Camera (video acquisition part) )
DESCRIPTION OF SYMBOLS 19 Illumination part 20 Hue detection part 21 Image | video process part 22 Endoscope 23 Processing apparatus 24 Monitor 25 Acceleration sensor (motion detection part)
26 Optical member 27 Imaging device 28 Illumination device 29 Substrate 100 Head mounted display system A Reference image B Inside C Frame image a Scan line

Claims

A video input unit to which video is input;
A video display unit that is mounted on the head of the wearer and is supported so as to be placed in front of the wearer's eyes, and displays an input video input to the video input unit;
A video determination unit for determining whether or not the input video is a video of an observation site of a subject;
A head-mounted display device comprising: a control unit configured to display the input video on the video display unit when the video determination unit determines that the input video is a video of an observation site inside the body.
A first video storage unit for storing the input video;
A second video storage unit for storing a predetermined reference video;
A similarity determination unit that determines whether or not the input video stored in the first video storage unit is similar to the reference video stored in the second video storage unit,
The video determination unit, when the similarity determination unit determines that the input video is similar to the reference video, the input video is a video of a non-observation site other than the observation site, or , Determine whether the image of the observation site,
The control unit hides the input video on the video display unit when the video determination unit determines that the input video is a video of the non-observation site, and the video determination unit controls the input video to be displayed. The head-mounted display device according to claim 1, wherein when the image is determined to be an image of the observation site, the input image is displayed on the image display unit.
The head-mounted display device according to claim 2, wherein the reference image is an image obtained by photographing an opening of a cylindrical trocar into which an endoscope can be inserted and removed.
A video storage unit for storing the input video;
A feature extraction unit that extracts a predetermined feature from the input video stored in the video storage unit;
A motion detector that detects the motion of the feature portion extracted by the feature portion extractor;
The video determination unit is a video of a non-observation site other than the observation site when the movement of the feature portion detected by the motion detection unit continues in a substantially same direction for a predetermined time. Determine whether or not it is an image of the observation site,
The control unit hides the input video on the video display unit when the video determination unit determines that the input video is a video of the non-observation site, and the video determination unit controls the input video to be displayed. The head-mounted display device according to claim 1, wherein when the image is determined to be an image of the observation site, the input image is displayed on the image display unit.
The head-mounted display device according to claim 4, wherein the characteristic portion is a contour when an opening of a cylindrical trocar into which an endoscope can be inserted and removed is photographed from inside.
A video storage unit for storing the input video;
A luminance distribution detector that detects a luminance distribution of a predetermined line of the input video stored in the video storage unit;
Determining whether the luminance distribution detected by the luminance distribution detection unit is bipolarized into a low luminance region having a luminance value lower than a predetermined threshold and a high luminance region having a luminance value higher than the predetermined threshold; An image processing unit for detecting a line width of the low luminance region when the luminance distribution is bipolar;
The video determination unit determines that the input video is a video of the observation site when the line width of the low luminance area detected by the video processing unit increases with time, and the video processing unit The head-mounted display device according to claim 1, wherein the input image is determined to be an image of a non-observation region other than the observation region when a line width of the detected low-luminance region decreases with time. .
A video switching unit that switches the video to be input to the video display unit between the input video and another arbitrary video;
The control unit causes the video switching unit to switch to the input video when the video determination unit determines that the input video is a video of the observation site, and the video determination unit causes the input video to be changed to the input video. The head-mounted display device according to claim 1, wherein when the video is determined to be an image of a non-observation site other than the observation site, the video switching unit switches to the arbitrary video.
The control unit turns on the display of the video display unit when the video determination unit determines that the input video is the video of the observation site, and the video determination unit sets the input video to the observation site. The head-mounted display device according to claim 1, wherein the display of the video display unit is turned off when it is determined that the video is an image of a non-observation site other than.
The head-mounted display device according to claim 1, further comprising a video acquisition unit that acquires a video of the subject and inputs the acquired video to the video input unit.
An endoscope provided with a motion detector for detecting movement in the insertion direction at the distal end;
A head-mounted display system comprising the head-mounted display device according to any one of claims 1 to 9, wherein an endoscope video imaged by the endoscope is input to the video input unit.