JP2008060981A - Image observation apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video see-through type image observation apparatus capable of preventing an image having after-image from being observed when imaging and observing a fast-moving object. <P>SOLUTION: The image observation apparatus 20 includes: an imaging system which includes an imaging element 2 and images an object; a display element 15 which displays an image generated based on an output signal from the imaging element; an eye-piece optical system 16 for observing the image displayed on the display element; and a motion detecting means 9 which detects the motion of the object from the generated image. A control means 11 controls an electronic shutter speed of the imaging element in accordance with the moving speed of the object detected by the motion detecting unit. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image observation apparatus that displays an external image acquired by an imaging system on a display element and observes the image through an eyepiece optical system.

  A so-called video see-through type image observation apparatus that acquires an external image using an imaging system including an image sensor such as a CCD sensor, displays the external image on a display element such as an LCD, and observes the image through an eyepiece optical system. Conventionally proposed (for example, Patent Document 1).

  Further, in such a video see-through type image observation apparatus, a virtual space image such as a CG (computer graphics) image or a recorded image obtained by an imaging device such as a video camera is combined with an external image (real space image) for display and observation. Sometimes it is possible.

  By the way, in such an image observation apparatus, it is necessary to display an external image at a high frame rate with as little delay as possible so that the observer feels as if the external environment is being observed with the naked eye. This is because if there is a delay in the display of the external image, the actual external movement (for example, the movement of the observer's own hand) and the movement on the observation image will occur, giving the viewer a sense of incongruity. Because.

  In general, humans can recognize patterns and images that change at high speeds in the order of 10 Hz. For this reason, when a black rectangular object is rotated 20 times per second, that is, at a high speed of 120 rpm or more as shown in FIG. 8, the human being has a gray circle object whose center is black and the surrounding area is brighter as shown in FIG. Recognized as

In other words, if the display delay of the external image is about 0.05 seconds, it is considered that the observer does not feel so uncomfortable.
JP 2001-133725 A (FIG. 1 etc.)

  However, in the image observation apparatus as described above, there is a delay of several frames from imaging (photoelectric conversion) to display on the display element by an external image capturing system due to a delay in electrical processing. appear. For example, when an image captured at a frame rate of 60 Hz (60 fps) is displayed on the display element, the observer sees an image of 2 to 3 frames within about 0.05 seconds of the time resolution.

  For this reason, for example, as shown in FIG. 10, when an object to be observed moves to some extent during 0.05 seconds, two or three images with different object positions overlap each other as shown in FIG. The image is observed as a so-called afterimage.

  An object of the present invention is to provide a video see-through type image observing apparatus and a control method thereof that can prevent an image having an afterimage from being observed when a fast-moving subject is imaged and observed. One.

  An image observation apparatus according to an aspect of the present invention includes an imaging device, an imaging system that images a subject, a display device that displays an image generated based on an output signal from the imaging device, and a display on the display device An eyepiece optical system for observing the generated image, and motion detection means for detecting the motion of the subject from the generated image. And it has the control means which controls the electronic shutter speed of an image sensor according to the motion speed of the subject detected by the motion detector.

  In addition, an image observation apparatus according to another aspect of the present invention includes an imaging device, an imaging system that images a subject, a display device that displays an image generated based on an output signal from the imaging device, and the display An eyepiece optical system for observing the image displayed on the element, a motion detection means for detecting the motion of the subject from the generated image, and the motion of the subject is detected by the motion detection means in the generated image Image processing means for performing pixel addition and smoothing processing in a pixel area including pixels. And it has the control means which changes the magnitude | size of the said pixel area according to the moving speed of the to-be-photographed object detected by the motion detection part, It is characterized by the above-mentioned.

  In addition, an image observation apparatus according to another aspect of the present invention includes an imaging device, an imaging system that images a subject, a display device that displays an image generated based on an output signal from the imaging device, and the display An eyepiece optical system for observing the image displayed on the element, a motion detection means for detecting the motion of the subject from the generated image, and the motion of the subject is detected by the motion detection means in the generated image Image processing means for performing pixel addition and smoothing processing in a pixel area including pixels. And a control unit that causes the image processing unit to perform pixel addition and smoothing processing in the pixel area when the motion speed of the subject detected by the motion detection unit is faster than the first motion speed. To do.

  Furthermore, an image observation apparatus according to another aspect of the present invention includes an imaging element, an imaging system that images a subject, a display element that displays an image generated based on an output signal from the imaging element, and the display An eyepiece optical system for observing an image displayed on the element, a motion detection means for detecting a motion of a subject from the generated image, and pixel addition for the generated image and a plurality of frame images Image processing means for performing processing. Then, when the motion speed of the subject detected by the motion detection unit is faster than the first motion speed, the image processing means is made to perform pixel addition processing on the plurality of frame images.

  In addition, a control method according to another aspect of the present invention includes an imaging device that includes an imaging device, images a subject, a display device that displays an image generated based on an output signal from the imaging device, and the display device And an eyepiece optical system for observing an image displayed on the screen. The control method includes a detection step of detecting the movement of the subject from the generated image, and a step of controlling the electronic shutter speed of the image sensor according to the movement speed of the subject detected in the detection step. It is characterized by.

  In addition, a control method according to another aspect of the present invention includes an imaging device that includes an imaging device, images a subject, a display device that displays an image generated based on an output signal from the imaging device, and the display device And an eyepiece optical system for observing an image displayed on the screen. The control method performs a pixel addition and smoothing process in a pixel area including a detection step of detecting the movement of the subject from the generated image and a pixel in which the movement of the subject is detected by the detection step in the generated image. Processing steps. In the processing step, the size of the pixel area is changed according to the movement speed of the subject detected in the detection step.

  In addition, a control method according to another aspect of the present invention includes an imaging device that includes an imaging device, images a subject, a display device that displays an image generated based on an output signal from the imaging device, and the display device And an eyepiece optical system for observing an image displayed on the screen. The control method performs a pixel addition and smoothing process in a pixel area including a detection step of detecting the movement of the subject from the generated image and a pixel in which the movement of the subject is detected by the detection step in the generated image. Processing steps. In the processing step, when the movement speed of the subject detected in the detection step is faster than the first movement speed, pixel addition and smoothing processing are performed in the pixel area.

  Furthermore, a control method according to another aspect of the present invention includes an imaging device that includes an imaging device, images a subject, a display device that displays an image generated based on an output signal from the imaging device, and the display device. And an eyepiece optical system for observing an image displayed on the screen. The control method includes a detection step of detecting a motion of a subject from the generated image, and a processing step of performing pixel addition processing on the generated image and a plurality of frame images. In the processing step, when the movement speed of the subject detected in the detection step is faster than the first movement speed, pixel addition processing is performed on a plurality of frame images.

  According to the present invention, it is possible to cause an observer to perform a good image observation without a sense of incongruity by performing a process of making an afterimage inconspicuous on an image obtained by imaging a fast-moving subject. .

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows the configuration of an image observation system that is Embodiment 1 of the present invention. In FIG. 1, reference numeral 20 denotes a video see-through head mounted display (image observation apparatus), and 30 denotes an image for supplying a virtual space image recorded by an imaging apparatus such as a CG image or a video camera to the head mounted display 20. It is a personal computer as a supply device.

  In the video see-through head mounted display 20, the subject image formed by the imaging lens 1 is formed on the imaging surface of the solid-state imaging device 2 constituted by a CCD sensor, a CMOS sensor, or the like. The image sensor 2 is driven by a drive pulse signal from a drive circuit 3 that operates based on a timing pulse signal output from the pulse generation circuit 4. The imaging lens 1 and the imaging element 2 constitute an imaging system.

  An imaging output signal from the imaging device 2 is amplified to a predetermined level by the preamplifier 5 and then input to the camera process circuit 6. The camera process circuit 6 performs signal processing such as gamma correction on the imaging output signal, and sequentially generates frame images.

  The frame images generated in the camera process circuit 6 are sequentially sent to and displayed on the display element 15 constituted by an LCD or the like. The observer observes the image displayed on the display element 15 through the eyepiece optical system 16. E is the observer's eye.

  One frame image generated in the camera process circuit 6 is written in the frame memory 7. Also, the previous frame image written in the frame memory 7 prior to this is written in the frame memory 8. Two frame images written in the frame memories 7 and 8 (that is, two images captured and generated with a time difference from each other) are input to a motion vector detection circuit 9 as a motion detection unit.

  The motion vector detection circuit 9 detects a motion vector representing the direction and magnitude (speed) of the subject motion for each pixel from the input two frame images using a known motion vector detection method. Then, the motion vector detection result is sent to the system control unit 11 constituted by a microcomputer.

  The system control unit 11 controls the electronic shutter speed of the image sensor 2 and the gain of the preamplifier 5 according to the motion speed of the subject indicated by the input motion vector (hereinafter referred to as motion vector speed). The electronic shutter temporarily stops accumulation of electric charge obtained by photoelectric conversion in the light receiving unit of the image sensor 2 at a predetermined time, and reads (transfers) the electric charge accumulated within the predetermined time as an imaging output signal through the register. In the moving image shooting, by repeating this operation, frame images are sequentially generated. The predetermined time corresponds to the electronic shutter speed.

  Here, the relationship between the motion vector speed and the electronic shutter speed in this embodiment will be described with reference to FIG. In FIG. 2, the imaging lens 1, the imaging device 2, and the subject A are shown.

  In general, the angle of view of the imaging system is wider in the horizontal direction (the direction parallel to the paper surface in FIG. 2) than in the vertical direction (the direction perpendicular to the paper surface in FIG. 2), and the movement of the subject to be imaged is also in the vertical direction. More horizontal movement than For this reason, in this embodiment, the horizontal direction is considered as a reference.

  The horizontal field angle of the imaging system is φ °, and the horizontal resolution of the imaging device 2 is ω pixels. Further, the moving speed of the subject on the image sensor 2, that is, the motion vector speed is set to P pixels / s. In this case, the resolution of the human eye is about 1 minute in terms of angle of view, and if the subject moves for 1 minute or more in terms of angle of view with a time resolution of 0.05 seconds, an image with an afterimage is observed. .

  One minute in terms of angle of view corresponds to ω / (60 × φ) pixels on the image sensor 2. Therefore, if the motion vector speed is equal to or higher than ω / (60 × φ) /0.05=ω/ (3 × φ) [pixel / s], the observer recognizes the observation image as an afterimage.

  Further, depending on the horizontal resolution of the image sensor 2, the number of pixels corresponding to 1 minute may be 2 pixels or less. In other words, when ω / (60 × φ) <2, the observer moves to the afterimage when the subject moves at a motion vector speed of 2 to 3 pixels, that is, 40 pixels / s or more in 0.05 seconds. Recognize an image.

From these, the threshold of the motion vector speed (first motion speed) for the control of the electronic shutter speed is
When ω / (60 × φ)> 2, ω / (3 × φ) [pixel / s]
When ω / (60 × φ) ≦ 2, 40 pixels / s
It is good to do.

In this embodiment, the drive frequency of the image sensor 2 is 60 Hz, and the electronic shutter speed can be changed from 1/500 seconds to 1/60 seconds, for example. The horizontal angle of view φ of the imaging lens 1 is 50 °, and the resolution (number of pixels) of the imaging device 2 is horizontal 640 (= ω) pixels and vertical 480 pixels corresponding to VGA output. For this reason,
ω / (60 × φ) ≈0.2 pixel ≦ 2
It is. Therefore, the threshold is 40 pixels / s.

  FIG. 3A shows a method of controlling the electronic shutter speed [second (s)] of the image sensor 2 with respect to the motion vector speed [pixel / s] in the present embodiment.

  In FIG. 3A, when the motion vector speed is zero (when the subject is stationary), an electronic shutter speed at a standard gain of the preamplifier 5 corresponding to the imaging environment, for example, 1/125 seconds is set. When the motion vector speed is not zero (when the subject is moving), the electronic shutter speed becomes slower to 1/60 seconds in proportion to the motion vector speed as the motion vector speed increases to 40 pixels / s. Set as follows.

  Further, the gain of the preamplifier 5 is also controlled in accordance with the change in the electronic shutter speed, and the exposure is kept constant. Specifically, the gain is set lower as the motion vector speed is faster.

  In this embodiment, since the image sensor 2 is driven at 60 Hz, three frame images are displayed in 0.05 seconds, which is a general human time resolution. For this reason, at a motion vector speed of 40 pixels / s or more on the image sensor 2 corresponding to 1 minute, which is a resolution converted to an angle of view, an afterimage is recognized by setting the electronic shutter speed to the slowest 1/60 second. This prevents the displayed image from being displayed.

  That is, when a scene in which the object shown in FIG. 10 moves from the left to the right in the figure is captured, if the electronic shutter speed is high, for example, 1/500 seconds, as shown in FIG. Image overlap) is recognized. However, as described above, by setting the electronic shutter speed to 1/60 seconds, as shown in FIG. 12, the image is blurred (blurred) and the afterimage becomes inconspicuous.

  The flowchart of FIG. 3B shows a procedure for controlling the electronic shutter speed of the image sensor 2 and the gain of the preamplifier 5 performed by the system control unit 11. This control is executed according to a computer program stored in the system control unit 11. This also applies to other embodiments described below.

  First, in step (denoted as S in the figure) 101, the system control unit 11 detects a motion vector from two frame images and calculates a motion vector speed.

  Next, in step 102, it is determined whether or not the motion vector speed is zero. If zero, the routine proceeds to step 104, where the electronic shutter speed is set to 1/125 seconds, and the gain of the preamplifier 5 is set to Low (standard gain).

  If the motion vector speed is not zero, the process proceeds to step 103 to determine whether the motion vector speed is 40 pixels / s or more. If it is slower than 40 pixels / s, the process proceeds to step 105, where the electronic shutter speed proportional to the motion vector speed and the gain of the preamplifier 5 corresponding thereto are set.

  On the other hand, if it is 40 pixels / s or more, the process proceeds to step 106, the motion vector speed is set to 1/60 seconds, and the gain of the preamplifier 5 is set to High. The gain set in step 105 and step 106 is set so that the same level of exposure as that obtained when the electronic shutter speed is 1/125 seconds and the gain of the preamplifier 5 is low is obtained.

  If the number of pixels corresponding to 1 minute ω / (60 × φ)> 2, it is determined in step 103 of FIG. 3B whether the motion vector speed is equal to or higher than ω / (3 × φ) [pixels / s]. If it is later than that, the process proceeds to Step 105, and if it is more than that, the process proceeds to Step 106.

  The electronic shutter speed for each motion vector speed described above is merely an example, and can be arbitrarily selected.

  In the present embodiment, the case where the gain of the preamplifier 5 is controlled according to the change in the electronic shutter speed has been described. However, the aperture provided in the imaging lens 1 is controlled instead of the gain of the preamplifier 5, You may make it maintain exposure constant.

  Furthermore, in this embodiment, the case where the subject moves with respect to the imaging system (head mounted display) has been described, but the same electronic shutter control can be performed when the imaging system moves with respect to the subject. The same applies to the following embodiments.

  FIG. 4 shows a video see-through head mounted display that is Embodiment 2 of the present invention. This embodiment is the same as the first embodiment in that an image supply apparatus is connected to a video see-through head mounted display to constitute an image observation system.

  In the video see-through head mount display 21, the subject image formed by the imaging lens 1 is formed on the imaging surface of the solid-state imaging device 2 constituted by a CCD sensor, a CMOS sensor, or the like. The image sensor 2 is driven by a drive pulse signal from a drive circuit 3 that operates based on a timing pulse signal output from the pulse generation circuit 4. The imaging lens 1 and the imaging element 2 constitute an imaging system.

  An imaging output signal from the imaging device 2 is amplified to a predetermined level by the preamplifier 5 and then input to the camera process circuit 6. The camera process circuit 6 performs signal processing such as gamma correction on the imaging output signal, and sequentially generates frame images.

  One frame image generated in the camera process circuit 6 is written in the frame memory 7. Also, the previous frame image written in the frame memory 7 prior to this is written in the frame memory 8. The two frame images written in the frame memories 7 and 8 are input to the motion vector detection circuit 9.

  The motion vector detection circuit 9 detects a motion vector representing the direction and magnitude (speed) of the subject motion for each pixel from the input two frame images using a known motion vector detection method. Then, the motion vector detection result is sent to the system control unit 11 constituted by a microcomputer.

  The frame image generated in the camera process circuit 6 is also input to the pixel addition circuit 10 as image processing means via the frame memory 7.

  The system control unit 11 determines the number of pixels to be subjected to pixel addition processing in the pixel addition circuit 10, that is, the size of the pixel area, in accordance with the motion speed of the subject indicated by the input motion vector (hereinafter referred to as motion vector speed). Control. The pixel addition circuit 10 performs pixel addition processing and smoothing processing with the number of pixels specified by the system control unit 11. Pixel addition processing adds (and averages) signals of a predetermined number of pixels (pixel areas) centered on or including pixels having a motion vector speed equal to or higher than a predetermined threshold in each frame image. Thus, this is a process of outputting the corresponding pixel signal in each frame image. The smoothing process is a process for smoothing the edge of the pixel area on which the pixel addition process is performed.

  The frame image after the pixel addition processing is sent to the display element 15 constituted by an LCD or the like via the camera process circuit 6 and displayed. The observer observes the image displayed on the display element 15 through the eyepiece optical system 16. E is the observer's eye.

Here, the threshold of the motion vector speed (first motion speed) for performing the pixel addition process is
When ω / (60 × φ)> 2, ω / (3 × φ) [pixel / s]
When ω / (60 × φ) ≦ 2, 40 pixels / s
And

In this embodiment, the drive frequency of the image sensor 2 is 60 Hz, and the electronic shutter speed can be changed from 1/500 seconds to 1/60 seconds, for example. The horizontal angle of view φ of the imaging lens 1 is 50 °, and the resolution (number of pixels) of the imaging device 2 is horizontal 640 (= ω) pixels and vertical 480 pixels corresponding to VGA output. For this reason,
ω / (60 × φ) ≈0.2 pixel ≦ 2
It becomes. Therefore, the threshold is 40 pixels / s.

  FIG. 5A shows the relationship between the motion vector speed and the number of pixels on which pixel addition is performed in the present embodiment. When the motion vector speed is 0 (pixel / s) or more and slower than 40 (pixel / s), the pixel addition process is not performed. When the motion vector speed is 40 (pixel / s) or higher and lower than 80 (pixel / s), pixel addition processing is performed in a pixel area of 4 × 4 pixels.

  When the motion vector speed is 80 (pixel / s) or higher and lower than 160 (pixel / s), pixel addition processing is performed in a pixel area of 8 × 8 pixels. Furthermore, when the motion vector speed is 160 (pixels / s) or more, pixel addition processing is performed in a pixel area of 16 × 16 pixels.

  In this embodiment, since the image sensor 2 is driven at 60 Hz, three frame images are displayed in 0.05 seconds, which is a general human time resolution. For this reason, by increasing the number of pixels to be added as the motion vector speed on the image sensor 2 increases, it is possible to avoid displaying an image in which an afterimage is recognized.

  In addition, the motion vector speed is calculated for each pixel, and pixel addition processing is performed in an area with a predetermined number of pixels centering on the pixels whose motion vector speed is equal to or greater than the threshold value, thereby reducing the image quality of the image portion such as the background with less motion. It is possible to remove afterimages on a subject moving at high speed without being reduced. Furthermore, by performing the smoothing process in the pixel area where the pixel addition process has been performed, it is possible to prevent an unnatural image in which the contour of the moving subject is jagged from being observed.

  That is, when a scene in which the object shown in FIG. 10 moves from the left to the right in the figure is captured, an afterimage (overlapping of three images) is not observed as shown in FIG. As shown in FIG. 12, the image is blurred (blurred) and the afterimage becomes inconspicuous.

  The flowchart of FIG. 5B shows a procedure for controlling the number of pixels in the pixel addition process performed by the system control unit 11.

  First, in step (denoted as S in the figure) 201, the system control unit 11 detects a motion vector from two frame images and calculates a motion vector speed.

  Next, in step 202, it is determined whether or not the motion vector velocity V is 0 ≦ V <40 (pixels / s). If the motion vector speed V is within this range, the process proceeds to step S205, and the pixel addition process is not performed. If the motion vector speed V is not within this range, the process proceeds to step S203.

  In step 203, it is determined whether or not the motion vector speed V is 40 ≦ V <80 (pixels / s). If the motion vector speed V is within this range, the process proceeds to step S206, and the number of pixels to be subjected to pixel addition processing is set to 4 × 4 pixels. If the motion vector speed V is not within this range, the process proceeds to step S204.

  In step 204, it is determined whether or not the motion vector speed V is 80 ≦ V <160 (pixels / s). If the motion vector speed V is within this range, the process proceeds to step S207, and the number of pixels to be subjected to pixel addition processing is set to 8 × 8 pixels. If the motion vector speed V is not within this range, the process proceeds to step S208.

  In step 208, assuming that V ≧ 160 (pixels / s), the number of pixels to be subjected to pixel addition processing is set to 16 × 16 pixels.

  Note that if ω / (60 × φ)> 2, whether or not 0 ≦ V <ω / (3 × φ) [pixel / s] in each of steps 202 and 203 in FIG. 5B, ω / (3 It may be determined whether or not xφ) ≦ V <80 pixels / s.

  The number of pixels to be subjected to pixel addition processing for each motion vector speed described above is merely an example, and can be arbitrarily selected.

  FIG. 6 shows a video see-through type head mounted display which is Embodiment 3 of the present invention. This embodiment is the same as the first embodiment in that an image supply apparatus is connected to a video see-through head mounted display to constitute an image observation system.

  In the video see-through type head mounted display 22, the subject image formed by the imaging lens 1 is formed on the imaging surface of the solid-state imaging device 2 constituted by a CCD sensor, a CMOS sensor, or the like. The image sensor 2 is driven by a drive pulse signal from a drive circuit 3 that operates based on a timing pulse signal output from the pulse generation circuit 4. The imaging lens 1 and the imaging element 2 constitute an imaging system.

  An imaging output signal from the imaging device 2 is amplified to a predetermined level by the preamplifier 5 and then input to the camera process circuit 6. The camera process circuit 6 performs signal processing such as gamma correction on the imaging output signal, and sequentially generates frame images.

  One frame image generated in the camera process circuit 6 is written in the frame memory 7. Also, the previous frame image written in the frame memory 7 prior to this is written in the frame memory 8. The two frame images written in the frame memories 7 and 8 are input to the motion vector detection circuit 9.

  The motion vector detection circuit 9 detects a motion vector representing the direction and magnitude (speed) of the subject motion for each pixel from the input two frame images using a known motion vector detection method. Then, the motion vector detection result is sent to the system control unit 11 constituted by a microcomputer.

  The frame image generated in the camera process circuit 6 is also input to the pixel addition circuit 10 ′ serving as image processing means via the frame memory 7. The pixel addition circuit 10 ′ performs pixel addition processing between frame images.

  The system control unit 11 controls the electronic shutter speed of the image sensor 2 and the frame image in the pixel addition circuit 10 ′ according to the motion speed of the subject (hereinafter referred to as motion vector speed) indicated by the input motion vector. Control of pixel addition processing between them. The method for controlling the electronic shutter speed is the same as that in the first embodiment. Further, the pixel addition processing between the frame images in the present embodiment is performed for all the pixels of each frame image.

  The frame image after the pixel addition processing is sent to the display element 15 constituted by an LCD or the like via the camera process circuit 6 and displayed. The observer observes the image displayed on the display element 15 through the eyepiece optical system 16. E is the observer's eye.

Here, the threshold of the motion vector speed (first motion speed) for performing the pixel addition process is
When ω / (60 × φ)> 2, ω / (3 × φ) [pixel / s]
When ω / (60 × φ) ≦ 2, 40 pixels / s
Good.

In this embodiment, the drive frequency of the image sensor 2 is 60 Hz, and the electronic shutter speed can be changed from 1/500 seconds to 1/60 seconds, for example. The horizontal angle of view φ of the imaging lens 1 is 50 °, and the resolution (number of pixels) of the imaging device 2 is horizontal 640 (= ω) pixels and vertical 480 pixels corresponding to VGA output. For this reason,
ω / (60 × φ) ≈0.2 pixel ≦ 2
It becomes. Therefore, the threshold is 40 pixels / s.

  Further, since the image sensor 2 is driven at 60 Hz, three frame images are displayed in 0.05 seconds, which is a general human time resolution. For this reason, it is possible to avoid displaying an image in which an afterimage is recognized by increasing the number of frames to which pixels are added as the motion vector speed on the image sensor 2 increases.

  That is, when the motion vector speed is 0 (pixel / s) or more and slower than 40 (pixel / s), pixel addition processing between frame images including the pixel is not performed. However, when the motion vector speed is 40 (pixels / s) or more, pixel addition processing is performed between frame images including the pixel.

  In this way, pixel addition processing between frame images is performed only on frame images that include pixels with a fast motion vector speed, thereby moving at high speed without degrading the image quality of image portions such as backgrounds with little motion. Afterimages on the subject can be removed.

  That is, when a scene in which the object shown in FIG. 10 moves from the left to the right in the figure is captured, an afterimage (overlapping of three images) is not observed as shown in FIG. As shown in FIG. 12, the image is blurred (blurred) and the afterimage becomes inconspicuous.

  Further, in conjunction with the pixel addition processing between the frame images, the electronic shutter speed control (and the gain or aperture control of the preamplifier 5) similar to that in the first embodiment is performed, so that the subject moving more effectively at high speed can be obtained. Afterimage can be removed.

  The flowchart of FIG. 5B shows a control procedure for pixel addition processing between the electronic shutter speed and the frame image performed by the system control unit 11.

  First, in step (denoted as S in the figure) 301, the system control unit 11 detects a motion vector from two frame images and calculates a motion vector speed.

  Next, in step 302, it is determined whether or not the motion vector speed V is 0 ≦ V <40 (pixels / s). If the motion vector speed V is within this range, the process proceeds to step S303, and the pixel addition process is not performed. If the motion vector speed V is not within this range, the process proceeds to step S304 to perform pixel addition processing between frame images.

  Further, after obtaining the motion vector speed in step 301, the process proceeds to step 102 shown in FIG. 3B in parallel with the processing in step 302 and subsequent steps, and the electronic shutter speed is controlled.

  Note that the number of frame images to be subjected to the pixel addition processing may be set to | 0.033 × H |

  In this embodiment, since H = 60 Hz, the number of frame images to be subjected to pixel addition processing is 1.98 frames or more, and at least two frames are added.

  In each of the above embodiments, the case where the motion vector is detected and the pixel addition process is performed with one frame image as one image unit is described. However, the motion vector is detected and the pixel addition process is performed with one field image as one image unit. You may do it.

1 is a block diagram showing a configuration of an image observation system that is Embodiment 1 of the present invention. FIG. 3 is an explanatory diagram regarding the angle of view, the horizontal resolution, and the movement speed of the subject in the head mounted display of the first embodiment. FIG. 6 is a diagram illustrating a relationship between a motion vector speed and an electronic shutter speed in the head mounted display according to the first embodiment. 5 is a flowchart illustrating a control procedure in the head mounted display according to the first embodiment. The block diagram which shows the structure of the head mounted display which is Example 2 of this invention. FIG. 10 is a diagram illustrating a relationship between a motion vector speed and the number of pixels on which image addition is performed in the head-mounted display of Example 2. 9 is a flowchart illustrating a control procedure in the head mounted display according to the second embodiment. FIG. 6 is a block diagram showing a configuration of a head mounted display that is Embodiment 3 of the present invention. 10 is a flowchart illustrating a control procedure in the head mounted display according to the third embodiment. The figure for demonstrating the subject of this invention. The figure for demonstrating the subject of this invention. The figure for demonstrating the subject of this invention. The figure for demonstrating the subject of this invention. The figure for demonstrating the effect of the Example of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging lens 2 Solid-state image sensor 3 Drive circuit 4 Pulse generation circuit 5 Preamplifier 6 Camera process circuit 7, 8 Frame memory 9 Motion vector detection circuit 10, 10 'Pixel addition circuit 11 System control part 15 Display element 16 Eyepiece optical system

Claims (17)

An imaging system including an imaging element and imaging a subject;
A display element for displaying an image generated based on an output signal from the imaging element;
An eyepiece optical system for observing an image displayed on the display element;
Movement detection means for detecting movement of a subject from the generated image;
An image observation apparatus comprising: a control unit that controls an electronic shutter speed of the image pickup device in accordance with a movement speed of a subject detected by the movement detection unit.   The image observation apparatus according to claim 1, wherein the control unit decreases the electronic shutter speed as the movement speed increases.   3. The image observation apparatus according to claim 1, wherein when the movement speed is higher than the first movement speed, the control unit decreases the electronic shutter speed as compared with a case where the movement speed is lower. A preamplifier for amplifying an output signal from the image sensor;
The image observation apparatus according to claim 1, wherein the control unit controls a gain of the preamplifier according to the movement speed.   The image observation apparatus according to claim 4, wherein the control unit decreases the gain as the moving speed increases.   The image observation apparatus according to claim 5, wherein the control unit lowers the gain when the movement speed is higher than the first movement speed as compared with a case where the movement speed is low. An imaging system including an imaging element and imaging a subject;
A display element for displaying an image generated based on an output signal from the imaging element;
An eyepiece optical system for observing an image displayed on the display element;
Movement detection means for detecting movement of a subject from the generated image;
Image processing means for performing pixel addition and smoothing processing in a pixel area including pixels in which movement of the subject is detected by the motion detection means in the generated image;
An image observation apparatus comprising: control means for changing the size of the pixel area in accordance with the movement speed of the subject detected by the movement detection unit. An imaging system including an imaging element and imaging a subject;
A display element for displaying an image generated based on an output signal from the imaging element;
An eyepiece optical system for observing an image displayed on the display element;
Movement detection means for detecting movement of a subject from the generated image;
Image processing means for performing pixel addition and smoothing processing in a pixel area including pixels in which movement of the subject is detected by the motion detection means in the generated image;
Control means for causing the image processing means to perform the pixel addition and smoothing processing in the pixel area when the motion speed of the subject detected by the motion detection unit is faster than the first motion speed. An image observation apparatus. An imaging system including an imaging element and imaging a subject;
A display element for displaying an image generated based on an output signal from the imaging element;
An eyepiece optical system for observing an image displayed on the display element;
Movement detection means for detecting movement of a subject from the generated image;
Image processing means for performing pixel addition processing on the generated image and a plurality of frame images;
An image observation apparatus that causes the image processing means to perform the pixel addition processing on the plurality of frame images when the motion speed of the subject detected by the motion detection unit is faster than a first motion speed. When the drive frequency of the image sensor is H hertz,
The image observation apparatus according to claim 9, wherein the image processing unit performs the pixel addition process on a frame image of | 0.033 × H | frames or more.   When the angle of view in the specific direction of the imaging system is φ ° and the resolution of the imaging element in the specific direction is ω pixels, and ω / (60 × φ)> 2, the first motion speed The image processing apparatus according to claim 3, 6, 8, or 9, wherein ω / (3 × φ) pixels / s.   When the angle of view in the specific direction of the imaging system is φ ° and the resolution of the imaging element in the specific direction is ω pixels, the first motion speed is obtained when ω / (60 × φ) ≦ 2. The image observation apparatus according to claim 3, wherein the image observation device is 40 pixels / s. An image observation apparatus according to any one of claims 1 to 12,
An image observation system comprising: an image supply device that supplies an image synthesized with an image generated based on an output signal from the image sensor to the image observation device. An imaging system that includes an imaging device and images a subject, a display device that displays an image generated based on an output signal from the imaging device, and an eyepiece optical system for observing the image displayed on the display device A method of controlling an image observation apparatus comprising:
A detection step of detecting a movement of a subject from the generated image;
And a step of controlling the electronic shutter speed of the image sensor in accordance with the movement speed of the subject detected in the detection step. An imaging system that includes an imaging device and images a subject, a display device that displays an image generated based on an output signal from the imaging device, and an eyepiece optical system for observing the image displayed on the display device A method of controlling an image observation apparatus comprising:
A detection step of detecting a movement of a subject from the generated image;
A process step of performing pixel addition and smoothing processing in a pixel area including a pixel in which movement of a subject is detected by the detection step in the generated image,
In the processing step, the size of the pixel area is changed in accordance with the movement speed of the subject detected in the detection step. An imaging system that includes an imaging device and images a subject, a display device that displays an image generated based on an output signal from the imaging device, and an eyepiece optical system for observing the image displayed on the display device A method of controlling an image observation apparatus comprising:
A detection step of detecting a movement of a subject from the generated image;
A process step of performing pixel addition and smoothing processing in a pixel area including a pixel in which movement of a subject is detected by the detection step in the generated image,
In the processing step, when the motion speed of the subject detected in the detection step is faster than the first motion speed, the pixel addition and smoothing processing is performed in the pixel area. Method. An imaging system that includes an imaging device and images a subject, a display device that displays an image generated based on an output signal from the imaging device, and an eyepiece optical system for observing the image displayed on the display device A method of controlling an image observation apparatus comprising:
A detection step of detecting a movement of a subject from the generated image;
And a processing step of performing pixel addition processing on the generated image and a plurality of frame images,
In the processing step, when the movement speed of the subject detected in the detection step is faster than the first movement speed, the pixel addition process is performed on the plurality of frame images. Control method.