JP6188326B2 - Image distortion correction method for 3D image display - Google Patents

Description

  The present invention relates to a three-dimensional display device, and more particularly to an image distortion correction method for displaying a geometrically accurate three-dimensional image.

  Patent Document 1 states that “a plurality of basic units that two-dimensionally scan a light beam and project it in the air are two-dimensionally arranged, and a predetermined image signal is input to each basic unit to cope with the movement of two-dimensional scanning. Thus, a three-dimensional image display device characterized in that the beam light source is brightness-modulated ”has been proposed, and it is possible to eliminate inevitable lens aberrations in integral photography using a conventional lens array. .

JP 2012-98341 A

  In Patent Document 1, since the two-dimensional scanning is performed with the movable mirror, the temperature characteristic and the change with time occur. When the temperature of each movable mirror rises with time, the spring constant and resonance frequency of the torsion spring supporting the movable mirror Changes, the direction of the light beam projected from each basic unit and the scanning angle change, and the three-dimensional image projected into the space is distorted.

  In the present invention, the invisible light beam is superimposed on the visible light beam of the basic unit that two-dimensionally scans the light beam and is projected in the air, and the cameras that can detect the invisible light are arranged in several places corresponding to the reference viewing area or the reference viewing area, A three-dimensional image display method that corrects image distortion by projecting invisible light images with different patterns for each frame from all basic units, and detecting distortion of the projected image with a camera. Therefore, it is intended to always provide a three-dimensional image display with little geometric distortion even when the scanning characteristic of the scanning mirror changes.

  In order to solve the above problems, the present invention provides the following means.

  A plurality of basic units that two-dimensionally scan the light beam emitted from the beam light source and project it into the air are arranged two-dimensionally, and a predetermined image signal is input to each basic unit, corresponding to the movement of the two-dimensional scanning. In the three-dimensional image display method for modulating the luminance of the beam light source, the invisible light beam is superimposed on the visible light beam of the basic unit, and cameras capable of detecting invisible light are arranged in several places corresponding to the reference viewing area or the reference viewing area. By projecting invisible light images with different patterns for each frame from the basic unit, and correcting the distortion of the image by detecting the distortion of the projected image with a camera, with a temperature change etc., Even if the scanning characteristics of the scanning mirror change, a three-dimensional image with little geometric distortion can always be displayed.

  In the present invention, the invisible light beam is superimposed on the visible light beam of the basic unit that two-dimensionally scans the light beam and is projected in the air, and the cameras that can detect the invisible light are arranged in several places corresponding to the reference viewing area or the reference viewing area, By projecting invisible light images with different patterns for each frame from all basic units and detecting the distortion of the projected image with a camera, the distortion of the image is corrected in real time during the display of the three-dimensional image. Thus, even if the scanning characteristics of the scanning mirror change, it is possible to always realize a three-dimensional image display with little geometric distortion.

The figure which shows the basic composition of the three-dimensional image display method of patent document 1 The figure which shows the structural example of the basic unit group of patent document 1 The figure which shows the structural example of the basic unit of patent document 1 The figure which shows the other structural example of the basic unit of patent document 1 The figure which shows the structural example of the basic unit used for this invention The figure which shows the other structural example of the basic unit used for this invention Basic configuration diagram of the present invention The figure which shows the projection image group from the basic unit group in the standard viewing zone The figure explaining the temperature change of the projection image from a basic unit The figure explaining the pixel which changes the invisible light image to superimpose The figure explaining the example of the time change of the invisible light image to superimpose The figure explaining the example of distortion of the image projected from one basic unit in a standard viewing zone at a certain temperature The figure explaining the example of the positional relationship between the camera arrange | positioned at the four corners of a reference | standard visual field at the time of a certain temperature, and the pixel of the image projected from one basic unit. The figure explaining the example of distortion correction of the image projected from a basic unit based on the distortion of the image detected with the camera arrange | positioned in the reference | standard visual field at a certain temperature The figure explaining the example of the positional change of the camera arrange | positioned at the four corners of a reference | standard viewing zone, and the pixel of the image which temperature changes and is projected from one basic unit. Explanatory drawing of the example of distortion correction of the image projected from a basic unit based on the change of the distortion of the image detected with the camera which temperature changed and was arrange | positioned in the reference | standard viewing zone The figure explaining the other example of arrangement | sequence of the pixel which changes the invisible light image to superimpose Diagram showing an example in which a half mirror and a reduction optical system are provided between light beams projected from the basic unit group to the reference viewing area, and the same information is detected as when cameras are arranged at the four corners of the reference viewing area. The figure which shows the other example which detects the same information as the case where the concave mirror of a half mirror is provided between the light ray groups projected from the basic unit group to the reference viewing area, and the cameras are arranged at the four corners of the reference viewing area

  Hereinafter, embodiments of the present invention will be described in detail.

  1 to 4 are diagrams showing an outline of Patent Document 1 as a conventional example.

  FIG. 1 is a diagram showing an example of a basic configuration. 8a, 8b, 8c,... Are basic units, 9a,... Are laser diodes, 10a,. ... Are biaxial scanning mirrors.

The light beams emitted from the laser diodes 9a... Pass through the beam shapers 10a... To form a sharp light beam that tends to spread in proportion to the distance, and are biaxially scanned through the fixed mirror 11a. It is incident on the mirror 12a.
3′a, 3′b, 3′c... Are image signals supplied to the basic units, and G3 ′ is a signal group supplied to all the basic unit groups.

  What is important here is that, for example, a three-dimensional image of 7 is displayed without distortion, the light rays representing the points (A, B, C...) Of the three-dimensional image are predetermined points. It is that it concentrates on.

  FIG. 2 is a perspective view of a configuration example of the same three-dimensional display device. Basic units 8a, 8b, 8c,... It is the whole basic unit group arranged two-dimensionally.

  FIG. 3 is a configuration example of a basic unit capable of color display. Light emitted from the red laser diode 9R, the green laser diode 9G, and the blue laser diode 9B is approximately distanced through the beam forming machines 10R, 10G, and 10B. A sharp light beam that tends to spread proportionally is combined into one light beam by the dichroic prism 13, enters the biaxial scanning mirror 12 via the fixed mirror 11, and the image is projected into the air by its reflection.

  FIG. 4 shows another configuration example of the basic unit capable of color display, in which 9R is a red laser diode, 9G is a green laser diode, and 9B is a blue laser diode, respectively, for the single mode optical fibers 14R, 14G, and 14B. Are combined. Since the core diameter of the single mode optical fiber is about the wavelength, the core portion of the end face becomes a point light source of each color. Further, after the light beams tend to spread almost in proportion to the distance through the convex lenses (collimator lenses 15R, 15G, 15B) focused on the color point light sources, the optical axes of the color light beams become parallel. Line up as close as possible.

  The light beams whose optical axes are parallel and close to each other are incident on the biaxial scanning mirror 12 via the fixed mirror 11 and are reflected to project an image in the air.

  Here, since each color light beam tends to spread, the light beam of each color becomes thick in the viewing zone at a distant position, and the region where each color is mixed becomes most of the light beam. For an observer, it is possible to observe light substantially equivalent to the light beam shown in FIG.

  FIG. 5 is a diagram showing an embodiment of the basic unit in the present invention.

  In addition to the red laser diode 9R, the green laser diode 9G, and the blue laser diode 9B, which are also in the conventional example, the light emitted from the infrared laser diode 9-IR is used as a beam forming machine 10R, 10G, 10B, 10-IR. Through which the light beam spreads in proportion to the distance, and is combined into a single light beam by the dichroic prism 13, enters the biaxial scanning mirror 12 through the fixed mirror 11, and the reflection reflects an image in the air. Project.

  The light emitted from the infrared laser diode 9-IR is projected to the viewing zone together with the visible light, but is invisible to humans, so even if it is modulated with any image signal, I can't feel it.

  FIG. 6 is a diagram showing another embodiment of the basic unit in the present invention.

  In addition to the red laser diode 9R, the green laser diode 9G, and the blue laser diode 9B that are also in the conventional example, the infrared laser diode 9-IR is coupled to the single mode optical fibers 14R, 14G, 14B, and 14-IR. ing. Since the core diameter of the single mode optical fiber is about the wavelength, the core portion of the end face becomes a point light source of each color. Further, after forming a sharp light beam that tends to spread in proportion to the distance via a convex lens (collimator lenses 15R, 15G, 15B, 15-IR) focused on each color point light source, the optical axis of each color light beam Line up close to each other in parallel.

  The light beams whose optical axes are parallel and close to each other are incident on the biaxial scanning mirror 12 via the fixed mirror 11 and are reflected to project an image in the air.

  In this embodiment as well, the light emitted from the infrared laser diode 9-IR is projected toward the viewing zone together with the visible light. However, since it is invisible to humans, it can be viewed with any image signal. It is not felt by human beings in the area.

  FIG. 7 is a diagram showing an embodiment of a method for detecting in real time the distortion of images projected from all the basic units, which is the key of the present invention.

  Here, G8 is a basic unit group in which the basic units 8a, 8b, 8c, 8d,... Are arranged two-dimensionally. In the four corners of the reference viewing zone 16, there is almost no sensitivity to visible light, and the infrared laser diode 9- Cameras 17a, 17b, 17c, and 17d that are sensitive to the wavelength of light from IR are arranged, and infrared images emitted from all the basic units are monitored.

  The observer places his eyes on the reference viewing zone 16 and observes the light beam from the basic unit group G8.

  As a matter of course, the observer recognizes only visible light among the light rays projected from the basic unit group G8, and cannot recognize infrared rays which are invisible light.

  FIG. 8 shows an example of image ranges (scanning ranges) 18a, 18b, 18c, 18d... Projected from the respective basic units 8a, 8b, 8c, 8d. In the figure, there are generally variations due to individual differences in basic units.

  FIG. 9 shows an example of the temperature change of the image range (scanning range) projected from one basic unit 8a. The image range (scanning range) 18a-1 projected at a certain temperature is a temperature change. The spring constant and resonance frequency of the torsion spring supporting the movable mirror change, and the image range (scanning range) projected at another temperature is, for example, 18a-2, and the size and position of the scanning range change. .

  This temperature change is caused by turning on the power of the three-dimensional image display device using the basic unit group G8 and supplying driving power to the laser diode group and the biaxial scanning mirror group of the basic unit group G8. This is due to the temperature rise of the basic unit group.

  In addition, the basic unit in the peripheral part has good heat dissipation and the temperature rise is small, but the central unit and the basic unit in the upper part tend to have a high temperature rise due to poor heat dissipation or the influence of the heat of other basic units, Since the temperature rise of each basic unit varies individually, it must be individually compensated.

  FIG. 10 shows an example of the order of pixels to be blinked in the projected image 3 ′ by infrared rays superimposed on the visible light, and FIG. 11 shows an example of the change pattern.

  The infrared image pattern change shown in FIG. 11 sequentially turns on the pixels with the numbers shown in FIG. 10 and increases the number of lit pixels for each frame. Returning to lighting, the pixels are sequentially lit, and the number of lit pixels is repeatedly increased for each frame (the pixels painted in black indicate that they are lit).

  Since the infrared image that changes from frame to frame is not visible to the observer of the three-dimensional image, it does not interfere with the observation of the three-dimensional image.

  The infrared image 3 ′ in which the order of pixels to be lit is determined as shown in FIG. 10 is distorted, for example, as an image 18-a shown in FIG. 12 on the reference viewing zone surface.

  FIG. 13 is a diagram for explaining a distortion detection method for such a distorted image. When the distorted infrared image 18a-1 is measured by the cameras 17a, 17b, 17c, and 17d arranged at the four corners of the reference viewing zone, In the camera, infrared rays are not detected from the 00th frame to the 06th frame, and infrared rays are detected from the 07th frame to the 25th frame.

  Similarly, in the camera of 17b, infrared rays are not detected from the 00th frame to the 16th frame, infrared rays are detected from the 17th frame to the 25th frame, and in the 17c camera, infrared rays are not detected in the 00th frame. Infrared light is detected from the 02nd frame to the 25th frame. In the 17d camera, infrared light is not detected from the 00th frame to the 07th frame, and infrared light is detected from the 08th frame to the 25th frame. .

  FIG. 14 is a diagram showing a method for correcting distortion using information on image distortion detected by the method shown in FIG. 13, and the first detected infrared ray of each camera is the image scanning range 19a-1. The drive signal of the biaxial scanning mirror is corrected so as to be at the four corners.

  As a result, the four corners of the image projected from the basic unit 8a coincide with the positions of the cameras 17a, 17b, 17c, and 17d in the reference viewing zone 16, and there is no original distortion as the projected image.

  FIG. 15 is a diagram illustrating a state in which the temperature has changed from the state of FIG. 13 and the distortion of the image has changed. The distorted infrared image 18a-2 is captured by cameras 17a, 17b, 17c, and 17d arranged at the four corners of the reference viewing zone. When measured, the camera of 17a does not detect infrared rays from the 00th frame to the 07th frame, and detects infrared rays between the 08th frame and the 25th frame.

  Similarly, in the camera of 17b, infrared rays are not detected from the 00th frame to the 17th frame, infrared rays are detected from the 18th frame to the 25th frame, and in the camera of 17c, the infrared rays are detected from the 00th frame to the 07th frame. Infrared light is not detected, infrared light is detected between the 08th frame and 25th frame, and the 17d camera does not detect infrared light from the 00th frame to the 12th frame, and between the 13th frame and the 25th frame. Infrared is detected.

  FIG. 16 is a diagram showing a method for correcting distortion using information on image distortion detected by the method shown in FIG. 15. As in the case of FIG. 14, the pixels where infrared rays are first detected by each camera are shown. The drive signals for the biaxial scanning mirror are corrected so as to be at the four corners of the image scanning range 19a-2.

  As a result, even when the temperature changes and the size and position of the projected image change, the four corners of the image projected from the basic unit 8a are at the positions of the cameras 17a, 17b, 17c, and 17d in the reference viewing zone 16. The projection image matches the original distortion.

  As shown in FIGS. 14 and 16, in the corrected state, the cameras 17a, 17b, 17c, and 17d do not detect infrared rays in the 00th frame, but detect infrared rays between the 00th frame and the 25th frame. It is a state to be done.

  That is, the image correction according to the present invention is such that the cameras 17a, 17b, 17c, and 17d do not detect infrared rays in the 00th frame and detect infrared rays between the 00th frame and the 25th frame. It can also be explained that the servo is applied to correct the image.

  Here, the cause of the change in image distortion is mainly a temperature change, so it does not change suddenly in several frames, but changes in units of minutes. The detection is a sufficiently fast detection interval, and the image change in minutes is corrected without substantial delay.

  Since the basic units 8a, 8b, 8c, 8d,... Have different temperature characteristics, it is necessary to detect the image distortion of all the basic units independently, but this is realized with the configuration shown in FIG. The following is described.

  In FIG. 7, light beams incident on the camera 17 a from the basic units 8 a, 8 b, 8 c, 8 d... Have different arrival directions, and therefore enter different pixels of the image sensor of the camera 17 a.

  As a result, which basic unit image distortion is detected can be determined by paying attention to the output of the corresponding predetermined pixel, and all the basic units can be detected independently as different pixels.

  Similarly, in the other cameras 17b, 17c, and 17d, the light beams from the basic units 8a, 8b, 8c, 8d,... It can be detected.

  Eventually, the light beam from which basic unit is identified as the pixel position of the image sensor, and the image distortion of each basic unit is identified as a time signal with a period of 26 frames. Can be corrected independently.

  The most characteristic feature of the image distortion correction method of the present invention is that the image distortion of all the basic units is always corrected in real time in the 3D image, so that the 3D image display with less distortion is always maintained and the geometric distortion is maintained. In other words, an accurate three-dimensional image is stably displayed.

  FIG. 17 is a diagram showing another embodiment of the order of the lit pixels of the infrared image. If the pixels are lit in the numerical order, the reference frame from the first frame in which infrared is detected by the camera during the 26 frame period is used. It does not change that the pixel position to be arranged at the corner of the viewing zone can be detected.

  The pixel lighting order is not limited to FIGS. 10 and 17 as long as distortion can be detected.

  Further, in the embodiment in which the number of pixels for sequentially turning on infrared rays is increased, image distortion can be detected by a lighting method in which only the pixel of the frame number is lit in each frame, and the pixel for each frame. If the distortion of the image can be detected, for example, it can be detected by a method of turning off the light, the lighting sequence is arbitrary.

In the embodiment, infrared rays have been described as invisible light rays. However, since similar effects can be obtained even when ultraviolet rays are used, any projection light for image distortion detection may be used as long as it is invisible light rays.

  FIG. 18 is a diagram showing an outline of the second embodiment of the present invention. G8 is a basic unit group, 16 is a reference viewing zone, 20 is a flat half mirror (beam splitter), 21 is a convex lens, and 16 ′ is a half mirror. In the area where the same light rays as the reference viewing area formed by the reduction optical system 20 and the convex lens 21 reach, cameras 17a to 17d are arranged at the four corners of the area, and signals similar to those in the first embodiment shown in FIG. Get.

  With this configuration, there is an advantage that the camera is not arranged in the reference viewing zone where the observer is present and does not interfere with the observation of the three-dimensional image.

  FIG. 19 is a diagram showing an outline of the third embodiment of the present invention. G8 is a basic unit group, 16 is a reference viewing zone, 22 is a concave half mirror (beam splitter), and 16 ′ is a concave half mirror 22. In the area where the same light beam as the reference viewing zone formed by the reduction optical system reaches, cameras 17a to 17d are arranged at the four corners of the area, and the same signals as in the embodiment of FIG. 18 are obtained.

  With this configuration, as in the second embodiment of FIG. 18, the camera is not arranged in the reference viewing area where the observer is present, and there is an advantage that the three-dimensional image is not obstructed.

  In the embodiments shown in FIGS. 7, 18 and 19, the case of four cameras has been described. However, the image distortion due to the biaxial scanning temperature of all the basic units is measured in advance, and the change is stable. In such a case, one to three cameras may be arranged at representative positions where distortion can be easily detected.

  In the three-dimensional image display apparatus according to the present invention, image distortion is always detected and corrected for all the basic units while the image is displayed, so that a geometrically accurate three-dimensional image can always be displayed. . This feature is particularly valuable when used for three-dimensional images for surgical navigation where position accuracy is required.

3 'Image signal input to basic unit 3'-2 Other examples of image pixel arrangement changed for each frame of image signal input to invisible laser of basic unit 3'a, 3'b, 3' c: Image signal 7 input to each basic unit 7 Three-dimensional image 8 Basic units 8a, 8b, 8c ... Each basic unit 9 Laser diode 9a ... Each laser diode 9B Blue laser diode 9G Green laser diode 9R Red laser diode 9-IR Invisible laser diode 10 Beam shaper 10a ... Each beam shaper 10B Blue laser diode beam shaper 10G Green laser diode beam shaper 10R Red laser diode beam shaper 10-IR Beam shaper 11 for invisible laser diodes Fixed mirror 1a ... each fixed mirror 12 biaxial scanning mirror 12a ... each biaxial scanning mirror 13 dichroic prism 14B single mode optical fiber 14G for blue light single mode optical fiber 14R for green light single mode optical fiber 14 for red light IR invisible light single mode optical fiber 15B Blue light collimating lens 15G Green light collimating lens 15R Red light collimating lens 15-IR Invisible light collimating lens 16 Reference viewing zone 16 'Area where the same rays as the reference viewing zone reach 17a, 17b ... Invisible light cameras 18a, 18b, 18c ... Image range 18a-1 projected from the basic units 8a, 8b, 8c ... on the surface of the reference viewing zone Reference at a certain temperature Range of images projected from the basic unit 8a in the plane of the viewing zone 18a-2 Range of image 19a-1 projected from base unit 8a in the plane of the reference viewing zone at another temperature Corrected image projected from base unit 8a in the plane of the reference viewing zone at a certain temperature The range of the corrected image projected from the basic unit 8a in the plane of the reference viewing zone at another temperature 20 plane half mirror 21 convex lens 22 concave half mirror A first vertex of a three-dimensional image B Second vertex of a 3D image C Third vertex G3 ′ of a 3D image Image signal group G8 basic unit group input to all basic units

Claims (3)

A plurality of basic units for two-dimensionally scanning a light beam emitted from a monochromatic or multi-color beam light source and projecting it in the air are arranged two-dimensionally,
In a three-dimensional image display method for inputting a predetermined image signal to each of the basic units and modulating the intensity of the beam light source in response to a two-dimensional scanning movement,
As the beam light source, a beam of invisible light other than visible light is superimposed, and an image by invisible light is projected in the air,
Change the pattern of the projected image by invisible rays for each frame,
With a camera that can detect the invisible light, arranged in one to several places in the reference viewing area, it detects distortions in the projected images from all the projection basic units,
Two-dimensional scanning of a beam with visible light and invisible light superimposed, or by deforming the image projected with a beam of visible light and invisible light so that the projected image with invisible light is sufficiently distorted in the viewing zone An image distortion correction method for three-dimensional image display, characterized in that the shape of the image is changed. A plurality of basic units for two-dimensionally scanning a light beam emitted from a monochromatic or multi-color beam light source and projecting it in the air are arranged two-dimensionally,
In a three-dimensional image display method for inputting a predetermined image signal to each of the basic units and modulating the intensity of the beam light source in response to a two-dimensional scanning movement,
As the beam light source, a beam of invisible light other than visible light is superimposed, and an image by invisible light is projected in the air,
Change the pattern of the projected image by invisible rays for each frame,
A reduction optical system using a half mirror is provided between a plurality of two-dimensionally arranged basic unit groups and a reference viewing area, and from one location corresponding to the reference viewing area reduced by the reduction optical system With cameras that can detect the invisible light arranged in several places, the distortion of the projected images from all the projection basic units is detected,
Two-dimensional scanning of a beam with visible light and invisible light superimposed, or by deforming the image projected with a beam of visible light and invisible light so that the projected image with invisible light is sufficiently distorted in the viewing zone An image distortion correction method for three-dimensional image display, characterized in that the shape of the image is changed. Using infrared as invisible light, three-dimensional image display image distortion correction method of claim 1 or claim 2, characterized in that the infrared detecting using a camera as possible.

Images