WO2019098265A1

WO2019098265A1 - Color vision support device

Info

Publication number: WO2019098265A1
Application number: PCT/JP2018/042261
Authority: WO
Inventors: 勇二関口; 木村　浩; 洋一藤岡; 悟須藤; 貴仁日高; 岩井　順一
Original assignee: 株式会社テレパシージャパン
Priority date: 2017-11-17
Filing date: 2018-11-15
Publication date: 2019-05-23
Also published as: JP7011295B2; JP2019096939A

Abstract

[Problem] To convert an image captured by a camera into a hue that is easy for an observer to discriminate, without degrading the image quality of the captured image. [Solution] A color vision support device 100 includes a camera 10 that captures an image of the outside world to generate image data, a display optical system 20 that emits image light corresponding to the image data, and an optical system 30 that guides the image light to a pupil of the observer. The display optical system 20 includes a light source unit 22 including a plurality of light emitting elements 21 having different emission wavelengths, an image element 25 converting irradiation light from the light source unit 22 into image light, and a control circuit 26 that independently controls light quantities of the plurality of light emitting elements 21. The display optical system outputs the image light in which the image data are converted into a hue that is easy for an observer to discriminate. Since the gradation of each primary color can be maintained by increasing or decreasing the light quantity of the primary color in the light source unit 22, it is possible to avoid degradation of image quality of the captured image.

Description

Color vision support device

The present invention relates to an eyepiece type color vision assistance device mounted on a head mounted display (HMD). Specifically, the color vision support device of the present invention is an optical device installed in front of the observer's eyes, and converts the image data taken by the camera into a hue that is easy for the observer to distinguish in color. The projection assists the observer's color vision.

It is called "colorless" that the appearance and feeling of color are different from those of general color vision users. In the human retina, cones having high sensitivity to light near long wavelength (565 nm), middle wavelength (545 nm) and short wavelength (440 nm) exist, respectively, and L cone, M cone and It is called S cone. L cones (red cones) mainly sense yellowish green to red light, M cones (green cones) mainly sense green to yellowish green light, S cones (blue cones) purple I mainly feel the blue light. Due to the difference in spectral sensitivities of these three types of cones (photocells), L cones mainly feel yellowish green to red, M cones mainly feel green to yellowish light, and S cones mainly I feel mainly purple to blue light. Then, when light enters the pupil, these cones react, the information is transmitted from the retina to the optic nerve and carried to the visual center of the cerebral cortex, and color vision corresponding to each wavelength of visible light (400 to 800 nm) occurs It is said that. The color-impaired person will have a slightly different color sensation from the normal one, for example, due to the weakness or lack of any of the L-, M-, and S-cone senses.

The type of color blindness is broadly divided into "type 1 color vision" in which the sense of the L cone is weak or missing, "type 2 color vision" in which the sense in the M cone is weak or missing, and S cone It is said that there is a "type 3 color vision" with weak or missing sensation. People with color weaknesses with such characteristics may find it difficult to distinguish between two adjacent colors (such as red and green).

As a device for assisting the color vision of such a color-impaired person, for example, Patent Document 1 discloses a head-mounted display type visual aid device. The visual aid device of Patent Document 1 stores in advance the hue at which the viewer's color weakness occurs, and when the hue at which the color weakness occurs is detected from the image data acquired by the camera, for example, The image data is converted to a hue that is easy for the observer to distinguish by making the light amount larger than the light amount of other hues and outputting.

JP, 2016-95577, A

By the way, a general LED is used as a light source of the visual aid device of patent document 1, and it is supposed that the irradiation light from a light source will be introduce | transduced into a light transmissive liquid crystal display, and will be converted into image light. A general LED emits white light. For example, a red, green, or blue light emitting diode emits light with the same light amount to generate white light (multichip), or a blue light emitting diode emits yellow fluorescence. One that produces white light by coating with a body (single chip) is known. In order to generate image light by introducing irradiation light from such a white LED to a light transmission type liquid crystal display, for example, the red light amount is increased more than green and blue for color-weakened people with poor red sensitivity. In this case, it is necessary to decrease the transmission amount (light amount) of green and blue in the liquid crystal display to make the red light amount relatively strong. That is, when a general LED is used as a light source, only white light is provided from the light emission source, so in order to adjust the hue of the image light for the color impaired, the transmission amount of each hue in the liquid crystal display There is no way but to adjust the However, if the amount of light of another hue is decreased to relatively increase the amount of light of a specific hue, the gradation of the other hues is degraded, and a false contour occurs in the image light and the image quality There was a problem of causing a decline.

Therefore, an object of the present invention is to provide a color vision support device capable of converting a captured image into a hue that is easy for the observer to discriminate colors without degrading the image quality of the captured image captured by a camera.

The inventor of the present invention has intensively studied the means for solving the problems of the prior art and, as a result, constitutes a light source with a plurality of light emitting elements having different light emitting colors and enables independently controlling the light quantity of each light emitting element In the light source, after increasing the light quantity of the hue that the observer can not distinguish in color, introducing the irradiation light from the light source to the image element such as a liquid crystal panel, the image light is not affected by the image quality We have found that we can project The present inventors considered that the problems of the prior art could be solved based on the above findings, and completed the present invention. Specifically, the present invention has the following configuration.

The present invention relates to a color vision support apparatus. A color vision support apparatus according to the present invention includes a camera 10 for photographing the outside world to generate image data, a display optical system 20 for emitting an image light corresponding to the image data, and guiding the image light to the observer's pupil And an eyepiece optical system 30. The display optical system 20 includes a light source unit 22, an image device 25, and a control circuit 26. The light source unit 22 includes a plurality of light emitting elements 21 having different emission wavelengths. In addition, although the example of the some light emitting element 21 is a light emitting diode which light-emits in three primary colors of red, green, and blue, it may be a laser. The light emission color of the light emitting element 21 is not limited to these, and for example, adjustment such as changing red to orange may be performed according to the observer's color vision. The video element 25 converts the irradiation light from the light source unit 22 into video light. The control circuit 26 controls the light amounts of the plurality of light emitting elements 21 independently. The display optical system 20 can output image light obtained by converting the image data into a hue that can be easily discriminated by the observer.

As in the above configuration, by enabling the light amount of each light emitting element 21 of the light source unit 22 to be adjusted independently, for example, the red light amount is increased more than green and blue for color weak who is poor in red sensitivity. In this case, the light amount of red in the light source unit 22 may be increased. As described above, in the prior art, it is necessary to relatively increase the amount of red light by reducing the transmission amount (light amount) of green and blue in the picture element 25 such as a liquid crystal panel. It is possible to intensify the red light amount without reducing the green and blue light amounts. Therefore, according to the present invention, it is possible to project image light onto the observer's pupil without degrading the image quality.

In the present invention, it is preferable that the control circuit 26 performs time division driving in which the plurality of light emitting elements 21 emit light sequentially. That is, time-division driving is time-divided by the observer by turning off the other light emitting elements 21 at the timing when one light emitting element 21 is lit and repeatedly performing this for each light emitting element 21 at high speed. In this driving method, the afterimage of image light is visually recognized and each image light is combined in the brain of the observer. The light source of the conventional color vision support device always lights all the light emitting elements (light emitting diodes) to emit white light, and transmits white light from the light source through the liquid crystal panel provided with the color filter. It was common to add color to the light. On the other hand, by driving each light emitting element 21 by time division as in the present invention, substantially to the eye of the observer (specifically, L cone, M cone, S cone of retina) The light emission color of each light emitting element 21 is separately projected. For this reason, the hue of the image light is not mixed at the stage of being projected onto the eye of the observer, so that it is easy for the color-weaker to perceive the image light.

In the present invention, the light source unit 22 may include four or more light emitting elements 21 having different emission wavelengths. In this case, it is preferable that the control circuit 26 select and drive at least three of the light emitting elements 21. Thus, for example, in addition to the light emitting elements 21 for general three primary colors (red, green, blue) for normal persons, the light emitting elements 21 for weakly colored persons are provided, and the light emitting elements 21 used as needed. Can be implemented to realize a general-purpose video display device that can be used by various people.

In the present invention, the light source unit 22 may be configured to physically exchange the first light source unit 41 and the second light source unit 42. The first light source unit 41 includes a plurality of light emitting elements 21 having different emission wavelengths. On the other hand, the second light source unit 42 includes at least one or more light emitting elements 21 having different emission wavelengths and at least one light emitting element 21 included in the first light source unit 41. By preparing such light source units 41 and 42 and attaching them to the color vision support apparatus as needed or replacing the unit itself, various persons including normal persons and persons with weak colors can use the present invention It will be.

In the present invention, it is preferable that the eyepiece optical system 30 has a prism 32 which reflects the image light and guides it to the pupil of the observer, and the back surface side of the reflection surface 32a of the prism 32 is shielded. If the prism 32 is translucent, the observer will visually recognize the background behind the prism 32, and even if the image light whose hue is adjusted is provided to the observer, the image light and the background are The effect of color vision assistance is halved because it will appear to overlap. Therefore, as described above, by making the back surface of the reflecting surface 32a of the prism 32 substantially in the light shielding state, the observer can effectively view the image light whose hue is adjusted.

In the present invention, the camera 10 preferably includes an autofocus sensor 11 for detecting the distance between the photographing lens and the subject. In this case, the display optical system 20 preferably has a plurality of modes for the method of converting the hue of the image data, and determines the mode in accordance with the distance detected by the autofocus sensor 11. With the mode of the hue conversion method, for example, the degree to which the light amount of the specific light emitting element 21 is increased may be divided into a plurality of modes. Also, white balance can be achieved by changing only the RGB ratio (see Fig. 7), changing the position of the RGB primary color chromaticity points (see Fig. 8), or changing the position of the RGB primary color chromaticity points and adjusting the ratio. A mode (see FIG. 10) to maintain (W / H) may be prepared. For example, change the mode of the hue conversion method depending on whether the observer is looking at a distant view or the task of making a color judgment at hand such as harvesting vegetables or connecting electrical wiring etc. Can provide appropriate color vision support depending on the situation.

In the present invention, the camera 10 preferably has an autofocus sensor 11 for detecting the distance between the lens and the subject. In this case, the control circuit 26 generates display image data obtained by cutting out a portion of the image data generated by the camera 10 based on the distance detected by the autofocus sensor 11, and displays the image for displaying the display image data. A signal may be sent to the light source unit 22 and the video device 25. By appropriately adjusting the image range extracted from the image data and made to be visible to the observer according to the distance to the object at which the observer is gazing, the image provided to the observer and the surrounding scenery are seamless. Will be connected to

According to the color vision aiding apparatus of the present invention, it is possible to convert the photographed image into a hue which is easy for the observer to discriminate the color without deteriorating the image quality of the photographed image photographed by the camera.

FIG. 1 shows an example of the appearance of a color vision assistance device (head mounted display type) according to the present invention. FIG. 2 is a block diagram showing a functional configuration of the color vision support apparatus. FIG. 3 is a view for explaining the effect of the color vision aiding apparatus according to the present invention, in which (a) is a normal display state using white light, and (b) a state in which a specific primary color is extracted from white light. (Conventional example) is shown. FIG. 4 is a diagram for explaining the effect of the color vision aiding apparatus according to the present invention, and (c) shows a state (in the present invention) in which the light quantity of a specific primary color is increased in the light emitting portion. FIG. 5 shows the wavelength sensitivity characteristics of the cone. FIG. 6 shows mixed color lines of types 1 2 3 3 weak. FIG. 7 shows the color reproduction range and the white chromaticity point when the primary color point intensity ratio is changed. FIG. 8 shows the color reproduction range and the white chromaticity point when the primary color chromaticity point is changed. FIG. 9 shows the effect when the primary color chromaticity point is changed, using a mixed color line of type 1 weak. FIG. 10 shows the color reproduction range and the color chromaticity point in the case of performing W / B adjustment after changing the primary color chromaticity point. FIG. 11 shows the change of the spectrum for changing the primary color chromaticity point in the light source. FIG. 12 shows an example of time-division driving of light emitting elements. FIG. 13 shows the configuration of a display optical system provided with a plurality of light sources. FIG. 14 shows the configuration of a display optical system in which the light source unit can be replaced. FIG. 15 shows an example of a method of generating display image data using autofocus. FIG. 16 shows an example of a method of generating display image data using autofocus. FIG. 17 schematically shows an example in which a display image is shown superimposed on a landscape.

Hereinafter, an embodiment of the present invention will be described using the drawings. The present invention is not limited to the embodiments described below, and includes those appropriately modified by the person skilled in the art from the following embodiments within the obvious scope.

FIG. 1 is an external perspective view showing an example of a head mounted display (HMD) type color vision assistance apparatus 100. The HMD type color vision assistance apparatus 100 is mounted on the head of the observer and projects an image on its pupil. In the present embodiment, the color vision support apparatus 100 is disposed only in front of the eye of one eye of the observer. The color vision support apparatus 100 basically includes a camera 10, a display optical system 20, and an eyepiece optical system 30. The camera 10 captures an external world in the same direction as the line of sight of the observer and generates the image data. The display optical system 20 subjects the image data captured by the camera 10 to predetermined color tone correction for color vision support, generates image light of the image data after the correction, and emits the light toward the eyepiece optical system 30. . The eyepiece optical system 30 reflects or refracts the image light emitted from the display optical system 20 and guides it to the pupil of the observer. In the example shown in FIG. 1, the image light emitted from the display optical system 20 in the lateral direction (X-axis direction) is reflected approximately at right angles by the eyepiece optical system 30 and travels in the depth direction (Z-axis direction) It enters the pupil of the observer. The space between the display optical system 20 and the eyepiece optical system 30 is a space, and the image light travels in the air.

In the embodiment shown in FIG. 1, the eyepiece optical system 30 has a configuration in which a prism 32 is disposed on a transparent substrate 31. The transparent substrate 31 is a plate-like member facing the observer's pupil, and at least a part thereof is made of a transparent member or a mesh member which transmits light. In the example shown in FIG. 1, the transparent substrate 31 is a transparent substrate formed entirely of a transparent member such as plastic or glass. Also, a prism 32 is provided on the outer surface of the transparent substrate 31. Further, the transparent substrate 31 is fixed to the housing of the HMD that accommodates the display optical system 20. Thus, the transparent substrate 31 functions as a support for positioning the prism 32 on the optical axis of the image light emitted from the display optical system 20. Since the transparent substrate 31 is transparent, it is possible to ensure a wide view of the observer who visually recognizes the image. In addition, the observer can visually recognize the background on the back side of the transparent substrate 31 together with the image. On the other hand, the reflective surface 32b of the prism 32 is coated with a light-shielding paint or the like so that the viewer can not visually recognize the background behind the prism 32. Therefore, for the observer, the background overlapping with the prism 32 is replaced with the image generated in the display optical system 20.

In addition, in the example shown in FIG. 1, the HMD type color vision assistance apparatus 100 may be equipped with an optical sensor, a touch panel, etc. other than the above-mentioned structure. The housing of the HMD contains a CPU, memory, various communication devices, an acceleration sensor, a gyro sensor, and a battery. The structure of HMD can employ | adopt a well-known structure suitably, and it does not specifically limit in this invention.

Subsequently, an example of an optical system provided in the color vision aiding device 100 will be described with reference to FIG. In particular, FIG. 2 shows an optical system provided with a transmission type image device (liquid crystal panel) 25. FIG. 2 is a drawing of the structure of the color vision assistance device 100 from a plane (XZ plane). As shown in FIG. 2, the color vision support apparatus 100 includes a camera 10, a display optical system 20 for generating and emitting an image light (one-dot chain line), and an eyepiece optical system 30 for guiding the image light to the observer's pupil. Equipped with Image data acquired by the camera 10 is corrected by the display optical system 20 and emitted as an image light, propagates through the air and is incident on the eyepiece optical system 30, and finally to the pupil E of the observer It is projected.

The camera 10 is an imaging device for acquiring image data of a still image or a moving image. The camera 10 is configured of, for example, a photographing lens, a mechanical shutter, a shutter driver, a photoelectric conversion element such as a CCD image sensor unit, a digital signal processor (DSP) that reads charge amount from the photoelectric conversion element and generates image data, and an IC memory Be done. The camera 10 also includes an autofocus sensor (AF sensor) for measuring the distance from the photographing lens to the subject, and a mechanism for adjusting the focal length of the photographing lens according to the distance detected by the AF sensor. Is preferred. The type of AF sensor is not particularly limited, but a known passive method such as a phase difference sensor or a contrast sensor may be used. In addition, as an AF sensor, it is possible to use an active type sensor which directs infrared rays or ultrasonic waves to a subject and receives the reflected light or the reflected wave. The image data acquired by the camera 10 is supplied to the control circuit 26, and after predetermined image processing is performed, image light corresponding to the image data is generated and output by the display optical system 20.

In the embodiment shown in FIG. 2, the display optical system 20 includes a light source unit 22 including a plurality of light emitting elements 21, an equalizing element (integrator) 23, a condenser lens 24, an image element 25, and a control circuit. And 26.

The light source unit 22 is configured to include a plurality of light emitting elements 21 having different emission wavelengths (that is, emission colors). An example of the light emitting element 21 is an LED (light emitting diode). The light emitting element 21 is provided with one capable of emitting light of three primary colors of, for example, red (R), green (G), and blue (B). However, the light emission color of the light emitting element 21 is not limited to the above three primary colors, and can be replaced with a light emission color that allows the viewer to easily distinguish in a situation where the viewer's color is weak. For example, use the orange one in place of the red light emitting element 21, use the blue green one in place of the green light emitting element 21, or use the blue one in place of the blue light emitting element 21. Is also possible. Moreover, in the present invention, the light quantity of each light emitting element 21 can be adjusted independently. That is, in a normal LED light source, white light is generated with the light amount of the light emitting element 21 of three primary colors being always uniform, but the light emitting unit 22 of the present invention is not limited to one generating white light. The amount of light of the element 21 may be increased to generate reddish irradiation light, or only the amount of light of the blue light emitting element 21 or the green light emitting element 21 may be increased. The light amount of each light emitting element 21 in the light emitting unit 22 is adjusted independently by the control circuit 26. The light source unit 22 needs power to emit light, so the light source unit 22 is connected to a battery (not shown) via a power supply line.

The light emitted from each light emitting element 21 of the light source unit 22 is input to the equalizing element 23. The equalizing element 23 adjusts the deviation of the optical path of the irradiation light output from each light emitting element 21 and makes the irradiation surface to the video element 25 uniform via the condenser lens 24 in the latter stage. The condenser lens 24 is formed of an aspheric lens or the like, and converts the irradiation light from each light emitting element 21 into substantially parallel light and emits the light toward the image element 25 so as to pass through the same optical path.

The video element 25 converts incident light into video light by modulating it according to display image data. In the present embodiment, a transmissive liquid crystal panel is used as the video element 25. Each pixel of the liquid crystal panel has, for example, a structure in which a liquid crystal layer is interposed between two glass substrates with transparent electrodes, and a polarizing plate is attached to each of the glass substrates. When no voltage is applied between the transparent electrodes, the liquid crystal molecules of the liquid crystal layer are arranged parallel to the glass surface, but when a voltage is applied between the transparent electrodes, the liquid crystal molecules of the liquid crystal layer are oriented in the direction perpendicular to the glass surface Will change. The combination of the movement of liquid crystal molecules and the polarization directions of the two polarizing plates adjusts the amount of transmission of light transmitted through each pixel of the liquid crystal panel. The liquid crystal panel has a structure in which such pixels are arranged in a matrix. In the present invention, as described later, when the light emitting element 21 is driven by time division, the irradiation light input from the light emitting device 21 to the image element 25 already contains a color component, It is not necessary to provide a color filter or the like on the video element 25. Control of a voltage or the like applied to the video element 25 is performed by the control circuit 26. The video element 25 is connected to the battery via a power supply line. Other than the transmissive liquid crystal panel described above, for example, a reflective liquid crystal panel can also be employed as the video element 25. When a reflective liquid crystal panel is used as the image element 25, the configuration of the “linear arrangement type eyepiece image display device” disclosed in Japanese Patent No. 6081508 may be referred to.

The control circuit 26 subjects the image data generated by the camera 10 to image processing such as correction processing of hue for people with color weakness and generates image data for display (display image data). Then, the control circuit 26 controls each of the light emitting elements 21 and the video element 25 constituting the light source unit 22 based on the display image data to generate and output video light.

In the present invention, the control circuit 26 can control the light amounts of the plurality of light emitting elements 21 constituting the light source unit 22 independently. Specifically, the maximum luminance of a certain light emitting element is preferably set to be twice or more, and may be three times or more or four times or more, with respect to the maximum luminance of another light emitting element.

Here, with reference to FIG. 3 and FIG. 4, the advantage of adjusting the light quantity of a primary color independently in the light source part 22 is demonstrated. FIGS. 3 and 4 show an example in which the light amounts of the respective primary colors of red, green and green are linearly controlled at 256 gradations by 8-bit signals. The pattern of (a) shown in FIG. 3 shows a general control method of the light source. In FIG. 3A, the maximum luminance of each primary color is set to be equal, and the light quantity of each primary color is controlled to 256 gradations between 0 and 100% with an 8-bit signal. In a general light source, only white light is required, so the light amounts of the respective primary colors are controlled to be always equal. On the other hand, in the pattern of (b) shown in FIG. 3, the light amount of each primary color is made equal and the white light is emitted from the light source, and only the light amount of the specific primary color is emphasized from white light on the image element (liquid crystal panel) side. Shows a control method for reducing the amount of light of the other primary colors. For example, FIG. 3 (b) shows an example in the case where the amount of red light is made four times that of green and blue. In this case, since the maximum luminance of each primary color in the light source is equal, the other green and blue colors are maintained at 100% at maximum while the amount of red light is maintained at a maximum of 100% in order to increase the amount of red light The only way to reduce the amount of light to 25%. In this case, although all 256 gradations can be used to display a red image, only 64 gradations, which is 1/4, can be used to display green and blue images. As such, when the gradation of a specific primary color decreases, there is a problem that the image quality of the image is degraded due to the occurrence of a false contour or the like. On the other hand, the pattern of (c) shown in FIG. 4 shows a control method of independently adjusting the light quantity (maximum luminance) of each primary color in the light source. For example, FIG. 4C shows an example in which the maximum luminance of the red light amount is four times the green and blue light amounts while maintaining the green and blue light amounts at a maximum of 100%. In this case, all 256 gradations can be used for each of the red, green and blue primary colors. Therefore, unlike the pattern of FIG. 3B, the pattern of FIG. 4C can avoid deterioration in image quality. As described above, by configuring the light amounts of the plurality of light emitting elements 21 constituting the light source unit 22 to be independently controllable, it is possible to increase the light amount of a specific primary color while avoiding the deterioration of the image quality of the image. Is possible.

As shown in FIG. 2, the eyepiece optical system 30 is an optical element which reflects or refracts the image light emitted from the image element 25 and guides it to the pupil E of the observer. In the present embodiment, the eyepiece optical system 30 has a prism 32. The eyepiece optical system 30 may also have an eyepiece lens 33. The prism 32 is a member for guiding the image light from the display optical system 20 inside. The prism 32 has, for example, a shape having an incident surface 32a of image light, a reflection surface 32b, and an emission surface 32c. The incident surface 32a of the prism 32 is provided to intersect the optical axis of the image light traveling in the lateral direction (X-axis direction) substantially perpendicularly. Further, the exit surface 32c is provided to face the pupil of the observer. The reflecting surface 32b has, for example, a rectangular shape (rectangular shape), and functions as means for bending the optical path of the image light at a right angle. Specifically, the reflecting surface 32b reflects the image light incident on the inside of the prism through the incident surface 32a in the Z direction. The prism 32 may be configured by a single prism, or may be configured by combining a plurality of prisms. The eyepiece 33 is attached to, for example, the incident surface 32 a of the prism 32. The eyepiece lens 33 has positive power and condenses the image light incident on the prism 32 on the pupil. The eyepiece 33 may be joined to the incident surface 32 a of the prism 32 or may be integrated with the prism 32. In addition, it is preferable that the rear surface of the reflection surface 32b of the prism 32 be subjected to light shielding processing for absorbing or totally reflecting light. For example, black paint may be applied to the back surface of the reflective surface 32b or mirror finish may be applied. In this case, it is possible to prevent external light from entering the prism 32 from the back surface of the reflecting surface 32b and the viewer from visually recognizing a scene overlapping the reflecting surface 32b.

In the above configuration, the display optical system 20 generates image light. That is, the irradiation light from the light source unit 22 is homogenized by the homogenizing element 23 and then condensed by the condenser lens 24 and enters the video element 25. The irradiation light is modulated by the video element 25 to become video light. Next, the image light emitted from the display optical system 20 enters the eyepiece optical system 30. In the eyepiece optical system 30, the image light enters the inside of the prism 32 through the eyepiece lens 33. Thereafter, the image light travels inside the prism 32 along the lateral direction (X-axis direction), the light path is bent at the reflecting surface 32b, and travels while changing the direction in the depth direction (Z-axis direction). Thereby, the image light is emitted through the emission surface 32 c of the prism 32 and guided to the pupil of the observer. Thus, the observer can observe the enlarged virtual image of the image generated by the display optical system 20 at the position of the pupil E.

Subsequently, with reference to FIGS. 5 to 11, an example of a method in which the control circuit 26 controls each of the light emitting elements 21 constituting the light source unit 22 will be described. Here, the case where the three light emitting elements 21 emit light in the three primary colors of red, green and blue will be described as an example.

FIG. 5 shows the sensitivity characteristics of the S-cone, M-cone, and L-cone, and the respective maximum values are normalized to one. The blue color of the primary color is perceived by the S cone, but the wavelengths sensed by the M cone and the L cone overlap, so red and green are perceived by calculation of the light reception amounts of the M cone and the L cone. Of these three cones, when one or two sensitivities are lowered, it becomes a state called so-called weak color, and the way of feeling the color differs from that of the healthy person. A color is usually defined by a number called "two-dimensional chromaticity" using x, y coordinates, but people with color blindness can only perceive one-dimensional change in color. Since the color-impaired person can sense the chromaticity only in one dimension in this way, even a color that can be distinguished by a healthy person may appear to be the same color to the color-weakened person. FIG. 6 is a diagram showing the colors felt by the color-impaired person, and it is shown that the colors on the same oblique line in the chromaticity diagram all appear to be the same color to the color-weakened person. If the sensitivity of the L cone is low (type 1 weak), if the sensitivity of the M cone is low (type 2 weak), the sensitivity of the S cone is low. It shows the weak case (3 color weak).

FIG. 7 shows the primary color points of the display, the color reproduction range, and the white (highest luminance) chromaticity point in the xy chromaticity diagram. In FIG. 7, the white chromaticity points in the case where the intensity ratio of the light quantity of each primary color (red, blue, green) is made equal in a general display are indicated by x marks. On the other hand, FIG. 7 shows an example where the control circuit 26 changes the intensity ratio of the primary colors of the light emitting element 21, specifically, the light quantity of red is set to four times that of other green and blue. . In this case, the white chromaticity point is at the position indicated by 印. As described above, it is effective to enhance the intensity (light quantity) of the red primary color as a support measure for the color-weak person with low red sensitivity (mainly type 1 weak). However, in such a case, although there is no change in each of the red, green and blue primary color points, it can be understood that the white chromaticity point (maximum luminance) approaches the red primary color point by increasing the red intensity . Then, if only the amount of red light is simply increased, the effect of feeling red increases for people with color weakness, but the whiteness will be felt reddish.

Subsequently, as shown in FIG. 8, the light quantity of each light emitting element 21 can be controlled by the control circuit 26 to change the position of the primary color itself. For example, it is effective to change the position of the primary color itself when the sensitivity drop of the cone of the observer is large and the intensity (light intensity) increase of the specific primary color is insufficient. In the example shown in FIG. 8, the color normally displayed in red is converted to orange and displayed as a support measure for the color weak who is sensitive to red (mainly type 1 weak). As a result, the color-impaired person can recognize that the red itself can not be recognized, and if it is orange, there is a color there. Also, in order to clarify the difference between orange and green, the green primary color is also converted to light blue. Thus, by appropriately converting the position of each primary color point, it becomes easy for the color-weak to recognize the color distinction. However, as shown in FIG. 8, also in this case, the chromaticity point of white (highest luminance) deviates from the original position, so the observer recognizes the white as a color different from the normal color.

In FIG. 9, by performing control to change the primary color point in the control circuit 26 as shown in FIG. 8 as described above, it is possible to distinguish colors existing on the same confusion color line even for the weak (for example, type 1) It shows that it becomes. In this way, in the state before changing the primary color point, even if the color was present on the same confusion color line, it is possible to shift from the same confusion color line by changing the primary color point. Even those colors can be distinguished as different colors. In addition, when primary color points are changed, colors that were originally distinguishable without problems may be arranged on the same line, so it is also possible to temporally switch between the state where primary color points are changed and the original state without change. It is valid. That is, by displaying two types of chromaticity perceived in one dimension temporally, two-dimensional chromaticity can be simulated in a pseudo manner.

Further, as shown in FIG. 10, even when the position of the primary color chromaticity point itself is changed, the white (highest luminance) chromaticity point is originally set by adjusting the intensity ratio of the light emitting element 21. It can be returned to the position. As described above, the control circuit 26 can also restore the white balance by adjusting the intensity ratio of each light emitting element 21 after changing the primary color chromaticity point.

FIG. 11 shows, as an example, the spectrum of a normal light source and the spectrum of the light source when red and green primary color points are moved. Although the movement of the primary color points can also be performed using color conversion of the image signal, it can be said that it is more preferable to use a light source having the primary color point as the dominant wavelength. For example, when the sensitivity of the L cone is low, using an orange wavelength light source is more efficient than when displaying a red primary color (eg, orange) changed by mixing red and green wavelengths. If such a light source is used, color conversion processing with a signal becomes unnecessary.

Subsequently, with reference to FIG. 12, an example in which the control circuit 26 drives each of the light emitting elements 21 constituting the light source unit 22 in a time division manner will be described. In time-division driving, the other light-emitting elements 21 are turned off at the timing when one light-emitting element 21 is lit, and this is repeatedly performed for each light-emitting element 21 at a high speed. It is a method of combining the image lights in the observer's brain by visually recognizing the afterimage.

FIG. 12 conceptually shows an example of time-division driving of three types of light emitting elements emitting red, green and blue light. As shown in FIG. 12, the red light emitting element, the green light emitting element, and the blue light emitting element emit light in this order, and another light emitting element is turned off at the timing when a certain light emitting element is lit. The primary color light emitted from each light emitting element is input to a video element (liquid crystal panel) and converted into video light there. The red image light, the green image light, and the blue image light are sequentially projected onto the observer's pupil, but these three types of image light are combined in the observer's brain to form one image. It becomes. In addition, as shown in FIG. 12, by adjusting the transmission amount of each image light with the image element, fine hue (specifically, the cube of 256 gradations) is obtained for an image obtained by combining three types of image light. = 16777216) can be expressed.

When each light emitting element 21 is driven in a time-division manner, the light emission color of each light emitting element 21 is substantially separately (when the light cone of the retina L cone, M cone and S cone) of the observer. Projected) For this reason, in the case of time-division driving, the hue of the image light is not mixed at the stage of being projected onto the eye of the observer, so that there is an advantage that the image light can be easily perceived by people with color blindness.

Subsequently, a configuration capable of switching the light emitting elements 21 constituting the light source unit 22 will be described with reference to FIGS. 13 and 14. FIG. 13 shows an example in which the light emitting element 21 to be driven is switched by a switch, and FIG. 14 shows an example in which the light emitting element 21 incorporated in the display optical system is physically replaced.

In the example shown in FIG. 13, in the display optical system 20, four or more types of light emitting elements 21 are provided in advance in the light emitting unit 22, and the control circuit 26 can select which light emitting element 21 is to be driven. It is configured. Specifically, the light emitting unit 22 includes a first light source 22 a including three types of light emitting elements 21 and a second light source 22 b including three types of light emitting elements 21. The light emitting elements 21 of the first light source 22 a and the light emitting elements 21 of the second light source 22 b may have different emission colors of at least one light emitting element 21, and the emission colors of all the light emitting elements 21 may be different. . For example, the first light source 22a is composed of light emitting elements 21 of three primary colors of red, green and blue for displaying a normal image, and the second light source 22b has a color different from the above three primary colors for the weak Light emitting elements 21 (for example, a combination of orange, light blue and blue, or a combination of red, yellow and blue green). The first light source 22a and the second light source 22b are connected to the control circuit 26, respectively. The control circuit 26 determines which of the first light source 22 a and the second light source 22 b is to be driven according to the mode selected by the observer. As described above, which of the first light source 22a and the second light source 22b is to be driven can be switched by the switch.

In the example shown in FIG. 14, a plurality of physically separated light source units 41, 42, 43 are prepared in advance, and the display optical system 20 can be mounted with an arbitrarily selected light source unit. It is configured. Specifically, the display optical system 20 and each light source unit 41, 42, 43 are provided with terminals for connection, respectively, and by connecting the two terminals, the display optical system 20 and each light source unit 41, 42 and 43 are electrically connected. In the example shown in FIG. 14, the first light source unit 41 includes three light emitting elements 21. Each light emitting element 21 emits light in three primary colors of red, green and blue for displaying an ordinary image. . The second light source unit 42 includes a first light source 42a and a second light source 42b. In the second light source unit 42, for example, the first light source 22a is composed of light emitting elements 21 of three primary colors of red, green and blue for displaying a normal image, and the second light source 22b is for the weak The light-emitting elements 21 (for example, a combination of orange, light blue, and blue, or a combination of red, yellow, and blue-green) that are different from the three primary colors are configured. When the second light source unit 42 is attached to the display optical system 20, the control circuit 26 can select which of the first light source 22a and the second light source 22b is to be driven. Similar to the second light source unit 42, the third light source unit 43 also includes the first light source 43a and the second light source 43b. The first light source 43a is composed of light emitting elements 21 of three primary colors of red, green and blue for displaying a normal image. On the other hand, the second light source 43b is assumed to be driven at the same time as the first light source 43a, and for the color-weaker, for example, the light emitting element 21 of the same luminescent color as one of the three primary colors of the first light source 43a. There are three. For example, the three light emitting elements 21 constituting the second light source 43a emit light of the same color, and are selected from red, green or blue. That is, when the third light source unit 43 is attached to the display optical system 20, the control circuit 26 selects whether to drive only the first light source 43a or to drive both the first light source 43a and the second light source 43b. Do. For example, when the second light source 43b is composed of three light emitting elements 41 emitting red light, only the first light source unit 41 is driven by driving both the first light source 43a and the second light source 43b. The amount of red light (maximum luminance) can be set to four times as much as sometimes.

Subsequently, with reference to FIGS. 15 to 17, processing for extracting display image data equivalent to the observer's visual angle from the image data acquired by the camera 10 in consideration of the parallax between the observer's pupil and the camera 10 Explain. As shown in FIG. 15, for example, the case where the camera 10 is disposed near the temple of the observer and the image light is projected onto one eye of the observer is considered. In particular, here, as in the configuration shown in FIG. 1 and the like, the case where image light is projected onto the pupil by the prism 32 by the prism 32 and the back surface of the prism 32 is shielded will be described as an example.

As shown in FIG. 15, a gap is present between the observer's one eye and the camera 10, and as a result, a parallax P is generated between the line of sight of the observer's one eye and the line of sight of the camera 10. In this case, if the image data captured by the camera 10 is projected as it is to the observer's one eye, it is possible to seamlessly overlap the background actually viewed by the observer with the image generated by the device under the influence of the parallax P. Can not. Therefore, in consideration of the parallax P, a range in which the gaze range of the observer and the object plane at which the observer is gazing overlap is extracted as a display image from the image data captured by the camera 10 (cut out ), It is effective to generate display image data. However, as can be seen by comparing FIG. 15 and FIG. 16, the range extracted as the display image fluctuates depending on the distance from the photographing lens of the camera 10 to the object plane at which the observer is gazing. Here, the distance from the photographing lens of the camera 10 to the object plane at which the observer is gazing can be measured by an autofocus lens provided in the camera 10. Therefore, the control circuit 26 preferably determines a range to be extracted as a display image from the image data based on the parallax P and the distance to the object plane measured by the autofocus lens. The parallax P can be arbitrarily set in consideration of the entire configuration of the color vision assistance device 100, and may be finely adjusted by the operation of the observer. In addition, the distance from the photographing lens of the camera 10 to the object plane at which the observer is gazing may be input to the control circuit 26 as the measurement value of the autofocus lens provided in the camera 10 as described above.

As described above, the control circuit 26 extracts display image data to be projected onto the observer's pupil from the image data acquired by the camera 10 based on the parallax P and the measurement value of the autofocus lens. Then, based on the display image data, the light source unit 22 and the video device 25 are controlled to generate video light. FIG. 17 shows an example in the case where an image generated by the color vision support apparatus 100 is superimposed and displayed on an actual landscape. If the display range of the image is reduced relative to the actual viewing angle (the apparent size), a gap may occur between the landscape and the image, which may cause the viewer to feel uncomfortable. On the other hand, when the display range of the video is equal to the actual viewing angle, the landscape and the video are seamlessly connected, and the observer can naturally view the state where the video is superimposed in the landscape. Such a display makes it possible to provide natural visual assistance to the observer.

In the present invention, the control circuit 26 of the front optical system 20 may select a mode for converting the hue of the image data in accordance with the distance detected by the autofocus sensor 11. Specifically, the control circuit 26 stores a plurality of modes for the method of converting the hue of the image data in a memory or the like, and determines the mode in accordance with the distance detected by the autofocus sensor 11. With the mode of the hue conversion method, for example, the degree to which the light amount of the specific light emitting element 21 is increased may be divided into a plurality of modes. Also, prepare a mode that changes only the RGB ratio, a mode that includes movement of the origin (white chromaticity point) in addition to changing the RGB ratio, or a mode that changes the RGB ratio but maintains the W / B balance. May be For example, depending on the situation, by changing the mode of the hue conversion method depending on whether the observer is looking at a distant view or the task of making a color judgment at hand It can provide appropriate color vision support. Specifically, color conversion of image data is performed in different modes between the case shown in FIG. 15 and the case shown in FIG.

Hereinabove, in order to express the content of the present invention, the embodiments of the present invention have been described with reference to the drawings. However, the present invention is not limited to the above embodiment, and includes modifications and improvements apparent to those skilled in the art based on the matters described in the present specification.

DESCRIPTION OF SYMBOLS 10 ... Camera 11 ... Auto-focus sensor 20 ... Display optical system 21 ... Light emitting element 22 ... Light source part 22a ... 1st light source 22b ... 2nd light source 23 ... Equalization element 24 ... Condensing lens 25 ... Picture element 26 ... Control circuit 30 ... eyepiece optical system 31 ... transparent substrate 32 ... prism 32a ... incident surface 32b ... reflection surface 32c ... emission surface 33 ... eyepiece lens 41 ... first light source unit 42 ... second light source unit 43 ... third light source unit 100 ... color vision support device

Claims

A camera (10) that captures the outside world and generates image data;
A display optical system (20) for emitting image light corresponding to the image data;
An eyepiece optical system (30) for guiding the image light to the observer's pupil,
The display optical system (20) is
A light source unit (22) including a plurality of light emitting elements (21) each having a different emission wavelength;
A video element (25) for converting light emitted from the light source unit (22) into the video light;
A control circuit (26) for independently controlling the light amounts of the plurality of light emitting elements (21);
The color vision assistance apparatus which can output the image light which converted the said image data into the hue which is easy to be color-distinctive of a viewer.
The color vision assistance device according to claim 1, wherein the control circuit (26) performs time-division driving in which the plurality of light emitting elements (21) sequentially emit light.
The light source unit (22) includes four or more light emitting elements (21) having different emission wavelengths,
The color vision assistance device according to claim 1, wherein the control circuit (26) selects and drives at least three of the light emitting elements (21).
The light source unit (22) includes a first light source unit (41) including a plurality of light emitting elements (21) having different emission wavelengths, and at least one light emitting element included in the first light source unit (41) The color vision assistance device according to claim 1, wherein the color vision assistance device is configured to be physically exchangeable with the second light source unit (42) including at least one or more light emitting elements (21) having different emission wavelengths.
The eyepiece optical system (30) has a prism (32) that reflects the image light and guides it to the pupil of the observer;
The color vision aiding apparatus according to claim 1, wherein the back surface side of the reflection surface (32a) of the prism (32) is shielded.
The camera (10) has an autofocus sensor (11) for detecting the distance between the photographing lens and the subject,
The display optical system (20) has a plurality of modes for the method of converting the hue of the image data, and determines the mode in accordance with the distance detected by the autofocus sensor (11). Color vision support device.
The camera (10) has an autofocus sensor (11) for detecting the distance between the lens and the subject,
The control circuit (26) generates display image data obtained by cutting out a part of image data generated by the camera (10) based on the distance detected by the autofocus sensor (11), and the display image The color vision support device according to claim 1, wherein a video signal for displaying data is sent to the light source unit (22) and the video element (25).