JP6803027B2 - Head-mounted image presentation device with visible light wavelength converter - Google Patents

Description

The present invention relates to a head-mounted image presentation device having a visible light wavelength conversion unit.

The annual number of blind people in Japan reaches 16,000, and the causes can be broadly divided into two types, except for severe trauma, which are caused by disorders of the inner retina and disorders of the outer layer. It is known that the outer layer of the retina is selectively degenerated in retinal degenerative diseases such as retinitis pigmentosa, age-related macular degeneration, and retinal detachment. Looking at the photoreceptor of the retina and its signal transmission mechanism, the incident light and image information are received by the photoreceptor cells (photoreceptors) located in the outer layer of the retina and transmitted to the inner layer of the retina. That information is transmitted to the brain by the ganglion cells that make up the optic nerve, located in the inner layer of the retina. In this way, the only cells that can receive light, that is, images, are photoreceptor cells in the retina, and if the photoreceptor cells degenerate or disappear for some reason, even if the other cells are normal, blindness will occur.

According to the photoreception mode of photoreceptor cells, it is known that the photoreceptive proteins present in photoreceptor cells require a chain reaction of various proteins after photoreception. From such a mode, it has been conventionally considered that it is impossible to confer photoacceptivity by introducing a single gene. In this connection, it is necessary to introduce at least three kinds of proteins (that is, arrestin, opsin, and G protein α subunit) into one cell in order to impart photoacceptivity to nerve cells. It has been reported (Non-Patent Document 1).

Nagal et al. (Non-Patent Document 2) found that channelrhodopsin-2 (hereinafter, also referred to as “ChR2”) isolated from the green alga Chlamydomonas has both photoacceptivity and cation-selective permeability, and ChR2. It was reported that the photoreceptive ability can be imparted to cultured mammalian cells (HEK293, BHK) and the like by introducing a gene. A similar gene that is sensitive to red color has been found in the green alga Volvox (Non-Patent Document 3).

The present inventors have obtained a modified Volvox carteri obtained by fusing the N-terminal region of channelrhodopsin derived from Chlamydomonas reinhardtii with channelrhodopsin derived from Volvox. It has been found that the expression efficiency of the channelrhodopsin on the cell membrane can be improved by using the photoreceptive channelrhodopsin protein (Patent Document 1).

WO2011 / 019081

Zemelman BV et al. , Neuron. 2002 Jan 3; 33 (1): 15-22 Nagel G et al. , Proc Natl Acad SciUS A. 2003 Nov 25; 100 (24): 13940-5 Freng Zhang et al. , Nature Neuroscience, Volume 11, Number 6, June 2008: p631-633

However, since Chlamydomonas-derived ChR2 is limited to blue, even if vision is restored, only blue can be seen. A similar gene with photosensitivity was found in the green alga Volvox, but visual recovery is limited to red. Although the modified Volvox-derived photoreceptive channel rhodopsin protein responds to a wider wavelength range, it is limited to around 450-600 nm and does not respond well at wavelengths of 380-450 nm and 600-750 nm.

In this way, by administering the protein to patients with retinal degeneration, it is possible to react in a certain wavelength region of the visible light wavelength region, but it is still possible to sufficiently react in other regions. I can't. No means has been realized for a person having reduced sensitivity to a part of the visible light wavelength range, including a patient with retinal degeneration, to recognize all visible light incident on the visual field.

The present invention has been made in view of the above problems and has the following features. That is, in the head-mounted image presenting apparatus according to the embodiment of the present invention, the sensitivity to a part of the visible light wavelength range is lower than that of the other wavelength range of the visible light wavelength range. A head-mounted image presenting device for a user who has an imaging unit that captures the user's visual direction to acquire first image information, and at least a part of the wavelength range in the first image information. A wavelength conversion unit that converts at least a part of the color signal for the above into a color signal that expresses colors in the other wavelength range and outputs a second video information based on the converted color signal, and the first. It is provided with an image presenting unit that presents to the user an image in the viewing direction of the user whose wavelength has been converted based on the image information of 2.

The wavelength conversion unit of the present invention includes a first table including spectral chromaticity coordinates in the XYZ color system, and the first table for converting wavelengths in a part of the wavelength regions into the other wavelength regions. The chromaticity coordinates associated with the wavelengths of the part of the wavelength range in the table are associated with the wavelengths of the other wavelength range, and the chromaticity associated with the wavelengths of the other wavelength range in the first table. Based on the first video information, the XYZ table includes a second table in which at least a part of the coordinates is associated with a chromaticity coordinate different from the chromaticity coordinate associated in the first table. An XYZ signal that expresses a color in a color system is acquired, a wavelength corresponding to the XYZ signal is specified based on the first table, and a wavelength corresponding to the specified wavelength is associated with the specified wavelength based on the second table. The XYZ signal for the converted XYZ signal may be acquired, and the second video information based on the converted XYZ signal may be output.

Further, the wavelength conversion unit acquires an RGB signal expressing a color in the RGB color system based on the first video information, calculates the stimulus sum of the XYZ signal based on the RGB signal, and calculates the XYZ signal. The r signal value obtained from is divided by the sum of stimuli to obtain the r signal value, and when the acquired r signal value is equal to or greater than the r signal upper limit value, the acquired r signal value is used as the r signal. Converted to the upper limit value, the converted r signal value is multiplied by the stimulus sum to obtain the converted R signal value, and the second video information based on the RGB signal including the converted R signal value is obtained. The r signal upper limit value may be output and set to a value equal to or less than the maximum r signal value in the other wavelength range obtained by RGB conversion of the spectral chromaticity coordinates by the XYZ color system.

Further, the wavelength conversion unit acquires the g signal value by dividing the G signal value acquired from the RGB signal by the stimulus sum, and when the acquired g signal value is equal to or less than the lower limit value of the g signal. , The g signal value is converted to the g signal lower limit value, the converted g signal value is multiplied by the stimulus sum to obtain the converted G signal value, and RGB including the converted G signal value is further included. The second video information based on the signal is output, and the g signal lower limit is the g signal value for the wavelengths at both ends in the other wavelength range obtained by RGB conversion of the spectral chromaticity coordinates by the XYZ color system. It may be set to be greater than or equal to the larger g signal value.

The wavelength conversion unit acquires an RGB signal expressing colors in the RGB color system based on the first video information, and RGB-converts the spectral chromaticity coordinates by the XYZ color system to obtain an RGB color system. The converted R signal value is obtained by multiplying the r signal conversion coefficient determined based on the spectral chromaticity coordinates according to the above by the R signal value included in the acquired RGB signal to obtain the converted R signal value. The second video information based on the RGB signal including RGB is output, and the r signal conversion coefficient is a value obtained when the r signal value in the spectral chromaticity coordinates by the RGB color system is multiplied by the r signal conversion coefficient. The maximum value of r signal is determined not to exceed the upper limit value of r signal, and the upper limit value of r signal is a value equal to or less than the maximum r signal value in other wavelength regions in the spectral chromaticity coordinates of the RGB color system. It may be.

Instead of acquiring the converted R signal value by multiplying the r signal conversion coefficient by the R signal value included in the acquired RGB signal, the stimulus sum calculated based on the acquired RGB signal. The r signal value obtained by dividing the R signal value included in the acquired RGB signal is multiplied by the r signal conversion coefficient to obtain the converted r signal value, and the converted r signal is obtained. The converted R signal value can also be obtained by multiplying the value by the sum of stimuli.

The wavelength conversion unit calculates the stimulus sum of the XYZ signal based on the acquired RGB signal, and divides the G signal value included in the acquired RGB signal by the stimulus sum to acquire the g signal value. Then, after multiplying the g signal value by a predetermined coefficient, the g signal lower limit value is added, and the stimulus sum is multiplied to obtain the converted G signal value, and RGB further including the converted G signal value. The second video information based on the signal is output, and the g signal lower limit value is the larger of the g signal values for the wavelengths at both ends in the other wavelength range in the spectral chromaticity coordinates by the RGB color system. It may be equal to or more than the g signal value.

The r signal upper limit value may be an r signal value for a wavelength of 600 nm.

The other wavelength range may be 450 nm to 600 nm.

The presentation unit can be a projector or a display.

The imaging unit captures a left eye image in the user's view direction for presentation to the user's left eye and a right eye image in the user's view direction for presentation to the user's right eye, which is different from the left eye image. The first image information captured includes the image information for the left eye and the image for the right eye, and the second image information includes the image for the left eye and the right image converted by the wavelength conversion unit. The image presenting unit is converted with respect to the user's left eye based on the image information for the converted left eye image included in the second image information, including the image information for the eye image. The left eye image is presented, and the converted right eye image is presented to the user's right eye based on the image information for the converted right eye image included in the second image information. You may do so.

The user can be a retinal degeneration patient who has been administered to the eye a protein that allows ganglion cells to receive light in the other wavelength range.

Further, the wavelength conversion device according to the embodiment of the present invention is for a user whose sensitivity to a part of the visible light wavelength range is lower than that of the other wavelength range of the visible light wavelength range. A wavelength conversion device that outputs a video signal for presenting a video based on a color signal expressing colors in the other wavelength range, and is at least in the first video information acquired by imaging the user's visual direction. At least a part of the color signal for the part of the wavelength range is converted into a color signal expressing the color in the other wavelength range, and the second video information based on the converted color signal is output.

Further, the program of one embodiment of the present invention may be applied to a user whose sensitivity to a part of the visible light wavelength range is lower than that of another wavelength range of the visible light wavelength range. It is a program for outputting a video signal for presenting a video based on a color signal expressing a color in the wavelength range of the above, and in the first video information acquired by imaging the user's viewing direction on a computer. A step of converting at least a part of a color signal for at least a part of the wavelength range into a color signal expressing a color in the other wavelength range, and a second video information based on the converted color signal. The process of outputting and the process of outputting can be executed.

The method of the embodiment of the present invention is within the other wavelength range for a user whose sensitivity to a part of the visible light wavelength range is lower than that of the other wavelength range of the visible light wavelength range. It is a method for outputting a video signal for presenting a video based on a color signal expressing the color of the above, and at least the above-mentioned one in the first video information acquired by a computer by imaging the viewing direction of the user. A step of converting at least a part of a color signal for a wavelength range of a part into a color signal expressing a color in the other wavelength range, and a step of outputting a second video information based on the converted color signal. And may be executed.

According to the present invention, even a user whose sensitivity to a part of the visible light wavelength range is lower than that of the other wavelength range of the visible light wavelength range is all of the visible light. Can be recognized.

It is a schematic perspective view of the head-mounted image presenting apparatus which concerns on one Embodiment of this invention. It is a schematic rear side perspective view of the head-mounted image presenting apparatus which concerns on one Embodiment of this invention. It is a schematic rear side perspective view of the head-mounted image presenting apparatus which concerns on one Embodiment of this invention. It is a schematic perspective view of the head-mounted image presenting apparatus which concerns on one Embodiment of this invention. It is a hardware block diagram of the head-mounted image presenting apparatus which concerns on one Embodiment of this invention. It is a figure which shows the wavelength region before and after conversion which concerns on one Embodiment of this invention. It is a flowchart which shows the information processing which concerns on one Embodiment of this invention. It is a flowchart which shows the information processing which concerns on one Embodiment of this invention. It is a graph of the spectral chromaticity coordinates by the XYZ color system which concerns on one Embodiment of this invention. It is a flowchart which shows the information processing which concerns on one Embodiment of this invention. It is a graph of the spectral chromaticity coordinates by the RGB color system which concerns on one Embodiment of this invention. It is a flowchart which shows the information processing which concerns on one Embodiment of this invention. It is a graph of the spectral chromaticity coordinates by the RGB color system which concerns on one Embodiment of this invention. It is a flowchart which shows the information processing which concerns on one Embodiment of this invention. It is a graph of the spectral chromaticity coordinates by the RGB color system which concerns on one Embodiment of this invention.

[First Embodiment]
Schematic drawings of the head-mounted image presentation device 100 according to one embodiment of the present invention are shown in FIGS. 1a to 1d. The head-mounted image presenting device 100 is used by being attached to the user's head, and includes an imaging unit 101 and an image presenting unit 102. The imaging unit 101 is arranged at a position where the user's field of view can be imaged, and the image presenting unit 102 is arranged at a position where the image can be presented to the user. The head-mounted image presenting device 100 performs wavelength conversion processing (color conversion processing) on the image in the visual field direction of the user captured by the imaging unit 101 by the wavelength conversion unit 103, and the wavelength-converted image on the image presenting unit 102. To the user. The wavelength conversion unit 103 is not shown in FIG.

FIG. 1a is a schematic perspective view of one configuration example of the head-mounted image presenting device 100, and FIGS. 1b and 1c are schematic rear perspective views thereof. FIG. 1d is a schematic perspective view of another configuration example of the head-mounted image presenting device 100. FIG. 1b is a configuration example when a projector is used as the image presentation unit 102. In this figure, a separate projector lens is provided for each of the left and right eyes, but for weight reduction and cost reduction, only one lens is provided and the image is presented to only one eye. May be good. FIG. 1c shows a configuration example when a display is used as the image presentation unit 102. FIG. 1d shows a case where a projector is used as the image presenting unit 102, and includes a projector housing 110 including a light source, prisms 111 and 112. The light emitted from the projector housing 110 is refracted in a U shape by the prisms 111 and 112 and emitted toward the user's right eye. The projector can make the user see an image by projecting directly onto the retina using, for example, a retinal projection method.

FIG. 2 shows a hardware configuration diagram of the head-mounted image presentation device 100 according to the first embodiment. As described above, the head-mounted image presenting device 100 includes an imaging unit 101, an image presenting unit 102, a wavelength conversion unit 103, and further includes a processing unit 104, a storage unit 105, and a bus 108. The wavelength conversion unit 103 includes an internal storage unit 107. The storage unit 105 stores the program 106.

The head-mounted image presentation device 100 is a device for a user whose sensitivity to a part of the visible light wavelength range is lower than that of the other wavelength range of the visible light wavelength range. .. Since the sensitivity of some visible light wavelength regions is reduced, this user has difficulty in recognizing light in that wavelength region or cannot recognize it at all. In the present invention, the reduced sensitivity includes the case where there is no sensitivity at all. On the other hand, with respect to light in other wavelength ranges, the sensitivity may be lower than that of a healthy person, but the sensitivity is higher than that of some of the wavelength ranges. The head-mounted image presenting device 100 performs wavelength conversion of light in a wavelength region in which the sensitivity is relatively low so that the light can be recognized in the wavelength region having a relatively high sensitivity.

It does not matter what the reason why the sensitivity of the target user of the present invention to the visible light wavelength is lowered. Examples of the causes include retinitis pigmentosa, age-related macular degeneration, retinal detachment and other retinal degeneration diseases. The target user of the present embodiment is a patient with retinal degeneration, but by administering a photoreceptive channel rhodopsin protein derived from a modified volbox, the sensitivity is improved for colors in the wavelength range of 450 to 600 nm in the visible light wavelength range. However, the color in the wavelength range of 380 to 450 nm and 600 to 750 nm is still very insensitive and cannot be recognized.

The imaging unit 101 captures the user's visual field direction to acquire the first video information. The wavelength conversion unit 103 compares at least a part of the color signal for a part of the wavelength range in which the sensitivity of the first video information acquired by the imaging unit 101 is reduced in the visible light wavelength range. It is converted into a color signal expressed by a color in another visible light wavelength range having high target sensitivity, and the second video information based on the converted color signal is output. The image presentation unit 102 presents the wavelength-converted image in the user's field of view to the user based on the second image information output by the wavelength conversion unit 103. The image presentation unit 102 may be a projector that presents an image, or may be a display device such as a liquid crystal display, an organic EL display, or a plasma display.

A user who cannot sufficiently recognize visible light with the amount of light emitted by an image presenting device such as a display even in a wavelength range in which the user's sensitivity to visible light is relatively good (the other wavelength range of the visible light wavelength range). There is also. For such users, it is desirable to use a projector as the image presentation unit 102. By directing high-intensity visible light directly into the user's eyes with a projector, the user can recognize the image. When a projector provided with two lenses is used as the image presentation unit 102 in the present embodiment, light for presenting the same image is emitted from both lenses based on the second image information. Further, when a projector including two lenses is used, it is preferable to present images with different wavelength conversions for each eye. When the sensitivity to each wavelength is different for each eye, wavelength conversion suitable for each eye can be performed.

The processing unit 104 controls the imaging unit 101, the image presentation unit 102, and the wavelength conversion unit 103 based on the instructions included in the program 106 stored in the storage unit 105 to execute information processing. The storage unit 105 includes a hard disk, a main memory, and a buffer memory. The program 106 is stored in the storage unit 105. However, the storage unit 105 may be any non-volatile storage or non-volatile memory as long as it can store information, and may be removable. The storage unit 105 stores the program 106 and various data that can be referred to when the program is executed. The wavelength conversion unit 103 may store information necessary for wavelength conversion in the internal storage unit 107, may be stored in the storage unit 105, or may be used in combination. The internal storage unit 107 includes a hard disk, a main memory, and a buffer memory like the storage unit 105. The wavelength conversion unit 103 may be realized by an integrated circuit, or the function as a wavelength conversion unit may be realized by the processing unit 104 executing the program 106 stored in the storage unit 105. The bus 108 connects between the hardware elements included in the spectacle-type image presenting device 100.

In the present embodiment, as shown in FIG. 3, light in the visible light wavelength range of 380 to 750 nm is wavelength-converted to light in the wavelength range of 450 to 600 nm. That is, visible light in the visible light wavelength range of 380 to 750 nm is converted into a wavelength in the wavelength range of 450 to 600 nm that can be recognized by the user targeted by the present embodiment and presented, and is presented at 450 nm or less and 600 nm or more. Wavelength should not be output or presented. Therefore, even if visible light having a wavelength that cannot be recognized originally is included in the user's field of view, it is possible to recognize an image of the field of view including the presence of the visible light, although the wavelength is different from the actual wavelength. It becomes.

The operation of the head-mounted image presenting device 100 according to the present embodiment will be described with reference to the flowchart shown in FIG. First, in step 401, the imaging unit 101 starts imaging in the user's field of view direction to acquire the first video information. The imaging unit 101 continues imaging unless the imaging is stopped, and continuously outputs the first video information to the wavelength conversion unit 103. In step 402, the wavelength conversion unit 103 acquires the first video information output from the imaging unit 101, and at least a part of the color signal for the wavelength light of 380 nm or less and 750 nm or more is 450 to 600 nm. It is converted into a color signal expressed by the color for the wavelength light of, and the second video information based on the converted color signal is output. The wavelength conversion may be performed on the entire visible light wavelength, or may be performed on any wavelength range as long as it includes wavelengths of 450 nm or less and 600 nm or more that cannot be recognized by the user. In step 403, the image presenting unit 102 presents the wavelength-converted image in the visual field direction of the user to the user based on the second image information output by the wavelength conversion unit 103. When the image presenting unit 102 is a projector, for example, the image is presented by emitting light based on the image information from the projector. After that, returning to step 402, the wavelength conversion unit 103 executes wavelength conversion on the first video information sequentially output by the imaging unit 101 to output the second video information, and in step 403, the video presentation unit 102 Repeats the process of presenting the image based on this.

By executing the above-mentioned processing of the present embodiment, as shown in FIG. 3, visible light in the wavelength range of 380 to 750 nm in the visible light wavelength range can be recognized by the user targeted by the present embodiment 450 to. It is possible to convert the wavelength into a wavelength within the wavelength range of 600 nm and display it, so that the user can recognize an image of the field of view including the presence of visible light having a wavelength that cannot be originally recognized. It is clear that the present invention can be carried out by the same method even if the wavelength range recognizable by the user is another wavelength range.

[Second Embodiment]
This embodiment differs from the first embodiment in that XYZ conversion is used as the wavelength conversion process. Other points are the same as those in the first embodiment. Hereinafter, the wavelength conversion process different from that of the first embodiment will be mainly described.

In the present embodiment, the wavelength conversion unit 103 includes a first table including spectral chromaticity coordinates in the XYZ color system, and a part in which the user's sensitivity is lower than that of other wavelength regions in the visible light region. In order to convert a wavelength in the visible light wavelength range into a wavelength in another wavelength range, the chromaticity coordinates associated with the wavelength in the visible light wavelength range in the first table are set in the other wavelength range. At least a part of the chromaticity coordinates associated with wavelengths and associated with wavelengths in the other wavelength range in the first table is different from the chromaticity coordinates associated with in the first table. Includes a second table associated with. In the present embodiment, these tables are stored in the internal storage unit 107 provided in the wavelength conversion unit 103, but may be stored in the storage unit 105. The wavelength conversion unit 103 acquires an XYZ signal that expresses a color in the XYZ color system based on the first video information, and identifies the wavelength corresponding to the XYZ signal based on the first table. Based on the second table, the XYZ signal for the converted wavelength associated with the specified wavelength is acquired, and the second video information based on the converted XYZ signal is output.

FIG. 5 shows a process using the XYZ conversion in the present embodiment as the wavelength conversion process (step 402). In step 501, one frame is taken out from the first video information acquired by the imaging unit 101. The first video information acquired by the imaging unit 101 is moving image data including a plurality of frames. For example, if the moving image data is 30 fps (Frames per Second), 30 frames are imaged per second. In the present embodiment, the wavelength conversion unit 103 reads one frame from the first video information, executes the wavelength conversion process, and the video presentation unit 102 repeatedly executes the process of presenting the video frame by frame.

The RGB signal of each pixel is acquired as the color signal in the video information of one frame extracted in step 501 in step 502. When the RGB signal of the video information acquired from the imaging unit 101 is an sRGB signal including gamma correction, the RGB signal is acquired by performing reverse gamma correction. Similarly, in other embodiments for acquiring RGB signals, reverse gamma correction is performed as necessary. If the input RGB signal value exceeds a predetermined value, appropriate wavelength conversion processing may not be performed. Therefore, the RGB signal is within a predetermined range by calibration or automatic gain control (AGC) based on the reference signal. Control may be performed so as to have an internal size. The AGC is executed based on the reference signal at an appropriate timing before the wavelength conversion. Then, in step 503, this RGB signal is converted into an XYZ signal (XYZ conversion). Various conversion formulas for converting an RGB signal into an XYZ signal are known, but in the present embodiment, the formula (1) by the CIE RGB method is used. Here, i indicates a pixel number, and can take a value from 0 to the maximum pixel number i_max (number of pixels-1). This XYZ conversion is performed on all pixels (i = 0 to i_max). Further, in step 504, the stimulus sum S [i] of the tristimulus value XYZ of each pixel is acquired by the calculation of the equation (2) and stored in the internal storage unit 107.

Then, in step 505, the XYZ values obtained as shown in the equation (3) are divided by the stimulus sum S [i] to obtain the normalized xyz values for all the pixels.

In step 506, the wavelength of each pixel is specified based on the normalized xyz value and the spectral chromaticity coordinates in the XYZ color system. The spectral chromaticity coordinates may be any as long as the correspondence between the normalized xyz value and the wavelength is shown. In this embodiment, the spectral chromaticity coordinates in the XYZ color system formulated by the CIE shown in Table 1 and FIG. 6 (a) are used. Since the spectral chromaticity coordinates themselves in this XYZ color system are well known to those skilled in the art, Table 1 shows only the wavelength range necessary for explanation, and the other areas are omitted for simplification. To do. Further, in the present embodiment, spectral chromaticity coordinates from 380 to 700 nm are used, but a table of spectral chromaticity coordinates including a wider range or a narrower range, for example, spectral chromaticity coordinates from 380 to 750 nm is used. It is clear that the present invention can be realized by the same method even if there is.

The xyz value of each pixel acquired in step 505 is compared with the xyz value in Table 1, and the corresponding wavelength λ is specified. For example, first, the x value is used as a reference to identify a wavelength that matches or has some close x values as candidates, and then the y value and the z value are used as a reference to sequentially perform a comparison to match or the closest xyz value. The wavelength having is specified as the wavelength for the pixel.

When the xyz value for the acquired pixel i is x [i] = 0.173332, y [i] = 0.00482, z [i] = 0.82188 (this is xyz [i] = ( In the case of (expressed as 0.17332, 0.00482, 0.82188)), first, the candidate wavelength λ is selected based on the x value. Since there is no wavelength λ having x = 0.17332 in Table 1, the wavelength λ = 400 nm having the closest x value (0.17344) and the wavelength λ = 401 nm having the second closest x value (0.17329). The wavelength λ = 399 nm having the third closest x value (0.173338) is selected as a candidate. Next, among the candidate wavelengths λ = 399 nm, 400 nm and 401 nm, the wavelength λ = 399 nm having the closest y value (0.00481) to y [i] = 0.00482 and the next closest y value (0.00480). The wavelength λ = 400 nm with is narrowed down as a candidate. Further, among the wavelengths λ = 399 nm and 400 nm, the wavelength λ = 400 nm having a z value (0.82186) closer to z [i] = 0.82188 is specified as the wavelength of the pixel i. In addition, those skilled in the art can identify the wavelength λ by calculating the correlation between the xyz value for the acquired pixel i and the xyz value in Table 1 by various methods and selecting the wavelength having the largest correlation. it is obvious.

This wavelength specifying process is performed for all pixels, and the wavelength λ [i] for each pixel is specified. In this embodiment, the spectral chromaticity coordinates in the wavelength range of 380 to 700 nm are used, but when a wavelength outside this range is input, the spectral chromaticity coordinates are used by the above method. Wavelengths in the range of 380-700 nm are specified.

Next, wavelength conversion is performed in step 507. The wavelength conversion converts a wavelength in the visible light wavelength range that cannot be recognized due to a decrease in the user's sensitivity into a wavelength in the visible light wavelength range that can be recognized by the user. In the present embodiment, the wavelength range recognizable by the user is also converted to another wavelength within the wavelength range recognizable by the user, but only the visible light wavelength range in which the sensitivity of the user is reduced may be converted. .. In this embodiment, wavelength conversion is performed using the following wavelength conversion table.

In the wavelength conversion table of Table 2, "pre-conversion wavelength λ (nm)" indicates the wavelength before wavelength conversion, and "post-conversion wavelength λ (nm)" indicates the wavelength after conversion (conversion destination). “X (λ)”, “y (λ)”, and “z (λ)” indicate x (λ), y (λ), and z (λ) values for expressing the wavelength λ after conversion. That is, this wavelength conversion table assigns xyz values for the converted wavelength λ (nm) and the converted wavelength λ (nm) to the pre-conversion wavelength λ (nm). For example, for the wavelength λ (nm) before conversion = 380 nm, 450 nm is assigned as the wavelength λ (nm) after conversion, and the xyz value (0.15664, 0.01771, 0.82565) for expressing the wavelength after conversion 450 nm is assigned. ) Is assigned. The xyz value for expressing the converted wavelength λ (nm) is obtained from the table of spectral chromaticity coordinates in the XYZ color system shown in Table 1. The converted wavelength λ (nm) has been described for the sake of understanding the invention, but may not be included in actual information processing.

In preparing this wavelength conversion table, the converted wavelength of 450 to 600 nm with respect to the wavelength of the visible light wavelength range of 380 to 700 nm is obtained by linear approximation or first-order approximation calculation using the equation (4).

Since the spectral chromaticity coordinates in the XYZ color system shown in Table 1 only indicate the xyz value for each 1 nm, the converted wavelength including the decimal point calculated by the equation (4) is appropriately rounded, rounded down, rounded up, etc. And assign it to the converted wavelength of the integer value. Further, since the spectral chromaticity coordinates in the XYZ color system show xyz values of 450 to 600 nm for each 1 nm, xyz values for 151 wavelengths are shown, whereas visible light wavelength range 380. If xyz is assigned to ~ 700 nm every 1 nm, 321 xyz values are required. Therefore, the number of xyz values shown in the spectral chromaticity coordinates is insufficient. Therefore, in the present embodiment, the xyz value for each wavelength 450 nm to 1 nm shown in the table of spectral chromaticity coordinates in the XYZ color system is assigned and assigned every 1 to 2 nm from the wavelength before conversion of 380 nm by the above-mentioned method. For the wavelengths before conversion that did not exist, the "wavelength after conversion λ (nm)" was left blank, and linear interpolation was performed based on the xyz values assigned to the wavelengths before and after the wavelength. This interpolation method is not limited to the nth-order approximation (n = 1 or more), and other methods may be used.

For example, an xyz value (0.15664, 0.01771, 0.82565) for a post-conversion wavelength 450 nm is assigned to a pre-conversion wavelength 380 nm, and an xyz value for a post-conversion wavelength 451 nm is assigned to a pre-conversion wavelength 382 nm. (0.15560, 0.01861, 0.82579) is assigned, and the pre-conversion wavelength λ381 nm in the meantime is the xyz value (mean value) between the xyz values for the post-conversion wavelengths 450 nm and 451 nm (0. 15612, 0.01816, 0.82572) is assigned.

In this embodiment, the wavelength conversion table is used to convert the wavelength specified in step 507. For example, when the wavelength λ [i] specified in step 506 is 399 nm, first, a row having a wavelength λ (nm) before conversion = 399 nm is searched. Then, the converted wavelength λ of the row identified by the search is 459 nm, and the output (0.14547, 0.02812, 0.82641) assigned as the xyz value for that purpose is output. That is, since the output (0.14472, 0.02891, 0.82638) representing the wavelength 459 nm is output as xyz [i] for the pixel i having the xyz value representing the wavelength 399 nm, the pixel [i] The wavelength is converted from 399 nm to 459 nm.

In step 508, the stimulus sum S [i] for the pixel i is read from the internal storage unit 107 and multiplied by the xyz [i] of all the converted pixels as shown in the equation (5) to perform wavelength conversion. Acquire the later XYZ value. Then, in step 509, the converted XYZ value is RGB-converted based on the equation (6), and in step 510, the converted RGB signal is output to the video presentation unit 102.

After that, in step 403, the image presenting unit 102 presents the image to the user based on the RGB signal output from the wavelength conversion unit 103 in step 507. Gamma correction may be performed if necessary. Similarly, in other embodiments that output RGB signals, gamma correction is performed as necessary.

After presenting the image of one frame by the above-mentioned processing, the process returns to step 402 to perform the wavelength conversion (step 402) and the image presentation (step 403) of the present embodiment for the next frame. In the present embodiment, the wavelength conversion (step 402) and the image presentation (step 403) are performed for each frame, but the image may be presented after the wavelength conversion is performed for a plurality of frames at once.

In this embodiment, wavelengths within the entire visible light wavelength range of 380 to 700 nm are converted into a user-recognizable wavelength range of 450 to 600 nm. FIG. 6 shows a graph showing the spectral chromaticity coordinates of the XYZ color system before and after the conversion. FIG. 6A shows a graph before conversion, and FIG. 6B shows a graph after conversion. Before conversion (FIG. 6 (a)), the original xyz value for expressing that wavelength is assigned to each wavelength, but after conversion (FIG. 6 (b)), each wavelength is assigned. , The xyz value for 450 to 600 nm is assigned almost evenly over 380 to 700 nm instead of the original xyz value for expressing the wavelength. Therefore, it is understood that the shape of the graph of 450 to 600 nm in FIG. 6 (a) is assigned to 380 to 700 nm in FIG. 6 (b) in a form stretched to the left and right. After the conversion, an xyz value for expressing the wavelength of 450 nm is assigned to the wavelength of 380 nm, so that the image presenting unit 102 outputs an output based on the xyz value of 450 nm in the pixel having the input of 380 nm. , Light with a wavelength of 450 nm will be observed by the user.

Therefore, by using this embodiment, the user can see the visible light converted into a wavelength that can be recognized with relatively good sensitivity even when the sensitivity is lowered and there is visible light in a wavelength range that cannot be recognized originally. You will be able to observe as. Furthermore, since the wavelength before conversion is almost linearly assigned to the recognizable wavelength range, it is possible for the user to recognize the expression such as the color relationship between adjacent parts with an expression close to the actual color expression. ..

In the present embodiment, an example of converting a wavelength within the visible light wavelength range of 380 to 700 nm into a wavelength range of 450 to 600 nm from one continuous wavelength range to one continuous wavelength range has been shown. It is also possible to convert from one continuous wavelength range to two or more separated wavelength ranges, or from two or more separated wavelength ranges to another two or more separated wavelength ranges by the same method. It is clear that

[Third Embodiment]
In this embodiment, the wavelength conversion process is different from that of the first and second embodiments. Other points are the same as those of the first and second embodiments. Hereinafter, the wavelength conversion processing different from that of the first and second embodiments will be mainly described.

In the present embodiment, the wavelength conversion step 402 does not perform the XYZ conversion, but performs the wavelength conversion in the RGB signal. Hereinafter, the wavelength conversion step 402 in this embodiment will be described with reference to the flowchart shown in FIG. Similar to the second embodiment, in step 701, the wavelength conversion unit 103 reads one frame from the first video information acquired from the imaging unit 101, and in step 702, the RGB signal of each pixel in this frame is output. get. Next, in step 703, the stimulus sum S [i] of each pixel is calculated based on the RGB signal using the equation (7) derived from the equation (1) and stored in the internal storage unit 107. The sum of stimuli [i] may be calculated by an approximate expression or the like.

Here, R [i], G [i], and B [i] are the R signal value, the G signal value, and the B signal value of the pixel i. Further, r [i], g [i] and b [i] are obtained by dividing R [i], G [i] and B [i] of pixel i by the sum of stimuli S [i]. In step 704, R [i] of each pixel is divided by the stimulus sum S [i] to calculate r [i].

In step 705, it is determined whether or not r [i] of each pixel is equal to or greater than the upper limit value of the r signal, and if it is equal to or greater than the upper limit value, it is converted to the upper limit value. If it is smaller than the upper limit value, the conversion is not performed and the raw r [i] is output as the converted r [i]. The spectral chromaticity coordinates according to the RGB color system used in this embodiment are shown in Table 3 and FIG. 8 (a). The spectral chromaticity coordinates based on the RGB color system in the present embodiment are converted to the spectral chromaticity coordinates based on the XYZ color system formulated by CIE shown in Table 1 and FIG. 6 (a) by the RGB conversion shown in the equation (6). Was created using. The rgb signal values shown in Tables 3 and 8 are values obtained by RGB-converting the normalized xyz signal values shown in the spectral chromaticity coordinates by the XYZ color system. The spectral chromaticity coordinates by the RGB color system may be any as long as the correspondence between the rgb signal value and the wavelength is shown based on the chromaticity coordinates by the XYZ color system. .. Assuming a patient whose visible light wavelength range is 450 to 600 nm due to retinal degenerative disease, even if an RGB signal for expressing a wavelength having a wavelength longer than 600 nm is output, it cannot be recognized. According to FIG. 8A, at wavelengths longer than 600 nm, the R signal is more dominant than the G and B signals. Therefore, in the present embodiment, when r [i] of each pixel i sets the maximum value of the r signal that can be recognized by the user, that is, the r signal value at a wavelength of 600 nm as the r signal upper limit value, and exceeds this. By converting to this r signal upper limit value, it is possible to always output an r signal that can be recognized by the user and prevent light having a wavelength of about 600 nm or more from being presented by the presenting unit 102. The r signal value (1.14888) at a wavelength of 600 nm is acquired from the spectral chromaticity coordinates shown in Table 3 and FIG. 8 (a). The upper limit of the r signal may be smaller than the maximum value of the r signal in the wavelength range recognizable by the user.

Then, in step 706, the converted r [i] is multiplied by the stimulus sum S [i] of the pixel i to obtain the converted R [i]. In step 707, the RGB signal including the converted R signal value is wavelengthed by replacing the R signal value included in the RGB signal of each pixel imaged and output by the imaging unit 101 with the converted R signal value. It is output from the conversion unit 103 to the image presentation unit 102. The G and B signal values are not converted, and the values output from the imaging unit 101 are used. The image presenting unit 102 presents a frame based on the converted RGB signal (step 403).

By such a conversion process, as shown in FIG. 8B, the r signal upper limit value is output as r [i] having a wavelength of 600 nm or more, and a wavelength having a wavelength of 600 nm or more is imaged by the imaging unit 101. However, light having a wavelength of about 600 nm is presented by the image presenting unit.

By using this embodiment, even a user who cannot recognize a wavelength longer than 600 nm can observe a wavelength light having a wavelength longer than 600 nm as a wavelength light having a wavelength of about 600 nm through the image presenting unit 102. In the wavelength range of 600 nm or less, the output is at the original wavelength, so that the user can recognize the expression such as the color relationship between adjacent portions in the actual color expression. Further, in the present invention, it is preferable that the wavelength conversion of the user's field of view can be executed in real time to provide the user's field of view in real time like glasses. However, in a device worn on the user's head such as a head-mounted display, the processing capacity of the hardware may be limited. In this embodiment, since RGB signals are directly processed without undergoing XYZ conversion to realize wavelength conversion, information processing is small and it is possible to present a converted wavelength image in real time even with limited hardware resources. ..

[Fourth Embodiment]
This embodiment is similar to the third embodiment in that the processing of RGB signal values is executed without XYZ conversion, but the specific wavelength conversion processing is different from that of the third embodiment. Hereinafter, the wavelength conversion process different from that of the third embodiment will be mainly described. In the present embodiment, it is assumed that the R signal value, the G signal value, and the B signal value in the RGB signal acquired from the imaging unit 101 do not take negative values.

The specific processing of the wavelength conversion step 402 in the present embodiment will be described with reference to the flowchart shown in FIG. Steps 901 to 903 are similar to steps 701 to 703 in the third embodiment. In step 904, not only R [i] of each pixel but also G [i] is divided by the stimulus sum S [i] to obtain r [i] and g [i]. Then, in step 905, if r [i] is equal to or greater than the r signal upper limit value, it is converted into an r signal upper limit value, and in step 906, if g [i] is equal to or less than the g signal lower limit value, the g signal lower limit value is converted. Convert to.

In the present embodiment, the r signal value (1.114888) and the g signal value (0) at a wavelength of 600 nm are based on the graph of the spectral chromaticity coordinates by the RGB color system shown in Table 3 and FIG. 8 (a). .20837) is determined as the upper limit of the r signal and the lower limit of the g signal, respectively. The upper limit of the r signal is preferably the maximum value of the r signal for a wavelength recognizable by the user, but it may be a value less than that. In the present embodiment, the imaging unit 101 does not return a negative value, so the value that is a negative value in the spectral chromaticity coordinates according to the RGB color system shown in Table 3 and FIG. 8A is read as 0. .. For example, the r signal value in 450 to 500 nm is 0, and the maximum value of the r signal value in this wavelength range is 0, so that the upper limit value of the r signal is 0. The lower limit of the g signal is the larger of the g signal values for the wavelengths at both ends of the wavelength range that can be recognized by the user. In this embodiment, a wavelength range of 450 to 600 nm that can be recognized by the user is assumed. When the g signal values at both ends of this wavelength range are compared, the signal value for the wavelength of 600 nm is larger, so the lower limit of the g signal is set to the value of 600 nm. For example, when the wavelength range is 500 to 600 nm that can be recognized by the user, the g signal value at the wavelength of 500 nm is larger, so the signal value for the wavelength of 500 nm is set as the g signal lower limit value. From FIG. 8A, the graph of the spectral chromaticity coordinates by the RGB color system for the g signal value has an upward convex shape, so that the g signal value is equal to or more than the lower limit value of the g signal as described above. By doing so, it is possible to prevent the presentation of light having a wavelength that the user cannot recognize.

Then, the wavelength conversion unit 103 obtains the converted R [i] and G [i] by multiplying the converted r [i] and g [i] by the stimulus sum S [i] (step 907). ), A converted RGB signal including these is output (step 908). For B [i], use the one that has not been converted. Based on this, the video presentation unit 102 presents the video (step 403).

By converting the r signal value and the g signal value in this way, the graph of the spectral chromaticity coordinates output by the present embodiment is shown in FIG. In the region where the wavelength is 600 nm or more, the r signal and the g signal output a value for a wavelength of 600 nm. In FIG. 8A, the b signal value for a wavelength of 600 nm or more is a negative value, but the imaging unit 101 in the present embodiment outputs 0 for a negative value. Therefore, even when the imaging unit 101 captures light of 600 nm or more, the wavelength conversion of the present embodiment converts it into an RGB signal for 600 nm and outputs it, so that the light of 600 nm is presented.

Further, even in a region where the wavelength is short, if g [i] is equal to or less than the lower limit value of g signal (0.20883), the value of g [i] is converted to the lower limit value of g signal and output. In the present embodiment, the g signal lower limit value is obtained when the wavelength is 479 nm, and g [i] is a value lower than this when the wavelength is shorter than that, so that the g signal lower limit value is converted. In the wavelength range shorter than the wavelength of 479 nm, the r signal value and the b signal value are almost constant values. Therefore, by fixing the g signal value to the lower limit value of the g signal, light having a wavelength of about 479 nm is emitted in this region. Will be done.

In the third embodiment, attention was paid to the fact that the r signal is largely dominant in the wavelength region where the wavelength is longer than 600 nm, and only the r signal is converted. In the present embodiment, the g signal value of 600 nm is output for the wavelength of 600 nm or more for the g signal as well as the r signal. By using this embodiment, even if the imaging unit 101 captures light of 450 nm or less and 600 nm or more, only light having a wavelength of approximately 479 to 600 nm is presented by the image presenting unit 102, so that the light is 450 to 600 nm. Even a user who can recognize only the wavelength range can recognize all visible light. Since it is output at the original wavelength in the wavelength range of 479 to 600 nm, it is possible for the user to recognize the expression such as the color relationship between adjacent portions with an expression close to the actual color expression. Further, as in the third embodiment, the wavelength conversion process can be executed without the XYZ conversion, so that the high-speed process becomes possible.

[Fifth Embodiment]
The present embodiment is similar to the third and fourth embodiments in that the processing of the RGB signal is executed without the XYZ conversion, but the specific wavelength conversion processing is different. Hereinafter, the wavelength conversion processing different from that of the third and fourth embodiments will be mainly described. Also in this embodiment, the R signal value, the G signal value, and the B signal value in the RGB signal acquired from the imaging unit 101 do not take negative values.

Hereinafter, the wavelength conversion step 402 in this embodiment will be described with reference to the flowchart shown in FIG. Steps 1101 to 1104 are similar to steps 901 to 904 in the fourth embodiment. Then, in step 1105, a predetermined calculation is performed on r [i] and g [i] to perform wavelength conversion. In the present embodiment, as an operation for r [i], the r signal conversion coefficient is set so that the maximum value of r [i] after conversion is equal to or less than the maximum r signal value in the wavelength range recognizable by the user. Multiply by [i]. The r signal conversion coefficient is determined so that the maximum value of the value obtained by multiplying the r signal value indicated in the spectral chromaticity coordinates by the RGB color system by this conversion coefficient does not exceed the upper limit value of the r signal. As the calculation for g [i], the calculation is performed so that the minimum value of g [i] after conversion is equal to or greater than the lower limit of the g signal value for the wavelength recognizable by the user. In this embodiment, r [i] and g [i] are converted based on the following equations.

Here, upper_limit_r is the upper limit value of the r signal after wavelength conversion, and peak_r is the maximum r signal value before wavelength conversion. The value obtained by dividing upper_limit_r by peak_r is used as the r signal conversion coefficient. As can be seen from the equation (8), the upper limit of the r signal is the maximum value of r [i] after conversion. The upper limit of the r signal is preferably the maximum value of the r signal for a wavelength recognizable by the user, but it may be a value less than that. In the present embodiment, the upper limit of the r signal is 1.14888, which is the r signal value at a wavelength of 600 nm, peak_r is 1.49990, which is the r signal value at a wavelength of 700 nm, and the r signal conversion coefficient is about 0.766 (1). .14888 / 1.49990). If this r signal conversion coefficient is used, the maximum value of the value obtained by multiplying the r signal value indicated in the spectral chromaticity coordinates by the RGB color system is the r signal upper limit value (1.14888). Similarly, the r [i] of each pixel can be linearly converted to a value having the upper limit of the r signal as the upper limit by multiplying the r signal conversion coefficient shown in the equation (8).

According to FIG. 8A, it is understood that a user who can recognize only the wavelength range of 450 to 600 nm cannot recognize when an r signal value larger than the r signal value for the wavelength of 600 nm is output. Therefore, in the present embodiment, the r signal value at a wavelength of 600 nm is set to upper_limit_r (r signal upper limit value) so that the r signal value for a wavelength of 600 nm becomes the maximum value after conversion, and the maximum r signal value before wavelength conversion is set. The r signal value having a wavelength of 700 nm indicating the above was defined as peak_r (maximum r signal value before conversion). The upper_limit_r and peak_r can be determined from the spectral chromaticity coordinates according to the RGB color system shown in Table 3 and FIG. 8A, and the r signal value conversion coefficient is a coefficient that can be determined in advance.

peak_g is the maximum g signal value before wavelength conversion, and lower_limit_g is the lower limit value of g signal after wavelength conversion, which is an offset value. In the present embodiment, peak_g is a g signal value (1.16241) at a wavelength of 518 nm, and lower_limit_g is a g signal value (0.20837) at a wavelength of 600 nm. The coefficient multiplied by r [i] in the right term of the equation (9) is the g signal conversion coefficient, so that the g signal value after conversion does not exceed the maximum value of the g signal value before conversion and is converted. The shape of the graph of spectral chromaticity coordinates for the later g signal is determined to resemble the shape before conversion. By performing the calculation shown in the equation (9), the upper limit is linearly set to the same value as before the conversion while the lower_limit_g (g signal lower limit) is set as the offset value with g [i] of each pixel as the offset value. It becomes possible to convert. According to FIG. 8 (a), the g-signal value is shown by a graph having an upwardly convex shape, and the g-signal value for the wavelengths at both ends in the wavelength range of 450 to 600 nm that can be recognized by the user is 600 nm. The g signal value for this is larger. Therefore, it is understood that this user can recognize a g signal value equal to or higher than the g signal value for a wavelength of 600 nm. Therefore, in the present embodiment, the g signal value at a wavelength of 600 nm is set as a lower_limit_g (g signal lower limit value) as an offset signal so that the g signal value for a wavelength of 600 nm becomes the minimum value, and the maximum g signal value before wavelength conversion is set. It was decided to perform an operation so as to maintain peak_g of the g signal value having a wavelength of 518 nm.

In step 1106, r [i] and g [i] converted by the equations (8) and (9) are multiplied by the stimulus sum S [i] to obtain the wavelength-converted R and G values, and the step 1106 is performed. In 1107, an RGB signal including these is output to the image presentation unit 102. The B signal value is not converted, and the value output from the imaging unit 101 is used. Since the wavelength conversion operation for r [i] is only multiplication, the sum of stimuli S [i] divided in step 1104 is canceled by the multiplication in step 1106. Therefore, the division and multiplication of the sum of stimuli in steps 1104 and 1106 may be omitted, and the wavelength conversion process may be executed with R [i] as it is.

The calculation method for wavelength conversion is not limited to the above equations (8) and (9). The calculation for r [i] is such that the maximum value of r [i] after conversion does not exceed the upper limit of the r signal that is equal to or less than the maximum value of the r signal that can be recognized by the user. It does not matter what kind of operation it is. For example, if the converted r signal does not exceed the upper limit of the r signal, the r signal conversion coefficient may be set to 2/3 and this may be simply multiplied by r [i]. As the calculation for g [i], if the minimum value of g [i] after conversion is equal to or greater than the above-mentioned lower_limit_g (g signal lower limit value), for example, g signal conversion coefficient = 1 is set. Lower_limit_g may be simply added to g [i], or lower_limit_g may be added after dividing g [i] by peak_g (g signal conversion coefficient = 1 / peak_g). Further, the wavelength conversion can be performed by processing only the R signal value or only the G signal value.

By using this embodiment, it is possible to convert the graph of the spectral chromaticity coordinates into an RGB signal for expressing a wavelength that can be recognized by the user while keeping the shape similar. Therefore, as compared with the embodiment in which the RGB signal value equal to or greater than the r signal upper limit value or less than or equal to the g signal lower limit value is fixed to the upper limit or lower limit value, the expression such as the color relationship between adjacent portions is closer to the actual color expression. Allows the user to recognize it. Further, as in the third and fourth embodiments, high-speed processing is possible because it does not involve XYZ conversion.

[Sixth Embodiment]
The first to fifth head-mounted video devices according to the present embodiment are 3D head-mounted video presentation devices that present images with parallax captured by the imaging unit 101 to each user's eyes. Is different from the embodiment of. In other respects, it is similar to other embodiments. Hereinafter, the points different from those of other embodiments will be mainly described.

As step 401 of FIG. 4, the imaging unit 101 captures an image in the user's visual field direction for presentation to the user's left eye and an image in the user's visual field direction for presentation to the user's right eye, and the left and right eyes. The first video information including the video information with parallax for presenting to is output. The images presented to each of the left and right eyes are images with parallax, and are different from each other. For example, two monocular cameras or stereo cameras arranged side by side may be used to capture image information to be presented to the left and right eyes. Left and right video information may be generated using the parallax calculated by one monocular camera.

As step 402, the wavelength conversion unit 103 performs wavelength conversion processing on each of the video information to be presented to the left and right eyes included in the first video information, and the parallax for presenting to the converted left and right eyes. Outputs the second video information including some video information. The wavelength conversion process may be any of the wavelength conversion processes described in the first to fifth embodiments. The video presentation unit 102 presents the video to the left eye of the user based on the video information to be presented to the left eye included in the second video information, and presents the video to the right eye included in the second video information. The image is presented to the user's right eye based on the image information for the purpose. For example, two displays may be used to present video information for presentation to the left and right eyes. FIG. 1b shows a configuration in which two projector lenses are provided as the image presentation unit 102. In the present embodiment, parallax image information to be presented to the left and right eyes from each lens is presented. By using this embodiment, it is possible to provide a three-dimensional image to the user. When the sensitivity to each wavelength is different for each eye, it is desirable to perform different wavelength conversion suitable for each eye and present an image having different wavelength conversion for each eye. Wavelength conversion adapted to the sensitivity of one eye may be performed on the image information of both eyes.

In the processes or operations described above, the processes, operations and combinations can be freely changed as long as there is no contradiction. Moreover, each embodiment described above is an example for explaining the present invention, and the present invention is not limited to these embodiments. The present invention can be carried out in various forms without departing from the gist thereof. Further, the effects described in the present embodiment merely list the most preferable effects arising from the present invention, and the effects according to the present invention are not limited to those described in the present embodiment.

100 Head-mounted image presentation device 101 Imaging unit 102 Image presentation unit 103 Wavelength conversion unit 104 Processing unit 105 Storage unit 106 Program 107 Internal storage unit 108 Bus 110 Projector housing 111 Prism 112 Prism

Claims (14)

A head-mounted image presentation device for users whose sensitivity to a part of the visible light wavelength range is lower than that of other wavelength ranges of the visible light wavelength range.
An imaging unit that captures the user's field of view and acquires the first video information,
At least a part of the signal for expressing the wavelength for at least a part of the wavelength range in the first video information is wavelength- converted into a signal for expressing the wavelength in the other wavelength range, and the wavelength is converted. A wavelength converter that outputs second video information based on the wavelength signal
An image presenting unit that presents a wavelength-converted image in the user's field of view to the user based on the second image information.
A head-mounted image presentation device equipped with. Sensitivity is lower than other wavelengths in the visible light wavelength range. Some wavelength ranges in the visible light wavelength range are different between the right eye and the left eye.
The wavelength conversion unit executes the wavelength conversion for each of the sensitivity of the left eye and the sensitivity of the right eye, and the second image information includes the image information for the right eye and the image information for the left eye, and the right eye. The video information for the left eye is based on the wavelength signal that has been wavelength-converted based on the sensitivity of the right eye, and the video information for the left eye is based on the wavelength signal that has been wavelength-converted based on the sensitivity of the left eye.
The image presenting unit presents the right eye image to the user's right eye based on the right eye image information included in the second image information, and the user's left based on the left eye image information. Presents a wavelength-converted left-eye image to the eye,
The head-mounted image presentation device according to claim 1. The wavelength conversion unit
A first table containing spectral chromaticity coordinates in the XYZ color system,
In order to convert the wavelength of the part of the wavelength range into the other wavelength range, the chromaticity coordinates associated with the wavelength of the part of the wavelength range in the first table are the wavelengths of the other wavelength range. In the first table, at least a part of the chromaticity coordinates associated with the wavelengths in the other wavelength ranges are set to chromaticity coordinates different from the chromaticity coordinates associated with in the first table. The associated second table and
Including
Based on the first video information, an XYZ signal expressing a color in the XYZ color system is acquired, and a wavelength corresponding to the XYZ signal is specified based on the first table.
Based on the second table, the XYZ signal for the converted wavelength associated with the identified wavelength is obtained.
Outputs the second video information based on the converted XYZ signal.
The head-mounted image presentation device according to claim 1 or 2 . The wavelength conversion unit
Based on the first video information, an RGB signal that expresses a color in an RGB color system is acquired, and an RGB signal is obtained.
The stimulus sum of the XYZ signal is calculated based on the RGB signal,
The R signal value acquired from the RGB signal is divided by the sum of stimuli to obtain the r signal value, and when the acquired r signal value is equal to or greater than the r signal upper limit value, the acquired r signal value is obtained. Is converted to the upper limit of the r signal,
The converted r signal value is multiplied by the stimulus sum to obtain the converted R signal value.
The second video information based on the RGB signal including the converted R signal value is output.
The r signal upper limit value is a value equal to or less than the maximum r signal value in the other wavelength region obtained by RGB conversion of the spectral chromaticity coordinates by the XYZ color system.
The head-mounted image presentation device according to claim 1 or 2 . The wavelength conversion unit
The G signal value acquired from the RGB signal is divided by the sum of stimuli to obtain the g signal value, and when the acquired g signal value is equal to or less than the lower limit value of the g signal, the g signal value is used as the g signal. Convert to the lower limit and
Multiply the converted g signal value by the stimulus sum to obtain the converted G signal value.
The second video information based on the RGB signal including the converted G signal value is output.
The lower limit of the g signal is equal to or greater than the larger g signal value of the g signal values for the wavelengths at both ends in the other wavelength range obtained by RGB-converting the spectral chromaticity coordinates of the XYZ color system.
The head-mounted image presentation device according to claim 4 . The wavelength conversion unit
Based on the first video information, an RGB signal that expresses a color in an RGB color system is acquired, and an RGB signal is obtained.
The r signal conversion coefficient determined based on the spectral chromaticity coordinates obtained by RGB conversion of the spectral chromaticity coordinates by the XYZ color system is converted into the R signal value included in the acquired RGB signal. Obtain the converted R signal value by multiplication,
The second video information based on the RGB signal including the converted R signal value is output.
The r signal conversion coefficient is determined so that the maximum value obtained when the r signal value in the spectral chromaticity coordinates of the RGB color system is multiplied by the r signal conversion coefficient does not exceed the upper limit value of the r signal. ,
The r signal upper limit value is a value equal to or less than the maximum r signal value in other wavelength regions in the spectral chromaticity coordinates of the RGB color system.
The head-mounted image presentation device according to claim 1 or 2 . Instead of acquiring the converted R signal value by multiplying the r signal conversion coefficient by the R signal value included in the acquired RGB signal.
It was converted by multiplying the r signal value obtained by dividing the R signal value included in the acquired RGB signal by the stimulus sum calculated based on the acquired RGB signal by the r signal conversion coefficient. The r signal value is acquired, and the converted r signal value is multiplied by the stimulus sum to obtain the converted R signal value.
The head-mounted image presentation device according to claim 6 . The wavelength conversion unit
The stimulus sum of the XYZ signal is calculated based on the acquired RGB signal,
The G signal value included in the acquired RGB signal is divided by the sum of stimuli to obtain the g signal value.
After multiplying the g signal value by a predetermined coefficient, the g signal lower limit value is added, and the stimulus sum is multiplied to obtain the converted G signal value.
The second video information based on the RGB signal including the converted G signal value is output, and the lower limit value of the g signal is the wavelength at both ends in the other wavelength range in the spectral chromaticity coordinates by the RGB color system. Is greater than or equal to the larger g signal value of the g signal values for
The head-mounted image presentation device according to claim 6 or 7 . The head-mounted video presentation device according to any one of claims 1 to 8 , wherein the video presentation unit is a projector or a display . The imaging unit captures a left-eye image in the user's visual field direction for presentation to the user's left eye and a right-eye image in the user's visual field direction for presentation to the user's right eye, which is different from the left-eye image. Take an image and
The first video information includes the video information for the left eye and the video for the right eye.
The second video information includes left-eye video information for the left-eye video and right-eye video information for the right-eye video that has been wavelength-converted by the wavelength conversion unit.
The image presenting unit presents the wavelength-converted left-eye image to the user's left eye based on the wavelength-converted left-eye image information included in the second image information, and presents the wavelength-converted left-eye image to the user's left eye. A wavelength-converted right-eye image is presented to the user's right eye based on the wavelength-converted right-eye image information included in the image information.
The head-mounted image presentation device according to any one of claims 1 to 9 . The user according to any one of claims 1 to 10 , wherein the user is a retinal degeneration patient and a user who has administered a protein to the eyeball that allows ganglion cells to receive light in the other wavelength range. Head-mounted image presentation device. A signal for expressing a wavelength in the other wavelength range to a user whose sensitivity to a part of the visible light wavelength range is lower than that of the other wavelength range in the visible light wavelength range. A wavelength conversion device that outputs a video signal for presenting a video based on the video.
To express at least a part of a signal for expressing a wavelength in at least a part of the wavelength range in the first video information acquired by imaging the user's visual direction to express a wavelength in the other wavelength range . Wavelength converted to signal,
Outputs the second video information based on the wavelength-converted signal.
Wavelength converter. A signal for expressing a wavelength in the other wavelength range to a user whose sensitivity to a part of the visible light wavelength range is lower than that of the other wavelength range in the visible light wavelength range. A method for outputting a video signal for presenting a video based on a computer.
At least a portion of the signal for representing at least the wavelength of a part of a wavelength region in the first image information obtained by imaging a field of view direction of the user, for expressing a wavelength of the other wavelength region The process of converting the wavelength to a signal and
A step of outputting the second video information based on the wavelength-converted signal, and
How to do it. A program that causes a computer to execute the method according to claim 13 .