JP2017037235A - Binoculars display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To propose a useful binoculars display device.SOLUTION: A binoculars display is provided that comprises binoculars display systems receiving image information from binoculars imaging system with optical axes in parallel, and having an optical axis width equivalent to an optical axis width of the binoculars imaging systems. The binoculars display is configured to provide a display virtual image at a distance equivalent to a reference subject distance of the binoculars imaging system. The binoculars display comprises a diopter adjustment function, and includes an image enlargement function or image contraction function. The binoculars display is provided that comprises: binoculars display system receiving the image information from the binoculars imaging systems with the optical axes in parallel, and having an optical axis width different from that of the binoculars imaging system; and an image shift function based on a difference between the optical axis width of the binoculars imaging system and the optical axis width of the binoculars display system. The binoculars display is provided that comprises: binoculars display systems including an optical axis width adjustment function; and an image shift function in accordance with an optical axis width adjustment in the binoculars display system. The binoculars display is provided that comprises: display optical systems receiving the image information from the binoculars imaging systems with the optical axes in parallel, and being different from the binoculars imaging systems; and an image shift function based on a difference of the display optical system.SELECTED DRAWING: Figure 11

Description

  The present invention relates to a binocular display device.

  Various binocular display devices and their applications have been studied. For example, application as a visual assistance system for visually impaired persons is being studied. The main causes of visual impairment include glaucoma cases, cataracts, night blindness, eye diseases such as age-related macular degeneration, or developmental disorders such as visual impairment in early childhood. A device has been proposed. In this case, the binocular display device is used as an auxiliary display device. As an example, in an eyeglass-type visual enhancement device, image processing is performed on an image of a region corresponding to the user's visual field among images captured by a CCD camera, and this can be visually recognized by a user using a virtual image display device. Proposed. As another example, the image information obtained from the imaging unit is processed and displayed on the display area of the display unit, and the image of the outside world and the processed image of the display area are simultaneously displayed on the display unit by the observer. It has been proposed that can be seen from. (Patent Document 2)

JP 2003-287708 A Japanese Patent No. 4600200

  However, there are many issues to be further studied regarding binocular display equipment.

  In view of the above, an object of the present invention is to propose a more useful binocular display device.

  To achieve the above object, the present invention provides a binocular display including a binocular display system that receives image information from a binocular imaging system having parallel optical axes and has an optical axis width equivalent to the optical axis width of the binocular imaging system. I will provide a.

  According to a specific feature, the binocular display provides a display virtual image at a distance equivalent to the standard subject distance of the binocular imaging system.

  According to another specific feature, the binocular display has a diopter adjustment function.

  According to another specific feature, the binocular display has an image enlargement function or an image reduction function.

  According to another aspect of the present invention, a binocular display system that receives image information from a binocular imaging system with parallel optical axes and has an optical axis width different from the optical axis width of the binocular imaging system, and the binocular imaging system A binocular display having an image shift function based on a difference between an optical axis width and an optical axis width of the binocular display system is provided.

  According to another aspect of the present invention, there is provided a binocular display having a binocular display system having an optical axis width adjustment function and an image shift function according to the adjustment of the optical axis width in the binocular display system.

  According to another aspect of the present invention, the image information is received from a binocular imaging system having parallel optical axes, the display optical system is different from the binocular imaging system, and an image shift function based on a difference between the display optical systems is provided. A binocular display is provided.

  As described above, more useful binocular display equipment is provided.

It is a block diagram which shows the whole structure in Example 1 of the visual assistance system of this invention. Example 1 3 is a basic flowchart illustrating an operation of a central control unit according to the first embodiment. It is a flowchart which shows the detail of step S22 of FIG. 2, and step S24. It is a flowchart which shows the detail of step S26 of FIG. 2, and step S28. It is a flowchart which shows the detail of step S34 of FIG. It is a flowchart which shows the detail of step S14 of FIG. It is a flowchart which shows the detail of the image sudden change mitigation process in Example 2 of the visual assistance system of this invention. (Example 2) It is a block diagram which shows the whole structure in Example 3 of the visual assistance system of this invention. (Example 3) 10 is a basic flowchart illustrating an operation of a central control unit according to a third embodiment. It is a flowchart which shows the detail of step S214 of FIG. It is a block diagram which shows the whole structure in Example 4 of the visual assistance system of this invention. Example 4 FIG. 9 is a schematic cross-sectional view illustrating the basics of eyepiece optical system diopter adjustment of Example 4. FIG. 10 is a schematic diagram illustrating a basic configuration of 3D image capturing and display in Example 4; FIG. 10 is a schematic diagram of a myopic person in Example 4. FIG. 10 is a schematic diagram of a farsighted person in Example 4. FIG. 10 is a schematic diagram of short-distance object points in Example 4. FIG. 10 is a schematic diagram of a long-distance object point in Example 4. FIG. 10 is a schematic diagram for enlarging a display image in Example 4. FIG. 10 is a schematic diagram of display image reduction in the fourth embodiment. It is a cross-sectional schematic diagram of Example 5 of the present invention. (Example 5) FIG. 21 is a schematic sectional view of Example 6 of the present invention. (Example 6) FIG. 22 is a schematic sectional view of Example 7 of the present invention. (Example 7)

  FIG. 1 is a block diagram showing an overall configuration in Example 1 of a visual assistance system according to an embodiment of the present invention. The visual assistance system according to the first embodiment includes a goggle type head mounted display (hereinafter, “HMD”) 2 and a controller 4 connected to the goggle type head mounted display (hereinafter, “HMD”). The cable serves as a parallel data communication line and a power supply line for the HMD 2 and the controller 4.

  The HMD 2 is put in front of normal glasses 10 put in front of the user's right eye 6 and left eye 8. As described above, the HMD 2 is used on the assumption that the problem of refraction of the user's right eye 6 and left eye 8 is solved by the glasses 10. For this purpose, the HMD 2 includes a main body portion 2a and a temple portion 2b. When the temple portion 2b is put on the ear from above the glasses 10, the main body portion 2a is arranged in front of the lens of the eyeglasses 10.

  Both the right-eye display 12 and the left-eye display 14 of the HMD in the main body 2a are configured by an OLED (organic light emitting diode) display panel using an organic EL phenomenon. The drive unit 16 drives the right-eye display 12 and the left-eye display 14 based on image signals sent from the controller 4 as described later, and displays a right-eye image and a left-eye image. The virtual image of the displayed image is guided to the right eye 6 and the left eye 8 by the right eyepiece optical system 18 and the left eyepiece optical system 20 along the lines of sight 6a and 8a, as indicated by broken line arrows. The drive unit 16 also adjusts the focus of the right-eye eyepiece optical system 18 and the left-eye eyepiece optical system 20 based on the control of the controller 4, and moves the optical axis in parallel to shift the line of sight from the line of sight 6a or 8a. Shift adjustment is also performed.

  The right-eye image pickup element 22 in the main body 2a has a right-eye bending zoom lens optical system 24 that bends light incident along the line of sight 6a of the right eye 90 degrees inward (right side in the drawing) as indicated by a broken arrow. By this, a real image of the object scene is formed. Similarly, the left-eye image pickup device 26 is covered by a left-eye bending zoom lens optical system 28 that bends light incident along the line of sight 8a of the left eye 90 ° inward (left side in the drawing) as indicated by a dashed arrow. A real image of the scene is formed. The object scene images picked up by the right-eye image pickup device 22 and the left-eye image pickup device 26 are sent to the controller 4 via the drive unit 16 as described later. Due to the zooming function of the right-eye bent zoom lens optical system 24 and the left-eye bent zoom lens optical system 28, the right-eye image pickup device 22 and the left-eye image pickup device 26 not only have the same magnification image but also the actual image. It is possible to form an image in which the field is enlarged or an image in which the wide area of the actual scene is aggregated. The former is suitable for magnifying observation, and the latter is suitable for providing a field image in the field of view that is wider than it would actually look for a user with field stenosis.

  In the imaging system using the right-eye bent zoom lens optical system 24 and the left-eye bent zoom lens optical system 28, the thickness in the direction of the incident optical axis is reduced so that the main body 2a does not protrude excessively forward. Further, the right-eye bent zoom lens optical system 24 and the left-eye bent zoom lens optical system 28 occupy a space in the direction perpendicular to the lines of sight 6a and 8a, respectively, and the right-eye image further inside the bending direction. An imaging element 22 and a left-eye image imaging element 26 are disposed. This arrangement avoids the arrangement of parts that block the outside of the lines of sight 6a and 8a, so that the actual field outside the lines of sight 6a and 8a can be seen directly. Although it is said that the human eye can recognize information on a wide-angle field of about 200 degrees, the information on the field displayed on the right-eye display 12 and the left-eye display 14 in Example 1 is about 40. Degree. Therefore, in the first embodiment, visual information can be obtained by directly viewing the object scene outside the image information displayed on the right-eye display 12 and the left-eye display 14.

  With the above-described configuration, the visual assistance system according to the first embodiment sends the object scene image captured by the right-eye image imaging element 22 and the left-eye image imaging element 26 to the controller 4 for processing according to the user's symptoms. By displaying the processed image returned from the controller 4 on the right-eye display 12 and the left-eye display 14 so as to be observable, it is possible to obtain better visual information than when the object scene is viewed directly. For example, a user with night blindness (dark adaptation disorder) is provided with a processed image in which the gain is increased and the dark portion is raised by gamma correction. On the other hand, a processed image in which a high-intensity portion is compressed by gamma correction is provided to a user of dawn (light adaptation disorder). In order to improve the legibility of characters and the like, it is also possible to provide a black and white inverted image. Furthermore, visual information can be obtained by directly viewing the object scene around the image information displayed on the right-eye display 12 and the left-eye display 14 as described above. As will be described later, the direct image is harmonized with the display image by controlling the transmittance.

  As described above, in the first embodiment of the present invention, in the normal state, the optical axis of the right-eye eyepiece optical system 18 that guides the virtual image of the right-eye display 12 to the right eye 6 is incident on the right-eye bent-smooth lens optical system 24. It coincides with the optical axis. Similarly, the optical axis of the left-eye eyepiece optical system 20 that guides the virtual image of the left-eye display 14 to the left eye 8 coincides with the incident optical axis of the left-eye bent smooth lens optical system 28. If necessary, the optical axis of the right-eye eyepiece optical system 18 or the left-eye eyepiece optical system 20 can be moved in parallel by the control of the drive unit 16 so as to be shifted from the line of sight 6a or 8a. This is because, for example, when there is a disturbance in central vision due to age-related macular degeneration or the like, the image of the center of the field imaged by the right-eye image pickup device 22 or the left-eye image pickup device 26 is other than the central portion of the retina where there is no failure. This is to shift the line of sight so that it can be seen.

  Further, as described above, the first embodiment of the present invention is configured so that the actual field outside the lines of sight 6a and 8a can be directly seen as the background. A right-eye variable transmittance ND filter 33 is provided in the optical path from the outside of the right eye indicated by the arrow 30. The right-eye variable transmittance ND filter 33 is constituted by, for example, a liquid crystal shutter, and the transmittance is variable between the maximum transmittance and the light-shielded state under the control of the driving unit 16. Similarly, a left-eye variable transmittance ND filter 36 is provided in the optical path from the outside of the left eye indicated by the arrow 34, and the transmittance is controlled between the maximum transmittance and the light-shielded state under the control of the driving unit 16. It is variable. Thus, the transmittance of the right-eye variable transmittance ND filter 32 and the left-eye variable transmittance ND filter 36 can be changed independently of each other. The change in transmittance is used to assist the pupil's adaptability to changes in ambient brightness, and the display is made easier to see by changing the transmittance according to the change in brightness of the display. It is used to achieve harmony with direct observation images. Further, when black and white reverse display is performed on the display unit, the right-eye variable transmittance ND filter 32 and the left-eye variable transmittance ND filter 36 are set in a light-shielded state so as not to interfere with black-and-white reverse image observation.

  As is clear from FIG. 1, all the components in the first embodiment described above are housed in the main body 2a, and there is no portion protruding from the front surface of the main body 2a. Therefore, when viewed from a person facing the user wearing the HMD2, the HMD2 is felt to be similar to normal sunglasses, and the uncomfortable feeling that it is observed with a special device is reduced.

  The drive unit 16 in the main body 2a of the HMD 2 is connected to the controller 4 through parallel data communication and a power supply line 38, and performs mutual communication and power supply from the controller 4 to the HMD 2. In addition, in order to change the image processing provided to the right-eye display 12 and the left-eye display 14 according to the brightness of the environment, an ambient light sensor 40 is provided on the vine portion 2b of the HMD 2 and the communication line 42 is used to Light information is sent to the controller 4. Further, an acceleration sensor 44 is provided in the temple portion 2b to alleviate a sudden change in the image especially when the face is changed, and the information such as the movement of the face is sent to the controller 4 through the communication line 46. Yes. The parallel data communication and power supply line 38 and the communication lines 42 and 46 are actually combined into one connection cable. 1 shows a configuration in which the ambient light sensor 40, the acceleration sensor 44, and the biological sensor 45 directly communicate with the controller. However, the parallel light communication and the power supply line 38 communicate with each other via the drive unit 16. It may be configured.

  The controller 4 has an input / output unit 48 for communicating with the HMD 2 and supplying power to the HMD 2 as described above. The image processing unit 50 of the controller 4 processes the image received by the supply line 38 via the drive unit 16 from the image sensor 22 for the right eye image and the image sensor 26 for the left eye image of the parallel data communication and power supply HMD 2, and Is sent to the display control unit 52 as image data suitable for assistance. Image data from the display control unit 52 is transmitted by parallel data communication and the power supply line 38, and the drive unit 16 drives the right-eye display 12 and the left-eye display 14 based on the received image data to display an image. The background control unit 54 controls the eye variable transmittance ND filter 32 and the left eye variable transmittance ND filter 36 through parallel data communication and the power supply line 38.

  The preset storage unit 56 stores image processing information corresponding to individual symptoms of the user and ambient light, and preset values of transmittance information in the variable transmittance ND filter. The operation unit 58 performs the preset value input operation and the image processing selection operation such as black and white reversal in cooperation with the display of the controller display unit 60. The central control unit 62 controls the image processing unit 50 by adding the operation of the operation unit 58 and the information from the ambient light sensor 40 and the acceleration sensor 44 to the image processing information of the preset storage unit 56. Further, the central control unit 58 controls the background control unit 50 by adding information from the ambient light sensor 40 to the transmittance information in the preset storage unit 56. The control data of the background control unit 50 is transmitted by the parallel data communication and the power supply line 38, and based on the data received by the driving unit 16, the transmittances of the right-eye variable transmittance ND filter 32 and the left-eye variable transmittance ND filter 36 are set. Vary the background brightness directly observed. The central control unit 62 further controls the display control unit 52 and the controller display unit 60 in relation to the above functions. The power supply unit 64 supplies power to the entire controller and also supplies power to the HMD 2 via the input / output unit 48.

  FIG. 2 is a basic flowchart for explaining the operation of the central control unit 62 in the first embodiment. The flow starts when power supply to the system is started, and it is checked in step S2 whether a preset value is stored. If there is a memory, the preset value is read from the preset memory 56, and the process proceeds to step S6. On the other hand, if the preset value is not stored in step S2, the process proceeds to step S8, a default value indicating that no correction is performed in the image processing is read, and the process proceeds to step S6.

  In step S6, imaging by the right-eye image imaging device 22 and the left-eye image imaging device 26 is started, and the process proceeds to step S10, where the standard state display control based on the predetermined brightness and the standard state background corresponding thereto are performed. Then, transmittance control of the right-eye variable transmittance ND filter 32 and the left-eye variable transmittance ND filter 36 is started.

  Next, in step S12, it is checked whether or not an operation for setting a preset value has been performed. If it is confirmed that the setting operation has been performed, the process proceeds to step S14 to perform a preset value setting process, and the process proceeds to step S16. . On the other hand, when the preset value setting operation is not confirmed in step S12, the process directly proceeds to step S16. Details of the preset value setting process in step S14 will be described later.

  In step S16, it is checked whether or not the fact that the user has a central visual field defect is stored as a preset value. If so, the process proceeds to step S16 to perform a line-of-sight shift process, and the process proceeds to step S20. On the other hand, when it is confirmed in step S16 that the user does not correspond to the central visual field disorder, the process directly proceeds to step S20. At this time, the normal state is reached, and the optical axes of the right-eye eyepiece optical system 18 and the right-eye eyepiece optical system 20 are the same as the incident optical axes of the right-eye bent-summed lens optical system 24 and the left-eye bent-summed lens optical system 26, respectively. Will match.

  In step S16, it is checked whether or not the brightness of the ambient light has changed. If there is a change, the right-eye display change process in step S22 and the right-eye display change process in step S24 are sequentially executed to reach step S26. . In this way, the display change processing for the right eye and the left eye is performed independently. In step S26, the right eye background change process is performed, and then the left eye background change process of step S28 is executed, and the process proceeds to step S30. In this way, independent processing is performed for the right-eye and left-eye background change processing. On the other hand, when there is no change in ambient light in step S20, the process directly proceeds to step S30.

  In step S30, it is checked whether or not a black and white reversal operation has been performed. If there is an operation, the flow proceeds to step S32 to perform black and white reversal display and the background is shifted to step S34 in a light-shielding state so as not to prevent observation of black and white reversal display. . On the other hand, when the black and white reversal operation cannot be performed in step S30, the process directly proceeds to step S34. In step S34, it is checked whether the power is being supplied. If the power is being supplied, the process returns to step S12. Thereafter, as long as it is confirmed that the power is being supplied in step S36, steps S12 to S36 are repeated. On the other hand, if it is not confirmed in step S36 that power is being supplied, the flow is immediately terminated.

  FIG. 3 is a flowchart showing details of the right-eye display change process in step S22 and the left-eye display change process in step S24 of FIG. 2, and the contents are common to both steps, but for the right eye as shown in FIG. And for the left eye respectively. When the flow starts, the process proceeds to step S42, and it is checked whether or not the change in ambient light detected in step S20 in FIG. 1 is greater than or equal to a predetermined value that is predetermined for display change processing. If the change is equal to or less than a predetermined value that does not require display change processing, the flow is immediately terminated, and the process proceeds to step S26.

  On the other hand, when a change greater than or equal to the predetermined value is detected in step S42, the process proceeds to step S44, and it is checked whether or not the ambient light has increased beyond a predetermined value due to the change. If it is detected in step S44 that the ambient light has increased above the predetermined value, the gain of the image sensor is decreased in step S46, and the process proceeds to step S48. In step S48, it is checked whether or not the user has a light adaptation disorder. If yes, the process proceeds to step S50 to perform gamma correction for compressing the high-intensity part, and the process proceeds to step S52. In step S52, contour enhancement processing is further performed, and the process proceeds to step S54. On the other hand, if it is not detected in step S44 that the ambient light has increased above the predetermined value, or if it is not confirmed in step S48 that the user is a light adaptation disorder, the process proceeds directly to step S54.

  In step S54, it is checked whether ambient light has decreased below a predetermined value. When it is detected that the ambient light has decreased below the predetermined value, the gain of the image sensor is increased in step S56, and the process proceeds to step S58. In step S58, it is checked whether or not the user has a dark adaptation failure, and if applicable, the process proceeds to 60 to perform gamma correction for lifting the low luminance part, and the process proceeds to step S62. In step S62, contour enhancement processing is further performed, and the process proceeds to step S64. On the other hand, if it is not detected in step S54 that the ambient light has decreased below the predetermined value, or if it is not confirmed in step S58 that the user is a dark adaptation disorder, the process directly proceeds to step S64.

  In step S54, a counter for correcting display change according to the time when the pupil reacts to the change in brightness is reset and started, and the process proceeds to step S66. In step S66, it is checked whether or not the pupil response correction based on the previous brightness change is being performed. If yes, the process proceeds to step S68, the previous pupil response correction is canceled, and the process proceeds to step S70. On the other hand, if it is not detected in step S66 that the previous pupil reaction correction is being performed, the process proceeds directly to step S70. In step S70, pupil response correction is started, and processing for automatically ending pupil response correction is started when the pupil response is completed based on the counter, and the flow ends.

  FIG. 4 is a flowchart showing details of the right-eye background changing process in step S26 and the left-eye background changing process in step S28 of FIG. 2, and the contents are common to both steps, but for the right eye as shown in FIG. And for the left eye respectively. When the flow starts, the process proceeds to step S82, and it is checked whether the ambient light has increased beyond a predetermined value due to the change in ambient light detected in step S20 in FIG. Normally, the predetermined value in step S82 in FIG. 4 is lower in level than the predetermined value in step S44 in FIG.

  If it is detected in step S82 that the ambient light has increased above the predetermined value, the process proceeds to step S84, and the transmittance of the variable transmittance ND filter is decreased corresponding to the increase in ambient light, and the process proceeds to step S86. In step S86, it is checked whether or not the display portion change processing has been performed based on the current ambient light change. If yes, the process proceeds to step S88 to change the transmittance of the variable transmittance ND filter in response to the display portion change. Then, step S90 is reached. On the other hand, if it is not detected in step S82 that the ambient light has increased above the predetermined value, or if it is not confirmed in step S86 that the display unit changing process has been performed, the process proceeds directly to step S90.

  In step S90, it is checked whether or not the ambient light has decreased below a predetermined value due to the change in ambient light detected in step S20 of FIG. Usually, the predetermined value in step S90 of FIG. 4 has a higher level than the predetermined value in step S54 of FIG. When it is detected in step S90 that the ambient light has decreased below the predetermined value, the process proceeds to step S92, and it is checked whether or not the transmittance of the variable transmittance ND filter has already been maximized. If it is not the maximum value, the process proceeds to step S94, the transmittance of the variable transmittance ND filter is increased corresponding to the increase in the ambient light, and the process proceeds to step S96. However, this increase is limited by the maximum transmittance. On the other hand, if it is detected in step S92 that the transmittance of the variable transmittance ND filter has already been maximized, the process proceeds directly to step S96.

  In step S96, it is checked whether or not the display portion changing process has been performed based on the current ambient light change. If applicable, the process proceeds to step S98 to change the transmittance of the variable transmittance ND filter in response to the display portion change. Then, the process proceeds to step S100. However, this increase is limited by the maximum transmittance. On the other hand, if it is not detected in step S90 that the ambient light has decreased below the predetermined value, or if it is not confirmed in step S96 that the display unit changing process has been performed, the process proceeds directly to step S90.

  In step S100, it is checked whether or not there is a pupil response correction process started for display change in FIG. 3, and if applicable, the process proceeds to step S102 to start the transmittance correction of the corresponding variable transmittance ND filter. In response to the trend response correction for display change, a process for automatically ending the correction is started and the flow is ended.

  FIG. 5 is a flowchart showing details of the sudden image change mitigation process in step S34 of FIG. When the flow starts, it is checked in step S112 whether the display magnification is equal to or larger than the same magnification. If this is not the case, that is, if the display magnification is the same as or lower than the background magnification and the face orientation is not changed, etc. Then, the process proceeds to step S36.

  On the other hand, when it is detected in step S112 that the display magnification is equal to or larger than the same magnification, the process proceeds to step S114, and it is detected whether or not an acceleration based on the change of the face orientation is detected. When acceleration is detected, the process proceeds to step S116, the display of the previous frame on the right-eye display 12 and the left-eye display 14 is maintained, and the process proceeds to step S118. In step S118, it is checked whether or not a predetermined time corresponding to the display magnification (for example, a time corresponding to 3 frames at a display magnification of 2) has elapsed. If no time has elapsed, the process returns to step S116, and thereafter, step S116 and step S118 are repeated until the time elapse is detected in step S118, and the previous frame is maintained. On the other hand, when the passage of time is detected in step S118, the process proceeds to step S120.

  In step S120, acceleration detection is performed again. When acceleration is no longer detected due to the stop of the movement of the face, the process proceeds to step S122, the next frame of the frame maintained in step S116 is displayed, and the process proceeds to step S126. . The next frame display in step S122 is performed at a frame rate faster than usual. In step S126, it is checked whether or not the current frame has been displayed. If the current frame has not been caught, the process returns to step S120. Thereafter, unless a new acceleration is detected in step S120 and the current frame cannot be caught in step S126, steps S120 to S126 are repeated, and return to the current frame is performed at a frame rate faster than the normal frame rate. Then, when it is detected in step S126 that the current frame is displayed and the camera is in the saddle state, the flow is terminated. This alleviates sudden changes in the image when the face direction is changed in a state where the magnification is large, and delays the movement of the image. This delay is quickly recovered when the facial movement stops. On the other hand, when the acceleration is detected in step S120 and the facial motion continues, the process proceeds to step S124, the current frame is displayed, and the flow is ended. Therefore, when the movement of the face continues, the sudden change of the image is reduced in a form that is thinned out of the frame.

  FIG. 6 is a flowchart showing details of the preset value setting process in step S14 of FIG. When the flow starts, it is checked in step S132 whether the setting is made by a doctor. If it corresponds, it progresses to step S134, a doctor setting process is performed, and it transfers to step S136. On the other hand, if it is not detected in step S132 that the setting is made by the doctor, the process directly proceeds to step S136. In step S136, it is checked whether or not the setting is made by a vision trainer. If it corresponds, it progresses to step S138, a vision trainer setting process is performed, and it transfers to step S140. On the other hand, if it is not detected in step S136 that the setting is made by the vision trainer, the process proceeds directly to step S140.

  In step S140, it is checked whether the setting is made by the user himself / herself. If applicable, right eye setting is started in step S142, and ambient light initial setting is performed in step S144. In the personal setting, the user actually wears the HMD 2 and observes the right-eye display 12 to determine whether the setting is appropriate. Specifically, the display correction parameter is changed by the person in step S146. In step S148, the person's judgment is obtained as to whether or not the image observed by the right-eye display 12 is optimal. If it cannot be determined that it is optimal, the process returns to step S146, and thereafter, step S146 and step S148 are repeated to repeat the parameter change and the person's determination. Then, when the optimum determination is made by the user in step S148 and the operation unit 58 is operated, the process proceeds to step S150, the parameters in that state are stored, and the process proceeds to step S152.

  In step S152, it is checked whether the parameter storage stored as described above has reached a predetermined number of times. If the accumulated storage has not reached the predetermined number in Step S152, the process returns to Step S146, and Steps S146 to S152 are repeated until the accumulated storage reaches the predetermined number. On the other hand, when the accumulated storage reaches a predetermined number in step S152, the process proceeds to step S154, and the set parameter is determined by averaging the stored parameters.

  Next, in step S156, the ambient light is automatically changed for the purpose of setting, and the process proceeds to step S158. In step S158, it is checked whether or not the ambient light changing process has been completed. If the change process has not ended, the process returns to step S146, and thereafter, unless the ambient light change is ended, steps S146 to S158 are repeated, and the setting for the right eye is continued. On the other hand, when the ambient light changing process ends in step S158, the process proceeds to the left eye setting process in step S160. The details of the left-eye setting process in step S160 are the same as those in the right-eye setting process in steps S146 to S158, but are collectively shown in step S160 to avoid complication. When the left eye setting process in step S160 ends, the flow ends, and the process proceeds to step S16 in FIG. On the other hand, if it is not detected in step S140 that the setting is made by the user, the flow is immediately terminated.

  The implementation of the various features shown in the first embodiment is not limited to the first embodiment, and other embodiments can be implemented as long as the advantages can be enjoyed. For example, in the flow of FIG. 3, contour enhancement is performed when the user has a light adaptation disorder, and contrast enhancement is performed when the user has a dark adaptation disorder. It is possible to employ contour enhancement and contrast enhancement regardless of whether there is a light adaptation disorder or a dark adaptation disorder.

  FIG. 7 is a flowchart showing the details of the sudden image change mitigation process in Example 2 of the visual assistance system according to the embodiment of the present invention. The overall configuration of the second embodiment is the same as the block diagram of the first embodiment in FIG. The basic operation is the same as the basic flowchart of the first embodiment shown in the figure. Therefore, about a common part, Example 1 is used and description is abbreviate | omitted. The second embodiment differs from the first embodiment in the specific configuration of the image sudden change mitigation process in step S34 of FIG. The flowchart of FIG. 7 shows the details of step S34 of FIG.

  When the flow of FIG. 7 is started, whether or not an acceleration of a predetermined amount or more is detected is chucked in step S162. If acceleration is detected, the process proceeds to step S164, and it is checked whether or not a predetermined time (for example, 2 seconds) has elapsed since the acceleration was first detected in a series of acceleration detection. If the predetermined time has not elapsed, the process proceeds to step S166, a series of acceleration detection history analysis is performed, and the process proceeds to step S168. In step S168, whether or not a predetermined time (for example, 0.5 seconds) has elapsed since the analysis was started is checked. If the predetermined time has not elapsed, the process returns to step S166, and thereafter, the analysis is continued by repeating steps S166 and S168 until the predetermined time elapses.

  On the other hand, when it is confirmed that a predetermined time has elapsed since the analysis was started in step S168, the process proceeds to step S170, and whether or not a series of acceleration changes detected as a result of the history analysis corresponds to minute vibrations. Is checked. When the minute vibration is detected, the process proceeds to step S172, the display of the previous frame is maintained, and the display change is stopped. This is to prevent image blurring due to minute vibrations caused by unintentional body tremors. In step S174, it is checked whether or not acceleration in the same direction is detected.

  If the acceleration in the same direction is not detected in step S174, the process returns to step S164, and steps S164 to S174 are repeated unless it is detected in step S164 that a predetermined time has elapsed since the acceleration was first detected. On the other hand, when acceleration in the same direction is detected in step S174, it is determined that an intentional operation such as changing the orientation of the face has been performed, and the display is changed from the change stop state to the current frame. The process proceeds to S178. If it is determined in step S170 that it is not a minute vibration as a result of history analysis, the process immediately proceeds to step S178.

  In step S178, the current display magnification is confirmed, and the process proceeds to step S180 to check whether the display magnification is equal to or larger than the current magnification. If it is equal to or greater than the same magnification, in step S182, the magnification dependent frame dropping is instructed, and the process proceeds to step S184. In the magnification-dependent frame dropping in step S182, for example, the frame rate is reduced to half when the magnification is 1.5 times, and the frame rate is reduced to one third when the magnification is 2 times. The higher the magnification is, the lower the frame rate is so that the image does not move finely in a short time, thereby preventing video sickness caused by the enlarged image. On the other hand, if it is determined in step S180 that the image is not equal to or larger than the same size, the process proceeds to step S186 to instruct display at the normal frame rate, and the process proceeds to step S184.

  In step S184, the presence / absence of acceleration detection is detected again. If acceleration is subsequently detected, the process returns to step S164, and unless it is detected in step S164 that a predetermined time has elapsed since the acceleration was first detected, step S164 is performed. To repeat step S188. On the other hand, if it is confirmed in step S184 that no acceleration is detected, the process proceeds to step S188 to instruct the display of the current frame, and the flow ends.

  If it is detected in step S164 that a predetermined time has elapsed since the acceleration was first detected, the flow is immediately ended even if the acceleration detection is continued. This is to prevent other tasks in FIG. 2 from being unable to be executed due to staying in the flow of FIG. 7 for a long time. As apparent from FIG. 2, if there are no other tasks, the process from step S12 to step S32 is repeated. In step S34, the flow in FIG. 7 is repeated, and if there is acceleration detection, the corresponding function in FIG. 7 can be continued. If no acceleration is detected in step S162, the process proceeds to step S190, the display at the normal frame rate is instructed, and the flow ends. In this case, substantially no operation is performed in the flow of FIG. 7, but step S190 is set for the case where acceleration is not detected because the process reaches step S162 in a state other than the normal display state.

  FIG. 8 is a block diagram showing an overall configuration in Example 3 of the visual assistance system according to the embodiment of the present invention. Since the configuration of the third embodiment in FIG. 8 is similar to that of the first embodiment in FIG. 1, the same parts are denoted by the same reference numerals and the description thereof is omitted. The first point that the third embodiment differs from the first embodiment is that a pulse sensor 66 for detecting a pulse is provided in the temple portion 2b, and information such as whether the user is in a resting state or in an active state such as walking. Is sent to the controller 4 via the communication line 68. Similar to the first embodiment, the parallel data communication and power supply line 38 and the communication lines 42, 46, and 68 are actually combined into one connection cable. 3 also shows a configuration in which the ambient light sensor 40, the acceleration sensor 44, and the biological sensor 66 directly communicate with the controller 4. However, each sensor and the controller 4 perform parallel data communication via the drive unit 16. The power supply line 38 may be used for communication.

  The second point in which the third embodiment shown in FIG. 8 differs from the first embodiment shown in FIG. 1 is that a line-of-sight sensor is provided in the HMD 2 and detects the movement of the line of sight of the user. Information such as the movement of the user's line of sight detected by the line-of-sight sensor 70 is sent to the controller 4 via the drive unit 16 by parallel data communication. Details of the pulse sensor 66 and the line-of-sight sensor 70 will be described later. The ambient light sensor 40, the acceleration sensor 44, the pulse sensor 66, the line-of-sight sensor 70, and the like in the third embodiment are collectively referred to as a situation sensor because they detect the situation when the HMD 2 is used.

  The third point in which the third embodiment shown in FIG. 8 is different from the first embodiment shown in FIG. It is a point that is registered together with a situation sensor for detecting a situation to be detected and learning information on the operation relationship of the operation unit 58. Then, in cooperation with the central control unit 74, the mode selection by the operation unit 58 is limited, or the registered mode is automatically selected. These details will also be described later.

  FIG. 9 is a basic flowchart for explaining the operation of the central control unit 74 in the third embodiment. Since the flow of FIG. 9 is often in common with the flow of the diagram in the third embodiment, common steps are denoted by the same step numbers and description thereof is omitted, and common step groups are also illustrated together. The description is omitted. That is, the start-up process in step S192 is a combination of steps S2, S4, and S8 in FIG. 2, step S194 is a combination of steps S12 to S18 in FIG. 2, and step S194 is a combination of steps S22 in FIG. Step S28 is summarized.

When the display / view change process in step S196 is followed to step S198, it is determined whether or not there has been a manual mode registration operation for registering an enlargement mode other than the normal mode, a wide mode, a black and white reversal mode, and the like. To check. When there is a mode registration operation, the process proceeds to step S200, a mode manual registration process according to the operation is executed, and the process proceeds to step S202. On the other hand, when the mode registration operation is not detected in step S198, the process directly proceeds to step S202.

  In step S202, it is checked whether or not a manual operation for selecting a mode has been performed. If there is a mode selection operation, the process proceeds to step S204 and the subsequent steps, and the state when the operation is performed is detected by each sensor. Specifically, ambient light is detected by the ambient light sensor 40 in step S204, acceleration is detected by the acceleration sensor 44 in step S206, and a pulse is detected by the pulse sensor 66 in step S208. In step S210, the line-of-sight sensor 70 detects the movement state of the line of sight.

  Next, in step S212, a mode automatic learning registration process is performed in which the detection state of each sensor is learned when the mode is manually selected, and the mode is automatically registered. In this process, for example, when the enlargement mode and the black and white mode are selected, if there is a rest state without acceleration detection and a rest state as viewed from the pulse, and the movement of the line of sight is limited, In the state, it is learned that the enlargement mode and the monochrome mode are selected, and these detection states are registered in the enlargement mode and the monochrome mode. In other words, in such a state, the user interprets and registers as having selected the enlargement mode and the monochrome mode for reading or viewing documents.

  The registration information by the mode automatic learning process in step S212 is used for cross-checking validity when automatically selecting a general mode as described later, and for establishing a specific learning result condition specific to the user. Used for automatic custom mode setting based. When the mode automatic learning process in step S212 ends, the process proceeds to step S214. On the other hand, if no mode selection operation is detected in step S202, the process directly proceeds to step S214. In step S214, mode change processing is executed, details of which will be described later. Step S34 and subsequent steps are the same as those in FIG.

  FIG. 10 is a flowchart showing details of the mode change process in step S214 of FIG. When the flow starts, it is checked in step S222 whether or not a manual operation for changing to the wide mode has been performed. If this operation is not detected, the process advances to step S224 to check whether or not a manual operation for changing to the enlargement mode has been performed. When a manual operation for changing to the enlargement mode is detected, the process proceeds to step S226, and it is checked whether or not an acceleration corresponding to the movement of the user is detected. If no acceleration is detected, the process proceeds to step S228, the magnification change according to the operation is performed (in this case, the magnification in the “enlargement mode”), and the process proceeds to step S230.

  On the other hand, when an acceleration corresponding to that the user is moving is detected in step S226, the process proceeds to step S232, and a notification display that enlargement is impossible is displayed on the controller display unit 60 to change the magnification. It transfers to step S230, without performing. Note that the message display indicating that enlargement is not possible may be superimposed on one of the images of the right-eye image sensor 22 and the left-eye image sensor 26.

  If it is detected in step S222 that a manual operation for changing to the wide mode has been performed, the process directly proceeds to step S228, and the magnification change according to the operation (in this case, the “wide mode” magnification) is executed. The process proceeds to step S230. In this way, if the operation is to reduce the magnification, even if the user is moving, there is little possibility of danger, so the acceleration is not detected and the operation is executed immediately. If the enlargement mode operation is not detected in step S224, it means that there is no manual operation for changing the magnification, and the process immediately proceeds to step S230.

  In step S230, it is confirmed whether or not the user is a visual field constrictor. If it is not a visual field constriction person, the process proceeds to step S236, the same size display is set to the standard mode, and the process proceeds to step S236. On the other hand, if it is confirmed in step S230 that the user is a visual field constrictor, the process proceeds to step S238, the wide display is set to the standard mode, and the process proceeds to step S236.

  Steps 236 and thereafter relate to automatic mode change. First, in step S236, it is checked based on the acceleration sensor 44 whether or not the user is in a stationary state. If it is in a stationary state, the process proceeds to step S240, and it is checked based on the pulse sensor 66 whether the user is in a resting state. If it is in the stationary state, the process proceeds to step S242, and it is checked whether the automatic setting in the enlarged mode in such a state is consistent with the manual setting action of the user so far. If there is no contradiction as a result, the enlargement mode is automatically set in step S244, and the process proceeds to step S246. On the other hand, if the stationary state cannot be detected in step S236, the resting state cannot be detected in step S240, or inconsistency with the learning information is detected in step S242, the enlargement mode is not automatically set, and step S246 is performed. Migrate to The contradiction with the learning information means that, in the automatic registration process from step S202 to step S212 in FIG. 9, the history of manually selecting the expansion mode is rare despite being in a stationary state and a resting state, or This is the case where there is a history in which the enlargement mode automatically set based on the stationary state and the resting state is manually canceled. In such a case, there is a high possibility that the automatic setting of the enlargement mode is contrary to the user's intention even if it is detected that the camera is in a stationary state and a resting state. Will do.

  In step S246, the movement of the user's line of sight based on the output of the line-of-sight sensor 70 is compared with the reference data indicating the movement of the reading pattern, and it is checked whether the movement of the user's line of sight corresponds to the movement of the reading pattern. To do. If it is determined that the reading pattern line-of-sight movement is detected, the process proceeds to step S240, and whether the black-and-white reversal mode is automatically set in a state where the reading pattern line-of-sight movement exists is consistent with the manual setting action of the user so far. To check. If there is no contradiction as a result, the monochrome inversion mode is automatically set in step S250, and the process proceeds to step S252. On the other hand, when the reading pattern line-of-sight movement cannot be detected in step S246, the black-and-white reversal mode is not automatically set, and the process proceeds to step S252. Note that the contradiction with the learning information in this case is the black and white reversal mode in the automatic registration process from step S202 to step S212 in FIG. Is a rare history, or a history in which the black-and-white reversal mode automatically set based on the reading pattern line-of-sight movement is manually canceled is recognized. In such a case, there is a high possibility that it is contrary to the user's intention to automatically set the black-and-white reversal mode even if the reading pattern line-of-sight movement is detected. Therefore, the process proceeds to step S254 with the automatic black-and-white reversal mode set ahead. become.

  In the automatic setting from step S236 to step S250, when the black / white reversal mode automatic setting in step S250 is performed after the enlargement mode automatic setting in step S244, the enlarged display is reversed in black and white. On the other hand, if the line-of-sight movement of the reading pattern is not detected in step S246 even if the enlargement mode is automatically set in step S244, there is a possibility that the user is sitting on a chair and performing assembly work or the like, which is suitable for character interpretation. Since it is often inappropriate to perform black and white reversal, only step S252 is reached by automatic setting of the enlargement mode.

  Further, the automatic setting in steps S236 to S250 described above relates to automatic setting of relatively general modes such as the enlargement mode and the black and white reversal mode. On the other hand, in steps S252 and S254, a plurality of modes that are custom-set for a specific user according to the user's symptoms such as an adaptation failure to light and dark, an abnormality in visual field, etc. (at least a specific mode under a specific condition) And a standard mode individually set for the user) are selected according to the situation and are automatically set. Specifically, it is checked whether or not the predetermined condition custom-set in step S252 is met, and if so, the automatic custom mode setting process in step S254 is entered to automatically change the mode setting. The process proceeds to S256. On the other hand, if it is not detected in step S252 that the custom setting condition is met, the process directly proceeds to step S256, and the current setting mode is maintained.

  In step S256, it is checked whether or not a movement acceleration is detected. When the movement acceleration is detected, the process proceeds to step S258, where the automatic return to the standard mode is performed and the flow ends. On the other hand, if the movement acceleration is not detected in step S256, the current setting mode is maintained and the flow ends. If the movement acceleration is detected in step S256, for example, a person sitting in a chair and reading or working may have stood up and started walking. In such a case, the enlargement mode and the black / white inversion mode are maintained. If it is done, it is dangerous, and in step S258, automatic return to the standard mode is performed. This standard mode is the same magnification mode or the wide mode set in step S234 or step S238. The standard mode can be custom set for a specific user.

  The implementation of the various features shown in the above embodiments is not limited to each embodiment, and other embodiments can be implemented as long as the advantages can be enjoyed. Further, in this specification, for the sake of simplicity, the explanations and illustrations in the respective embodiments are concentrated on portions different from those in the other embodiments. However, the embodiments having the respective characteristics described in the respective embodiments are replaced. Needless to say, the present invention can be implemented by combining a plurality of features or a plurality of features.

  FIG. 11: is a block diagram which shows the whole structure in Example 4 of the visual assistance system which concerns on embodiment of this invention. Since the configuration of the fourth embodiment in FIG. 11 is in common with the first embodiment in FIG. 1, the same parts are denoted by the same reference numerals and the description thereof is omitted. The first point that the fourth embodiment is different from the first embodiment is that the fourth embodiment is configured to be used without using the normal glasses 10. That is, diopter adjustment for myopia or hyperopia is performed by adjusting the focus of the right-eye eyepiece optical system 18 and the left-eye eyepiece optical system 20, and for astigmatism, the right-eye toric lens 76 and the left-eye toric lens 78 can be inserted and removed. Correct by. The toric lens 76 for the right eye and the toric lens 78 for the left eye can be independently replaced with lenses suitable for the power of the user, and are removed when not necessary. In addition, when the right-eye toric lens 76 and the left-eye toric lens 78 are inserted, they are rotated around the lines of sight 6a and 8a, respectively, so that correction according to the astigmatic axis angle of the user is performed. The first point in which the fourth embodiment differs from the first embodiment is that the central control unit 80 and the image processing unit execute specific functions, which will be described later.

  FIG. 12 is a schematic cross-sectional view illustrating the basics of diopter adjustment for myopia and hyperopia in the eyepiece optical systems 18 and 20 of the fourth embodiment. For simplicity, illustration and description are given only for the right eye, but the same applies to the left eye. In FIG. 12A, the eyepiece 18a in the right eyepiece optical system 18 is movable between the display display surface 12a and the right eye 6 along the line of sight 6a for diopter adjustment. The eyepiece 18a is set at the standard position in the state of FIG. In this state, the virtual image 12b on the display display surface 12a is located in the visual acuity adjustment range 82 of the healthy person. On the other hand, in the state of FIG. 12A, the position of the virtual image 12b on the display surface 12a is out of the visual acuity adjustment range 84 for the nearsighted person and the visual acuity adjustment range 86 for the farsighted person.

  FIG. 12B shows the diopter adjusted by shifting the eyepiece 18a toward the display display surface 12a so that the display virtual image 12c falls within the visual acuity adjustment range 84 of the nearsighted person. FIG. 12C shows the diopter adjustment by shifting the eyepiece 18a to the right eye side so that the display virtual image 12d falls within the range of visual acuity adjustment of the farsighted person.

  13 to 19 examine various states in the fourth embodiment based on the relationship shown in FIG. 12, and mainly explain the relationship between parallax and convergence angle in 3D image capturing and display. As will be described in detail below, according to the fourth embodiment, the imaging system and the display system are configured to have predetermined conditions, thereby adjusting diopter for nearsightedness and hyperopia, changing object distance, and expanding the display image. In both reduction and reduction (widening if there is a peripheral image), 3D display can be made almost without any sense of incongruity. 13 to 17 consider a single object point and the convergence angle of the eye for simplicity, and FIGS. 18 and 19 illustrate the reality of appearance based on the convergence angle when the display image is enlarged or reduced. Are considering. The 3D image recognition processing by the brain is not only based on the element of the convergence angle of the eye with respect to a single object point, but also greatly contributes to, for example, disparity information of a plurality of object point groups with different distances. However, the elements studied are not inconsistent with the overall 3D recognition results by eye movements and brain.

  FIG. 13 illustrates a basic configuration of 3D image capturing and display in the fourth embodiment. FIG. 13A shows a basic configuration of a 3D parallel binocular imaging system including a right-eye bent zoom lens optical system 24 and a left-eye bent zoom lens optical system 28. For the sake of simplicity, the optical path that is actually bent is shown as being extended to a linear optical path equivalent to this. As apparent from FIG. 13A, the optical axes of the right-eye bent zoom lens optical system 24 and the left-eye bent zoom lens optical system 28 are parallel to each other, and the width thereof is set to the average human eye width. Yes. Further, the right-eye bent zoom lens optical system 24 and the left-eye bent zoom lens optical system 28 are both set to the standard focal length, and the focus is adjusted to focus on the object point 88 at the standard distance. The standard distance of the object point 88 is set to the same distance as the position of the virtual image 12b on the display display surface 12a in FIG. With such a configuration, parallax occurs at the object point 88 viewed from the right-eye bent zoom lens optical system 24 and the left-eye bent zoom lens optical system 28, and the image is formed on the imaging surface 22 a of the right-eye image pickup device 22. The positions of the image point 90 and the image point 92 formed on the image pickup surface 26a of the left-eye image pickup device 26 are shifted to the opposite sides from the optical axis. This is imaged by the right-eye image sensor 22 and the left-eye image sensor 26 as parallax information of the object point 88.

FIG. 13B uses a parallel binocular display whose eye width is equal to the width between the optical axes of the parallel binocular lenses 24 and 28, and in FIG. 13A, the image sensor 22 for the right eye image and the image sensor 26 for the left eye image. The captured image information is displayed on the right-eye display display surface 12a and the left-eye display display surface 14a, respectively. 13B, the positions of the right-eye display display surface 12a and the right-eye eyepiece 18a are set in the same manner as in FIG. 12A, and the positions of the left-eye display display surface 14a and the left-eye eyepiece 20a are also the same. It is. The positions of the virtual image 12b on the right-eye display display surface 12a and the virtual image 14b on the left-eye display display surface 14a are as follows:
It will appear at the same position as the object point 88 at the standard distance.

  In FIG. 13A, an image point 90 imaged on the image pickup surface 22a and an image point 92 imaged on the image pickup surface 26a are real images, and are vertically and horizontally reversed. Therefore, when these are displayed on the right-eye display display surface 12a and the left-eye display display surface 14a and the virtual image is observed, the images are respectively rotated by 180 degrees so as to be in an upright state. As a result, the image information of the image point 90 on the imaging surface 22a and the image point 92 on the imaging surface 26a is displayed as display points 94 and 96 on the right-eye display display surface 12a and the left-eye display display surface 14a, respectively. Is done.

  As described above, the right-eye display display surface 12a and the left-eye display display surface 14a on which the display points 94 and 96 are respectively displayed are the virtual image display points 94a and 96a by the right-eye eyepiece 18a and the left-eye eyepiece 20, respectively. It appears as virtual images 12b, 14b on the displayed display surface. At this time, the central control unit 80 controls the image processing unit 81 to adjust the display points 94 and 96 so that the virtual image display points 94a and 96a coincide with each other. In FIG. 13B, when the right eye 6 and the left eye 8 gaze at the virtual image display points 94a and 96a, the real images 94b and 96b of the virtual image display points 94a and 96a become the centers of the retina of the right eye 6 and the left eye 8, respectively. The state in which an image is formed is shown. By this gaze, a convergence angle is generated in the right eye 6 and the left eye 8, and it is possible to feel the distance to the coincident virtual image display points 94a and 96a.

  FIG. 13C shows the state of convergence angles of the right eye 6 and the left eye 8 when a healthy person actually views the object point 88 of FIG. 13A with the naked eye. Since the convergence angle in FIG. 13B substantially matches the convergence angle in FIG. 13C, the observation in FIG. 13B gives the user the same feeling as in FIG. 13C. The above is the description of the healthy person, and corresponds to the state described with reference to FIG.

  FIG. 14 shows the same thing in Example 4 examined for a myopic person. As described in FIG. 12A, the virtual images 12b and 14b on the display surface in the state of FIG. 13B are out of the visual acuity adjustment range 84 of the nearsighted person and cannot be clearly seen. 14A is similar to FIG. 12B in that the right-eye eyepiece 18a and the left-eye eyepiece 18 are displayed on the display display surface so that the display virtual images 12c and 14c are within the visual acuity adjustment range 84 of the nearsighted person. The diopter is adjusted by shifting to the 12a, 14a side. The right-eye display display surface 12a, the left-eye display display surface 14a, and the display points 94 and 96 displayed on these are the same as in FIG. 13B.

  14A, similarly to FIG. 12B, the virtual image 12c of the right-eye display display surface 12a on which the display points 94 and 96 are displayed and the virtual image 14c of the left-eye display display surface 14a are the right eye 6 and It approaches the left eye 8 and enters the visual acuity adjustment range 84 of the nearsighted person. However, at this time, the virtual image display points 94c and 96c seen on the virtual images 12c and 14c on the display surface do not coincide with each other on the virtual images 12c and 14c in the visual acuity adjustment range 84. However, in the right eye 6 and the left eye 8 that try to see the virtual image display points 94c and 96c as the same point, the real images 94b and 96b of the virtual image display points 94c and 96c are formed at the centers of the retinas of the right eye 6 and the left eye 8, respectively. Take a convergence angle like this. As a result, what is visible is recognized as matching points 94d and 96d.

  FIG. 14B shows the state of convergence angles of the right eye 6 and the left eye 8 that a nearsighted person sees in real life by correcting the object point 88 of FIG. ing. Since the convergence angle of FIG. 14 (A) substantially matches the convergence angle of FIG. 14 (B), the observation of the coincidence points 94d and 96d in FIG. A feeling equal to the observation of the object point 88 in everyday life with the glasses 98 as in B) is given. Strictly speaking, the positions of the coincidence points 94d and 96d in FIG. 14A are different from the virtual image display points 94a and 96a in FIG. 13 but not a significant difference, so that the view between FIG. 13 and FIG. Shifting and correcting the positions of the display points 94 and 96 displayed on the right-eye display display surface 12a and the left-eye display display surface 14a according to the degree adjustment can be omitted.

  FIG. 15 shows the same results as described above in Example 4 for a hyperopic person. As described with reference to FIG. 12A, the virtual images 12b and 14b on the display surface in the state of FIG. 13B are out of the visual acuity adjustment range 86 of the farsighted person and cannot be clearly seen. Therefore, in FIG. 15A, in the same manner as FIG. 12C, the right eyepiece 18a and the left eyepiece 18a are placed in the right eye 6 so that the display virtual images 12d and 14d are within the range of visual acuity adjustment 86 of the farsighted person. And diopter adjustment by shifting to the left eye 8 side. The right-eye display display surface 12a, the left-eye display display surface 14a, and the display points 94 and 96 displayed on these are the same as in FIG. 13B.

  15A, similarly to FIG. 12C, the virtual image 12d of the display display surface 12a for the right eye and the virtual image 14d of the display display surface 14a for the left eye on which the display points 94 and 96 are respectively displayed are the right eye 6 and It moves away from the left eye 8 and enters the range of visual acuity adjustment for the farsighted person. However, at this time, the virtual image display points 94e and 96e that are visible on the virtual images 12d and 14d on the display surface do not coincide with each other on the virtual images 12d and 14d in the visual acuity adjustment range. (The positions are reversed and reversed.) However, the right eye 6 and the left eye 8 trying to see the virtual image display points 94e and 96e as the same point are the real images 94b and 96b of the virtual image display points 94e and 96e. The angle of convergence is such that an image is formed at the center of the retina of each of the left eyes 8. As a result, what is visible is recognized as matching points 94f and 96f.

  FIG. 15 (B) shows the state of convergence angles of the right eye 6 and the left eye 8 that a hyperopic person sees in everyday life by correcting the object point 88 of FIG. 13 (A) by wearing glasses 100 of convex lenses. Show. The vergence angle in FIG. 15A also substantially coincides with the vergence angle in FIG. 15B. Therefore, the observation of the coincidence points 94f and 96f in FIG. A feeling equivalent to the observation of the object point 88 in everyday life with the glasses 100 as in 15 (B) is given. Strictly speaking, the positions of the coincidence points 94f and 96f in FIG. 15A are different from the virtual image display points 94a and 96a in FIG. 13, but for the right eye according to the diopter adjustment as in the case of myopia. Shifting and correcting the positions of the display points 94 and 96 displayed on the display display surface 12a and the left-eye display display surface 14a can be omitted.

  The above is the description when the normal person, the nearsighted person, and the farsighted person observe the display on the right-eye display display surface 12a and the left-eye display display surface 14a based on the same image information captured in FIG. is there. As described above, in the fourth embodiment, the parallax information of the object point 88 is obtained by the parallel binocular imaging system in which the optical axes are parallel to each other and the width is set to the average human eye width. Then, the image information including the parallax information is displayed on each parallel binocular display whose eye width is equal to the width between the optical axes of the parallel binocular binocular imaging system, so that the object points on the left and right displays coincide with each other. The stereoscopic view is reproduced by the vergence angle of the eyes. Then, by adjusting diopter, it corresponds to near-sighted and far-sighted and enables reproduction according to a healthy person.

  Next, the case where the position of the object point is different from the object point 88 set at the same distance as the position of the virtual image 12b on the display display surface 12a will be described with reference to FIGS. For the sake of simplicity, only a normal person will be described, but a nearsighted person and a farsighted person can be similarly understood according to FIGS. 14 and 15.

  FIG. 16 is the same as FIG. 13, but as shown in FIG. 16A, the right-eye bent zoom lens optical system 24 and the left-eye bent zoom lens optical system 28 are the object points of the standard distance in FIG. The focus is adjusted to focus on the near-distance object point 102 closer to 88. It is assumed that this short distance object point 102 is within the range of visual acuity adjustment for a healthy person as will be described later. In this state, the parallax of the short-distance object point 102 viewed from the right-eye bent zoom lens optical system 24 and the left-eye bent zoom lens optical system 28 is larger than that in the case of FIG. The positions of the image point 104 formed on the upper side and the image point 106 formed on the image pickup surface 26a of the left-eye image pickup device 26 are greatly shifted on the opposite sides from the optical axis. This is imaged by the right-eye image sensor 22 and the left-eye image sensor 26 as parallax information of the short-distance object point 102.

  FIG. 16B shows image information captured by the right-eye image sensor 22 and the left-eye image sensor 26 in the state of FIG. 16A on the right-eye display display surface 12a and the left-eye display display surface 14a, respectively. It is displayed. In FIG. 16B, the positions of the right-eye display display surface 12a and the right-eye eyepiece 18a are set in the same manner as in FIG. 12A, and the positions of the left-eye display display surface 14a and the left-eye eyepiece 20a are also the same. It is. In this respect, FIG. 16B is the same as FIG.

  However, the display points 108 and 110 displayed on the right-eye display display surface 12a and the left-eye display display surface 14a are largely shifted from the display center to the opposite sides because the short-distance object point 102 is close. Accordingly, the right-eye display display surface 12a and the left-eye display display surface 14a on which the display points 108 and 110 are displayed are displayed as virtual images 12b and 14b on the display display surface by the right-eye eyepiece 18a and the left-eye eyepiece 20, respectively. When visible, the virtual image display points 108a, 110a on them do not coincide. (Positions are reversed and reversed.) However, the right image 6 and the left eye 8 that try to see the virtual image display points 108a and 110a as the same point are the real images 108b and 110b of the virtual image display points 108a and 110a. Further, the angle of convergence is more tight so as to form an image at the center of the retina of each of the left eyes 8. As a result, what is visible is recognized as the coincidence points 108c and 110c at a closer distance.

  FIG. 16C shows the state of convergence angles of the right eye 6 and the left eye 8 when a healthy person actually views the short distance object point 102 of FIG. 16A with the naked eye. As described above, the short-distance object point 102 is within the visual acuity adjustment range of a healthy person. Since the convergence angle in FIG. 16B substantially coincides with the convergence angle in FIG. 16C, the observation of the coincidence points 108c and 110c in FIG. A feeling equivalent to the observation of the point 102 is given.

  FIG. 17 is the same as FIG. 13 and FIG. 16, except that the right-eye bent zoom lens optical system 24 and the left-eye bent zoom lens optical system 28 have the standard distance in FIG. The focus is adjusted to focus on a long-distance object point 112 farther than the object point 88. It is assumed that the long-distance / short-distance object point 112 is also within the range of visual acuity adjustment for a healthy person. In this state, the parallax of the long-distance object point 112 viewed from the right-eye bent zoom lens optical system 24 and the left-eye bent zoom lens optical system 28 is smaller than that in the case of FIG. The positions of the image point 114 formed above and the image point 116 formed on the image pickup surface 26a of the left-eye image pickup device 26 are shifted, but closer to the optical axis. This is imaged by the right-eye image sensor 22 and the left-eye image sensor 26 as parallax information of the long-distance object point 112.

  FIG. 17B shows image information captured by the right-eye image imaging element 22 and the left-eye image imaging element 26 in the state of FIG. 17A on the right-eye display display surface 12a and the left-eye display display surface 14a, respectively. It is displayed. In FIG. 17B, the positions of the right-eye display display surface 12a and the right-eye eyepiece 18a are set in the same manner as in FIG. 12A, and the positions of the left-eye display display surface 14a and the left-eye eyepiece 20a are also the same. It is. In this respect, FIG. 17B is the same as FIGS. 13B and 16B.

  However, the display points 118 and 120 displayed on the right-eye display display surface 12a and the left-eye display display surface 14a are closer to the display center because the long-distance object point 112 is far away. Therefore, the right-eye display display surface 12a and the left-eye display display surface 14a on which the display points 118 and 120 are displayed are respectively converted into virtual images 12b and 14b on the display display surface by the right-eye eyepiece lens 18a and the left-eye eyepiece lens 20. When visible, the virtual image display points 118a, 120a on them do not coincide. However, in the right eye 6 and the left eye 8 that try to see the virtual image display points 118a and 120a as the same point, the real images 118b and 120b of the virtual image display points 118a and 120a are formed at the centers of the retinas of the right eye 6 and the left eye 8, respectively. Take a looser angle of convergence. As a result, what is visible is recognized as the coincidence points 118d and 120d at a farther distance.

  FIG. 17C shows the state of convergence angles of the right eye 6 and the left eye 8 when the normal object is viewed with the naked eye by the long distance object point 112 of FIG. As described above, the short-distance object point 112 is within the visual acuity adjustment range of a healthy person. Since the convergence angle in FIG. 17B substantially matches the convergence angle in FIG. 17C, the observation of the coincidence points 118d and 120d in FIG. A feeling equivalent to the observation of the point 112 is given.

  As described above, the image information captured by the parallel binocular imaging system in which the optical axes are parallel to each other and the width is set to the average human eye width is the width between the optical axes of the parallel binocular imaging system in which the eye width is parallel. In the configuration of the fourth embodiment, which is displayed on a parallel binocular display equal to each other, stereoscopic viewing is possible even if the distance between the object points varies depending on the convergence angle of the eye to be seen so that the object points on the left and right displays coincide with each other. Can be reproduced.

  FIGS. 18 and 19 relate to the case where the central control unit 80 controls the image processing unit 81 to enlarge and reduce (widen) the display images on the right-eye display display surface 12a and the left-eye display display surface 14a, respectively. Yes, from the viewpoint of the angle of convergence, we are examining the reality of appearance when expanding and contracting.

  FIG. 18 relates to a case where display images on the right-eye display display surface 12a and the left-eye display display surface 14a are enlarged. First, FIG. 18A shows a standard magnification state before enlargement, and corresponds to the state of FIG. At this time, a display image 122 and a display image 124 are displayed on the right-eye display display surface 12a and the left-eye display display surface 14a, respectively. (Shown as a large image to show enlargement and reduction.) Similarly to FIG. 13B, the display image 122 and the display image 124 are displayed on the display display surface 12a for the right eye and the display display surface 14a for the left eye. By the right eyepiece 18a and the left eyepiece 20, the virtual image display images 122a and 124a can be seen as virtual images 12b and 14b on the display surface on which they are displayed. At this time, the virtual image display images 122a and 124a match in the same manner as in FIG. In FIG. 18B, when the right eye 6 and the left eye 8 gaze at the virtual image display images 122a and 124a, the real images 122b and 124b of the virtual image display images 122a and 124a (shown with dots to show only the position). Indicates a state in which images are formed at the centers of the retinas of the right eye 6 and the left eye 8, respectively. This gaze causes a convergence angle in the right eye 6 and the left eye 8, and the distance to the coincident virtual image display images 122a and 124a can be felt as in FIG. 13B.

  On the other hand, FIG. 18B shows a case where the display images on the right-eye display display surface 12a and the left-eye display display surface 14a are enlarged to be enlarged display images 126 and 128, respectively. What is important at this time is not to enlarge the enlarged display images 126 and 128 themselves as the center, but to enlarge the entire image around the central portions of the entire right-eye display display surface 12a and the left-eye display display surface 14a. . As a result, the display images 122 and 124 that are not in the center of the entire right-eye display display surface 12a and the left-eye display display surface 14a in the state of FIG. 18A are displayed as enlarged display images 126 and 128 in FIG. When it becomes, it will shift to the opposite side more greatly with expansion. (Leave away from the center of the image.)

  Accordingly, the right-eye display display surface 12a and the left-eye display display surface 14a on which such enlarged display images 126 and 128 are displayed are displayed on the display display surface virtual images 12b and 14b by the right-eye eyepiece 18a and the left-eye eyepiece 20, respectively. , The virtual image enlarged display images 126a and 128a thereon do not match. (The positions are reversed and reversed.) However, the right eye 6 and the left eye 8 trying to see the virtual image enlarged display images 126a and 128b as the same point are the real images 126b and 128b of the virtual image enlarged display images 126a and 128b ( Since only the position is shown, the angle of convergence is such that the image of the right eye 6 and the left eye 8 is focused on the center of the retina. As a result, what is visible is recognized as two coincident images 126c and 128c at a closer distance. Thereby, the coincidence images 126c and 128c appear closer together with enlargement, and the reality as a 3D image can be realized.

  On the other hand, FIG. 19 relates to a case where the display images on the right-eye display display surface 12a and the left-eye display display surface 14a are reduced (widened if there are peripheral images). FIG. 19A is the same as FIG. 18A and shows a standard magnification state before enlargement.

  On the other hand, FIG. 19B shows a case where the display images on the right-eye display display surface 12a and the left-eye display display surface 14a are reduced to reduced display images 130 and 132, respectively. What is important at this time is not to reduce the reduced display images 130 and 132 themselves as the centers, as in FIG. 18, but to center the central portions of the right-eye display display surface 12a and the left-eye display display surface 14a. It is to reduce the whole. As a result, when the display images 122 and 124 in the state of FIG. 19A become the reduced display images 130 and 132 of FIG. 19B, the images approach each other along with the reduction.

  Accordingly, the right-eye display display surface 12a and the left-eye display display surface 14a on which such reduced display images 130 and 132 are displayed are displayed on the display display surface virtual images 12b and 14b by the right-eye eyepiece 18a and the left-eye eyepiece 20, respectively. , The virtual image reduced display images 130a and 132a thereon do not match. However, the right eye 6 and the left eye 8 that try to see the virtual image reduced display images 130a and 132a as the same point have the real images 130b and 132b of the virtual image reduced display images 130a and 132a (shown with dots to show only the position) as the right eye. The 6 and the left eye 8 each take a looser convergence angle that forms an image at the center of the retina. As a result, what is visible is recognized as the coincidence images 130c and 132c at a farther distance. As a result, the coincidence images 132c and 132c appear farther with reduction, and the reality as a 3D image can be realized.

  20 to 22 illustrate the parallax / congestion in the case of using an imaging system and a display system that are in a mutually different relationship from the conditions described in FIGS. 13 to 19.

  FIG. 20 is a schematic sectional view of Example 5 of the present invention. In Example 5 of FIG. 20, image information captured by a small parallel binocular imaging system in which the optical axes are parallel to each other and the width thereof is set to be narrower than the average human eye width, the average human eye width is obtained. Are displayed on the parallel binocular display set to. Hereinafter, the processing of the fifth embodiment when the optical axis widths of the parallel binocular imaging system and the parallel binocular imaging system are different will be described. In addition, since structures other than the above are common to the fourth embodiment, FIG. 11 is used for details of the structure.

  FIG. 20A shows a basic configuration of a 3D parallel binocular imaging system including a right-eye bent zoom lens optical system 134 and a left-eye bent zoom lens optical system 136. Similarly to FIG. 13, for the sake of simplicity, the optical path is extended. As in the fourth embodiment, the optical axes of the right-eye bent zoom lens optical system 134 and the left-eye bent zoom lens optical system 136 are parallel to each other, but their width is narrower than the average human eye width. The right-eye bent zoom lens optical system 134 and the left-eye bent zoom lens optical system 136 are both set to the standard focal length and the focus is adjusted to focus on the object point 88 at the standard distance. The standard distance of the object point 88 is set to the same distance as the position of the virtual image 12b on the display display surface 12a in FIG.

  Image points 142 and 144 imaged on the imaging surface 138a of the right-eye image pickup device and the imaging surface 140a of the right-eye image pickup device by the right-eye bent zoom lens optical system 134 and the left-eye bent zoom lens optical system 136, respectively. Deviate from the optical axis to the opposite side. However, the shift amount is smaller than that in FIG. 13A because the optical axis widths of the right-eye bent zoom lens optical system 134 and the left-eye bent zoom lens optical system 136 are narrow.

  FIG. 20B shows a parallel binocular display in which the eye width is equal to the average eye width of a human as described above, and the configuration is the same as FIG. 13B. However, if the display on the right-eye display display surface 12a and the left-eye display display surface 14a are displayed on the basis of image information in which the shift amounts of the image points 142 and 144 are small as shown in FIG. 148 is closer to the display center. Accordingly, when the right-eye display display surface 12a and the left-eye display display surface 14a on which such display points 146 and 148 are respectively displayed are seen as virtual images 12b and 14b on the display display surface, the virtual image display points 146a and 148a do not match. . The right eye 6 and the left eye 8 have looser convergence angles such that the real images 146b and 148b are formed at the centers of the retinas of the right eye 6 and the left eye 8 in an attempt to view the virtual image display points 146a and 148a as the same point. As a result, what is visible is recognized as the coincidence points 146c and 148c at a distance farther than the object point 88. Thus, when the widths between the optical axes of the parallel binocular imaging system and the parallel binocular imaging system do not match, the displayed matching points 146c and 148c appear to be at a different distance from the object point 88 to be imaged, resulting in a sense of incongruity. .

  FIG. 20C shows a configuration for eliminating the above-mentioned uncomfortable feeling. Specifically, as shown in FIG. 20C, the central control unit 80 controls the image processing unit 81 so that the virtual image display points 150d and 152d coincide on the virtual images 12b and 14b on the display surface, respectively. The display points on the right-eye display display surface 12a and the left-eye display display surface 14a are shifted. That is, when the display points 146 and 148 shown in FIG. 20 are shifted toward the outer side of the image indicated by the arrows in FIG. 20C and set as the display points 150 and 152, on the virtual images 12b and 14b on the display surface. , The virtual image display points 150d and 152d coincide. As a result, the displayed coincidence points 150d and 152d are in a state of convergence angle where they can be seen at the same position as the object point 88 to be imaged, and the uncomfortable feeling can be eliminated.

  FIG. 21 is a schematic sectional view of Example 6 of the present invention. The sixth embodiment is configured such that the eye width of the parallel binocular display can be adjusted. Specifically, Example 5 in FIG. 21 shows image information captured by a small parallel binocular imaging system in which the optical axes are parallel to each other and the width thereof is set narrower than the average human eye width. Are each displayed on an adjustable parallel binocular display.

  FIG. 21A is the same as FIG. That is, even in Example 6, the width of the optical axis of the right-eye bent zoom lens optical system 134 and the left-eye bent zoom lens optical system 136 is narrower than the average human eye width, and the imaging of the right-eye image pickup device is performed. The shift amounts of the image points 142 and 144 formed on the imaging surface 140a of the surface 138a and the imaging device for the right eye image are determined by the optical axis widths of the right-eye bent zoom lens optical system 134 and the left-eye bent zoom lens optical system 136, respectively. Since it is narrow, it is smaller than FIG.

  FIG. 21B is a schematic cross-sectional view of a parallel binocular display similar to FIG. 20B, but the left-eye display display surface 14a and the left-eye eyepiece 20a are connected to the right-eye display display surface 12a by adjusting the eye width. The eye width of the parallel binocular display is equal to the width between the optical axes of the parallel binocular binocular imaging system. Such a state is suitable as a display for children with a narrow eye width. Then, as a result of matching the width between the optical axes of the parallel binocular imaging system and the parallel binocular imaging system, the relationship between FIG. 21A and FIG. 21B is as shown in FIG. 13A and FIG. The virtual image display points 146a and 148a on the virtual images 12b and 14b on the display surface coincide with each other. The position looks the same distance as the object point 88 to be imaged, and there is no sense of incongruity. In this way, the sense of discomfort can be eliminated by adjusting the width between the optical axes.

  FIG. 21C is the same as FIG. 20C, and shows a case where the eye width of the parallel binocular display is increased by adjusting the eye width, for example, for adults. In this case, as in the fifth embodiment, the central control unit 80 controls the image processing unit 81 to shift the display points on the right-eye display display surface 12a and the left-eye display display surface 14a, respectively. Eliminate.

  FIG. 22 is a schematic sectional view of Example 7 of the present invention. The seventh embodiment is suitable when a parallel binocular imaging system and a parallel binocular display are separated and various parallel binocular imaging systems and parallel binocular displays are combined.

  FIG. 22A shows a parallel binocular display used for the basic combination, and its configuration is the same as FIG. 21B. However, the eye width is fixed in a narrow state. In the basic combination of the seventh embodiment, the parallel binocular display shown in FIG. 22A is combined with the parallel binocular binocular imaging system shown in FIG. Since this combination is equivalent to the relationship between FIG. 21B and FIG. 21A, the display does not feel strange.

  FIG. 22B shows another parallel binocular display combined with the parallel binocular binocular imaging system shown in FIG. In the parallel binocular display of FIG. 22B, the width between the optical axes coincides with that of the parallel binocular binocular imaging system, but the display display surfaces 154a and 156a are located far from the right eye 6 and the left eye 812. Accordingly, the eyepiece lenses 158a and 160a are different optical systems. For this reason, the relationship between the parallel binocular binocular imaging system and the parallel binocular display in this combination is the same as that in FIGS. 20 (A) and 20 (B). For this reason, as shown in FIG. 22C, the same correction as in FIG. 20C is performed to eliminate the uncomfortable feeling. For this purpose, each binocular binocular imaging system and parallel binocular display hold information necessary for correction and communicate with each other.

  The implementation of the various features shown in each of the above embodiments is not limited to each embodiment, and the advantages thereof can be enjoyed.

  For example, the binocular imaging system and the binocular display device are integrated, and various features of the embodiment configured as a visual assistance system for viewing the visual field in front of the eye in real time include the binocular imaging system and the binocular display device as in the seventh embodiment. The present invention can also be applied to a separate structure composed of a 3D camera and a 3D head-mounted display. In this case, the 3D moving image content shot by the 3D camera can be viewed on the head mounted 3D display at different times and / or at different places.

  The present invention can be applied to visual assistance systems and binocular display equipment.

24, 28, 134, 136 Binocular imaging system
12a, 14a, 18a, 20a Binocular display system 80, 81 Image shift function

Claims (8)

  A binocular display provided with a binocular display system that receives image information from a binocular imaging system having parallel optical axes and has an optical axis width equivalent to the optical axis width of the binocular imaging system.   The binocular display according to claim 1, wherein a display virtual image is provided at a distance equivalent to a standard subject distance of the binocular imaging system.   The binocular display according to claim 1 or 2, further comprising a diopter adjustment function.   4. The binocular display according to claim 1, wherein the binocular display has an image enlargement function.   5. The binocular display according to claim 1, further comprising an image reduction function.   A binocular display system that receives image information from a binocular imaging system with parallel optical axes and has an optical axis width different from the optical axis width of the binocular imaging system, the optical axis width of the binocular imaging system, and the light of the binocular display system A binocular display having an image shift function based on a difference from an axial width.   A binocular display having a binocular display system having an optical axis width adjustment function and an image shift function according to an adjustment of an optical axis width in the binocular display system.   A binocular display that receives image information from a binocular imaging system having parallel optical axes, has a display optical system different from the binocular imaging system, and has an image shift function based on a difference between the display optical systems.

Images