WO2022024917A1 - 撮像装置、調整方法、及び調整プログラム - Google Patents

撮像装置、調整方法、及び調整プログラム Download PDF

Info

Publication number
WO2022024917A1
WO2022024917A1 PCT/JP2021/027296 JP2021027296W WO2022024917A1 WO 2022024917 A1 WO2022024917 A1 WO 2022024917A1 JP 2021027296 W JP2021027296 W JP 2021027296W WO 2022024917 A1 WO2022024917 A1 WO 2022024917A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
adjustment
field lens
multispectral camera
optical system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/027296
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
和佳 岡田
慶延 岸根
睦 川中子
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Corp
Original Assignee
Fujifilm Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujifilm Corp filed Critical Fujifilm Corp
Priority to JP2022540241A priority Critical patent/JP7522198B2/ja
Publication of WO2022024917A1 publication Critical patent/WO2022024917A1/ja
Priority to US18/153,359 priority patent/US12181663B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/0081—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for altering, e.g. enlarging, the entrance or exit pupil
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28—Investigating the spectrum
    • G01J3/30—Measuring the intensity of spectral lines directly on the spectrum itself
    • G01J3/36—Investigating two or more bands of a spectrum by separate detectors
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B13/00—Optical objectives specially designed for the purposes specified below
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B13/00—Optical objectives specially designed for the purposes specified below
    • G02B13/0095—Relay lenses or rod lenses
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B15/00—Optical objectives with means for varying the magnification
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B3/00—Simple or compound lenses
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B5/00—Optical elements other than lenses
    • G02B5/005—Diaphragms
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B5/00—Optical elements other than lenses
    • G02B5/20—Filters
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B5/00—Optical elements other than lenses
    • G02B5/30—Polarising elements
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/006—Filter holders
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/02—Mountings, adjusting means, or light-tight connections, for optical elements for lenses
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B11/00—Filters or other obturators specially adapted for photographic purposes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H04N23/11—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths for generating image signals from visible and infrared light wavelengths
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/45—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/633—Control of cameras or camera modules by using electronic viewfinders for displaying additional information relating to control or operation of the camera
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/71—Circuitry for evaluating the brightness variation

Definitions

  • the present invention relates to an image pickup device for capturing a multispectral image, and an adjustment method and adjustment program for the image pickup device.
  • Patent Document 1 describes an imaging device including a spectroscopic filter array and a field lens.
  • One embodiment according to the technique of the present disclosure provides an image pickup apparatus, an adjustment method, and an adjustment program capable of acquiring a multispectral image of good image quality.
  • the image pickup device is an image pickup device arranged on the image side of another optical system, and is a multi-spectral camera that acquires images for a plurality of wavelength bands, and a multi-spectral camera with another optical system.
  • the multispectral camera is equipped with a field lens that relays to the spectrum camera and an adjustment mechanism that adjusts the conjugate relationship between the exit pupil position of another optical system and the incident pupil position of the multispectral camera.
  • the multispectral camera has a pupil position or a pupil position.
  • a frame that is arranged in the vicinity of and has a plurality of aperture regions, and has a frame having different centers of gravity of the plurality of aperture regions, and a frame that is arranged in the plurality of aperture regions and has at least a part of the wavelength bands transmitting different light.
  • a wavelength polarizing filter unit having a plurality of optical filters including one or more optical filters and a plurality of polarizing filters arranged in a plurality of aperture regions and having different polarization directions, and light transmitted through any of the plurality of aperture regions. It includes an image pickup element including a plurality of pixel groups that receive light, and a processor that generates an image based on a plurality of image signals output from the image pickup element.
  • the adjusting mechanism adjusts the distance between the other optical system and the field lens.
  • the image pickup apparatus is the first aspect, in which the adjustment mechanism adjusts the distance between the field lens and the multispectral camera.
  • the image pickup apparatus is in any one of the first to third aspects, and the adjustment mechanism is the distance between the exit pupil position of another optical system and the field lens, and the field lens and the multispectral camera. Adjust the distance from the entrance pupil position.
  • the image pickup apparatus is one of the first to fourth aspects, and the adjustment mechanism adjusts the image magnification while keeping it constant.
  • the adjusting mechanism changes the position of the wavelength polarizing filter unit and / or the focal length of the field lens to keep the image magnification constant.
  • the image pickup apparatus is any one of the first to sixth aspects, and the adjustment mechanism is the attachment / detachment mechanism of the wavelength polarizing filter unit.
  • the processor outputs image support information necessary for adjustment.
  • the imaging device further includes a display device in the eighth aspect, and the processor outputs image support information based on the light / dark information of at least one spectroscopic image obtained by the multispectral camera.
  • the processor outputs an adjustment procedure as image support information based on the direction of the aperture region and the light / dark information.
  • the image pickup apparatus further includes a display device in any one of the eighth to tenth aspects, and the display device is adjusted as image assist information and image assist information output from the processor. Show at least one of the steps.
  • the adjustment method according to the twelfth aspect of the present invention is a frame body arranged at the pupil position or in the vicinity of the pupil position and having a plurality of opening regions, and a plurality of frames having different centers of gravity of the plurality of opening regions.
  • a plurality of optical filters including two or more optical filters arranged in an aperture region and transmitting light having at least a part of wavelength bands different from each other, and a plurality of polarizing filters arranged in a plurality of aperture regions and having different polarization directions.
  • a multi-spectral camera that acquires images for multiple wavelength bands, a field lens that relays between another optical system and the multi-spectral camera, and the exit pupil position of the other optical system and the incident pupil position of the multi-spectral camera. It is an adjustment method of an image pickup apparatus arranged on the image side of another optical system, comprising an adjustment mechanism for adjusting a conjugate relationship, and has an output step for outputting image support information necessary for adjustment.
  • the adjustment method according to the thirteenth aspect is the twelfth aspect, in which the image support information based on the light / dark information of at least one spectral image obtained by the multispectral camera is displayed on the display device in the output step.
  • the adjustment method according to the fourteenth aspect is the thirteenth aspect, in which in the output step, the adjustment procedure as the image support information is displayed on the display device based on the direction of the opening region and the light / dark information.
  • the adjustment program according to the fifteenth aspect of the present invention causes a computer to execute the adjustment method according to any one of the twelfth to the fourteenth.
  • FIG. 1 is a diagram showing a configuration of an imaging system according to the first embodiment.
  • FIG. 2 is a diagram showing a configuration of an image pickup apparatus.
  • FIG. 3 is a conceptual diagram showing how the conjugate relationship is adjusted by exchanging each element.
  • FIG. 4 is a diagram showing an example of an adjustment mechanism of a field lens.
  • FIG. 5 is a diagram showing another example of the adjustment mechanism of the field lens.
  • FIG. 6 is a perspective view of a multispectral camera.
  • FIG. 7 is a cross-sectional view of a multispectral camera.
  • FIG. 8 is a diagram showing the configuration of the frame body.
  • FIG. 9 is a diagram showing a configuration of a wavelength polarizing filter unit.
  • FIG. 10 is a diagram showing the polarization direction of the polarizing filter.
  • FIG. 9 is a diagram showing a configuration of a wavelength polarizing filter unit.
  • FIG. 10 is a diagram showing the polarization direction of the polarizing filter.
  • FIG. 11 is a diagram showing a configuration of an image pickup device.
  • FIG. 12 is a diagram showing a processor configuration.
  • FIG. 13 is a table showing an example of the relationship between the relative position of the field lens and the elements to be adjusted.
  • FIG. 14 is a diagram showing a state of adjustment in cases 1 to 4.
  • FIG. 15 is another diagram showing the state of adjustment in the case 5.
  • FIG. 16 is a flowchart (1/2) showing a procedure for adjusting the conjugate relationship.
  • FIG. 17 is a flowchart (2/2) showing a procedure for adjusting the conjugate relationship.
  • FIG. 18 is a conceptual diagram showing the relationship between the direction of the aperture region and the light / dark information of the spectroscopic image.
  • FIG. 19 is a table showing the influence of the shape of the opening region on the adjustment of the conjugate relationship.
  • FIG. 20 is a diagram showing a specific example of conjugation relationship adjustment in consideration of the shape of the opening region.
  • FIG. 21 is a diagram showing a state of connection with a microscope.
  • FIG. 22 is a diagram showing a state of connection with the zoom optical system.
  • FIG. 23 is a diagram showing a system configuration when the elements are not connected to each other.
  • FIG. 1 is a diagram showing a configuration of an imaging system 10 (imaging system, imaging device) according to the first embodiment.
  • the image pickup system 10 includes an optical system 20 (another optical system; a lens 22 is included), an image pickup device 100, a display device 300 (display device such as a liquid crystal display), a storage device 310 (photomagnetic recording device, semiconductor memory, etc.), and the like.
  • the image pickup device 100 is arranged on the image side of the optical system 20 and is composed of an operation unit 320 (keyboard, mouse, switch, etc.).
  • a speaker that outputs image support information described later by voice may be provided.
  • An example of the optical system 20 will be described later (see FIGS. 21 and 22).
  • FIG. 2 is a diagram showing a configuration of an image pickup device 100 (imaging device).
  • the image pickup device 100 includes a mount adapter 110 (adjustment mechanism), a field lens unit 120 (field lens), a multispectral camera 130 (multispectral camera), and an image pickup device main body 140 (multispectral camera). It is equipped with a camera).
  • the field lens 122 is a lens that relays the optical system 20 and the multispectral camera 130.
  • the screw mount standards include, for example, C mount and CS mount.
  • the C mount has a diameter of 25.4 mm, a screw pitch of 0.794 mm, and a flange back of 17.526 mm (see FIG. 2).
  • the adjustment of the conjugate relationship means that the exit pupil position of the optical system 20 (another optical system) and the entrance pupil position of the image pickup device 100 (imaging device) are coupled to each other.
  • the conjugate relationship is adjusted by attaching / detaching elements with different sizes and characteristics, moving the lens, etc. (adjustment by the adjustment mechanism). This makes it possible to prevent a decrease in the amount of peripheral light (vignetting) in the spectroscopic image.
  • the conjugation relationship can be adjusted based on the light and dark information of the spectroscopic image. Specifically, as will be described in detail later, the processor 142 calculates the light / dark information (light amount distribution in the image) of at least one spectral image obtained by the multispectral camera 130, and the conjugate relationship is based on this light / dark information. Generates and outputs information (image support information) necessary for adjustment. The user can adjust the conjugation relationship by attaching / detaching / exchanging or moving the components according to the image support information.
  • FIG. 3 is a conceptual diagram showing how the conjugate relationship is adjusted by exchanging each element.
  • the image pickup apparatus 100 includes a mount adapter 110A (adjustment mechanism), a field lens unit 120A including a field lens 122A, and a multispectral camera 130A (the lens and the image pickup apparatus main body are not shown).
  • a mount adapter 110A adjustment mechanism
  • a field lens unit 120A including a field lens 122A
  • a multispectral camera 130A the lens and the image pickup apparatus main body are not shown.
  • the mount adapter 110B adjustment mechanism having a size (length in the optical axis direction) different from that of the mount adapter 110A, the optical system 20 and the image pickup are performed.
  • the conjugation relationship can be adjusted by adjusting the distance to the device 100.
  • a field lens unit 120B having a field lens 122B (adjustment mechanism) having a focal length and / or an image magnification different from that of the field lens 122A (adjustment mechanism). It is also possible to adjust the conjugate relationship by adjusting the distance between the optical system 20 and the field lens. When the adjustment is performed by the field lens unit, the focal length and / or the image magnification may be changed by moving the lens forward and backward while the unit having a specific configuration is attached.
  • the part (d) in FIG. 3 shows the state of adjustment by the multispectral camera, and is coupled by attaching the multispectral camera 130B having a lens having a focal length and / or an image magnification different from that of the multispectral camera 130A. Can be adjusted. Further, when the adjustment is performed by the multispectral camera, the focal length and / or the image magnification may be changed by moving the lens forward and backward while the camera having a specific configuration is attached.
  • Adjustment by the mount adapter, field lens unit, and multi-spectral camera may be performed by only one of them or by combining a plurality of them. For example, by exchanging the mount adapter and the field lens unit, the distance between the exit pupil position of the optical system 20 (another optical system) and the field lens, and the distance between the field lens and the entrance pupil position of the multispectral camera can be adjusted. can do.
  • the field lens unit 120 includes a lens barrel 121 and a field lens 122 (field lens).
  • the field lens 122 moves forward and backward in the direction of the optical axis L by the user operating the adjustment mechanism described later.
  • the field lens 122 may be composed of one lens or a plurality of lenses.
  • FIG. 4 is a diagram showing an example of the adjustment mechanism of the field lens.
  • the part (a) in the figure is a perspective view of the lens barrel 124A, and the lens barrel 124A is formed with three slits 126A in a direction parallel to the optical axis L.
  • Part (b) of the figure is a front view of the lens unit 128, showing a field lens 122 and three arms 127.
  • the portion (c) in the figure shows a state in which the arm 127 is inserted into the slit 126A and the lens unit 128 is attached to the lens barrel 124A.
  • the arm 127 can move in the slit 126A in the direction of the optical axis L, whereby the user can move the field lens 122 forward and backward to adjust the conjugation relationship.
  • the field lens 122 is composed of a plurality of lenses, such an adjustment mechanism may be provided for a part of the plurality of lenses, or an adjustment mechanism may be provided for all of the plurality of lenses. ..
  • FIG. 5 is a diagram showing another example of the adjustment mechanism of the field lens.
  • the part (a) in the figure is a perspective view of the lens barrel 124B, and the lens barrel 124B is formed with a spiral slit 126B centered on the optical axis L.
  • the portion (b) in the figure shows a state in which the lens unit 128 similar to that in FIG. 4 is attached to the lens barrel 124B (a state in which the arm 127 is inserted into the slit 126B). In the state shown in the portion (b) of the figure, the lens unit 128 can move forward and backward while rotating about the optical axis L, whereby the user can move the field lens 122 forward and backward.
  • the distance between the field lens 122 and the optical system 20 (another optical system) and / or the multispectral camera 130 can be changed by an adjustment mechanism as shown in FIGS. 4 and 5. Further, by moving the field lens 122 forward and backward, the focal length may be changed to keep the image magnification constant.
  • ⁇ Adjustment of conjugate relationship with multispectral camera> 6 and 7 are perspective views and cross-sectional views of the multispectral camera 130, respectively.
  • an optical system including a first lens 132 and a second lens 136 is arranged in a lens barrel 131, and these lenses rotate the first lever 104 and the second lever 106, respectively.
  • the first lens 132 and the second lens 136 may be a lens group composed of a plurality of lenses.
  • the same mechanism (lens barrel, arm, slit, etc.) as in the case of the field lens unit can be used for advancing and retreating the first lens 132 and the second lens 136 (see FIGS. 4 and 5). That is, these mechanisms constitute an adjustment mechanism.
  • the lens barrel 131 is formed with a slit 108 (wavelength polarization filter unit attachment / detachment mechanism) at or near the pupil position of the image pickup device 100, and the wavelength polarization filter unit 134 (wavelength polarization) is formed in the slit 108.
  • the filter unit is inserted, and the optical axis is arranged so as to coincide with the optical axis L of the imaging optical system (first lens 132, second lens 136).
  • FIG. 8 is a diagram showing the configuration of the frame body 135, and FIG. 9 is a diagram showing the configuration of the wavelength polarizing filter unit 134.
  • the portions (a) to (f) in FIG. 8 are a rear view, a top view, a left side view, a bottom view, a perspective view, and a front view of the frame body 135 and / or the wavelength polarizing filter unit 134, respectively. Is.
  • the frame body 135 includes four opening regions 135A to 135D (plural opening regions).
  • the centers of gravity of these opening regions 135A to 135D are different from each other, and are also different from the center of gravity 135G as a whole.
  • the shapes of the opening regions 135A to 135D are not limited to the fan shape as shown in FIGS. 8 and 9, and may be other shapes such as a circle, a rectangle, and a polygon. Further, the shape and size may differ between the opening regions.
  • Filter sets 137A to 137D are arranged in these opening regions (the back surface side of the frame body 135), respectively, as shown in the portion (a) of FIG. 8 and FIG. ..
  • the filter sets 137A to 137D may be fixed with an adhesive.
  • the filter set 137A is configured by superimposing an optical filter 138A and a polarizing filter 139A.
  • the filter set 137B is configured by superimposing the optical filter 138B and the polarizing filter 139B.
  • the filter set 137C is configured by superimposing the optical filter 138C and the polarizing filter 139C.
  • the filter set 137D comprises an optical filter 138D and a polarizing filter 139D.
  • the optical filters 138A to 138D are a plurality of optical filters (color filters) including two or more optical filters that transmit light having at least a part of different wavelength bands, and the polarizing filters 139A to 139D are a plurality of polarized light having different polarization directions. It is a filter.
  • FIG. 10 is a diagram showing an example of the polarization direction of the polarizing filter, and as illustrated in the portions (a) to (d) of the figure, the polarizing directions of the polarizing filters 139A to 139D have a maximum of four directions (opening region). Same as the number of; for example, 0 °, 45 °, 90 °, 135 °).
  • the polarizing filters 139A to 139D may be filters that are polarized by a polarizing film, or may be filters that are polarized by a wire grid or a plurality of slits.
  • the wavelength polarizing filter unit 134 having the above-described configuration can be inserted and removed from the slit 108, whereby the attachment / detachment mechanism of the wavelength polarizing filter unit 134 is configured.
  • the user can select and use a wavelength polarizing filter unit having a desired wavelength band and a wavelength polarizing filter unit having a small decrease in peripheral illumination (vignetting).
  • the wavelength polarizing filter unit may be moved back and forth in the direction of the optical axis L by the mechanism (adjustment mechanism) as described above with respect to FIGS. 4 and 5. As a result, the conjugation relationship can be adjusted without changing the image magnification (see case 6 in FIG. 13).
  • the aperture region may be 3 or less, and correspondingly, the optical filter (color filter) and the polarizing filter may be 3 types or less.
  • the opening areas 135A to 135D may be shielded by a shielding member or the like.
  • FIG. 11 is a diagram showing the configuration of the image pickup device 138.
  • the image pickup element 138 is a CMOS (Complementary Metal-Oxide Semiconductor) type image pickup element (image sensor), and is a monochrome type image pickup element having a pixel array layer 211, a polarization filter element array layer 213, and a microlens array layer 215. be.
  • Each layer is arranged in the order of the pixel array layer 211, the polarizing filter element array layer 213 (a plurality of polarizing elements), and the microlens array layer 215 from the image plane side toward the object side.
  • the image sensor 138 is not limited to the CMOS type, but may be an XY address type or a CCD (Charge Coupled Device) type image sensor.
  • CCD Charge Coupled Device
  • the pixel array layer 211 is configured by two-dimensionally arranging a large number of photodiodes 212 (a plurality of pixel groups). One photodiode 212 constitutes one pixel. Each photodiode 212 is regularly arranged along the horizontal direction (x direction) and the vertical direction (y direction).
  • the polarizing filter element array layer 213 is configured by arranging four types of polarizing filter elements 214A, 214B, 214C, 214D (a plurality of polarizing elements) having different polarization directions (polarization directions of transmitted light) in a two-dimensional manner. ..
  • the polarization directions of the polarizing filter elements 214A, 214B, 214C, and 214D can be, for example, 0 °, 45 °, 90 °, and 135 °. Further, these polarization directions can correspond to the polarization directions (see FIG. 10) of the polarization filters 139A to 139D in the wavelength polarization filter unit 134 described above.
  • the image pickup device 138 includes a plurality of image groups that receive any of the light transmitted through the plurality of aperture regions by the polarizing filter elements 214A to 214D. These polarizing filter elements 214A to 214B are arranged at the same intervals as the photodiode 212, and are provided for each pixel.
  • the microlens array layer 215 includes microlenses 216 arranged in each pixel.
  • the image sensor 138 includes an analog amplification unit (not shown), an A / D converter (Analog-to-Digital Converter), and an image sensor drive unit.
  • an analog amplification unit not shown
  • an A / D converter Analog-to-Digital Converter
  • the image pickup apparatus main body 140 includes a processor 142.
  • the processor 142 (processor, computer) has each part (function) of an image acquisition unit 142A, a relative position detection unit 142B, an image support information generation unit 142C, and a display control unit 142D, and has a spectroscopic image. Acquisition (including interference removal), detection of relative position between the field lens and other optical system and / or multispectral camera, generation and output of image support information, etc.
  • the processing of the adjustment method by the processor 142 will be described in detail later.
  • the above-mentioned function of the processor 142 can be realized by using various processors.
  • the various processors include, for example, a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) to realize various functions.
  • the various processors described above include a GPU (Graphics Processing Unit), which is a processor specialized in image processing.
  • the various processors described above include PLD (Programmable Logic Device), which is a processor whose circuit configuration can be changed after manufacturing such as FPGA (Field Programmable Gate Array).
  • the above-mentioned various processors include a dedicated electric circuit, which is a processor having a circuit configuration specially designed for executing a specific process such as an ASIC (Application Specific Integrated Circuit).
  • each function of the processor 142 may be realized by one processor or may be realized by a plurality of processors. Further, one processor may support a plurality of functions. Further, each function of the processor 142 may be realized by a circuit, or a part of each function may be realized by a circuit and the rest may be realized by a processor.
  • the above-mentioned processor or electric circuit executes software (program)
  • the code that can be read by the processor (computer) of the software to be executed and the data necessary for executing the software are stored non-temporarily by flash memory 144 (Flash Memory) or the like. It is stored in a target recording medium, and the processor refers to the software or data.
  • the software stored in the non-temporary recording medium includes an adjustment program for executing the adjustment method according to the present embodiment.
  • the code or data may be recorded on a non-temporary recording medium using various optical magnetic recording devices, a semiconductor memory, or the like instead of the flash memory 144.
  • the "semiconductor memory” includes a ROM (Read Only Memory) and an EEPROM (Electronically Erasable and Programmable ROM) in addition to the flash memory.
  • ROM Read Only Memory
  • EEPROM Electrically Erasable and Programmable ROM
  • RAM 146 is used as a temporary storage area.
  • the processor 142 detects the position of the field lens 122 (the position relative to the optical system 20 and the multispectral camera 130), and based on this position. Information necessary for adjusting the conjugate relationship (image support information) may be generated and output.
  • FIG. 13 is a table showing an example of the relationship between the relative position of the field lens and the elements to be adjusted. Further, FIG. 14 is a diagram showing a state of adjustment in cases 1 to 4, and FIG. 15 is a diagram showing a state of adjustment in case 5. As shown in these figures, a plurality of elements may be moved together. In this way, which element should be adjusted depends on the relative position and other conditions.
  • the relative position detection unit 142B (processor 142: see FIG. 12) is a field lens 122 and a multispectral camera 130 by means of a photo interrupter (not shown), an MR sensor (MR: Magneto Resistive Sensor / magnetoresistive sensor), and the like.
  • MR Magneto Resistive Sensor / magnetoresistive sensor
  • the positions of the lens (first lens 132, second lens 136) and the wavelength polarization filter unit 134 can be detected.
  • step S100 The user attaches the field lens unit 120 and the multispectral camera 130 to the image side of the optical system 20 (another optical system) (step S100), and the optical axis of the optical system 20, the field lens unit 120, and the multispectral camera 130. Make adjustments.
  • the image acquisition unit 142A (processor 142) uses the optical filters 138A to 138D (plural) based on the plurality of image signals output from the image sensor 138.
  • a plurality of images (spectral images) corresponding to the wavelength bands of the optical filter) are generated (step S120: imaging step).
  • the three aperture regions 135A to 135C are used by shielding any one of the aperture regions 135A to 135D (for example, the aperture region 135D) (that is, the image for the three wavelength bands ⁇ 1 to ⁇ 3). To obtain).
  • the ratio (interference ratio) at which the light of each wavelength band ⁇ 1 to ⁇ 3 emitted from the image pickup apparatus 100 is received by each pixel is the setting of the wavelength bands ⁇ 1 to ⁇ 3 of the light transmitted by the optical filters 138A to 138C, and the polarizing filter 139A to. It is uniquely determined from the setting of the polarization direction of the light transmitted by the 139C and the polarization direction (four directions) of the light received by each pixel of the image pickup element 138, and can be obtained in advance.
  • the image acquisition unit 142A uses a plurality of shielding members for shielding the interference ratio other than the specific aperture region among the plurality of aperture regions, and one of the plurality of shielding members is attached to the lens device. It can be calculated from a plurality of images acquired in the state of being.
  • the image acquisition unit 142A calculates a coefficient group (each element of the interference elimination matrix) for the interference elimination process from these images, and stores these coefficient groups in the flash memory 144.
  • the image acquisition unit 142A calculates a pixel signal corresponding to the wavelength bands ⁇ 1 to ⁇ 3 from the pixel signals obtained from each pixel, and uses the coefficient group acquired from the flash memory 144 to create an image (interference) in the wavelength bands ⁇ 1 to ⁇ 3. Is removed, and a spectroscopic image) is generated. Images in the wavelength bands ⁇ 1 to ⁇ 3 are output to the outside and stored in a storage device (not shown) as needed. Further, the display control unit 142D (processor) displays the spectroscopic image on the display device 300 (display device) (step S125: imaging step). The display of the spectroscopic image and the adjustment of the conjugate relationship based on the light / dark information of the spectroscopic image may be performed for all wavelength bands or for some wavelength bands.
  • the image support information generation unit 142C (processor) generates information (image support information) necessary for adjusting the conjugate relationship based on the light / dark information (light amount distribution) of at least one spectral image obtained by the multispectral camera 130.
  • Step S130 Generation step.
  • the image support information includes, for example, attachment / detachment or replacement of the mount adapter 110 between the optical system 20 (another optical system) and the multispectral camera 130, replacement of the field lens unit 120 and / or the multispectral camera 130, and field lens 122.
  • At least of the advancing / retreating direction of the first lens 132 and / or the advancing / retreating direction of the second lens 136, the advancing / retreating direction of the wavelength polarizing filter unit 134, and the replacement of the wavelength polarizing filter unit 134 may be replaced.
  • One may be included.
  • the image support information generation unit 142C can generate the above-mentioned image support information based on the direction of the aperture region and the light / dark information of the spectroscopic image. Specifically, as described above with respect to FIG. 8 and the like, the image support information can be generated by utilizing the fact that the center of gravity of the opening regions 135A to 135D is different from the center of gravity 135G as a whole.
  • FIG. 18 is a conceptual diagram showing the relationship between the direction of the aperture region and the light / dark information of the spectroscopic image.
  • an optical filter in this case, in this case
  • Optical filters 138B to 138D arranged in the aperture regions 135B to 135D
  • the first lens 132 is not shown in FIG.
  • the image 133 is an image of the wavelength polarization filter unit 134 formed by the first lens 132 (the entrance pupil of the multispectral camera 130).
  • Part (a) in FIG. 18 is a “diaphragm” (short thick line in the figure; wavelength polarizing filter) for such a specific wavelength band of light rays (upper ray 150, main ray 152, lower ray 154) transmitted through the field lens 122. It indicates a state in which it is not blocked (no vignetting has occurred) by (indicated by 134, which is the reference code of). In this state, no light-dark distribution occurs on the light-receiving surface of the image sensor 138.
  • the main ray 152 and the lower ray 154 are not blocked, but are above.
  • the light beam 150 is blocked by the "aperture", resulting in a brighter upper side and a darker lower side of the figure (limb darkening).
  • the upper ray 150 and the main ray 152 are not blocked, but the lower ray 154. Is blocked by the "aperture", and as a result, the lower side of the figure becomes brighter and the upper side becomes darker (the occurrence of limb darkening).
  • image support information can be generated in consideration of the relationship between the direction of such an opening region and the light / dark information of the spectroscopic image. For example, when the upper side of the spectroscopic image is bright and the lower side is dark, it is considered that the state is as shown in the portion (b) of FIG. By approaching the state shown in (1), the conjugation relationship of the pupil can be adjusted so that the light amount distribution of the spectroscopic image can be uniformly approached.
  • the display control unit 142D causes the display device 300 to display (output) the image support information (in the above example, the moving direction of the field lens 122) generated in this way (step S140: output step).
  • the message "Please lower the field lens backward" (message indicating the moving direction of the field lens 122) can be displayed (output) on the display device 300.
  • the display control unit 142D displays (outputs) a symbol such as an arrow indicating the moving direction (forward or alternate) in place of or in addition to displaying such a message (image support information) in characters. You may. Further, the display control unit 142D may display (output) the adjustment procedure as the image support information.
  • the display device 300 displays at least one of the image support information and the adjustment procedure as the image support information according to the output from the display control unit 142D.
  • the user can lower the field lens 122 backward (retract in the direction of the optical axis L) according to this image support information (step S150), and when adjustment (movement) is made, the image acquisition unit 142A and the display control unit 142D (processor). ) Makes the display device 300 display the image (spectral image) in the state after the movement (step S160: display step).
  • the image support information generation unit 142C determines whether or not the brightness of the image in the moved state is equal to or higher than the threshold value (step S170: brightness determination step, output step), and if the determination is affirmed (step S170: brightness determination step, output step). That is, when the image becomes brighter than the threshold value and vignetting is reduced), the process is terminated.
  • the image support information generation unit 142C for example, "sum of the brightness values of the entire screen when the maximum brightness of the image is standardized to 1" and "the reciprocal of the distance to the position of the center of gravity of the brightness value at the center of the screen". It can be adopted as the definition of "brightness". Even if the image support information generation unit 142C and the display control unit 142D display the image support information indicating that the adjustment of the conjugate relationship is completed on the display device 300 when the determination in step S170 is affirmed and the process is terminated. good.
  • step S170 determines whether or not the image has become brighter due to the movement (step S180: brightness determination step, output step). If this determination is denied, the image support information generation unit 142C and the display control unit 142D generate, as image support information, a message prompting the field lens 122 to reverse the direction of movement and display it on the display device 300 ( Step S190: generation step, output step). In step S195, the image support information generation unit 142C sets the brightness after movement as a new brightness (brightness determination step, generation step, output step), and returns to step S140. In the examples shown in FIGS. 16 and 17, the conjugate relationship can be adjusted in the same manner as in driving the focus lens by the “mountain climbing method” in a general camera system.
  • the conjugate relationship can be adjusted in this way, and a multispectral image with good image quality is obtained. be able to.
  • FIG. 19 is a table showing the influence of the shape of the opening region on the adjustment of the conjugate relationship.
  • the "Parameters related to the movement of the field lens” column summarizes the characteristics related to the movement of the field lens
  • the "Detachability” column summarizes the characteristics related to the attachment / detachment / replacement of the field lens
  • the "Other” column summarizes the characteristics related to the field lens. It summarizes the basic characteristics when using in combination.
  • the influence on the conjugation relationship adjustment differs depending on the shape of the opening region.
  • the centers of gravity of the plurality of aperture regions are different from each other (each aperture region is arranged asymmetrically with respect to the center of gravity of the entire aperture region).
  • the centers of gravity of the plurality of aperture regions are the same (for example, when they are concentric circles)
  • the amount of peripheral light is reduced and the amount of movement of the field lens is large.
  • the center of gravity of the aperture region is different, it is easy to determine the moving direction of the field lens as in the example described later.
  • the attachment / detachment of the field lens and the applicable F number also depend on the shape of the aperture region.
  • the image support information generation unit 142C and the display control unit 142D can generate and display image support information in consideration of such circumstances.
  • FIG. 20 is a diagram showing a specific example (when the field lens 122 is moved) of the conjugate relationship adjustment in consideration of the shape of the aperture region.
  • the opening regions 160A to 160D are fan-shaped, and their centers of gravity are different.
  • the remaining opening regions 160A, 160C, 160D act as a kind of "aperture” or "shielding member" as described above.
  • Each aperture region shall be fitted with an optical filter that differs in at least a portion of the wavelength band).
  • the image support information generation unit 142C and the display control unit 142D urge the user to increase the distance between the field lens 122 and the multispectral camera 130 (the field lens 122 is extended forward) (image support).
  • Information is generated (generation step, output step) and displayed on the display device 300 (output step).
  • the part (b4) in FIG. 20 is an example of a spectroscopic image in a state where the user moves the field lens 122 based on the image assist information, but in this example, a dark region still remains on the right side of the image. Therefore, the image support information generation unit 142C and the display control unit 142D again generate information prompting the user to increase the distance between the field lens 122 and the multispectral camera 130, and display the information on the display device 300.
  • the part (b4) in FIG. 20 is an example of a spectroscopic image in a state where the user moves the field lens again, and the brightness becomes equal to or higher than the threshold value and the distribution of light and darkness is reduced, so that the process is terminated.
  • the portion (c1) in FIG. 20 is a diagram showing an opening region 160D in which the left side is open, and in this case, the right side of the spectroscopic image becomes bright as shown in the portion (c2) in the same figure. Therefore, the image support information generation unit 142C and the display control unit 142D also prompt the user to increase the distance between the field lens 122 and the multispectral camera 130, as in the parts (b2) to (b4) in the figure. Image support information) is generated (generation step, output step) and displayed on the display device 300 (output step). As a result, as shown in the portion (c4) of FIG. 20, when the brightness becomes equal to or more than the threshold value and the distribution of light and dark becomes small, the process ends.
  • the image pickup device imaging device 100, image pickup system 10
  • the adjustment method, and the adjustment program according to the first embodiment it is possible to acquire a multispectral image with good image quality.
  • FIG. 21 is a diagram showing a state of connection with a microscope 30 (an aspect of “another optical system”).
  • the microscope 30 includes an optical system including an objective lens 32, and the user can observe a specimen or the like placed on the stage 34 through the eyepiece 36.
  • the microscope 30 is provided with a camera connection unit 38, and an image pickup device 100 (one aspect of the image pickup device according to the present invention) can be connected to the camera connection unit 38.
  • a part of the luminous flux from the specimen or the like is guided to the image pickup apparatus 100 via the camera connection portion 38, and the conjugate relationship of the pupils is adjusted by the image pickup apparatus 100 in the same manner as described above for the first embodiment. It is possible to acquire a multispectral image of image quality.
  • the processing of the adjustment method is performed by the processor 142 and the computer 330 (processor), and the acquired image can be displayed on the display device 300. The user can perform operations necessary for these processes via the operation unit 320.
  • the above-mentioned image pickup apparatus main body 140 may be connected to the side of the microscope 30, or may be configured integrally with the computer 330 as shown in FIG. 21.
  • FIG. 22 is a diagram showing a state of connection with the zoom optical system. Also in this case, similarly to the first embodiment and the aspect of FIG. 21, the conjugate relationship of the pupils can be adjusted by the image pickup apparatus 100 to obtain a multispectral image with good image quality.
  • the polarizing filters 139A to 139D provided in the wavelength polarizing filter unit 134 and the polarizing filter elements 214A to 214D provided in the image pickup device 138 receive light that has passed through any of the aperture regions.
  • the present invention is not limited to such an embodiment, and if the image pickup device uses a pupil-split type multispectral camera, the conjugate relationship can be adjusted by moving the field lens or the like without using polarized light. This can be done so that a multispectral image with good image quality can be obtained.
  • the “other optical system” and the image pickup device (imaging device 100) according to the present invention are connected via a mount adapter 110 (adjustment mechanism) or the like, and each element is also connected in the image pickup device 100. Although connected, in the present invention these elements do not necessarily have to be mechanically connected.
  • the optical system 20 (another optical system) and the image pickup device 101 (imaging device) can be arranged apart from each other, and the image pickup device 101 also has a field lens unit.
  • the 120 and the multispectral camera 130 can be arranged apart from each other.
  • the optical system 20, the field lens unit 120, and the multispectral camera 130 are each held by a member (not shown) such as a tripod or a guide rail, or placed on an adjustment table.
  • the spacing of each element can be adjusted, the lenses (field lens 122, first lens 132, second lens 136) can be moved forward and backward, and can be attached / detached / replaced in the same manner as in the above-described embodiment.
  • This makes it possible to adjust the conjugation relationship of the pupil and obtain a multispectral image with good image quality.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Chemical & Material Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Analytical Chemistry (AREA)
  • Studio Devices (AREA)
  • Blocking Light For Cameras (AREA)
  • Spectrometry And Color Measurement (AREA)
  • Lens Barrels (AREA)
  • Camera Bodies And Camera Details Or Accessories (AREA)
  • Polarising Elements (AREA)
PCT/JP2021/027296 2020-07-28 2021-07-21 撮像装置、調整方法、及び調整プログラム Ceased WO2022024917A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2022540241A JP7522198B2 (ja) 2020-07-28 2021-07-21 撮像装置、調整方法、及び調整プログラム
US18/153,359 US12181663B2 (en) 2020-07-28 2023-01-12 Imaging device, adjustment method, and adjustment program

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020-127274 2020-07-28
JP2020127274 2020-07-28

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/153,359 Continuation US12181663B2 (en) 2020-07-28 2023-01-12 Imaging device, adjustment method, and adjustment program

Publications (1)

Publication Number Publication Date
WO2022024917A1 true WO2022024917A1 (ja) 2022-02-03

Family

ID=80036839

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/027296 Ceased WO2022024917A1 (ja) 2020-07-28 2021-07-21 撮像装置、調整方法、及び調整プログラム

Country Status (3)

Country Link
US (1) US12181663B2 (https=)
JP (1) JP7522198B2 (https=)
WO (1) WO2022024917A1 (https=)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023157396A1 (ja) * 2022-02-15 2023-08-24 富士フイルム株式会社 レンズ装置、情報処理装置、プログラム、及び撮像装置の製造方法
WO2024042783A1 (ja) * 2022-08-22 2024-02-29 富士フイルム株式会社 画像処理装置、画像処理方法、及びプログラム

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004219397A (ja) * 2002-11-20 2004-08-05 Nikon Corp 光学系、それを備えた固体撮像素子照明装置及び固体撮像素子の検査方法
JP2008064933A (ja) * 2006-09-06 2008-03-21 Opcell Co Ltd 光学ユニット
JP2019082412A (ja) * 2017-10-31 2019-05-30 株式会社ニコン 撮像装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9395516B2 (en) * 2012-05-28 2016-07-19 Nikon Corporation Imaging device
JP7292848B2 (ja) * 2018-10-16 2023-06-19 キヤノン株式会社 光学装置及びそれを備える撮像システム
CN114631057B (zh) * 2019-10-30 2024-07-26 富士胶片株式会社 摄像装置、摄像光学系统及摄像方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004219397A (ja) * 2002-11-20 2004-08-05 Nikon Corp 光学系、それを備えた固体撮像素子照明装置及び固体撮像素子の検査方法
JP2008064933A (ja) * 2006-09-06 2008-03-21 Opcell Co Ltd 光学ユニット
JP2019082412A (ja) * 2017-10-31 2019-05-30 株式会社ニコン 撮像装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023157396A1 (ja) * 2022-02-15 2023-08-24 富士フイルム株式会社 レンズ装置、情報処理装置、プログラム、及び撮像装置の製造方法
WO2024042783A1 (ja) * 2022-08-22 2024-02-29 富士フイルム株式会社 画像処理装置、画像処理方法、及びプログラム

Also Published As

Publication number Publication date
US12181663B2 (en) 2024-12-31
JPWO2022024917A1 (https=) 2022-02-03
US20230204950A1 (en) 2023-06-29
JP7522198B2 (ja) 2024-07-24

Similar Documents

Publication Publication Date Title
JP4483951B2 (ja) 撮像装置
JP5901246B2 (ja) 撮像装置
JP6584149B2 (ja) 撮像装置
JP4941332B2 (ja) 撮像装置
JP5067154B2 (ja) 撮像装置
CN102422216B (zh) 测距装置以及摄像装置
JP7373592B2 (ja) レンズ装置、撮像装置、撮像方法、撮像プログラム、及び記録媒体
US12177550B2 (en) Lens device for forming image, and imaging apparatus and imaging method with lens device
CN102348060A (zh) 图像拍摄设备、图像拍摄装置、图像拍摄方法和程序
JP7478727B2 (ja) アライメント支援のための一体化瞳カメラを有する低コスト眼底撮像器
JP2001083400A (ja) 撮像光学系
JP7100735B2 (ja) 撮像素子及び撮像装置
US20220038613A1 (en) Imaging device
WO2022024917A1 (ja) 撮像装置、調整方法、及び調整プログラム
JP2021135404A (ja) レンズ装置、撮像装置、撮像方法、及び撮像プログラム
JP5251323B2 (ja) 撮像装置
US12197109B2 (en) Lens device and imaging apparatus
JP2018056799A (ja) プロジェクターシステム及び撮像装置
JP6997295B2 (ja) 撮像装置、撮像方法、及びプログラム
WO2022092007A1 (ja) 光学部材、レンズ装置及び撮像装置
JPWO2022024917A5 (https=)
JP4545418B2 (ja) 眼底カメラ
JP2010102281A (ja) 可視光・赤外光撮影用レンズアダプタ
CN107924107B (zh) 摄像装置
JPH08110486A (ja) 内視鏡装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21850569

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022540241

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21850569

Country of ref document: EP

Kind code of ref document: A1