WO2023157396A1

WO2023157396A1 - Lens device, information processing device, program, and method for manufacturing imaging device

Info

Publication number: WO2023157396A1
Application number: PCT/JP2022/041531
Authority: WO
Inventors: 睦川中子; 和佳岡田; 慶延岸根; 友也平川; 高志椚瀬
Original assignee: 富士フイルム株式会社
Priority date: 2022-02-15
Filing date: 2022-11-08
Publication date: 2023-08-24

Abstract

This lens device has a polarization unit disposed on an optical path. The polarization unit has a plurality of openings located on the optical path. A first polarizer is provided to at least one of the openings. The lens device has a changing mechanism for changing the angle of a transmission axis of the first polarizer.

Description

LENS DEVICE, INFORMATION PROCESSING DEVICE, PROGRAM, AND IMAGE SENSING DEVICE MANUFACTURING METHOD

The technology of the present disclosure relates to a lens device, an information processing device, a program, and a method of manufacturing an imaging device.

Japanese Patent Application Laid-Open No. 2001-042380 discloses a photographing apparatus in which an optical filter is arranged in the optical path of a condensing optical system having at least one lens, wherein the optical filter covers the required angle of view on the imaging plane. An imaging device is disclosed that is positioned near the most closed position.

International Publication No. 2019/155908 pamphlet has a disk provided with a plurality of filters including an ND filter whose transmittance is continuously variable, and among the plurality of filters provided on the disk, the rotation direction of the disk A filter unit is disclosed in which a filter corresponding to the posture of is arranged on the optical axis of incident light to an image sensor.

Japanese Patent Application Laid-Open No. 2014-183516 discloses a camera having a polarizing filter, photographing means, detection means, photometry means, and control means. The polarizing filter is rotatable on a rotation axis that includes and is parallel to the optical axis of the optical lens. A photographing means photographs an image through an optical lens and a polarizing filter. The detection means detects the rotation angle of the polarizing filter with respect to the reference angle. The photometry means measures the luminance value of light incident through the optical lens and the polarizing filter. The control means obtains a first rotation angle of the polarizing filter at which the luminance value measured by the photometry means is maximized, and the first rotation angle of the polarization filter photographed by the photographing means at the first rotation angle of the polarization filter at which the luminance value is maximized. A difference image is created by obtaining a difference between the image and a second image captured by the imaging means at a second rotation angle of the polarizing filter designated in advance, and the first image and the difference image are stored in association with each other. Record on media.

In International Publication No. 2019/102609 pamphlet, an image sensor, a first polarizing filter that is arranged in a light receiving path of the image sensor and allows linearly polarized light to pass through, and a first polarizing filter that changes the direction of the polarization axis of the first polarizing filter A monitoring device is disclosed comprising a device, a light source, a second polarizing filter disposed in the light projection path of the light source for passing linearly polarized light, and a second device for changing the direction of the polarization axis of the second polarizing filter. there is

In International Publication No. 2019/116646 pamphlet, a polarizer in which a plurality of types of polarizing members are arranged by rotating or symmetrically moving the polarizing members having the same internal structure, and a plurality of types of polarizing members. An imaging device is disclosed that includes a photoelectric conversion element that converts the emitted light into an electric charge.

One embodiment according to the technology of the present disclosure reduces the error even if an error occurs in the angle of the transmission axis of the first polarizer due to, for example, variations in the mounting angle of the lens device with respect to the body of the imaging device. Provided are a lens device, an information processing device, a program, and a method for manufacturing an imaging device that can

A first aspect of the technology of the present disclosure is a lens apparatus having a polarizing unit arranged in an optical path, the polarizing unit having a plurality of apertures positioned in the optical path, at least of the plurality of apertures A first polarizer is provided in one aperture, and the lens device is a lens device having a changing mechanism for changing the angle of the transmission axis of the first polarizer.

A second aspect of the technology of the present disclosure is the lens device according to the first aspect, wherein the changing mechanism has a rotating member that rotates around the optical axis of the lens device, and the rotating member is coupled to the polarization unit. It is a lens device that has

A third aspect of the technology of the present disclosure is the lens device according to the second aspect, wherein the rotation member connects the first member and the polarization unit with the polarization unit arranged in the optical path. and a second member for providing a lens device.

A fourth aspect of the technology of the present disclosure is the lens device according to the third aspect, wherein the first member is a ring member having a slot in a partial region, and the second member is a slot It is a lens device that is assembled to the

A fifth aspect of the technology of the present disclosure is a lens device according to any one of the second to fourth aspects, wherein a lens barrel that rotatably supports a rotating member; and a fixing member that fixes the rotating member.

A sixth aspect of the technology of the present disclosure is the lens device according to any one of the first to fifth aspects, wherein the lens device includes a first image sensor having a second polarizer. 1 is a lens device having a first mount attached to a second mount provided on an imaging device body.

A seventh aspect of the technology of the present disclosure is the lens device according to the sixth aspect, wherein the first mount is formed with a first threaded portion that is screwed with the second threaded portion formed on the second mount. It is a lens device that has

An eighth aspect of the technology of the present disclosure is the lens device according to the sixth aspect or the seventh aspect, wherein the first mount and the second mount are C mounts.

A ninth aspect of the technology of the present disclosure is applied to an imaging device including the lens device according to any one of the first to eighth aspects and a second imaging device body, and a processor wherein the processor, when the lens device is attached to the second imaging device body, first information that is output data from a second image sensor provided in the second imaging device body is an information processing device that outputs second information about an angle based on .

A tenth aspect of the technology of the present disclosure is the information processing device according to the ninth aspect, wherein the processor outputs a signal indicating whether or not the angle is within the first predetermined angle range based on the second information. It is an information processing device that

An eleventh aspect of the technology of the present disclosure is the information processing device according to the ninth or tenth aspect, wherein the information processing device is an imaging device.

A twelfth aspect of the technology of the present disclosure is the information processing device according to any one of the ninth to eleventh aspects, wherein the first information is the spectrum of light incident on the second image sensor and It is an information processing device that is information based on the optical properties of the polarizing unit.

A thirteenth aspect of the technology of the present disclosure is the information processing device according to any one of the ninth to twelfth aspects, wherein the second information is the number of pixels included in the second image sensor. It is an information processing device that is information based on output data output from at least one specific pixel.

A fourteenth aspect of the technology of the present disclosure is the information processing device according to any one of the ninth to thirteenth aspects, wherein the second information is at least a luminance value, a degree of polarization, and a polarization angle. It is an information processing device including one.

A fifteenth aspect of the technology of the present disclosure is an information processing device according to any one of the ninth to fourteenth aspects, wherein the processor outputs the second information to the display. .

A sixteenth aspect of the technology of the present disclosure is an information processing device according to any one of the ninth to fifteenth aspects, further comprising a memory for storing second information.

A seventeenth aspect of the technology of the present disclosure is the information processing device according to any one of the ninth to sixteenth aspects, wherein the processor performs interference elimination processing on the output data. is an information processing device that acquires a multispectral image.

An eighteenth aspect of the technology of the present disclosure is processing applied to an imaging device including the lens device according to any one of the first to eighth aspects and a second imaging device body wherein the processing is output data from the second image sensor provided in the second imaging device body when the lens device is attached to the second imaging device body A program including outputting second information about an angle based on first information.

A nineteenth aspect of the technology of the present disclosure is an imaging device for assembling an imaging device including the lens device according to any one of the first to eighth aspects and a second imaging device body. A manufacturing method of an imaging device, comprising: attaching a lens device to a second imaging device body; and changing an angle with a changing mechanism while the lens device is attached to the second imaging device body. is.

A twentieth aspect of the technology of the present disclosure is the method of manufacturing an imaging device according to the nineteenth aspect, wherein the change of the angle is performed when focusing is performed using the lens device. manufacturing method.

A twenty-first aspect of the technology of the present disclosure is, in the imaging device manufacturing method according to the nineteenth aspect or the twentieth aspect, the output data from the second image sensor provided in the second imaging device body. The imaging device manufacturing method further comprises acquiring second information about an angle based on the first information, and changing the angle based on the second information.

A twenty-second aspect of the technology of the present disclosure is the imaging device manufacturing method according to any one of the nineteenth to twenty-first aspects, wherein the angle falls within the first predetermined angle range based on the second information. The imaging device manufacturing method further includes outputting a signal indicating whether or not the imaging device is settled.

A twenty-third aspect of the technology of the present disclosure is the method for manufacturing an imaging device according to any one of the nineteenth aspect to the twenty-second aspect, when the angle falls within the second predetermined angle range, A method of manufacturing an imaging device, further comprising fixing a polarizing unit to a barrel of a lens device.

It is a perspective view which shows an example of an imaging device. 2 is an exploded side view showing an example of a lens device and an imaging device body; FIG. It is an exploded perspective view showing an example of a lens device. FIG. 4 is an exploded perspective view showing an example of a pupil division filter; 1 is an exploded longitudinal sectional view showing an example of a lens device; FIG. It is a longitudinal cross-sectional view showing an example of an assembled state of the lens device. It is a block diagram which shows an example of the hardware constitutions of a lens apparatus. It is explanatory drawing which shows an example of a photoelectric conversion element. It is a figure which shows an example of the 1st process of the manufacturing method of an imaging device. It is a figure which shows an example of the 2nd process of the manufacturing method of an imaging device. It is a figure which shows an example of the 3rd process of the manufacturing method of an imaging device. It is a figure which shows an example of the 4th process of the manufacturing method of an imaging device. It is a figure which shows an example of the 5th process of the manufacturing method of an imaging device. It is a figure which shows an example of the 6th process of the manufacturing method of an imaging device. It is a figure which shows an example of the 7th process of the manufacturing method of an imaging device. 4 is a flow chart showing an example of the flow of a method for manufacturing an imaging device; 2 is a block diagram showing an example of a functional configuration of an imaging device; FIG. FIG. 10 is an explanatory diagram showing an example of a first operation of the processor of the imaging device; FIG. 10 is an explanatory diagram showing an example of a second operation of the processor of the imaging device; 4 is a flow chart showing an example of a flow of operations of a processor of an imaging device; It is a block diagram which shows an example of the hardware constitutions of a determination assistance apparatus. It is a block diagram which shows an example of a functional structure of a determination support apparatus. FIG. 4 is an explanatory diagram showing an example of the operation of the processor of the determination support device; 4 is a flow chart showing an example of the operation flow of the processor of the determination support device; It is a block diagram which shows an example of the manufacturing system which concerns on a modification. It is a block diagram which shows an example of the imaging device which concerns on a modification.

An example of an embodiment of a lens device, an information processing device, a program, and a method for manufacturing an imaging device according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

First, the wording used in the following explanation will be explained.

　I/F is an abbreviation for "Interface". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". CCD is an abbreviation for "Charge Coupled Device". NVM is an abbreviation for "Non-Volatile Memory". RAM is an abbreviation for "Random Access Memory". CPU is an abbreviation for "Central Processing Unit". GPU is an abbreviation for "Graphics Processing Unit". EEPROM is an abbreviation for "Electrically Erasable and Programmable Read Only Memory". HDD is an abbreviation for "Hard Disk Drive". TPU is an abbreviation for "Tensor processing unit". SSD is an abbreviation for "Solid State Drive". USB is an abbreviation for "Universal Serial Bus". ASIC is an abbreviation for "Application Specific Integrated Circuit". FPGA is an abbreviation for "Field-Programmable Gate Array". PLD is an abbreviation for "Programmable Logic Device". SoC is an abbreviation for "System-on-a-chip." IC is an abbreviation for "Integrated Circuit".

In the description of this specification, the "center" is an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, in addition to the perfect center, and is not contrary to the spirit of the technology of the present disclosure. It refers to the center in the sense of including the error of In the description of this specification, "orthogonal" is an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, in addition to perfect orthogonality, and is not contrary to the spirit of the technology of the present disclosure. It refers to orthogonality in the sense of including the error of In the description of this specification, a "straight line" is an error that is generally allowed in the technical field to which the technology of the present disclosure belongs, in addition to a perfect straight line, and is not contrary to the spirit of the technology of the present disclosure. It refers to a straight line in the sense of including the error of

Next, the structural configuration of the imaging device 10 according to this embodiment will be described.

As shown in FIG. 1 as an example, the imaging device 10 is a multispectral camera capable of outputting a multispectral image 128 and includes a lens device 12 and an imaging device body 14 . The imaging device 10 is an example of an “imaging device” according to the technology of the present disclosure. The lens device 12 is an example of a “lens device” according to the technology of the present disclosure. The imaging device body 14 is an example of the "first imaging device body" according to the technology of the present disclosure. Multispectral image 128 is an example of a “multispectral image” according to the technology of this disclosure.

As shown in FIG. 2 as an example, the lens device 12 has a first mount 16 and the imaging device body 14 has a second mount 18 . Each of the first mount 16 and the second mount 18 is a C-mount and is formed in an annular shape. The first mount 16 is provided coaxially with the optical axis OA of the lens device 12 . A screw 20 is formed on the outer peripheral surface of the first mount 16 , and a screw hole 22 is formed inside the second mount 18 .

The first mount 16 is attached to the second mount 18 by screwing the screw 20 into the screw hole 22 . The lens device 12 is attached to the imaging device body 14 by attaching the first mount 16 to the second mount 18 . The first mount 16 is an example of a "first mount" according to the technology of the present disclosure. The second mount 18 is an example of a "second mount" according to the technology of the present disclosure. The screw 20 is an example of the "first threaded portion" according to the technology of the present disclosure, and the screw hole 22 is an example of the "second threaded portion" according to the technology of the present disclosure.

A screw hole 22 may be formed inside the first mount 16 and a screw 20 may be formed on the outer peripheral surface of the second mount 18 . In this case, the screw hole 22 is an example of the "first screw portion" according to the technology of the present disclosure, and the screw 20 is an example of the "second screw portion" according to the technology of the present disclosure.

As shown in FIG. 3 as an example, the lens device 12 has a pupil division filter 24, a connecting pin 26, a lens barrel 28, a rotating member 30, an end frame 32, and a set screw 34.

As shown in FIG. 4 as an example, the pupil division filter 24 has a frame 36, filters 38A to 38C, polarizers 40A to 40C, and a shielding member 42. Pupil division filter 24 is an example of a “polarization unit” according to the technology of the present disclosure.

The frame 36 has openings 44A to 44D. Hereinafter, the openings 44A to 44D will be referred to as "openings 44" when there is no need to distinguish between the openings 44A to 44D. A plurality of apertures 44 are arranged along the direction around the optical axis OA. The opening 44 is an example of an "opening" according to the technology of the present disclosure.

The filters 38A to 38C are provided in the openings 44A to 44C, respectively, and the shielding member 42 is provided in the opening 44D. Filter 38A has a first transmission wavelength band λ ₁ , filter 38B has a second transmission wavelength band λ ₂ , and filter 38C has a third transmission wavelength band λ ₃ . The first transmission wavelength band λ ₁ , the second transmission wavelength band λ ₂ , and the third transmission wavelength band λ ₃ are wavelength bands different from each other. The first transmission wavelength band λ ₁ , the second transmission wavelength band λ ₂ , and the third transmission wavelength band λ ₃ may each be set to arbitrary bands. Hereinafter, the filters 38A to 38C will be referred to as "filters 38" when there is no need to distinguish between the filters 38A to 38C.

The polarizers 40A to 40C are provided in the openings 44A to 44C, respectively, and overlapped with the filters 38A to 38C. The polarizer 40A is a polarizer whose transmission axis is set at an azimuth angle of 0°. The polarizer 40B is a polarizer whose transmission axis is set at an azimuth angle of 45°. The polarizer 40C is a polarizer whose transmission axis is set at an azimuth angle of 90°. Hereinafter, the polarizers 40A to 40C will be referred to as "polarizers 40" when there is no need to distinguish between the polarizers 40A to 40C. The polarizer 40 is an example of the "first polarizer" according to the technology of the present disclosure.

Although the number of openings 44 is four in the example shown in FIG. 4, the number of openings 44 may be any number. Also, although the number of filters 38 is three in the example shown in FIG. 4, the number of filters 38 may be any number. Also, although the number of polarizers 40 is three in the example shown in FIG. 4, the number of polarizers 40 may be any number.

Although the number of filters 38 is less than the number of apertures 44 in the example shown in FIG. Also, although the number of polarizers 40 is less than the number of apertures 44 in the example shown in FIG. In the pupil division filter 24 , the polarizer 40 may be provided in at least one aperture 44 among the plurality of apertures 44 .

As shown in FIGS. 5 and 6 as an example, the lens device 12 has a first lens 46 and a second lens 48 . Also, the lens barrel 28 has a main body 50 , a lens frame 52 and an outer cylinder 54 . The lens frame 52 is provided inside the body 50 , and the outer cylinder 54 is provided outside the body 50 . The outer cylinder 54 and the lens frame 52 are connected. The lens frame 52 holds the first lens 46 . The first lens 46 is, for example, a focus lens. A lever 56 is provided on the outer cylinder 54 . A cam (not shown) and/or a helicoid (not shown) are provided between the lens frame 52 and the main body 50 .

When the lens frame 52 rotates around the optical axis OA integrally with the outer cylinder 54 by moving the lever 56 in the direction around the optical axis OA, the lens frame 52 is integrated with the first lens 46 by the cam and/or the helicoid. , in the direction of the optical axis OA. The focus of the lens device 12 is adjusted by moving the first lens 46 in the direction of the optical axis OA.

The lens barrel 28 is formed with an insertion opening 58 for inserting the pupil division filter 24 inside the lens barrel 28 . The insertion opening 58 opens in a direction perpendicular to the optical axis OA. A slit 60 is formed in the inner peripheral surface of the lens barrel 28 . The slit 60 extends in an arc around the optical axis OA.

The pupil division filter 24 is inserted inside the lens barrel 28 through the insertion opening 58 . The pupil division filter 24 is arranged in the optical path 62 provided inside the lens device 12 by being inserted inside the lens barrel 28 . A plurality of apertures 44 (see FIG. 4) are located in the optical path 62 when the pupil division filter 24 is placed in the optical path 62 . The optical path 62 is an example of an "optical path" according to the technology of the present disclosure.

When the pupil division filter 24 is arranged in the optical path 62 , the frame 36 of the pupil division filter 24 is inserted into the slit 60 . The pupil division filter 24 is rotatably supported by the lens barrel 28 around the optical axis OA with the frame 36 inserted into the slit 60 . The connecting pin 26 is fixed to the upper portion of the frame 36 and extends above the frame 36 . The pupil division filter 24 rotates about the optical axis OA with respect to the lens barrel 28 within the range in which the connecting pin 26 moves inside the insertion opening 58 .

The rotating member 30 is a member for changing the angle of the transmission axis of each polarizer 40 (see FIG. 4) provided in the pupil division filter 24. The rotary member 30 is formed in an annular shape and provided outside the lens barrel 28 . The rotating member 30 is rotatably supported by the lens barrel 28 around the optical axis OA. The rotating member 30 is an example of a “changing mechanism” and a “rotating member” according to the technology of the present disclosure. The lens barrel 28 is an example of a "lens barrel" according to the technology of the present disclosure.

The rotating member 30 specifically has a ring member 64 and a lid member 66 . The ring member 64 is formed in an annular shape. The ring member 64 is rotatably attached to the outer peripheral surface of the lens barrel 28 . Ring member 64 has a notch 68 . The cutout portion 68 is formed in a concave shape by cutting out a portion of the ring member 64 in the circumferential direction. The notch portion 68 is an example of a “slot portion” according to the technology of the present disclosure. The ring member 64 may be made of metal or resin. Moreover, the notch portion 68 may be formed by notching, or may be formed by molding using a mold.

The lid member 66 has a shape corresponding to the notch portion 68 and is attached to the notch portion 68 . The lid member 66 is fixed to the ring member 64 by, for example, screws (not shown). The cover member 66 is connected to the connecting pin 26 with the pupil division filter 24 arranged in the optical path 62 . The rotating member 30 is connected to the pupil division filter 24 via a connecting pin 26 . The rotary member 30 is connected to the pupil division filter 24 via a connection pin 26 to rotate integrally with the pupil division filter 24 . The ring member 64 is an example of the "first member" and the "ring member" according to the technology of the present disclosure, and the lid member 66 is an example of the "second member" according to the technology of the present disclosure.

In the lens device 12, the pupil division filter 24 rotates integrally with the rotating member 30, so that any one of the plurality of polarizers 40 provided in the pupil division filter 24 (hereinafter referred to as a "specific polarizer 40”) can be kept within a predetermined angle range. For the predetermined angle range, for example, an interference removal matrix (hereinafter also referred to as "interference removal parameter") used when obtaining the multispectral image 128 based on the captured image data 96 as described later is commonly used for each imaging device 10. Even when it is used, it is set within a range where the image quality of the multispectral image 128 can be ensured by each imaging device 10 . The predetermined angle range is an example of the "first predetermined angle range" and the "second predetermined angle range" according to the technology of the present disclosure.

A threaded hole 70 is formed in the ring member 64 , and the set screw 34 is screwed into the threaded hole 70 . The set screw 34 is screwed into the screw hole 70 , and when the tip of the set screw 34 hits the outer peripheral surface of the lens barrel 28 , the rotating member 30 is fixed to the lens barrel 28 . The set screw 34 is an example of a "fixing member" according to the technology of the present disclosure.

The set screw 34 is used to fix the rotating member 30 to the lens barrel 28, but the fixing member for fixing the rotating member 30 to the lens barrel 28 is a screw of a type other than the set screw 34. or other types of fasteners than screws. Alternatively, the fixing member may be a fixing material such as solder or adhesive.

The end frame 32 is attached to the lens barrel 28. When the end frame 32 is attached to the lens barrel 28, the end frame 32 restricts the movement of the rotating member 30 along the direction of the optical axis OA.

As shown in FIG. 7 as an example, the imaging device body 14 includes an image sensor 72, a control driver 74, an input/output I/F 76, a computer 78, a display 80, and a communication I/F 82.

When the pupil division filter 24 is arranged in the optical path 62, the first lens 46, the pupil division filter 24, and the second lens 48 extend along the optical axis OA of the lens device 12 from the subject 4 side to the imaging device body 14 side. A first lens 46, a pupil division filter 24, and a second lens 48 are arranged in this order along the line. The first lens 46 transmits light obtained by reflecting the light emitted from the light source 2 by the subject 4 (hereinafter referred to as “subject light”) through the pupil division filter 24 . The second lens 48 forms an image of the subject light that has passed through the pupil division filter 24 on the light receiving surface 84A of the photoelectric conversion element 84 provided in the image sensor 72 .

In FIG. 7, for convenience, the plurality of apertures 44 provided in the pupil division filter 24 are shown arranged linearly along the direction perpendicular to the optical axis OA. , are arranged along the direction around the optical axis OA (see FIG. 4).

The image sensor 72 has a photoelectric conversion element 84 and a signal processing circuit 86 . The image sensor 72 is, for example, a CMOS image sensor. In this embodiment, a CMOS image sensor is exemplified as the image sensor 72, but the technology of the present disclosure is not limited to this. The technology of the present disclosure is established. The image sensor 72 is an example of the "first image sensor" according to the technology of the present disclosure.

As an example, FIG. 7 shows a schematic configuration of the photoelectric conversion element 84 . As an example, FIG. 8 specifically shows the configuration of part of the photoelectric conversion element 84 . The photoelectric conversion element 84 has a pixel layer 88 , a polarizing filter layer 90 and a spectral filter layer 92 .

The pixel layer 88 has a plurality of pixels 94 . A plurality of pixels 94 are arranged in a matrix and form a light receiving surface 84A of the photoelectric conversion element 84 . Each pixel 94 is a physical pixel having a photodiode (not shown), photoelectrically converts received light, and outputs an electrical signal corresponding to the amount of received light.

Hereinafter, the pixels 94 provided in the photoelectric conversion elements 84 are referred to as "physical pixels" in order to distinguish them from the pixels forming the multispectral image 128. The pixels forming the multispectral image 128 are also referred to as "image pixels."

The photoelectric conversion element 84 outputs electrical signals output from the plurality of physical pixels 94 to the signal processing circuit 86 as captured image data 96 . The signal processing circuit 86 digitizes the analog captured image data 96 input from the photoelectric conversion element 84 .

A plurality of physical pixels 94 form a plurality of pixel blocks 98 . Each pixel block 98 is formed by four physical pixels 94 . In FIG. 7, for the sake of convenience, the four physical pixels 94 forming each pixel block 98 are shown arranged linearly along the direction perpendicular to the optical axis OA. Also, the four physical pixels 94 are arranged adjacent to each other in the vertical and horizontal directions of the photoelectric conversion element 84 .

The polarizing filter layer 90 has polarizers 100A to 100D. The polarizer 100A is a polarizer whose transmission axis is set at an azimuth angle of 90°. The polarizer 100B is a polarizer whose transmission axis is set at an azimuth angle of 135°. The polarizer 100C is a polarizer whose transmission axis is set at an azimuth angle of 0°. The polarizer 100D is a polarizer whose transmission axis is set at an azimuth angle of 45°. The polarizers 100A to 100D correspond to the four physical pixels 94, respectively, and are superimposed on the four physical pixels 94, respectively. Hereinafter, the polarizers 100A to 100D will be referred to as "polarizers 100" when there is no need to distinguish between the polarizers 100A to 100D. A polarizer 100 corresponding to each physical pixel 94 is an example of a “second polarizer” according to the technology of the present disclosure.

The spectral filter layer 92 has B filters 102A, G filters 102B, and R filters 102C. The B filter 102A is a blue bandpass filter that transmits most of the light in the blue wavelength band among the light in the plurality of wavelength bands. The G filter 102B is a green bandpass filter that transmits the light in the green wavelength band most among the light in the plurality of wavelength bands. The R filter 102C is a red band filter that transmits most of the light in the red wavelength band among the light in the plurality of wavelength bands. A B filter 102 A, G filter 102 B, and R filter 102 C are assigned to each pixel block 98 .

In FIG. 7, for the sake of convenience, the B filter 102A, G filter 102B, and R filter 102C are shown arranged linearly along the direction perpendicular to the optical axis OA. Furthermore, the B filters 102A, G filters 102B, and R filters 102C are arranged in a matrix in a predetermined pattern arrangement. In the example shown in FIG. 8, the B filters 102A, G filters 102B, and R filters 102C are arranged in a matrix in a Bayer pattern as an example of a predetermined pattern arrangement. Note that the predetermined pattern arrangement may be an RGB stripe arrangement, an R/G checkered arrangement, an X-Trans (registered trademark) arrangement, a honeycomb arrangement, or the like, in addition to the Bayer arrangement.

Hereinafter, the B filter 102A, the G filter 102B, and the R filter 102C are each referred to as the "filter 102" when it is not necessary to distinguish between the B filter 102A, the G filter 102B, and the R filter 102C.

As shown in FIG. 7 as an example, the input/output I/F 76 is connected to a signal processing circuit 86, a control driver 74, a computer 78, a display 80, and a communication I/F 82.

The computer 78 has a processor 110 , NVM 112 and RAM 114 . The processor 110 controls the imaging device 10 as a whole. The processor 110 is, for example, an arithmetic processing device including a CPU and a GPU. The GPU operates under the control of the CPU and is responsible for executing image processing. Although processing units including CPUs and GPUs are mentioned here as an example of processor 110, this is merely an example, and processor 110 may be one or more CPUs that integrate GPU functionality. , may be one or more CPUs that do not integrate GPU functionality. Processor 110 , NVM 112 , and RAM 114 are connected via bus 166 , which is connected to input/output I/F 76 .

The NVM 112 is a non-temporary storage medium and stores various parameters and various programs. For example, NVM 112 is flash memory (eg, EEPROM). However, this is merely an example, and an HDD or the like may be applied as the NVM 112 together with the flash memory. A RAM 114 temporarily stores various information and is used as a work memory.

The processor 110 reads necessary programs from the NVM 112 and executes the read programs on the RAM 114 . Processor 110 controls control driver 74 and signal processing circuit 86 according to a program executed in RAM 114 . Control driver 74 controls photoelectric conversion element 84 under the control of processor 110 . The display 80 is, for example, a liquid crystal display and displays various images including the multispectral image 128 .

The communication I/F 82 is communicably connected to a determination support device 140 (see FIG. 14), which will be described later. The communication I/F 82 may be communicably connected to the determination support device 140 according to a predetermined wireless communication standard, or may be communicably connected to the determination support device 140 according to a predetermined wired communication standard. Examples of the default wireless communication standard include Bluetooth (registered trademark). Note that wireless communication standards other than this (for example, Wi-Fi, 5G, etc.) may be used. The communication I/F 82 manages exchange of information with each of the determination support devices 140 . For example, communication I/F 82 transmits information in response to a request from processor 110 to determination support device 140 . Communication I/F 82 also receives information transmitted from determination support device 140 and outputs the received information to processor 110 via bus 116 .

Next, the operation of the structural configuration of the imaging device 10 according to this embodiment will be described with reference to FIGS. 9 to 16. FIG. 9 to 15 show an example of each process of the method for manufacturing the imaging device 10 according to this embodiment, and FIG. 16 shows an example of the flow of the method for manufacturing the imaging device 10 according to this embodiment. It is shown.

In the manufacturing method of the imaging device 10 shown in FIG. 16, first, in step ST10, an operator (not shown) inserts the pupil division filter 24 to which the connecting pin 26 is attached into the lens barrel 28 through the insertion opening 58. (See FIG. 9). After the process of step ST10 is performed, the manufacturing method of the imaging device 10 proceeds to step ST12.

At step ST12, the operator rotatably attaches the ring member 64 to the outer peripheral surface of the lens barrel 28 (see FIG. 10). Also, the operator attaches the end frame 32 to the outer peripheral surface of the lens barrel 28 (see FIG. 10). After the process of step ST12 is executed, the manufacturing method of the imaging device 10 proceeds to step ST14.

At step ST14, the operator attaches the first mount 16 to the second mount 18 by screwing the screw 20 of the first mount 16 into the screw hole 22 of the second mount 18 (see FIG. 11). Thereby, the lens device 12 is attached to the imaging device body 14 . After the process of step ST14 is performed, the manufacturing method of the imaging device 10 proceeds to step ST16.

At step ST16, the operator focuses the lens device 12 by moving the lever 56 in the direction around the optical axis OA (see FIG. 12). After the process of step ST16 is performed, the manufacturing method of the imaging device 10 proceeds to step ST18.

In step ST18, the operator connects the cover member 66 to the connecting pin 26 with the pupil division filter 24 placed in the optical path 62 (see FIG. 13). Also, the operator assembles the cover member 66 into the notch portion 68 of the ring member 64 (see FIG. 13). As a result, the rotary member 30 is connected to the pupil division filter 24 via the connection pin 26 , and the rotary member 30 rotates integrally with the pupil division filter 24 . After the process of step ST18 is performed, the manufacturing method of the imaging device 10 proceeds to step ST20.

In step ST20, the operator connects the imaging device 10 to the determination support device 140, and determines the captured image data 96 obtained by imaging the subject 4 with the image sensor 72 (see FIG. 7) of the imaging device 10. It is transmitted to the support device 140 (see FIG. 14). The determination support device 140 displays angle information 186 regarding the angle of the transmission axis of the specific polarizer 40 on the display 150 based on the received captured image data 96, as will be described in detail later. Angle information 186 includes, for example, luminance values, degrees of polarization, and polarization angles. The brightness value, the degree of polarization, and the polarization angle are displayed on the display 150 as specific numerical values. Further, the display 150 of the determination support device 140 displays determination result information 188 indicating whether or not each value of the luminance value, the degree of polarization, and the polarization angle falls within a predetermined range (described later). . After the process of step ST20 is performed, the manufacturing method of the imaging device 10 proceeds to step ST22.

In step ST22, the operator looks at the brightness value, the degree of polarization, and the polarization angle displayed on the display 150 and/or the determination result displayed on the display 150, and moves the pupil division filter 24 integrally with the rotary member 30. is rotated (see FIG. 14). After the process of step ST22 is performed, the manufacturing method of the imaging device 10 proceeds to step ST24.

At step ST24, the operator determines whether or not the angle of the transmission axis of the specific polarizer 40 is within the predetermined angle range (see FIG. 14). For example, the operator determines whether each value of the luminance value, the degree of polarization, and the polarization angle is within a predetermined range, and/or whether each value of the luminance value, the degree of polarization, and the polarization angle is within the predetermined range. It is determined whether or not the angle of the transmission axis of the specific polarizer 40 is within the predetermined angle range based on whether or not the result of determination that the angle is within the range is obtained. In step ST24, if the angle of the transmission axis of the specific polarizer 40 does not fall within the predetermined angle range, the determination is negative, and the manufacturing method of the imaging device 10 proceeds to step ST20. In step ST24, when the angle of the transmission axis of the specific polarizer 40 is within the predetermined angle range, the determination is affirmative, and the manufacturing method of the imaging device 10 proceeds to step ST26.

At step ST26, the operator screws the set screw 34 into the screw hole 70 (see FIG. 15). As a result, the rotation member 30 is fixed to the lens barrel 28 , thereby fixing the pupil division filter 24 to the lens barrel 28 . After the process of step ST26 is performed, the manufacturing method of the imaging device 10 ends. The imaging device 10 including the lens device 12 and the imaging device body 14 is assembled by the method of manufacturing the imaging device 10 described above. The manufacturing method of the imaging device 10 is an example of the “imaging device manufacturing method” according to the technology of the present disclosure.

Next, the functional configuration of the imaging device 10 according to this embodiment will be described.

As shown in FIG. 17 as an example, the NVM 112 stores a multispectral image generation program 120 . Processor 110 reads multispectral image generation program 120 from NVM 112 and executes read multispectral image generation program 120 on RAM 114 . Processor 110 performs multispectral image generation processing to generate multispectral image 128 according to multispectral image generation program 120 running on RAM 114 .

The multispectral image generation processing is realized by the processor 110 operating as the output value acquisition unit 122, the interference removal processing unit 124, and the multispectral image acquisition unit 126 according to the multispectral image generation program 120.

As an example, as shown in FIG. 18, when captured image data 96 output from the image sensor 72 is input to the processor 110, the output value acquisition unit 122 outputs each physical pixel 94 based on the captured image data 96. Get the value Y. The output value Y of each physical pixel 94 corresponds to the brightness value of each pixel included in the captured image indicated by the captured image data 96 .

Here, the output value Y of each physical pixel 94 is a value including interference (that is, crosstalk). That is, since light in each transmission wavelength band λ of the first transmission wavelength band λ ₁ , the second transmission wavelength band λ ₂ , and the third transmission wavelength band λ ₃ is incident on each physical pixel 94, the output value Y is , a value corresponding to the amount of light in the first transmission wavelength band _λ1 , a value corresponding to the amount of light in the second transmission wavelength band _λ2 , and a value corresponding to the amount of light in the third transmission wavelength band _λ3 .

In order to acquire the multispectral image 128 (see FIG. 17), the processor 110 separates and extracts the value corresponding to each transmission wavelength band λ from the output value Y for each physical pixel 94, that is, the interference The output value Y needs to be subjected to interference removal processing, which is processing for removing . Therefore, in the present embodiment, the interference removal processing unit 124 performs interference removal processing on the output value Y of each physical pixel 94 acquired by the output value acquisition unit 122 .

Here, the interference removal processing will be described. The output value Y of each physical pixel 94 includes luminance values of red, green, and blue as output value Y components. The output value Y of each physical pixel 94 is represented by Equation (1). _YR is the luminance value of red in the output value Y, _YG is the luminance value of green in the output value Y, and _YB is the luminance value of blue in the output value Y. is.

The pixel value X of each image pixel forming the multispectral image 128 is the luminance value of light in the first transmission wavelength band λ ₁ , the luminance value of light in the second transmission wavelength band λ ₂ , and the third transmission wavelength band λ _{3 .} , as a component of the pixel value X. A pixel value X of each image pixel is represented by Equation (2). However, the luminance value X _λ1 is the luminance value of light in the first transmission wavelength band λ ₁ of the pixel value X, and the luminance value X _λ2 is the light of the second transmission wavelength band λ ₂ in the pixel value X. and the luminance value X _λ3 is the luminance value of the light in the third transmission wavelength band λ ₃ of the pixel value X.

Assuming that the interference matrix is A, the output value Y of each physical pixel 94 is represented by Equation (3).

The interference matrix A (not shown) is based on the spectrum of the subject light, the spectral transmittance of the first lens 46, the spectral transmittance of the second lens 48, the spectral transmittances of the plurality of filters 38, and the spectral sensitivity of the image sensor 72. is a matrix defined by

Assuming that the interference cancellation matrix, which is the general inverse matrix of the interference matrix A, is A ⁺ , the pixel value X of each image pixel is represented by Equation (4).

Like the interference matrix A, the interference cancellation matrix A ⁺ also includes the spectrum of the subject light, the spectral transmittance of the first lens 46, the spectral transmittance of the second lens 48, the spectral transmittances of the plurality of filters 38, and the image sensor 72 is a matrix defined based on the spectral sensitivity of The interference cancellation matrix A ⁺ is pre-stored in NVM 112 .

The interference cancellation processing unit 124 acquires the interference cancellation matrix A ⁺ stored in the NVM 112 and the output value Y of each physical pixel 94 acquired by the output value acquisition unit 122, and combines the acquired interference cancellation matrix A ⁺ Based on the output value Y of each physical pixel 94, the pixel value X of each image pixel is output by Equation (4).

Here, as described above, the pixel value X of each image pixel is the brightness value X λ1 of light in the first transmission wavelength band λ ₁ , the brightness value X _λ2 of light in the second transmission wavelength band λ ₂ , and the brightness value X _λ2 of light in the third transmission wavelength band λ 2 . The luminance value X _λ3 of light in the wavelength band λ ₃ is included as a component of the pixel value X. The brightness value X _λ1 of the light in the first transmission wavelength band λ ₁ is indicated by the first image data of the captured image data 96 . The brightness value X _λ2 of light in the second transmission wavelength band λ ₂ is indicated by the second image data of the captured image data 96 . The brightness value X _λ3 of light in the third transmission wavelength band λ ₃ is indicated by the third image data of the captured image data 96 .

As described above, the interference removal processing is executed by the interference removal processing unit 124, so that the captured image data 96 is the first image data that is image data representing the luminance value _Xλ1 of the light in the first transmission wavelength band _λ1 . , second image data representing the brightness value _Xλ2 of light in the second transmission wavelength band _λ2 , and third image data representing _{the brightness value Xλ3} _of light in the third transmission wavelength band λ3. image data. That is, the captured image data 96 is separated into image data for each transmission wavelength band of the plurality of filters 38 . Interference removal processing is an example of "interference removal processing" according to the technology of the present disclosure.

As an example, as shown in FIG. 19 , the multispectral image acquisition unit 126 generates first image data, second image data, and third image data generated by the interference removal processing performed by the interference removal processing unit 124. to obtain multispectral image data. Multispectral image data is image data representing a multispectral image 128 . The multispectral image data is output to display 80, for example. Display 80 displays multispectral image 128 based on the multispectral image data.

Next, the action of the functional configuration of the imaging device 10 according to this embodiment will be described with reference to FIG. FIG. 20 shows an example of the flow of multispectral image generation processing according to this embodiment.

In the multispectral image generation process shown in FIG. 20, first, in step ST30, the output value acquisition unit 122 acquires the output value Y of each physical pixel 94 based on the captured image data 96 output from the image sensor 72. (See Figure 18). After the process of step ST30 is executed, the multispectral image generation process proceeds to step ST32.

In step ST32, the interference removal processing unit 124 obtains the interference removal matrix A ⁺ stored in the NVM 112 and the output value Y of each physical pixel 94 obtained in step ST30, and calculates the obtained interference removal matrix A ⁺ and the output value Y of each physical pixel 94, the pixel value X of each image pixel is output (see FIG. 18). By executing the interference elimination process in step ST32, the captured image data 96 is changed from the first image data, which is the image data indicating the brightness value _Xλ1 of the light in the first transmission wavelength band _λ1 , and the second transmission wavelength band The _second image data is image data representing the brightness value _Xλ2 of light of λ2, and the third image data is image data representing the brightness value _Xλ3 of light of the third transmission wavelength band λ3 _. . After the process of step ST32 is executed, the multispectral image generation process proceeds to step ST34.

In step ST34, the multispectral image acquisition unit 126 obtains the multispectral image 128 based on the first image data, the second image data, and the third image data generated by executing the interference removal process in step ST32. (see FIG. 19). After the process of step ST34 is executed, the multispectral image generation process proceeds to step ST36.

At step ST36, the processor 110 determines whether or not the condition for terminating the multispectral image generation process (that is, the termination condition) is satisfied. An example of the termination condition is a condition that the user has given an instruction to the imaging device 10 to terminate the multispectral image generation processing. In step ST36, if the termination condition is not met, the determination is negative, and the multispectral image generation process proceeds to step ST30. In step ST36, if the termination condition is met, the determination is affirmative and the multispectral image generation process is terminated.

Next, the determination support device 140 according to this embodiment will be described.

As shown in FIG. 21 as an example, the determination support device 140 is a device applied to the imaging device 10 . Specifically, as described above, the determination support device 140 determines whether the angle information 186 regarding the angle of the transmission axis of the specific polarizer 40 (see FIG. 4) and whether the angle of the transmission axis of the specific polarizer 40 falls within the predetermined angle range. By displaying on the display 150 the determination result information 188 regarding the determination of whether or not the (see FIG. 14))). The determination support device 140 is an example of an “information processing device” according to the technology of the present disclosure.

The determination support device 140 includes a computer 148, a display 150, and a communication I/F 152. The computer 148, the display 150, and the communication I/F 152 are realized by hardware resources similar to the computer 78, the display 80, and the communication I/F 82 provided in the imaging device 10, for example. The communication I/F 152 is communicably connected to the imaging device 10 . Computer 148 has a processor 160 , NVM 162 and RAM 164 . Processor 160 is an example of a "processor" according to the technology of the present disclosure.

As shown in FIG. 22 as an example, the NVM 162 stores a determination support program 170 . The processor 160 reads the determination support program 170 from the NVM 162 and executes the read determination support program 170 on the RAM 164 . The processor 160 executes determination support processing for assisting determination by the operator according to a determination support program 170 executed on the RAM 164 .

The determination support processing is realized by operating the processor 160 as an acquisition unit 172, a derivation unit 174, a storage control unit 176, a display control unit 178, a determination unit 180, and a determination result output unit 182 according to the determination support program 170. . The determination support program 170 is an example of a "program" according to the technology of the present disclosure. Determination support processing is an example of “processing” according to the technology of the present disclosure.

As an example, as shown in FIG. 23, the acquisition unit 172 acquires the image of the subject 4 (see FIG. 7) by the image sensor 72 of the imaging device 10 with the lens device 12 attached to the imaging device body 14 (see FIG. 14). Captured image data 96 obtained by capturing an image is acquired. The captured image data 96 includes output data 132 output from each physical pixel 94 . The obtaining unit 172 selects one physical pixel 94 from among the plurality of physical pixels 94 as the specific physical pixel 94A.

For example, the obtaining unit 172 selects the physical pixel 94 arranged in the center of the light receiving surface 84A of the photoelectric conversion element 84 among the plurality of physical pixels 94 as the specific physical pixel 94A. Then, the acquisition unit 172 acquires the output data 132 output from the specific physical pixel 94A among the plurality of output data 132 included in the captured image data 96 . The output data 132 output from the specific physical pixel 94</b>A is data output based on the spectrum of subject light incident on the image sensor 72 and the optical characteristics of the pupil division filter 24 . The optical properties of the pupil division filter 24 unit include the transmittance of each filter 38, the angle of the transmission axis of each polarizer 40, and the like.

The imaging device body 14 is an example of a "second imaging device body" according to the technology of the present disclosure. The image sensor 72 is an example of a "second image sensor" according to the technology of the present disclosure. The specific physical pixel 94A is an example of a "specific pixel" according to the technology of the present disclosure. The output data 132 output from the specific physical pixel 94A is an example of "first information" and "output data" according to the technology of the present disclosure.

Based on the output data 132 acquired by the acquisition unit 172, the derivation unit 174 acquires the angle information 186 regarding the angle of the transmission axis of the specific polarizer 40 among the plurality of polarizers 40 provided in the pupil division filter 24. do. Angle information 186 includes, for example, luminance values, degrees of polarization, and polarization angles.

The polarization angle refers to the angle of the polarization direction of subject light incident on the light receiving surface 84A of the photoelectric conversion element 84. The degree of polarization refers to a ratio in which the total amount of subject light incident on the light receiving surface 84A of the photoelectric conversion element 84 is the denominator and the amount of the polarized component of the subject light is the numerator. The higher the degree of polarization, the better the separation when the captured image data 96 is separated into the first image data, the second image data, and the third image data (see FIG. 18). The luminance value indicates the amount of subject light received by the light receiving surface 84A of the photoelectric conversion element 84. FIG. The higher the brightness value, the brighter the multispectral image 128 (see FIG. 19). Note that the angle information 186 may include at least one of the brightness value, the degree of polarization, and the angle of polarization. The angle information 186 is an example of "second information" according to the technology of the present disclosure.

The storage control unit 176 causes the NVM 162 to store the angle information 186 . NVM 162 is an example of "memory" according to the technology of the present disclosure. Display control unit 178 outputs angle information 186 to display 150 . Display 150 displays angle information 186 . As a result, the display 150 displays specific numerical values of the luminance value, the degree of polarization, and the angle of polarization included in the angle information 186 . The display 150 is an example of a "display" according to the technology of the present disclosure.

The determination unit 180 determines whether each value of the luminance value, the degree of polarization, and the polarization angle falls within a predetermined range. The above-described predetermined ranges for each value of luminance value, degree of polarization, and polarization angle are set corresponding to predetermined angle ranges (see FIG. 14). The determination result output unit 182 outputs determination result information 188 indicating the determination result by the determination unit 180 to the display 150 .

Specifically, when the determination unit 180 determines that each value of the luminance value, the degree of polarization, and the polarization angle falls within the predetermined range, the determination result output unit 182 outputs the luminance value, the degree of polarization, and the polarization angle are within the predetermined range (that is, a signal indicating affirmative information) is output to the display 150 . On the other hand, when at least one of the brightness value, the degree of polarization, and the polarization angle is out of the predetermined range, the determination result output unit 182 determines that each value of the brightness value, the degree of polarization, and the polarization angle is within the predetermined range. output to display 150 determination result information 188 (that is, a signal indicating negative information) indicating the determination result that the values do not fall within the range.

As a result, the display 150 displays the judgment result as to whether or not each value of the luminance value, the degree of polarization, and the polarization angle is within the predetermined range. For example, if it is determined that the brightness value, the degree of polarization, and the polarization angle are within the predetermined ranges, the display 150 displays the characters "OK" as the determination result. On the other hand, when it is determined that the brightness value, the degree of polarization, and the polarization angle are not within the predetermined ranges, the display 150 displays "NG" as the determination result. The determination result information 188 is an example of "a signal indicating whether or not the angle is within the first predetermined angle range".

Next, the action of the functional configuration of the determination support device 140 according to this embodiment will be described with reference to FIG. FIG. 24 shows an example of the flow of determination support processing according to this embodiment.

In the determination support process shown in FIG. 24 , first, in step ST40, the acquisition unit 172 acquires an image of the subject by the image sensor 72 of the imaging device 10 with the lens device 12 attached to the imaging device body 14. The obtained captured image data 96 is obtained, and the output data 132 output from the specific physical pixel 94A among the plurality of output data 132 included in the obtained captured image data 96 is obtained (see FIG. 23). After the process of step ST40 is executed, the determination support process proceeds to step ST42.

In step ST42, the derivation unit 174 obtains angle information about the angle of the transmission axis of the specific polarizer 40 among the plurality of polarizers 40 provided in the pupil division filter 24, based on the output data 132 acquired in step ST40. 186 (see FIG. 23). After the process of step ST42 is executed, the determination support process proceeds to step ST44.

At step ST44, the storage control unit 176 causes the NVM 162 to store the angle information 186 (see FIG. 23). After the process of step ST44 is executed, the determination support process proceeds to step ST46.

At step ST46, the display control unit 178 outputs the angle information 186 to the display 150 (see FIG. 23). As a result, the display 150 displays specific numerical values of the luminance value, the degree of polarization, and the angle of polarization included in the angle information 186 . After the process of step ST46 is executed, the determination support process proceeds to step ST48.

In step ST48, the determination section 180 determines whether or not each value of the brightness value, degree of polarization, and polarization angle falls within a predetermined range (see FIG. 23). In step ST48, if the brightness value, the degree of polarization, and the polarization angle are all within the predetermined ranges, the determination is affirmative, and the determination support process proceeds to step ST50. In step ST48, if the luminance value, the degree of polarization, and the polarization angle do not fall within the predetermined ranges, the determination is negative, and the determination support processing proceeds to step ST52.

In step ST50, the determination result output unit 182 displays determination result information 188 (that is, affirmative information) indicating that each value of the luminance value, the degree of polarization, and the polarization angle is within the predetermined range on the display 150. (see FIG. 23). As a result, the display 150 displays a determination result indicating that each value of the luminance value, the degree of polarization, and the polarization angle is within the predetermined range. After the process of step ST50 is executed, the determination support process proceeds to step ST54.

In step ST52, the determination result output unit 182 displays the determination result information 188 (that is, negative information) indicating that each value of the luminance value, the degree of polarization, and the polarization angle does not fall within the predetermined ranges on the display 150. (see FIG. 23). As a result, the display 150 displays a judgment result indicating that each value of the luminance value, the degree of polarization, and the polarization angle is not within the predetermined range. After the process of step ST52 is executed, the determination support process proceeds to step ST54.

At step ST54, the processor 160 determines whether or not the condition for terminating the determination support process (that is, the termination condition) is satisfied. An example of the termination condition is a condition that the user gives an instruction to the determination support device 140 to terminate the determination support process. In step ST54, if the termination condition is not met, the determination is negative, and the determination support process proceeds to step ST40. In step ST54, if the termination condition is met, the determination is affirmative and the determination support process is terminated. The information processing method described as the action of the functional configuration of the determination support device 140 described above is an example of the "information processing method" according to the technology of the present disclosure.

As described above, the imaging device 10 (see FIG. 11) according to this embodiment includes the lens device 12 and the imaging device body 14 . The lens device 12 has a first mount 16 and the imaging device body 14 has a second mount 18 . The first mount 16 and the second mount 18 are each C mounts. A screw 20 is formed on the outer peripheral surface of the first mount 16 , and a screw hole 22 is formed inside the second mount 18 . Therefore, by screwing the screw 20 into the screw hole 22 , the first mount 16 can be attached to the second mount 18 , and the lens device 12 can be attached to the imaging device body 14 .

Further, in the imaging apparatus 10 (see FIG. 7) according to the present embodiment, the lens device 12 has the pupil division filter 24 arranged in the optical path 62, and the pupil division filter 24 is positioned in the optical path 62. It has a plurality of openings 44 through which it extends. A polarizer 40 is provided in each of the plurality of openings 44 . The lens device 12 also includes a rotating member 30 for changing the angle of the transmission axis of the polarizer 40 . Therefore, for example, due to manufacturing errors in the screw 20 and/or the screw hole 22, the mounting angle of the lens device 12 with respect to the imaging device body 14 varies, resulting in an error in the angle of the transmission axis of the polarizer 40. Even in this case, by changing the angle of the transmission axis of the polarizer 40 with the rotating member 30, the error in the angle of the transmission axis can be reduced.

Also, the rotating member 30 is connected to the pupil division filter 24 and rotates around the optical axis OA of the lens device 12 . Therefore, the angle of the transmission axis of the polarizer 40 can be changed more easily than, for example, when the pupil division filter 24 needs to be replaced in order to change the angle of the transmission axis of the polarizer 40 .

Also, the rotating member 30 has a ring member 64 and a lid member 66 . The lid member 66 connects the pupil division filter 24 to the ring member 64 while the pupil division filter 24 is arranged in the optical path 62 . Therefore, for example, even when the ring member 64 is rotatably supported by the lens barrel 28 , the pupil division filter 24 arranged in the optical path 62 can be connected to the ring member 64 by the lid member 66 .

Also, the ring member 64 is a ring-shaped member having a cutout portion 68 in a partial region, and the lid member 66 is assembled to the cutout portion 68 . Therefore, for example, the circularity of the rotating member 30 can be ensured as compared with the case where the rotating member 30 is not provided with the ring member 64 and is divided into a plurality of members in the circumferential direction.

Also, the rotating member 30 is rotatably supported with respect to the lens barrel 28 , and the rotating member 30 is fixed to the lens barrel 28 by a set screw 34 . Therefore, by fixing the rotation member 30 to the lens barrel 28 by the set screw 34, it is possible to maintain the state in which the angle of the transmission axis of the specific polarizer 40 among the plurality of polarizers 40 falls within the predetermined angle range. can.

Further, the processor 160 (see FIG. 23) of the determination support device 140, when the lens device 12 is attached to the imaging device body 14, based on the output data 132 from the image sensor 72 provided in the imaging device body 14 to output angle information 186 regarding the angle of the transmission axis of the specific polarizer 40 . Therefore, based on the angle information 186 (see also FIG. 14), the operator can determine whether or not the angle of the transmission axis of the specific polarizer 40 is within the predetermined angle range.

Also, the processor 160 outputs determination result information 188 indicating whether or not the angle of the transmission axis of the specific polarizer 40 is within the predetermined angle range based on the angle information 186 . Therefore, based on the determination result information 188 (see also FIG. 14), the operator can determine whether or not the angle of the transmission axis of the specific polarizer 40 is within the predetermined angle range.

Also, the angle information 186 is information based on the spectrum of light incident on the image sensor 72 and the optical characteristics of the pupil division filter 24 . Therefore, based on the angle information 186, which is information based on the spectrum of light incident on the image sensor 72 and the optical characteristics of the pupil division filter 24, it is determined whether the angle of the transmission axis of the specific polarizer 40 is within the predetermined angle range. can be determined by the operator.

Also, the angle information 186 is information based on the output data 132 output from at least one specific physical pixel 94A among the plurality of physical pixels 94 included in the image sensor 72. Therefore, for example, based on the output data 132 output from the specific physical pixel 94A, the operator can determine whether or not the angle of the transmission axis of the specific polarizer 40 is within the predetermined angle range.

Also, the angle information 186 includes at least one of a luminance value, a degree of polarization, and a polarization angle. Therefore, based on at least one of the brightness value, the degree of polarization, and the polarization angle, the operator can determine whether or not the angle of the transmission axis of the specific polarizer 40 falls within the predetermined angle range.

Processor 160 also outputs angle information 186 to display 150 . Therefore, based on the angle information 186 (see also FIG. 14) displayed on the display 150, the operator can determine whether the angle of the transmission axis of the specific polarizer 40 is within the predetermined angle range.

Also, the angle information 186 is stored in the NVM 162 . Therefore, even if the angle of the transmission axis of the polarizer 40 is changed by the rotating member 30, it is possible to grasp the error that occurred in the angle of the transmission axis before the change based on the angle information 186 stored in the NVM 162. .

In addition, the manufacturing method of the imaging device 10 according to the present embodiment (see FIGS. 9 to 16) includes attaching the lens device 12 to the imaging device body 14 and the state in which the lens device 12 is attached to the imaging device body 14 and changing the angle of the transmission axis of the polarizer 40 by the rotating member 30 . Therefore, for example, due to manufacturing errors in the screw 20 and/or the screw hole 22, the mounting angle of the lens device 12 with respect to the imaging device body 14 varies, resulting in an error in the angle of the transmission axis of the polarizer 40. Even in this case, by changing the angle of the transmission axis of the polarizer 40 with the rotating member 30, the error in the angle of the transmission axis can be reduced.

Further, the method for manufacturing the imaging device 10 according to the present embodiment changes the angle of the transmission axis of the polarizer 40 when focusing using the lens device 12 is performed. Therefore, for example, compared to changing the angle of the transmission axis of the polarizer 40 before focusing using the lens device 12, errors in the angle of the transmission axis can be reduced.

Further, in the method for manufacturing the imaging device 10 according to the present embodiment, the transmission axis of the specific polarizer 40 among the plurality of polarizers 40 is determined based on the output data 132 from the image sensor 72 provided in the imaging device body 14. obtaining angle information 186 about the angle of , and changing the angle of the transmission axis of the polarizer 40 is performed based on the angle information 186 . Therefore, based on the output data 132 from the image sensor 72, the angle of the transmission axis of the polarizer 40 can be changed.

Further, the method for manufacturing the imaging device 10 according to the present embodiment outputs determination result information 188 indicating whether or not the angle of the transmission axis of the specific polarizer 40 is within the predetermined angle range based on the angle information 186. further provide. Therefore, for example, the operator can easily determine whether or not the angle of the transmission axis of the specific polarizer 40 is within the predetermined angle range, compared to the case where the determination result information 188 is not output.

Further, in the manufacturing method of the imaging device 10 according to the present embodiment, when the angle of the transmission axis of the specific polarizer 40 is within the predetermined angle range, the pupil division filter 24 is attached to the lens barrel 28 of the lens device 12. Further comprising fixing. Therefore, by fixing the rotary member 30 to the lens barrel 28, it is possible to maintain a state in which the angle of the transmission axis of the specific polarizer 40 falls within the predetermined angle range.

It should be noted that, in the above-described embodiment, the method for manufacturing the imaging device 10 is performed by an operator. However, as shown in FIG. 25 as an example, the manufacturing method of the imaging device 10 may be executed by a manufacturing system 200. FIG. Manufacturing system 200 includes controller 202 , first device 204 , second device 206 , third device 208 , fourth device 210 , fifth device 212 , sixth device 214 , and seventh device 216 .

The controller 202 is, for example, a computer including a processor, NVM, and RAM (all of which are not shown). A sixth device 214 and a seventh device 216 are controlled. The first device 204, the second device 206, the third device 208, the fourth device 210, the fifth device 212, and the seventh device 216 are, for example, assembly devices including robot hands and/or actuators.

The first device 204 inserts the pupil division filter 24 to which the connecting pin 26 is attached inside the lens barrel 28 through the insertion port 58 (see FIG. 9). The second device 206 rotatably attaches the ring member 64 to the outer peripheral surface of the lens barrel 28 (see FIG. 10). The second device 206 attaches the end frame 32 to the outer peripheral surface of the lens barrel 28 (see FIG. 10). The third device 208 attaches the first mount 16 to the second mount 18 by screwing the screws 20 of the first mount 16 into the screw holes 22 of the second mount 18 (see FIG. 11).

The fourth device 210 focuses the lens device 12 by moving the lever 56 (see FIG. 12). The fifth device 212 connects the cover member 66 to the connecting pin 26 with the pupil division filter 24 arranged in the optical path 62 (see FIG. 13). Also, the fifth device 212 assembles the cover member 66 into the notch portion 68 of the ring member 64 (see FIG. 13).

The controller 202 outputs the angle information 186 regarding each value of the luminance value, the degree of polarization, and the polarization angle, and/or the determination result of whether or not each value of the luminance value, the degree of polarization, and the polarization angle falls within a predetermined range. is acquired from the determination support device 140 (see FIG. 25).

The sixth device 214 rotates the pupil division filter 24 integrally with the rotating member 30 based on the angle information 186 and/or the determination result information 188 (see FIG. 14). The controller 202 acquires the angle information 186 and/or the determination result information 188 from the determination support device 140, and determines the angle of the transmission axis of the specific polarizer 40 based on the acquired angle information 186 and/or the determination result information 188. It is determined whether or not the angle is within the range (see FIG. 14).

The seventh device 216 fixes the rotation member 30 to the lens barrel 28 by screwing the set screw 34 into the screw hole 22 when the angle of the transmission axis of the specific polarizer 40 is within the predetermined angle range. (See FIG. 15). According to the manufacturing system 200 described above, it is possible to save work by the operator.

In the above embodiment, the determination support processing is executed by the processor 160 included in the determination support device 140, which is a device different from the imaging device 10 (see FIG. 23). However, as shown in FIG. 26 as an example, the determination support processing may be executed by the processor 110 included in the imaging device 10 . Then, the angle information 186 regarding each value of the luminance value, the degree of polarization, and the polarization angle, and/or the determination result of whether or not each value of the luminance value, the degree of polarization, and the polarization angle is within a predetermined range. Determination result information 188 may be displayed on display 80 .

In the example shown in FIG. 26, the imaging device 10 is an example of the "information processing device" according to the technology of the present disclosure, and the processor 110 is an example of the "processor" according to the technology of the present disclosure. Also, in the example illustrated in FIG. 26 , the display 80 is an example of the “display” according to the technology of the present disclosure, and the NVM 112 is an example of the memory according to the technology of the present disclosure.

In the above embodiment, a pupil division filter 24 having a plurality of filters 38 and a plurality of polarizers 40 is used. may be used.

In the above embodiment, the rotating member 30 is used as a changing mechanism for changing the angle of the transmission axis of the polarizer 40 . That is, the imaging device 10 has a rotary changing mechanism including the rotating member 30 . However, in order to change the angle of the transmission axis of the polarizer 40, the imaging device 10 may be provided with, for example, a slide-type change mechanism including a slide member, or a replacement-type change mechanism including a replacement member. may be provided.

In the above embodiment, as an example of the multispectral image 128, a multispectral image generated based on light split into three transmission wavelength bands λ was described as an example, but the three transmission wavelength bands λ are , is merely an example, and may be four or more transmission wavelength bands λ. That is, the imaging device 10 may be a multispectral camera capable of imaging a subject with a higher wavelength resolution than a multispectral camera capable of imaging light split into three transmission wavelength bands λ.

In the above embodiment, the processor 110 was illustrated for the imaging device 10, but instead of the processor 110 or together with the processor 110, at least one other CPU, at least one GPU, and/or at least one TPU may be used. may be used.

In the above embodiment, the NVM 112 stores the multispectral image generation program 120 as an example, but the technology of the present disclosure is not limited to this. For example, the multispectral image generation program 120 may be stored in a portable non-transitory computer-readable storage medium such as an SSD or USB memory (hereinafter simply referred to as "non-temporary storage medium"). A multispectral image generation program 120 stored in a non-transitory storage medium is installed in the computer 78 of the imaging device 10, and the processor 110 performs multispectral image generation processing according to the multispectral image generation program 120. FIG.

In addition, the multispectral image generation program 120 is stored in a storage device such as another computer or server device connected to the imaging device 10 via a network, and the multispectral image generation program 120 is stored in response to a request from the imaging device 10. may be downloaded and installed on computer 78 .

In addition, it is not necessary to store all of the multispectral image generation program 120 in a storage device such as another computer or server device connected to the imaging device 10, or in the NVM 112, and part of the multispectral image generation program 120 It may be stored.

In addition, although the computer 78 is built into the imaging device 10 , the technology of the present disclosure is not limited to this, and the computer 78 may be provided outside the imaging device 10 , for example.

Although the above embodiment illustrates computer 78 including processor 110, NVM 112, and RAM 114, the technology of the present disclosure is not limited to this, and computer 78 may include an ASIC, FPGA, and/or PLD. device may be applied. Also, instead of the computer 78, a combination of hardware and software configurations may be used.

In the above embodiment, the processor 160 was illustrated for the determination support device 140, but instead of the processor 160 or together with the processor 160, at least one other CPU, at least one GPU, and/or at least one TPU may be used.

In the above embodiment, an example of the form in which the determination support program 170 is stored in the NVM 162 has been described, but the technology of the present disclosure is not limited to this. For example, the determination support program 170 may be stored in a non-temporary storage medium such as SSD or USB memory. A determination support program 170 stored in a non-temporary storage medium is installed in the computer 148 of the determination support device 140 , and the processor 160 executes determination support procedures according to the determination support program 170 .

Further, the determination support program 170 is stored in a storage device such as another computer or server device connected to the determination support device 140 via a network, and the determination support program 170 is downloaded in response to a request from the determination support device 140. and may be installed on computer 148 .

In addition, it is not necessary to store all of the determination support program 170 in another computer or a storage device such as a server device connected to the determination support device 140, or in the NVM 162, and a part of the determination support program 170 may be stored. You can leave it.

In addition, although the computer 148 is built in the determination support device 140, the technology of the present disclosure is not limited to this, and the computer 148 may be provided outside the determination support device 140, for example.

Although the above embodiment illustrates computer 148 including processor 160, NVM 162, and RAM 164, the technology of the present disclosure is not limited thereto, and computer 148 may include an ASIC, FPGA, and/or PLD. device may be applied. Also, instead of the computer 148, a combination of hardware and software configurations may be used.

Also, the following various processors can be used as hardware resources for executing the various processes described in the above embodiments. Examples of processors include CPUs, which are general-purpose processors that function as hardware resources that execute various processes by executing software, that is, programs. Also, processors include, for example, FPGAs, PLDs, ASICs, and other dedicated electric circuits that are processors having circuit configurations specially designed to execute specific processing. A memory is built in or connected to each processor, and each processor uses the memory to perform various processes.

Hardware resources that perform various processes may be configured with one of these various processors, or a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or CPUs). and FPGA). Also, the hardware resource for executing various processes may be one processor.

As an example of configuration with one processor, first, there is a form in which one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes various processes. Secondly, as typified by SoC, etc., there is a mode of using a processor that implements the function of the entire system including a plurality of hardware resources for executing various processes with a single IC chip. In this way, various processes are realized using one or more of the above various processors as hardware resources.

Furthermore, as the hardware structure of these various processors, more specifically, an electric circuit in which circuit elements such as semiconductor elements are combined can be used. Also, the line-of-sight detection process described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps added, and the order of processing may be changed without departing from the scope of the invention.

The descriptions and illustrations shown above are detailed descriptions of the parts related to the technology of the present disclosure, and are merely examples of the technology of the present disclosure. For example, the above descriptions of configurations, functions, actions, and effects are descriptions of examples of configurations, functions, actions, and effects of portions related to the technology of the present disclosure. Therefore, unnecessary parts may be deleted, new elements added, or replaced with respect to the above-described description and illustration without departing from the gist of the technology of the present disclosure. Needless to say. In addition, in order to avoid complication and facilitate understanding of the portion related to the technology of the present disclosure, the descriptions and illustrations shown above require particular explanation in order to enable implementation of the technology of the present disclosure. Descriptions of common technical knowledge, etc., that are not used are omitted.

In this specification, "A and/or B" is synonymous with "at least one of A and B." That is, "A and/or B" means that only A, only B, or a combination of A and B may be used. Also, in this specification, when three or more matters are expressed by connecting with "and/or", the same idea as "A and/or B" is applied.

All publications, patent applications and technical standards mentioned herein are expressly incorporated herein by reference to the same extent as if each individual publication, patent application and technical standard were specifically and individually noted to be incorporated by reference. incorporated by reference into the book.

Claims

A lens device in which a polarization unit is arranged in an optical path,
the polarization unit has a plurality of apertures positioned in the optical path;
At least one of the plurality of openings is provided with a first polarizer,
A lens device, wherein the lens device includes a change mechanism that changes an angle of the transmission axis of the first polarizer.
The changing mechanism has a rotating member that rotates around the optical axis of the lens device,
The lens device according to claim 1, wherein the rotating member is connected with the polarization unit.
The rotating member is
a first member;
a second member that connects the polarization unit to the first member with the polarization unit arranged in the optical path;
The lens device according to claim 2, comprising:
The first member is a ring member having a slot in a partial area,
The lens device according to claim 3, wherein the second member is attached to the slot portion.
a lens barrel that rotatably supports the rotating member;
a fixing member that fixes the rotating member to the lens barrel;
The lens device according to any one of claims 2 to 4, further comprising: a.
6. The lens arrangement according to any one of claims 1 to 5, wherein the lens arrangement comprises a first mount attached to a second mount provided on a first imaging device body comprising a first image sensor with a second polarizer. The lens device as described.
7. The lens device according to claim 6, wherein the first mount has a first threaded portion that screws together with a second threaded portion formed on the second mount.
8. The lens device according to claim 6, wherein the first mount and the second mount are C mounts, respectively.
An information processing device that is applied to an imaging device that includes the lens device according to any one of claims 1 to 8 and a second imaging device body and that includes a processor,
The processor, when the lens device is attached to the second imaging device body, based on first information that is output data from a second image sensor provided in the second imaging device body, An information processing device that outputs second information about an angle.
The information processing apparatus according to claim 9, wherein the processor outputs a signal indicating whether or not the angle is within a first predetermined angle range based on the second information.
The information processing device according to claim 9 or 10, wherein the information processing device is the imaging device.
The information processing apparatus according to any one of claims 9 to 11, wherein the first information is information based on a spectrum of light incident on the second image sensor and optical characteristics of the polarizing unit.
The second information according to any one of claims 9 to 12, wherein the second information is information based on output data output from at least one specific pixel among a plurality of pixels included in the second image sensor. information processing equipment.
The information processing device according to any one of claims 9 to 13, wherein the second information includes at least one of a luminance value, a degree of polarization, and a polarization angle.
The information processing apparatus according to any one of claims 9 to 14, wherein the processor outputs the second information to a display.
The information processing apparatus according to any one of claims 9 to 15, further comprising a memory that stores the second information.
The information processing apparatus according to any one of claims 9 to 16, wherein the processor acquires a multispectral image by performing interference cancellation processing on the output data.
A program for causing a computer to execute processing applied to an imaging device comprising the lens device according to any one of claims 1 to 8 and a second imaging device body,
The processing is based on first information, which is output data from a second image sensor provided in the second imaging device body, when the lens device is attached to the second imaging device body. A program comprising outputting second information about an angle.
A manufacturing method of an imaging device for assembling an imaging device comprising the lens device according to any one of claims 1 to 8 and a second imaging device body, comprising:
attaching the lens device to the second imaging device body; and
A method of manufacturing an imaging device, comprising: changing the angle by the changing mechanism while the lens device is attached to the second imaging device body.
20. The method of manufacturing an imaging device according to claim 19, wherein changing the angle is performed when focusing is performed using the lens device.
Further comprising acquiring second information about the angle based on first information that is output data from a second image sensor provided in the second imaging device body,
21. The method of manufacturing an imaging device according to claim 19, wherein changing the angle is performed based on the second information.
22. The method of manufacturing an imaging device according to claim 21, further comprising outputting a signal indicating whether the angle is within a first predetermined angle range based on the second information.
23. The imaging according to any one of claims 19 to 22, further comprising fixing the polarization unit with respect to the lens barrel of the lens device when the angle is within a second predetermined angle range. Method of manufacturing the device.