WO2023181558A1 - 撮像装置 - Google Patents

撮像装置 Download PDF

Info

Publication number
WO2023181558A1
WO2023181558A1 PCT/JP2022/047594 JP2022047594W WO2023181558A1 WO 2023181558 A1 WO2023181558 A1 WO 2023181558A1 JP 2022047594 W JP2022047594 W JP 2022047594W WO 2023181558 A1 WO2023181558 A1 WO 2023181558A1
Authority
WO
WIPO (PCT)
Prior art keywords
light
image
image sensor
infrared light
visible light
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/047594
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
拓洋 澁谷
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kokusai Denki Electric Inc
Original Assignee
Hitachi Kokusai Electric Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hitachi Kokusai Electric Inc filed Critical Hitachi Kokusai Electric Inc
Priority to JP2024509768A priority Critical patent/JPWO2023181558A1/ja
Publication of WO2023181558A1 publication Critical patent/WO2023181558A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B11/00—Filters or other obturators specially adapted for photographic purposes
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B15/00—Special procedures for taking photographs; Apparatus therefor
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B7/00—Control of exposure by setting shutters, diaphragms or filters, separately or conjointly
    • G03B7/08—Control effected solely on the basis of the response, to the intensity of the light received by the camera, of a built-in light-sensitive device
    • G03B7/091—Digital circuits
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/45—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/75—Circuitry for compensating brightness variation in the scene by influencing optical camera components

Definitions

  • the present invention relates to an imaging device, and particularly to an imaging device that expands the apparent depth of field.
  • Means to increase the brightness level are: [1] Open the aperture to increase the amount of light received, [2] Increase the exposure time by lengthening the time for one frame, [3] Irradiate light onto the subject. [4] There is a method of applying gain correction to the video signal.
  • Patent Document 1 discloses that it is determined whether it is daytime, nighttime, early morning, or evening based on the brightness of the photographed image acquired by the camera and the brightness of the reference image of the road surface, and the aperture of the camera is adjusted according to the determination result. A technique for controlling this is disclosed.
  • an object of the present invention to provide an imaging device that can obtain an image with a deep apparent depth of field even when the aperture is opened.
  • one of the typical imaging devices of the present invention is capable of separately receiving visible light and near-infrared light with respect to incident light, and is capable of receiving a visible light signal based on the visible light and a near-infrared light signal based on the visible light.
  • an image sensor unit that outputs a near-infrared light signal based on infrared light, and a process for extracting a contour component from an image generated by the near-infrared light signal and superimposing the extracted contour component on the image generated by the visible light signal.
  • a video signal processing section is capable of separately receiving visible light and near-infrared light with respect to incident light, and is capable of receiving a visible light signal based on the visible light and a near-infrared light signal based on the visible light.
  • FIG. 1 is a block diagram of a computer system for implementing aspects according to embodiments of the present disclosure.
  • FIG. 2 is a block diagram showing an example of the configuration of an imaging device according to the present invention.
  • FIG. 3 is a diagram showing a first example of the image sensor section in the image pickup apparatus of the present invention.
  • FIG. 4 is a diagram showing a second example of the image sensor section in the image pickup apparatus of the present invention.
  • FIG. 5 is a diagram showing a third example of the image sensor section in the image pickup apparatus of the present invention.
  • FIG. 6 is a diagram showing a fourth example of the image sensor section in the image pickup apparatus of the present invention.
  • FIG. 7 is a diagram showing a fifth example of the image sensor section in the image pickup apparatus of the present invention.
  • FIG. 1 is a block diagram of a computer system for implementing aspects according to embodiments of the present disclosure.
  • FIG. 2 is a block diagram showing an example of the configuration of an imaging device according to the present invention.
  • FIG. 8 is a diagram showing an example of a flowchart of the imaging apparatus of the present invention.
  • FIG. 9 is a diagram illustrating a specific example of superimposing contour components from a visible light image and a near-infrared light image in the imaging apparatus of the present invention.
  • FIG. 1 is a block diagram of a computer system 300 for implementing aspects according to embodiments of the present disclosure.
  • the mechanisms and apparatus of the various embodiments disclosed herein may be applied to any suitable computing system.
  • the main components of computer system 300 include one or more processors 302 , memory 304 , terminal interface 312 , storage interface 314 , I/O (input/output) device interface 316 , and network interface 318 . These components may be interconnected via memory bus 306, I/O bus 308, bus interface unit 309, and I/O bus interface unit 310.
  • Computer system 300 may include one or more processing devices 302A and 302B, collectively referred to as processor 302. Each processor 302 executes instructions stored in memory 304 and may include onboard cache. In some embodiments, computer system 300 may include multiple processors, and in other embodiments, computer system 300 may be a single processing unit system. Processing devices include CPU (Central Processing Unit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), and DSP (Digital Processing Unit). al Signal Processor) etc. can be applied.
  • CPU Central Processing Unit
  • FPGA Field-Programmable Gate Array
  • GPU Graphics Processing Unit
  • DSP Digital Processing Unit
  • memory 304 may include random access semiconductor memory, storage devices, or storage media (either volatile or nonvolatile) for storing data and programs.
  • memory 304 represents the entire virtual memory of computer system 300 and may include the virtual memory of other computer systems connected to computer system 300 via a network.
  • memory 304 may be conceptually considered a single entity, in other embodiments this memory 304 may be a more complex arrangement, such as a hierarchy of caches and other memory devices. .
  • memory may exist as multiple levels of caches, and these caches may be divided by function. As a result, one cache may hold instructions while the other cache holds non-instruction data used by the processor.
  • Memory may be distributed and associated with various different processing units, such as in a so-called NUMA (Non-Uniform Memory Access) computer architecture.
  • NUMA Non-Uniform Memory Access
  • Memory 304 may store all or a portion of the programs, modules, and data structures that perform the functions described herein.
  • memory 304 may store latent factor identification application 350.
  • latent factor identification application 350 may include instructions or writings that perform functions described below on processor 302, or may include instructions or writings that are interpreted by other instructions or writings.
  • latent factor identification application 350 may be applied to semiconductor devices, chips, logic gates, circuits, circuit cards, and/or other physical hardware instead of or in addition to processor-based systems. It may also be implemented in hardware via a device.
  • latent factor identification application 350 may include data other than instructions or descriptions.
  • cameras, sensors, or other data input devices may be provided to communicate directly with bus interface unit 309, processor 302, or other hardware of computer system 300. . Such a configuration may reduce the need for processor 302 to access memory 304 and latent factor identification applications.
  • Computer system 300 may include a bus interface unit 309 that provides communication between processor 302 , memory 304 , display system 324 , and I/O bus interface unit 310 .
  • I/O bus interface unit 310 may be coupled to I/O bus 308 for transferring data to and from various I/O units.
  • I/O bus interface unit 310 connects, via I/O bus 308, a plurality of I/O interface units 312, 314, 316, also known as I/O processors (IOPs) or I/O adapters (IOAs). and 318.
  • Display system 324 may include a display controller, display memory, or both. A display controller may provide video, audio, or both data to display device 326.
  • Computer system 300 may also include devices, such as one or more sensors, configured to collect data and provide the data to processor 302.
  • computer system 300 may include environmental sensors that collect humidity data, temperature data, pressure data, etc., motion sensors that collect acceleration data, motion data, etc., and the like. Other types of sensors can also be used.
  • the display memory may be a dedicated memory for buffering video data.
  • Display system 324 may be connected to a display device 326, such as a standalone display screen, a television, a tablet, or a handheld device.
  • display device 326 may include speakers to render audio.
  • a speaker for rendering audio may be connected to the I/O interface unit.
  • the functionality provided by display system 324 may be implemented by an integrated circuit that includes processor 302.
  • bus interface unit 309 may be implemented by an integrated circuit that includes processor 302.
  • the I/O interface unit has the ability to communicate with various storage or I/O devices.
  • the terminal interface unit 312 may include a user output device such as a video display device, a speaker television, or a user input device such as a keyboard, mouse, keypad, touch pad, trackball, button, light pen, or other pointing device.
  • user I/O devices 320 such as: Using the user interface, a user operates a user input device to input input data and instructions to user I/O device 320 and computer system 300, and to receive output data from computer system 300. Good too.
  • the user interface may be displayed on a display device, played through a speaker, or printed through a printer, for example, via the user I/O device 320.
  • Storage interface 314 may include one or more disk drives or direct access storage devices 322 (typically magnetic disk drive storage devices, but an array of disk drives or other storage devices configured to appear as a single disk drive). ) can be installed. In some embodiments, storage device 322 may be implemented as any secondary storage device. The contents of memory 304 may be stored in storage device 322 and read from storage device 322 as needed.
  • Network interface 318 may provide a communication pathway so that computer system 300 and other devices can communicate with each other. This communication path may be, for example, network 330.
  • the computer system 300 shown in FIG. 1 includes a bus structure that provides a direct communication path between a processor 302, a memory 304, a bus interface 309, a display system 324, and an I/O bus interface unit 310;
  • computer system 300 may include point-to-point links in a hierarchical, star, or web configuration, multiple hierarchical buses, and parallel or redundant communication paths.
  • I/O bus interface unit 310 and I/O bus 308 are shown as a single unit, computer system 300 may actually include multiple I/O bus interface units 310 or multiple I/O bus A bus 308 may also be provided.
  • multiple I/O interface units are shown to separate I/O bus 308 from various communication paths leading to various I/O devices, in other embodiments, one of the I/O devices may Some or all may be directly connected to one system I/O bus.
  • computer system 300 is a device that receives requests from other computer systems (clients) that do not have a direct user interface, such as a multi-user mainframe computer system, a single-user system, or a server computer. There may be. In other embodiments, computer system 300 may be a desktop computer, portable computer, laptop, tablet computer, pocket computer, telephone, smart phone, or any other suitable electronic device.
  • FIG. 2 is a block diagram showing an example of the configuration of an imaging device according to the present invention.
  • the imaging device 1 includes a lens section 2, an image sensor section 3, a video signal processing section 4, and a control section 5.
  • the imaging device 1 captures video at, for example, 3 frames per second (3 fps) or more, and performs processing to be described later.
  • Incident light from a subject is imaged by the lens section 2, and photoelectrically converted into an electrical signal by the image sensor section 3.
  • the lens section 2 includes concave and convex lenses 6 that adjust zoom and focus, an aperture 7 that adjusts the amount of light, and a near-infrared light cut filter 8. These are provided in this order from the incident light side: each concave-convex lens 6, an aperture 7, and a near-infrared light cut filter 8. Note that various configurations can be applied to the configuration of each concave-convex lens 6 depending on the purpose.
  • the aperture 7 adjusts the amount of light entering from the concave-convex lens 6 by changing the size of the central hole using a structure such as a plurality of blades. Opening the aperture increases the amount of light, allowing you to capture good images even in dark surroundings. However, in this case, the depth of field becomes shallow, and objects at a certain distance are in focus, while objects at other distances tend to be blurred. This makes it difficult to identify objects at a distance other than a specific distance. On the other hand, if the surroundings are bright, there will be enough light even when the aperture is closed, and the depth of field will be deep. In this case, it is easy to focus over a wide range of different distances, and even at distances that are out of focus, the degree of blur is small.
  • the near-infrared light cut filter 8 is a filter that cuts only near-infrared light and allows visible light to pass through.
  • near-infrared light is light with a wavelength of approximately 800 to 2,500 nanometers.
  • visible light is light with a wavelength of about 380 to 770 nanometers.
  • the near-infrared light cut filter 8 is located adjacent to the diaphragm 7, and is open only in the center, so that it is transparent without a filter.
  • the size of the aperture in the central part where there is no filter can be set to be smaller. For example, the aperture may be made smaller than the aperture when the diaphragm 7 is at its most closed position.
  • the aperture may be smaller than the aperture of the diaphragm 7 at the time of the brightness level threshold value in step S101 in FIG. 8, which will be described later. As a result, it is possible to set a deep depth of field for near-infrared light.
  • the image sensor section 3 converts the incident light that has passed through the lens section 2, the aperture 7, and the near-infrared cut filter 8 into an electrical signal, and transmits the signal to the video signal processing section 4.
  • Examples of the image sensor here include a CCD (Charge-Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and the like.
  • the image sensor section 3 is configured to be able to separately receive visible light and near-infrared light, and the detailed configuration will be described later with reference to FIGS. 3 to 7.
  • the video signal processing unit 4 performs various video signal processing such as gain correction, gamma correction, knee correction, contour correction, and color correction on the signal input from the image sensor unit 3.
  • the video signal processing section 4 includes a contour extraction section 9.
  • the contour extraction section 9 performs contour correction processing on the near-infrared light signal input from the image sensor section 3 and outputs it as a contour signal. Then, the above contour signal is added to the visible light signal inputted from the image sensor section 3 to create a video signal.
  • An HD-SDI (High Definition Serial Digital Interface) signal is generated from this video signal and output to the outside. Note that the video signal output from the video signal processing unit 4 is not limited to the HD-SDI described above, and other types of video signals such as compressed and encrypted video signals may be applied.
  • the control unit 5 controls the lens unit 2, the image sensor unit 3, and the video signal processing unit 4 of the imaging device 1.
  • the control unit 5 can be configured by, for example, a CPU.
  • the computer system 300 shown in FIG. 1 can be applied to the video signal processing section 4 and the control section 5.
  • FIG. 3 is a diagram showing a first example of the image sensor section in the image pickup device of the present invention.
  • the image sensor section 3 shown in FIG. 3 includes an image sensor 31 having a filter 31a.
  • the filter 31a is a filter installed on the incident light side, and is a filter in which a plurality of visible light filters “Vis” and a plurality of near-infrared light filters “IR” are arranged alternately in a mosaic pattern (checkerboard pattern). be.
  • the visible light filter “Vis” is a filter that transmits only visible light.
  • the near-infrared light filter "IR” is a filter that transmits only near-infrared light. Therefore, the image sensor section 3 in FIG.
  • FIG. 3 is a monochrome single-plate type, and serves as an image sensor for monochrome images.
  • the incident light passes through the filter 31a and reaches the image sensor 31, but at this time, the visible light filter "Vis” and the near-infrared light filter “IR” separate visible light and near-infrared light for each light receiving element. Receive light. Then, it is output to the video signal processing unit 4 as a visible light signal and a near-infrared light signal.
  • FIG. 3 shows an example of 16 filters, the arrangement is not limited to this and can be arranged according to the number of light receiving elements.
  • FIG. 4 is a diagram showing a second example of the image sensor section in the imaging device of the present invention.
  • the image sensor section 3 shown in FIG. 4 includes a prism 12, an image sensor 32, and an image sensor 33.
  • the prism 12 is a prism that separates incident light into visible light and near-infrared light.
  • the image sensor 32 is an image sensor that receives visible light separated by the prism 12.
  • the image sensor 33 is an image sensor that receives near-infrared light separated by the prism 12.
  • the image sensor section 3 in FIG. 4 is a monochrome two-plate type using two monochrome image sensors that each receive visible light and near-infrared light. The incident light passes through the prism 12 and is separated into visible light and near-infrared light.
  • the image sensor 32 receives only visible light and outputs it to the video signal processing section 4 as a visible light signal.
  • the image sensor 33 receives only near-infrared light and outputs it to the video signal processing section 4 as a near-infrared light signal.
  • the video signal processing unit 4 generates a monochrome image based on the received visible light signal. Therefore, both image sensors do not have a mosaic filter as shown in FIG. 3.
  • FIG. 5 is a diagram showing a third example of the image sensor section in the imaging device of the present invention.
  • the image sensor section 3 shown in FIG. 5 includes an image sensor 34 having a filter 34a.
  • the filter 34a is a filter installed on the incident light side, and includes a plurality of red light filters "R”, a plurality of green light filters “G”, a plurality of blue light filters “B”, and a plurality of near-infrared light filters.
  • Optical filters "IR” are arranged alternately in a mosaic pattern.
  • multiple patterns are lined up with four patterns: a green light filter “G” in the upper left, a red light filter “R” in the upper right, a green light filter “G” in the lower left, and a near-infrared light filter “IR” in the lower right.
  • Green light filter “G” is a filter that passes only green light.
  • the red light filter “R” is a filter that passes only red light.
  • Blue light filter “B” is a filter that passes only blue light.
  • the near-infrared light filter “IR” is a filter that transmits only near-infrared light.
  • the image sensor section 3 in FIG. 5 is of a color single-plate type, and serves as an image sensor for color images.
  • the incident light passes through the filter 34a and reaches the image sensor 34, but at this time, the visible light passes through the red light filter “R” as red light and passes through the green light filter “G”. The light that has passed is received as green light, and the light that has passed through the blue light filter “B” is received as blue light by the image sensor 34. Signals based on these received lights are output to the video signal processing section 4 as visible light signals. Further, the light that has passed through the near-infrared light filter "IR” is received by the image sensor 34 as near-infrared light. A signal based on this received light is output to the video signal processing section 4 as a near-infrared light signal. This makes it possible to receive visible light and near-infrared light separately for each light receiving element.
  • FIG. 5 shows an example of 16 filters, the arrangement is not limited to this and can be arranged according to the number of light receiving elements.
  • FIG. 6 is a diagram showing a fourth example of the image sensor section in the imaging device of the present invention.
  • the image sensor section 3 shown in FIG. 6 includes a prism 12, an image sensor 35, and an image sensor 36.
  • the image sensor 35 has a filter 35a.
  • the prism 12 is a prism that separates incident light into visible light and near-infrared light, and is similar to the prism 12 in FIG. 4 .
  • the filter 35a is a filter in which a plurality of red light filters "R", a plurality of green light filters "G”, and a plurality of blue light filters "B" are alternately arranged in a mosaic pattern.
  • FIG. 6 shows a Bayer array filter 35a that receives visible light.
  • the image sensor 35 is an image sensor that receives visible light separated by the prism 12.
  • the image sensor 36 is an image sensor that receives near-infrared light separated by the prism 12.
  • the image sensor section 3 in FIG. 6 is a color two-plate type using an image sensor that receives visible light and a monochrome image sensor that receives near-infrared light.
  • incident light passes through a prism 12 and is separated into visible light and near-infrared light.
  • the image sensor 35 receives only visible light and outputs it to the video signal processing section 4 as a visible light signal.
  • the image sensor 36 receives only near-infrared light and outputs it to the video signal processing section 4 as a near-infrared light signal.
  • the image sensor 35 can receive light as an RGB color image through the filter 35a.
  • FIG. 6 shows an example of 16 filters, the arrangement is not limited to this and can be arranged according to the number of light receiving elements.
  • FIG. 7 is a diagram showing a fifth example of the image sensor section in the imaging device of the present invention.
  • the image sensor section 3 shown in FIG. 7 includes a prism 15, an image sensor 37, an image sensor 38, an image sensor 39, and an image sensor 40.
  • the prism 15 is a prism that separates incident light into red light, green light, blue light, and near-infrared light, respectively.
  • the image sensor 37 is an image sensor that receives red light separated by the prism 15.
  • the image sensor 38 is an image sensor that receives green light separated by the prism 15.
  • the image sensor 39 is an image sensor that receives blue light separated by the prism 15.
  • the image sensor 40 is an image sensor that receives near-infrared light separated by the prism 15.
  • the image sensor section 3 in FIG. 7 is a color four-plate type using four monochrome image sensors that receive red light, green light, blue light, and near-infrared light, respectively.
  • incident light passes through a prism 15 and is separated into red light, green light, blue light, and near-infrared light.
  • the image sensor 37 receives only red light
  • the image sensor 38 receives only green light
  • the image sensor 39 receives only blue light. Signals based on these received lights are output to the video signal processing section 4 as visible light signals.
  • the image sensor 40 receives only near-infrared light. A signal based on this received light is output to the video signal processing section 4 as a near-infrared light signal.
  • the video signal processing unit 4 generates a color video based on the received red light, green light, and blue light. Therefore, both image sensors do not have a mosaic filter as shown in FIG. 6.
  • FIG. 8 is a diagram showing an example of a flowchart of the imaging apparatus of the present invention. The operation of expanding the apparent depth of field when the aperture 7 of the lens section 2 is opened will be described using FIG. 8. Here, unless otherwise explained, processing in the video signal processing section 4 is shown.
  • step S101 it is determined whether the brightness level of the video input to the video signal processing unit 4 is smaller than a threshold value. If it is determined that the brightness level of the input video is smaller than the threshold, the process advances to step S102. If the brightness level of the input video is equal to or higher than the threshold, the process ends.
  • the brightness level threshold can be set in advance. As the input video here, a video based on a visible light signal can be used.
  • step S102 the aperture 7 of the lens section 2 is opened so that a predetermined brightness level is obtained.
  • the predetermined brightness level here may be, for example, the brightness level threshold value in step S101. If the brightness level is low, the image will be dark, so by opening the aperture 7, the brightness of the image can be maintained at a predetermined level.
  • the degree to which the aperture 7 opens may be determined according to the brightness level detected in step S101.
  • the aperture 7 can be controlled and opened under the control of the control section 5 according to the brightness level detected by the video signal processing section 4.
  • opening the aperture 7 increases the amount of light and raises the brightness level, the depth of field becomes shallower and the degree of blur in the image at a distance other than the specific distance increases.
  • the contour extraction unit 9 extracts contour components from the near-infrared light signal.
  • the near-infrared light signal is a signal of near-infrared light obtained by passing through a near-infrared light cut filter 8 adjacent to the aperture 7.
  • the near-infrared light cut filter 8 is open only at the center and is transparent, so that most of the near-infrared light does not pass through, but only a part of the center passes through. That is, the optical path of the near-infrared light is not affected by the diaphragm 7, and is always in the same state as if the diaphragm were closed. As a result, an image with a deep depth of field can be obtained although the brightness level of the near-infrared light signal is low.
  • the video signal processing unit 4 creates a video (near-infrared video) based on the near-infrared light signal, but since the near-infrared video has a low brightness level, the video is dark as a whole. For this reason, the near-infrared light image may be subjected to gain correction as necessary, or may be passed through an LPF (Low-Pass Filter) to remove noise.
  • LPF Low-Pass Filter
  • Extraction of the contour component can be performed by performing image processing on each image of the near-infrared light video and extracting the contour of each subject.
  • step S104 the video signal processing unit 4 superimposes the contour component extracted from the near-infrared light image onto the visible light image.
  • the visible light image is an image created by a visible light signal that has passed through the aperture 7.
  • the superimposition involves aligning and superimposing outline components from visible light images and near-infrared light images taken at the same time. As a result, the outline component of the near-infrared light image is added to the blurred visible light image with a shallow depth of field, so the outline of each subject can be made clear.
  • FIG. 9 is a diagram illustrating a specific example of superimposing contour components from a visible light image and a near-infrared light image in the imaging apparatus of the present invention.
  • the visible light image 201 is an image obtained when the aperture 7 is opened in step S102 of FIG. Although the brightness level can be increased by opening the aperture 7, the depth of field becomes shallower. In the example of FIG. 9, the car in the foreground is in focus, and other subjects are out of focus, resulting in a blurred image (an image with blurred colors).
  • the brightness level of the image is maintained at a predetermined level so as not to become dark.
  • the contour image 202 is an image representing the contour components extracted in step S103 of FIG. 8.
  • the near-infrared light passes through the near-infrared light cut filter 8 that is open only in the center, resulting in an image with a deep depth of field, but initially the entire image is dark. For this reason, gain correction is performed to increase the gain and brighten the image. At this time, since noise also increases, noise correction such as DNR (Digital Noise Reduction) is also applied.
  • DNR Digital Noise Reduction
  • contour components are extracted based on the image.
  • the contour image 202 in FIG. 9 has a deep depth of field, so that not only the car in the foreground but also buildings, roads, billboards, and other cars in the background are in focus or slightly out of focus. Therefore, it is possible to capture the contour components of each subject (each object) through image processing.
  • the contour component of the contour image 202 can represent the contour of each subject with a line or the like.
  • the superimposed video 203 is a video that has undergone the superimposition process in step S104 of FIG. 8.
  • a visible light image 201 and an outline image 202 are superimposed. Since the visible light image 201 maintains the brightness level, the image does not become dark as a whole. Furthermore, since the contour image 202 adds contour components over a wide range, the contour can be made clear and the apparent depth of field can be increased. This allows you to clearly capture not only the car in the foreground, but also other subjects.
  • the present invention is not limited to the above-described embodiments, and includes various modifications.
  • the embodiments described above are described in detail to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to having all the configurations described.
  • it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment and it is also possible to add the configuration of another embodiment to the configuration of one embodiment.
  • the present invention can be applied not only when the amount of light is insufficient, but also when the amount of light is sufficient.
  • a predetermined effect can be obtained by combining it with other operations that affect the brightness level. For example, it is possible to increase the frame rate by shortening the exposure time by the amount of light increased by opening the aperture 7 of the lens, or to improve the image quality by performing gain correction in the compression direction.
  • FIG. 2 shows an example in which the aperture 7 and the near-infrared light cut filter 8 are close to each other
  • the present invention is also applicable even if they are separated.
  • the aperture 7 is narrowed down to the same size as the aperture of the near-infrared light cut filter 8
  • near-infrared light will not reach anything other than parallel light from the central aperture, but in this case, the visible light image will be Because the depth is deep, contour correction is not necessary.
  • the zoom magnification it can be corrected by correcting it using aberration correction (digital zoom).
  • Computer system 302 ...Processor, 302A, 302B...Processing device, 304...Memory, 306...Memory bus, 308...I/O bus, 309...Bus interface unit, 310...I/O bus interface unit, 312...Terminal interface unit, 314...Storage Interface, 316...I/O device interface, 318...Network interface, 320...User I/O device, 324...Display system, 326...Display device, 330...Network, 350...Latent factor identification application

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Color Television Image Signal Generators (AREA)
PCT/JP2022/047594 2022-03-24 2022-12-23 撮像装置 Ceased WO2023181558A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2024509768A JPWO2023181558A1 (https=) 2022-03-24 2022-12-23

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022-047858 2022-03-24
JP2022047858 2022-03-24

Publications (1)

Publication Number Publication Date
WO2023181558A1 true WO2023181558A1 (ja) 2023-09-28

Family

ID=88100459

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/047594 Ceased WO2023181558A1 (ja) 2022-03-24 2022-12-23 撮像装置

Country Status (2)

Country Link
JP (1) JPWO2023181558A1 (https=)
WO (1) WO2023181558A1 (https=)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006180269A (ja) * 2004-12-22 2006-07-06 Sony Corp 画像処理装置、画像処理方法、撮像装置、プログラム、及び記録媒体
JP2013152369A (ja) * 2012-01-25 2013-08-08 Nippon Seimitsu Sokki Kk 絞り装置およびカメラ
JP2015029841A (ja) * 2013-08-06 2015-02-16 三菱電機エンジニアリング株式会社 撮像装置および撮像方法

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160255323A1 (en) * 2015-02-26 2016-09-01 Dual Aperture International Co. Ltd. Multi-Aperture Depth Map Using Blur Kernels and Down-Sampling

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006180269A (ja) * 2004-12-22 2006-07-06 Sony Corp 画像処理装置、画像処理方法、撮像装置、プログラム、及び記録媒体
JP2013152369A (ja) * 2012-01-25 2013-08-08 Nippon Seimitsu Sokki Kk 絞り装置およびカメラ
JP2015029841A (ja) * 2013-08-06 2015-02-16 三菱電機エンジニアリング株式会社 撮像装置および撮像方法

Also Published As

Publication number Publication date
JPWO2023181558A1 (https=) 2023-09-28

Similar Documents

Publication Publication Date Title
US11582400B2 (en) Method of image processing based on plurality of frames of images, electronic device, and storage medium
US11184553B1 (en) Image signal processing in multi-camera system
EP2589226B1 (en) Image capture using luminance and chrominance sensors
JP7077395B2 (ja) 多重化高ダイナミックレンジ画像
CN110248106B (zh) 图像降噪方法、装置、电子设备以及存储介质
CN109089046B (zh) 图像降噪方法、装置、计算机可读存储介质及电子设备
WO2020207262A1 (zh) 基于多帧图像的图像处理方法、装置、电子设备
JP2020150331A (ja) 画像処理装置、画像処理装置の制御方法、システム、及び、プログラム
WO2020207261A1 (zh) 基于多帧图像的图像处理方法、装置、电子设备
CN110213462B (zh) 图像处理方法、装置、电子设备、图像处理电路及存储介质
KR20240155200A (ko) 포비티드 감지
EP3148177A1 (en) Image generation method based on dual camera module and dual camera module
EP2133238A1 (en) Imaging apparatus and image correction method
CN110930440B (zh) 图像对齐方法、装置、存储介质及电子设备
US8836800B2 (en) Image processing method and device interpolating G pixels
US20250330722A1 (en) Adaptive foveation sensing systems
US20080158372A1 (en) Anti-aliasing in an imaging device using an image stabilization system
CN115529411B (zh) 视频虚化方法和装置
US8976286B2 (en) Imaging apparatus, lens unit, and imaging unit
JP2021532640A (ja) ただ二つのカメラを備える装置およびこの装置を用いて二つの画像を生成する方法
JP7052811B2 (ja) 画像処理装置、画像処理方法及び画像処理システム
US20210377428A1 (en) Monitor camera
WO2023181558A1 (ja) 撮像装置
WO2021039113A1 (ja) 撮像素子、撮像装置、撮像素子の作動方法、及びプログラム
CN115767287B (zh) 图像处理方法与电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22933685

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024509768

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22933685

Country of ref document: EP

Kind code of ref document: A1