KR102200540B1 - Apparatus and method for obtaining multi-brightness image - Google Patents

Abstract

The present invention relates to a device and a method for acquiring multiple luminosity images, wherein images having multiple luminosities in visible-ray and infrared wavelength domains can be photographed and synthesized by using optical filter transmittance adjustment. The device may comprise: a camera unit for photographing a predetermined subject and acquiring images thereof; a lighting unit disposed on the periphery of the camera unit so as to emit infrared light; an optical filter unit disposed on the front surface of the at least one of the camera unit and the lighting unit so as to adjust the amount of transmission of light directed to the camera unit or the amount of emission of infrared light emitted from the lighting unit; a light multiplexing unit for generating a light multiplexing signal so as to control the transmittance of the optical filter unit based on the distance from the subject and the photography environment; and an image processing unit for synthesizing and processing multiple photographed images.

Description

Multi-luminosity image acquisition apparatus and method {APPARATUS AND METHOD FOR OBTAINING MULTI-BRIGHTNESS IMAGE}

The present invention relates to a multi-luminosity image acquisition apparatus, and more particularly, to a multi-luminosity image acquisition apparatus and method capable of capturing and synthesizing multi-luminosity images in visible and infrared wavelength regions by using optical filter transmittance adjustment.

In general, in order to identify outdoor objects such as CCTV surveillance cameras, vehicles, robots, drones, etc., cameras have shooting conditions from daytime to night, dark indoors, tunnels, basements, night lights, vehicle headlights, no light at all. It can cover indoor environment, etc.

Currently, in various environments, in accordance with consumer demand for high-precision image quality, it is being switched to HD-class IP cameras, and LED lighting is supplemented to increase the sensitivity of the camera sensor, or to shoot at night or dark places in closed spaces. It is a common trend to use it.

Cameras used at night use a lot of IR (infrared) LED lights that do not have a visual effect for security or crime prevention, and a lot of use of new LED lights to clearly identify objects in dark spaces in low light. Research is ongoing.

In addition, technology for preventing IR oversaturation by increasing the shutter speed by using LED flashing synchronized with camera shooting or detecting the distance of the subject is disclosed as Korean Patent No. 10-0983346 (2010.09.20). It has been registered.

However, in the existing technology, when environmental changes such as moving photographing are severe, it is difficult to adjust the camera and LED lighting adaptively to the distance and amount of light of a subject, and hardware configuration corresponding to various functions may be complicated.

In addition, simply by increasing the sensitivity of the camera sensor, it may be difficult to shoot an image at a level in which the human vision adapts to and perceives a sudden change in lighting conditions and a wide area of luminance experienced in everyday life.

Accordingly, there is a need to develop a multi-luminosity image acquisition device that can be easily manufactured using an existing low-cost sensor and can realize image synthesis that can visually identify and recognize objects in various lighting environments.

The technical problem to be achieved by an embodiment of the present invention is to perform a single luminous intensity multiplexing method using the transmittance adjustment of a light filter, thereby implementing an image synthesis capable of visually identifying and recognizing objects in various lighting environments. An image acquisition apparatus and method are provided.

In addition, the technical problem to be achieved by an embodiment of the present invention is a multi-luminosity image acquisition device that can easily be manufactured using a low-cost sensor by obtaining distance information from a distance sensor and photographing environment information from an illuminance sensor, and I want to provide a way.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present invention belongs from the following description. I will be able to.

In order to solve the above technical problem, the apparatus for obtaining a multi-luminosity image according to an embodiment of the present invention includes a camera unit for capturing an image by photographing a predetermined subject, and an illumination unit disposed around the camera unit to emit infrared light. , An optical filter unit that is disposed on the front of at least one of the camera unit and the lighting unit to control the amount of transmitted light incident to the camera unit or the amount of infrared light emitted from the lighting unit, and the transmittance of the optical filter unit based on the distance to the subject and the shooting environment. A light intensity multiplexer that generates an optical multiplexing signal to control, and an image processor that synthesizes and processes a plurality of captured images.

In addition, a multi-luminosity image acquisition apparatus according to another embodiment of the present invention includes a camera unit for capturing a predetermined subject to obtain an image, an illumination unit disposed around the camera unit to emit infrared light, a camera unit, and the front of the illumination unit. An optical filter unit that is arranged at the same time and controls the amount of transmitted light incident on the camera unit or the amount of infrared light emitted from the illumination unit, and a light intensity multiplexing unit that generates an optical multiplexing signal to control the transmittance of the optical filter unit based on the distance to the subject and the shooting environment. And an image processing unit for synthesizing and processing a plurality of photographed images, the camera unit includes an infrared cutoff switch for switching whether to block infrared light according to a shooting environment, and the optical filter unit includes a front surface of the camera unit and the lighting unit It may include a fixed optical filter arranged to cover at the same time.

In addition, a multi-luminosity image acquisition apparatus according to another embodiment of the present invention includes a camera unit for capturing a predetermined subject to obtain an image, an illumination unit disposed around the camera unit to emit infrared light, a camera unit, or a front side of the illumination unit. An optical filter unit that is arranged to adjust the transmission amount of light incident on the camera unit or the projection amount of infrared light emitted from the illumination unit, a light intensity multiplexing unit that generates a light multiplexing signal to control the transmittance of the optical filter unit based on a distance to a subject and a photographing environment, And an image processing unit for synthesizing and processing a plurality of captured images, and the optical filter unit includes a movable optical filter that is moved to cover the front of the camera unit or to cover the front of the lighting unit, and a position of the movable optical filter. It may include an optical filter switch to move to or move to the front of the lighting unit.

Here, the optical filter switch is electrically connected to the motor switch to control the positional movement of the movable optical filter according to the shooting environment, and the motor switch includes a motor electrically connected to the optical filter switch, and the day mode ( Day mode) or a night mode (night mode) may include a day / night switch for switching the motor.

In addition, the optical filter switch moves the position of the movable optical filter to the front of the camera unit in a day mode in which the measured illuminance is greater than or equal to the preset illuminance in the shooting environment, and the night mode in which the measured illuminance is less than the preset illuminance in the shooting environment In (night mode), the position of the movable optical filter can be moved to the front of the lighting unit.

In addition, the optical filter switch moves the position of the movable optical filter to the front of the camera unit in the day mode of the shooting environment, and the movable optical filter is in the night mode of the shooting environment. You can also move the position to the front of the lighting unit.

In addition, a multi-luminosity image acquisition apparatus according to another embodiment of the present invention includes a camera unit for capturing a predetermined subject to obtain an image, an illumination unit disposed around the camera unit to emit infrared light, and a camera disposed in front of the camera unit. An optical filter unit that adjusts the amount of transmitted light incident to the negative or the amount of infrared light reflected on the subject, a light intensity multiplexing unit that generates a light multiplexing signal to control the transmittance of the optical filter unit based on the distance to the subject and the shooting environment, and a number of shots An image processing unit for synthesizing and processing images of, and the camera unit includes a beam splitter for separating incident light into visible and infrared light, a first camera for photographing multiple optical images of the separated visible light band, and It includes a second camera for photographing multiple optical images of the separated infrared wide band, and the optical filter unit is a fixed type that is disposed on the optical separation surface of the beam splitter and covers the front surfaces of the first and second cameras of the camera unit excluding the illumination unit at the same time. It may include an optical filter.

On the other hand, the multi-luminosity image acquisition method of the multi-luminosity image acquisition apparatus according to an embodiment of the present invention includes a multi-luminosity image acquisition device including a light intensity multiplexer for controlling the transmittance of the camera unit, the lighting unit, the image processing unit, and the optical filter unit. A method of obtaining a luminous intensity image, comprising: checking whether an image capturing request is received by a luminous intensity multiplexer, acquiring distance information and photographing environment information from a subject when an image capturing request is received by the luminous intensity multiplexer, and Calculating an optical multiplexing signal function based on distance information and photographing environment information of, and controlling the transmittance of the optical filter unit by generating an optical multiplexing signal based on the optical multiplexing signal function calculated by the luminous intensity multiplexing unit, and controlling the transmittance of the optical filter unit by the illumination unit Infrared light is emitted during the time when is modulated, the camera unit photographs multi-luminosity images during the time when the transmittance of the optical filter unit is modulated, and the image processing unit synthesizes and processes the captured multi-luminosity images. .

The effects of the apparatus and method for obtaining a multi-luminosity image according to the present invention will be described as follows.

In the present invention, by performing a single luminous intensity multiplexing method by using a transmittance adjustment of an optical filter, it is possible to implement image synthesis capable of visually identifying and recognizing objects in various lighting environments.

In addition, according to the present invention, by obtaining information on a distance to a subject from a distance sensor and information on a photographing environment from an illuminance sensor, it is possible to easily utilize and manufacture a low-cost sensor.

In addition, the present invention acquires multi-band multiple images using light intensity modulation in visible and near-infrared bands, and synthesizes them into high dynamic range (HDR) visual images, thereby underexposure of light experienced in images in various environments at day and night. Or you can overcome partial overexposure.

In addition, the present invention proposes a method of modulating the intensity of light incidence of photographing using an optical filter control compliant with light and a method of modulating the intensity of outgoing light of an IR lamp, so that it is possible to respond in real time to changes in external light that changes in various places.

In addition, the present invention can be easily extended and grafted to a mobile camera system that can be mounted on vehicles, robots, drones, etc., as well as fixed CCTV cameras, and manufacturing cost is low.

As such, the present invention provides various image signal processing fields that acquire and express WDR (Wide Dynamic Range) images with multiple intensity and bands for visible and IR light, along with fixed and mobile surveillance camera systems, autonomous vehicles, and unmanned drones. It can be actively used in various image visualization fields, including the object recognition field, which is emerging recently.

In particular, changes in external light that occur during self-driving video are frequently outside the recognition range of general sensors, such as when entering or exiting the basement or tunnel, looking outside or facing the opposite situation, dark parking lot, and strong vehicle headlight changes. Can occur.

A sensor-based driving situation detection area among autonomous vehicle systems is a process of recognizing such an external environment and is an essential element in recognizing a fixed or moving object or a path.

At this time, protection of life and property by recognizing an unexpected situation occurring in real time and preventing collisions should be prioritized over some other advanced functions.

In addition, the present invention meets the need for surveillance and security that are increasing globally due to increasing social tensions and crimes, and it is possible to implement a high-efficiency video surveillance system.

In addition, the present invention can be used in social security, industry, and broadcasting sites.

That is, the present invention can be used in security CCTV, vehicle-cell phone-drone camera, black box market, and industrial fields where image sensors are utilized, and can be used in the field of video-based video signal or broadcast video signal processing. .

In addition, the present invention can be used in related studies and research fields.

That is, the present invention will contribute to the development of intelligent image analysis technologies such as the field of image fusion, object search, face search, and environment monitoring, and can be used as an alternative technology that can improve the performance of existing image sensor hardware with limited performance, Research result data and simulation data in the form of reports and papers are expected to help develop linked technologies in the future, and are expected to be used in high-tech fields such as airports, ports, military facilities, and construction sites.

Further scope of applicability of the present invention will become apparent from the detailed description below. However, since various changes and modifications within the spirit and scope of the present invention can be clearly understood by the person concerned, it should be understood that the detailed description and specific embodiments such as preferred embodiments of the present invention are given by way of example only.

1 is a block diagram illustrating an apparatus for obtaining a multi-luminosity image according to an exemplary embodiment of the present invention.
2 is a timing diagram showing an image capture sequence of a camera and adjusting transmittance of an optical filter by an optical multiplexing signal.
3 is a diagram illustrating a level multiplexing process for image multiplexing corresponding to visible light in a day mode.
4 is a diagram illustrating a level multiplexing process for multiplexing a composite image corresponding to visible and infrared light in a night mode.
5 is a block diagram illustrating a multi-luminosity image acquisition apparatus according to another embodiment of the present invention.
6 is a block diagram illustrating an apparatus for obtaining a multi-luminosity image according to another embodiment of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

The suffixes "module" and "unit" for the constituent elements used in the following description are simply given in consideration of ease of writing in the present specification, and the "module" and "unit" may be used interchangeably with each other.

Further, embodiments of the present invention will be described in detail with reference to the accompanying drawings and contents described in the accompanying drawings, but the present invention is not limited or limited by the embodiments.

The terms used in the present specification have selected general terms that are currently widely used as possible while considering functions in the present invention, but this may vary according to the intention or custom of a technician working in the art, or the emergence of new technologies. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in the description of the corresponding invention. Therefore, it should be noted that terms used in the present specification should be interpreted based on the actual meaning of the term and the entire contents of the present specification, not a simple name of the term.

1 is a block diagram illustrating an apparatus for obtaining a multi-luminosity image according to an exemplary embodiment of the present invention.

As shown in Figure 1, the present invention, the camera unit 100, the lighting unit 200, the optical filter unit 300, the intensity multiplexing unit 400, the image storage unit 600, and the image processing unit 500 It may include.

Here, the camera unit 100 may acquire an image by photographing a predetermined subject.

As an example, the camera unit 100 is a single sensor camera, and a lens, an IR-cut switch, an image sensor, an electronic shutter, and an exposure meter may be configured as a single module.

At this time, the infrared cut-off switch enables infrared cut-on (IR-cut on) at high illuminance (day) to capture a visible light image, and infrared cut-off (IR-cut off) at low illuminance (night) to broadband (visible light) +Near-infrared) Allows you to shoot video.

Further, the camera unit 100 may include a broadband camera that captures a broadband image including both visible and infrared light.

In addition, the camera unit 100 may capture multiple optical images of a visible light band or multiple optical images of a composite band including both visible and infrared light according to a photographing environment.

Here, the camera unit 100 captures multiple images of light in the visible light band when the measured illuminance in the photographing environment is greater than or equal to the preset illuminance, and when the measured illuminance in the photographing environment is less than the preset illuminance, the camera unit 100 includes both visible and infrared light. It is possible to take multi-band optical images.

In addition, the camera unit 100 may include an infrared cut-off switch for switching whether to cut off infrared light according to a photographing environment.

In this case, the optical filter unit 300 may include a camera unit 100 including an infrared cutoff switch and a fixed optical filter disposed to cover the front surface of the lighting unit 200 at the same time.

Here, the camera unit 100 captures a visible light image by controlling the infrared cutoff switch to turn on to block infrared light when the measured illuminance in the photographing environment is equal to or higher than a preset illuminance, and the measured illuminance in the photographing environment is preset. If it is less than the illuminance, the infrared cutoff switch is turned off so as not to block the infrared light, so that a broadband image including both visible and infrared light can be captured.

Next, the lighting unit 200 may include a fixed infrared lamp that is independently disposed on one side of the camera unit 100 to emit infrared light.

Here, the fixed infrared lamp may be supplied with only DC power without inputting a signal for synchronizing shooting with the camera unit 100.

Subsequently, the optical filter unit 300 may include a fixed optical filter disposed to simultaneously cover the front surfaces of the camera unit 100 and the lighting unit 200.

Here, the fixed optical filter basically uses a liquid crystal (LC) filter capable of varying the transmittance according to the applied voltage, but all electro-optical devices whose transmittance is adjusted are applied.

In some cases, in the fixed optical filter, a plurality of polarizing plates having different transmittances may be disposed.

For example, a polarizing plate having a first transmittance may be disposed to face the camera unit 100, and a polarizing plate having a second transmittance may be disposed to face the lighting unit 200.

In addition, the light intensity multiplexing unit 400 includes a sensing unit 410 for sensing a distance to a subject and a photographing environment, and an optical multiplexing signal to control the transmittance of the optical filter unit 300 based on the distance to the subject and a photographing environment. It may include an optical filter control unit 420 that generates.

Here, the sensing unit 410 may include a distance sensor 412 that senses a distance to a subject, and an illuminance sensor 414 that senses ambient illuminance in a photographing environment.

As an example, the distance sensor 412 may include at least one of a region of field (RoF) sensor and an ultrasonic sensor, but is not limited thereto.

In addition, the illuminance sensor 414 may include at least one of a cadmium sulfide (CdS) device and a solar panel, but is not limited thereto.

Subsequently, the optical filter control unit 420 of the luminous intensity multiplexing unit 400 controls the transmittance of the optical filter unit 300 based on the distance to the subject and the shooting environment, and the transmittance control unit 422 and the camera unit 100 are A timing control unit 424 that controls the signal timing to synchronize the period and the transmittance change period of the optical filter unit 300, and a multiplexing signal that generates a multiplexed signal according to control signals from the transmittance control unit 422 and the timing control unit 424 It may include a generator 426.

In addition, when generating the optical multiplexing signal, the light intensity multiplexing unit 400 calculates a light modulation contrast value for controlling the transmittance of the optical filter unit 300 based on a distance to a subject and a photographing environment, and the camera unit 100 ), and generates a timing pulse for synchronizing the period of change in transmittance of the optical filter unit 300, calculates the minimum transmittance level corresponding to the photographing environment, and based on the optical modulation contrast value, the timing pulse, and the minimum transmittance level. An optical multiplexed signal function may be calculated, and an optical multiplexed signal may be generated based on the calculated optical multiplexed signal function.

For example, when calculating the optical multiplexing signal function, the luminous intensity multiplexer 400 recognizes as a day mode if the measured illuminance in the photographing environment is more than a preset illuminance, and if it is recognized as the day mode, it measures in the photographing environment. Calculates the light modulation contrast value based on the illuminance, generates a timing pulse for synchronizing the period of photographing of the camera unit 100 and the period of change of transmittance of the optical filter unit 300, and calculates the minimum transmission level in the day mode. , The optical multiplexing signal function of the day mode can be calculated based on the optical modulation contrast value, the timing pulse, and the minimum transmission level.

Here, the minimum transmittance level may be the lowest transmittance ratio set relatively to the maximum open transmittance of the optical filter unit 300 during the day time.

Therefore, when calculating the optical multiplexing signal function of the day mode, the optical intensity multiplexing unit 400 calculates the optical multiplexing signal function = (1-minimum transmission level (day mode)) × timing pulse × optical modulation contrast value. can do.

In addition, when calculating the optical multiplexing signal function, the luminous intensity multiplexer 400 recognizes as a night mode if the measured illuminance in the photographing environment is less than a preset illuminance, and determines the distance to the subject when it is recognized as a night mode. Based on the optical modulation contrast value is calculated, a timing pulse for synchronizing the transmittance change period of the optical filter unit 300 based on the photographing period of the camera unit 100, and calculating the minimum transmission level in the night mode, The optical multiplexing signal function can be calculated based on the optical modulation contrast value, the timing pulse, and the minimum transmission level.

Here, the minimum transmittance level may be the lowest transmittance ratio set relatively to the maximum open transmittance of the optical filter unit 300 at night.

Therefore, when calculating the optical multiplexing signal function, the optical multiplexing unit 400 can calculate the optical multiplexing signal function = (1-minimum transmission level (night mode)) × timing pulse × optical modulation contrast value. .

Next, the image storage unit 600 includes at least one of an optical multiplex image of a visible light band, an optical multiplex image of an infrared light band, and a multiplex optical image including both visible and infrared light captured from the camera unit 100. You can save one.

In addition, the image processing unit 500 synthesizes any one of optical multiple images in the visible light band, optical multiple images in the infrared light band, and optical multiple images in the complex band including both visible and infrared light according to the photographing environment. Can be handled.

For example, the shooting environment is a first shooting environment in which daylight conditions are daytime, a second shooting environment in the night time zone in dark or weak lighting conditions, a moving shooting condition, or a constant condition in which lighting and external light changes are severe. 3 The shooting environment may be included, but is not limited thereto.

That is, if the photographing environment is a first photographing environment, the image processing unit 500 synthesizes and processes a high-exposure image and a low-exposure image from multiple optical images in the visible light band, and if the photographing environment is a second photographing environment, the infrared light band A high-exposure image and a low-exposure image are synthesized and processed from multiple optical images, and if the photographing environment is a third photographing environment, a high-exposure image and a low-exposure image are generated from optical multiple images of a composite band including both visible and infrared light. It can be synthesized and processed.

As shown in FIG. 1, the apparatus for obtaining a multi-luminosity image according to an embodiment of the present invention includes a camera unit 100 for capturing an image by photographing a predetermined subject, and is disposed around the camera unit 100 to provide infrared light. The illumination unit 200, the camera unit 100, and the front surface of the illumination unit 200 are simultaneously disposed to control the transmission amount of light incident to the camera unit 100 or the projection amount of infrared light emitted from the illumination unit 200. The filter unit 300, a light intensity multiplexing unit 400 that generates an optical multiplexing signal to control the transmittance of the optical filter unit 300 based on a distance to a subject and a photographing environment, and synthesizes and processes a plurality of photographed images. It may include an image processing unit 500.

Here, the camera unit 100 includes an infrared cut-off switch for switching whether or not to block the infrared light according to a photographing environment, and the optical filter unit 300 simultaneously displays the front of the camera unit 100 and the lighting unit 200 It may include a fixed optical filter arranged to cover.

At this time, the camera unit 100 captures a visible light image by controlling the infrared cutoff switch to turn on to block infrared light when the measured illuminance in the shooting environment is higher than a preset illuminance, and the measured illuminance in the shooting environment is preset. If it is less than the illuminance, the infrared cutoff switch is turned off so as not to block the infrared light, so that a broadband image including both visible and infrared light can be captured.

As described above, in the embodiment of FIG. 1, a camera which is a single sensor (broadband) may be applied, an IR lamp may be synchronized using an optical filter, and a fixed IR lamp may be used.

The embodiment of FIG. 1 is an apparatus for acquiring a high luminance band image in an environment with a large luminance range, an optical filter that adjusts the transmitted amount of incident light or the projection amount of IR light, an illuminometer that measures the luminous intensity of the environment, and measures the distance of a subject. It includes a rangefinder, a camera that acquires images by accumulating electric charges according to the amount of light transmitted through the optical filter, and an optical filter control unit that controls the transmittance of the optical filter. It may include a timing control unit for.

In addition, in the embodiment of FIG. 1, the timing control unit of the optical filter control unit generates an external trigger signal to synchronize the optical multiplexing signal and the photographing timing signal, or shoots using a vertical synch signal of a camera. The timing and optical multiplexing signals can be synchronized.

Next, the photographing environment is analyzed from the illuminometer and rangefinder signals, and the light transmittance of the optical filter can be adjusted to a desired level by adjusting the voltage level of the low-level signal among the optical multiplexed signals.

The single camera sensor captures multiple images of visible light during the day when IR light is blocked depending on the brightness of the shooting environment, and multiple images of a complex band that includes both visible and IR light at night when IR light passes. do.

At this time, there is no need to input a signal to the IR lamp for synchronizing shooting with the camera, and only DC power is supplied to the IR lamp (fixed IR lamp), and instead, the camera shooting and synchronization can be performed by an optical filter in front of the IR lamp.

In addition, since the modulated IR light source can be created by adjusting the light transmittance from the optical multiplexed signal, it is possible to use an independent lamp because a separate synchronization circuit connection is not required.

Here, there are two areas where it is more advantageous to use standalone IR lighting than built-in IR lighting.

Most network cameras using standalone IR lighting have a longer surveillance distance than cameras with built-in IR lighting, and a wider range of cameras using standalone IR lighting, giving you more flexibility in camera selection.

FIG. 2 is a timing diagram showing an image capture sequence of a camera and adjusting transmittance of an optical filter by an optical multiplexing signal, and FIG. 3 is a diagram illustrating a level multiplexing process for image multiplexing corresponding to visible light in a day mode.

FIG. 2 is a conceptual diagram of multi-image photographing of visible light in a day mode, and (a) of FIG. 2 shows an optical multiplexing signal for photographing a 60 Hz double-exposure image of a visible light image, and (b) of FIG. ) Shows the sequence of changing the light transmittance of the optical filter.

As shown in FIG. 2, when a voltage is not applied to the optical filter, about 50% light transmittance is shown (high region), and when the voltage is increased, the light transmittance decreases (low region).

Here, when the voltage is adjusted so that the light transmittance of the low region is about 3% to about 5%, the light exposure ratio of the two images may be about 10:1.

In this case, the camera may take an image alternately for two exposure states during a period of light modulation.

In addition, since the image processing unit synthesizes the high dynamic range (HDR) image by two frames at one frame interval, a frame rate can be maintained.

In addition, the difference in the level of the optical multiplexing signal may be adjusted differently for the change in brightness of the photographing environment obtained from the light meter signal information.

The illuminometer may be a cadmium sulfide (CdS) device or a solar panel.

In addition, the present invention, as shown in FIG. 3, has the maximum light modulation contrast in a bright environment (high light amount), and the darker the light (low light amount), the smaller the light modulation contrast to compensate for the insufficient light amount. , This can be set as an optional function.

In addition, the function for the optical multiplexed signal in the day mode is as shown in Equation 1 below.

Here, the minimum transmittance level may be the lowest transmittance ratio set relative to the maximum open transmittance of the optical filter during the day time.

4 is a diagram illustrating a level multiplexing process for multiplexing a composite image corresponding to visible and infrared light in a night mode.

As shown in FIG. 4, in the present invention, in a night mode, the amount of IR light is modulated according to a near subject distance to capture and synthesize a multiple exposure image of a composite band of visible and infrared light.

In the present invention, first, distance information for a nearby subject is obtained from a response of a distance sensor, and accordingly, a transmittance adjustment signal of an IR lamp light can be generated.

Further, in the present invention, the light amount modulation of the fixed IR lamp may be synchronized with the image capturing frame rate through modulation of the transmittance of the optical filter synchronized with the camera capturing.

The final IR intensity is adjusted to a high intensity (for shooting a distant subject) and a low intensity (for shooting a near-field subject) during one shooting cycle (timing control), and the IR lamp as a whole is alternately lit at two levels of high-low. It can produce the effect of being.

That is, according to the present invention, color information and near object contour information can be obtained at a low level, and distant object contour information can be obtained at a high level.

Subsequently, in a night mode, an optical multiplexing signal function can be obtained from the rangefinder signal information.

Here, as the rangefinder, an ultrasonic sensor or a region of field or infrared (RoF) sensor may be used.

In addition, the function for the optical multiplexed signal in the night mode is shown in Equation 2 below.

Here, the minimum transmittance level may be the lowest transmittance ratio set relative to the maximum open transmittance of the optical filter at night.

In addition, the minimum light intensity level of the IR lamp during the half cycle time is a level at which a nearby subject is identified, and is set by adjusting the transmittance of the optical filter in consideration of a distance.

5 is a block diagram illustrating a multi-luminosity image acquisition apparatus according to another embodiment of the present invention.

As shown in FIG. 5, the present invention includes a camera unit 100, an illumination unit 200, an optical filter unit 300, a luminous intensity multiplexing unit 400, an image storage unit 600, and an image processing unit 500. It may include.

Here, the camera unit 100 may acquire an image by photographing a predetermined subject.

As an example, the camera unit 100 is a single sensor camera, and a lens, an image sensor, an electronic shutter, and an exposure meter may be configured as one module.

Further, the camera unit 100 may include a broadband camera that captures a broadband image including both visible and infrared light.

In addition, the camera unit 100 may capture multiple optical images of a visible light band or multiple optical images of a composite band including both visible and infrared light according to a photographing environment.

Here, the camera unit 100 captures multiple images of light in the visible light band when the measured illuminance in the photographing environment is greater than or equal to the preset illuminance, and when the measured illuminance in the photographing environment is less than the preset illuminance, the camera unit 100 includes both visible and infrared light. It is possible to take multi-band optical images.

Next, the lighting unit 200 may include a fixed infrared lamp that is independently disposed on one side of the camera unit 100 to emit infrared light.

Here, the fixed infrared lamp may be supplied with only DC power without inputting a signal for synchronizing shooting with the camera unit 100.

Subsequently, the optical filter unit 300 is disposed on the front side of the camera unit 100 or the illumination unit 200 to adjust the transmission amount of light incident to the camera unit 100 or the projection amount of infrared light emitted from the illumination unit 200. have.

For example, the optical filter unit 300 includes a movable optical filter 310 that is moved to cover the front of the camera unit 100 or to cover the front of the lighting unit 200, and the position of the movable optical filter 310 The optical filter switch 320 may be moved to the front of the camera unit 100 or to the front of the lighting unit 200.

Here, the mobile optical filter 310 basically uses a liquid crystal (LC) filter capable of varying the transmittance according to the applied voltage, but all electro-optical devices whose transmittance is adjusted are applied.

In addition, the optical filter switch 320 may be electrically connected to the motor switch 330 to control the positional movement of the movable optical filter 310 according to the photographing environment.

Here, the motor switch 330 switches the motor 332 electrically connected to the optical filter switch 320 and the motor 332 in a day mode or a night mode according to a photographing environment. It may include a day / night switch 334.

For example, the day/night switch 334 may be switched to a day mode or a night mode based on the measured illuminance sensed by the illuminance sensor 414.

That is, the optical filter switch 320 moves the position of the movable optical filter 310 to the front of the camera unit 100 in the day mode in which the measured illuminance is equal to or greater than the preset illuminance in the photographing environment, and If the measurement illuminance is less than the preset illuminance in a night mode, the position of the movable optical filter 310 may be moved to the front of the lighting unit 200.

In some cases, the optical filter switch 320 moves the position of the movable optical filter 310 to the front of the camera unit 100 in the day mode, which is a day time of the photographing environment, and at night time of the photographing environment. In the in-night mode, the position of the movable optical filter 310 may be moved to the front of the lighting unit 200.

In addition, the luminous intensity multiplexing unit 400 may generate an optical multiplexing signal to control the transmittance of the optical filter unit 300 based on a distance to a subject and a photographing environment.

In addition, the sensing unit may include a distance sensor for sensing a distance to a subject, and an illuminance sensor 414 for sensing ambient illuminance in a photographing environment.

As an example, the distance sensor may include at least one of a RoF (Region of Field or infrared) sensor and an ultrasonic sensor, but is not limited thereto.

In addition, the illuminance sensor 414 may include at least one of a cadmium sulfide (CdS) device and a solar panel, but is not limited thereto.

Next, when generating the optical multiplexing signal, the light intensity multiplexing unit 400 calculates a light modulation contrast value for controlling the transmittance of the optical filter unit 300 based on the distance to the subject and the photographing environment, and the camera unit 100 ), and generates a timing pulse for synchronizing the period of change in transmittance of the optical filter unit 300, calculates the minimum transmittance level corresponding to the photographing environment, and based on the optical modulation contrast value, the timing pulse, and the minimum transmittance level. An optical multiplexed signal function may be calculated, and an optical multiplexed signal may be generated based on the calculated optical multiplexed signal function.

For example, when calculating the optical multiplexing signal function, the luminous intensity multiplexer 400 recognizes as a day mode if the measured illuminance in the photographing environment is more than a preset illuminance, and if it is recognized as the day mode, it measures in the photographing environment. Calculates the light modulation contrast value based on the illuminance, generates a timing pulse for synchronizing the period of photographing of the camera unit 100 and the period of change of transmittance of the optical filter unit 300, and calculates the minimum transmission level in the day mode. , The optical multiplexing signal function of the day mode can be calculated based on the optical modulation contrast value, the timing pulse, and the minimum transmission level.

Here, the minimum transmittance level may be the lowest transmittance ratio set relatively to the maximum open transmittance of the optical filter unit 300 during the day time.

Therefore, when calculating the optical multiplexing signal function of the day mode, the optical intensity multiplexing unit 400 calculates the optical multiplexing signal function = (1-minimum transmission level (day mode)) × timing pulse × optical modulation contrast value. can do.

In addition, when calculating the optical multiplexing signal function, the luminous intensity multiplexer 400 recognizes as a night mode if the measured illuminance in the photographing environment is less than a preset illuminance, and determines the distance to the subject when it is recognized as a night mode. Based on the optical modulation contrast value is calculated, a timing pulse for synchronizing the transmittance change period of the optical filter unit 300 based on the photographing period of the camera unit 100, and calculating the minimum transmission level in the night mode, The optical multiplexing signal function can be calculated based on the optical modulation contrast value, the timing pulse, and the minimum transmission level.

Here, the minimum transmittance level may be the lowest transmittance ratio set relatively to the maximum open transmittance of the optical filter unit 300 at night.

Therefore, when calculating the optical multiplexing signal function, the optical multiplexing unit 400 can calculate the optical multiplexing signal function = (1-minimum transmission level (night mode)) × timing pulse × optical modulation contrast value. .

Next, the image storage unit 600 includes at least one of an optical multiplex image of a visible light band, an optical multiplex image of an infrared light band, and a multiplex optical image including both visible and infrared light captured from the camera unit 100. You can save one.

In addition, the image processing unit 500 synthesizes any one of optical multiple images in the visible light band, optical multiple images in the infrared light band, and optical multiple images in the complex band including both visible and infrared light according to the photographing environment. Can be handled.

For example, the shooting environment is a first shooting environment in which daylight conditions are daytime, a second shooting environment in the night time zone in dark or weak lighting conditions, a moving shooting condition, or a constant condition in which lighting and external light changes are severe. 3 The shooting environment may be included, but is not limited thereto.

That is, if the photographing environment is a first photographing environment, the image processing unit 500 synthesizes and processes a high-exposure image and a low-exposure image from multiple optical images in the visible light band, and if the photographing environment is a second photographing environment, the infrared light band A high-exposure image and a low-exposure image are synthesized and processed from multiple optical images, and if the photographing environment is a third photographing environment, a high-exposure image and a low-exposure image are generated from optical multiple images of a composite band including both visible and infrared light. It can be synthesized and processed.

As described above, in the embodiment of FIG. 5, a single sensor (broadband) camera may be applied, an IR lamp may be synchronized using an optical filter, and a fixed IR lamp and an optical filter switch may be used.

The embodiment of FIG. 5 is a device for acquiring a high luminance band image in an environment with a large luminance range, an optical filter that adjusts the transmitted amount of incident light or the projection amount of IR light, an illuminometer that measures the luminous intensity of the environment, and measures the distance of a subject. It includes a rangefinder, a camera that acquires images by accumulating electric charges according to the amount of light transmitted through the optical filter, and an optical filter control unit that controls the transmittance of the optical filter. It may include a timing control unit for.

In addition, in the embodiment of FIG. 5, an optical filter switch is added so that the position of the optical filter can be changed according to the day mode or the night mode, and a motor switch is connected to multiplex the visible light exposure of the camera using the optical filter during the daytime. And, at night time, the light exposure of the IR lamp can be multiplexed by moving the optical filter toward the fixed IR lamp.

In the embodiment of FIG. 5, an external independent IR lamp can be combined with a single broadband sensor, and synchronization between the IR lamp and the camera is not required, and the luminous intensity multiplexing unit 400 has the same operation procedure of the embodiment of FIG. 1. .

6 is a block diagram illustrating an apparatus for obtaining a multi-luminosity image according to another embodiment of the present invention.

As shown in Figure 6, the present invention, the camera unit 100, the lighting unit 200, the optical filter unit 300, the intensity multiplexing unit 400, the image storage unit 600, and the image processing unit 500 It may include.

Here, the camera unit 100 may acquire an image by photographing a predetermined subject.

For example, the camera unit 100, as a dual sensor camera, is composed of two identical single sensors, and an IR-cut switch may be removed.

As an example, the camera unit 100 includes a beam splitter 700 for separating incident light into visible light and infrared light, a first camera 110 for photographing multiple optical images of the separated visible light band, and It may include a second camera 120 for photographing multiple optical images of the separated infrared light band.

Here, the optical filter unit 300 is disposed on the light separation surface of the beam splitter 700 to simultaneously cover the front surfaces of the first and second cameras 110 and 120 of the camera unit 100 excluding the illumination unit 200 It may include a fixed optical filter.

In this case, the first and second cameras 110 and 120 may be broadband cameras that capture a broadband image including both visible and infrared light.

In addition, the first and second cameras 110 and 120 may be arranged perpendicular to each other around the light separation surface of the beam splitter 700.

In addition, the optical filter unit 300 and the beam splitter 700 may be sequentially separated or bonded in the light incident direction, and the optical filter unit 300 is separated from the beam splitter 700 with respect to the optical path and vertically Alternatively, it may be bonded to the beam splitter 700 and disposed at an angle of about 45 ^o .

Next, the lighting unit 200 may include a fixed infrared lamp that is independently disposed on one side of the camera unit 100 to emit infrared light.

Here, the fixed infrared lamp may be supplied with only DC power without inputting a signal for synchronizing shooting with the camera unit 100.

Subsequently, the optical filter unit 300 may include a fixed optical filter disposed to cover the front surface of the camera unit 100 excluding the lighting unit 200.

Here, the fixed optical filter basically uses a liquid crystal (LC) filter capable of varying the transmittance according to the applied voltage, but all electro-optical devices whose transmittance is adjusted are applied.

In addition, the luminous intensity multiplexing unit 400 may generate an optical multiplexing signal to control the transmittance of the optical filter unit 300 based on a distance to a subject and a photographing environment.

In addition, the sensing unit may include a distance sensor for sensing a distance to a subject, and an illuminance sensor for sensing ambient illuminance in a photographing environment.

As an example, the distance sensor may include at least one of a RoF (Region of Field or infrared) sensor and an ultrasonic sensor, but is not limited thereto.

In addition, the illuminance sensor may include at least one of a cadmium sulfide (CdS) device and a solar panel, but is not limited thereto.

Next, when generating the optical multiplexing signal, the light intensity multiplexing unit 400 calculates a light modulation contrast value for controlling the transmittance of the optical filter unit 300 based on the distance to the subject and the photographing environment, and the camera unit 100 ), and generates a timing pulse for synchronizing the period of change in transmittance of the optical filter unit 300, calculates the minimum transmittance level corresponding to the photographing environment, and based on the optical modulation contrast value, the timing pulse, and the minimum transmittance level. An optical multiplexed signal function may be calculated, and an optical multiplexed signal may be generated based on the calculated optical multiplexed signal function.

For example, when calculating the optical multiplexing signal function, the luminous intensity multiplexer 400 recognizes as a day mode if the measured illuminance in the photographing environment is more than a preset illuminance, and if it is recognized as the day mode, it measures in the photographing environment. Calculates the light modulation contrast value based on the illuminance, generates a timing pulse for synchronizing the period of photographing of the camera unit 100 and the period of change of transmittance of the optical filter unit 300, and calculates the minimum transmission level in the day mode. , The optical multiplexing signal function of the day mode can be calculated based on the optical modulation contrast value, the timing pulse, and the minimum transmission level.

Here, the minimum transmittance level may be the lowest transmittance ratio set relatively to the maximum open transmittance of the optical filter unit 300 during the day time.

Therefore, when calculating the optical multiplexing signal function of the day mode, the optical intensity multiplexing unit 400 calculates the optical multiplexing signal function = (1-minimum transmission level (day mode)) × timing pulse × optical modulation contrast value. can do.

In addition, when calculating the optical multiplexing signal function, the luminous intensity multiplexer 400 recognizes as a night mode if the measured illuminance in the photographing environment is less than a preset illuminance, and determines the distance to the subject when it is recognized as a night mode. Based on the optical modulation contrast value is calculated, a timing pulse for synchronizing the transmittance change period of the optical filter unit 300 based on the photographing period of the camera unit 100, and calculating the minimum transmission level in the night mode, The optical multiplexing signal function can be calculated based on the optical modulation contrast value, the timing pulse, and the minimum transmission level.

Here, the minimum transmittance level may be the lowest transmittance ratio set relatively to the maximum open transmittance of the optical filter unit 300 at night.

Therefore, when calculating the optical multiplexing signal function, the optical multiplexing unit 400 can calculate the optical multiplexing signal function = (1-minimum transmission level (night mode)) × timing pulse × optical modulation contrast value. .

Next, the image storage unit 600 includes at least one of an optical multiplex image of a visible light band, an optical multiplex image of an infrared light band, and a multiplex optical image including both visible and infrared light captured from the camera unit 100. You can save one.

In addition, the image processing unit 500 synthesizes any one of optical multiple images in the visible light band, optical multiple images in the infrared light band, and optical multiple images in the complex band including both visible and infrared light according to the photographing environment. Can be handled.

For example, the shooting environment is a first shooting environment in which daylight conditions are daytime, a second shooting environment in the night time zone in dark or weak lighting conditions, a moving shooting condition, or a constant condition in which lighting and external light changes are severe. 3 The shooting environment may be included, but is not limited thereto.

That is, if the photographing environment is a first photographing environment, the image processing unit 500 synthesizes and processes a high-exposure image and a low-exposure image from multiple optical images in the visible light band, and if the photographing environment is a second photographing environment, the infrared light band A high-exposure image and a low-exposure image are synthesized and processed from multiple optical images, and if the photographing environment is a third photographing environment, a high-exposure image and a low-exposure image are generated from optical multiple images of a composite band including both visible and infrared light. It can be synthesized and processed.

As described above, in the embodiment of FIG. 6, a camera that is a dual sensor (visible light and near-infrared band) is applied, IR incident light is synchronized using a beam splitter and an optical filter, and a fixed IR lamp may be used.

That is, in the embodiment of FIG. 6, visible light multiplexing and IR light multiplexing image capturing may occur simultaneously (always), an IR-cut switch is not required, and a fixed IR lamp may be used.

Here, the dual sensor may mean two single sensors from which the infrared cutoff switch is removed.

In the exemplary embodiment of FIG. 6, a dual optical path method in which two image sensors are spatially arranged side-by-side for photographing may be applied.

In this case, in the embodiment of FIG. 6, relatively simple manufacturing is possible without an additional device, but image alignment is performed for a subject in a short distance (within about 3 m to 5 m) because a difference in view point occurs between the two sensors. Complex viewing angle correction processing may be required.

Accordingly, the embodiment of FIG. 6 may apply a single light path method in which visible light and IR images are captured by beam splitting on the same incident light path.

In the embodiment of FIG. 6, an optical filter and a beam splitter may be sequentially bonded in the light incident direction, and may be disposed at an angle of 45 ^° to the optical path.

In addition, the two cameras are capable of capturing a broadband image, but since the visible light and the IR band are separated through a beam splitter, each camera can be in charge of capturing visible light and IR images.

Here, the visible light camera captures a multi-exposure visible light image light modulated by the optical filter by the light intensity multiplexing unit 400, and at the same time, the IR camera modulates the IR light reflected by the fixed IR lamp with the optical filter to multiple exposure. You can take an IR image.

Here, the light multiplexing signal function of the light intensity multiplexer 400 may be generated from the day mode as shown in Equation 1 above the reference illuminance, and may be generated from the night mode as shown in Equation 2 below the reference illuminance.

In this case, in a bright environment, the visible light image centered on the visible light image is multiplied while the IR image including the maximum transmittance is simultaneously multiplexed. Can be multiple shots.

In addition, the embodiment of FIG. 6 is a dual camera system, which may be operated at all times, and the image storage unit 600 may simultaneously store a multi-level visible light image and an IR image.

Here, when performing image fusion, it is useful to synthesize a multi-exposure visible light image in the daytime when the outside light is strong or at night when the illumination light is sufficient, and in the absence of lighting or in a low-light environment (night mode), the IR image synthesis is useful. I can.

In addition, when illumination light (building lighting, vehicle headlight) is irregularly present even at night, or when outside light is insufficient, such as a corridor or indoors, even during daytime, it suddenly becomes dark when moving indoors or underground (darkness + some strong external light). , When strong lighting (vehicle headlights, building interior lights) is turned on at night, or the image is locally brightened by strong lighting, it is possible to synthesize composite-band HDR images from constant multiple images in visible and IR bands, and local light is insufficient. And image reproduction can be performed without saturation.

As an example, three cases of image synthesis are as follows.

First, in the case of daylight hours (daylight conditions), a multi-exposure visible light image may be synthesized (IR blocked) using an optical filter and a visible light camera.

Second, in the case of night time (dark or weak lighting conditions), multi-IR band image synthesis (IR pass through) can be performed using an optical filter, an IR camera, and an IR lamp.

Third, in the case of constant conditions (moving photography, lighting, and external light fluctuations), synthesis of an IR image and a visible light composite image may be performed using an optical filter, a visible-IR camera, and an IR lamp.

For example, in the case of a shooting environment in which the shutter speed of a night surveillance camera is set slowly because the average incident light is small, and a sudden change of light (vehicle headlight) or a strong indoor light flows in, the conventional method is strong visible light only by IR multiplexing. Although it is not possible to overcome the local saturation phenomenon of the present invention, the method of the present invention takes a visible light camera with an appropriate exposure and synthesizes the multiple-exposure image in a dark environment on average. Local saturation can be improved by synthesizing visible light images.

In addition, in the case of a shooting environment in a tunnel or in a shaded place in an average bright environment, the constant IR image capturing of the present invention enables synthesis and reproduction of an image without a shadow area.

In addition, in the case of daytime shooting in bright outside light, in a shooting environment where the field of view is not clearly secured due to fog or haze, the IR image is captured simultaneously with the visible light multiple images (in this case, the IR lamp does not operate). de-hazing) video can be played back.

As described above, a method of obtaining a multi-luminosity image according to a multi-luminosity image acquisition device including a light intensity multiplexer for controlling the transmittance of the camera unit, the illumination unit, the image processing unit, and the optical filter unit is as follows.

First, the luminous intensity multiplexer may check whether an image capturing request is received.

In addition, the luminous intensity multiplexer may obtain distance information from a subject and information on a photographing environment when an image photographing request is received.

Here, the luminous intensity multiplexer may acquire distance information by sensing a distance to a subject, and acquire illuminance information by sensing ambient illuminance in a photographing environment.

Subsequently, the luminous intensity multiplexer may calculate an optical multiplexing signal function based on the acquired distance information and photographing environment information.

Here, the luminous intensity multiplexing unit calculates a light modulation contrast value for controlling the transmittance of the optical filter unit based on the distance to the subject and the shooting environment, and generates a timing pulse for synchronizing the shooting period of the camera unit and the transmittance change period of the optical filter unit. , The minimum transmission level corresponding to the photographing environment may be calculated, and an optical multiplexing signal function may be calculated based on the optical modulation contrast value, the timing pulse, and the minimum transmission level.

For example, when calculating the light modulation contrast value, the luminous intensity multiplexer recognizes as a day mode if the measured illuminance in the photographing environment is greater than or equal to a preset illuminance. A modulation contrast value is calculated, and if the measured illuminance in the photographing environment is less than a preset illuminance, it is recognized as a night mode, and when it is recognized as a night mode, a light modulation contrast value may be calculated based on the distance to the subject.

In addition, when calculating the minimum transmittance level, the luminous intensity multiplexer recognizes as a day mode if the measured illuminance in the shooting environment is more than a preset illuminance, and calculates the minimum transmittance level in the day mode when it is recognized as a day mode. , If the measured illuminance in the photographing environment is less than a preset illuminance, it is recognized as a night mode, and if it is recognized as a night mode, a minimum transmittance level in the night mode may be calculated.

Here, the minimum transmittance level in the day mode is the lowest transmittance ratio set relative to the maximum open transmittance of the optical filter during the day time, and the minimum transmittance level in the night mode is the maximum open transmittance of the optical filter at night. It may be the lowest transmittance ratio set relative to.

In addition, when calculating the optical multiplexing signal function, the luminous intensity multiplexing unit is the optical multiplexing signal function = (1-minimum transmission level (day mode or night mode)) × timing pulse × optical modulation contrast value (day mode or night mode). It can be calculated with an equation.

Next, the luminous intensity multiplexer may control the transmittance of the optical filter unit by generating an optical multiplexed signal based on the calculated optical multiplexing signal function.

In addition, the illumination unit may emit infrared light during a time when the transmittance of the optical filter unit is modulated, and the camera unit may capture multiple luminous intensity images during a time when the transmittance of the optical filter unit is modulated.

Here, the camera unit, when capturing multi-luminosity images, includes optical multi-images in the visible light band, light multi-images in the infrared light band, and light in the composite band including all visible and infrared light during a time when the transmittance of the optical filter unit is modulated. At least one of the multiple images may be photographed and stored.

Subsequently, the image processing unit may synthesize and process the captured multi-luminosity images.

Here, the image processing unit, when synthesizing and processing the photographed multi-luminosity images, according to the photographing environment, optical multiple images of the visible light band, optical multiple images of the infrared light band, and a composite band including both visible and infrared light. Any one of optical multiple images may be synthesized and processed.

For example, the shooting environment is a first shooting environment in which daylight conditions are daytime, a second shooting environment in the night time zone in dark or weak lighting conditions, a moving shooting condition, or a constant condition in which lighting and external light changes are severe. 3 The shooting environment may be included, but is not limited thereto.

Here, the image processing unit combines and processes a high-exposure image and a low-exposure image from multiple optical images in the visible light band if the photographing environment is the first photographing environment, and if the photographing environment is the second photographing environment, the optical multiple image in the infrared broad band High-exposure image and low-exposure image are synthesized and processed from the field, and if the shooting environment is a third shooting environment, high-exposure and low-exposure images are synthesized and processed from optical multiple images of a complex band that includes both visible and infrared light. can do.

In the conventional HDR image synthesis, an HDR image can be reproduced by synthesizing an image photographed with multiple exposures to overcome a limited dynamic luminance range of the visible light band of a camera sensor, but the present invention uses a multi-intensity and multi-band image. A composite HDR image synthesis method can be proposed.

The present invention acquires and synthesizes multiple exposure images by adjusting the visible light exposure of a sensor, acquires and synthesizes multiple IR images by adjusting the intensity of IR light, and provides a visible light band image to improve the image expression at night with irregular lighting. It is also possible to reproduce a wideband image by synthesizing the images in the and IR bands.

In addition, the present invention enables detection of both near and far objects by multi-photographing and synthesizing an IR-blocked image and an IR-passed image.

The hybrid HDR image synthesis method of the present invention is to synthesize a multi-exposure image of the first visible light band and a multi-lighting image of the IR light band, and synthesize a second visible light HDR image and an IR HDR image. It may be a method of complex synthesis.

As described above, according to the present invention, by performing the unified luminous intensity multiplexing method using the transmittance adjustment of the optical filter, image synthesis capable of visually identifying and recognizing objects for various lighting environments can be implemented.

In addition, according to the present invention, by obtaining information on a distance to a subject from a distance sensor and information on a photographing environment from an illuminance sensor, it is possible to easily utilize and manufacture a low-cost sensor.

In addition, the present invention acquires multi-band multiple images using light intensity modulation in visible and near-infrared bands, and synthesizes them into high dynamic range (HDR) visual images, thereby underexposure of light experienced in images in various environments at day and night. Or you can overcome partial overexposure.

In addition, the present invention proposes a method for modulating photographing incident light intensity using optical filter control that is adaptive to light and modulating the light intensity of IR lamp output, so that it is possible to respond in real time to changes in external light that change in various places.

In addition, the present invention can be easily extended and grafted to a mobile camera system that can be mounted on vehicles, robots, drones, etc., as well as fixed CCTV cameras, and manufacturing cost is low.

As such, the present invention provides various image signal processing fields that acquire and express WDR (Wide Dynamic Range) images with multiple intensity and bands for visible and IR light, along with fixed and mobile surveillance camera systems, autonomous vehicles, and unmanned drones. It can be actively used in various image visualization fields, including the object recognition field, which is emerging recently.

In particular, changes in external light that occur during self-driving video are frequently outside the recognition range of general sensors, such as when entering or exiting the basement or tunnel, looking outside or facing the opposite situation, dark parking lot, and strong vehicle headlight changes. Can occur.

A sensor-based driving situation detection area among autonomous vehicle systems is a process of recognizing such an external environment and is an essential element in recognizing a fixed or moving object or a path.

At this time, protection of life and property by recognizing an unexpected situation occurring in real time and preventing collisions should be prioritized over some other advanced functions.

In addition, the present invention meets the need for surveillance and security that are increasing globally due to increasing social tensions and crimes, and it is possible to implement a high-efficiency video surveillance system.

In addition, the present invention can be used in social security, industry, and broadcasting sites.

That is, the present invention can be used in security CCTV, vehicle-cell phone-drone camera, black box market, and industrial fields where image sensors are utilized, and can be used in the field of video-based video signal or broadcast video signal processing. .

In addition, the present invention can be used in related studies and research fields.

That is, the present invention will contribute to the development of intelligent image analysis technologies such as the field of image fusion, object search, face search, and environment monitoring, and can be used as an alternative technology that can improve the performance of existing image sensor hardware with limited performance, Research result data and simulation data in the form of reports and papers are expected to help develop linked technologies in the future, and are expected to be used in high-tech fields such as airports, ports, military facilities, and construction sites.

Features, structures, effects, and the like described in the present inventions above are included in at least one embodiment of the present invention, and are not necessarily limited to only one embodiment. Further, the features, structures, effects, etc. illustrated in each embodiment may be implemented by combining or modifying other embodiments by a person having ordinary knowledge in the field to which the embodiments belong. Accordingly, contents related to such combinations and modifications should be interpreted as being included in the scope of the present invention.

In addition, although the embodiments have been described above, these are only examples and do not limit the present invention, and those of ordinary skill in the field to which the present invention belongs are illustrated above within the scope not departing from the essential characteristics of the present embodiment. It will be seen that various modifications and applications that are not available are possible. For example, each component specifically shown in the embodiment can be modified and implemented. And differences related to these modifications and applications should be construed as being included in the scope of the present invention defined in the appended claims.

100: camera unit
200: lighting unit
300: optical filter unit
400: intensity multiplexing unit
500: image processing unit
600: image storage unit

Claims (24)

A camera unit that captures a predetermined subject and acquires an image;
An illumination unit disposed around the camera unit to emit infrared light;
An optical filter unit disposed on a front surface of at least one of the camera unit and the illumination unit to adjust the transmission amount of light incident to the camera unit or the projection amount of infrared light emitted from the illumination unit;
A light intensity multiplexing unit generating an optical multiplexing signal to control transmittance of the optical filter unit based on a distance to the subject and a photographing environment; And,
An image processing unit for synthesizing and processing the plurality of captured images,
The luminous intensity multiplexing unit,
When generating the optical multiplexing signal, an optical modulation contrast value for controlling the transmittance of the optical filter unit is calculated based on the distance to the subject and the photographing environment, and the photographing period of the camera unit and the transmission change period of the optical filter unit are synchronized. To generate a timing pulse for, calculate a minimum transmission level corresponding to the photographing environment, calculate an optical multiplexing signal function based on the optical modulation contrast value, timing pulse, and minimum transmission level, and the calculated optical multiplexing signal function A multi-luminosity image acquisition device, characterized in that generating the optical multiplexed signal based on. The method of claim 1, wherein the camera unit,
A multi-luminosity image acquisition device comprising a broadband camera for photographing a broadband image including both visible and infrared light. The method of claim 1, wherein the camera unit,
A multi-luminosity image acquisition device, characterized in that photographing multiple optical images of a visible light band or multiple optical images of a complex band including both visible and infrared light according to the photographing environment. The method of claim 3, wherein the camera unit,
If the measured illuminance in the photographing environment is equal to or greater than a preset illuminance, the optical multi-image of the visible light band is captured
A multi-luminosity image acquisition device, characterized in that, when the measured illuminance in the photographing environment is less than a preset illuminance, an optical multi-image of a composite band including both visible and infrared light is captured. The method of claim 1, wherein the camera unit,
And an infrared cut-off switch for switching whether to cut off the infrared light according to the photographing environment. The method of claim 5, wherein the optical filter unit,
A multi-luminosity image acquisition device comprising: a fixed optical filter disposed to cover the front of the camera unit including the infrared cutoff switch and the lighting unit at the same time. The method of claim 1, wherein the camera unit,
A beam splitter for separating the incident light into visible light and infrared light;
A first camera for photographing multiple optical images of the separated visible light band; And,
And a second camera for capturing multiple optical images of the separated infrared light band. The method of claim 7, wherein the optical filter unit,
And a fixed optical filter disposed on the light splitting surface of the beam splitter to simultaneously cover front surfaces of the first and second cameras of the camera unit except for the illumination unit. The method of claim 7, wherein the first and second cameras,
A multi-luminosity image acquisition device, characterized in that it is a broadband camera that captures a broadband image including both visible and infrared light. The method of claim 7, wherein the first and second cameras,
Multi-luminosity image acquisition apparatus, characterized in that arranged perpendicular to each other about the light splitting surface of the beam splitter. The method of claim 1, wherein the optical filter unit,
A movable optical filter that is moved to cover the front surface of the camera unit or the illumination unit; And,
And an optical filter switch for moving the position of the movable optical filter to the front of the camera unit or to the front of the lighting unit. The method of claim 11, wherein the optical filter switch,
A multi-luminosity image acquisition device, characterized in that electrically connected to a motor switch to control a positional movement of the movable optical filter according to the photographing environment. The method of claim 1, wherein the light intensity multiplexing unit,
A sensing unit for sensing a distance to the subject and a photographing environment; And,
And an optical filter control unit generating the optical multiplexing signal to control the transmittance of the optical filter unit based on the distance to the subject and the photographing environment. The method of claim 13, wherein the optical filter control unit,
A transmittance control unit controlling transmittance of the optical filter unit based on a distance to the subject and a photographing environment;
A timing controller for controlling a signal timing so as to synchronize a photographing period of the camera unit and a transmission change period of the optical filter unit; And,
And a multiplex signal generator configured to generate the multiplex signal according to control signals from the transmittance control unit and the timing control unit. delete The method of claim 1, wherein the light intensity multiplexing unit,
When calculating the optical multiplexing signal function, if the measured illuminance in the photographing environment is greater than or equal to a preset illuminance, it is recognized as a day mode, and if recognized as the day mode, an optical modulation contrast value based on the measured illuminance in the photographing environment And generating a timing pulse for synchronizing the photographing period of the camera unit and the transmission change period of the optical filter unit, calculating the minimum transmission level in the day mode, and the optical modulation contrast value, the timing pulse, and the minimum transmission The multi-luminosity image acquisition device, characterized in that calculating the optical multiplexing signal function of the day mode based on the level. The method of claim 16, wherein the minimum transmission level,
A multi-luminosity image acquisition device, characterized in that the lowest transmittance ratio set relatively to the maximum open transmittance of the optical filter during day time. The method of claim 16, wherein the light intensity multiplexer,
When calculating the optical multiplexing signal function of the day mode, the optical multiplexing signal function = (1-minimum transmission level (day mode)) × timing pulse × optical modulation contrast value, characterized in that it is calculated by calculating the multi-luminosity image acquisition Device. The method of claim 1, wherein the light intensity multiplexing unit,
When calculating the optical multiplexing signal function, if the measured illuminance in the photographing environment is less than a preset illuminance, it is recognized as a night mode, and if recognized as the night mode, an optical modulation contrast value is calculated based on the distance to the subject. And generates a timing pulse for synchronizing the transmittance change period of the optical filter unit based on the photographing period of the camera unit, calculates the minimum transmission level in the night mode, and calculates the optical modulation contrast value, the timing pulse and the minimum transmission A multi-luminosity image acquisition device, characterized in that calculating an optical multiplexing signal function based on a level. The method of claim 19, wherein the minimum transmission level is
A multi-luminosity image acquisition device, characterized in that the lowest transmittance ratio set relatively to the maximum open transmittance of the optical filter unit at night time. The method of claim 19, wherein the light intensity multiplexing unit,
When calculating the optical multiplexing signal function, the optical multiplexing signal function = (1-minimum transmittance level (night mode)) × timing pulse × optical modulation contrast value, characterized in that it is calculated by a formula. In the multi-luminosity image acquisition method of a multi-luminosity image acquisition device including a light intensity multiplexer for controlling transmittance of a camera unit, an illumination unit, an image processing unit, and an optical filter unit,
Checking, by the luminosity multiplexer, whether an image capturing request is received;
Obtaining, by the luminous intensity multiplexer, information about a distance to a subject and information about a photographing environment when an image photographing request is received;
Calculating, by the luminous intensity multiplexing unit, an optical multiplexing signal function based on the acquired distance information and photographing environment information;
Generating the optical multiplexed signal based on the calculated optical multiplexing signal function by the optical multiplexing unit to control transmittance of the optical filter;
The illumination unit emits infrared light during a time when the transmittance of the optical filter unit is modulated, and the camera unit photographs multiple luminous intensity images during a time when the transmittance of the optical filter unit is modulated; And,
The image processing unit comprises the step of synthesizing and processing the photographed multi-luminosity images,
The step of calculating the optical multiplexing signal function,
Calculating an optical modulation contrast value for controlling transmittance of the optical filter unit based on a distance to the subject and a photographing environment;
Generating a timing pulse for synchronizing a photographing period of the camera unit and a transmission change period of the optical filter unit;
Calculating a minimum transmission level corresponding to the photographing environment; And,
And calculating an optical multiplexing signal function based on the optical modulation contrast value, timing pulse, and minimum transmission level. The method of claim 22, wherein obtaining distance information and photographing environment information from the subject comprises:
A method of obtaining a multi-luminosity image, comprising sensing the distance to the subject to obtain the distance information, and sensing the surrounding illuminance in the photographing environment to obtain illuminance information. delete

Images