KR20220086213A - Camera system and method for improving low-light image, computer program stored in medium for executing the method, and the computer-readable recording medium - Google Patents

Abstract

The present invention provides a camera system for improving low-light images, comprising: a camera that takes an image within an observable area (FOV) and outputs an image signal; a motion sensor for detecting a motion of an object within the observable area of the camera and outputting a motion detection signal; an illuminance sensor for estimating ambient illuminance; and receiving the motion detection signal, determining whether the estimated ambient illuminance is low illuminance or high illuminance, and when the determination result is low illuminance, a first image including color information corresponding to a visible ray region frequency among the image signals It relates to a camera system, including an image processing unit for calling the second image including brightness information corresponding to the infrared region frequency together with the second image and mixing it at a predetermined ratio, and image processing the resulting mixed image. Accordingly, it is possible to obtain a high-contrast image even in a low-light environment.

Description

A camera system and method for improving low-light images, a computer program stored in a medium for executing the method, and a computer-readable recording medium thereof , AND THE COMPUTER-READABLE RECORDING MEDIUM}

The present invention relates to a camera system and method for improving a low-light image, and more particularly, to a camera system and method for improving a low-light image, capable of acquiring a high-contrast image even in a low-light environment in a low-power standby mode.

In the conventional case, high-contrast images could be acquired only when ambient illumination was secured to a certain degree, such as in broad daylight, and additional lighting was required in environments where it is difficult to obtain high-contrast low-illuminance images, such as at night or in basements. .

In addition, in order to minimize power consumption, in the related art, it is difficult to obtain high-contrast, high-quality images in a low-light environment by continuously shooting by lowering the number of video recording frames per second or by using a radar sensing and recording method in a low-power mode.

Accordingly, even in a low-light environment in a low-power standby mode that minimizes power consumption, for example, even a low-light image acquired by a vehicle black box of a vehicle parked in an underground parking lot can be improved to be a high-contrast image camera system and There is a need for a method.

KR 2016-0047196 A KR 2019-0106928 A

Accordingly, an object of the present invention is to provide a camera system and method for improving a low-illuminance image, which can acquire a high-contrast image even in a low-light environment.

Another object of the present invention is to provide a camera system and method for improving a low-illuminance image for a vehicle, capable of acquiring a high-contrast image even in a low-light environment in a low-power standby mode.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

The object is, according to a first aspect of the present invention,

In a camera system for improving low-light images,

a camera that captures an image within an observable area (FOV) and outputs an image signal;

a motion sensor for detecting a motion of an object within the observable area of the camera and outputting a motion detection signal;

an illuminance sensor for estimating ambient illuminance; and

Receives the motion detection signal, determines whether the estimated ambient illuminance is low illuminance or high illuminance, and when the determination result is low illuminance, a first image including color information corresponding to a visible ray region frequency among the image signals; Including an image processing unit for calling the second image including the brightness information corresponding to the infrared region frequency and mixing it at a certain ratio, and image processing the resulting mixed image,

achieved by the camera system.

In this case, a predetermined ratio of mixing the first image and the second image is preferably 1:2 or 2:3.

Furthermore, the image processing unit receives the motion detection signal, determines whether the estimated ambient illuminance is low illuminance or high illuminance, and when the determination result is high illuminance, the image processing unit provides a color corresponding to a visible ray region frequency among the image signals. Only the first image including information may be called and image processed.

In addition to this, the camera further includes a dual bandpass filter that passes a visible ray region frequency and an infrared region frequency of the image signal,

Receives the motion detection signal, determines whether the estimated ambient illuminance is low illuminance or high illuminance, and when the determination result is high illuminance, the image processing unit, a redish component with respect to the infrared region frequency passed among the image signals It may be characterized in that it is corrected.

In particular, the camera is a vehicle camera,

The image processing of the mixed image may be performed starting from a low power standby mode.

On the other hand, the above object is also according to the second aspect of the present invention,

In the low-illuminance image improvement method executed in the image processing unit of the camera system for improving the low-light image,

Receiving an image signal from a camera that takes an image in the viewable area (FOV);

receiving a motion detection signal from a motion detection sensor that detects a motion of an object within an observable area of the camera and outputs a motion detection signal;

receiving an output signal from an illuminance sensor for estimating ambient illuminance; and

Receives the motion detection signal, determines whether the estimated ambient illuminance is low illuminance or high illuminance, and when the determination result is low illuminance, a first including color information corresponding to a visible ray region frequency among the received image signals Calling a second image including the brightness information corresponding to the infrared region frequency together with the image and mixing it at a predetermined ratio, comprising the step of image processing the resulting mixed image,

This is achieved by a method for improving low-light images.

In this case, a predetermined ratio of mixing the first image and the second image is preferably 1:2 or 2:3.

Furthermore, it receives the motion detection signal, determines whether the estimated ambient illuminance is low illuminance or high illuminance, and, if the determination result is high illuminance, includes color information corresponding to a visible ray region frequency among the received image signals It may further include the step of calling only the first image to process the image.

In addition to this, the camera further includes a dual bandpass filter that passes a visible ray region frequency and an infrared region frequency of the image signal,

Receives the motion detection signal, determines whether the estimated ambient illuminance is low illuminance or high illuminance, and if the determination result is high illuminance, correcting the redish component with respect to the frequency of the infrared region passed among the received image signals It may include further steps.

In particular, the camera is a vehicle camera,

The image processing of the mixed image may be performed starting from a low power standby mode.

Said object is also according to a third aspect of the invention,

It is achieved by a computer-readable recording medium recording a program for performing the above method.

Furthermore, the object is also according to a fourth aspect of the invention,

It is achieved by a computer program stored in a medium for executing the above method through combination with hardware.

According to the camera system and method for improving a low-light image of the present invention as described above, a high-contrast image is obtained even in a low-light environment by synthesizing an image including color information and an image including brightness information at a predetermined ratio without additional lighting. There are advantages to being able to

In addition, according to the camera system and method for improving low-light images of the present invention, there is an advantage that high-contrast images can be obtained even in low-light environments in low-power standby mode.

1 is a configuration diagram of a camera system for improving low-light images according to the present invention.
2 is a view showing the selective transmittance of the dual bandpass filter employable in the present invention.
3 is a flowchart of a method for improving a low-light image that is executable in a camera system for improving a low-light image according to the present invention.
4 is a diagram illustrating an example of generating a mixed image by mixing a first image and a second image according to the present invention.
5 is a view showing a histogram of an image in various situations.

Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the contents described in the accompanying drawings. However, the present invention is not limited or limited by the exemplary embodiments. The same reference numerals provided in the respective drawings indicate members that perform substantially the same functions.

Terms including an ordinal number such as 1st, 2nd, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise.

The terms used in the present invention have been selected as currently widely used general terms as possible while considering the functions in the present invention, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than the name of a simple term.

In the entire specification, when a part "includes" a certain element, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated.

1 is a block diagram of a camera system for improving low-light images according to the present invention.

Referring to FIG. 1 , the camera system 1 of the present invention includes: a camera 10 for outputting an image signal by essentially taking an image within an observable area (FOV); a motion detection sensor 20 for detecting a motion of an object within the observable area of the camera 10 and outputting a motion detection signal; an illuminance sensor 30 for estimating ambient illuminance; and an image processing unit 40 that processes the image received from the camera 10 according to the present invention in the case of low light. In addition, the camera system 1 may further include a storage unit 50 for storing the received image and/or the image-processed image for a certain period of time.

The camera 10 may be a digital camera that receives an image through a lens, projects it to a CCD or CMOS image sensor, takes a picture, and stores the image in a storage medium such as a memory card. In particular, the camera 10 is a camera mounted inside a vehicle, and may be a vehicle camera, which is a vehicle black box that can record images inside and outside the vehicle to provide information necessary for understanding an accident situation occurring during driving and parking.

In addition to this, the camera 10 may further include a bandpass filter capable of selectively filtering a desired wavelength band from the input signal, for example, but not limited thereto, as shown in FIG. 2 , in the range of 380 to 650 nm. It may further include a dual bandpass filter that passes the visible ray region frequency and the infrared region frequency centered on 850 nm.

The motion sensor 20 is a sensor that is mounted near the camera and senses the motion of an object, such as a human body, within the observable area of the camera, and may be, for example, a passive infrared sensor (PIR sensor).

The illuminance sensor 30 is a sensor that estimates ambient illuminance and outputs a value corresponding thereto as an output signal, and may be, for example, an illuminance sensor built into the vehicle.

The image processing unit 40 differently image-processes the image received from the camera 10 according to the present invention based on whether the light is low. The operation of the image processing unit 40 will be described with reference to FIG. 3 , which is a flowchart of a low-light image improvement method executable in a camera system for low-light image improvement according to the present invention.

Referring to the drawings, the low-light image improvement method of the present invention may be started from, for example, a low-power standby mode ( S100 ), which is being executed in a vehicle black box of a vehicle parked at night or in an underground parking lot.

First, in the low-power standby mode (S100), when a motion is detected (S200), it is determined whether the light is low or high based on the ambient light (S400). At this time, if no movement is detected, it continuously stays in the low power standby mode (S100).

The reception (S300) of the output signal from the illuminance sensor corresponding to the ambient illuminance is illustrated in FIG. 3 after the reception of the motion detection signal (S200), but is not necessarily limited thereto. For example, the ambient illuminance may be estimated periodically and/or aperiodically based on an event by the illuminance sensor and output to the image processing unit. Accordingly, the reception (S300) of the output signal from the illuminance sensor corresponding to the ambient illuminance may be performed before the reception of the motion detection signal (S200).

Similarly, although the step of determining whether the light is low or high (S400) is shown after receiving the motion detection signal (S200) in FIG. 3, it is not necessarily limited thereto. For example, after receiving the output signal from the illuminance sensor (S300), it is possible to determine whether it is a low illuminance or a high illuminance based on this at any time, so it may be performed before the motion detection signal reception (S200).

Similarly, the step of receiving the image signal from the camera (S500, S500') is also illustrated in FIG. 3 after the step of determining whether the light is low or high light (S400), but is not necessarily limited thereto. For example, when the image signal is received, the camera may periodically and/or aperiodically capture the image based on an event, and the image signal may be output to the image processing unit. Accordingly, the image signal reception (S500, S500') may be performed before/after the motion detection signal reception (S200).

Next, it receives the motion detection signal, determines whether the estimated ambient illuminance is low illuminance or high illuminance, and when the determination result is low illuminance, a first The second image including the brightness information corresponding to the frequency in the infrared region is called together with the image and mixed at a predetermined ratio (S600), and the resulting mixed image is image-processed (S700) and stored (S800).

In this case, the first image is an image including color information or saturation such as Red, Blue, Green, Magenta, Cyan, Yellow, White, and Black, and the second image is an image including brightness information or brightness, and the first image and The second image may be extracted from one image signal (same frame) photographed at one time, or may be extracted from another image signal (different frame) photographed separately.

On the other hand, the second image including brightness information may be expressed as in FIG. 4(b) as a factor determining the image brightness in a mixed image generated by mixing.

5 is a diagram showing a histogram of an image in various situations. 5 (a) and (b) are graphs in which the contrast is uniformly distributed over the entire area, (c) and (d) of FIG. Each of e) and (f) is a graph corresponding to a low-light image and a brightly saturated image. If the histogram distribution is biased toward the bright or dark side, the contrast is lowered, but the object can be identified, and the low-light or brightly saturated image corresponds to a state in which it is difficult to identify the object itself.

In the case of a normal low-light image, it may be the situation of (c) or (e) of FIG. 5 . In the case of (c) of FIG. 5, the first image and the second image are mixed in a ratio of 2:3, and in the case of FIG. 5(e), when the first image and the second image are mixed in a ratio of 1:2, It was confirmed that the distribution was similar to that of (a) of FIG. 5 . Furthermore, when the mixing ratio of the first image and the second image is greater than 1:2 or 2:3, image saturation may occur due to excessive contrast enhancement, and thus image quality may be reduced.

Accordingly, a predetermined ratio of mixing the first image and the second image is preferably 1:2 or 2:3. This blending ratio is a weight for color information included in the first image and brightness information included in the second image, and is applied when mixing.

Accordingly, the image in which the first image (refer to (a) of FIG. 4) and the second image (refer to (b) of FIG. 4) according to the present invention is mixed, as in (c) of FIG. 4, contrasts even in low light can be generated as an improved low-light high-contrast image.

In particular, in the case where the camera is equipped with a dual bandpass filter that passes the visible ray frequency and the infrared ray frequency, the image acquired through the infrared region of the dual bandpass filter is not only about color information, but also about the human body, engine, etc. Since it further includes the brightness information of the object, it can help to obtain a more improved high-contrast image.

The image processing step (S700) of the mixed image may include general image signal processing such as exposure adjustment, gamma correction, noise reduction, etc., in order to improve the image, and in particular, binarization and edge detection and number/character recognition, etc. It may further include an image analysis process using the technique of

On the other hand, it receives the motion detection signal, determines whether the estimated ambient illuminance is low illuminance or high illuminance, and when the determination result is high illuminance, the first Only the image is called (S600'), the image is processed (S700'), and the image is stored (S800).

In this case, since the first image corresponds to an image including color information or chroma such as Red, Blue, Green, Magenta, Cyan, Yellow, White, and Black acquired at high illuminance, it is not necessary to compare it with other images including brightness information or brightness. No need to mix to improve contrast.

Furthermore, it receives the motion detection signal, determines whether the estimated ambient illuminance is low illuminance or high illuminance. Therefore, it may be necessary to correct the redish component for this.

In particular, in the case where the camera is equipped with a dual bandpass filter that passes the visible ray frequency and the infrared ray frequency, it may also be necessary to correct the radish component for the pass infrared region of the dual bandpass filter.

The image processing of the received image ( S700 ′) may include general image signal processing such as exposure adjustment, gamma correction, noise reduction, and WDR in order to improve the image.

Meanwhile, the methods according to an embodiment of the present invention may be at least partially implemented in the form of program instructions and recorded in a computer-readable recording medium. For example, embodied with a program product consisting of a computer-readable medium containing program code, which may be executed by a processor for performing any or all steps, operations, or processes described.

The computer may be a desktop computer, a laptop computer, a notebook computer, a smart phone, or any device that may be incorporated into or may be a similar computing device. A computer is a device having one or more alternative and special purpose processors, memory, storage, and networking components (either wireless or wired). The computer may run, for example, an operating system compatible with Microsoft's Windows, Apple OS X or iOS, a Linux distribution, or an operating system such as Google's Android OS.

The program instruction form may be collectively referred to as software, which may include a computer program, code, instructions, or a combination of one or more of these, and configure the processing device to operate as desired. may be configured or independently or collectively instruct the processing device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. may be permanently or temporarily embody in The software may be distributed over networked computing devices, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded in the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Terms used in this specification generally are intended to be "open-ended" terms, particularly in claims (eg, the body of claims) (eg, "comprising" means "including but not limited to" , "having" should be construed as "having at least more" and "comprising" as "including but not limited to" It is expressly recited in the claims, and in the absence of such recitation, no such intent is understood.

Only specific features of the invention have been shown and described herein, and various modifications and variations will occur to those skilled in the art. It is therefore to be understood that the claims are intended to cover changes and modifications that fall within the spirit of the invention.

1: camera system 10: camera
20: motion sensor 30: illuminance sensor
40: image processing unit 50: storage unit

Claims (12)

In a camera system for improving low-light images,
a camera that captures an image within an observable area (FOV) and outputs an image signal;
a motion sensor for detecting a motion of an object within the observable area of the camera and outputting a motion detection signal;
an illuminance sensor for estimating ambient illuminance; and
Receive the motion detection signal, determine whether the estimated ambient illuminance is low illuminance or high illuminance, and when the determination result is low illuminance, a first image including color information corresponding to a visible ray region frequency among the image signals; Including an image processing unit for calling the second image including the brightness information corresponding to the infrared region frequency and mixing it at a certain ratio, and image processing the resulting mixed image,
camera system. The method of claim 1,
A predetermined ratio of mixing the first image and the second image is 1:2 or 2:3,
camera system. 3. The method of claim 2,
Receives the motion detection signal, determines whether the estimated ambient illuminance is low illuminance or high illuminance, and when the determination result is high illuminance, the image processing unit may include color information corresponding to a visible ray region frequency among the image signals Calling only the first image including the image processing,
camera system. 4. The method of claim 3,
The camera further includes a dual bandpass filter that passes a visible ray region frequency and an infrared region frequency of the image signal,
Receives the motion detection signal, determines whether the estimated ambient illuminance is low illuminance or high illuminance, and when the determination result is high illuminance, the image processing unit, a redish component with respect to the infrared region frequency passed among the image signals characterized by correcting
camera system. 5. The method of claim 4,
The camera is a vehicle camera,
Image processing of the mixed image is performed starting from a low-power standby mode,
camera system. In the low-illuminance image improvement method executed in the image processing unit of the camera system for improving the low-light image,
Receiving an image signal from a camera that takes an image in the viewable area (FOV);
receiving a motion detection signal from a motion detection sensor that detects a motion of an object within an observable area of the camera and outputs a motion detection signal;
receiving an output signal from an illuminance sensor estimating ambient illuminance; and
Receives the motion detection signal, determines whether the estimated ambient illuminance is low illuminance or high illuminance, and when the determination result is low illuminance, a first Calling a second image including brightness information corresponding to the infrared region frequency together with the image and mixing it at a certain ratio, comprising the step of image processing the resulting mixed image,
How to improve low light video. 7. The method of claim 6,
A predetermined ratio of mixing the first image and the second image is 1:2 or 2:3,
How to improve low light video. 8. The method of claim 7,
Receives the motion detection signal, determines whether the estimated ambient illuminance is low illuminance or high illuminance, and, if the determination result is high illuminance, includes color information corresponding to a visible ray region frequency among the received image signals Further comprising the step of processing the image by calling only one image,
How to improve low light video. 9. The method of claim 8,
The camera further includes a dual bandpass filter that passes a visible ray region frequency and an infrared region frequency of the image signal,
Receives the motion detection signal, determines whether the estimated ambient illuminance is low illuminance or high illuminance, and if the determination result is high illuminance, correcting the redish component with respect to the frequency of the infrared region passed among the received image signals further comprising steps,
How to improve low light video. 10. The method of claim 9,
The camera is a vehicle camera,
The image processing of the mixed image is performed starting from a low-power standby mode,
How to improve low light video. A computer-readable recording medium recording a program for performing the method of any one of claims 6 to 10. A computer program stored in a medium for executing the method of any one of claims 6 to 10 through combination with hardware.

Images