KR102378372B1 - Data processing system for cmos image sensor capable of mitigating banding noise and processing method thereof - Google Patents

Abstract

Provided is a system for receiving one long exposure channel data and P number of short exposure channel data from a CMOS image sensor, and processing the data of the CMOS image sensor. The system includes a controller which generates a control signal for controlling the operation of the CMOS image sensor. The P number of short exposure channel data may be composed of first short exposure channel data to P^th short exposure channel data. The controller shifts the readout position of the Y^th short exposure channel data for the K^th line of the (N+1)^th frame by a predetermined time from the readout position of the Y^th short exposure channel data for the K^th line of the N^th frame to generate the control signal.

Description

A CMOS image sensor data processing system capable of mitigating banding noise and a processing method therefor

The present invention relates to a data processing system for a CMOS image sensor capable of mitigating banding noise and a processing method therefor.

FIG. 1 is an explanatory diagram illustrating the principle of generation of banding noise caused by a light collecting structure of a rolling shutter of a CMOS image sensor in a photographing environment using synchronous lighting. In addition, FIG. 2 shows an exemplary view of an image in which actual banding noise is generated.

In FIG. 1, since a 60Hz fluorescent lamp is used by full-wave rectification, the actual AC power frequency of the externally radiated light becomes 120Hz. In addition, FIG. 1 exemplifies a case in which an internal synchronization signal (vertical sync) of a used camera is 30 FPS (Frame Per Second) progressive scan.

Since the data of each line exposed by the rolling shutter is sequentially transmitted from the CMOS image sensor, the exposure start time of each line is not the same as in Global Shutter such as CCD, but only by the time of the line sequence. will be pushed out

Accordingly, when the CMOS image sensor reflects the sine wave of synchronous lighting such as indoor fluorescent lamps as it is, the amount of wave energy focused on each line is different. , banding noise appears as shown in FIG. 2 . That is, the brightness of the illumination is different when each of the M lines constituting one frame of the image is exposed, and this is the same continuously in the next frame. Banding noise in a fixed form due to the difference in brightness continuously appears in the image even if the frame is changed.

Within the general system configuration range in which the power frequency and the vertical sync cycle inside the camera are synchronized (eg, NTSC, PAL video standard) as shown in FIG. 1, the position of the noise does not show a random or non-normal distribution tendency. Since it is fixed, it is difficult to remove it with a general noise reducer, and it remains in the form of FPN (Fixed Pattern Noise).

Synchronous lighting is defined as lighting with a frequency characteristic of multiples synchronized with the image synchronization signal of the sensor. When shooting a video in such an indoor synchronous lighting environment, the appropriate amount of light from the light source and the amount of exposure of the sensor may vary depending on many necessary variables. At this time, in the multi-frame synthesis method of WDR, due to the rolling shutter condensing structure of the CMOS image sensor according to the short exposure control of the short channel, multiple It shows a wide banding noise phenomenon.

The present invention aims to solve the technical problem as described above, and it is possible to obtain a broadband image with high visibility by reducing a banding noise component that occurs when acquiring an indoor image in a synchronous lighting environment. An object of the present invention is to provide a data processing system for a CMOS image sensor capable of reducing banding noise and a processing method thereof.

The present invention is a system for receiving one long exposure channel data and P short exposure channel data from a CMOS image sensor and processing the data of the CMOS image sensor, and a control for controlling the operation of the CMOS image sensor A controller for generating a signal, wherein the controller is configured to determine the readout position of the Y-th short exposure channel data for the K-th line of the N+1th frame and the Y-th short exposure for the K-th line of the Nth frame. The control signal is generated to shift a readout position of the channel data by a predetermined time, wherein the P short exposure channel data includes a first short exposure channel data to a Pth short exposure channel data, and the N, the K and each of P is a natural number of 1 or more, and Y is a natural number of 1 or more and P or less.

In addition, the controller, for the Nth frame to the N+Lth frame set as one frame set, the degree of shift in the next frame from one frame of the readout position of the Yth short exposure channel data for the Kth line The control signal is generated so that L is a natural number of 2 or more.

In addition, the controller, P short exposure times for P short exposure channels and one long exposure time for one long exposure channel of the N th frame to the N + L th frame set as one frame set, respectively, The control signal is generated to have the same value in the case of the same channel for the Nth frame to the N+Lth frame, wherein L is a natural number of 1 or more.

(상기 수학식에서, Piir(N)은 N번째 프레임에서의 해당 픽셀의 누적 휘도값, W는 미리 설정된 가중치 및 Pcur(N+1)은 N+1번째 프레임에서의 해당 픽셀의 실제 휘도값을 각각 나타낸다.) 상기 N+1번째 프레임에서의 해당 픽셀의 누적 휘도값을 산출 후, 산출된 상기 N+1번째 프레임에서의 해당 픽셀의 누적 휘도값으로 N+1번째 프레임에서의 해당 픽셀의 실제 휘도값을 대체하는 것에 의해 잡음을 감소시키는 것을 특징으로 한다.In addition, the system of the present invention includes: a channel separator that receives the one long exposure channel data and the P short exposure channel data from the CMOS image sensor, and separates and outputs each channel data; a wideband image synthesizer for receiving one long exposure channel data and P short exposure channel data separated and output from the channel separator and synthesizing them into a wideband image; and a noise attenuator for receiving a wideband image input from the wideband image synthesizer and reducing noise, wherein the noise attenuator includes the cumulative luminance value Piir( Calculate N+1),

(In the above equation, Piir(N) is the accumulated luminance value of the corresponding pixel in the Nth frame, W is a preset weight, and Pcur(N+1) is the actual luminance value of the corresponding pixel in the N+1th frame, respectively. After calculating the cumulative luminance value of the corresponding pixel in the N+1th frame, the calculated cumulative luminance value of the corresponding pixel in the N+1th frame is used as the actual luminance of the corresponding pixel in the N+1th frame. It is characterized in that the noise is reduced by substituting the values.

In addition, the weight is preferably set using a difference value between the accumulated luminance value of the corresponding pixel in the Nth frame and the actual luminance value of the corresponding pixel in the N+1th frame. Specifically, the weight is a second weight value when the difference between the accumulated luminance value of the corresponding pixel in the Nth frame and the actual luminance value of the corresponding pixel in the N+1th frame is less than a preset first difference value. When the difference between the accumulated luminance value of the corresponding pixel in the Nth frame and the actual luminance value of the corresponding pixel in the N+1th frame is greater than or equal to the preset first difference value, as the difference increases, the second 2 It may be set to a value gradually decreasing from the weight value.

The present invention is a method of receiving one long exposure channel data and P short exposure channel data from a CMOS image sensor and processing the data of the CMOS image sensor, and a control for controlling the operation of the CMOS image sensor A control step of generating a signal; including, in which the control step sets the readout position of the Y-th short exposure channel data for the K-th line of the N+1th frame, and the Y-th for the K-th line of the Nth frame The control signal is generated to shift a readout position of the short exposure channel data by a predetermined time, wherein the P short exposure channel data is composed of a first short exposure channel data to a Pth short exposure channel data, the N; Each of K and P is a natural number equal to or greater than 1, and Y is a natural number equal to or greater than 1 and equal to or less than P.

Specifically, in the control step, for the Nth frame to the N+Lth frame set as one frame set, from one frame of the readout position of the Yth short exposure channel data for the Kth line to the next frame The control signal is generated so that the degree of shift is the same, wherein L is a natural number of 2 or more.

In addition, in the control step, P short exposure times for P short exposure channels and one long exposure time for one long exposure channel of the Nth frame to the N+Lth frame set as one frame set are respectively , the control signal is generated to have the same value in the case of the same channel for the Nth frame to the N+Lth frame, wherein L is preferably a natural number greater than or equal to 1.

(상기 수학식에서, Piir(N)은 N번째 프레임에서의 해당 픽셀의 누적 휘도값, W는 미리 설정된 가중치 및 Pcur(N+1)은 N+1번째 프레임에서의 해당 픽셀의 실제 휘도값을 각각 나타낸다.) 상기 N+1번째 프레임에서의 해당 픽셀의 누적 휘도값을 산출 후, 산출된 상기 N+1번째 프레임에서의 해당 픽셀의 누적 휘도값으로 N+1번째 프레임에서의 해당 픽셀의 실제 휘도값을 대체하는 것에 의해 잡음을 감소시키는 것을 특징으로 한다.In addition, the method of the present invention may include: a channel separation step of receiving the one long exposure channel data and the P short exposure channel data from the CMOS image sensor, and separating and outputting each channel data; a wideband image synthesis step of receiving one long exposure channel data and P short exposure channel data separated and output from the channel separation step and synthesizing them into a wideband image; and a noise attenuating step of receiving a wideband image input from the wideband image synthesis step and reducing noise, wherein the noise attenuating step includes: the cumulative luminance of the corresponding pixel in the N+1th frame using the following equation Calculate the value Piir(N+1),

(In the above equation, Piir(N) is the accumulated luminance value of the corresponding pixel in the Nth frame, W is a preset weight, and Pcur(N+1) is the actual luminance value of the corresponding pixel in the N+1th frame, respectively. After calculating the cumulative luminance value of the corresponding pixel in the N+1th frame, the calculated cumulative luminance value of the corresponding pixel in the N+1th frame is used as the actual luminance of the corresponding pixel in the N+1th frame. It is characterized in that the noise is reduced by substituting the values.

In addition, the weight is preferably set using a difference value between the accumulated luminance value of the corresponding pixel in the Nth frame and the actual luminance value of the corresponding pixel in the N+1th frame. Specifically, the weight is set as a second weight value when the difference between the accumulated luminance value of the corresponding pixel in the Nth frame and the actual luminance value of the corresponding pixel in the N+1th frame is less than a preset first difference value. and when the difference between the accumulated luminance value of the corresponding pixel in the Nth frame and the actual luminance value of the corresponding pixel in the N+1th frame is equal to or greater than a preset first difference value, as the difference increases, the second It may be set to a value gradually decreasing from the weight value.

According to the processing system and the processing method for alleviating banding noise of a CMOS image sensor of the present invention, it is possible to obtain a broadband image with high visibility by reducing a banding noise component generated when acquiring an indoor image in a synchronous lighting environment.

1 is an explanatory diagram illustrating the principle of generation of banding noise.
2 is an exemplary view of an image in which actual banding noise is generated.
3 is a block diagram of a data processing system of a CMOS image sensor capable of mitigating banding noise according to the present invention.
4 is an explanatory view of controlling the operation of the CMOS image sensor by the controller of the present invention.
5 is an exemplary view in which the readout position is shifted by half a period in the next frame;
6 is a control explanatory diagram of a long exposure time and a short exposure time.
Fig. 7 is an explanatory diagram of a wideband image synthesizing method;
Fig. 8 is an explanatory diagram of the operation of the noise attenuator;
9 is an exemplary diagram for a method of setting a weight;
10 is a flowchart of a data processing method of a CMOS image sensor capable of mitigating banding noise;

Hereinafter, a data processing system of a CMOS image sensor capable of mitigating banding noise according to embodiments of the present invention and a processing method thereof will be described in detail with reference to the accompanying drawings. Of course, the following examples of the present invention are not intended to limit or limit the scope of the present invention only to embody the present invention. What an expert in the technical field to which the present invention pertains can easily infer from the detailed description and embodiments of the present invention is construed as belonging to the scope of the present invention.

3 is a block diagram of a data processing system 100 of a CMOS image sensor (CI) capable of mitigating banding noise according to an exemplary embodiment of the present invention.

As can be seen from Fig. 3, the processing system 100 of the present invention comprises a controller 10, a channel separator 20, a wideband image synthesizer 30, a noise attenuator 40, an image signal processor 50 and a memory. (60) may be included.

The controller 10 may be implemented by a processor. In addition, the channel separator 20 , the wideband image synthesizer 30 , and the noise attenuator 40 may be implemented by a circuit or a processor, or a combination of a circuit and a processor. However, the channel separator 20, the wideband image synthesizer 30, and the noise attenuator 40 are preferably implemented by circuits. In addition, the processing system 100 of the present invention may be implemented in the form of a SoC (System on Chip).

The controller 10 generates control signals for controlling the operations of the CMOS image sensor CI, the wideband image synthesizer 30 and the noise attenuator 40 .

For reference, it is assumed that the CMOS image sensor CI adopts a rolling shutter condensing structure in a line interleaved multi channel readout method.

The CMOS image sensor CI preferably outputs data of one long exposure channel and P short exposure channels. That is, the processing system 100 of the present invention receives one long exposure channel data and P short exposure channel data from the CMOS image sensor CI. The P short exposure channel data includes the first short exposure channel data to the Pth short exposure channel data. In addition, P is a natural number of 1 or more.

For example, a CMOS image sensor CI that outputs two channels may include one long exposure channel and one short exposure channel. In addition, in the case of a CMOS image sensor CI that outputs three channels, one long exposure channel and two short exposure channels may be provided.

4 is an explanatory diagram of control of the operation of the CMOS image sensor CI by the controller 10 of the present invention.

As in FIG. 1, in FIG. 4, since a 60Hz fluorescent lamp is used by full-wave rectification, the actual AC power frequency of the externally radiated light becomes 120Hz. In addition, FIG. 4 exemplifies a case in which an internal synchronization signal (vertical sync) of a used camera is 30 FPS (Frame Per Second) progressive scan.

In addition, one frame is configured to include M lines.

In Fig. 4, one long exposure channel and one short exposure channel are cross-controlled for wideband image synthesis. In the long exposure channel, the light sensitivity of banding noise is insignificant, whereas in the short exposure channel, the banding noise is generated. Accordingly, it can be seen that the imaging sensitivity of each line according to the illumination frequency period is different.

In the present invention, in order to offset the FPN (Fixed Pattern Noise) tendency of the banding noise, the readout position is set at a predetermined time interval to change the short exposure time point for the control of the N+1th frame following the Nth frame. Shift as much as possible.

Accordingly, the dimming amount for each line of the short channel for the illumination phase, which is a given light source, in the N+1th frame appears different from the previous Nth frame according to the shift position.

Specifically, the controller 10 sets the readout position of the Yth short exposure channel data for the Kth line of the N+1th frame, and the readout position of the Yth short exposure channel data for the Kth line of the Nth frame. It is preferable to generate a control signal for controlling the operation of the CMOS image sensor CI so as to shift by a predetermined time t(b). Here, each of N and K is a natural number of 1 or more, and Y is a natural number of 1 or more and P or less.

For reference, the readout position refers to a time at which data for a corresponding line is transmitted from the CMOS image sensor (CI) after one long exposure time and P short exposure times are finished, respectively. That is, it means a position from which the data is read in the processing system 100 of the present invention. This readout position can be described in relation to a reference phase of illumination by a full-wave rectified AC power supply. For example, the readout position may be described as a value shifted by a constant value of phase, that is, time from a zero-crossing point where the phase of the illumination is 0 degrees.

Specifically, when the readout position of the Y-th short exposure channel data for the K-th line of the N-th frame is a value shifted by t(a) from the zero-crossing point, the Y-th for the K-th line of the N+1th frame The readout position of the short exposure channel data may be expressed as a value shifted by t(a)+t(b) from the zero crossing point.

Alternatively, the readout position of the Yth short exposure channel data for the Kth line of the N+1th frame is 1/ of the illumination from the readout position of the Yth shortexposure channel data for the Kth line of the Nth frame It may be explained using the period of illumination, such as a value shifted by two periods. A half cycle of illumination has the same meaning as a phase shift of 90 degrees.

For reference, the controller 10 sets the readout position of the Yth short exposure channel data for the 1st line of the N+1th frame from the readout position of the Yth short exposure channel data for the 1st line of the Nth frame. By shifting the constant time, it is possible to shift the lead-out positions for the remaining lines by the same constant time. This is because the readout interval of each line constituting the frame is kept constant.

The controller 10 may set a frame set for shifting the lead-out position, that is, a group to L frames. That is, the N+L-th frame from the N-th frame operates as one set. In addition, it is preferable that the N+2L+1-th frame from the N+L+1-th frame again operates as a set. That is, the controller 10 may set the L frames as one set, that is, a group to control the exposure time and the like in units of sets.

Specifically, the controller 10 generates a control signal such that the shifted degrees of the readout positions in the Nth frame to the N+Lth frame are all equally spaced. Set the maximum shift range of the readout position allowed by the CMOS image sensor (CI), and perform continuous control so that the readout position is equally spaced during one set of predefined Nth frames to N+Lth frames In this case, it is also possible to control the banding noise to appear in a visually flowing form.

That is, the controller 10, for the Nth frame to the N+Lth frame set as one frame set, from one frame of the readout position of the Yth short exposure channel data for the Kth line to the next frame It is preferable to generate a control signal for controlling the operation of the CMOS image sensor CI so that the shift degree is the same. Here, L is a natural number of 1 or more.

For example, the shift degree of the Yth short exposure channel data readout position for the Kth line of the N+1th frame from the readout position of the Yth short exposure channel data for the Kth line of the Nth frame is t( b) Let's do it. In this case, the shift degree of the readout position of the Yth short exposure channel data for the Kth line of the N+2th frame from the readout position of the Yth short exposure channel data for the Kth line of the N+1th frame is also becomes t(b), and from the readout position of the Y-th short exposure channel data for the K-th line of the N+L-1th frame, the Y-th short-exposure channel data for the K-th line of the N+L-th frame is read The shift degree of the out position also becomes t(b).

FIG. 5 is an exemplary view in which the readout position is shifted by half a period in the next frame as shown in FIG. 4 .

By shifting the readout position, it can be seen that the position of the banding noise that can be clearly confirmed in the fluorescent lamp has shifted.

That is, in the present invention, by moving the position of the banding noise for each frame by shifting the readout position, the noise attenuator 40 that attenuates the noise by comparing the front and rear frames can be effectively performed.

The FPN tendency of banding noise can be avoided by defining one set of combination control of the N+L-th frame from the N-th frame and repeating this. In addition, it is possible to visually observe only the short channel portion in the form of flicking.

At this point, it is important to note that the blinking result should only appear positionally. Since the exposure amount should not change in each frame included in a frame set, the exposure amount in each frame must always be the same value, and always to keep it constant.

6 is an explanatory diagram of control of a long exposure time and a short exposure time. Specifically, FIG. 5 is a graph showing a change in the maximum exposure range when the exposure time is changed from 1 to 2.

In case of automatic exposure control, the value at exposure time 1 becomes the exposure value for short exposure, and the value at exposure time 2 becomes the exposure value for long exposure.

That is, the controller 10 calculates each of one long exposure time and P short exposure times for a plurality of frames included in one frame set by an automatic exposure control algorithm, and then the longest exposure time of the smallest value. For each value and P short exposure times, the smallest short exposure time value can be set as the long exposure time and the P short exposure times for the set.

Specifically, the controller 10 needs to manage each value of one long exposure time and P short exposure times of the N+L-th frame from the N-th frame set as one frame set as preset values. That is, the set value for the long exposure time may be set to t(L), and the set value for the short exposure time may be set to t(S_1) to t(S_P), respectively.

In other words, in the CMOS image sensor (CI), P short exposure times for P short exposure channels of the Nth frame to the N+Lth frame set as one frame set and one long exposure channel for one long exposure channel The long exposure time has the same value in the case of the same channel for the Nth frame to the N+Lth frame, respectively, so that the controller 10 generates a control signal.

The channel separator 20 receives one long exposure channel data and P short exposure channel data from the CMOS image sensor CI, and separates and outputs each channel data.

The wideband image synthesizer 30 receives one long exposure channel data and P short exposure channel data separated and output from the channel separator 20 and combines them into a wideband image.

7 is an explanatory diagram of a method for synthesizing a wideband image.

The input frame is long channel (Long Ch), 1st short channel (Short1 Ch), 2nd short channel (Short2 Ch),... It is processed sequentially in the batch order of the scene, and when the last frame of the scene arrives, the pre-stored frame is read and the bit extension and dynamic range extension are performed.

[Equation 1] below is an example of a gain obtained when synthesizing multiple exposures of two sheets, and multiple exposure processing of two or more sheets is possible depending on the intensity of the light source. However, it is required to increase the size of the memory 60 and consider the combined bit rate according to the increase in frames.

Meanwhile, the synthesized bit depth is compressed through a compressed mapping process to match the final output specification of the image (Tone Mapping), and in this process, a tone curve is used to secure visual visibility. apply together

Depending on the result of broadband image synthesis, banding noise or low-contrast image (Low Dynamic Range) across the screen that occurs during normal single-frame shooting can be significantly improved. The fundamental problem of banding noise due to (long exposure) remains insignificant, and banding noise in which only the light-emitting surface is flickered due to the short exposure of the short exposure channel remains.

The noise attenuator 40 receives a wideband image from the wideband image synthesizer 30 and serves to reduce noise.

The noise attenuator 40 preferably uses an IIR (Infinite Impulse Response) filter to reduce noise by the following [Equation 2].

In [Equation 2], Piir(N) is the cumulative luminance value of the corresponding pixel in the Nth frame, Piir(N+1) is the cumulative luminance value of the corresponding pixel in the N+1th frame, and W is a preset weight , Pcur(N+1) represents the actual luminance value of the corresponding pixel in the N+1th frame, respectively.

That is, the noise attenuator 40 calculates the accumulated luminance value of the corresponding pixel in the N+1th frame, and uses the calculated cumulative luminance value of the corresponding pixel in the N+1th frame for the corresponding pixel in the N+1th frame. Reduce the noise by replacing the actual luminance value of

In addition, the weight is set using a difference value between the accumulated luminance value of the corresponding pixel in the Nth frame and the actual luminance value of the corresponding pixel in the N+1th frame.

The noise attenuator 40 is a powerful noise canceller that takes temporal IIR feedback filtering for a single spatial coordinate using the memory 60, and can easily adjust the NR intensity according to the Weight, which is efficient compared to the resources of the Logic Element. While it is a noise filter, it has a weak characteristic to the motion blur effect of a moving subject due to a strong weight.

8 is an explanatory diagram of the operation of the noise attenuator 40. As shown in FIG.

It can be seen that the banding noise appears in the form of flickering over time, and a low pass filter (LPF) effect by the noise attenuator 40 appears.

If the weight of the weight in Equation 2 is increased, that is, if the weight of the feedback is increased, a more smoothed (LPF) result is obtained, and as a result, the banding noise is dispersed to obtain a stable image.

9 shows an exemplary diagram of a method of setting a weight.

In addition, a variable weight may be considered as a method corresponding to the motion blur effect by the noise attenuator 40 . This may use a difference value between the accumulated luminance value of the corresponding pixel in the previous step and the actual luminance value of the current corresponding pixel.

That is, the weight is set as the second weight value when the difference between the accumulated luminance value of the corresponding pixel in the Nth frame and the actual luminance value of the corresponding pixel in the N+1th frame is less than a preset first difference value, , when the difference between the accumulated luminance value of the corresponding pixel in the Nth frame and the actual luminance value of the corresponding pixel in the N+1th frame is equal to or greater than a preset first difference value, the second weight value increases as the difference value increases It can be set to a value that gradually decreases from .

That is, when the difference value is less than the preset first difference value, it is a motion region due to the shift of the banding noise, and when the difference value is equal to or greater than the preset first difference value, it corresponds to a normal motion region.

By additionally applying the weight curve in this way, it is possible to separately apply the weight of each motion area due to the shift of the motion of the general dynamic subject and the banding noise. The banding motion region takes a strongly stabilized image and applies a low weight to the other motion-generating regions to obtain a clear image without dragging.

The image signal processor 50 performs image processing using data output from the noise attenuator 40 .

10 is a flowchart of a data processing method of a CMOS image sensor (CI) capable of mitigating banding noise according to an exemplary embodiment of the present invention.

Since the data processing method of the CMOS image sensor (CI) capable of mitigating banding noise according to the present invention uses the data processing system 100 of the CMOS image sensor (CI) capable of reducing the banding noise described above, a separate It goes without saying that all features of the data processing system 100 of the CMOS image sensor (CI) capable of mitigating banding noise are included even if there is no explanation.

As can be seen from FIG. 10 , the data processing method of the CMOS image sensor CI capable of mitigating banding noise according to the present invention includes a control for generating a control signal for controlling the operation of the CMOS image sensor CI. step (S10); a channel separation step (S20) of receiving one long exposure channel data and P short exposure channel data from the CMOS image sensor (CI), and separately outputting each channel data; a wideband image synthesis step (S30) of receiving one long exposure channel data and P short exposure channel data separated and output from the channel separation step (S20) and synthesizing them into a wideband image; and a noise reduction step (S40) of reducing noise by receiving a wideband image from the wideband image synthesis step (S30).

The CMOS image sensor CI outputs one long exposure channel data and P short exposure channel data. In addition, it is preferable that the P short exposure channel data consists of the first short exposure channel data to the Pth short exposure channel data.

Specifically, in the control step S10, the readout position of the Y-th short exposure channel data for the K-th line of the N+1th frame and the Y-th short exposure channel data for the K-th line of the Nth frame are readout A control signal for controlling the operation of the CMOS image sensor CI may be generated to shift the position from the position by a predetermined time. Each of N, K and P is a natural number of 1 or more, and Y is a natural number of 1 or more and P or less.

In addition, in the control step (S10), from one frame of the readout position of the Y-th short exposure channel data for the K-th line to the next frame for the N-th frame to the N+L-th frame set as one frame set, in the next frame It is preferable to generate a control signal for controlling the operation of the CMOS image sensor CI so that the shift degree of is the same. L is a natural number of 1 or more.

In addition, the control step S10 includes P short exposure times for P short exposure channels and one long exposure time for one long exposure channel of the Nth frame to the N+Lth frame set as one frame set. generates a control signal for controlling the operation of the CMOS image sensor CI to have the same value in the case of the same channel for the Nth frame to the N+Lth frame, respectively.

In the noise reduction step (S40), after calculating Piir(N+1) using [Equation 2], the calculated Piir(N+1) replaces the actual luminance value of the corresponding pixel in the N+1th frame. It is characterized in that the noise is reduced by

In addition, in the noise reduction step (S40), the weight is set using a difference value between the accumulated luminance value of the corresponding pixel in the Nth frame and the actual luminance value of the corresponding pixel in the N+1th frame. .

Specifically, the weight is set as a second weight value when the difference between the accumulated luminance value of the corresponding pixel in the Nth frame and the actual luminance value of the corresponding pixel in the N+1th frame is less than a preset first difference value, , when the difference between the accumulated luminance value of the corresponding pixel in the Nth frame and the actual luminance value of the corresponding pixel in the N+1th frame is equal to or greater than a preset first difference value, the second weight value increases as the difference value increases Preferably, it is set to a value that gradually decreases from .

As described above, according to the data processing system 100 of a CMOS image sensor (CI) capable of mitigating banding noise of the present invention and its processing method, the banding noise component that occurs when acquiring an indoor image in a synchronous lighting environment is reduced It can be seen that a broadband image with high visibility can be obtained by doing this.

100: processing system
10: controller
20: channel separator
30: broadband image synthesizer
40: noise attenuator
50: image signal processor
60 : memory
CI: CMOS image sensor

Claims (12)

delete A system for processing data of the CMOS image sensor by receiving one long exposure channel data and P short exposure channel data from a CMOS image sensor, the system comprising:
Including; a controller for generating a control signal for controlling the operation of the CMOS image sensor;
The controller, for the Nth frame to the N+Lth frame set as one frame set, the degree of shift from one frame to the next frame of the readout position of the Yth short exposure channel data for the Kth line is the same to generate the control signal,
The P short exposure channel data is composed of 1st short exposure channel data to Pth short exposure channel data,
wherein N, K and P are each a natural number of 1 or more,
Y is a natural number that is 1 or more and P or less,
The system, characterized in that L is a natural number of 2 or more. A system for processing data of the CMOS image sensor by receiving one long exposure channel data and P short exposure channel data from a CMOS image sensor, the system comprising:
Including; a controller for generating a control signal for controlling the operation of the CMOS image sensor;
In the controller, P short exposure times for P short exposure channels and one long exposure time for one long exposure channel of the Nth frame to the N+Lth frame set as one frame set are the Nth generating the control signal so as to have the same value in the case of the same channel for the frame to the N+L-th frame,
The P short exposure channel data is composed of 1st short exposure channel data to Pth short exposure channel data,
The system according to claim 1, wherein each of N, P and L is a natural number of 1 or more. 4. according to any one of claims 2 or 3,
The system is
a channel separator that receives the one long exposure channel data and the P short exposure channel data from the CMOS image sensor, and separates and outputs each channel data;
a wideband image synthesizer for receiving one long exposure channel data and P short exposure channel data separated and output from the channel separator and synthesizing them into a wideband image; and
A noise attenuator for reducing noise by receiving a wideband image from the wideband image synthesizer;
The noise attenuator calculates the accumulated luminance value Piir(N+1) of the corresponding pixel in the N+1th frame by using the following equation,

(In the above equation, Piir(N) is the accumulated luminance value of the corresponding pixel in the Nth frame, W is a preset weight, and Pcur(N+1) is the actual luminance value of the corresponding pixel in the N+1th frame, respectively. indicate.)
After calculating the cumulative luminance value of the corresponding pixel in the N+1th frame, the calculated cumulative luminance value of the corresponding pixel in the N+1th frame is replaced with the actual luminance value of the corresponding pixel in the N+1th frame A system characterized in that the noise is reduced by 5. The method of claim 4,
The weight is
The system, characterized in that it is set using a difference value between the accumulated luminance value of the corresponding pixel in the Nth frame and the actual luminance value of the corresponding pixel in the N+1th frame. 6. The method of claim 5,
The weight is
When the difference between the accumulated luminance value of the corresponding pixel in the Nth frame and the actual luminance value of the corresponding pixel in the N+1th frame is less than a preset first difference value, it is set as a second weight value,
When the difference between the accumulated luminance value of the corresponding pixel in the Nth frame and the actual luminance value of the corresponding pixel in the N+1th frame is greater than or equal to a preset first difference value, the second weight value increases as the difference value increases A system characterized in that it is set to a value that gradually decreases from . delete A method for processing data of the CMOS image sensor by receiving one long exposure channel data and P short exposure channel data from a CMOS image sensor, the method comprising:
A control step of generating a control signal for controlling the operation of the CMOS image sensor;
In the control step, for the Nth frame to the N+Lth frame set as one frame set, the degree of shift from one frame to the next frame of the readout position of the Yth short exposure channel data for the Kth line is To be the same, generating the control signal,
The P short exposure channel data is composed of 1st short exposure channel data to Pth short exposure channel data,
wherein N, K and P are each a natural number of 1 or more,
Y is a natural number that is 1 or more and P or less,
Wherein L is a natural number of 2 or more. A method for processing data of the CMOS image sensor by receiving one long exposure channel data and P short exposure channel data from a CMOS image sensor, the method comprising:
A control step of generating a control signal for controlling the operation of the CMOS image sensor;
In the control step, P short exposure times for P short exposure channels and one long exposure time for one long exposure channel of the Nth frame to the N+Lth frame set as one frame set are N generating the control signal so as to have the same value in the case of the same channel for the th frame to the N+L th frame,
The P short exposure channel data is composed of 1st short exposure channel data to Pth short exposure channel data,
Each of N, P, and L is a natural number of 1 or more. 10. The method of any one of claims 8 or 9,
The method is
a channel separation step of receiving the one long exposure channel data and the P short exposure channel data from the CMOS image sensor, and separately outputting each channel data;
a wideband image synthesis step of receiving one long exposure channel data and P short exposure channel data separated and output from the channel separation step and synthesizing them into a wideband image; and
Further comprising; a noise attenuation step of reducing noise by receiving a wideband image from the wideband image synthesis step;
In the noise reduction step, the cumulative luminance value Piir(N+1) of the corresponding pixel in the N+1th frame is calculated using the following equation,

(In the above equation, Piir(N) is the accumulated luminance value of the corresponding pixel in the Nth frame, W is a preset weight, and Pcur(N+1) is the actual luminance value of the corresponding pixel in the N+1th frame, respectively. indicate.)
After calculating the cumulative luminance value of the corresponding pixel in the N+1th frame, the calculated cumulative luminance value of the corresponding pixel in the N+1th frame is replaced with the actual luminance value of the corresponding pixel in the N+1th frame A method characterized in that the noise is reduced by 11. The method of claim 10,
The weight is
The method of claim 1, wherein the value is set using a difference between the accumulated luminance value of the corresponding pixel in the Nth frame and the actual luminance value of the corresponding pixel in the N+1th frame. 12. The method of claim 11,
The weight is
When the difference between the accumulated luminance value of the corresponding pixel in the Nth frame and the actual luminance value of the corresponding pixel in the N+1th frame is less than a preset first difference value, it is set as a second weight value,
When the difference between the accumulated luminance value of the corresponding pixel in the Nth frame and the actual luminance value of the corresponding pixel in the N+1th frame is greater than or equal to a preset first difference value, the second weight value increases as the difference value increases A method characterized in that it is set to a value that gradually decreases from .

Images