KR101521458B1 - Apparatus and method for removing smear - Google Patents

Abstract

The present invention relates to an apparatus capable of effectively removing a smear generated in an image through detecting the smear generated in the image obtained by a CCD sensor and post-processing that of removing the smear and restoring to the original image, and a removing method thereof. The apparatus for removing a smear according to an embodiment of the present invention comprises: an image input unit for receiving an image obtained by a CCD sensor; a detection unit for detecting a first column position where a smear is generated among columns composing the image when the smear is generated in the image inputted in the image input unit; and a removal unit for removing the smear from the image based on the detected first column position.

Description

[0001] APPARATUS AND METHOD FOR REMOVING SMEAR [0002]

The present invention relates to an apparatus and method for removing smears generated in an image, and more particularly, to an apparatus and method for removing smears generated in an image obtained by a CCD sensor, .

Conventionally, a film is used in photographing, but an image sensor capable of replacing a film has been developed and an optical image is obtained through an image sensor. At this time, the image sensor means an element which converts an optical image into an electric signal.

An image sensor is formed by arranging a plurality of pixels in a two-dimensional structure. Each pixel of the image sensor converts it into an electrical signal according to the brightness of the light. By measuring the electrical signal, the amount of light entering each pixel can be known. The amount of light can be converted into an electrical signal to form an image of a pixel unit. Thus, an electrical signal corresponding to the amount of light can be expressed numerically, which is called a pixel value. Here, each pixel value may be represented by a value between 0 and 255. [

Such an image sensor can be classified into a charge coupled device (CCD) type image sensor and a complementary MOS (CMOS) type according to the way in which electrons and holes are transmitted to an output unit. ) Type image sensors are widely used.

The CCD type image sensor and the CMOS type image sensor commonly have a light receiving section that receives light and converts it into an electric signal. The CCD type image sensor transmits the electric signal through the CCD and converts it to the voltage at the last stage. On the other hand, the CMOS image sensor converts the signal to a voltage at each pixel and transfers it to the outside.

That is, the CCD type image sensor moves the electrons generated by the light directly to the output part by using the gate pulse, and the CMOS type image sensor converts the electrons generated by the light into voltage in each pixel, There is a difference in output through the switch.

In the current CMOS image sensor, a CCD image sensor is widely used in comparison with a CCD image sensor due to its relative advantages including mass production. However, CCD image sensors have been widely used due to the problem that the signal is distorted due to noise, CCD type image sensors are widely used.

However, a smear phenomenon may occur in a CCD image sensor depending on the signal processing method. The smear phenomenon refers to a phenomenon in which a line of vertical lines appears on the screen when a strong reflected light of a light source or an illumination lamp is photographed. It is often seen when using high-speed shutter and is often seen when shooting very bright objects such as light sources. The CCD type image sensor has a structure in which only one light is present in one cell. When a charge that can be stored in one cell overflows due to reflection phenomenon and interference phenomenon between cells, a smear phenomenon occurs.

The smear phenomenon is easily generated in the buffer area for storing or transmitting to the image sensor according to the exposure of the light in the high-speed shutter setting. The high-speed shutter of the CCD adjusts the exposure by the exposure time of the CCD through the shutter of the camera body and by directly controlling the CCD at a shutter speed higher than the synchronization speed. If the shutter of the camera body is opened when acquiring an image using the electronic shutter of the CCD, the light continues to be incident on the photodiode and the charge is overflowed in the stored space. If the charge of the CCD composed of the longitudinal array is read, And a smear phenomenon is generated.

The smear phenomenon thus generated causes distortion of photographed images, detection of a vehicle type in a vehicle detecting or interrupting system, and trouble of obstructing the identification of a vehicle number. Referring to FIG. 1, in a road environment, a chromatic, a glass, a coated surface, an emblem or the like of a vehicle is generated in a region having a shimmer, and the width varies depending on the area of the reflection surface.

Currently, the cause of the smear phenomenon has been revealed and various solutions have been proposed.

As an example of a solution, a method of using a mechanical shutter of a camera is proposed for a single image captured by a camera, and a method of grouping pixels at the boundary of a CCD image sensor using an optical block region (OBR) (Grouping) method is also proposed.

However, this method has a problem that the complexity increases in structure or the cost increases rapidly.

Therefore, it is required to detect and remove the smear on the image in the post-processing process using the image processing rather than the sensor level unit.

Korean Patent Application Publication No. 1990-0019481

SUMMARY OF THE INVENTION Accordingly, the present invention has been made to solve the above-mentioned problems occurring in the prior art, and it is an object of the present invention to provide a method and apparatus for detecting smears generated in an image obtained by a CCD sensor, And an object thereof is to provide a user with a device that can be effectively removed and a method of removing the device.

Specifically, the present invention eliminates the smear phenomenon through post-processing using image processing rather than a sensor level unit, thereby eliminating image smear elimination apparatus that can be implemented at a low cost and efficient operation, And the purpose of providing the removal method to the user.

It is another object of the present invention to provide a user with an image smear removal device and a removal method thereof, which can be widely applied to an image pickup device using a CCD image sensor and have a wide range of application fields.

It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are not intended to limit the invention to the precise form disclosed. It can be understood.

According to another aspect of the present invention, there is provided a smear removal apparatus including: a video input unit receiving an image acquired by a CCD sensor; A detector for detecting a position of a first column in which the smear is generated among columns constituting the image when a smear is generated in the image input to the image input unit; And removing the smear from the image based on the detected position information of the first column.

The detecting unit may include an extracting unit for extracting a signal distribution curve for each column constituting the image using the image input to the image input unit; And a conversion unit converting the signal distribution curve into a normal distribution curve, wherein the signal distribution curve represents a sum of gray values of a plurality of pixels constituting each column constituting the image.

The detection unit may generate a binary pattern map by comparing the normal distribution curve with a preset threshold value. In a region where the normal distribution curve is smaller than the threshold value, the binary pattern map has a value of 0, And the binary pattern map has a value of 1 in an area larger than the threshold value.

In addition, a region having the value of 1 in the binary pattern map corresponds to the first column of the image.

에 따라 결정될 수 있다. 상기 수학식에서, 상기

는 상기 임계치이고, 상기

는 상기 신호분포 곡선의 평균이며, 상기

는 가중치이고, 상기

는 상기 신호분포 곡선의 표준편차이다.Further,

≪ / RTI > In the above equation,

Is the above-mentioned threshold value,

Is an average of the signal distribution curves,

Is a weight,

Is the standard deviation of the signal distribution curve.

In addition, the detection section may extend the width of the first column by adding a part of the column adjacent to the first column to the first column.

The elimination unit may subtract a gray value of each of a plurality of pixels constituting the binary pattern map from a gray value of each of a plurality of pixels constituting the image, Can be removed.

The image processing apparatus may further include a reconstruction unit that reconstructs the image of the first row from which the smear is removed by using a predetermined interpolation method.

The interpolation method of the restoration unit may include calculating a priority for each of a plurality of pixels in the patch and determining a highest priority pixel having the highest priority among the calculated priorities, Wherein the patch comprises at least a portion of the pixels constituting the first column, and wherein the patch comprises at least a portion of the pixels constituting the first column.

Also, the restoring unit applies the interpolation method until restoration is performed on all the pixels constituting the first column.

Also, the priority may be calculated according to the following equation.

는 픽셀

의 우선순위이고, 상기

는 픽셀의 신뢰도이며, 상기

는 상기 패치이고, 상기

는 상기 패치의 크기이며, 상기

는 입력된 이미지이고, 상기

는 복원할 영역이며, 상기

는 구조체의 방향 단위 벡터이고, 상기

는 픽셀

에서의 경계(contour)에 대한 법선 단위 벡터이며, 상기

는 정규화 상수(normalization constant)이다.In the above equation,

Gt;

Is the priority of

Is the reliability of the pixel,

Is the patch,

Is the size of the patch,

Is an input image,

Is an area to be restored,

Is the direction unit vector of the structure,

Gt;

Is a normal unit vector for the contour in

Is a normalization constant.

According to another aspect of the present invention, there is provided a smear removal method including: a first step of receiving an image acquired by a CCD sensor; A second step of, when a smear is generated in the input image, detecting a position of the first column in which the smear is generated among columns constituting the image; And a third step of removing the smear from the image based on the detected position information of the first column.

The second step may include: extracting a signal distribution curve for each column constituting the image using the input image; Transforming the signal distribution curve into a normal distribution curve; And generating a binary pattern map by comparing the normal distribution curve with a preset threshold value, wherein the signal distribution curve represents a sum of gray values of a plurality of pixels constituting each column constituting the image, , The binary pattern map has a value of 0 in an area where the normal distribution curve is smaller than the threshold value, and the binary pattern map has a value of 1 in an area where the normal distribution curve is larger than the threshold value.

And a fourth step of restoring an image of the first column in which the smear is removed by using a predetermined interpolation method, and the fourth step is a step of calculating a priority for each of the plurality of pixels in the patch step; Determining a highest priority pixel having the highest priority among the calculated priorities; And performing restoration through comparison of the similarity between the highest priority pixel and a pixel not constituting the first column among the plurality of pixels in the patch, As shown in FIG.

The present invention relates to a device capable of effectively removing smear generated in an image through a post-processing process of detecting smears generated in an image acquired by a CCD sensor and restoring the smear to an original image after removing the smear, can do.

Specifically, the present invention eliminates the smear phenomenon through post-processing using image processing rather than a sensor level unit, thereby eliminating image smear elimination apparatus that can be implemented at a low cost and efficient operation, The user can be provided with the removal method.

In addition, the present invention can provide a user with an image smear removal device and a removal method thereof, which can be widely applied to an image pickup device using a CCD image sensor and have a wide range of application fields.

SUMMARY OF THE INVENTION The present invention has been conceived to solve the above-mentioned problems, and it is an object of the present invention to provide a method and apparatus for detecting smear through image processing of an image obtained through a CCD and removing detected smear, Apparatus and method to a user.

It should be understood, however, that the effects obtained by the present invention are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those skilled in the art to which the present invention belongs It will be possible.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate a preferred embodiment of the invention and, together with the description, serve to provide a further understanding of the technical idea of the invention, It should not be construed as limited.
FIG. 1 shows an example of a situation where smear is generated on the CCTV.
FIG. 2 is a photograph of an example of smear generated in an image obtained in CCTV.
Fig. 3 shows an example of a block diagram of a smear removal device applicable to the present invention.
Figure 4 is a flow diagram of a method for detecting and removing smear in an image associated with an example of the present invention to reconstruct an image.
5 shows an example of a signal distribution curve according to each column of the image obtained in the image pickup device.
FIG. 6 shows a normal distribution curve obtained by taking the signal distribution curve of FIG. 5 as an input.
7A is an image in which smear is generated in the first experiment.
Fig. 7B is a signal distribution curve relating to the image in the first experiment.
7C is a normal distribution curve for the signal distribution curve in the first experiment.
FIG. 7D is a binary pattern map based on the threshold value in the normal distribution curve in the first experiment. FIG.
7E is a smear region detected using the binary pattern map in the first experiment.
7F is an image reconstructed by removing the smear and applying an inpainting technique in the first experiment.
8A is an image in which a smear is generated in the second experiment.
8B is a signal distribution curve for an image in the second experiment.
8C is a normal distribution curve for the signal distribution curve in the second experiment.
FIG. 8D is a binary pattern map based on the threshold value in the normal distribution curve in the second experiment. FIG.
8E is a smear region detected using the binary pattern map in the second experiment.
FIG. 8F is an image reconstructed by removing the smear and applying an inpainting technique in the second experiment.
9A is an image in which smear is generated in the third experiment.
FIG. 9B is a signal distribution curve for an image in the third experiment. FIG.
Figure 9c is a normal distribution curve for the signal distribution curve in the third experiment.
FIG. 9D is a binary pattern map based on the threshold value in the normal distribution curve in the third experiment. FIG.
9E is a smear region detected using the binary pattern map in the third experiment.
9F is an image reconstructed by removing the smear and applying an inpainting technique in the third experiment.
10A is an image in which a smear is generated in the fourth experiment.
FIG. 10B is a signal distribution curve for an image in the fourth experiment. FIG.
10C is a normal distribution curve for the signal distribution curve in the fourth experiment.
FIG. 10D is a binary pattern map based on the threshold value in the normal distribution curve in the fourth experiment. FIG.
10E is a smear region detected using the binary pattern map in the fourth experiment.
FIG. 10F is an image reconstructed by removing the smear and applying an inpainting technique in the fourth experiment.

Hereinafter, a preferred embodiment of the present invention will be described with reference to the drawings. It should be noted that the embodiments described below do not unduly limit the contents of the present invention described in the claims, and the entire constitution described in this embodiment is not necessarily essential as a means for solving the present invention.

In recent years, images are often acquired using image sensors instead of conventional films. The image sensor is largely classified into a CMOS method and a CCD method. Conventionally, a CCD image sensor has been widely used since it shows superiority in low-noise and low-illuminance photographing.

In particular, such CCD image sensors have been widely used in the unmanned control system using CCTV, such as speeding vehicle enforcement, signaling vehicle enforcement, illegal parking, and so on.

However, such a CCD image sensor has a problem that a smear phenomenon occurs, and various methods have been studied to solve this problem.

As an example of a solution, a method of using a mechanical shutter of a camera is proposed for a single image captured by a camera, and a method of grouping pixels at the boundary of a CCD image sensor using an optical block region (OBR) (Grouping) method is also proposed.

However, according to such a method, there arises a problem that the structure becomes complicated or the cost increases rapidly. Particularly, there is a problem that the hardware can not be replaced in the case of CCTV installed in the past, and therefore, there is a demand for a method of detecting and removing smear on an image using image processing.

Hereinafter, a method for detecting and removing smear on an image using the image processing to be proposed by the present invention will be described.

<Removal order of smear in image>

Although the present invention describes a method for detecting and removing smear based on CCTV, the scope of right of the present invention is not limited to the CCTV, and the scope of the present invention is not limited to CCTV, but may be widely applied to digital cameras and imaging devices using CCD image sensors It is obvious that it can be applied.

FIG. 1 shows an example of a situation where smear is generated on a CCTV, and FIG. 2 is a photograph of an example of smear generated on an image obtained in a CCTV.

Referring to FIGS. 1 and 2, the smear phenomenon occurs frequently when strong reflected light such as a light source or illumination is photographed using an image pickup device equipped with a CD image sensor. Light is especially good in strong summer days, and occurs in places where reflection is good, such as the top of a vehicle or chrome plated parts.

Further, when the position where the smear is generated is near the number plate of the vehicle or occurs on the number, the number of the number plate may be unrecognized or mistakenly recognized, so that the effect of unattended interrupting may be reduced.

Accordingly, in order to solve the above problem, a method of removing smear and restoring an image through an image processing process will be described.

Fig. 3 shows an example of a block diagram of a smear removal device applicable to the present invention. Referring to FIG. 3, the smear removal apparatus 100 of the present invention may include an image input unit 10, a detection unit 20, a removal unit 30, and a restoration unit 40.

However, the components shown in Fig. 3 are not essential, so that a smear removal device 100 having more or fewer components may be implemented. Also, it is possible that the components shown in FIG. 1 are connected to each other in an interdependent manner, and that the components are separately or integrally implemented as shown in FIG.

The image input unit 10 is connected to an image pickup element, and receives an image or an image captured by the image pickup element. Here, the image pickup device will be limited to equipment capable of acquiring images or images using a CCD sensor.

The detection unit 20 determines whether a smear is generated in the image input to the image input unit 10. When it is determined that a smear has occurred, the detection unit 20 can detect the position of the smear generated column (first column).

The detecting unit 20 may further include an extracting unit and a converting unit. The extraction unit extracts signal distribution curves for the respective columns constituting the image using the image input to the image input unit 10. [ The converting unit converts the signal distribution curve generated by the extracting unit into a normal distribution curve.

The removal unit 30 may remove the smear generated in the image based on the position information of the first column detected by the detection unit 20. [

The restoring unit 40 may restore the original image of the first column from which the smear is removed by using an interpolation method based on the priority of the patch.

More specifically, the restoration unit 40 calculates a priority for each of the plurality of pixels in the patch, determines the highest priority pixel having the highest priority among the calculated priorities, The restoration can be performed through comparison of the similarity of the pixels that do not constitute the first column among the plurality of pixels in the image.

4 is a flowchart illustrating a method of detecting and removing smear in an image related to an example of the present invention, and FIG. 5 is a view showing an example of a signal distribution curve according to each column of an image obtained in the image pickup device And FIG. 6 shows a normal distribution curve obtained by taking the signal distribution curve of FIG. 5 as an input.

Referring to FIG. 3, the image input unit 10 starts a method of receiving an image from a predetermined image pickup device (S110). As described above, the image pickup device is limited to equipment using a CCD image sensor. CMOS image sensors are not structurally smearing.

After inputting the image from the image pickup device, the step of analyzing the input image statically (S120) is proceeded. As shown in FIG. 5, the extracting unit of the detecting unit 20 may curve the input image into a signal distribution for a column unit signal. The signal distribution curve represents the sum of the gray values of the plurality of pixels constituting each column constituting the image.

6, the converting unit of the detecting unit 20 may convert a signal distribution curve related to the input image into a normal distribution curve. That is, when the smear is generated at a specific place from the source of sunlight or passive light by the vehicle in general, it can be expressed as a normal distribution.

After the image is expressed by the normal distribution, the step of determining whether or not the smear exists is performed (S130). It can be concluded that the smear is generated in columns in the image due to the characteristics of the smear, and particularly in the white and bright forms, and in the white and bright forms.

Thus, the sum of the gray values along the direction of the distribution generated in the smear and other sections in the signal distribution curve and the value of the maximum estimate for the sum of the column distribution curve, i.e., the blurring, In the curve, when a portion having a specific and significantly higher frequency than other portions in the normal distribution exists, it can be determined that the smear is generated in the image.

After the presence of the smear in the image is determined, the step of determining the position of the smear is performed (S140). In step S140, it is possible to determine a location where the smear is generated by determining a portion having a specific and significantly higher frequency than the other portions in the signal distribution curve of the image as a section in which the smear occurs.

It is determined that the smear region exists, and after the position is determined, smear is removed and a binary pattern map (Alpha map) for restoration is generated (S150).

The smear intensity and the exact background intensity are estimated to remove the smear. A binary pattern map is generated by applying an average filter to the signal strength in each column of the image.

In this case, the binary pattern map has a value of 1 when the signal intensity on the normal distribution is larger than a predetermined threshold value, and the column has a value of 1, and the column has a value of 0 when the signal intensity is small.

After the binary pattern map is generated, the smear position is rearranged (S160) using the binary pattern map. It consists of vehicle, noise, background, and smear when analyzing each pixel station. The intensity of the smear signal is estimated by reconstructing the smear region search size in order to align the gray values of the pixels in the column using the applied filter, and the accurate position is determined and rearranged.

After the smear position is rearranged, smear is removed (S170). Removing the smear may be removed from the entire image using the smear region and intensity determined in step S160.

After the smear is removed, the image is restored (S180). There are various methods of restoring an image, but in the case of the present invention, inpainting is applied. In particular, although the image can be reconstructed using the interpolation method, since the image is not suitable for the region, the image can be reconstructed using a patch having a predetermined size in the periphery.

&Lt; Modeling on noise of smear phenomenon >

Hereinafter, modeling for smear will be described in order to remove the smear contained in the image obtained from the CCD image sensor.

First, the number of photons input for the predetermined exposure interval in the CCD image sensor is recognized as shown in Equation (1).

는 노출간격이고,

는 노출시간동안 CCD 이미지 센서에 입력된 광자의 총 수이며,

는 비례상수이고,

는 복사강도이다.here

Is the exposure interval,

Is the total number of photons input to the CCD image sensor during the exposure time,

Is a proportional constant,

Is the radiation intensity.

Further, in the case where the smear phenomenon occurs, the number of photons generated during the time of transferring the photoelectrons in one row can be recognized as shown in Equation (2).

는 생성된 광자의 수이고,

는 다른 하나의 열에서 광전자를 전송하는 시간을 의미하며,

는 광자 수의 비율이다. 노출시간의 누적과 광전가 전송에 따라 생성된 스미어 현상은

인 때 발생된다.here,

Is the number of generated photons,

Means the time for transferring photoelectrons from one column to another,

Is the ratio of the number of photons. The accumulation of exposure time and the smear phenomenon caused by photoelectric transfer

.

의 수는 수학식 3에 나타난 것과 같다.Therefore, in the smear phenomenon,

Is expressed by Equation (3).

가 광전자 전송의 요인이라 하면, 픽셀에 대한 빛의 강도(Intensity)

는 수학식 4에 나타난 바와 같다.if

Is a factor of the photoelectric transfer, the intensity of the light to the pixel,

Is expressed by Equation (4).

If the value of Equation (3) is substituted into Equation (4), it can be expressed as Equation (5) below.

는 빛에 대한 누수 효과가 없는 원래의 강도이고,

는 스미어 현상에 발생된 강도이다.here,

Is the original strength without leakage effect on light,

Is the intensity generated in the smear phenomenon.

는 동일한 열(column) 픽셀 강도를 합해서 얻어진 배경 강도

를 이용하여 추정할 수 있다.According to Equation (5), the intensity of the smear phenomenon

The background intensity obtained by adding the same column pixel intensities

. &Lt; / RTI &gt;

Through the above-described modeling, it is possible to estimate and determine the presence of smear generated in the CCD image sensor.

<Generation of Signal Distribution Curve of Image through Normal Distribution>

Hereinafter, a description will be given of a method of determining whether smear is included in an image obtained by a CCD image sensor and generating a signal distribution curve of an image through a normal distribution as a previous step for removing smear when included. do.

In the present specification, the present invention will be described with reference to a still image of a vehicle from an image of a CCD sensor due to the frequency of sunlight for smear detection.

Assuming no noise, the response of the CCD pixel is as shown in Equation (6).

는 하나의 CCD 픽셀로 그레이(Gray) 값을 갖는 확률밀도함수(Probability density function, PDF)이고,

는 태양광의 빈도성분에 대한 CCD 픽셀,

는 생성된 차량의 순수한 CCD 픽셀이다.

는 모든 픽셀들의 그레이 값(gray value)에 대한 PDF,

은 차량의 모든 픽셀들의 그레이 값을 나타낸다.here,

Is a probability density function (PDF) having a Gray value as one CCD pixel,

A CCD pixel for the frequency component of sunlight,

Is a pure CCD pixel of the generated vehicle.

Is a PDF for the gray value of all pixels,

Represents the gray value of all the pixels of the vehicle.

Generally, it can be expressed as a normal distribution if it is generated at a specific place from the source of sunlight or passive light by the vehicle.

는 수학식 7로 나타낼 수 있다.In order to express the normal distribution, by concretely summarizing the above parameters,

Can be expressed by Equation (7).

는 차량에 대한 예외적인 값(the exception of Vehicle),

는 분산을 의미한다.here,

Is the exception of the vehicle,

Means dispersion.

는 수학식 8과 같이 표현될 수 있다. CCD 픽셀에서 차량의 빛 발생은 시스템에서 강한 빛으로부터 생성되므로, 상대적으로 잡음은 무시할 수 있기 때문에, 2차원 가우시안 곡선에 태양광에 대한 픽셀의 그레이(gray)로 표현된다.And,

Can be expressed by Equation (8). Since the light emission of the vehicle in the CCD pixel is generated from strong light in the system, it is represented by the gray of the pixel for the sunlight in the two-dimensional Gaussian curve, since the noise is relatively negligible.

는 픽셀

에 대한 그레이 값(gray value),

와

는 차량의 중심,

는 CCD 센서 영상에 대한 차량의 진폭크기,

는 확산반경을 의미한다.here,

Gt;

A gray value for a pixel,

Wow

The center of the vehicle,

The amplitude magnitude of the vehicle with respect to the CCD sensor image,

Is the diffusion radius.

을 수학식 9와 같이 나타낼 수 있다.Using Equation (8), the center pixel distance of the vehicle light source can be considered as energy for weak light. For the noise present at this time, the average value

Can be expressed by Equation (9).

은 차량반경률,

는 확산반경을 의미한다.here

The vehicle radius,

Is the diffusion radius.

가 차량 반경률

보다 큰경우 (

) 분포

로 표현된다.At this time, ignoring the quantization error,

The vehicle radius ratio

Is greater than

) Distribution

Lt; / RTI &gt;

열(column)의 픽셀 값에 대한 그레이 값 PDF인

는 수학식 10과 같이 나타낼 수 있다.so,

The gray value for the pixel value of the column.

Can be expressed by Equation (10).

는 차량 광원에서 광자를 받지 않은 경우에 대한 PDF이다. here,

Is a PDF for the case where no photons are received from the vehicle light source.

열에 있는 그레이 값도 포함한다. At this time, since there is actually noise, when the noise also satisfies the normal distribution, equation (10)

It also contains the gray value in the column.

Also, when estimating the background noise, the accuracy is poor unless the light source distribution of the vehicle is considered. This can be seen as an area with a higher gray value than the background components in the entire image.

In this way, a signal distribution curve of the image through the normal distribution can be generated.

<Smear Zone Detection>

Hereinafter, a method for detecting a smear region in an image in which smears are generated will be discussed.

Since the smear is generated only in the image obtained from the CCD image sensor as described above and is generated only in the column, it is a main method to find the smear occurrence column in the image image.

However, when a smear is generated in one image, it may occur in a plurality of areas, not only in one place, so it is very difficult to find an accurate position.

In the present invention, in order to detect a smear region, it is determined that a smear region is present for white and bright columns transmitted to the supersaturated object, and the gray characteristic is analyzed and detected.

Specifically, we use the sum of gray values and the distribution curve of the column along the direction of the distribution not only in smear but also in other sections. In the curve, the other areas show relatively similar components or even distributions as the background, vehicle components.

However, unlike other regions, there is a property having a remarkably high maximum estimated value in the section where the smear occurs.

Therefore, in order to detect the smear occurrence period, the values of the maximum estimated value (Peak value) with respect to the sum of blurring are searched and if the value of the maximum estimate is significantly higher than other regions of the thermal distribution curve, Can be assumed to have occurred.

Equation (11) means a smear brightness intensity estimation curve model.

는 원 영상의 크기이고,

과

는 각각 행과 열을 나타낸다.

는

열에서 그레이 값이고,

는

내에서 픽셀에 대한 그레이 값을 나타낸다. here

Is the size of the original image,

and

Denote rows and columns, respectively.

The

The gray value in the column,

The

&Lt; / RTI &gt; represents a gray value for a pixel within a pixel.

는 수학식 12에 나타낸 바와 같다.On the other hand, the threshold value set for the smear region is

Is expressed by Equation (12).

는 전체 영상에서 통계적 분석에 따른 평균(mean)이고,

는 표준편차이며,

는 가중치를 의미한다.here

Is the mean according to the statistical analysis in the whole image,

Is the standard deviation,

Means a weight.

는, 수학식 13으로 나타낼 수 있다.Regarding the correlation between the equations (11) and (12), it can be seen that the smear phenomenon occurs at the highest estimated value among the curves of the gray sum, as described above.

Can be expressed by Equation (13).

That is, when the smear brightness intensity is higher than the threshold value, the smear region is determined.

과 관련된, 시작 좌표

과 끝 좌표

을 얻을 수 있다. After finding the smear position coordinates,

Associated with the start coordinates

And end coordinates

Can be obtained.

However, it is possible to catch the smear occurrence region in an area wider than the start coordinates and the end coordinates thus obtained. For example, as shown in Equation (14) below, it is possible to determine the region of the smear occurrence to be 2 pixels before the start coordinate and 2 pixels after the end coordinate.

과

는 스미어 영역에 대한 결과를 의미한다.
here,

and

Means the result for the smear zone.

<Estimation and removal of smear signal intensity>

Hereinafter, a method of detecting and eliminating smear by estimating the intensity of smear and the intensity of background in a state in which smear is removed in an image in which smear is generated will be examined.

Apply a mean filter to the signal strength for each column and use the applied filter to reconstruct the smear region search size to align the gray values of the pixels in the column.

Equation (15) is an expression relating to an average filter applied to the signal strength of each column.

는 검색 크기에서 좌표에 대응하는 영상 데이터 중간위치이고,

는 선택된 위치 영역의 반경(radius),

'는 선택된 검색크기에서 정렬된 벡터이다.here,

Is an image data intermediate position corresponding to coordinates in the search size,

Is the radius of the selected location area,

'Is an ordered vector of the selected search size.

In the case of Equation (15), when analyzing each pixel column, the gray values of the pixel are sorted including the vehicle, noise, background and smear signal.

스미어와

배경성분의 강도를 정확하게 얻을 수 있다. According to the above method,

Smear and

The intensity of the background component can be accurately obtained.

It is possible to estimate the smear intensity, determine the difference from the pure background intensity, and determine the position and area of the smear.

After determining the intensity and area of the smear, the smear is removed from the entire image.

은 스미어가 제거된

번째 영상을 의미한다.here,

The smear was removed

.

를 제거하는 것이다. In other words,

.

Through such a method, it is possible to detect smear in the image including the smear, to grasp the position of the smear, and to remove the smear from the image.

<Restoration of removed smear area image>

In the following, after the smear region is removed from the image, a method of restoring the image for the relevant portion will be discussed.

In general, in the image processing, a method of removing noise and restoring an image is to restore an image that is not distorted from a binary pattern map (alpha map). In particular, among various restoration methods, ) Method is applied.

A common image restoration method is interpolation. This is simply applicable if the missing area is not large, but there is an inadequate interpolation method for the area because the error propagates. To improve this, a patch method having a predetermined size is used.

The patch method involves the following steps: i) calculating the priority of the patch, ii) using a similar texture for the highest priority patch, iii) following the method of priority recalculation, Until all of the masked areas are interpolated.

Priority calculation of patches is a calculation of the priority for pixels on the boundary of the area to be interpolated in the calculation step, and priority is very important for correct texture and structure interpolation.

의 우선순위를 의미 하는

는,The priority of the patch is determined by the product of the confidence of the center point of the patch centered at the pixel on the boundary and the value associated with the structure in the patch,

Means the priority of

Quot;

. &Lt; / RTI &gt;

는 패치내의 픽셀에서의 신뢰도 값,

는 보간 해야 할 영역을 나타낸다.

는 패치를 의미하고

는 패치의 크기,

는 영상내 구조체의 방향 단위 벡터,

는 픽셀

에서의 경계(contour)에 대한 법선 단위 벡터를 나타낸다.

는 정규화 상수(normalization constant)이고, 픽셀의 신뢰도는 소스영역(S)에 포함되는 픽셀에 대해서는 1, 그 외에는 0으로 정하였다. here,

Is the confidence value at the pixel in the patch,

Represents an area to be interpolated.

Means a patch

The size of the patch,

Is the direction unit vector of the structure in the image,

Gt;

The unit vector of the normal to the contour in the image.

Is the normalization constant, and the reliability of the pixel is set to 1 for pixels included in the source region S, and to 0 otherwise.

는 영상 내 구조체의 방향과 일치하는 방향의 픽셀에게 더 높은 우선순위를 부여함으로써 구조체를 더 올바르게 복원 할 수 있도록 하는 값이며, 주변에 신뢰도가 높은 픽셀이 많이 분포하는 위치부터 보간하게 함으로써 보다 정확한 보간이 가능하도록 하는 우선순위 값을 의미한다. In other words,

Is a value that allows a structure to be restored more correctly by giving a higher priority to pixels in a direction coinciding with the direction of the structure in the image. By interpolating from a position where many reliable pixels are distributed in the periphery, This is a priority value that enables the operation of the system.

A patch having the highest priority can be calculated through the above equation.

After calculating a patch having a highest priority, a pixel having a maximum similarity degree is compared with gray intensity of a patch within a predetermined range through template matching, Blending with the patch region on the pixel.

Particularly in the case of the present invention, a sum of absolute difference (SAD) in a gray-scale space was used as a template matching method.

After the above process is completed, the process of (i) and (ii) is performed again. That is, the highest priority among the pixels on the boundary is recalculated and the above process is repeated until all of the target pixels are interpolated, and all of the target pixels can be interpolated.

In this way, the smear-removed image can be restored.

&Lt; First Experiment Result >

Hereinafter, the first experimental result of applying the present invention will be described.

7B is a signal distribution curve for an image in the first experiment, FIG. 7C is a normal distribution curve for a signal distribution curve in the first experiment, FIG. 7D is a normal distribution curve for the signal distribution curve in the first experiment, FIG. 7E is a smear region detected using the binary pattern map in the first experiment, FIG. 7F is a smear region obtained by removing the smear in the first experiment, Inpainting technique is applied to image restoration.

Referring to FIG. 7A, the smear position is a partial area including a reflector, and is distributed at chrome or mirror positions of the vehicle. The position of the license plate is influenced by the position of the chrome, which causes the number or the letter of the license plate to be distorted, thereby failing to achieve the purpose of the CCTV. In the case of Fig. 7A, in the first experiment, a smear was generated in the chrome portion on the front of the vehicle, which does not hinder the recognition of the license plate. However, it is necessary to remove the smear for a clear vehicle image.

Referring to FIG. 7B, it can be seen that a signal distribution per each column is known, and a peak having a peak value farther from the periphery is present at a position of about 430 pixels from the left side of the image. In this way, it is possible to detect that a smear has occurred in the image.

Referring to FIG. 7C, it can be seen that the signal distribution curves of the image can be normalized and the smear change curve is distributed in the highest region (paek) than the change curve including the background, noise, and object. This is because it has a strong intensity reaching the reflector of the vehicle from the light source. At this time, the change curve is reconstructed according to the threshold value in the normal distribution signal, and the smear is determined in a region located in a region higher than the set value.

Referring to FIG. 7D, when a smear region is determined, it can be converted into a binary pattern map (Alpha map), and it can be determined as a smear region and detected. That is, in the normal distribution signal, the change curve is reconstructed according to the threshold value, and the section located in a region higher than the preset value is determined as the position where the smear is generated.

Referring to FIG. 7E, it can be seen that the smear is generated and the judgment area is slightly enlarged to remove it from the image. Extending the region is to obtain a more accurate correction result by utilizing the pixel information in a wider area than the initial retrieved position.

When the smear is removed from the image, it can be seen that the original image is not restored but the smear is present on the image at that position.

Referring to FIG. 7F, an image can be restored by applying an inpainting technique on a region where the smear is removed. As mentioned above, the priority is determined and the image is sequentially restored to the smear part through the patch method.

The minimum and maximum pixel intensity values (Mean intensity), Mean (Mean), Sum of absolute differences (SAD), and PSNR between the input image and the reconstructed image can be compared with each other. Maximum signal-to-noise ratio, and peak-to-noise ratio).

In the case of the comparison, if the values between the image before the restoration and the image after the restoration exist in similar values, it can be understood that the method proposed by the present invention is an approximated approach.

Table 1 shows the data obtained before and after the restoration by the results of the first experiment.

Test Image Intensity value (Min) Mean SAD PSNR input result input result The first experiment 18 21 112.19 111.54 3.907 26.33 (dB)

According to Table 1, there is no significant difference in the minimum pixel intensity value (Min) and the mean value (Mean) when comparing the values before and after the restoration. In case of SAD, It can be confirmed that the restoration has been successfully performed.

That is, as can be seen from Figs. 7A and 7F, the restoration can be confirmed visually, and it can be confirmed that the restoration has been successful also by the data.

&Lt; Second Experimental Result >

Hereinafter, a second experimental result using the present invention will be described.

8A is a signal distribution curve for an image in the second experiment, FIG. 8C is a normal distribution curve for a signal distribution curve in the second experiment, FIG. FIG. 8E is a smear region detected using a binary pattern map in the second experiment, FIG. 8F is a smear region obtained by removing smear in the second experiment, Inpainting technique is applied to image restoration.

Referring to FIG. 8A, the position where the smear is generated exists in the license plate, so that an erroneous or unrecognized expression may be generated for the recognition of the number of the vehicle. Therefore, it is recognized that it is necessary to remove the smear generated on the license plate and restore the image.

Referring to FIG. 8B, it can be seen that a signal distribution per each column is known, and a peak having a peak value far higher than that of the surrounding is present at a position of about 440 pixels away from the left side of the image. In this way, it is possible to detect that a smear has occurred in the image.

Referring to FIG. 8C, it can be seen that the signal distribution curves of the image can be normalized, and the smear change curve is distributed in the highest region (paek) than the change curve including the background, noise, and object. This is because it has a strong intensity reaching the reflector of the vehicle from the light source. At this time, the change curve is reconstructed according to the threshold value in the normal distribution signal, and the smear is determined in a region located in a region higher than the set value.

Referring to FIG. 8D, when a smear region is determined, it can be converted into a binary pattern map (alpha map), and it can be determined as a smear region and detected. That is, in the normal distribution signal, the change curve is reconstructed according to the threshold value, and the section located in a region higher than the preset value is determined as the position where the smear is generated.

Referring to FIG. 8E, it can be seen that the smear is generated and the judgment area is somewhat extended to remove it from the image. Extending the region is to obtain a more accurate correction result by utilizing the pixel information in a wider area than the initial retrieved position.

When the smear is removed from the image, it can be seen that the original image is not restored but the smear is present on the image at that position.

Referring to FIG. 8F, an image can be restored by applying an inpainting technique to a region where the smear is removed. As mentioned above, the priority is determined and the image is sequentially restored to the smear part through the patch method.

8A and 8F, the license plate on which the smear is generated in FIG. 8A is an image in which an error may occur in the case of automatic number recognition, but the image restored in FIG. 8F is capable of automatic number recognition, The image is restored so that it is not restored.

As in the first experimental result, the second experimental result can be judged by comparing the data obtained before the restoration with the data obtained after the restoration, as shown in Table 2.

Test Image Intensity value (Min) Mean SAD PSNR input result input result Second experiment 12 13 78.93 77.81 3.672 25.72 (dB)

According to Table 2, there is no significant difference between the minimum pixel intensity value (Min) and the mean value (Mean) when comparing the values before and after the restoration. In case of SAD, It can be confirmed that the restoration has been successfully performed.

That is, as can be seen from Figs. 8A and 8F, the restoration can be visually confirmed, and it can be confirmed that the restoration has been successful also by the data.

&Lt; Third Experimental Result >

Hereinafter, the third experimental result using the present invention will be described.

FIG. 9A is a signal distribution curve relating to an image in the third experiment, FIG. 9C is a normal distribution curve with respect to the signal distribution curve in the third experiment, FIG. FIG. 9E is a smear region detected using a binary pattern map in the third experiment, FIG. 9F is a smear region obtained by removing smear in the third experiment, Inpainting technique is applied to image restoration.

Referring to FIG. 9A, a smear is generated in a low illuminance distribution, and unlike other experimental images, a reflector position by a vehicle has a space shape rather than an edge. That is, even if the illuminance is low, a strong illuminance may be generated and smear may occur depending on the position of the installed camera and the light source.

Referring to FIG. 9B, it can be seen that the signal distribution per each column of the image is known, and that there exists a column having a peak value that is much higher than the surrounding area at a position about 1100 pixels away from the left side of the image. In this way, it is possible to detect that a smear has occurred in the image.

Referring to FIG. 9C, it can be seen that the signal distribution curves of the image can be normalized, and the smear change curve is distributed in the highest region (paek) than the change curve including the background, noise, and object. This is because it has a strong intensity reaching the reflector of the vehicle from the light source. At this time, the change curve is reconstructed according to the threshold value in the normal distribution signal, and the smear is determined in a region located in a region higher than the set value.

Referring to FIG. 9D, when a smear region is determined, it can be converted into a binary pattern map (Alpha map), and it can be determined as a smear region and detected. That is, in the normal distribution signal, the change curve is reconstructed according to the threshold value, and the section located in a region higher than the preset value is determined as the position where the smear is generated.

Referring to FIG. 9E, it can be seen that the smear is generated, and the judgment area is somewhat expanded to remove it from the image. Extending the region is to obtain a more accurate correction result by utilizing the pixel information in a wider area than the initial retrieved position.

When the smear is removed from the image, it can be seen that the original image is not restored but the smear is present on the image at that position.

Referring to FIG. 9F, an image can be restored by applying an inpainting technique to a region where the smear is removed. As mentioned above, the priority is determined and the image is sequentially restored to the smear part through the patch method.

Table 3 shows the results of the third experiment by comparing the data obtained before and after the restoration.

Test Image Intensity value (Min) Mean SAD PSNR input result input result Third experiment One 0 63.44 62.68 4.428 22.46 (dB)

Table 3 shows that there is not a large difference between the minimum pixel intensity value (Min) and the mean value (Mean) when comparing the values before and after the restoration. In the case of SAD, It can be confirmed that the restoration has been successfully performed.

That is, as can be seen from Figs. 9A and 9F, the restoration can be confirmed visually, and it can be confirmed that the restoration has been successful also by the data.

&Lt; Fourth Experimental Result >

Hereinafter, the fourth experimental result using the present invention will be described.

10A is a signal distribution curve relating to an image in the fourth experiment, FIG. 10C is a normal distribution curve for a signal distribution curve in the fourth experiment, FIG. 10E is a smear region detected using a binary pattern map in the fourth experiment, FIG. 10F is a smear region obtained by removing smear in the fourth experiment and performing in-painting (FIG. Inpainting technique is applied to image restoration.

Referring to FIG. 10A, it can be seen that a plurality of smears are generated in one image. Especially, plural smears can be a problem especially because they have different strengths.

Specifically, the smear phenomenon differs depending on the reflector and the intensity, but there is a clear smear phenomenon, and smear due to the weak intensity is present, so that the pattern is similar to background, object, and noise. There is a difficult problem.

Therefore, if the threshold value is larger than the threshold value, the smear corresponding to the strongest intensity is obtained. However, in the case of the weak intensity, the original image is maintained.

Therefore, it is reasonable to maintain the original image signal without processing the smear generated from a small amount of light source, since the original image itself is distorted and the distortion is more severe in the reconstructed result.

In the case of FIG. 10A, since all of the smears generated are larger than the threshold value, all of them can be removed.

Referring to FIG. 10B, it can be seen that the signal distribution per each column of the image is known. As a result, it can be seen that there exists a column having a peak value that is much higher than the surrounding area at a position of about 550 and 1050 pixels away from the left side of the image have. Thus, it is possible to detect that a plurality of smears are generated in the image.

Referring to FIG. 10C, it can be seen that the signal distribution curves of the image can be normalized, and the smear change curve is distributed in the highest region (paek) than the change curve including the background, noise, and object. This is because it has a strong intensity reaching the reflector of the vehicle from the light source. At this time, the change curve is reconstructed according to the threshold value in the normal distribution signal, and the smear is determined in a region located in a region higher than the set value.

Referring to FIG. 10D, when a smear region is determined, it can be converted into a binary pattern map or an Alpha map, and it can be determined as a smear region and detected. That is, in the normal distribution signal, the change curve is reconstructed according to the threshold value, and the section located in a region higher than the preset value is determined as the position where the smear is generated.

Referring to FIG. 10E, it can be seen that the smear is generated, and the judgment area can be expanded to remove it from the image. Extending the region is to obtain a more accurate correction result by utilizing the pixel information in a wider area than the initial retrieved position.

When the smear is removed from the image, it can be seen that the original image is not restored but the smear is present on the image at that position.

Referring to FIG. 10F, an image can be restored by applying an inpainting technique on a region where the smear is removed. As mentioned above, the priority is determined and the image is sequentially restored to the smear part through the patch method.

The results of the fourth experiment can also be determined by comparing the data obtained before and after the restoration, as shown in Table 4.

Test Image Intensity value (Min) Mean SAD PSNR input result input result The fourth experiment 13 10 122.11 121.19 5.384 23.93 (dB)

According to Table 4, there is no significant difference in the minimum pixel intensity value (Min) and the mean value (Mean) when comparing the values before and after restoration. In the case of SAD, It can be confirmed that the restoration has been successfully performed.

That is, as can be seen in FIGS. 10A and 10F, the restoration can be confirmed visually, and it can be confirmed that the restoration has been successfully performed by the data.

Through the above-described methods and experiments, it is possible to provide a user with an apparatus and method for detecting smear by using image processing of an image obtained through CCD, and for obtaining image data of sharp image quality by removing detected smear .

In particular, in the case of the CCTV pre-installed using the conventional CCD image sensor, additional devices for preventing smear are not installed, and the smear can be removed without incurring additional expenses. In this way, it is possible to reduce the replacement cost of the existing CCTV and increase the number recognition rate of the vehicle.

The present invention can also be embodied as computer-readable codes on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may be implemented in the form of a carrier wave (for example, transmission over the Internet) .

In addition, the computer-readable recording medium may be distributed over network-connected computer systems so that computer readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers of the technical field to which the present invention belongs.

In addition, the apparatus and method as described above can be applied to a case where the configuration and method of the embodiments described above are not limitedly applied, but the embodiments may be modified so that all or some of the embodiments are selectively combined .

10:
20:
30: Removal
40:
100: Smear removal device

Claims (14)

An image input unit receiving an image acquired by the CCD sensor;
A detector for detecting a position of a first column in which the smear is generated among columns constituting the image when a smear is generated in the image input to the image input unit; And
And removing the smear from the image based on the detected position information of the first column,
Wherein:
An extracting unit for extracting a signal distribution curve for each column constituting the image using the image input to the image input unit; And
And a conversion unit for converting the signal distribution curve into a normal distribution curve,
Wherein the signal distribution curve represents a sum of gray values of a plurality of pixels constituting each column constituting the image. delete The method according to claim 1,
Wherein the detector compares the normal distribution curve with a predetermined threshold to generate a binary pattern map,
Wherein the binary pattern map has a value of 0 in an area where the normal distribution curve is smaller than the threshold value and the binary pattern map has a value of 1 in an area where the normal distribution curve is larger than the threshold value. . The method of claim 3,
Wherein the region having the value of 1 in the binary pattern map corresponds to the first column of the image. The method of claim 3,
Wherein the threshold value is determined according to the following equation.
Equation

In the above equation,

Is the above-mentioned threshold value,

Is an average of the signal distribution curves,

Is a weight,

Is the standard deviation of the signal distribution curve. 5. The method of claim 4,
Wherein:
Wherein a width of the first column is increased by adding a part of the column adjacent to the first column to the first column. The method according to claim 6,
The removing unit
Subtracting a gray value of each of a plurality of pixels constituting the binary pattern map from a gray value of each of a plurality of pixels constituting the image,
And removing the smear generated in the first column by using the subtracted result. An image input unit receiving an image acquired by the CCD sensor;
A detector for detecting a position of a first column in which the smear is generated among columns constituting the image when a smear is generated in the image input to the image input unit;
A removal unit for removing the smear from the image based on the detected position information of the first column; And
And a reconstruction unit for reconstructing the image of the first column from which the smear is removed by using a predetermined interpolation method. 9. The method of claim 8,
The interpolation method of the reconstruction unit may include:
Calculating a priority for each of the plurality of pixels in the patch, determining a highest priority pixel having the highest priority among the calculated priorities, and determining the highest priority pixel among the plurality of pixels in the patch, The restoration is performed by comparing the similarity of the pixels not constituting the column,
Wherein the patch comprises at least a portion of pixels constituting the first column. 10. The method of claim 9,
The restoration unit,
Wherein the interpolation method is applied until restoration is performed on all the pixels constituting the first column. 10. The method of claim 9,
Wherein the priority order is calculated according to the following equation.
Equation

In the above equation,

Gt;

Is the priority of

Is the reliability of the pixel,

Is the patch,

Is the size of the patch,

Is an input image,

Is an area to be restored,

Is the direction unit vector of the structure,

Gt;

Is a normal unit vector for the contour in

Is a normalization constant. A first step of receiving an image acquired by the CCD sensor;
A second step of, when a smear is generated in the input image, detecting a position of the first column in which the smear is generated among columns constituting the image; And
And removing the smear from the image based on the detected position information of the first column,
The second step comprises:
Extracting a signal distribution curve for each column constituting the image using the input image;
Transforming the signal distribution curve into a normal distribution curve; And
Comparing the normal distribution curve with a predetermined threshold value to generate a binary pattern map,
Wherein the signal distribution curve represents a sum of gray values of a plurality of pixels constituting each column constituting the image,
Wherein the binary pattern map has a value of 0 in an area where the normal distribution curve is smaller than the threshold value and the binary pattern map has a value of 1 in an area where the normal distribution curve is larger than the threshold value. . delete A first step of receiving an image acquired by the CCD sensor;
A second step of, when a smear is generated in the input image, detecting a position of the first column in which the smear is generated among columns constituting the image;
A third step of removing the smear from the image based on the detected position information of the first column; And
And a fourth step of restoring an image of the first column from which the smear is removed by using a predetermined interpolation method,
In the fourth step,
Calculating a priority for each of the plurality of pixels in the patch;
Determining a highest priority pixel having the highest priority among the calculated priorities; And
And performing a restoration through comparison of similarity of pixels of the highest priority among the plurality of pixels in the patch that do not constitute the first column,
Wherein the patch comprises at least a portion of the pixels constituting the first column.

Images