KR20110123512A - Partial image extractor and its partial image extracting method - Google Patents

Abstract

PURPOSE: A partial image extractor and a partial image extracting method thereof are provided to extract the license plate of a vehicle by determining the color parameters of the vehicle and the license plate. CONSTITUTION: An image pickup unit(110) picks up images about an external environment. A color coordination converting unit(120) converts the images into HIS color coordinates data. A binarization unit(130) generates a candidate area through the binarization of the HIS color coordinates data. A labeling unit(135) generates label data from the candidate area through recursive labeling algorithm. A filter unit(140) extracts the final candidate area by filtering the label data.

Description

Partial image extractor and its partial image extracting method}

The present invention relates to a partial image extractor and a partial image extraction method, and more particularly, to a partial image extractor and a partial image extraction method for extracting a partial image from an entire image using the color of the image.

The vehicle number recognition system is a system for recognizing a vehicle number of a vehicle entering a gate installed in a parking lot or a highway toll gate, a road, or the like. It is also a system for recognizing the vehicle number of a speeding vehicle in a speeding prevention system. The vehicle number recognition system extracts numeric or character information corresponding to the vehicle number by analyzing an image including the license plate of the vehicle.

However, the conventional system for extracting a license plate has a problem in that the success rate of the license plate extraction is significantly lowered when the color of the vehicle is similar to the color of the license plate.

An object of the present invention for solving the above problems is to extract the license plate of the vehicle by determining the color parameters of the license plate and the vehicle color by intensively analyzing the fine color difference when the license plate of the vehicle and the color of the vehicle is similar. To provide a partial image extractor and partial image extraction method.

Partial image extractor according to an embodiment of the present invention for solving the above problems, the image pickup unit for imaging the image of the external environment, the color coordinate conversion unit for receiving the image from the image pickup unit to convert the HSI color coordinate data A binarization unit generating a candidate region by binarizing the HSI color coordinate data, a labeling unit generating label data using a recursive labeling algorithm in the candidate region, and And a filter unit for filtering the label data to extract the final candidate region.

Here, the binarization unit binarizes the HSI color coordinate data using a threshold value that is set as a result value through a statistical method.

The partial image extractor may further include an adjusting unit configured to determine a main axis of rotation passing through the final candidate area and to rotate the final candidate area along the main axis of rotation.

In addition, the partial image extractor according to an embodiment of the present invention further includes an image decomposition unit for generating the partial image by decomposing the rotated final candidate region.

The partial image extractor may further include a storage unit for storing the image and the histogram pattern captured by the imaging unit, histogram, normalize, and filter an image region corresponding to the partial image on the stored image, and then output the result value to the storage unit. The apparatus may further include an information reader configured to recognize and output the information included in the partial image in comparison with the stored histogram pattern.

Meanwhile, the partial image extraction method according to an embodiment of the present invention comprises the steps of capturing an image of an external environment, receiving the captured image and converting the image into HSI color coordinate data, and binarizing the HSI color coordinate data. A binarization step of generating a candidate region, a labeling step of generating labeled data using a recursive labeling algorithm on the candidate region, and a final candidate by filtering the candidate region And filtering the regions.

Here, the binarization step binarizes the HSI color coordinate data by using a threshold value that is set as a result value through a statistical method.

The partial image extraction method may further include an adjusting step of determining a main axis of rotation passing through the final candidate area and rotating the final candidate area along the main axis of rotation.

The partial image extraction method may further include an image decomposition step of generating the partial image by decomposing the rotated final candidate region.

In addition, the partial image extraction method stores the photographed image and the histogram pattern, histograms, normalizes, and filters an image region corresponding to the partial image on the stored image, and compares the result value with the stored histogram pattern. The method may further include an information reading step of recognizing and outputting information included in the image.

1 and 2 are block diagrams illustrating a configuration of a partial image extractor according to an exemplary embodiment.
3A to 3C are views for explaining a rotation adjustment operation according to an embodiment of the present invention.
4A to 4G are diagrams for describing a process of reading information from a candidate area according to an embodiment of the present invention.
5 illustrates a modulated iterative labeling algorithm according to an embodiment of the present invention.
6 is a flowchart illustrating a method of extracting a partial image, according to an exemplary embodiment.

Hereinafter, with reference to the drawings will be described the present invention in more detail.

1 is a block diagram illustrating a configuration of a partial image extractor 100 according to an exemplary embodiment. The partial image extractor 100 refers to a device that collects an image of the surrounding environment and extracts a partial region of interest from the collected entire images. The partial image extractor 100 may be implemented as an apparatus for extracting an automobile license plate or an apparatus for extracting an intruder shape in an intruder monitoring system. However, the present disclosure focuses on an embodiment of an apparatus for extracting an automobile license plate. In particular, it is assumed that the license plate has a color of green, yellow, and white for convenience of description.

Referring to FIG. 1, the partial image extractor 100 may include an imaging unit 110, a color coordinate conversion unit 120, a binarization unit 130, and a filter unit 140.

The imaging unit 110 collects and receives an image of the surrounding environment of the image extractor 100. That is, the imaging unit 110 may be a general camera as a device for receiving an image of the external environment. However, a pan-tilt-zoom (PTZ) camera may be preferably used as the imaging unit 110 in case all cars do not move along the same path. By using a PTZ camera, it will be easy to collect an image including the license plate even when the car is moving at a distance or being deflected. In addition, the imaging unit 110 may be implemented in a form including one or more cameras.

The color coordinate converter 120 may perform an operation of converting an image of the external environment collected by the imaging unit 110 into color coordinates. In detail, the color coordinate conversion unit 120 may convert the color coordinates of the image data for license plate recognition into a HSI (Hue, Saturation, Intensity) color coordinate system. The HSI color coordinate system is a coordinate system in which color is expressed in Hue, saturation, and intensity. In the present embodiment, the HSI color coordinate system is used, but the present invention is not limited thereto, and various color coordinate systems such as YIQ, YCBCR, CIELUV, HSV, and HSL may be used.

The image at the time point inputted to the color coordinate converter 120 is represented by an RGB color space. Therefore, the color coordinate converter 120 may convert the image displayed in the RGB coordinate system into the HSI color coordinate system. The process of converting the color coordinates is based on Equations 1 to 3 below. Equation 1 represents a color value, Equation 2 represents a saturation value, and Equation 3 represents a brightness value.

The color value calculated according to Equation 1 may have a value from 0 to 360. When the color value is 0, it represents red. The color value 60 indicates yellow, the color value 120 indicates green, and the color 240 indicates blue.

The saturation value calculated according to Equation 2 may have a value from 0 to 1.

The color coordinate converter 120 may transmit data represented by the HSI model calculated according to Equations 1 to 3 to the binarization unit 130.

The binarization unit 130 may binarize the data represented by the HSI color coordinates.

In detail, the binarization unit 130 may perform the binarization operation using the average value and the standard deviation of the hue values. In addition, the binarization unit 130 may use a hue parameter of the HSI coordinate system to detect green and yellow. In addition, the binarization unit 130 does not use color parameters when detecting white, and may mainly use saturation and intensity parameters.

Color parameters, saturation parameters, and brightness parameters used in the binarization operation according to an embodiment of the present invention may be set through a statistical method from different images including a license plate.

Here, the statistical method refers to a color parameter obtained by photographing a license plate in various environments, and then collecting hue, saturation, and brightness from the image samples photographed in various environments, and calculating the average of the respective hue, saturation, and brightness values. After setting to, it may mean a method of setting a predetermined range of parameters in consideration of the standard deviation by each sample. Certain constants in the following equations (4) and (5) can be determined through a statistical method.

Equation 4 below is an equation used in the binarization process when detecting green and yellow color.

Here, H (x, y), S (x, y), and I (x, y) mean components of the x-th and y-th pixels of hue, saturation, and lightness, respectively. μ ^H and σ ^H mean color value mean and color value standard deviation, respectively.

In addition, Equation 5 below represents a binarization process for white when the surrounding environment, that is, a vehicle including a license plate is exposed to strong light or weak light. The subscripts n, l, and h denote the case where the environment in which the plates are exposed is exposed to the general amount of light, when the light is exposed to weak light, and when the light is exposed to strong light.

Here, S (x, y) and I (x, y) represent saturation values and brightness values of the x and y th pixels, respectively. μ ^S and σ ^S represent the mean of the chroma values and the standard deviation values of the chroma values, respectively. μ ^H and σ ^H represent the mean of the brightness values and the standard deviation values of the brightness values, respectively.

The candidate region may be obtained through a binarization process using Equations 4 and 5. In this case, the candidate region is preferably a partial region of interest, that is, the license plate is represented by 1, and parts other than the license plate may be represented by 0, so that the license plate part may be distinguished from other parts.

However, when the color of the license plate and the car is similar, the parts of the car other than the license plate may also be displayed as 1, which may cause a problem in that the license plate may not be extracted accurately. In order to solve this problem, the labeling unit 135 uses an iterative labeling algorithm in the filtering step, which will be described later in more detail.

The labeling unit 135 receives the binarization data from the binarization unit 130 and performs labeling to generate label data. In addition, the labeling unit 135 transmits the generated label data to the filter unit 140.

The filter unit 140 may receive one or more candidate regions including the license plate from the labeling unit 135 and finally filter the candidate regions of the license plate to generate the final candidate region.

The final candidate area may include a license plate and may include portions having a color similar to the license plate. In addition, the filter unit 140 may exclude candidate regions that are estimated to not be license plates from among several candidate regions.

Table 1 is a table of criteria for excluding candidate regions according to an embodiment of the present invention.

Filtering parameters Green yellow White Bounding box [0.6, 1.0] [0.7,1.0] [0.7, 1.0] Aspect ratio [1.0, 3.0] [1.0,2.0] [1.0,6.0] Possible license plate shape Rectangle

The filtering process, that is, the process of excluding a candidate region that is not considered to be a license plate, may use the external characteristics of the license plate. Here, the external characteristic may be a bounding box and a horizontal-to-vertical ratio.

The filter unit 140 may determine that the candidate region that does not satisfy the preset conditions as shown in Table 1 is not the license plate, and exclude the candidate region from the candidate region. In this way, the filter unit 140 may finally set the remaining portion excluding the candidate region as a candidate region of the license plate and provide the final candidate region to the user.

In more detail, the bounding box may refer to a region connecting intersection points of one left end, one right end, an upper end, and a lower end of the region extracted as a rectangle. Here, the width of the bounding box is determined by the pixel and may represent the relative size of the image pickup object.

The aspect ratio is a value representing the ratio of the length of the bounding box to the width and length. The aspect-ratio may be determined from the extracted image, and may be expressed by Equation 6 below.

Here, c and r respectively represent the length of the column and the length of the column of the bounding box. In the case of a green plate, pA represents a value of 1 to 3, in the case of a yellow plate, 1 to 2, and a white plate to a value of 1 to 6.

The filter unit 140 may finally extract the candidate region of the license plate by excluding the regions included in the candidate region without the license plate by excluding a candidate region not included in the preset range.

2 is a block diagram illustrating a configuration of a partial image extractor 100 according to an exemplary embodiment.

Referring to FIG. 2, the partial image extractor 100 may further include an adjusting unit 150, an image resolving unit 160, an information reading unit 170, and a storage unit 180.

The final candidate area filtered from the filter unit 140 is transmitted to the image decomposition unit 160 after the preset adjustment operation through the adjustment unit 150 before being provided to the user, and then is transferred to the original image before the color coordinate conversion. Can be provided. In addition, the information reader 170 may extract information included in the partial image from the original partial image.

According to an embodiment of the present disclosure, the adjustment unit 150 may receive the final candidate region from the filter unit 140 and perform a rotation adjustment operation on the final candidate region.

When the vehicle passes around the partial image extractor 100, the images collected by the imaging unit 110 may be different because they do not always move in the same direction and at the same distance, and may be captured by different imaging angles. Can be. An image of a rectangular license plate imaged at these different imaging angles may not be in the form of a perfect rectangle.

Therefore, the adjustment unit 150 may adjust the candidate area in a rectangular shape by performing a rotation adjustment operation before providing the candidate area to the user. 3A to 3C illustrate a rotation adjustment process according to an embodiment of the present invention. This will be described later with reference to FIGS. 3A to 3C.

The extracted candidate regions are expressed only with values of 0 and 1 since they have undergone color coordinate transformation and binarization. In order to provide meaningful information to the user, the original image should be provided instead of the data represented by these zeros and ones.

The storage unit 170 may receive and store an image of the external environment captured by the image capturing unit 110. The image separation unit 160 may receive the adjusted candidate region from the adjustment unit 150, and may receive the original image from the storage unit 170.

The image separation unit 160 may compare two received images, extract a portion corresponding to the adjusted candidate region from the original image, and transmit the extracted image to the information reader 180.

The information reader 180 may read information of interest to the user from the partial image transmitted from the image resolver 160. For example, if the information of interest to the user is information on the license plate, the information extracting unit 180 may read information of numbers and letters from a partial image corresponding to the license plate. A method of extracting information about letters and numbers from the license plate will be described later in more detail with reference to FIGS. 4A to 4G. Meanwhile, the extracted information may be provided to the user or may be stored in the storage unit 170. When the extracted information is provided to the user, the extracted information may be provided in various ways such as display, wired or wireless information transmission, and voice output.

3A to 3C illustrate a rotation adjustment operation in the adjustment unit 150 according to an embodiment of the present invention.

The adjuster 150 may first determine the center of the candidate area and determine a minimum second moment axis.

Here, the center of the candidate region may be determined by the row and column axis based on the spatial distribution in the candidate region. Equation 7 shows an equation for determining the center of the candidate region.

Here, r _center and c _center represent the coordinates of the center of the column and row direction, respectively, r and c represent the column and row coordinates, the subscript i means the i-th pixel, Ai indicates the width of the area, and the pixel Can be calculated by

The central axis and the central moment can be determined as the minimum second moment. The equation for determining the center moment is expressed in Equation 8 below.

Also, the angle θ to be finally rotated based on the determined center axis and the center moment is determined by the following equation (9).

By rotating by θ by Equation 9, the candidate region may have a shape close to a rectangle.

3A illustrates a final candidate region filtered by the filter unit 140.

3b shows the minimum secondary moment axis for the rotation adjustment operation. 3C shows the candidate area of the rectangular shape obtained after the rotation adjustment operation.

4A to 4G illustrate a process of extracting information of letters and numbers from a partial image transmitted from the image separation unit 160 according to an embodiment of the present invention.

4A illustrates a partial image obtained by an embodiment of the present invention.

4B shows a longitudinal position histogram obtained by scanning a license plate longitudinally. Having a value close to 100 in the region near 0 and 250 in FIG. 4B indicates that there is a border of the license plate at 0 and 250.

4C shows a lateral position histogram obtained by scanning a license plate in the lateral direction. It can be seen that Figure 4c also includes a license plate border at the bottom. However, since the boundary of the license plate is an unnecessary area for identifying numbers and letters, it is preferable to be excluded in detecting information of the candidate area.

4D shows a horizontal position histogram after the boundary of the license plate has been excluded, ie filtered.

The information reader 180 may read information of the candidate area from the candidate area based on the candidate area excluding the boundary of the license plate.

4E is a normalized image of the histogram excluding the boundary of the license plate of the vehicle, and it can be seen that information of '10 7177 'can be obtained.

4F shows a longitudinal position histogram. As shown in Fig. 4f, there are seven peaks, which correspond to the respective numbers and letters. The information reader 180 may read the information of the license plate by comparing the previously stored pattern of the histogram with the read histogram.

4G illustrates information of the read license plate, and the information reading unit 180 may read information of seven numbers and letters.

FIG. 5 shows an iterative labeling algorithm capable of extracting a license plate area as a candidate area when the color of the car is similar to the color of the license plate.

Even when the color of the car is similar to that of the license plate, there is a difference between the color of the car and the license plate. The iterative labeling algorithm may extract the license plate as a candidate region based on the color difference.

Referring to FIG. 5, two four connected pixels may be grouped when the equation 10 is satisfied. Dist of Equation 10 is determined by Equation 11.

Here, the superscript R in I ^R _{i , j} represents red, G represents green, and B represents blue. I denotes brightness, and the subscripts i, j denote the i-th pixel in the horizontal direction and the j-th pixel in the vertical direction.

The first figure of FIG. 5 shows a binarized region. A region representing 1 in the binarized region is extracted as a candidate region, and accordingly, a labeling operation is performed on each region as shown in the second drawing of FIG. 5. The second drawing shows a candidate region after a connection is performed in the case where there is an association by determining an association with pixels existing in up, down, left, and right directions about one pixel. As a result of the operation according to the exemplary embodiment of the present disclosure, in the first drawing, arbitrary pixels may be connected to each other when the relationship with the pixels existing in the up, down, left, and right directions satisfies Equation (10).

The pixels connected to each other may be grouped as shown in the third drawing of FIG. 5, so that when the color of the car is similar to the color of the license plate, the license plate may be efficiently extracted as the final candidate area.

6 is a flowchart illustrating a partial image extraction method according to an embodiment of the present invention.

Referring to FIG. 6, the partial image extraction method includes an image capturing step (S610), a color coordinate conversion step (S620), a binarization step (S630), a labeling step (S640), a filtering step (S650), an adjusting step (S660), and an image. It may include a decomposition step (S670) and information reading step (S680).

The imaging unit 110 may collect and receive an image of the surrounding environment of the image extractor 100 (S610).

The color coordinate converter 120 may perform an operation of converting an image of the external environment collected by the imaging unit 110 into color coordinates. The color coordinate conversion unit 120 may convert the color coordinates of the image data for license plate recognition into a HSI (Hue, Saturation, Intensity) color coordinate system (S620). In the present embodiment, the HSI color coordinate system is used, but the present invention is not limited thereto, and various color coordinate systems such as YIQ, YCBCR, CIELUV, HSV, and HSL may be used.

The image at the time point inputted to the color coordinate converter 120 is represented by an RGB color space. Therefore, the color coordinate converter 120 may convert the image displayed in the RGB coordinate system into the HSI color coordinate system.

The binarization unit 130 may binarize the data represented by the HSI color coordinates (S630).

The binarization unit 130 according to an embodiment of the present invention may perform a binarization operation by using an average value and a standard deviation of hue values. In order to detect green and yellow, a hue parameter of the HSI coordinate system may be used. In the case of detecting white, the color parameter is not used, and the saturation and intensity parameters may be mainly used.

Color parameters, saturation parameters, and brightness parameters used in the binarization operation according to an embodiment of the present invention may be set through a statistical method from different images including a license plate.

The statistical method is to set the color parameters by capturing the license plate in various environments, and then collecting the hue, saturation, and brightness from the image samples taken in various environments, and then calculating the average of the respective hue, saturation, and brightness values. After that, it may mean a method of setting a predetermined range of parameters in consideration of the standard deviation by each sample.

 Candidate regions can be obtained through the binarization process. In this case, the candidate region is preferably a partial region of interest, that is, the license plate is represented by 1, and parts other than the license plate may be represented by 0, so that the license plate part may be distinguished from other parts.

However, when the color of the license plate and the car is similar, the parts of the car other than the license plate may also be displayed as 1, which may cause a problem in that the license plate cannot be accurately extracted. In order to solve this problem, the labeling unit 135 may use an iterative labeling algorithm in the filtering step.

The labeling unit 135 receives the binarization data from the binarization unit 130 and performs labeling to generate label data (S640). In addition, the labeling unit 135 transmits the generated label data to the filter unit 140.

The filter unit 140 may receive one or more candidate regions including the license plate from the labeling unit 135 and finally filter the candidate regions of the license plate (S650).

The final candidate area may include a license plate and may include portions having a color similar to the license plate. In addition, the filter unit 140 may exclude candidate regions that are estimated to not be license plates from among several candidate regions.

The filtering process, that is, the process of excluding a candidate region that is not considered to be a license plate, may use the external characteristics of the license plate. Here, the external characteristic may be a bounding box and a horizontal-to-vertical ratio. The description thereof is as described above with reference to Table 1.

The filter unit 140 may determine that the candidate region that does not satisfy the preset conditions as shown in Table 1 is not the license plate, and exclude the candidate region from the candidate region. In this way, the filter unit 140 may finally set the remaining portion excluding the candidate region as the candidate region of the license plate and provide the final candidate region to the user.

The final candidate area filtered from the filter unit 140 is transmitted to the image decomposition unit 160 after the preset adjustment operation through the adjustment unit 150 before being provided to the user, and then is transferred to the original image before the color coordinate conversion. It may be provided (S660).

According to an embodiment of the present disclosure, the adjustment unit 150 may receive the final candidate region from the filter unit 140 and perform a rotation adjustment operation on the final candidate region.

When the vehicle passes around the partial image extractor 100, the images collected by the imaging unit 110 may be different because they do not always move in the same direction and at the same distance, and may be captured by different imaging angles. Can be. An image of a rectangular license plate imaged at these different imaging angles may not be in the form of a perfect rectangle. Therefore, the adjustment unit 150 may adjust the candidate area in a rectangular shape by performing a rotation adjustment operation before providing the candidate area to the user.

The extracted candidate regions are expressed only with values of 0 and 1 since they have undergone color coordinate transformation and binarization. In order to provide meaningful information to the user, the original image should be provided instead of the data represented by these zeros and ones.

The storage unit 170 may receive and store an image of the external environment captured by the image capturing unit 110. The image separation unit 160 may receive the adjusted candidate region from the adjustment unit 150, and may receive the original image from the storage unit 170.

The image decomposition unit 160 may compare two received images, extract a portion corresponding to the adjusted candidate region from the original image, and transmit the extracted image to the information reader 180 (S670).

The information reader 180 may read information of interest to the user from the partial image transmitted from the image resolver 160 (680). For example, if the information of interest to the user is information on the license plate, the information extracting unit 180 may read information of numbers and letters from a partial image corresponding to the license plate. The description thereof is the same as that of the partial image extractor to read the information of the character. The extracted information may be provided to the user or may be stored in the storage unit 170. The method provided to the user may be provided in various ways, such as display, wired or wireless information transmission, and voice output.

While the above has been shown and described with respect to preferred embodiments of the present invention, the present invention is not limited to the specific embodiments described above, it is usually in the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

100: partial image extractor 110: imaging unit
120: color coordinate conversion unit 130: binarization unit
135 labeling portion 140 filter portion
150: adjusting unit 160: image decomposition unit
170: storage unit 180: information reading unit

Claims (10)

In the partial image extractor,
An imaging unit for capturing an image of an external environment;
A color coordinate conversion unit receiving the image from the imaging unit and converting the image into HSI color coordinate data;
A binarization unit for generating candidate regions by binarizing the HSI color coordinate data;
A labeling unit configured to generate labeled data from the candidate region using a recursive labeling algorithm; And
And a filter unit for filtering the label data to extract a final candidate region. The method of claim 1,
The binarization unit is a partial image extractor, characterized in that to binarize the HSI color coordinate data using a predetermined threshold value as a result of the statistical method. The method of claim 1,
And a controller configured to determine a main axis of rotation passing through the final candidate area and to adjust the rotation of the final candidate area along the main axis of rotation. The method of claim 3, wherein
And an image decomposition unit for generating the partial image by decomposing the rotated final candidate region. The method of claim 4, wherein
A storage unit which stores an image and a histogram pattern captured by the imaging unit; And
An information reader configured to histogram, normalize, and filter an image region corresponding to the partial image on the stored image, and compare the result value with a histogram pattern stored in the storage to recognize and output information included in the partial image; Partial image extractor, characterized in that it further comprises. In the partial image extraction method,
Imaging an image of an external environment;
Receiving the captured image and converting the captured image into HSI color coordinate data;
Binarizing the HSI color coordinate data to generate a candidate region;
A labeling step of generating labeled data from the candidate region using a recursive labeling algorithm; And
And filtering the candidate region to extract a final candidate region. The method according to claim 6,
The binarization step is a partial image extraction method, characterized in that to binarize the HSI color coordinate data using a threshold set as a result value through a statistical method. The method according to claim 6,
And adjusting the final candidate area along the main axis of rotation by determining a main axis of rotation passing through the final candidate area; and further comprising: adjusting the final candidate area along the main axis of rotation. The method of claim 8,
And an image decomposition step of decomposing the rotated final candidate region to generate a partial image. The method of claim 9,
Information stored in the partial image by storing the photographed image and the histogram pattern, and performing histogram, normalization, and filtering on an image area corresponding to the partial image on the stored image, and comparing the result value with the stored histogram pattern. And reading the information and outputting the detected information.

Images