CN105740860B

CN105740860B - Retail shop's label Chinese character region automatic testing method in natural scene

Info

Publication number: CN105740860B
Application number: CN201610059409.XA
Authority: CN
Inventors: 张重生; 赵晓东
Original assignee: Henan University
Current assignee: Henan University
Priority date: 2016-01-28
Filing date: 2016-01-28
Publication date: 2018-04-06
Anticipated expiration: 2036-01-28
Also published as: CN105740860A

Abstract

The invention discloses retail shop's label Chinese character region automatic testing method in a kind of natural scene, comprise the following steps：A, the vectorial V1 and V2 of reflection sample region image main pixel color are obtained；B, the background profile of original image is determined using vectorial V1 and V2；C, hsv color space is divided into black, white, red, yellow, green, cyan, blueness, pinkish red eight pieces of color spaces；D, colo(u)r breakup is carried out to original image, obtains 8 distribution of color binary maps；E, Chinese character connected domain analysis is carried out to 8 distribution of color binary maps, obtains 8 line of text binary maps；F, the line of text that Chinese-character writing rule is not met in each line of text binary map is filtered, the Chinese character region after being refined；G, the Chinese character region of refining of different colours is merged, obtains the Chinese character region in original image.The present invention can effectively improve the degree of accuracy of Chinese character region detection, significant to the application based on the Chinese Character Recognition under natural scene.

Description

Retail shop's label Chinese character region automatic testing method in natural scene

Technical field

The present invention relates to retail shop's label in the technical field of image processing under natural scene, more particularly to a kind of natural scene Chinese character region automatic testing method.

Background technology

In recent years, as the popularization of smart machine, people can easily shoot various pictures in natural scene.Picture In word be important information that image includes, thus the word in image is detected and then accurately identified, helped very much In the understanding to picture material.Although existing OCR character recognition technologies have tended to be ripe, due to the bat under natural scene Take the photograph there may be uneven illumination, angle not just and it is serious reflective a series of problems, such as, while the picture background shot is complicated, often Include multiple fonts, font size even deformed letters so that OCR does not reach preferable effect to the text detection under natural scene.

The content of the invention

It is an object of the invention to provide retail shop's label Chinese character region automatic testing method, Neng Gouzhun in a kind of natural scene The really Chinese character region in detection natural scene on retail shop's label, improves the effect that automatic marking and image understanding are carried out to street view image Rate.

To achieve the above object, the technical solution adopted by the present invention is：

Retail shop's label Chinese character region automatic testing method, comprises the following steps in natural scene：

A, the retail shop's label scanned in natural scene obtains original image, and the sample region figure of rectangle is intercepted from original image Picture, and the RGB triple channels of sample region image are deployed, forms the matrix of the row of M*N rows 3, and wherein M is the width of sample region image, N For the height of sample region image, Kmeans clusters, setting are carried out using every a line of matrix as a point of three dimensions respectively The classification number of cluster is 2, obtains two cluster centre points, respectively vectorial V1 and V2, subsequently into step B；

B, each pixel of original image is calculated respectively to vectorial V1 and V2 distance, and creates two and original image Size identical initializes binary map bg1 and bg2, and the pixel value for initializing all pixels point in binary map bg1 and bg2 is 0, Setpoint distance threshold X, compare each pixel of original image successively to vectorial V1 distance and X magnitude relationship, when original , will be with the pixel point coordinates identical picture in binary map bg1 is initialized when the pixel of image to vectorial V1 distance are less than X The pixel value of vegetarian refreshments is set to 1, obtains coarse contour binary map BG1；Compare each pixel of original image successively to vectorial V2's Distance and X magnitude relationship, when pixel to the vectorial V2 of original image distance are less than X, in binary map bg2 is initialized Pixel value with the pixel point coordinates identical pixel is set to 1, coarse contour binary map BG2 is obtained, respectively to coarse contour two Value figure BG1 and BG2 do Morphological scale-space and carry out connected domain analysis, and area maximum is selected from coarse contour binary map BG1 and BG2 Connected domain repaired and filled after be used as background profile, obtain background profile binary map；

C, hsv color space is divided into black, white, red, yellow, green, cyan, blueness, pinkish red eight pieces of colors Space, the division rule of eight pieces of color spaces are as follows：

Black space, v<0.3or(v<0.5and s<=0.25)；

White space, v>=0.5and s<=0.25；

Red space, (h<1/12or h>=11/12) and v>0.3and s>=0.25；

Yellow space, h>=1/12and h<3/12and v>0.3and s>=0.25；

Vert space, h>=3/12and h<5/12and v>0.3and s>=0.25；

Cyan space, h>=5/12and h<7/12and v>0.3and s>=0.25；

Blue color space, h>=7/12and h<9/12and v>0.3and s>=0.25；

Pinkish red space, h>=9/12and h<11/12and v>0.3and s>=0.25；

H represents the tone in hsv color space in formula, and the saturation degree in behalf hsv color space, v represents hsv color Lightness in space, subsequently into step D；

D, 8 and original image size identical binary map are created, and by the pixel of all pixels point in every binary map Value is set to 0, and original image is transformed into hsv color space from rgb space, then judges original image in hsv color space Each pixel belonging to color type, all pixels point for belonging to same color type is mapped to pair of a binary map Opening position is answered, and the pixel value of the pixel of corresponding position in binary map is set to 1, finally obtains 8 distribution of color two-values Figure, subsequently into step E；

E, 8 distribution of color binary maps are done with carrying out Morphological scale-space after computing again with background profile binary map respectively, The tiny gully in 8 distribution of color binary maps and fine rule are filtered out, obtains 8 thin contoured profile binary map, respectively to 8 thin wheel Exterior feature distribution binary map carries out connected domain scanning, obtains mark figure corresponding to connected graph using eight connectivity labelling method, and calculate each The attribute of connected domain, then the connected domain that Chinese-character stroke feature is not met in 8 thin contoured profile binary map is filtered, obtained Binary map is distributed to 8 stroke levels, is distributed the connected domain for meeting Chinese-character stroke feature in binary map to each stroke level respectively Merge, multiple connected domains after merging are designated as a marked region and determine mark value, for any two mark zone Domain, if the center-of-mass coordinate of a marked region is located in the minimum enclosed rectangle of another marked region, by the marked region Mark value be set to the mark value of another marked region, be distributed binary map finally for each stroke level, successively will mark Value identical marked region is designated as a line of text, 8 line of text binary maps is obtained, subsequently into step F；

F, for any one line of text in 8 line of text binary maps, when the height of the minimum enclosed rectangle of line of text Less than the minimum enclosed rectangle height of background profile 1/20, the ratio of width to height of the minimum enclosed rectangle of line of text be less than 7/10 or The area of line of text and the area ratio of its minimum enclosed rectangle are less than 1/10 or during more than 4/5, by this article one's own profession from corresponding text Deleted in one's own profession binary map, 8 obtained character picture binary map, calculate all texts in each character picture binary map respectively The area of one's own profession and, when the area being calculated and less than background profile binary map minimum enclosed rectangle area 1/200 when, Corresponding character picture binary map is deleted, and the minimum for calculating all line of text in remaining character picture binary map respectively is outer The coordinate of rectangle is connect, subsequently into step G；

G, one and original image size identical blank image are created, successively by institute in remaining character picture binary map The minimum enclosed rectangle for having line of text is marked in the corresponding position of blank image, primary identification image is obtained, respectively by primary Known in identification image by the border that two or more minimum enclosed rectangle forms in the isolated area of irregular polygon as one Other region, for each identification region, the coordinate on each summit is counted, and the minimum value of x coordinate is selected from apex coordinate It is designated as x_min, the maximum for selecting x coordinate is designated as x_max, the minimum value for selecting y-coordinate is designated as y_min, select y-coordinate maximum note For y_max, with (x_min, y_min), (x_max, y_min), (x_min, y_max), (x_max, y_max) four coordinates create rectangular area as summit, Remaining minimum enclosed rectangle composition in all rectangular areas then obtained from each identification region and primary identification image Region be Chinese character region in original image.

Described step A comprises the following steps：

Retail shop's label in A1, scanning natural scene obtains original image, and 1/3 length is more than in scan line and is less than 2/3 Appoint to take in the section of length and be a little used as basic point, point builds R*kR rectangularly-sampled area centered on this basic point, and wherein R is The width in rectangularly-sampled area, R value are more than the 1/7 of original image width, and 1/4, the k less than original image width adopts for rectangle The ratio of width to height in sample area, k span is 1/4<k<4, rectangularly-sampled area is cut from original image, obtains sample region image, Subsequently into step A2；

A2, the RGB triple channels expansion by sample region image, the matrix of the row of M*N rows 3 is formed, wherein M is sample region image Width, N are the height of sample region image, a point using every a line of matrix as three dimensions, are carried out using Euclidean distance Kmeans is clustered, and is set the classification number of cluster as 2, is obtained two cluster centre points, respectively vectorial V1 and V2.

Described step B comprises the following steps：

B1, each pixel in original image is labeled as 1,2 successively ... ..., n, n are pixel in original image Number, calculates the pixel x and vectorial V1 and V2 of original image Euclidean distance L1 (x) and L2 (x), wherein x=1, and 2 ... ..., N, subsequently into step B2；

B2, two and original image size identical initialization binary map bg1 and bg2 are created, and two are initialized by two The pixel value of all pixels point is set to 0 in value figure bg1 and bg2, setpoint distance threshold X, when pixel x and the vector of original image During V1 Euclidean distance L1 (x) ＜ X, the pixel value initialized in binary map bg1 with pixel x position identical pixel is put For 1, coarse contour binary map BG1 is obtained, will be just as the pixel x and vectorial V2 of original image Euclidean distance L2 (x) ＜ X Pixel value in beginningization binary map bg2 with pixel x position identical pixel is set to 1, obtains coarse contour binary map BG2, so Enter step B3 afterwards；

B3, Morphological scale-space is done to coarse contour binary map BG1 and BG2 respectively, i.e., thick wheel is removed by erosion operation first Wide binary map BG1 and BG2 fine rule, the tiny of coarse contour binary map BG1 and BG2 is then removed by dilation operation and closed operation Gully, connected domain analysis finally is carried out to coarse contour binary map BG1 and BG2, face is selected from coarse contour binary map BG1 and BG2 The maximum connected domain of product is as background profile, subsequently into step B4；

Hole in B4, filling background profile, and the depression on background profile edge is repaired, obtain background profile two-value Figure.

Described step E comprises the following steps：

E1,8 distribution of color binary maps are done with background profile binary map respectively and computing, obtain 8 coarse contours distributions Binary map, subsequently into step E2；

E2, Morphological scale-space is done to 8 coarse contour distribution binary maps respectively, i.e., to each coarse contour binary map vertical Closed operation and opening operation with 7 pixels are done in horizontal both direction, filter out tiny gully and fine rule, obtain 8 thin profile point Cloth binary map, subsequently into step E3；

E3,8 connected domain scannings are carried out to 8 thin contoured profile binary map respectively, and calculate each thin contoured profile two It is worth area, center-of-mass coordinate and the minimum enclosed rectangle of each connected domain in figure, subsequently into step E4；

E4, for any one connected domain in each thin contoured profile binary map, when it meets following three filterings During at least one in condition, the pixel value of all pixels point in this connected domain is set to 0, obtains 8 stroke levels distribution two-values Figure, subsequently into step E5, wherein three filter conditions are respectively：

A, the area ratio of the area of connected domain and its minimum enclosed rectangle is less than 1/5；

B, the width of the minimum enclosed rectangle of connected domain is less than the 1/200 of original image width or more than original image width 1/2；

C, the ratio of width to height of the minimum enclosed rectangle of connected domain is less than 1/10 or more than 10；

E5, binary map is distributed for 8 obtained stroke levels, each stroke level distribution binary map is located as follows Reason：

E5-1, by stroke level be distributed binary map in each connected domain be labeled as C successively₁, C₂... ..., C_i, i is stroke level The number of connected domain in binary map is distributed, respectively by C₁, C₂... ..., C_iMark value be set to respective subscript value, i.e. C₁Mark Note value is 1, C₂Mark value be 2, by that analogy, C_iMark value be i, and define independent communication domain set P, independent communication domain Initial connected domain number is 0 in set P；

E5-2, from C₁, C₂... ..., C_iThe middle minimum enclosed rectangle for choosing any two connected domain is to axis of ordinates upslide Shadow, two vertical projection line segments are obtained, it is the overlap length between two vertical projection line segments to define disH, and is provided when two perpendicular DisH values are 0 when straight Projection Line Segment does not have overlapping, from C₁, C₂... ..., C_iThe middle minimum external square for choosing any two connected domain Shape obtains two floor projection line segments to projecting on axis of abscissas, and it is the most short distance between two floor projection line segments to define dis From, when two floor projection line segments are adjacent or overlapping, dis values are 0, when two floor projection line segments do not have overlapping, dis values For left side floor projection line segment right endpoint the distance between to the left end point of right side floor projection line segment；

E5-3, definition avgW are character mean breadth, and avgW initial value is C₁Minimum enclosed rectangle width, definition Sum is character row overall width, and sum initial value is avgW, and definition cnt is counter, and cnt initial value is 1；

Choose C₁And C₂Two vertical projection line segments being obtained to axis of ordinates upslide movie queen of minimum enclosed rectangle, when two Overlap length disH between bar vertical projection line segment is less than C₁Minimum enclosed rectangle height 1/4 when, by C₂Middle all pixels point Pixel value be set to 0, and by C₂Mark value be set to 0, when the overlap length disH between two vertical projection line segments is more than or equal to C₁ Minimum enclosed rectangle height 1/4 when, choose C₁And C₂Obtained to axis of abscissas upslide movie queen two of minimum enclosed rectangle Floor projection line segment, when the beeline dis between two floor projection line segments is less than or equal to 1.5 times of avgW, by C₂Mark Note value is set to C₁Mark value 1, while update sum value and add C for sum₂Minimum enclosed rectangle width, then by cnt value Add 1, and the value for updating avgW is sum divided by cnt, when the beeline dis between two floor projection line segments is more than the 1.5 of avgW Times when, by C₂It is put into independent communication domain set P；

Choose C₁And C₃Two vertical projection line segments being obtained to axis of ordinates upslide movie queen of minimum enclosed rectangle, when two Overlap length disH between bar vertical projection line segment is less than C₁Minimum enclosed rectangle height 1/4 when, by C₃Middle all pixels point Pixel value be set to 0, and by C₃Mark value be set to 0, when the overlap length disH between two vertical projection line segments is more than or equal to C₁ Minimum enclosed rectangle height 1/4 when, choose C₁And C₃Obtained to axis of abscissas upslide movie queen two of minimum enclosed rectangle Floor projection line segment, when the beeline dis between two floor projection line segments is less than or equal to 1.5 times of avgW, by C₃Mark Note value is set to C₁Mark value 1, while update sum value and add C for sum₃Minimum enclosed rectangle width, then by cnt value Add 1, and the value for updating avgW is sum divided by cnt, when the beeline dis between two floor projection line segments is more than the 1.5 of avgW Times when, by C₃It is put into independent communication domain set P；

By that analogy, until choosing C₁And C_iObtain to axis of ordinates upslide movie queen two of minimum enclosed rectangle it is vertical Projection Line Segment, when the overlap length disH between two vertical projection line segments is less than C₁Minimum enclosed rectangle height 1/4 when, will C_iThe pixel value of middle all pixels point is set to 0, and by C_iMark value be set to 0, the overlap length between two vertical projection line segments DisH is more than or equal to C₁Minimum enclosed rectangle height 1/4 when, choose C₁And C_iMinimum enclosed rectangle to axis of abscissas upslide Two floor projection line segments that movie queen obtains, when the beeline dis between two floor projection line segments is less than or equal to the 1.5 of avgW Times when, by C_iMark value be set to C₁Mark value 1, while update sum value and add C for sum_iMinimum enclosed rectangle width, Then cnt value is added 1, and the value for updating avgW is sum divided by cnt, as the beeline dis between two floor projection line segments During more than 1.5 times of avgW, by C_iIt is put into independent communication domain set P, subsequently into step E5-4；

Connected domain number in E5-4, statistical iteration connected domain set P, when the connected domain in independent communication domain set P When number is more than or equal to 2, into step E5-5, when the connected domain number in independent communication domain set P is 0 or 1, into step E5-7；E5-5, by independent communication domain set P multiple connected domains all take out, set P in independent communication domain is turned into empty set, Then multiple connected domains of taking-up being arranged in order from small to large according to subscript size, definition avgW is character mean breadth, AvgW initial value is the width of the minimum enclosed rectangle of first connected domain, and definition sum is character row overall width, and sum's is first Initial value is avgW, and definition cnt is counter, and cnt initial value is 1；

What the minimum enclosed rectangle of first connected domain of selection and second connected domain obtained to axis of ordinates upslide movie queen Two vertical projection line segments, when the overlap length disH between two vertical projection line segments is less than the minimum external of first connected domain Rectangular elevation 1/4 when, the pixel value of all pixels point in second connected domain is set to 0, and by the mark of second connected domain Note value is set to 0, when the overlap length disH between two vertical projection line segments is more than or equal to the minimum external square of first connected domain Shape height 1/4 when, choose the minimum enclosed rectangle of first connected domain and second connected domain to axis of abscissas upslide movie queen Two obtained floor projection line segments, when the beeline dis between two floor projection line segments is less than or equal to 1.5 times of avgW When, the mark value of second connected domain is set to the mark value of first connected domain, while the value for updating sum adds second for sum The width of the minimum enclosed rectangle of individual connected domain, cnt value is then added 1, and the value for updating avgW is sum divided by cnt, when two When beeline dis between bar floor projection line segment is more than 1.5 times of avgW, second connected domain is put into independent communication domain collection Close in P；

What the minimum enclosed rectangle of first connected domain of selection and the 3rd connected domain obtained to axis of ordinates upslide movie queen Two vertical projection line segments, when the overlap length disH between two vertical projection line segments is less than the minimum external of first connected domain Rectangular elevation 1/4 when, the pixel value of all pixels point in the 3rd connected domain is set to 0, and by the mark of the 3rd connected domain Note value is set to 0, when the overlap length disH between two vertical projection line segments is more than or equal to the minimum external square of first connected domain Shape height 1/4 when, choose the minimum enclosed rectangle of first connected domain and the 3rd connected domain to axis of abscissas upslide movie queen Two obtained floor projection line segments, when the beeline dis between two floor projection line segments is less than or equal to 1.5 times of avgW When, the mark value of the 3rd connected domain is set to the mark value of first connected domain, while the value for updating sum adds the 3rd for sum The width of the minimum enclosed rectangle of individual connected domain, cnt value is then added 1, and the value for updating avgW is sum divided by cnt, when two When beeline dis between bar floor projection line segment is more than 1.5 times of avgW, the 3rd connected domain is put into independent communication domain collection Close in P；

By that analogy, until choosing the minimum enclosed rectangle of first connected domain and last connected domain to axis of ordinates Two vertical projection line segments that upslide movie queen obtains, when the overlap length disH between two vertical projection line segments is less than first company When leading to the 1/4 of the minimum enclosed rectangle height in domain, the pixel value of all pixels point in last connected domain is set to 0, and will The mark value of last connected domain is set to 0, when the overlap length disH between two vertical projection line segments is more than or equal to first The minimum enclosed rectangle height of connected domain 1/4 when, choose first connected domain and the minimum external square of last connected domain Two floor projection line segments that shape obtains to axis of abscissas upslide movie queen, as the beeline dis between two floor projection line segments During less than or equal to 1.5 times of avgW, the mark value of last connected domain is set to the mark value of first connected domain, while more New sum value adds the width of the minimum enclosed rectangle of last connected domain for sum, cnt value then is added into 1, and update AvgW value is sum divided by cnt, will most when the beeline dis between two floor projection line segments is more than 1.5 times of avgW The latter connected domain is put into independent communication domain set P, subsequently into step E5-6；

E5-6, according to the method in step E5-5 set P in independent communication domain is reprocessed, until independent communication domain Connected domain number in set P is 0 or 1, subsequently into step E5-7；

E5-7, mark value identical connected domain in stroke level distribution binary map is designated as a marked region successively, and will Mark value of the mark value of connected domain as the marked region, area, the center-of-mass coordinate of each marked region are then calculated respectively And minimum enclosed rectangle, subsequently into step E6；

E6, binary map is distributed for 8 obtained stroke levels of step E5, each stroke level distribution binary map is made such as Lower processing：E6-1, for any two marked region, if the center-of-mass coordinate of a marked region is located at another marked region In minimum enclosed rectangle, then the mark value of the marked region is set to the mark value of another marked region, subsequently into step E6-2；

E6-2, mark value identical marked region in stroke level distribution binary map is designated as a line of text successively, then Width, height, area and the minimum enclosed rectangle of each line of text are calculated, obtains line of text binary map.

Described step F comprises the following steps：

F1,8 line of text binary maps for obtaining, for any one text in each line of text binary map OK, when it meets at least one in following three filter conditions, the pixel value of all pixels point in this article one's own profession is set to 0,8 candidate image binary maps are obtained, subsequently into step F2, wherein three filter conditions are respectively：

D, the height of line of text is less than the 1/20 of the minimum enclosed rectangle height of background profile；

E, the ratio of width to height of line of text is less than 7/10；

F, the area ratio of the area of line of text and its minimum enclosed rectangle is less than 1/10 or more than 4/5；

F2,8 candidate image binary maps for obtaining, for any one text in each candidate image binary map One's own profession, when it meets at least one in following two filter conditions, the pixel value of all pixels point in this article one's own profession is put For 0,8 character picture binary maps are obtained, subsequently into step F3, two of which filter condition is respectively：

G, the minimum enclosed rectangle of line of text is summed by row, obtains a series of row projection value, row projection value exceedes most 70% continuous columns of small boundary rectangle height is more than 0.7 times of minimum enclosed rectangle height；

H, the area of line of text is less than the 1/1000 of the minimum enclosed rectangle area of background profile binary map；

F3,8 character picture binary maps for obtaining, calculate formed in each character picture binary map first successively The area of all connected domains of each line of text and, when the area being calculated and minimum external less than background profile binary map Rectangular area 1/200 when, by corresponding character picture binary map delete, then calculate remaining character picture binary map respectively In all line of text minimum enclosed rectangle coordinate.

Described step G comprises the following steps：

G1, one and original image size identical blank image are created, successively by remaining character picture binary map The minimum enclosed rectangle of all line of text is marked in the corresponding position of blank image, obtains primary identification image, primary is known Minimum enclosed rectangle in other image is labeled as R successively₁, R₂... ..., R_j, j is minimum enclosed rectangle in primary identification image Number, and the centre coordinate of each minimum enclosed rectangle is calculated, subsequently into step G2；

G2, R is judged successively₂, R₃... ..., R_jCenter and R₁Position relationship, from R₁, R₂... ..., R_jMiddle Selection Center Positioned at R₁Internal or borderline all minimum enclosed rectangles, are put into R₁Merge in set；

R is judged successively₁, R₃... ..., R_jCenter and R₂Position relationship, from R₁, R₂... ..., R_jMiddle Selection Center is located at R₂Internal or borderline all minimum enclosed rectangles, are put into R₂Merge in set；

By that analogy, until judging R successively₁, R₂... ..., R_j-1Center and R_jPosition relationship, from R₁, R₂... ..., R_j Middle Selection Center is located at R_jInternal or borderline all minimum enclosed rectangles, are put into R_jMerge in set, subsequently into step G3；

G3, to R₁Merge set, R₂Merge set ... ..., R_jMerge set carry out condition merging, comprise the following steps：

G3-1, compare R₁Merge set and R₂Merge set, work as R₂Merge set and include R₁Merge at least one in set During minimum enclosed rectangle, to R₁Merge set and R₂Merge set and seek union, work as R₂Merge set and do not include R₁Merge in set During any one minimum enclosed rectangle, by R₂Merge set to be put into isolated area set Q；

Compare R₁Merge set and R₃Merge set, work as R₃Merge set and include R₁Merge at least one minimum in set During boundary rectangle, to R₁Merge set and R₃Merge set and seek union, work as R₃Merge set and do not include R₁Merge any in set During one minimum enclosed rectangle, by R₃Merge set to be put into isolated area set Q；

By that analogy, until comparing R₁Merge set and R_jMerge set, work as R_jMerge set and include R₁Merge in set During at least one minimum enclosed rectangle, to R₁Merge set and R_jMerge set and seek union, work as R_jMerge set and do not include R₁Merge During any one minimum enclosed rectangle in set, by R_jMerge set to be put into isolated area set Q, subsequently into step G3-2；

Merging set number in G3-2, statistical iteration regional ensemble Q, when the merging set in isolated area set Q When number is more than or equal to 2, into step G3-3, when the merging set number in isolated area set Q is 0 or 1, by isolated area Merging set in set Q is as a union, into step G4.

G3-3, by isolated area set Q multiple merging set all take out, isolated area set Q is turned into empty set, Then multiple merging set to taking-up are arranged in order from small to large according to subscript size, are compared first and are merged set and second It is individual to merge set, when merging set comprising first at least one minimum enclosed rectangle merged in set for second, to the One merges set and second merging set and seeks union, and merging set when second does not include appointing during first merging is gathered During one minimum enclosed rectangle of meaning, second merging set is put into isolated area set Q；

Compare first and merge set and the 3rd merging set, set is merged when the 3rd and includes first merging set In at least one minimum enclosed rectangle when, to first merge set and the 3rd merge set seek union, when the 3rd conjunction And when gathering any one minimum enclosed rectangle not included during first merging is gathered, the 3rd merging set is put into independence In regional ensemble Q；By that analogy, until comparing first merges set and last merging set, when last merging When set is comprising first at least one minimum enclosed rectangle merged in set, set and last conjunction are merged to first And gather and seek union, when last merging set does not include first any one minimum enclosed rectangle merged in set When, last merging set is put into isolated area set Q, subsequently into step G3-4.

G3-4, according to the method in step G3-3 isolated area set Q is reprocessed, until isolated area set Merging set number in Q is 0 or 1, using the merging set in isolated area set Q as a union, subsequently into step G4。

G4, each union obtained in step G3 handled as follows, statistics one is simultaneously concentrated all The apex coordinate of minimum enclosed rectangle, the minimum value that x coordinate is selected from apex coordinate are designated as x_min, select the maximum of x coordinate It is designated as x_max, the minimum value for selecting y-coordinate is designated as y_min, the maximum for selecting y-coordinate is designated as y_max, with (x_min, y_min), (x_max, y_min), (x_min, y_max), (x_max, y_max) four coordinates create rectangular area as summit, then from it is each and concentrate obtain it is all Rectangular area is the Chinese character region in original image.

The present invention realizes the detection to retail shop's label background profile by scan line sampling and Kmeans clusters, is advantageous to Retail shop's label is accurately positioned in natural scene, avoids the interference of other complex backgrounds；The present invention is real using the division of hsv color space Now to the colo(u)r breakup of retail shop's label Chinese character foreground, the Chinese character area of different colours on complete detection retail shop label can be ensured Domain, reduce the misclassification rate of Chinese character detection；The present invention is filtered by the analysis to each layer of Chinese character foreground, is realized to retail shop's label The detection of middle a variety of colors Chinese character, eventually through each layer Chinese character region is integrated, realize to label Chinese character area of retail shop in natural scene The automatic detection in domain.The present invention can effectively improve the degree of accuracy of Chinese character region detection, to being known based on the Chinese character under natural scene Other application is significant.

Brief description of the drawings

Fig. 1 is the schematic flow sheet of the present invention；

Fig. 2 is black and white division schematic diagram in the division of hsv color space of the present invention；

Fig. 3 is the division schematic diagram of other colors in the division of hsv color space of the present invention；

Fig. 4 is the original image scanned in the embodiment of the present invention under natural scene；

Fig. 5 is the area schematic of plate with trademark background profile in the embodiment of the present invention.

Embodiment

As shown in figure 1, retail shop's label Chinese character region automatic testing method in a kind of natural scene of the present invention, including Following steps：

A, the retail shop's label scanned in natural scene obtains original image, and the sample region figure of rectangle is intercepted from original image Picture, and the RGB triple channels of sample region image are deployed, forms the matrix of the row of M*N rows 3, and wherein M is the width of sample region image, N For the height of sample region image, Kmeans clusters, setting are carried out using every a line of matrix as a point of three dimensions respectively The classification number of cluster is 2, obtains two cluster centre points, respectively vectorial V1 and V2, subsequently into step B.

In order to ensure that vectorial V1 and V2 can reflect the main pixel distribution of sample region image, step A can be divided into following two Step；

Retail shop's label in A1, scanning natural scene obtains original image, and 1/3 length is more than in scan line and is less than 2/3 Appoint to take in the section of length and be a little used as basic point, point builds R*kR rectangularly-sampled area centered on this basic point, and wherein R is The width in rectangularly-sampled area, R value are more than the 1/7 of original image width, and 1/4, the k less than original image width adopts for rectangle The ratio of width to height in sample area, k span is 1/4<k<4, rectangularly-sampled area is cut from original image, obtains sample region image. Wherein, the restriction to rectangularly-sampled zone position and size can ensure that the main contents of original image are covered in rectangularly-sampled area, keep away Exempt from the missing inspection to different colours Chinese character region.

A2, the RGB triple channels expansion by sample region image, the matrix of the row of M*N rows 3 is formed, wherein M is sample region image Width, N are the height of sample region image, a point using every a line of matrix as three dimensions, are carried out using Euclidean distance Kmeans is clustered, and is set the classification number of cluster as 2, is obtained two cluster centre points, respectively vectorial V1 and V2, vectorial V1 and V2 The main pixel distribution of sample region image is reflected, for the identification to retail shop's label background profile in natural scene.

B, each pixel of original image is calculated respectively to vectorial V1 and V2 distance, and creates two and original image Size identical initializes binary map bg1 and bg2, and the pixel value for initializing all pixels point in binary map bg1 and bg2 is 0, Setpoint distance threshold X, compare each pixel of original image successively to vectorial V1 distance and X magnitude relationship, when original , will be with the pixel point coordinates identical picture in binary map bg1 is initialized when the pixel of image to vectorial V1 distance are less than X The pixel value of vegetarian refreshments is set to 1, obtains coarse contour binary map BG1；Compare each pixel of original image successively to vectorial V2's Distance and X magnitude relationship, when pixel to the vectorial V2 of original image distance are less than X, in binary map bg2 is initialized Pixel value with the pixel point coordinates identical pixel is set to 1, coarse contour binary map BG2 is obtained, respectively to coarse contour two Value figure BG1 and BG2 carry out connected domain analysis after doing Morphological scale-space, and area maximum is selected from coarse contour binary map BG1 and BG2 Connected domain repaired and filled, obtain background profile binary map.

Step B is the detection to original image background profile, and the background profile binary map to enable to obtain fully reflects The scope of retail shop's label, following steps can be classified as：

B1, each pixel in original image is labeled as 1,2 successively ... ..., n, n are pixel in original image Number, calculates the pixel x and vectorial V1 and V2 of original image Euclidean distance L1 (x) and L2 (x), wherein x=1, and 2 ... ..., n。

B2, two and original image size identical initialization binary map bg1 and bg2 are created, and two are initialized by two The pixel value of all pixels point is set to 0 in value figure bg1 and bg2, setpoint distance threshold X, X span should combine L1 (x) and L2 (x) is empirically determined, as the pixel x and vectorial V1 of original image Euclidean distance L1 (x) ＜ X, will initialize two Pixel value in value figure bg1 with pixel x position identical pixel is set to 1, obtains coarse contour binary map BG1, works as original graph , will be identical with pixel x position in initialization binary map bg2 when the pixel x and vectorial V2 of picture Euclidean distance L2 (x) ＜ X The pixel value of pixel be set to 1, obtain coarse contour binary map BG2.

B3, Morphological scale-space is done to coarse contour binary map BG1 and BG2 respectively, i.e., thick wheel is removed by erosion operation first Wide binary map BG1 and BG2 fine rule, the tiny of coarse contour binary map BG1 and BG2 is then removed by dilation operation and closed operation Gully, connected domain analysis finally is carried out to coarse contour binary map BG1 and BG2, face is selected from coarse contour binary map BG1 and BG2 The maximum connected domain of product is as background profile.

Black space, v<0.3or(v<0.5and s<=0.25)；

White space, v>=0.5and s<=0.25；

Red space, (h<1/12or h>=11/12) and v>0.3and s>=0.25；

Yellow space, h>=1/12and h<3/12and v>0.3and s>=0.25；

Vert space, h>=3/12and h<5/12and v>0.3and s>=0.25；

Cyan space, h>=5/12and h<7/12and v>0.3and s>=0.25；

Blue color space, h>=7/12and h<9/12and v>0.3and s>=0.25；

Pinkish red space, h>=9/12and h<11/12and v>0.3and s>=0.25；

H represents the tone in hsv color space in formula, and the saturation degree in behalf hsv color space, v represents hsv color Lightness in space, subsequently into step D.Hsv color space can reflect perception of the human eye to color well, according to above-mentioned Division rule divides to hsv color space, a variety of colors that can reflect comprehensively in natural scene.Fig. 2 is HSV of the present invention Black and white division schematic diagram in color space division, Fig. 3 are the division of other colors in the division of hsv color space of the present invention Schematic diagram.

D, 8 and original image size identical binary map are created, and by the pixel of all pixels point in every binary map Value is set to 0, and original image is transformed into hsv color space from rgb space, then judges original image in hsv color space Each pixel belonging to color type, all pixels point for belonging to same color type is mapped to pair of a binary map Opening position is answered, and the pixel value of the pixel of the corresponding position of binary map is set to 1, finally obtains 8 distribution of color two-values Figure, subsequently into step E.

E, 8 distribution of color binary maps are done with carrying out Morphological scale-space after computing again with background profile binary map respectively, The tiny gully in 8 distribution of color binary maps and fine rule are filtered out, obtains 8 thin contoured profile binary map, respectively to 8 thin wheel Exterior feature distribution binary map carries out connected domain scanning, obtains mark figure corresponding to connected graph using eight connectivity labelling method, and calculate each The attribute of connected domain, then the connected domain that Chinese-character stroke feature is not met in 8 thin contoured profile binary map is filtered, obtained Binary map is distributed to 8 stroke levels, is distributed the connected domain for meeting Chinese-character stroke feature in binary map to each stroke level respectively Merge, multiple connected domains after merging are designated as a marked region and determine mark value, for any two mark zone Domain, if the center-of-mass coordinate of a marked region is located in the minimum enclosed rectangle of another marked region, by the marked region Mark value be set to the mark value of another marked region, be distributed binary map finally for each stroke level, successively will mark Value identical marked region is designated as a line of text, 8 line of text binary maps is obtained, subsequently into step F.

What each connected domain in 8 distribution of color binary maps reflected is each stroke for forming Chinese character, by each The connected domain that stroke feature is not met in distribution of color binary map is filtered, and the connected domain to meeting stroke feature is closed And 8 line of text binary maps are obtained, specifically include following steps：

E1,8 distribution of color binary maps are done with background profile binary map respectively and computing, obtain 8 coarse contours distributions Binary map.

E2, Morphological scale-space is done to 8 coarse contour distribution binary maps respectively, i.e., to each coarse contour binary map vertical Closed operation and opening operation with 7 pixels are done in horizontal both direction, filter out tiny gully and fine rule, obtain 8 thin profile point Cloth binary map.

E3,8 connected domain scannings are carried out to 8 thin contoured profile binary map respectively, and calculate each thin contoured profile two It is worth area, center-of-mass coordinate and the minimum enclosed rectangle of each connected domain in figure.

E4, for any one connected domain in each thin contoured profile binary map, when it meets following three filterings During at least one in condition, the pixel value of all pixels point in this connected domain is set to 0, obtains 8 stroke levels distribution two-values Figure, wherein three filter conditions are respectively：

C, the ratio of width to height of the minimum enclosed rectangle of connected domain is less than 1/10 or more than 10.

E5, binary map is distributed for 8 obtained stroke levels of step E4, each stroke level distribution binary map is made such as Lower processing：

E5-1, by stroke level be distributed binary map in each connected domain be labeled as C successively₁, C₂... ..., C_i, i is stroke level The number of connected domain in binary map is distributed, respectively by C₁, C₂... ..., C_iMark value be set to respective subscript value, i.e. C₁Mark Note value is 1, C₂Mark value be 2, by that analogy, C_iMark value be i, and define independent communication domain set P, independent communication domain Initial connected domain number is 0 in set P.

E5-2, from C₁, C₂... ..., C_iThe middle minimum enclosed rectangle for choosing any two connected domain is to axis of ordinates upslide Shadow, two vertical projection line segments are obtained, it is the overlap length between two vertical projection line segments to define disH, and is provided when two perpendicular DisH values are 0 when straight Projection Line Segment does not have overlapping, from C₁, C₂... ..., C_iThe middle minimum external square for choosing any two connected domain Shape obtains two floor projection line segments to projecting on axis of abscissas, and it is the most short distance between two floor projection line segments to define dis From, when two floor projection line segments are adjacent or overlapping, dis values are 0, when two floor projection line segments do not have overlapping, dis values For left side floor projection line segment right endpoint the distance between to the left end point of right side floor projection line segment.

By that analogy, until choosing C₁And C_iObtain to axis of ordinates upslide movie queen two of minimum enclosed rectangle it is vertical Projection Line Segment, when the overlap length disH between two vertical projection line segments is less than C₁Minimum enclosed rectangle height 1/4 when, will C_iThe pixel value of middle all pixels point is set to 0, and by C_iMark value be set to 0, the overlap length between two vertical projection line segments DisH is more than or equal to C₁Minimum enclosed rectangle height 1/4 when, choose C₁And C_iMinimum enclosed rectangle to axis of abscissas upslide Two floor projection line segments that movie queen obtains, when the beeline dis between two floor projection line segments is less than or equal to the 1.5 of avgW Times when, by C_iMark value be set to C₁Mark value 1, while update sum value and add C for sum_iMinimum enclosed rectangle width, Then cnt value is added 1, and the value for updating avgW is sum divided by cnt, as the beeline dis between two floor projection line segments During more than 1.5 times of avgW, by C_iIt is put into independent communication domain set P, subsequently into step E5-4.

Connected domain number in E5-4, statistical iteration connected domain set P, when the connected domain in independent communication domain set P When number is more than or equal to 2, into step E5-5, when the connected domain number in independent communication domain set P is 0 or 1, into step E5-7。

E5-5, multiple connected domains in independent communication domain set P are all taken out, set P in independent communication domain is turned into empty Then multiple connected domains of taking-up are arranged in order by collection from small to large according to subscript size, definition avgW is character mean breadth, AvgW initial value is the width of the minimum enclosed rectangle of first connected domain, and definition sum is character row overall width, and sum's is first Initial value is avgW, and definition cnt is counter, and cnt initial value is 1；

E5-6, according to the method in step E5-5 set P in independent communication domain is reprocessed, until independent communication domain Connected domain number in set P is 0 or 1, subsequently into step E5-7.

E5-7, mark value identical connected domain in stroke level distribution binary map is designated as a marked region successively, and will Mark value of the mark value of connected domain as the marked region, area, the center-of-mass coordinate of each marked region are then calculated respectively And minimum enclosed rectangle, subsequently into step E6.

E6, binary map is distributed for 8 obtained stroke levels of step E5, each stroke level distribution binary map is made such as Lower processing：E6-1, for any two marked region, if the center-of-mass coordinate of a marked region is located at another marked region In minimum enclosed rectangle, then the mark value of the marked region is set to the mark value of another marked region, subsequently into step E6-2。

F, for any one line of text in 8 line of text binary maps, when the height of the minimum enclosed rectangle of line of text Less than the minimum enclosed rectangle height of background profile 1/20, the ratio of width to height of the minimum enclosed rectangle of line of text be less than 7/10 or The area of line of text and the area ratio of its minimum enclosed rectangle are less than 1/10 or during more than 4/5, by this article one's own profession from corresponding text Deleted in one's own profession binary map, 8 obtained character picture binary map, calculate all texts in each character picture binary map respectively The area of one's own profession and, when the area being calculated and less than background profile binary map minimum enclosed rectangle area 1/200 when, Corresponding character picture binary map is deleted, and the minimum for calculating all line of text in remaining character picture binary map respectively is outer The coordinate of rectangle is connect, subsequently into step G.

What each line of text in 8 line of text binary maps reflected is the Chinese character region of different colours in retail shop's label, is led to Cross and the line of text that Chinese-character writing rule is not met in each line of text binary map is filtered, the Chinese character area after being refined Domain, specifically include following steps：

F1,8 line of text binary maps for obtaining, for any one text in each line of text binary map OK, when it meets at least one in following three filter conditions, by all pixels point of each connected domain in this article one's own profession Mark value be set to 0,8 candidate image binary maps are obtained, wherein three filter conditions are respectively：

D, height is less than the 1/20 of the minimum enclosed rectangle height of background profile；

E, the ratio of width to height of one's own profession is less than 7/10；

F, the area ratio of area and its minimum enclosed rectangle is less than 1/10 or more than 4/5.

F2,8 candidate image binary maps for obtaining, for any one text in each candidate image binary map One's own profession, when it meets at least one in following two filter conditions, by all pixels of each connected domain in this article one's own profession The mark value of point is set to 0, obtains 8 character picture binary maps, two of which filter condition is respectively：

G, the minimum enclosed rectangle region of line of text is summed by row, obtains a series of row projection value, row projection value surpasses The continuous columns for crossing the 70% of minimum enclosed rectangle height is more than 0.7 times of minimum enclosed rectangle height；

H, the area of line of text is less than the 1/1000 of the minimum enclosed rectangle area of background profile binary map.

Line of text in each character picture binary map represents the Chinese character region of refining of different colours respectively, by will not Merged in Chinese character region of refining with color, you can obtain the Chinese character region in original image, specifically include following steps：

G1, one and original image size identical blank image are created, successively by remaining character picture binary map The minimum enclosed rectangle of all line of text is marked in the corresponding position of blank image, obtains primary identification image, primary is known Minimum enclosed rectangle in other image is labeled as R successively₁, R₂... ..., R_j, j is minimum enclosed rectangle in primary identification image Number, and the centre coordinate of each minimum enclosed rectangle is calculated, subsequently into step G2.

By that analogy, until judging R successively₁, R₂... ..., R_j-1Center and R_jPosition relationship, from R₁, R₂... ..., R_j Middle Selection Center is located at R_jInternal or borderline all minimum enclosed rectangles, are put into R_jMerge in set, subsequently into step G3。

By that analogy, until comparing R₁Merge set and R_jMerge set, work as R_jMerge set and include R₁Merge in set During at least one minimum enclosed rectangle, to R₁Merge set and R_jMerge set and seek union, work as R_jMerge set and do not include R₁Merge During any one minimum enclosed rectangle in set, by R_jMerge set to be put into isolated area set Q, subsequently into step G3-2。

G4, each union obtained in step G3 handled as follows, statistics one is simultaneously concentrated all The apex coordinate of minimum enclosed rectangle, the minimum value that x coordinate is selected from apex coordinate are designated as x_min, select the maximum of x coordinate It is designated as x_max, the minimum value for selecting y-coordinate is designated as y_min, the maximum for selecting y-coordinate is designated as y_max, with (x_min, y_min), (x_max, y_min), (x_min, y_max), (x_max, y_max) four coordinates form rectangular area as summit, then from it is each and concentrate obtain it is all Rectangular area is the Chinese character region in original image.

Illustrate retail shop's label Chinese character region automatic detection in natural scene of the present invention with reference to specific embodiment Method, comprise the following steps：

A, by scan line sampling and Kmeans cluster obtain reflection sample region image main pixel color vectorial V1 and V2, comprise the following steps：

Retail shop's label in A1, scanning natural scene, obtains original image as shown in Figure 4, the midpoint of scan line is made For basic point, the rectangularly-sampled area of one R*2R of point structure centered on this basic point, wherein R is the width in rectangularly-sampled area, and R's takes It is worth for the 1/6 of original image width, rectangularly-sampled area is cut from original image, obtains sample region image.Samples of green inframe Sample region image contain the color of main word and background in original image.

A2, the RGB triple channels expansion by sample region image, the matrix of the row of M*N rows 3 is formed, wherein M is sample region image Width, N are the height of sample region image, a point using every a line of matrix as three dimensions, are carried out using Euclidean distance Kmeans is clustered, and is set the classification number of cluster as 2, is obtained two cluster centre points, respectively vectorial V1 and V2, vectorial V1 and V2 Reflect the main pixel color of sample region image.

B, the background profile of original image is determined using vectorial V1 and V2, is comprised the following steps：

B2, two and original image size identical initialization binary map bg1 and bg2 are created, and two are initialized by two The pixel value of all pixels point is set to 0 in value figure bg1 and bg2, and setpoint distance threshold X is 50, when original image pixel x with During vectorial V1 Euclidean distance L1 (x) ＜ 50, the picture in binary map bg1 with pixel x position identical pixel will be initialized Plain value is set to 1, coarse contour binary map BG1 is obtained, as the pixel x and vectorial V2 of original image Euclidean distance L2 (x) ＜ 50 When, the pixel value initialized in binary map bg2 with pixel x position identical pixel is set to 1, obtains coarse contour binary map BG2。

Hole in B4, filling background profile, and the depression on background profile edge is repaired, obtain background profile two-value Figure, as shown in Figure 5.Background profile binary map can accurately reflect the position of retail shop's label in original image, effectively avoid background Outside interference of the noise to Chinese character region detection.

Black space, v<0.3or(v<0.5and s<=0.25)；

White space, v>=0.5and s<=0.25；

Red space, (h<1/12or h>=11/12) and v>0.3and s>=0.25；

Yellow space, h>=1/12and h<3/12and v>0.3and s>=0.25；

Vert space, h>=3/12and h<5/12and v>0.3and s>=0.25；

Cyan space, h>=5/12and h<7/12and v>0.3and s>=0.25；

Blue color space, h>=7/12and h<9/12and v>0.3and s>=0.25；

Pinkish red space, h>=9/12and h<11/12and v>0.3and s>=0.25；

H represents the tone in hsv color space in formula, and the saturation degree in behalf hsv color space, v represents hsv color Lightness in space, subsequently into step D.Hsv color space can reflect perception of the human eye to color well, according to above-mentioned Division rule divides to hsv color space, a variety of colors that can reflect comprehensively in natural scene.

D, colo(u)r breakup is carried out to original image, detailed process is：Create 8 and original image size identical two-value Figure, and the pixel value of all pixels point in every binary map is set to 0, original image is transformed into hsv color sky from rgb space Between in, then judge the color type belonging to each pixel of original image in hsv color space, same color class will be belonged to The all pixels point of type is mapped to the corresponding position of a binary map, and by the picture of the pixel of the corresponding position of binary map Plain value is set to 1,8 distribution of color binary maps is finally obtained, subsequently into step E.

E, Chinese character connected domain analysis is carried out to 8 distribution of color binary maps, specifically includes following steps：

By that analogy, until choosing the minimum enclosed rectangle of first connected domain and last connected domain to axis of ordinates Two vertical projection line segments that upslide movie queen obtains, when the overlap length disH between two vertical projection line segments is less than first company When leading to the 1/4 of the minimum enclosed rectangle height in domain, the pixel value of all pixels point in last connected domain is set to 0, and will The mark value of last connected domain is set to 0, when the overlap length disH between two vertical projection line segments is more than or equal to first The minimum enclosed rectangle height of connected domain 1/4 when, choose first connected domain and the minimum external square of last connected domain Two floor projection line segments that shape obtains to axis of abscissas upslide movie queen, as the beeline dis between two floor projection line segments During less than or equal to 1.5 times of avgW, the mark value of last connected domain is set to the mark value of first connected domain, while more New sum value adds the width of the minimum enclosed rectangle of last connected domain for sum, cnt value then is added into 1, and update AvgW value is sum divided by cnt, will most when the beeline dis between two floor projection line segments is more than 1.5 times of avgW The latter connected domain is put into independent communication domain set P, subsequently into step E5-6.

E6, binary map is distributed for 8 obtained stroke levels, each stroke level distribution binary map is located as follows Reason：

E6-1, for any two marked region, if the center-of-mass coordinate of a marked region is located at another marked region Minimum enclosed rectangle in, then the mark value of the marked region is set to the mark value of another marked region, subsequently into step Rapid E6-2.

F, what each line of text in 8 line of text binary maps reflected is the Chinese character region of different colours in retail shop's label, By being filtered to the line of text that Chinese-character writing rule is not met in each line of text binary map, the Chinese character area after being refined Domain, specifically include following steps：

E, the ratio of width to height of line of text is less than 7/10；

F, the area ratio of the area of line of text and its minimum enclosed rectangle is less than 1/10 or more than 4/5.

G, the line of text in 8 character picture binary maps represents the Chinese character region of refining of different colours respectively, by will not Merged in Chinese character region of refining with color, you can obtain the Chinese character region in original image, specifically include following steps：

The present invention realizes the detection to retail shop's label background profile by scan line sampling and Kmeans clusters, is advantageous to Retail shop's label is accurately positioned in natural scene, avoids the interference of other complex backgrounds；Realized using the division of hsv color space to business Spread the colo(u)r breakup of label Chinese character foreground and analysis filtering is carried out to each layer of Chinese character foreground, complete detection business can be ensured The Chinese character region of different colours on label is spread, reduces the misclassification rate of Chinese character detection；Eventually through each layer Chinese character region is integrated, realize To the automatic detection in retail shop's label Chinese character region in natural scene.The present invention can effectively improve the accurate of Chinese character region detection Degree, it is significant to the application based on the Chinese Character Recognition under natural scene.

Claims

1. retail shop's label Chinese character region automatic testing method in natural scene, it is characterised in that comprise the following steps：

A, the retail shop's label scanned in natural scene obtains original image, and the sample region image of rectangle is intercepted from original image, And deploy the RGB triple channels of sample region image, the matrix of the row of M*N rows 3 is formed, wherein M is the width of sample region image, and N is The height of sample region image, carries out Kmeans clusters using every a line of matrix as a point of three dimensions respectively, and setting is poly- The classification number of class is 2, obtains two cluster centre points, respectively vectorial V1 and V2, subsequently into step B；

B, each pixel of original image is calculated respectively to vectorial V1 and V2 distance, and creates two and original image size Identical initializes binary map bg1 and bg2, and the pixel value for initializing all pixels point in binary map bg1 and bg2 is 0, setting Distance threshold X, compare each pixel of original image successively to vectorial V1 distance and X magnitude relationship, work as original image Pixel to vectorial V1 distance when being less than X, will be with the pixel point coordinates identical pixel in binary map bg1 is initialized Pixel value be set to 1, obtain coarse contour binary map BG1；Compare each pixel of original image successively to vectorial V2 distance , will be with binary map bg2 is initialized when pixel to the vectorial V2 of original image distance are less than X with X magnitude relationship The pixel value of the pixel point coordinates identical pixel is set to 1, coarse contour binary map BG2 is obtained, respectively to coarse contour binary map BG1 and BG2 does Morphological scale-space and carries out connected domain analysis, and the maximum company of area is selected from coarse contour binary map BG1 and BG2 Logical domain is used as background profile after being repaired and being filled, and obtains background profile binary map；

C, hsv color space is divided into black, white, red, yellow, green, cyan, blueness, pinkish red eight pieces of color spaces, The division rule of eight pieces of color spaces is as follows：

Black space, v<0.3 or(v<0.5 and s<=0.25)；

White space, v>=0.5 and s<=0.25；

Red space, (h<1/12 or h>=11/12) and v>0.3 and s>=0.25；

Yellow space, h>=1/12 and h<3/12 and v>0.3 and s>=0.25；

Vert space, h>=3/12 and h<5/12 and v>0.3 and s>=0.25；

Cyan space, h>=5/12 and h<7/12 and v>0.3 and s>=0.25；

Blue color space, h>=7/12 and h<9/12 and v>0.3 and s>=0.25；

Pinkish red space, h>=9/12 and h<11/12 and v>0.3 and s>=0.25；

H represents the tone in hsv color space in formula, and the saturation degree in behalf hsv color space, v represents hsv color space In lightness, subsequently into step D；

D, 8 and original image size identical binary map are created, and the pixel value of all pixels point in every binary map is put For 0, original image is transformed into hsv color space from rgb space, then judges the every of original image in hsv color space Color type belonging to individual pixel, all pixels point for belonging to same color type is mapped to the corresponding position of a binary map Place is put, and the pixel value of the pixel of corresponding position in binary map is set to 1, finally obtains 8 distribution of color binary maps, so Enter step E afterwards；

E, 8 distribution of color binary maps are done with background profile binary map and carries out Morphological scale-space after computing again respectively, filter out 8 The tiny gully in distribution of color binary map and fine rule are opened, obtains 8 thin contoured profile binary map, respectively to 8 thin profile point Cloth binary map carries out connected domain scanning, obtains mark figure corresponding to connected graph using eight connectivity labelling method, and calculate each connection The attribute in domain, then the connected domain that Chinese-character stroke feature is not met in 8 thin contoured profile binary map is filtered, obtains 8 Stroke level distribution binary map is opened, is distributed the connected domain progress for meeting Chinese-character stroke feature in binary map to each stroke level respectively Merge, multiple connected domains after merging are designated as a marked region and determine mark value, for any two marked region, if The center-of-mass coordinate of one marked region is located in the minimum enclosed rectangle of another marked region, then by the mark of the marked region Value is set to the mark value of another marked region, is distributed binary map finally for each stroke level, successively that mark value is identical Marked region be designated as a line of text, 8 line of text binary maps are obtained, subsequently into step F；

F, for any one line of text in 8 line of text binary maps, when the height of the minimum enclosed rectangle of line of text is less than The 1/20 of the minimum enclosed rectangle height of background profile, the ratio of width to height of the minimum enclosed rectangle of line of text is less than 7/10 or text The area ratio of capable area and its minimum enclosed rectangle is less than 1/10 or during more than 4/5, by this article one's own profession from corresponding line of text Deleted in binary map, 8 obtained character picture binary map, calculate all line of text in each character picture binary map respectively Area and, when the area being calculated and less than background profile binary map minimum enclosed rectangle area 1/200 when, will be right The character picture binary map answered is deleted, and calculates the minimum external square of all line of text in remaining character picture binary map respectively The coordinate of shape, subsequently into step G；

G, one and original image size identical blank image are created, successively by all texts in remaining character picture binary map The minimum enclosed rectangle of one's own profession is marked in the corresponding position of blank image, is obtained primary identification image, is respectively identified primary The border being made up of in image two or more minimum enclosed rectangle is in the isolated area of irregular polygon as a cog region Domain, for each identification region, the coordinate on each summit is counted, and the minimum value for selecting from apex coordinate x coordinate is designated as x_min, the maximum for selecting x coordinate is designated as x_max, the minimum value for selecting y-coordinate is designated as y_min, the maximum for selecting y-coordinate is designated as y_max, with (x_min, y_min), (x_max, y_min), (x_min, y_max), (x_max, y_max) four coordinates create rectangular area as summit, then Remaining minimum enclosed rectangle composition in all rectangular areas obtained from each identification region and primary identification image Region is the Chinese character region in original image.

2. retail shop's label Chinese character region automatic testing method in natural scene as claimed in claim 1, it is characterised in that described Step A comprise the following steps：

Retail shop's label in A1, scanning natural scene obtains original image, is less than 2/3 length more than 1/3 length in scan line Section in appoint take a little be used as basic point, centered on this basic point point build a R*kR rectangularly-sampled area, wherein R is rectangle The width of sample region, R value are more than the 1/7 of original image width, and 1/4, the k less than original image width is rectangularly-sampled area The ratio of width to height, k span is 1/4<k<4, rectangularly-sampled area is cut from original image, obtains sample region image, then Into step A2；

A2, the RGB triple channels expansion by sample region image, the matrix of the row of M*N rows 3 is formed, wherein M is the width of sample region image Degree, N are the height of sample region image, a point using every a line of matrix as three dimensions, are carried out using Euclidean distance Kmeans is clustered, and is set the classification number of cluster as 2, is obtained two cluster centre points, respectively vectorial V1 and V2.

3. retail shop's label Chinese character region automatic testing method in natural scene as claimed in claim 1, it is characterised in that described Step B comprise the following steps：

B1, by each pixel in original image successively be labeled as 1,2 ... ..., n, n be original image in pixel Number, calculates the pixel x and vectorial V1 and V2 of original image Euclidean distance L1 (x) and L2 (x), wherein x=1,2 ... ..., n, Subsequently into step B2；

B2, two and original image size identical initialization binary map bg1 and bg2 are created, and binary map is initialized by two The pixel value of all pixels point is set to 0 in bg1 and bg2, setpoint distance threshold X, as the pixel x and vectorial V1 of original image During Euclidean distance L1 (x) ＜ X, the pixel value initialized in binary map bg1 with pixel x position identical pixel is set to 1, Coarse contour binary map BG1 is obtained, as the pixel x and vectorial V2 of original image Euclidean distance L2 (x) ＜ X, will be initialized Pixel value in binary map bg2 with pixel x position identical pixel is set to 1, obtains coarse contour binary map BG2, Ran Houjin Enter step B3；

B3, Morphological scale-space is done to coarse contour binary map BG1 and BG2 respectively, i.e., coarse contour two is removed by erosion operation first Value figure BG1 and BG2 fine rule, coarse contour binary map BG1 and BG2 tiny ditch are then removed by dilation operation and closed operation Gully, connected domain analysis finally is carried out to coarse contour binary map BG1 and BG2, area is selected from coarse contour binary map BG1 and BG2 Maximum connected domain is as background profile, subsequently into step B4；

Hole in B4, filling background profile, and the depression on background profile edge is repaired, obtain background profile binary map.

4. retail shop's label Chinese character region automatic testing method in natural scene as claimed in claim 1, it is characterised in that：It is described Step E comprise the following steps：

E1,8 distribution of color binary maps are done with background profile binary map respectively and computing, obtain 8 coarse contours distribution two-values Figure, subsequently into step E2；

E2, Morphological scale-space is done to 8 coarse contour distribution binary maps respectively, i.e., to each coarse contour binary map in vertical and water Closed operation and the opening operation of 7 pixels are done in flat both direction, tiny gully and fine rule is filtered out, obtains 8 thin contoured profile two Value figure, subsequently into step E3；

E3,8 connected domain scannings are carried out to 8 thin contoured profile binary map respectively, and calculate each thin contoured profile binary map In each connected domain area, center-of-mass coordinate and minimum enclosed rectangle, subsequently into step E4；

E4, for any one connected domain in each thin contoured profile binary map, when it meets following three filter conditions In it is at least one when, the pixel value of all pixels point in this connected domain is set to 0, obtains 8 stroke levels distribution binary maps, so Enter step E5 afterwards, wherein three filter conditions are respectively：

B, the width of the minimum enclosed rectangle of connected domain is less than the 1/200 or 1/ more than original image width of original image width 2；

E5, binary map is distributed for 8 obtained stroke levels, each stroke level distribution binary map is handled as follows：

E5-1, by stroke level be distributed binary map in each connected domain be labeled as C successively₁, C₂... ..., C_i, i is the distribution of stroke level The number of connected domain in binary map, respectively by C₁, C₂... ..., C_iMark value be set to respective subscript value, i.e. C₁Mark value For 1, C₂Mark value be 2, by that analogy, C_iMark value be i, and define independent communication domain set P, independent communication domain set P In initial connected domain number be 0；

E5-2, from C₁, C₂... ..., C_iThe middle minimum enclosed rectangle for choosing any two connected domain obtains to projecting on axis of ordinates To two vertical projection line segments, it is the overlap length between two vertical projection line segments to define disH, and provides to throw vertically when two DisH values are 0 when hachure section does not have overlapping, from C₁, C₂... ..., C_iIt is middle choose any two connected domain minimum enclosed rectangle to Projected on axis of abscissas, obtain two floor projection line segments, it is the beeline between two floor projection line segments to define dis, when Dis values are 0 when two floor projection line segments are adjacent or overlapping, and when two floor projection line segments do not have overlapping, dis values are left side The right endpoint of floor projection line segment is the distance between to the left end point of right side floor projection line segment；

E5-3, definition avgW are character mean breadth, and avgW initial value is C₁Minimum enclosed rectangle width, define sum be Character row overall width, sum initial value is avgW, and definition cnt is counter, and cnt initial value is 1；

Choose C₁And C₂Two vertical projection line segments being obtained to axis of ordinates upslide movie queen of minimum enclosed rectangle, when two it is perpendicular Deliver directly the intersegmental overlap length disH of hachure and be less than C₁Minimum enclosed rectangle height 1/4 when, by C₂The picture of middle all pixels point Plain value is set to 0, and by C₂Mark value be set to 0, when the overlap length disH between two vertical projection line segments is more than or equal to C₁Most Small boundary rectangle height 1/4 when, choose C₁And C₂Two levels being obtained to axis of abscissas upslide movie queen of minimum enclosed rectangle Projection Line Segment, when the beeline dis between two floor projection line segments is less than or equal to 1.5 times of avgW, by C₂Mark value It is set to C₁Mark value 1, while update sum value and add C for sum₂Minimum enclosed rectangle width, cnt value is then added 1, And the value for updating avgW is sum divided by cnt, when the beeline dis between two floor projection line segments is more than 1.5 times of avgW When, by C₂It is put into independent communication domain set P；

Choose C₁And C₃Two vertical projection line segments being obtained to axis of ordinates upslide movie queen of minimum enclosed rectangle, when two it is perpendicular Deliver directly the intersegmental overlap length disH of hachure and be less than C₁Minimum enclosed rectangle height 1/4 when, by C₃The picture of middle all pixels point Plain value is set to 0, and by C₃Mark value be set to 0, when the overlap length disH between two vertical projection line segments is more than or equal to C₁Most Small boundary rectangle height 1/4 when, choose C₁And C₃Two levels being obtained to axis of abscissas upslide movie queen of minimum enclosed rectangle Projection Line Segment, when the beeline dis between two floor projection line segments is less than or equal to 1.5 times of avgW, by C₃Mark value It is set to C₁Mark value 1, while update sum value and add C for sum₃Minimum enclosed rectangle width, cnt value is then added 1, And the value for updating avgW is sum divided by cnt, when the beeline dis between two floor projection line segments is more than 1.5 times of avgW When, by C₃It is put into independent communication domain set P；

By that analogy, until choosing C₁And C_iTwo vertical projections being obtained to axis of ordinates upslide movie queen of minimum enclosed rectangle Line segment, when the overlap length disH between two vertical projection line segments is less than C₁Minimum enclosed rectangle height 1/4 when, by C_iIn The pixel value of all pixels point is set to 0, and by C_iMark value be set to 0, the overlap length between two vertical projection line segments DisH is more than or equal to C₁Minimum enclosed rectangle height 1/4 when, choose C₁And C_iMinimum enclosed rectangle to axis of abscissas upslide Two floor projection line segments that movie queen obtains, when the beeline dis between two floor projection line segments is less than or equal to the 1.5 of avgW Times when, by C_iMark value be set to C₁Mark value 1, while update sum value and add C for sum_iMinimum enclosed rectangle width, Then cnt value is added 1, and the value for updating avgW is sum divided by cnt, as the beeline dis between two floor projection line segments During more than 1.5 times of avgW, by C_iIt is put into independent communication domain set P, subsequently into step E5-4；

Connected domain number in E5-4, statistical iteration connected domain set P, when the connected domain number in independent communication domain set P is big When equal to 2, into step E5-5, when the connected domain number in independent communication domain set P is 0 or 1, into step E5-7；

E5-5, by independent communication domain set P multiple connected domains all take out, set P in independent communication domain is turned into empty set, so Multiple connected domains of taking-up are arranged in order from small to large according to subscript size afterwards, definition avgW is character mean breadth, avgW Initial value be the width of the minimum enclosed rectangle of first connected domain, definition sum is character row overall width, sum initial value For avgW, definition cnt is counter, and cnt initial value is 1；

Choose two that the minimum enclosed rectangle of first connected domain and second connected domain obtains to axis of ordinates upslide movie queen Vertical projection line segment, when the overlap length disH between two vertical projection line segments is less than the minimum enclosed rectangle of first connected domain Height 1/4 when, the pixel value of all pixels point in second connected domain is set to 0, and by the mark value of second connected domain 0 is set to, the minimum enclosed rectangle that first connected domain is more than or equal to as the overlap length disH between two vertical projection line segments is high Degree 1/4 when, the minimum enclosed rectangle for choosing first connected domain and second connected domain obtains to axis of abscissas upslide movie queen Two floor projection line segments, will when the beeline dis between two floor projection line segments is less than or equal to 1.5 times of avgW The mark value of second connected domain is set to the mark value of first connected domain, while the value for updating sum adds second connection for sum The width of the minimum enclosed rectangle in domain, cnt value is then added 1, and the value for updating avgW is sum divided by cnt, when two levels When beeline dis between Projection Line Segment is more than 1.5 times of avgW, second connected domain is put into independent communication domain set P；

Choose two articles that the minimum enclosed rectangle of first connected domain and the 3rd connected domain obtains to axis of ordinates upslide movie queen Vertical projection line segment, when the overlap length disH between two vertical projection line segments is less than the minimum enclosed rectangle of first connected domain Height 1/4 when, the pixel value of all pixels point in the 3rd connected domain is set to 0, and by the mark value of the 3rd connected domain 0 is set to, the minimum enclosed rectangle that first connected domain is more than or equal to as the overlap length disH between two vertical projection line segments is high Degree 1/4 when, the minimum enclosed rectangle for choosing first connected domain and the 3rd connected domain obtains to axis of abscissas upslide movie queen Two floor projection line segments, will when the beeline dis between two floor projection line segments is less than or equal to 1.5 times of avgW The mark value of 3rd connected domain is set to the mark value of first connected domain, while the value for updating sum adds the 3rd connection for sum The width of the minimum enclosed rectangle in domain, cnt value is then added 1, and the value for updating avgW is sum divided by cnt, when two levels When beeline dis between Projection Line Segment is more than 1.5 times of avgW, the 3rd connected domain is put into independent communication domain set P；

By that analogy, until choosing the minimum enclosed rectangle of first connected domain and last connected domain to axis of ordinates upslide Two vertical projection line segments that movie queen obtains, when the overlap length disH between two vertical projection line segments is less than first connected domain Minimum enclosed rectangle height 1/4 when, the pixel value of all pixels point in last connected domain is set to 0, and will be last The mark value of one connected domain is set to 0, when the overlap length disH between two vertical projection line segments is more than or equal to first connection The minimum enclosed rectangle height in domain 1/4 when, choose the minimum enclosed rectangle of first connected domain and last connected domain to Two floor projection line segments that axis of abscissas upslide movie queen obtains, when the beeline dis between two floor projection line segments is less than During equal to 1.5 times of avgW, the mark value of last connected domain is set to the mark value of first connected domain, updated simultaneously Sum value is the width for the minimum enclosed rectangle that sum adds last connected domain, cnt value then is added into 1, and update avgW Value be sum divided by cnt, when the beeline dis between two floor projection line segments is more than 1.5 times of avgW, by last Individual connected domain is put into independent communication domain set P, subsequently into step E5-6；

E5-6, according to the method in step E5-5 set P in independent communication domain is reprocessed, until independent communication domain is gathered Connected domain number in P is 0 or 1, subsequently into step E5-7；

E5-7, mark value identical connected domain in stroke level distribution binary map is designated as a marked region successively, and will connection Mark value of the mark value in domain as the marked region, the area of each marked region, center-of-mass coordinate and most are then calculated respectively Small boundary rectangle, subsequently into step E6；

E6, binary map is distributed for 8 obtained stroke levels of step E5, each stroke level distribution binary map is located as follows Reason：

E6-1, for any two marked region, if the center-of-mass coordinate of a marked region is located at another marked region most In small boundary rectangle, then the mark value of the marked region is set to the mark value of another marked region, subsequently into step E6- 2；

E6-2, mark value identical marked region in stroke level distribution binary map is designated as a line of text successively, then calculated Width, height, area and the minimum enclosed rectangle of each line of text, obtain line of text binary map.

5. retail shop's label Chinese character region automatic testing method in natural scene as claimed in claim 1, it is characterised in that：It is described Step F comprise the following steps：

F1,8 line of text binary maps for obtaining, for any one line of text in each line of text binary map, when When it meets at least one in following three filter conditions, the pixel value of all pixels point in this article one's own profession is set to 0, obtained 8 candidate image binary maps, subsequently into step F2, wherein three filter conditions are respectively：

E, the ratio of width to height of line of text is less than 7/10；

F2,8 candidate image binary maps for obtaining, for any one text in each candidate image binary map OK, when it meets at least one in following two filter conditions, the pixel value of all pixels point in this article one's own profession is set to 0,8 character picture binary maps are obtained, subsequently into step F3, two of which filter condition is respectively：

G, the minimum enclosed rectangle of line of text is summed by row, obtains a series of row projection value, row projection value exceedes minimum outer The continuous columns for connecing the 70% of rectangular elevation is more than 0.7 times of minimum enclosed rectangle height；

F3,8 character picture binary maps for obtaining, calculate successively first formed in each character picture binary map it is each The area of all connected domains of line of text and, when the area being calculated and the minimum enclosed rectangle less than background profile binary map Area 1/200 when, by corresponding character picture binary map delete, then calculate institute in remaining character picture binary map respectively There is the coordinate of the minimum enclosed rectangle of line of text.

6. retail shop's label Chinese character region automatic testing method in natural scene as claimed in claim 1, it is characterised in that：It is described Step G comprise the following steps：

G1, one and original image size identical blank image are created, will owned successively in remaining character picture binary map The minimum enclosed rectangle of line of text is marked in the corresponding position of blank image, obtains primary identification image, by primary identification figure Minimum enclosed rectangle as in is labeled as R successively₁, R₂... ..., R_j, j is the number of minimum enclosed rectangle in primary identification image, And the centre coordinate of each minimum enclosed rectangle is calculated, subsequently into step G2；

G2, R is judged successively₂, R₃... ..., R_jCenter and R₁Position relationship, from R₁, R₂... ..., R_jMiddle Selection Center is located at R₁ Internal or borderline all minimum enclosed rectangles, are put into R₁Merge in set；

R is judged successively₁, R₃... ..., R_jCenter and R₂Position relationship, from R₁, R₂... ..., R_jMiddle Selection Center is located at R₂It is interior Portion or borderline all minimum enclosed rectangles, are put into R₂Merge in set；

By that analogy, until judging R successively₁, R₂... ..., R_j-1Center and R_jPosition relationship, from R₁, R₂... ..., R_jMiddle choosing Take and be centrally located at R_jInternal or borderline all minimum enclosed rectangles, are put into R_jMerge in set, subsequently into step G3；

G3-1, compare R₁Merge set and R₂Merge set, work as R₂Merge set and include R₁Merge at least one minimum in set During boundary rectangle, to R₁Merge set and R₂Merge set and seek union, work as R₂Merge set and do not include R₁Merge any in set During one minimum enclosed rectangle, by R₂Merge set to be put into isolated area set Q；

Compare R₁Merge set and R₃Merge set, work as R₃Merge set and include R₁Merge at least one external square of minimum in set During shape, to R₁Merge set and R₃Merge set and seek union, work as R₃Merge set and do not include R₁Merge set in any one most During small boundary rectangle, by R₃Merge set to be put into isolated area set Q；

By that analogy, until comparing R₁Merge set and R_jMerge set, work as R_jMerge set and include R₁Merge in set at least During one minimum enclosed rectangle, to R₁Merge set and R_jMerge set and seek union, work as R_jMerge set and do not include R₁Merge set In any one minimum enclosed rectangle when, by R_jMerge set to be put into isolated area set Q, subsequently into step G3-2；

Merging set number in G3-2, statistical iteration regional ensemble Q, when the merging set number in isolated area set Q is big When equal to 2, into step G3-3, when the merging set number in isolated area set Q is 0 or 1, by isolated area set Merging set in Q is as a union, into step G4；

G3-3, by isolated area set Q multiple merging set all take out, isolated area set Q is turned into empty set, then Multiple merging set to taking-up are arranged in order from small to large according to subscript size, are compared first and are merged set and second conjunction And gather, when merging set comprising first at least one minimum enclosed rectangle merged in set for second, to first Merge to gather and seek union with second merging set, merge when second any one during set is gathered not comprising first merging During individual minimum enclosed rectangle, second merging set is put into isolated area set Q；

Compare first merging set and the 3rd merging is gathered, when the 3rd merging set is comprising in first merging set During at least one minimum enclosed rectangle, set is merged to first and the 3rd merges set and seek union, merges collection when the 3rd When closing any one minimum enclosed rectangle not included during first merging is gathered, the 3rd merging set is put into isolated area In set Q；By that analogy, until comparing first merges set and last merging set, when last merging set During comprising first at least one minimum enclosed rectangle merged in set, set is merged to first and last merging collects Union is sought in conjunction, will when last merging set does not include first any one minimum enclosed rectangle merged in set Last merges set and is put into isolated area set Q, subsequently into step G3-4；

G3-4, according to the method in step G3-3 isolated area set Q is reprocessed, until in isolated area set Q Merging set number be 0 or 1, using the merging set in isolated area set Q as a union, subsequently into step G4；

G4, each union obtained in step G3 handled as follows, statistics one simultaneously concentrates all minimums The apex coordinate of boundary rectangle, the minimum value that x coordinate is selected from apex coordinate are designated as x_min, the maximum for selecting x coordinate is designated as x_max, the minimum value for selecting y-coordinate is designated as y_min, the maximum for selecting y-coordinate is designated as y_max, with (x_min, y_min), (x_max, y_min), (x_min, y_max), (x_max, y_max) four coordinates create rectangular area as summit, then from each and concentrate obtained all rectangles Region is the Chinese character region in original image.