JP6540220B2 - IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND IMAGE PROCESSING PROGRAM - Google Patents

Description

  The present invention relates to a technique for detecting regular patterns in an image.

  Video that includes regular patterns (striped, swirled, concentric, etc.) can cause viewers to develop photosensitive seizures. A photosensitive seizure is a symptom of an abnormal response to a light stimulus that jumps into the eye, and is a seizure resembling a hemorrhoid accompanied by convulsions and impaired consciousness.

  In order to suppress such an adverse effect on the human body, attempts are being made worldwide to suppress the distribution of video content including regular patterns. For example, the International Telecommunication Union (ITU) has established guidelines for reducing video-induced photosensitive seizures (see Non-Patent Document 1). In Non-Patent Document 1, the ITU recommends video distribution organizations to make video content producers aware of videos that may cause photosensitive seizures, including regular patterns. Furthermore, in Japan, the Japan Broadcasting Association and the Japan Civil Broadcasting Association have established guidelines especially for animation production (see Non-Patent Document 2). In Non-Patent Document 2, a warning is made so as to avoid the regular pattern from occupying a large part of the screen.

  However, in order to find out a regular pattern unintentionally mixed in video content, it is necessary to watch all the video content, which is very time-consuming. In addition, due to oversight by manual operation and differences in judgment criteria of workers, the point that the accuracy is unstable becomes a problem. Therefore, a technique for automatically detecting a regular pattern from an image included in a video is required.

  The regular pattern includes a plurality of specific patterns with high contrast as shown in FIG. Patent Document 1 describes an example of a related art that determines whether an image contains a regular pattern based on contrast and spatial frequency based on this characteristic. Specifically, this related art detects an image area in the image where the intensity of the contrast is out of the allowable range and the peak value of the spatial frequency is in a predetermined frequency band. Then, this related art determines that the image contains a regular pattern, when the area ratio of the detected image area to the entire image is out of the allowable range.

"Guidance for the reduction of photosensitive epilepsy caused by television", Rec. ITU-R BT.1702, ITU, 2005 "Guidelines on video techniques such as animation", Japan Broadcasting Association, Japan Commercial Broadcasting Federation, 1998 (2006 partially revised)

Patent No. 2898269

  However, in the related art described in Patent Document 1, for the regular pattern to be detected, it is necessary to designate in advance a possible frequency band of the peak value of the spatial frequency as the image feature. Therefore, this related art has a problem that it can not detect unknown regular patterns.

  The present invention has been made to solve the above-mentioned problems. That is, an object of the present invention is to provide a technique for detecting a regular pattern in an image without designating the image feature in advance.

  The image processing apparatus according to the present invention is not limited to 1 based on the similarity between the brightness gradient vector and a brightness gradient vector calculation unit that calculates a brightness gradient vector representing the feature of the change in brightness for each partial region of the input image. A local integration unit that generates a local integrated region integrating one or more partial regions and a similarity (local integrated region feature) of the local integrated region calculated based on the luminance gradient vector in the local integrated region The integrated area in a regular pattern based on a global integration unit that generates an integrated area integrating the one or more local integrated areas based on the gender and the distribution characteristic of the luminance gradient vector in the integrated area And a regular pattern determination unit that determines whether there is any and outputs the determination result.

  In the image processing method according to the present invention, a luminance gradient vector representing a feature of change in luminance is calculated for each partial region of the input image, and one or more of the above are calculated based on the similarity of the luminance gradient vectors. One or more local integration areas are generated by integrating partial areas, and one or more are generated based on the similarity of the features (local integration area features) of the local integration area calculated based on the luminance gradient vector in the local integration area. An integrated area in which the locally integrated area is integrated is generated, and whether or not the integrated area is a regular pattern is determined based on the distribution characteristic of the luminance gradient vector in the integrated area, and a determination result is output.

  In the image processing program according to the present invention, a luminance gradient vector calculating step of calculating a luminance gradient vector representing a characteristic of change in luminance for each partial region of the input image, and similarity of the luminance gradient vector. A locally integrated step of generating a locally integrated region integrating one or more of the partial regions, and a feature (locally integrated region feature) of the locally integrated region calculated based on the luminance gradient vector in the locally integrated region Based on the similarity of one or more of the locally integrated regions to generate an integrated region, and based on the distribution characteristic of the luminance gradient vector in the integrated region, the integrated region is regular. It is determined whether or not it is a pattern, and a regular pattern determination step of outputting a determination result is executed by the computer device.

  The present invention can provide a technique for detecting regular patterns in an image without designating the image features in advance.

FIG. 1 is a block diagram showing a configuration of an image processing apparatus as a first embodiment of the present invention. It is a figure showing an example of the hardware constitutions of the image processing device as a 1st embodiment of the present invention. It is a figure explaining the feature of a regular pattern. FIG. 7 illustrates an example of an image region having features similar to regular patterns but not regular patterns. It is a flowchart explaining operation | movement of the image processing apparatus as the 1st Embodiment of this invention. It is a block diagram showing composition of an image processing device as a 2nd embodiment of the present invention. It is a figure explaining an example of calculation processing of a brightness gradient vector in a 2nd embodiment of the present invention. It is a figure explaining direction ID used by the determination processing of the similarity of the brightness gradient vector in a 2nd embodiment of the present invention. It is a figure explaining the threshold value used in the determination processing of the similarity of the brightness | luminance gradient vector in the 2nd Embodiment of this invention. It is a figure explaining an example of the distribution characteristic of the brightness gradient vector in the image field which is not a regular pattern. It is a flowchart explaining operation | movement of the image processing apparatus as the 2nd Embodiment of this invention. It is a figure which shows the specific example of the image input in the 2nd Embodiment of this invention. It is a figure which shows the specific example of the partial area | region used in the 2nd Embodiment of this invention. It is a figure which shows the example of the brightness | luminance gradient vector produced | generated in the 2nd Embodiment of this invention. It is a figure explaining the specific example of unification of the partial area in a 2nd embodiment of the present invention. It is a figure which shows the example of the local integrated area | region produced | generated in the 2nd Embodiment of this invention. It is a figure which shows the tendency of the normalization histogram showing the characteristic about the specific example of the local integrated area | region produced | generated in the 2nd Embodiment of this invention. It is a figure which shows the specific example of the integrated area | region produced | generated in the 2nd Embodiment of this invention. It is a figure which shows the area | region of the regular pattern detected in 2nd Embodiment of this invention. FIG. 6 illustrates various regular patterns.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

First Embodiment
A functional block configuration of an image processing apparatus 1 as a first embodiment of the present invention is shown in FIG. In FIG. 1, the image processing apparatus 1 includes a brightness gradient vector calculation unit 11, a local integration unit 12, a global integration unit 13, and a regular pattern determination unit 14. The image processing apparatus 1 is an apparatus for detecting an area of a regular pattern from an image.

  Here, the image processing apparatus 1 can be configured by hardware elements as shown in FIG. In FIG. 2, the image processing apparatus 1 includes a central processing unit (CPU) 1001, a memory 1002, an output device 1003, and an input device 1004. The memory 1002 is configured by a random access memory (RAM), a read only memory (ROM), an auxiliary storage device (such as a hard disk), and the like. The output device 1003 is configured by a device that outputs information, such as a display device or a printer. The input device 1004 is configured by a device such as a keyboard and a mouse that receives an input of a user operation. In this case, each functional block of the image processing apparatus 1 is configured by the CPU 1001 that reads and executes a computer program stored in the memory 1002 and controls the output device 1003 and each unit of the input device 1004. Note that the hardware configuration of the image processing apparatus 1 and each functional block thereof is not limited to the above-described configuration.

  Next, the features of the regular pattern will be described. First, as an example of a regular pattern, a diagonal stripe pattern and a spiral pattern are shown in FIG. In these regular patterns, there are a plurality of partial areas (for example, areas surrounded by broken lines) having a specific pattern with high contrast. Moreover, the upper part of FIG. 3 shows a partially enlarged view of each of these regular patterns. As shown in the enlarged view, in the vicinity of the contour of each pattern in the regular pattern, pixel pairs having the same luminance difference along the contour are arranged in the direction indicated by the double arrow. Also, when each regular pattern is viewed in the entire image area, local specific patterns are dispersed and present throughout.

  On the other hand, FIG. 4 shows an example of an image area having features similar to regular patterns but not regular patterns. In FIG. 4, an area A is an area of characters superimposed on the image. In the area A, pixel pairs having the same luminance difference exist near the outline of the character, but no specific pattern is dispersed and present in the entire character area. Further, the area B is an area where an object having an appearance similar to a regular pattern is photographed. In the region B, pixel pairs having the same luminance difference exist in the vicinity of a straight line extending in the vertical direction, and a plurality of specific patterns including straight lines extending in the vertical direction exist in the entire region. However, in the region B, a portion having a small contrast (a portion having a small luminance difference) is present in part of the outline of a straight line extending in the vertical direction.

  In order to detect a regular pattern using such regular pattern features, the image processing apparatus 1 calculates a brightness gradient vector for each partial region of the image. Then, the image processing device 1 integrates the partial regions based on the local and global similarities using the brightness gradient vector (that is, aggregates specific patterns existing in a dispersed manner). Thereby, the image processing apparatus 1 detects a regular pattern. Hereinafter, details of the functional blocks of the image processing apparatus 1 will be described.

  The luminance gradient vector calculation unit 11 calculates a luminance gradient vector for each partial region in the input image. The luminance gradient vector is characteristic of the change in luminance in the partial region. For example, even before the calculation of the brightness gradient vector, the brightness gradient vector calculation unit 11 may obtain, as input data, an image designated as an inspection target via the input device 1004 among the images stored in the memory 1002. Good. In addition, the luminance gradient vector calculation unit 11 may execute a process of dividing an input image into partial regions based on a predetermined method.

  The local integration unit 12 generates a locally integrated region in which one or more partial regions are integrated based on the similarity of the luminance gradient vectors. In addition, the local integration unit 12 determines whether there is another partial area in which the luminance gradient vector has similarity to at least one of the partial areas included in the already integrated local integrated area. If there is another partial area having similarity, the local integration unit 12 further integrates the other partial area into the already integrated local integration area to make a new local integration area.

  The global integration unit 13 generates an integrated area in which one or more local integrated areas are integrated based on the similarity of the local integrated area features. The locally integrated region feature is information representing the feature of the locally integrated region, and is calculated based on the luminance gradient vector in the locally integrated region. Note that the brightness gradient vector in the locally integrated region is a brightness gradient vector calculated for each partial region included in the locally integrated region. In addition, the global integration unit 13 determines whether or not there is another locally integrated region in which the locally integrated region feature has similarity to at least one locally integrated region included in the already integrated region. If there are other locally integrated regions having similarities, another locally integrated region is further integrated into the already integrated region, to form a new integrated region.

  The regular pattern determination unit 14 determines whether the integrated area is a regular pattern based on the distribution characteristic of the luminance gradient vector in the integrated area. The brightness gradient vector in the integrated region is a brightness gradient vector calculated for each partial region included in the integrated region. Then, the regular pattern determination unit 14 outputs the determination result. For example, the regular pattern determination unit 14 may output information indicating an area determined to be a regular pattern on the input image so as to be output to the output device 1003.

  The operation of the image processing apparatus 1 configured as described above will be described with reference to FIG.

  First, the brightness gradient vector calculation unit 11 divides the input image into partial regions (step S1).

  Next, the luminance gradient vector calculation unit 11 calculates a luminance gradient vector for each partial region (step S2).

  Next, the local integration unit 12 generates a locally integrated region in which one or more partial regions are integrated based on the similarity of the luminance gradient vector (step S3).

  Next, the global integration unit 13 generates an integrated area in which one or more local integrated areas are integrated based on the similarity of the local integrated area features (step S4).

  Next, the regular pattern determination unit 14 determines whether or not the integrated area is a regular pattern based on the distribution characteristic of the luminance gradient vector in the integrated area, and outputs the determination result (step S5).

  Thus, the image processing device 1 ends the operation.

  Next, the effects of the first embodiment of the present invention will be described.

  The image processing apparatus as the first embodiment of the present invention can detect regular patterns in an image without designating the image features in advance.

  I will explain the reason. In the present embodiment, the luminance gradient vector calculation unit calculates, for each partial region obtained by dividing the input image, a luminance gradient vector representing the characteristic of the change in luminance. Then, the local integration unit generates a local integration region integrating one or more partial regions based on the similarity of the luminance gradient vectors, and the global integration unit generates a local integration region based on the similarity of the local integration region features. , Generate an integrated area integrating one or more local integrated areas. Then, the regular pattern determination unit determines whether or not the integrated area is a regular pattern based on the distribution characteristic of the luminance gradient vector in the integrated area, and outputs the determination result.

  As described above, in the present embodiment, the local integration unit corresponds to the contour of the regular pattern using the characteristic “the luminance gradient vectors at the close positions on the contour of the regular pattern are similar”. Partial areas are integrated into one to be a local integrated area. Then, the global integration unit repeats the same type among the locally integrated regions corresponding to the contour of the regular pattern, utilizing the characteristic that “the same kind of local pattern repeatedly exists on the image in the regular pattern”. Integrate the existing areas into a single integrated area. Thus, according to the present embodiment, it is possible to determine whether or not the pattern is a regular pattern based on the distribution characteristic of the luminance gradient vector for the integrated region in which the local pattern may be repeated. Therefore, in the present embodiment, it is not necessary to presume a regular pattern to be detected in advance, and it becomes possible to detect a region where an unknown regular pattern exists.

Second Embodiment
Next, a second embodiment of the present invention will be described in detail with reference to the drawings. In the drawings to which reference is made in the description of the present embodiment, steps having the same configuration and operation as those of the first embodiment of the present invention will be assigned the same reference numerals and detailed description in the present embodiment. I omit explanation.

  First, FIG. 6 shows the configuration of an image processing apparatus 2 as a second embodiment of the present invention. In FIG. 6, the image processing device 2 includes a luminance gradient vector calculation unit 21, a local integration unit 22, a global integration unit 23, and a regular pattern determination unit 24. The image processing device 2 is a device that detects an area of a regular pattern from an image.

<Configuration of Brightness Gradient Vector Calculation Unit>
The luminance gradient vector calculating unit 21 divides the input image into partial areas, substantially the same as the luminance gradient vector calculating unit 11 according to the first embodiment of the present invention, and generates luminance gradient vectors for each of the divided partial areas. Although it calculates, the details are different. A process of generating a partial area and a process of calculating a luminance gradient vector in the present embodiment will be described.

  In the present embodiment, the luminance gradient vector calculation unit 21 divides the image into grid-like partial regions. The luminance gradient vector calculation unit 21 may calculate the luminance gradient vector of each partial region after converting the image into an 8-bit grayscale image. Alternatively, the brightness gradient vector calculation unit 21 may calculate the brightness gradient vector of each partial region after converting the image into an image with another number of gradations. Further, the luminance gradient vector calculation unit 21 selects one of the color components of RGB (Red Green Blue) of the image, converts the selected color component amount into a grayscale image having the luminance value, and A luminance gradient vector may be calculated.

  In detail, for example, the luminance gradient vector calculating unit 21 may divide the input image into a grid shape with a predetermined width and height, and set each generated block as a partial region. Alternatively, the luminance gradient vector calculation unit 21 reduces each input image into a predetermined image size, and then divides each reduced image into a grid having a predetermined width and height to generate blocks. It may be a partial area. The predetermined image size may be, for example, a QVGA (Quarter Video Graphics Array) image size or a square image size such as 64 × 64. The luminance gradient vector calculation unit 21 may divide the image in units of a predetermined specific shape as well as the grid-like partial regions, and may use the generated unit regions as partial regions.

  Further, the luminance gradient vector calculation unit 21 calculates, as a luminance gradient vector, a vector having, as components, luminance fluctuation amounts in the horizontal direction and the vertical direction in the partial region.

  Here, an example of the calculation process of the luminance gradient vector will be described with reference to FIG. In this case, the luminance gradient vector calculation unit 21 divides the partial region into a predetermined number (here, 2 × 2) of subblocks. Then, the brightness gradient vector calculation unit 21 obtains brightness values Y1, Y2, Y3, and Y4 representative of each sub block. Then, the luminance gradient vector calculation unit 21 sets the difference between the luminance values Y1 and Y3 representing the upper left and lower right sub blocks as the first component, and sets the luminance values Y2 and Y4 representing the upper right and lower left sub blocks. A vector having the difference as the second component may be used as the luminance gradient vector. As the luminance values Y1, Y2, Y3 and Y4 representing each sub block, it is possible to adopt the average value or the mode value of the luminance values of the sub blocks. In this case, the effect of making the regular pattern area robustly detectable to noise can be expected. Alternatively, as the luminance values Y1, Y2, Y3 and Y4 representing each sub block, it is possible to adopt a luminance value that maximizes each component of the vector. In this case, it is possible to expect the effect of making the regular pattern area robustly detectable against blurring of the contour.

  In addition, another example of the calculation process of the luminance gradient vector will be described. The luminance gradient vector calculation unit 21 takes horizontal and vertical differentials for each pixel constituting the partial region, and sets a vector whose maximum value in the partial region of the differential value in each direction is x and y components, It may be a brightness gradient vector.

  Further, still another example of the calculation process of the luminance gradient vector will be described. The luminance gradient vector calculation unit 21 calculates a luminance vector for each of the small areas included in the partial area. In this case, the small areas may overlap. Further, any one of the above-described calculation processes may be used for the calculation process of the luminance vector in the small area. Then, the luminance gradient vector calculation unit 21 sets all luminance vectors of a predetermined number or more having substantially the same size and direction as the luminance gradient vector of the partial region among the luminance vectors calculated for the small regions included in the partial region. It is also good. In the case of this calculation process, one or more brightness gradient vectors are calculated for each partial region. That is, in the case of this calculation process, a plurality of luminance gradient vectors may be calculated for each partial region.

  For example, in the case where the partial area is configured by 3 × 3 pixels, it can be considered that the partial area includes four small areas of 2 × 2 pixels. Therefore, in each small area of 2 × 2 pixels, the brightness gradient vector calculation unit 21 regards each pixel as a sub-block in FIG. 7 and calculates the brightness gradient vector by the calculation process of FIG. 7. That is, four luminance gradient vectors are calculated in the 3 × 3 partial region. Here, it is assumed that four luminance gradient vectors are (1, 2), (1, 2) (2, 3), (3, 4), respectively. Further, it is assumed that “select all two or more luminance vectors having the same size and direction”. In this case, in this partial area, two or more vectors having the same size and direction are only (1, 2). Therefore, the luminance gradient vector calculation unit 21 sets the luminance gradient vector of this partial region as (1, 2). Further, it is assumed that “select all one or more luminance vectors having the same size and direction”. In this case, the luminance gradient vector calculation unit 21 sets the luminance gradient vectors of this partial region as (1, 2), (2, 3), (3, 4).

<Configuration of Local Integration Unit>
The local integration unit 22 generates a locally integrated region by performing integration based on the similarity of luminance gradient vectors between adjacent partial regions. In addition, the local integration unit 22 determines the similarity of the luminance gradient vector based on at least one of the magnitude and the direction of the luminance gradient vector.

  Specifically, the local integration unit 22 receives information on each partial region (the position of the partial region and the luminance gradient vector) as input, and calculates information indicating the proximity of the partial regions and the similarity of the luminance gradient vector, respectively. Do. The local integration unit 22 integrates the partial regions to generate a local integrated region when each of the proximity between the partial regions and the similarity of the luminance gradient vector satisfies a predetermined condition.

  At this time, the local integration unit 22 may set only partial regions having a luminance gradient vector whose vector size is equal to or more than a predetermined value as an integration target. In this case, the local integration unit 22 may couple a partial area having a luminance gradient vector whose vector size is less than a predetermined value to another partial area that satisfies the predetermined condition. For example, the predetermined condition is that the spatial distance is less than a specified value and the vector size of the brightness gradient vector is equal to or more than a predetermined value for a partial region having a brightness gradient vector whose vector size is less than a predetermined value. And, the color information may be similar. In this case, the partial regions that have become one by coupling are targets of integration by the local integration unit 22. Then, a luminance gradient vector having a vector size equal to or larger than a predetermined value is employed as a luminance gradient vector of a partial region which has become one by combining.

  Thus, the local integration unit 22 generates the local integration regions based on the proximity between the partial regions and the similarity of the brightness gradient vectors between the partial regions.

[Proximity between partial areas]
Here, the process of determining the proximity between partial areas will be described. The local integration unit 12 determines, based on the spatial distance D of the two partial areas, whether the two partial areas are close to each other. The spatial distance D may be the distance between the center coordinates of each partial area. In addition, the spatial distance D may be a distance between pixel pairs closest to each other among the pixels included in the partial region. Also, the spatial distance D may be a distance between pixel pairs that are most distant from one another among the pixels included in the partial region. Then, the local integration unit 12 may determine whether or not the distances between the partial regions are close based on the magnitude relationship between the spatial distance D between the partial regions and the threshold. For example, it is assumed that the central coordinates of partial areas A and B are (Xa, Ya) and (Xb, Yb), respectively, and the threshold is Th. In this case, the local integration unit 12 may determine that the partial regions A and B are close to each other when the following equation (1) is satisfied.

Note that “||” represents an absolute value.

[Similarity of luminance gradient vector]
Next, the process of determining the similarity of the brightness gradient vector will be described. The local integration unit 22 determines similarity between two partial regions based on at least one of the magnitude or the direction of the luminance gradient vector.

For example, when the luminance gradient vector is (Ua, Va), the magnitude thereof can be calculated by the following equation (2).

In this case, the local integration unit 22 may determine that the two luminance gradient vectors are similar when the difference value between the magnitudes of the two luminance gradient vectors is less than the threshold THs.

Further, the direction (unit: radian) of the luminance gradient vector (Ua, Va) can be calculated by the following equation (3).

Note that [rad] represents that the unit is radian. In this case, the local integration unit 22 may determine that the two luminance gradient vectors are similar if the difference value in the direction of the two luminance gradient vectors is less than the threshold THr.

  Alternatively, the local integration unit 22 may determine the similarity based on the ID given in the direction of the two luminance gradient vectors. For example, as shown in FIG. 8, it is assumed that the direction (angle) of the vector is quantized to a predetermined number (for example, 12) and the direction ID is determined. In this case, the local integration unit 22 assigns, to the luminance gradient vector, a direction ID to which the vector belongs. For example, in FIG. 8, the direction ID “2” is assigned to the luminance gradient vector (Ua, Va). Thus, the local integration unit 22 assigns a direction ID to the two luminance gradient vectors. Then, for example, the local integration unit 22 may determine that the two luminance gradient vectors are similar when the direction ID of the two luminance gradient vectors is the same. In addition, the local integration unit 22 may be configured such that the direction IDs of two luminance gradient vectors are adjacent (for example, “1” and “2”, “3” and “2”, “0” and “11”, etc.) The two luminance gradient vectors may be determined to be similar.

  In addition, the local integration unit 22 may determine that the two brightness gradient vectors are similar when it is determined that the two are similar by any of the determination processing of similarity based on the above-described size and direction. Alternatively, the local integration unit 22 may determine that the two brightness gradient vectors are similar when it is determined that the two are similar by both of the determination processing of similarity based on the above-described size and direction.

  In addition, the local integration unit 22 may set the thresholds THs and THr in accordance with the spatial distance between the partial regions to which two luminance gradient vectors that determine the similarity belong. For example, as shown in FIG. 9, the threshold value THs or THr may be set to be a larger value as the spatial distance D between partial areas becomes larger. In this case, as in the example of FIG. 9, the threshold value THs or THr may have an upper limit value. The maximum value of THs may be 255 in the case of an 8-bit grayscale image. Further, in the case of a grayscale image of another grayscale, the maximum value of THs may be a value corresponding to the number of grayscales. The maximum value of THr may be π or 2π. Thus, by setting threshold values THs and THr according to the spatial distance D between the partial regions, the direction of the brightness gradient vector gradually changes as the spatial distance D increases, such as, for example, a spiral or a concentric circle. Also in those cases, these partial areas are properly integrated. Also, even when there is a difference in magnitude of the brightness gradient vector between distant partial regions due to the influence of noise or the like, those partial regions are properly integrated.

  A case where a plurality of luminance gradient vectors are calculated for one partial region will be described. In this case, the local integration unit 22 selects two partial regions based on a combination in which at least one of the magnitude and the direction is most similar among the combinations of luminance gradient vectors selected one by one from the two partial regions. The similarity of may be determined. Alternatively, the local integration unit 22 calculates at least one of the magnitude and the direction for all combinations of luminance gradient vectors selected one by one from two partial regions, and the statistical amount (for example, average value, etc.) The similarity between the two partial regions may be determined based on The magnitude or direction of the combination of the luminance gradient vectors selected one by one from the two partial regions can be calculated, for example, as the above-mentioned equation (2), equation (3) or direction ID.

[Post-processing for locally integrated area]
In this manner, the local integration unit 22 integrates two partial regions into a locally integrated region, when the proximity and the similarity of the luminance vector both satisfy predetermined conditions between any two partial regions. Generate Then, the local integration unit 22 determines whether the similarity between the proximity and the luminance vector both satisfy a predetermined condition between any of the partial regions included in the already generated local integrated region and the other partial regions not included. It is determined whether or not. If it is determined that both the proximity and the similarity satisfy the predetermined condition, the local integration unit 22 integrates other partial areas into the already generated local integrated area to generate a new local integrated area. Do.

  In addition, the local integration unit 22 may set the partial region as a local integration region for a partial region not integrated with any other partial region.

  In addition, the local integration unit 22 may evaluate the finally generated locally integrated region, and remove the locally integrated region not satisfying the defined condition as a false detection. As the prescribed condition, a condition based on the number of partial regions constituting the locally integrated region or a condition based on the area of the locally integrated region may be adopted. For example, the local integration unit 22 may remove the locally integrated region as a false detection when the number of partial regions constituting the locally integrated region is less than a specified number. In addition, when the area of the locally integrated region is less than the specified value, the locally integrated unit 22 may remove the locally integrated region as a false detection. In addition, the local integration unit 22 removes the locally integrated area as a false detection when both the number of partial areas constituting the locally integrated area and the area of the locally integrated area are smaller than prescribed values respectively determined. May be

<Configuration of Global Integration Unit>
The global integration unit 23 integrates the locally integrated regions based on the similarity of the locally integrated region features calculated from the luminance gradient vector, substantially the same as the global integration unit 13 in the first embodiment of the present invention. Create an integrated area, but the details are different.

[Local integrated area feature]
First, details of the calculation process of the locally integrated region feature by the global integration unit 23 will be described. The global integration unit 23 calculates a normalized histogram based on at least one of the magnitude or the direction of the luminance gradient vector as a local integrated region feature.

(23-1) Normalized Histogram Based on Intensity of Luminance Gradient Vector Here, as an example of calculation processing of the local integrated region feature, calculation processing of a normalized histogram based on the magnitude of the luminance gradient vector will be described. In this case, the global integration unit 23 generates a histogram of the number of bins predetermined for the vector size for the luminance gradient vector in the local integration region. Then, the global integration unit 23 generates a normalized histogram by dividing (normalize) the frequency of each bin by the number of luminance gradient vectors used to generate the histogram in the generated histogram. Note that the global integration unit 23 generates a normalized histogram according to the size using a luminance gradient vector having a vector size equal to or larger than a specified value THb (> = 0) among the luminance gradient vectors in the locally integrated region. Good.

For example, for an 8-bit grayscale image, it is assumed that a normalized histogram based on the vector size is generated using a luminance gradient vector whose vector size is equal to or larger than a predetermined value THb. In this case, the global integration unit 23 generates a bin width (255−THb) / Nb1 histogram obtained by dividing the vector size range THb to 225 by a predetermined bin number Nb1. Here, "/" represents division. Specifically, the global integration unit 23 determines to which bin the size of each of the luminance gradient vectors having the vector size THb or more belonging to the local integrated region i belongs. Thereby, the global integration unit 23 calculates a histogram H1i [j] (0 <= j <Nb1, 0 <= i <Nr, Nr is the number of locally integrated regions) of the locally integrated region i. Then, the global integration unit 23 divides H1i [j] by the number Nv of luminance gradient vectors having a vector size THb or more belonging to the local integration region i to obtain a normalized histogram rH1i [ j] is generated.

(23-2) Normalized Histogram According to Direction of Luminance Gradient Vector Next, as another example of calculation processing of a local integrated region feature, calculation processing of a normalized histogram based on the direction of the luminance gradient vector will be described. In this case, the global integration unit 23 generates a histogram of the number of bins predetermined for the direction of the vector for the luminance gradient vector in the local integration region. Then, the global integration unit 23 generates a normalized histogram by dividing (normalize) the frequency of each bin by the number of luminance gradient vectors used to generate the histogram. Note that the global integration unit 23 generates a normalized histogram according to direction, using a luminance gradient vector having a vector size equal to or larger than a prescribed value THb (> 0) among the luminance gradient vectors included in the locally integrated region. It is also good.

For example, for an 8-bit grayscale image, it is assumed that a direction-based normalized histogram is generated using a luminance gradient vector whose vector size is equal to or greater than a predetermined value THb. In this case, the global integration unit 23 generates a histogram of a bin width π / Nb2 [rad] obtained by dividing the value range 0 to π [rad] of the direction by a predetermined bin number Nb2. Specifically, the global integration unit 23 determines to which bin the direction of each luminance gradient vector belonging to the local integration region i and not smaller than the vector size THb belongs. Thereby, the global integration unit 23 calculates the histogram H2i [j] (0 <= j <Nb2, 0 <= i <Nr, Nr is the number of locally integrated regions) of the locally integrated region i. Then, the global integration unit 23 divides H2i [j] by the number Nv of luminance gradient vectors having a vector size THb or more belonging to the local integration region i to obtain a normalized histogram rH2i [ j] is generated.

(23-3) Normalized Histogram Based on Direction and Vector Size of Intensity Gradient Vector Next, as another example of calculation processing of a local integrated region feature, calculation of a normalized histogram based on the direction of intensity gradient vector and vector size The process will be described. In this case, the global integration unit 23 generates a histogram of the number of bins predetermined for the vector direction and the vector size of the luminance gradient vector included in the local integration region. Then, the global integration unit 23 generates a normalized histogram by dividing (normalize) the frequency of each bin by the number of luminance gradient vectors used to generate the histogram. Note that the global integration unit 23 uses the luminance gradient vector whose vector size is equal to or larger than the specified value THb (> = 0) among the luminance gradient vectors included in the locally integrated region, to obtain a normalized histogram by the direction and the magnitude. It may be generated.

For example, for an 8-bit grayscale image, it is assumed that a normalized histogram based on direction and vector size is generated using a luminance gradient vector whose vector size is equal to or larger than a predetermined value THb. In this case, the global integration unit 23 determines the number of bins Nb3 (= Nb1 × Nb2) determined from the quantization number Nb1 for the vector size range THb to 225 and the quantization number Nb2 for the range 0 to π [rad] of the direction. ) Will be used to generate a histogram. Specifically, the global integration unit 23 determines to which bin the combination of the direction and the size of each of the brightness gradient vectors having the vector size THb or more belonging to the local integration region i belongs. Thereby, the global integration unit 23 calculates a histogram H3i [j] (0 <= j <Nb3, 0 <= i <Nr, Nr is the number of locally integrated regions) of the locally integrated region i. Then, the global integration unit 23 divides H3i [j] by the number Nv of luminance gradient vectors having a vector size THb or more belonging to the local integration region i, to obtain a normalized histogram rH3i [ j] is generated.

  As described above, (23-1), (23-2), and (23-3) are shown as examples of calculation processing of the locally integrated region feature. In addition, even in the case where a plurality of luminance gradient vectors are calculated for one partial region, the calculation processing of these locally integrated region features is applicable. In addition to these examples, the global integration unit 23 calculates, as a locally integrated region feature, another image feature amount such as a normalized histogram of color and the like by adding it to the normalized histogram regarding the luminance gradient vector described above. It is also good. In addition, the global integration unit 23 may use at least one of the magnitude and the direction of the luminance gradient vector at the position of each partial region included in the local integrated region as the local integrated region feature. Here, the brightness gradient vector at the position of each partial region included in the local integrated region is the brightness gradient vector of the partial region in which the pixel is included at the position of each pixel included in the local integrated region. In addition, the global integration unit 23 may use pixel values included in the locally integrated region as a locally integrated region feature.

[Similarity of locally integrated region features]
Next, determination processing of similarity of locally integrated region features will be described. Here, it is assumed that the locally integrated region feature is, for example, a normalized histogram calculated by (23-1), (23-2), or (23-3) described above. In this case, the global integration unit 23 may perform determination by calculating information representing the similarity between locally integrated region features for any two locally integrated regions based on the distance between normalized histograms. For example, information representing the similarity between the locally integrated regions A and B can be calculated by the following equation (7) using the normalized histograms rHA and rHB (bin number Nb).

  The information representing the similarity between locally integrated region features calculated by equation (7) has a larger value as the similarity is higher. Therefore, the global integration unit 23 may generate an integrated area obtained by integrating the two locally integrated areas, when the value calculated by the equation (7) exceeds the threshold value, for the two locally integrated areas. .

  In addition, the global integration unit 23 may calculate information representing the similarity between locally integrated region features for any two locally integrated regions by template matching. In this case, it is possible to use a brightness gradient vector at the position of each partial area included in the local integrated area as the local integrated area feature. Moreover, it is also possible to use together the pixel value of each pixel in a local integrated area as a local integrated area feature.

  Specifically, the global integration unit 23 scans one of the locally integrated regions A and B as a template in order to calculate information representing the similarity between the locally integrated regions A and B. For example, the global integration unit 23 may use the smaller one of the local integration regions A and B as a template. Then, the global integration unit 23 calculates the difference of the luminance gradient vector at the corresponding position of the overlapping regions in the local integration regions A and B for each scanning position. The difference of the brightness gradient vector may be a difference of at least one of the magnitude and the direction of the brightness gradient vector. Then, the global integration unit 23 may calculate the sum of the differences for each scanning position, and the minimum value among them may be information representing the similarity between the local integration area features.

  Specifically, the global integration unit 23 calculates the following difference value for each pixel of the overlapping region of the local integration regions A and B at a certain scanning position. The difference value calculated for each pixel is the difference between the luminance gradient vector calculated in the partial region including the pixel in the local integrated region A and the luminance gradient vector calculated in the partial region including the pixel in the local integrated region B It is.

  Note that when a plurality of luminance gradient vectors are calculated for one partial region, difference values calculated for each pixel will be described. In this case, the difference value calculated for each pixel is the difference of the combination of the luminance gradient vectors respectively selected from the partial area including the pixel in the local integrated area A and the partial area including the pixel in the local integrated area B It may be the minimum of the values. Further, the difference value calculated for each pixel is a difference between all combinations of luminance gradient vectors respectively selected from the partial region including the pixel in the local integrated region A and the partial region including the pixel in the local integrated region B. It may be a statistical value (for example, an average value, etc.) of the values.

  Then, the global integration unit 23 may calculate, for each scanning position, the sum of the calculated difference values for all pixels in the area where the local integrated areas A and B overlap. The global integration unit 23 may calculate the sum of pixel value differences in the overlapping area in addition to the sum of differences in luminance gradient vectors for each scanning position. For example, the global integration unit 23 may add the sum of differences of pixel values to the sum of differences of luminance gradient vectors.

  Then, the global integration unit 23 adopts the minimum value among the sums of the differences at each scanning position calculated as described above as information representing the similarity between the locally integrated region features. Here, the information indicating the similarity based on the sum of differences has a smaller value as the similarity is higher. Therefore, the global integration unit 23 generates an integrated area obtained by integrating two locally integrated areas, when the information representing the similarity based on the sum of differences is less than the threshold value, for the two locally integrated areas. Good.

Post-processing for integrated area
In this manner, the global integration unit 23 integrates an integrated area obtained by integrating two locally integrated areas, when the similarity between locally integrated area features satisfies a predetermined condition between any two locally integrated areas. Generate Then, the global integration unit 23 determines whether the similarity of locally integrated area features satisfies a predetermined condition between any of the locally integrated areas included in the already generated integrated area and any other locally integrated area not included. It is determined whether or not. If it is determined that the similarity of the locally integrated area feature satisfies the predetermined condition, the global integration unit 23 integrates another locally integrated area into the already generated integrated area to generate a new integrated area.

  The global integration unit 23 may set the locally integrated region as an integrated region for a locally integrated region that is not integrated with any other locally integrated region.

  In addition, the global integration unit 23 may evaluate the finally generated integrated area and remove an integrated area that does not satisfy the defined condition as a false detection. As the specified condition, a condition based on the number of partial regions constituting the integrated region or a condition based on the area of the integrated region may be adopted. For example, when the number of partial areas constituting the integrated area is less than the specified value, the global integration unit 23 may remove the integrated area as a false detection. In addition, when the area of the integrated area is less than the specified value, the global integration unit 23 may remove the integrated area as a false detection. In addition, the global integration unit 23 may remove the integrated area as a false detection when both the number of partial areas constituting the integrated area and the area of the integrated area are smaller than the predetermined values respectively defined. .

<Configuration of Regular Pattern Determination Unit>
Similar to the regular pattern determination unit 24 in the first embodiment of the present invention, the regular pattern determination unit 24 regularly determines the integrated region based on the distribution characteristic of the luminance gradient vector included in the integrated region. It is determined whether or not it is a pattern, but the details are different.

  Further, the regular pattern determination unit 24 may output each of the integrated areas determined as the regular pattern as a regular pattern area. Alternatively, the regular pattern determination unit 24 may output, as a regular pattern area, an area including all of the integrated areas determined to be regular patterns. The included region may be, for example, a circumscribed rectangle.

[Determination process of distribution characteristics]
Here, the details of the determination process of the distribution characteristic by the regular pattern determination unit 24 will be described. As the distribution characteristic, for example, density, concentration region, unidirectionality, point symmetry and the like can be mentioned.

(24-1) Determination Based on Density Here, the determination process based on density will be described as an example of the determination process of a regular pattern based on distribution characteristics. In this case, the regular pattern determination unit 24 may determine whether or not the integrated area is a regular pattern based on the density of the partial area in which the vector size of the luminance gradient vector is larger than the threshold in the integrated area. For example, it is considered that the integrated area having such a density higher than the specified value includes many edge areas (outline portions of regular patterns) having high luminance gradients. Therefore, the regular pattern determination unit 24 may determine that the integrated area in which the density of the partial area where the vector size of the luminance gradient vector is larger than the threshold is higher than the specified value is the regular pattern.

  Here, the density of the partial region having a luminance gradient vector whose vector size is larger than the threshold can be calculated, for example, as follows. For example, such a density may be a ratio of the area of a partial area having a luminance gradient vector whose vector size is equal to or larger than the threshold Tc in the integrated area with respect to the area of the integrated area. Alternatively, such a density may be a ratio of the area of a partial area having a brightness gradient vector with a vector size equal to or larger than the threshold Tc in the integrated area to the area of the circumscribed rectangle including the integrated area .

  Alternatively, instead of determining whether or not the regular pattern determination unit 24 separately determines whether the regular pattern is separately for each integrated area, whether or not the regular pattern is based on the density for all the integrated areas collectively. It may be determined. In this case, the regular pattern determination unit 24 sets a portion having a luminance gradient vector having a vector size equal to or larger than the threshold Tc among partial areas belonging to any of the integrated areas with respect to the area of the circumscribed rectangle including all integrated areas. The percentage of the area of the area may be calculated as the density. Then, in this case, if such a density is higher than the specified value, the regular pattern determination unit 24 may collectively determine that all the integrated regions are regular patterns.

(24-2) Determination Based on Concentration Region Here, the determination processing based on a concentration region will be described as an example of determination processing of a regular pattern based on distribution characteristics. In this case, the regular pattern determination unit 24 determines whether or not the integrated area is a regular pattern based on the position of the area where the partial area where the vector size of the luminance gradient vector is larger than the threshold in the integrated area is concentrated. It is also good. For example, it is considered that the integrated area in which such concentrated area is concentrated near the end is likely to be a rectangular frame used for video editing.

  Here, an example of a rectangular frame used for video editing is shown in FIG. FIG. 10 is called PinP (Picture in Picture), and shows other images and still images arranged in the image. Such PinP is often surrounded by a frame as illustrated. In addition, in a video, a character area is often surrounded by a frame for emphasizing characters. Therefore, the regular pattern determination unit 24 may determine that the integrated area in which the partial area where the vector size of the luminance gradient vector is larger than the threshold is concentrated near the end is not a regular pattern. Further, the regular pattern determination unit 24 may determine that the integrated area in which the partial area where the vector size of the luminance gradient vector is larger than the threshold is not concentrated near the end is a regular pattern.

  Here, whether or not the concentrated position of the partial region where the vector size of the luminance gradient vector is larger than the threshold is near the end can be determined as follows. For example, the regular pattern determination unit 24 determines the partial area included in the integrated area as an end area including a predetermined number of partial areas from the end of the integrated area, and a central area that is the other partial area. Divide into Then, the regular pattern determination unit 24 compares the number in the end region with the number in the central region for a partial region having a brightness gradient vector whose vector size is equal to or larger than the specified value Tc in the integrated region. Then, when the number in the end area is larger than the number in the center area by a large difference (difference larger than the threshold), the regular pattern determination unit 24 concentrates such partial areas near the end. It may be determined that Alternatively, the regular pattern determination unit 24 may divide the partial area within the specified range from the end of the circumscribed rectangle that includes a certain integrated area as the end area and divide the partial area within the other circumscribed rectangle as the central area. . The regular pattern determination unit 24 compares the number in the end region with the number in the central region for partial regions having a luminance gradient vector whose vector size is equal to or greater than the specified value Tc in the integrated region. It may be determined whether the partial region is concentrated near the end.

  Or, instead of the regular pattern determination unit 24 separately determining whether or not each integrated area is a regular pattern, whether all the integrated areas are regular patterns collectively based on the concentration area? It may be determined whether or not. In this case, in the circumscribed rectangle that includes all the integrated regions, the regular pattern determination unit 24 sets a partial region within a prescribed range from the end of the circumscribed rectangle as an end region and a partial region within the other circumscribed rectangles as a central region. It may be divided. Then, the regular pattern determination unit 24 compares, among partial areas belonging to any of the integrated areas, partial areas having a luminance gradient vector whose vector size is equal to or larger than the threshold Tc, the numbers of end areas and central areas. May be And, in this case, when the number in the end area is larger than the number in the central area by a large difference (difference larger than the threshold), the regular pattern determination unit 24 concentrates such partial areas in the end area. It may be determined that all integrated regions are collectively not a regular pattern. In addition, when it is determined that such a partial area is not concentrated on the end area, the regular pattern determination unit 24 may collectively determine that all the integrated areas are regular patterns.

(24-3) Determination Based on Unidirectionality Here, determination processing based on unidirectionality will be described as an example of regular pattern determination processing based on distribution characteristics. In this case, the regular pattern determination unit 24 may determine whether or not the integrated area is a regular pattern, based on the unidirectionality of the luminance gradient vector calculated for the partial area included in the integrated area. For example, if the directions of the luminance gradient vectors included in the integrated area are substantially the same regardless of the spatial position in the integrated area, the integrated area is highly likely to be diagonal lines or stripes. Therefore, the regular pattern determination unit 24 may determine that the integrated area in which the directions of the brightness gradient vectors are substantially the same is a regular pattern.

  Here, the unidirectionality of the luminance gradient vector can be determined, for example, as follows. For example, the regular pattern determination unit 24 generates a normalized histogram based on the direction of the luminance gradient vector in the integrated region. Such a normalized histogram can be calculated in the same manner as the processing example of (23-2) described for the global integration unit 23. Then, the regular pattern determination unit 24 determines the direction of the luminance gradient vector included in the integrated region when the number of bins having frequencies equal to or more than the specified value is within the predetermined number (for example, two) in the normalized histogram based on the direction. May be determined to be substantially the same.

  Alternatively, the regular pattern determination unit 24 may generate a normalized histogram based on the direction of the luminance gradient vector and the vector size in the integrated region. Such a normalized histogram can be calculated in the same manner as the processing example of (23-3) described for the global integration unit 23. Then, in the normalized histogram based on the direction and the size, the regular pattern determination unit 24 determines the luminance gradient included in the integrated area when the number of bins having frequencies equal to or more than the specified value is within a predetermined number (for example, two). It may be determined that the directions of the vectors are substantially the same.

  Alternatively, instead of determining whether the regular pattern is separately for individual combined regions, the regular pattern determination unit 24 collectively determines the rule based on the unidirectionality of the luminance gradient vector for all the combined regions. It may be determined whether or not it is a target pattern. In this case, the regular pattern determination unit 24 normalizes the above-mentioned normalized histogram based on the direction of the brightness gradient vector or the normalized histogram based on the direction and the vector size of the brightness gradient vector for the entire region including all integrated regions. Generate Then, the regular pattern determination unit 24 determines that the direction of the brightness gradient vector in all integrated regions is within the predetermined number (for example, two) of bins having frequencies greater than or equal to the specified value in the generated normalized histogram. It may be determined that they are substantially the same. Then, in this case, the regular pattern determination unit 24 may collectively determine that all the integrated regions are regular patterns.

(24-4) Determination Based on Point Symmetry Here, determination processing based on point symmetry will be described as an example of determination processing of a regular pattern based on distribution characteristics. In this case, the regular pattern determination unit 24 may determine whether the integrated area is a regular pattern based on the point symmetry of the luminance gradient vector included in the integrated area. For example, if the difference in the direction of the brightness gradient vector of a partial region pair included in the integrated region and located at point symmetry with each other is 0 or π [rad], the integrated region is likely to be concentric or spiral pattern . Therefore, the regular pattern determination unit 24 may determine that the integrated region having point symmetry in the luminance gradient vector is a regular pattern.

  Here, the point symmetry of the luminance gradient vector can be determined as follows. For example, the regular pattern determination unit 24 selects, for a partial region of interest in the integrated region, a partial region located at a point symmetrical position with the central coordinates of the integrated region as the origin. Here, the selected partial region is included in the same integrated region as the target partial region, and the vector size of the luminance gradient vector is substantially the same between the selected partial region and the target partial region and the difference in direction is 0 or It is assumed that it is π [rad]. In this case, the regular pattern determination unit 24 determines that the target partial area has point symmetry. Then, when the ratio of the area of the partial area having point symmetry among the partial areas in the integrated area to the area of the integrated area is equal to or more than the specified value, the regular pattern determination unit 24 determines the integrated area It may be determined that there is point symmetry.

  Alternatively, instead of the regular pattern determination unit 24 separately determining whether or not each integrated area is a regular pattern, it is an ordered pattern based on point symmetry collectively for all integrated areas. It may be determined whether or not. In this case, the regular pattern determination unit 24 may determine, for each partial area belonging to any one of the integrated areas, point symmetry with the central coordinates of the circumscribed rectangle including all the integrated areas as the origin. Then, the regular pattern determination unit 24 determines all integrated areas when the ratio of the area of the partial area having point symmetry to the area of the circumscribed rectangle including all integrated areas is equal to or more than the specified value. It may be determined collectively as a regular pattern.

<Description of operation>
The operation of the image processing apparatus 2 configured as described above will be described with reference to FIG.

  In FIG. 11, first, the luminance gradient vector calculation unit 21 divides the grayscale image generated from the input image into grid-like partial regions (step S11).

  Next, the brightness gradient vector calculation unit 21 calculates, for each partial region, a brightness gradient vector having the horizontal and vertical brightness fluctuation amounts in the partial region as components (step S12).

  Next, the local integration unit 22 generates a locally integrated region by integrating the adjacent partial regions based on the similarity of the direction and / or the vector size of the luminance gradient vector (step S13).

  As described above, the local integration unit 22 integrates subregions of any two subregions when the spatial proximity and the similarity of the luminance gradient vector satisfy predetermined conditions respectively. It may be a locally integrated area. Then, the local integration unit 22 adds, to the locally integrated region, another partial region that satisfies the above-described proximity and similarity with each of the predetermined regions between any of the partially integrated regions included in the locally integrated region. It should be integrated.

  In addition, as described above, the local integration unit 22 excludes the locally integrated region as a false detection when the number or area of partial regions constituting the finally generated locally integrated region is less than a specified value. You may

  Next, the global integration unit 23 generates an integrated area by integrating the locally integrated area based on the similarity of the locally integrated area features based on the luminance gradient (step S14).

  As described above, when the locally integrated region features are similar between any two locally integrated regions, the global integration unit 23 may integrate the locally integrated regions into an integrated region. Then, the global integration unit 23 may add and integrate another locally integrated region having similar locally integrated region features with any of locally integrated regions included in the integrated region to the integrated region.

  Also, as described above, the global integration unit 23 may use a normalized histogram based on at least one of the direction and the magnitude of the luminance gradient vector as the local integration region feature.

  In addition, the global integration unit 23 may exclude the integrated area as a false detection when the number or area of partial areas constituting the integrated area is less than a specified number as a result of the global integration.

  Next, the regular pattern determination unit 24 determines whether or not the integrated area is a regular pattern based on the distribution characteristic of the luminance gradient vector in the integrated area. As described above, the regular pattern determination unit 24 regularly uses the density of the brightness gradient vector, the concentration region, one-way property, point symmetry, or a combination thereof as the distribution characteristic of the brightness gradient vector. It is sufficient to judge the pattern. Then, the regular pattern determination unit 24 outputs the determination result (step S15). For example, as described above, the regular pattern determination unit 24 may output, as the regular pattern area, a rectangular area including the integrated area determined to be the regular pattern.

  Thus, the image processing apparatus 2 ends the operation.

  Next, the operation of the image processing apparatus 2 will be shown as a specific example.

  For example, it is assumed that a grayscale image generated from an input image is FIG. In this case, the luminance gradient vector calculation unit 21 generates lattice-like partial regions as shown in FIG. 13 (step S11).

  Next, the brightness gradient vector calculation unit 21 calculates a brightness gradient vector for each partial region of FIG. 13 (step S12). The calculated brightness gradient vector is, for example, as shown by an arrow in FIG. FIG. 14 is an enlarged view of a region surrounded by a dotted line in FIG.

  Next, the local integration unit 22 integrates partial regions which are spatially close to each other and to which the luminance gradient vectors are similar, and generates a local integrated region (thick rectangle) as shown in FIG. 15 (step S13). ). In addition, the local integration unit 22 excludes the partial area (hatched portion) not integrated with any partial area from the local integration area. Thus, in step S13, luminance gradients similar to each other among partial regions corresponding to the contour of the regular pattern are based on the characteristic that "the luminance gradient vectors of the close positions on the contour of the regular pattern are similar". Close subregions having vectors are integrated into one.

  As a result of step S13, for example, as shown in FIG. 16, it is assumed that locally integrated regions a1 to a12 and b1 to b12 are generated.

  Here, the global integration unit 23 generates a normalized histogram based on the direction of the luminance gradient vector as the local integration region feature for the local integration regions a1 to a12 and b1 to b12. Then, the global integration unit 23 generates an integrated area in which the locally integrated area is integrated based on the similarity of the locally integrated area feature (step S14).

  Here, the normalized histograms based on the directions of the luminance gradient vectors in the local integrated regions a1 to a12 shown in FIG. 16 have a tendency as shown in FIG. That is, the locally integrated region features (normalized histograms) of the locally integrated regions a1 to a12 are determined to be similar according to equation (7). Therefore, as shown in FIG. 18, the global integration unit 23 integrates the locally integrated regions a1 to a12 to generate an integrated region A.

  Further, the normalized histogram based on the direction of the brightness gradient vector in the local integrated regions b1 to b12 shown in FIG. 16 has a tendency as shown in FIG. 17 (b). That is, the locally integrated region features (normalized histograms) of the locally integrated regions b1 to b12 are determined to be similar according to equation (7). Therefore, as shown in FIG. 18, the global integration unit 23 integrates the locally integrated regions b1 to b12 to generate an integrated region B.

  Thus, in step S14, based on the characteristic that "the same kind of local pattern repeatedly exists in the regular pattern", the configuration of the luminance gradient vectors similar to each other among the locally integrated regions corresponding to the contour of the regular pattern The locally integrated area having is integrated into one. Note that the configuration of the luminance gradient vector refers to what magnitude and direction of the luminance gradient vector is present in the local integrated region at what rate.

  Next, the regular pattern determination unit 24 determines whether each of the integrated regions A and B is a regular pattern (step S15). Here, the case of performing the determination process using the concentration area described in (24-2) as the distribution characteristic of the luminance gradient vector will be described. In this case, in the circumscribed rectangle including the integrated area A, many luminance gradient vectors having vector sizes larger than the threshold exist in the central area rather than the end area. The same applies to the integrated area B. Therefore, the regular pattern determination unit 24 determines that the integrated regions A and B are regular patterns.

  Then, as shown in FIG. 19, the regular pattern determination unit 24 outputs, as a regular pattern area, a rectangular area including all of the integrated areas A and B determined to be regular patterns.

  This is the end of the description of the specific operation of the image processing apparatus 2.

  Next, the effects of the second embodiment of the present invention will be described.

  The image processing apparatus as the second embodiment of the present invention can detect regular patterns in an image with higher accuracy without designating the image features in advance.

  I will explain the reason. In the present embodiment, the luminance gradient vector calculation unit calculates a luminance gradient vector representing the amount of change in luminance in the horizontal direction and the vertical direction for each partial region obtained by dividing the input image into a grid. Then, the local integration unit generates a locally integrated region in which one or more adjacent partial regions are integrated based on the similarity of at least one of the magnitude or the direction of the luminance gradient vector. As a result, when there is a regular pattern in the image, a locally integrated region corresponding to a portion of the contour where the brightness gradient is similar is generated. Then, the global integration unit generates an integrated area integrating the one or more local integrated areas based on the similarity of the local integrated area features based on the luminance gradient vector. As a result, when the image has a regular pattern, an integrated area corresponding to a portion where contours having similar luminance gradients are repeated is generated. Then, the regular pattern determination unit determines whether the integrated area is a regular pattern based on the density of the brightness gradient vector in the integrated area, the concentration area, the one-way property, the point symmetry, a combination thereof, or the like. This is because the determination result is output.

  As described above, in this embodiment, a locally integrated region in which partial regions corresponding to the contour of the regular pattern are integrated into one is generated with higher accuracy. Then, in the present embodiment, an integrated area in which areas in which the same kind is repeated among local integrated areas (local patterns) corresponding to the contour of the regular pattern are integrated into one is generated with higher accuracy. As a result, in the present embodiment, the determination based on the density of the brightness gradient vector, the concentration area, the one-way property, the point symmetry or the like is performed on the integrated area where the local pattern may be repeated. It is possible to determine more accurately whether or not the pattern is regular. Therefore, in the present embodiment, it is not necessary to presume a regular pattern to be detected in advance, and a region where an unknown regular pattern exists can be detected with high accuracy.

  In the present embodiment, as an example of luminance gradient vector calculation processing, processing using a sub-block, processing using differentiation, and one having a predetermined number or more of substantially the same luminance gradient vectors in a small area Although the process to employ is shown, it is not restricted to this. The brightness gradient vector calculation unit may calculate the brightness gradient vector by other processing as long as it is information representing the feature of the change in brightness in the partial region.

  Moreover, although the process using Formula (2) or Formula (3) and the process using direction ID were shown as an example of the determination process of the similarity of a brightness | luminance gradient vector in this Embodiment, it does not restrict to this. The local integration unit may determine the similarity by another process based on at least one of the direction and the magnitude of the luminance gradient vector.

  Further, in the present embodiment, an example using Equations (4) to (6) which are normalized histograms based on at least one of the magnitude and the direction of the brightness gradient vector has been described as the locally integrated region feature. Not limited to. The locally integrated region feature may be other information as long as it is information representing the feature of the configuration of the luminance gradient vector in the locally integrated region.

  Also, in the present embodiment, processing using equation (7), which is the distance between normalized histograms, and processing by template matching are shown as determination processing of similarity of locally integrated area features, but processing is limited to this. Absent. The global integration unit may determine the similarity of the local integration region features using other processes that compare the composition of the intensity gradient vectors between the local integration regions.

  Further, in the present embodiment, as the determination process of the regular pattern of the integrated area, the process based on the density of the luminance gradient vector, the concentration area, the one-way property, the point symmetry or the combination thereof is shown. Not limited to. The regular pattern determination unit may determine whether it is a regular pattern based on other information representing the distribution characteristic of the luminance gradient vector in the integrated area.

  In each of the above-described embodiments of the present invention, the image processing apparatus has been described as the image to be inspected, but the image processing apparatus considers an image formed of a plurality of frame images as an inspection object. You may get it. Then, in this case, the image processing apparatus may be configured to extract a frame image from the video and function on the extracted frame image. In this case, each embodiment can be applied as a function of detecting a video including a regular pattern in a video editing system for editing a video recorded in an auxiliary storage device or the like. In addition, each embodiment may be applied to a video camera, a display terminal or the like having a function of detecting a regular pattern by being configured to function on a frame image group held on the main storage device. it can.

  Also, in each of the above-described embodiments of the present invention, the example has been mainly described in which each functional block of the image processing apparatus is realized by a CPU that executes a computer program stored in a storage device or a ROM. The present invention is not limited to this, and some, all, or a combination of each functional block may be realized by dedicated hardware.

  Further, in each embodiment of the present invention described above, the functional blocks of the image processing apparatus may be distributed to a plurality of apparatuses and realized.

  Further, in each embodiment of the present invention described above, the operation of the image processing apparatus described with reference to the flowcharts is stored as a computer program of the present invention in a storage device (storage medium) of the computer apparatus. Then, the CPU may read out and execute the computer program. And, in such a case, the present invention is constituted by such computer program code or storage medium.

  Moreover, each embodiment mentioned above can be implemented combining suitably.

  Furthermore, the present invention is not limited to the above-described embodiments, and can be implemented in various aspects.

Moreover, although a part or all of each embodiment mentioned above may be described also as the following additional remarks, it is not restricted to the following.
(Supplementary Note 1)
A brightness gradient vector calculation unit that calculates a brightness gradient vector representing the feature of the change in brightness for each partial region of the input image;
A local integration unit that generates a local integration region integrating one or more of the partial regions based on the similarity of the luminance gradient vectors;
An integrated area integrating one or more of the locally integrated areas is generated based on the similarity of features (locally integrated area features) of the locally integrated area calculated based on the luminance gradient vector in the locally integrated area. A global integration department,
A regular pattern determination unit that determines whether or not the integrated area is a regular pattern based on distribution characteristics of the luminance gradient vector in the integrated area, and outputs a determination result;
Image processing device equipped with
(Supplementary Note 2)
The luminance gradient vector calculation unit uses the respective blocks obtained by dividing the image in a grid shape as the partial region, and the vectors each having horizontal and vertical luminance fluctuation amounts in the partial region as components. The image processing apparatus according to claim 1, which is calculated as
(Supplementary Note 3)
The local integration unit integrates one or more of the partial regions to generate the local integration region based on the similarity in at least one of the magnitude and the direction of the luminance gradient vector. The image processing apparatus according to 1 or 2.
(Supplementary Note 4)
The local integration unit integrates the spatially adjacent front partial regions based on the similarity of the luminance gradient vectors to generate the local integration region. The image processing device according to one.
(Supplementary Note 5)
The global integration unit calculates a normalized histogram relating to at least one of the magnitude and the direction of the luminance gradient vector in the locally integrated region as the locally integrated region feature, and compares the normalized histograms among the locally integrated regions. The image processing according to any one of appendices 1 to 4, characterized in that the locally integrated area is integrated to generate the integrated area by determining the similarity of the locally integrated area feature. apparatus.
(Supplementary Note 6)
The global integration unit uses, as the local integration region feature, at least one of the magnitude and the direction of the luminance gradient vector at the position of each of the partial regions included in the local integration region, between the local integration regions. The local integrated area is integrated to generate the integrated area by comparing the magnitudes and / or directions of the luminance gradient vectors at corresponding positions to determine the similarity of the local integrated area feature. The image processing apparatus according to any one of appendixes 1 to 5 (claim 7)
The regular pattern determination unit is characterized in that whether or not the integrated area is a regular pattern is determined based on the density of a partial area having a luminance gradient vector having a size equal to or larger than a specified value in the integrated area. The image processing apparatus according to any one of appendices 1 to 6.
(Supplementary Note 8)
The regular pattern determination unit is characterized in that whether or not the integrated area is a regular pattern is determined based on an area where a luminance gradient vector having a size equal to or larger than a specified value is concentrated in the integrated area. The image processing apparatus according to any one of appendices 1 to 6.
(Appendix 9)
The regular pattern determination unit determines whether or not the integrated area is a regular pattern based on the one-way property of the luminance gradient vector in the integrated area. The image processing apparatus according to any one.
(Supplementary Note 10)
The regular pattern determination unit determines whether or not the integrated area is a regular pattern based on point symmetry of the luminance gradient vector in the integrated area. The image processing apparatus according to any one.
(Supplementary Note 11)
For each partial region of the input image, a luminance gradient vector representing the characteristic of the change in luminance is calculated
Generating a locally integrated region integrating one or more of the partial regions based on the similarity of the luminance gradient vectors;
An integrated area obtained by integrating one or more of the locally integrated areas is generated based on the similarity of features (locally integrated area features) of the locally integrated area calculated based on the luminance gradient vector in the locally integrated area. ,
An image processing method, which determines whether or not the integrated area is a regular pattern based on the distribution characteristic of the luminance gradient vector in the integrated area, and outputs a determination result.
(Supplementary Note 12)
A luminance gradient vector calculating step of calculating a luminance gradient vector representing the characteristic of the change in luminance for each partial region of the input image;
A local integration step of generating a local integration region integrating one or more of the partial regions based on the similarity of the luminance gradient vectors;
An integrated area integrating one or more of the locally integrated areas is generated based on the similarity of features (locally integrated area features) of the locally integrated area calculated based on the luminance gradient vector in the locally integrated area. Global integration steps,
A regular pattern determination step of determining whether or not the integrated region is a regular pattern based on the distribution characteristic of the luminance gradient vector in the integrated region, and outputting a determination result;
An image processing program that causes a computer device to execute

Reference Signs List 1 image processing device 2 image processing device 11, 21 luminance gradient vector calculation unit 12, 22 local integration unit 13, 23 global integration unit 14, 24 regular pattern determination unit 1001 CPU
1002 memory 1003 output device 1004 input device

Claims (10)

A brightness gradient vector calculation unit that calculates a brightness gradient vector representing the feature of the change in brightness for each partial region of the input image;
A local integration unit that generates a local integration region integrating one or more of the partial regions based on the similarity of the luminance gradient vectors;
An integrated area integrating one or more of the locally integrated areas is generated based on the similarity of features (locally integrated area features) of the locally integrated area calculated based on the luminance gradient vector in the locally integrated area. A global integration department,
A regular pattern determination unit that determines whether or not the integrated area is a regular pattern based on distribution characteristics of the luminance gradient vector in the integrated area, and outputs a determination result;
Image processing device equipped with   The luminance gradient vector calculation unit uses the respective blocks obtained by dividing the image in a grid shape as the partial region, and the vectors each having horizontal and vertical luminance fluctuation amounts in the partial region as components. The image processing apparatus according to claim 1, wherein:   The local integration unit integrates one or more of the partial areas to generate the local integration area based on the similarity of at least one of the magnitude and the direction of the luminance gradient vector. An image processing apparatus according to claim 1 or 2.   The global integration unit calculates a normalized histogram relating to at least one of the magnitude and the direction of the luminance gradient vector in the locally integrated region as the locally integrated region feature, and compares the normalized histograms among the locally integrated regions. The local integrated area is integrated to generate the integrated area by determining the similarity of the local integrated area feature, and the integrated area is generated according to any one of claims 1 to 3. Image processing device.   The regular pattern determination unit is characterized in that whether or not the integrated area is a regular pattern is determined based on the density of a partial area having a luminance gradient vector having a size equal to or larger than a specified value in the integrated area. The image processing apparatus according to any one of claims 1 to 4, wherein   The regular pattern determination unit is characterized in that whether or not the integrated area is a regular pattern is determined based on an area where a luminance gradient vector having a size equal to or larger than a specified value is concentrated in the integrated area. The image processing apparatus according to any one of claims 1 to 4.   The said regular pattern determination part determines whether the said integrated area | region is a regular pattern based on the one-way nature of the said brightness | luminance gradient vector in the said integrated area | region, The image processing device according to any one of 4.   The said regular pattern determination part determines whether the said integrated area | region is a regular pattern based on the point symmetry of the said brightness | luminance gradient vector in the said integrated area | region, It is characterized by the above-mentioned. The image processing device according to any one of 4. For each partial region of the input image, a luminance gradient vector representing the characteristic of the change in luminance is calculated,
Generating a locally integrated region integrating one or more of the partial regions based on the similarity of the luminance gradient vectors;
An integrated area obtained by integrating one or more of the locally integrated areas is generated based on the similarity of features (locally integrated area features) of the locally integrated area calculated based on the luminance gradient vector in the locally integrated area. ,
An image processing method, which determines whether or not the integrated area is a regular pattern based on the distribution characteristic of the luminance gradient vector in the integrated area, and outputs a determination result. A luminance gradient vector calculating step of calculating a luminance gradient vector representing the characteristic of the change in luminance for each partial region of the input image;
A local integration step of generating a local integration region integrating one or more of the partial regions based on the similarity of the luminance gradient vectors;
An integrated area integrating one or more of the locally integrated areas is generated based on the similarity of features (locally integrated area features) of the locally integrated area calculated based on the luminance gradient vector in the locally integrated area. Global integration steps,
A regular pattern determination step of determining whether or not the integrated region is a regular pattern based on the distribution characteristic of the luminance gradient vector in the integrated region, and outputting a determination result;
An image processing program that causes a computer device to execute