JP6700873B2 - Image processing device, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing device that processes an input image, an image processing method, and a program.

  Conventionally, it is possible to determine whether or not each pixel in an image of a subject or the like is a pixel forming a line, and smooth the luminance value of each pixel determined to form a line. There is known a method capable of reducing noise in an image while maintaining the shape of a subject image in the image. There are various known techniques for determining whether or not pixels are pixels that form a line. For example, in Non-Patent Document 1, a line area and a background area are set for an image, an average value of pixel luminance values is calculated for each area, and the average value of the line area is larger than the average value of the background area. A technique is disclosed in which, when different, each pixel in the line region is determined to be a pixel forming a line. In the technique described in Non-Patent Document 1, various directions are set by setting line areas in a plurality of directions and checking whether or not the average value of the line areas is significantly different from the average value of the background area in each of the set directions. The line of can be detected.

G. J. VanderBrug, "Line Detection In Satellite Image," [February 17, 2016 search], Internet <URL: http://docs.lib.purdue.edu/lars_symp/55/>

  In the case of the above-mentioned conventional technique, if the image area used for determining an area having a predetermined shape (linearity) such as a line area from the image is a small image area of about 3×3 pixels, for example. The image processing execution time is short. However, if, for example, the image area is enlarged in order to improve the determination accuracy of areas such as lines, the amount of calculation for obtaining the average value of the brightness values of each pixel in the image area increases, and the execution time of the image processing is increased. Will be long.

  Therefore, an object of the present invention is to make it possible to execute image processing for determining an area having a predetermined shape property in a short time regardless of the size of the area used for image processing.

The present invention generates a rotated image obtained by rotating an input image, generates an integral image obtained by integrating each pixel value of the input image and an integral image obtained by integrating each pixel value of the rotated image, and The feature is that the line region is specified based on the integral value calculated from the region of the integral image corresponding to the neighborhood region.

  According to the present invention, it is possible to execute image processing for determining an area having a predetermined shape property in a short time regardless of the size of the area used for image processing.

It is a figure which shows the function structure of the image processing apparatus of embodiment. It is a figure used for description of an integral image. It is a figure which shows the example of a setting of a line area and a background area. 7 is a flowchart of image processing in the image processing apparatus. It is a flowchart of the process which calculates|requires linearity from the integral image of an input image. It is a flowchart of the process which calculates|requires linearity from the integral image of a rotation image. It is a figure used for explaining the relation and coordinates of an input image and a rotation image. It is a figure which shows the example of a setting of a line area and background area which differ in thickness. It is a figure which shows the example of a setting of a line area and background area which differ in length.

Embodiments of the present invention will be described below with reference to the drawings.
<Configuration Example of Image Processing Device (Configuration Example of Parallel Processing)>
FIG. 1A is a block diagram showing the functional arrangement of the image processing apparatus 100 of this embodiment. Each unit illustrated in FIG. 1A of the image processing apparatus 100 of the present embodiment may be formed as a hardware configuration, or may be realized by a CPU or the like executing the image processing program of the present embodiment. Good.

  The image processing apparatus 100 is supplied with, for example, an image signal obtained by photographing a subject as an input image 101, and generates an image signal generated by performing image processing such as noise reduction processing on the input image 101. This is a device for outputting as an output image 102. It should be noted that the input image 101 is not limited to an image signal in which the image pickup apparatus is photographing an object, and various paths such as an image signal read from a recording medium and an image signal supplied via a communication path or the like. It may be any of the image signals via the. The image processing apparatus 100 of this embodiment includes a line area determination unit 103 and a line area processing unit 104.

  The line area determination unit 103 determines whether or not a line area, which is an example of an area having a predetermined shape, exists in the input image 101, and when the line area exists, at which position the line area is located, The direction of the line is detected. Specifically, the line region determination unit 103 includes a plurality of image rotation units, an integral image generation unit, and a linearity calculation unit, and also includes one linearity maximum region selection unit. The image rotation unit is an example of a rotation unit, the integral image generation unit is an example of a generation unit, and the linearity calculation unit and the linearity maximum region selection unit are an example of a specification unit. In the example of FIG. 1A, only the plurality of image rotation units 105B to 105D, the plurality of integrated image generation units 106A to 106D, and the plurality of linearity calculation units 107A to 107D are included in the line region determination unit 103. , And one linearity maximum region selection unit 108.

  The image rotation units 105B to 105D generate a plurality of rotated images in which the input image 101 is rotated at different rotation angles so as to be tilted. For example, the image rotation unit 105B performs a rotation process of rotating the input image 101 by, for example, 22.5 degrees and resampling it to generate a rotated image. Similarly, the image rotation unit 105C rotates the input image 101 by, for example, 45 degrees, and the image rotation unit 105D rotates the input image 101 by, for example, 67.5 degrees, and performs rotation processing for resampling, respectively, to generate rotated images. .. These rotation angles are examples. Each rotation angle can be arbitrarily set according to the direction of the line to be detected from the input image 101 (angle with respect to the horizontal (x axis) or vertical (y axis) of the image). When the rotation angle with respect to the input image 101 is set more finely than the above-mentioned 22.5 degrees, 45 degrees, and 67.5 degrees, the image rotation unit uses the example of FIG. 1A according to the set angles. Many are also provided. In this case, the integrated image generation unit and the linearity calculation unit in the subsequent stage are similarly provided in correspondence with the image rotation unit. Further, in the present embodiment, linear interpolation is used at the time of resampling of the rotation processing, but other known interpolation methods may be used as the interpolation method at the time of resampling. Each of the image rotation units 105B to 105D outputs a signal of a rotated image, and also outputs information of an image rotation angle (rotation amount) at the time of rotation processing and coordinate information of each pixel of the input image 101. The signals of the rotated images, the information of the rotation angles, and the coordinate information of each pixel of the input image 101, which are respectively output from the image rotation units 105B to 105D, are sent to the corresponding integral image generation units 106.

  The integral image generation unit 106A generates an integral image from the input image 101. Further, the integral image generation units 106B to 106D generate integral images from the rotation images supplied from the corresponding image rotation units 105B to 105D, respectively.

  The integrated image will be described below with reference to FIGS. 2(A) to 2(C). 2A to 2C, each square delimited by a dotted line represents a pixel. 2A, when the pixel p11 at the upper left corner of the input image 101 is set as the origin, the pixels p11 to p13,..., P21 to p23,. , And an example of each luminance value of each pixel (each numerical value in the drawing represents a luminance value). Further, FIG. 2B shows an integral image calculated from the image of FIG. Numerical values of the pixels P11 to P13,..., P21 to P23,..., P31 to P33,... Of the integrated image in FIG. 2B represent integrated values. The respective integrated values of the respective pixels P11 to P33 of FIG. 2B are included in a rectangular area composed of horizontal and vertical pixels from the origin pixel p11 of FIG. 2A to the target pixel. It is a value obtained by integrating each luminance value. In the present embodiment, the target pixel is a notation used when a description is given by focusing on a certain pixel among all the pixels forming the image, and all the pixels in the image are the target pixel. To do.

  Here, when the integrated image is generated, the input image 101 is scanned, for example, from the left end to the right end of the horizontal line, and after reaching the right end, the line is lowered vertically by one line and the horizontal line is again scanned. It is assumed that scanning is performed from the left end to the right end. In this case, the integrated value of the pixel of interest is the luminance value of each pixel on the left side of the pixel of interest in the row of the pixel of interest and the integrated value of the pixel vertically adjacent to the pixel of interest in the row for which the integrated value has already been calculated. It is calculated by adding and. For example, when the pixel p33 in FIG. 2A is the target pixel and the integral value of the pixel P33 corresponding to the target pixel p33 in FIG. 2A is obtained in FIG. 2B, the calculation is performed as follows. To be done. In the row of the target pixel p33, the brightness values of the pixels p31 and p32 on the left side of the target pixel p33 and the target pixel p33 are “3, 1, 5”. In the row for which the integral value has been calculated, the integral value of the pixel vertically adjacent to the target pixel p33 is the integral value “20” of the pixel P20 in FIG. 2B. Therefore, the luminance value “3, 1, 5” of each pixel p31, p32, p33 in the row of the target pixel p33 and the integral value “20” of the pixel P20 adjacent to the target pixel p33 in the row for which the integral value has been calculated. 2 is calculated, the integrated value “29” of the pixel P33 in FIG. 2B is calculated. In this way, the amount of calculation per pixel is constant in the integral image generation process, and the integral image can be generated by scanning the entire image once.

  If an integral image is used, the sum of the brightness values of the pixels in an arbitrary rectangular area of the integral image, that is, the integral value of the rectangular area is calculated with a constant calculation amount regardless of the size of the rectangular area. be able to. For example, in the input image 101 shown in FIG. 2C, when obtaining the total luminance value of each pixel in the rectangular area 220 indicated by the thick line in the figure, that is, the integrated value of the rectangular area 220, the following is performed. Can be asked. The integrated value of the rectangular area 220 is read from the integrated image of each integrated value of the adjacent pixels 201 to 204 indicated by circled circles (◯) of the rectangular area 220, and the integrated value of the adjacent pixels 201 and 204 is added to determine the adjacent value. It is obtained by subtracting the added value of the integrated values of the pixels 202 and 203.

Returning to FIG.
The integral image generation unit 106A receives the image signal of the integral image, information about the rotation angle of the input image 101 (the rotation angle is 0 degrees because the input image 101 is not rotated), and coordinate information about each pixel of the input image 101. Is output to the linearity calculation unit 107A. Since the rotation angle of the input image 101 is 0 degree, the integral image generation unit 106A may not output the rotation angle information of the input image 101. In addition, the integral image generating units 106B to 106D respectively correspond to the linearity corresponding to the image signals of the integral images generated by the integral image generating units 106B to 106D together with the rotation angle information from the image rotating units 105B to 105D and the coordinate information of each pixel of the input image 101. It outputs to calculation part 107B-107D.

  The linearity calculation unit 107A determines each pixel (each pixel of interest) of the input image 101 based on the integral image generated by the integral image generation unit 106A, the rotation angle information, and the coordinate information of each pixel of the input image 101. A value representing a predetermined shape property is calculated. Specifically, when each pixel of the input image 101 is a pixel of interest, the linearity calculation unit 107A calculates a value representing a predetermined shape of an integrated image region corresponding to a region near the pixel of interest. It is set as a judgment area, and a value representing the shape is calculated for the pixel of interest in each judgment area. In the case of the present embodiment, the linearity calculation unit 107A calculates a value representing the linear shape of the image (hereinafter referred to as “linearity”) as the value representing the predetermined shape. In addition, in the present embodiment, the linearity calculation unit 107A sets, as the determination area (integral image area), a rectangular determination area formed of at least two areas, a first area and a second area. Details of the determination region, the linearity, and the value indicating the linearity will be described later.

  Similarly, the linearity calculation units 107B to 107D input the input images based on the integral images supplied from the corresponding integral image generation units 106B to 106D, the rotation angle information, and the coordinate information of each pixel of the input image 101. A value representing the linearity corresponding to each pixel 101 is calculated. The linearity calculation units 107B to 107D use the determination area set for each pixel of interest of the input image 101 to calculate a value representing linearity for each image of interest.

  The determination region will be described below by taking the linearity calculation unit 107A as an example. The linearity calculation unit 107A sets a determination region centered on the pixel of interest of the input image 101, for example. 3A and 3B show determination regions 300 and 310 as an example.

  FIG. 3A shows a determination area 300 for detecting whether or not the target pixel pt is highly likely to be a pixel forming a line in the horizontal direction (horizontal direction) in the input image 101. The determination area 300 is a rectangular area of 9×9 pixels centered on the target pixel pt. In the determination area 300, a horizontal line area 301 is set as a first area, and a background area 302 other than the line area is set as a second area. The background area 302 is set so as to sandwich the line area 301. In the determination area 300 shown in FIG. 3A, the ratio of the widths of the line area 301 and the background area 302 in the vertical direction is set to 1:8. As described above, each of the line area 301 and the background area 302 is a rectangular area. Therefore, in the present embodiment, by using the integral image of the input image 101, the integral value of the line area 301 and the integral value of the background area 302 of those rectangular areas are constant, regardless of the size of the rectangular area. Can be calculated by

  FIG. 3B shows a determination area 310 for detecting whether or not the target pixel pt is highly likely to be a pixel forming a line in the vertical direction (vertical direction) in the input image 101. The determination region 310 is a rectangular region of 9×9 pixels centered on the pixel of interest pt, a vertical line region 311 is set as the first region, and a background that sandwiches the line region 311 as the second region. The area 312 is the set area. Further, in the determination area 310 shown in FIG. 3B, the ratio of the widths of the line area 311 and the background area 312 in the horizontal direction is set to 1:8. As in the case of FIG. 3A, the line area 311 and the background area 312 are rectangular areas. Therefore, by using the integral image of the input image 101, the integral value of the line area 311 and the integral value of the background area 312 of those rectangular areas can be calculated with a constant calculation amount regardless of the size of the rectangular area. ..

  The linearity calculation unit 107A sets the determination area 300 of FIG. 3A and the determination area 310 of FIG. 3B as described above for the target pixel pt. Then, in the linearity calculation unit 107A, the pixel of interest pt forms a horizontal line in the input image 101 based on the integrated value of the line area 301 and the integrated value of the background area 302 of the determination area 300 of FIG. A value representing the likelihood of being a pixel, that is, a value representing linearity is calculated. Similarly, the linearity calculation unit 107A causes the target pixel pt to form a vertical line in the input image 101 based on the integrated value of the line area 311 and the integrated value of the background area 312 of the determination area 310 in FIG. 3B. A linearity value representing the high probability of being a constituent pixel is calculated. Details of the linearity and the calculation process of the value will be described later.

  The linearity calculation units 107B to 107D set the same determination regions as described above for the integral images supplied from the corresponding integral image generation units 106B to 106D. However, the determination area in this case is set as an area centered on the pixel corresponding to the target pixel of the input image 101 (hereinafter, this pixel is also referred to as the target pixel) in the integrated image generated from the rotated image. To be done. Then, the linearity calculation units 107B to 107D calculate the linearity value for the pixel of interest within the determination region set in the integral image generated from the rotated image. Details of the determination region setting processing and the linearity value calculation performed by these linearity calculation units 107B to 107D will be described later. The linearity calculation units 107A to 107D output the linearity value of the pixel of interest calculated for each determination region to the maximum linearity region selection unit 108 together with the respective rotation angle information and coordinate information.

  The maximum linearity region selection unit 108 selects a determination region corresponding to the pixel of interest having the largest linearity value among the linearity values calculated by the linearity calculation units 107A to 107D, and makes the selected determination. The line area of the input image 101 is specified based on the area. Here, in the present embodiment, the determination region selected by the maximum linearity region selection unit 108 is the input image 101 (rotation angle is 0 degree), rotations of 22.5 degrees, 45 degrees, and 67.5 degrees. It is the determination area set in any of the images. Therefore, when the maximum linearity region selection unit 108 selects the determination region of the 45-degree rotated image, for example, the linearity maximum region selection unit 108 tilts 45 degrees to the position corresponding to the determination region of the 45-degree rotated image in the input image 101. The presence of a line area can be detected. Although details will be described later, there is a case where the determination area is not selected. Then, the maximum linearity region selection unit 108 uses the linearity value of the target pixel corresponding to the line region of the selected determination region, the rotation angle information, and the coordinate information to perform the line region smoothing of the line region processing unit 104. It is output to the unit 109.

  The line area smoothing unit 109 of the line area processing unit 104 is an example of a smoothing unit. The input image 101 is also supplied to the line region smoothing unit 109. The line area smoothing unit 109 calculates the input image 101 based on the linearity value of the target pixel corresponding to the line area of the determination area selected by the line area determination unit 103, the rotation angle information, and the coordinate information. The inside line area is identified, and the smoothing process is performed on the line area. Then, the line area smoothing unit 109 updates the line area of the input image 101 using the smoothed line area. The smoothing process in the line region smoothing unit 109 is, for example, a process of obtaining an average value of each pixel of the line region and updating each pixel value of the line region by the average value. In the present embodiment, smoothing is performed such that each pixel value in the line area is updated based on the average value of each pixel in the line area, but this is an example, and other smoothing processing may be performed. Good. The image signal updated by the smoothed line area is displayed on, for example, a display device (not shown) as an output image 102 after noise reduction by the image processing device 100, or is recorded on a recording medium by a recording device, or printed by a printing device. , Or by a communication device.

  FIG. 4A is a flowchart showing a processing procedure of image processing in the image processing apparatus 100 of FIG. The flowchart of FIG. 4A is a process executed by the line area determination unit 103 of FIG. Further, FIG. 4B is a flowchart of processing executed by the line area processing unit 104. The processing of the flowcharts of FIGS. 4A and 4B may be realized by executing an image processing program of the present embodiment stored in the ROM by the CPU or the like (not shown). In the following description, each step in the flowchart will be described only by reference numerals S105B to S105D, S106A to 106D, S107A to S107D, and the like, and the same applies to other flowcharts described later.

First, the flowchart of the image processing shown in FIG. 4A will be described.
4A is a process performed by the integral image generation unit 106A, and S107A is a process performed by the linearity calculation unit 107A. Further, S105B is processing performed by the image rotation unit 105B, S106B is processing performed by the integral image generation unit 106B, and S107B is processing performed by the linearity calculation unit 107B. Similarly, S105C is an image rotation unit 105C, S106C is an integral image generation unit 106C, S107C is a linearity calculation unit 107C, S105D is an image rotation unit 105D, S106D is an integral image generation unit 106D, and S107D is a linearity calculation unit 107D. This is the process performed.

  In the flowchart of FIG. 4A, the processing of the line area determination unit 103 starts when the input image 101 is supplied. When the process of the line area determination unit 103 starts, the processes of S106A to S107A, S105B to S107B, S105C to S107C, and S105D to S107D are executed in parallel.

The processing of S106A and S107A will be described below.
In S106A, the integral image generation unit 106A generates an integral image of the input image 101. After S106A, the process of the line area determination unit 103 proceeds to S107A performed by the linearity calculation unit 107A. In S107A, the linearity calculation unit 107A sets the above-described determination region for the integral image generated by the integral image generation unit 106A. Then, the linearity calculation unit 107A calculates the linearity value of the pixel of interest corresponding to the determination region as described above.

  FIG. 5 is a flowchart of detailed processing from the determination area setting to the linearity calculation performed by the linearity calculation unit 107A in S107A of FIG. 4A. The process of the flowchart of FIG. 5 is a process performed for each pixel (that is, each pixel of interest) of the input image 101.

  In step S301, the linearity calculation unit 107A sets the determination areas 300 and 310 in FIGS. 3A and 3B described above centered on the pixel of interest of the input image 101, for example. After the region setting process of S301, the process of the linearity calculation unit 107A proceeds to S302.

  In S302, the linearity calculation unit 107A calculates the integrated value of the line region 301 (sum of luminance values of pixels in the line region 301) and the background region 302 for the determination region 300 of FIG. 3A set in S301. An integral value (sum of luminance values of pixels in the background area 302) is calculated. Similarly, the linearity calculation unit 107A calculates the integrated value of the line area 311 and the integrated value of the background area 312 for the determination area 310 of FIG. After the integrated value calculation process of S302, the process of the linearity calculation unit 107A proceeds to S303.

  In S303, the linearity calculation unit 107A calculates an average value by dividing the integrated value of the line area 301 by the number of pixels in the area for the determination area 300, and calculates the average value of the integrated value of the background area 302 by the number of pixels in the area. Divide by to calculate the average value. Similarly, the linearity calculation unit 107A calculates the average value of the line area 311 and the average value of the background area 312 for the determination area 310. After the average value calculation processing of S303, the processing of the linearity calculation unit 107A proceeds to S304.

  In S304, the linearity calculation unit 107A calculates the ratio between the average value of the line area 301 and the average value of the background area 302 in the determination area 300. Similarly, the linearity calculation unit 107A calculates the ratio between the average value of the line area 311 and the average value of the background area 312 of the determination area 310. Here, the larger the difference between the average values of the line area 301 and the background area 302 in the determination area 300 and the larger the ratio of the average values, the more the target pixel pt of the determination area 300 is in the input image 101. It can be said that the pixels are highly likely to form a line in the horizontal direction. In addition, the larger the difference between the average values of the line area 311 and the background area 312 in the determination area 310 and the larger the ratio of the average values, the more the pixel of interest pt in the determination area 310 in the vertical direction in the input image 101. It can be said that the pixels are highly likely to form a line in the (vertical) direction. As described above, the value of the ratio calculated in S304 is a value indicating the possibility that the target pixel pt is a pixel forming a line, that is, a value indicating linearity. The linearity calculation unit 107A calculates the value of the ratio calculated in S304 as a value representing the linearity of the target pixel pt.

  Returning to the flowchart of FIG. 4A, the processing of S105C to S107C will be described. The processes of S105B to S107B and S105D to S107D are the same as the processes of S105C to S107C except that the rotation angle of the rotated image is different, and thus the description thereof will be omitted.

  In S105C, the image rotation unit 105C performs a rotation process of rotating the input image 101 by 45 degrees and resampling it as described above, and generates a rotated image. After S105C, the process of the line area determination unit 103 proceeds to S106C performed by the integral image generation unit 106C. In S106C, the integral image generation unit 106C generates an integral image from the rotation image supplied from the image rotation unit 105C. After S106C, the process of the line area determination unit 103 proceeds to S107C performed by the linearity calculation unit 107C.

  FIG. 6 is a detailed flowchart of the processing performed by the linearity calculation unit 107C in S107C of FIG. The process of the flowchart of FIG. 6 is a process performed on a pixel corresponding to each target pixel of the input image 101 in the integrated image of the rotated image obtained by rotating the input image 101 by 45 degrees. When the linearity calculation unit 107C proceeds to the processing of S107C in FIG. 4A, it first performs the processing of S501. S501 to S506 are processes performed in correspondence with each pixel of interest of the input image 101.

  In S501, the linearity calculation unit 107C calculates the coordinates of the target image sent from the integral image generation unit 106C (that is, the coordinate value of the target pixel of the input image 101) in the rotated image that is rotated by 45 degrees in S105C. Convert to coordinates in the coordinate system. The coordinate information conversion process is the same as the process used at the time of image rotation in S105C.

  Here, the relationship between each pixel of the input image 101 and the rotated image will be described in detail with reference to FIGS. 7(A) to 7(C). FIG. 7A shows a part of the input image 101, and it is assumed that each square delimited by a dotted line in the figure is a pixel. 7B and 7C show a part of the rotated image 700 after the input image 101 has been rotated by 45 degrees, and each square delimited by the solid line in the drawing indicates each of the rotated image 700. It is assumed to be a pixel. Further, it is assumed that the range surrounded by a frame 701 in FIGS. 7B and 7C corresponds to the input image 101 having the size shown in FIG. 7A. It should be noted that FIGS. 7B and 7C also show the input image 101 for easy understanding of the correspondence with the rotated image 700.

  As shown in FIG. 7B, the coordinates (center coordinates) of each pixel of the input image 101 and the coordinates (center coordinates) of each pixel of the rotated image 700 do not match and may be different. Easy to understand. Further, FIG. 7C shows the relationship between the target pixel pt of the input image 101 and the neighboring pixels in the rotated image 700 corresponding to the target pixel pt. As shown in FIG. 7C, the target pixel pt of the input image 101 can be obtained by interpolating from, for example, four neighboring pixels in the thick frame 710 of the rotated image 700. Therefore, in S501, the linearity calculation unit 107C performs coordinate conversion to calculate the coordinates of each neighboring pixel of the rotated image 700 from the coordinates of the target pixel pt of the input image 101. After S501, the processing of the linearity calculation unit 107C proceeds to S502.

  S502 to S505 are the processes performed for each neighboring pixel of the rotated image 700. S502 to S505 are the same processes as S301 to S304 described above except that the pixel of interest in the flowchart of FIG. 5 described above is a neighboring pixel, and thus detailed description of these S502 to S505 will be omitted. After S505, the processing of the linearity calculation unit 107C proceeds to S506.

  In S506, the linearity calculation unit 107C calculates a linearity value corresponding to the target pixel of the input image 101 from the linearity value obtained for each neighboring pixel in S502 to S505, for example, by linear interpolation. Although the linear interpolation is used here, the linearity value of the pixel of interest may be obtained by another known interpolation method.

  Description will be returned to the flowchart of FIG. As described above, the linearity value corresponding to the pixel of interest in each determination region calculated by the linearity calculation units 107A to 107D in S107A to S107D is the maximum linearity together with the corresponding rotation angle information and coordinate information. It is sent to the area selection unit 108. Then, the processing of the line area determination unit 103 proceeds to S108 performed by the maximum linearity area selection unit 108. Note that the line area determination unit 103 also adjusts the timing at which the linearity values, the rotation angle information, and the coordinate information are supplied to the linearity maximum area selection unit 108 from the linearity calculation units 107A to 107. ing. For the timing adjustment, for example, the integral images of the integral image generation units 106A to 106D, or the linearity values, the rotation angle information, and the coordinate information by the linearity calculation units 107A to 107 are temporarily stored in a memory or the like (not shown). This is done by adjusting the read timing.

  In S108, the maximum linearity region selection unit 108 selects the determination region corresponding to the pixel of interest having the largest linearity among the linearity values obtained in S107A to S107D, and selects the input image. The line area in 101 is specified. However, in the present embodiment, the maximum linearity region selection unit 108 does not select the determination region itself using the linearity value if the largest linearity value is equal to or less than the predetermined threshold value. .. In the case of the present embodiment, the maximum linearity region selection unit 108 uses, as the predetermined threshold value, the ratio value when the average value of the line region of the determination region is twice the average value of the background region. The predetermined threshold may be another threshold such as a preset threshold, or the predetermined threshold may not be used when selecting the determination region. When the predetermined threshold value is not used, as described above, the determination area corresponding to the target pixel having the largest linearity is selected. After S108, the line area determination unit 103 ends the processing of the flowchart of FIG. Then, from the maximum linearity region selection unit 108, as information indicating the specified line region, the linearity value of the target pixel corresponding to the line region and the rotation angle information and coordinate information corresponding to the target pixel are obtained. , To the line region smoothing unit 109 of the line region processing unit 104.

  FIG. 4B is a flowchart of processing performed by the line area smoothing unit 109 of the line area processing unit 104. The process of the flowchart of FIG. 4B is started when the process of S108 of FIG. 4A is completed. When the processing of the flowchart of FIG. 4B is started, the line area smoothing unit 109 performs the smoothing processing of the line area in S109, and updates the input image 101 using the smoothed line area. To do.

  In S109, the line area smoothing unit 109, based on the linearity value of the target pixel corresponding to the line area identified by the line area determination unit 103, and the rotation angle information and coordinate information of the target pixel, the input image. The line area in 101 is identified, and the line area is smoothed and updated. After S109, the processing of the line area processing unit 104 ends.

  As described above, the image processing apparatus 100 according to the present embodiment generates an integral image from the input image 101 and a plurality of rotated images obtained by rotating the input image 101 at different rotation angles, and the integrated image is generated. A predetermined determination area for detecting a line area is set for. Then, the image processing apparatus 100 calculates the linearity value for each determination area based on the integral value of the predetermined determination area, and the input image 101 is calculated according to the determination area having the largest linearity value. The line area of is detected. As described above, according to the image processing apparatus 100 of the present embodiment, since the integral value of the integral image is used, the amount of calculation per pixel becomes constant regardless of the size of the determination region used for detecting the line region. It is possible to detect line areas in various directions at high speed. Further, since the image processing apparatus 100 performs processing on the input image 101 and a plurality of rotated images in parallel, high speed processing is possible. As a result, the image processing apparatus 100 according to the present embodiment can perform the noise reduction processing of the line area at high speed.

<Another Configuration Example of Image Processing Device (Configuration Example of Sequential Processing)>
In FIG. 1A, a rotation image having a different rotation angle is generated in parallel from an input image 101 having a rotation angle of 0 degree, and the generation of an integral image and the linearity value are performed in parallel for the input image 101 and each rotation image. Although the example of performing the calculation is given, the respective processes may be performed sequentially. FIG. 1B shows an example of the configuration of the image processing apparatus 120 in the case where the processes of generating a rotated image, generating an integral image, and calculating the linearity value are sequentially processed each time the image is rotated. . Each unit of the image processing apparatus 120 in FIG. 1B may be formed as a hardware configuration, or may be realized by a CPU or the like executing an image processing program. The image processing apparatus 120 shown in FIG. 1B includes a line area determination unit 130 and a line area processing unit 104. In FIG. 1B, the same components as those shown in FIG. 1A are designated by the same reference numerals, and their description will be omitted.

  In the case of the configuration example of FIG. 1B, the line area determination unit 130 includes a holding unit 131, a control unit 132, and one image rotation unit 105, an integral image generation unit 106, a linearity calculation unit 107, and linearity. And a maximum area selection unit 108.

  The holding unit 131 holds the input image 101. The input image 101 held in the holding unit 131 is sent to the image rotation unit 105 under the control of the control unit 132. In the image rotation unit 105, the image rotation angle is sequentially set by the control unit 132. In the present embodiment, the control unit 132 reads the input image 101 from the holding unit 131 every time the rotation angle is 0 degree, 22.5 degrees, 45 degrees, and 67.5 degrees with respect to the image rotation section 105. , The image rotation angle is set to be sequentially changed. As a result, the image rotation unit 105 sequentially outputs the input images 101 with a rotation angle of 0 degrees, the rotation images with 22.5 degrees, 45 degrees, and 67.5 degrees. If the image quality is not deteriorated by the rotation process, the image after being rotated by the image rotating unit 105 is fed back to the image rotating unit 105 and further rotated to generate each rotated image. May be.

  The image signals sequentially output from the image rotation unit 105 are sent to the integral image generation unit 106. The integral image generation unit 106 sequentially generates integral images from the images of the rotation angles 0 degree, 22.5 degrees, 45 degrees, and 67.5 degrees sequentially supplied from the image rotation section 105. Each integral image generated by the integral image generating unit 106 is sequentially sent to the linearity calculating unit 107.

  The linearity calculation unit 107 is based on each integral image sequentially generated by the integral image generating unit 106, rotation angle information corresponding to each integral image, and coordinate information of each pixel (each pixel of interest) of the input image 101. Then, the linearity value corresponding to the pixel of interest of the input image 101 is sequentially calculated. Note that the rotation angle information and the coordinate information may be supplied from the image rotation unit 105 to the linearity calculation unit 107 via the integral image generation unit 106, as in the case of FIG. In the example of B), an example in which the control unit 132 generates and supplies the linearity calculation unit 107 is given. As in the case of the linearity calculation units 107A to 107D in FIG. 1A described above, the linearity calculation unit 107 sets a predetermined determination region corresponding to each pixel of interest of the input image 101, and makes these determinations. A linearity value corresponding to the pixel of interest in the area is calculated. Then, the linearity calculation unit 107 sends the linearity value calculated corresponding to each target pixel of the input image 101, the rotation angle information and the coordinate information of the target pixel to the maximum linearity region selection unit 108. Output.

  The maximum linearity region selection unit 108 and the line region smoothing unit 109 of the line region processing unit 104 are the same as those in FIG. 1A described above. As a result, the output image 102 after noise reduction is obtained from the image processing apparatus 120 of FIG.

  FIG. 4C is a flowchart showing a processing procedure of image processing in the image processing apparatus 120 of FIG. 1B that determines a line area by sequential processing. The flowchart of FIG. 4C is a process executed by the line area determination unit 130 of FIG. 1B, and is a process performed each time the rotation angle is sequentially set by the control unit 132. The process of the flowchart of FIG. 4C may be realized by executing the image processing program of this embodiment stored in the ROM by the CPU (not shown) or the like.

  In the flowchart of FIG. 4C, the image rotation unit 105 rotates the input image 101 by the set rotation angle and generates a rotation image every time the rotation angle is sequentially set by the control unit 132. .. For example, when an image (input image 101) with a rotation angle of 0 degrees is output from the image rotation unit 105 in S105, the process of the line area determination unit 130 proceeds to S106 performed by the integral image generation unit 106.

  In S106, the integral image generation unit 106 sequentially generates integral images from the rotation image supplied from the image rotation unit 105, similarly to the integral image generation units 106A to 106D in FIG. 1A described above. In S106, for example, when an image (input image 101) with a rotation angle of 0 degrees is supplied from the image rotation unit 105, an integral image is generated from the image. Then, after S106, the processing of the line area determination unit 103 proceeds to S107 performed by the linearity calculation unit 107.

  In S107, the linearity calculation unit 107 sequentially calculates the linearity value from the integral image supplied from the integral image generation unit 106, similarly to the linearity calculation units 107A to 107D in FIG. In S107, for example, when the integral image of the image (input image 101) having the rotation angle of 0 degree is supplied from the integral image generating unit 106, the linearity value is calculated from the integral image.

  When the process of S107 ends, the process returns to the process of S105, and the image rotation unit 105 is set to the next rotation angle by the control unit 132. When the rotation angle is set to 0 degree as described above, the next rotation angle is set to 22.5 degrees, for example. The setting of this rotation angle is an example, and another rotation angle may be set. In this way, the processing of S105 to S107 is performed every time the rotation angle is sequentially set.

  Then, when the setting of the rotation angle by the control unit 132 is completed and the linearity values calculated for each of the rotation angles are output to the linearity maximum region selection unit 108 in S107, FIG. The process of the flowchart ends.

  The image processing apparatus 120 that performs the sequential processing of FIG. 1B and FIG. 4C takes a slightly longer processing time than the image processing apparatus 100 that performs the parallel processing described above, but the noise reduction processing of the line region at high speed. It can be performed. On the other hand, the image processing apparatus 120 that performs sequential processing can be made smaller than the configuration of the image processing apparatus 100 that performs parallel processing. For example, in the case of the image processing apparatus 100 that performs the parallel processing described above, if the rotation angle of the rotation processing is set finely, the number of image rotation units, integral image generation units, and linearity calculation units increases. On the other hand, in the case of the image processing device 120 that performs sequential processing, only one image rotation unit, one integral image generation unit, and one linearity calculation unit are required, so that the configuration can be downsized and the cost can be reduced. Further, in the case of the image processing apparatus 120 that performs sequential processing, since the control unit 132 controls the setting of the rotation angle, it is very easy to change the setting of the rotation angle.

<Other example of determination area>
In the above example, the determination area 300 of FIG. 3A and the determination area 310 of FIG. Similarly, line determinations in two directions are performed from the rotated images of 45 degrees and 67.5 degrees. As described above, in the above-described example, lines in eight directions in total can be detected. However, if the rotation angle is set more finely, lines in more directions can be detected.

  Further, in the above-described example, the determination region 300 of FIG. 3A and the determination region 310 of FIG. 3B are given as the predetermined determination regions, but the determination region is not limited to these examples. For example, a determination area as shown in FIG. 3C may be set. Similar to FIG. 3A, FIG. 3C is a setting example of the determination area 320 for detecting a line in the horizontal (horizontal) direction corresponding to the target pixel pt. In the determination area 320 of FIG. 3C, the background areas 322 and 323 are set so as to approximate the line area 321 to a circle having the diameter of the line area 321. For example, if there is a vertical line 324 at the end or the like in the determination area 320 and the example of FIG. 3A is used, the value of the ratio of the average values of the line area 301 and the background area 302 is , The presence of the line 324 may lead to low values (low linearity values). On the other hand, in the case of the example of FIG. 3C, even if the vertical line 324 exists at the end or the like in the determination area 320, the value of the ratio of the average values of the line area 321 and the background area 322 is obtained. Does not have a low value due to the line 324, and a high linearity value is obtained. Therefore, by using the determination area 320 in FIG. 3C, even if a line other than the horizontal line to be detected exists at the end or the like, the linearity value of the horizontal line can be accurately obtained.

  As another example, a determination area 330 as shown in FIG. 3D may be set. In the determination area 330 of FIG. 3D, a line area 331 in the horizontal (horizontal) direction and a line area 332 in the vertical (vertical) direction are set to be orthogonal to each other at the pixel of interest pt, and other than the line areas 331 and 332. Is set as the background area 333. By using the determination area 330 of FIG. 3D, it is possible to detect the linearity value of the line in the horizontal (horizontal) direction and the line in the vertical (vertical) direction that are orthogonal to each other. In addition to the examples shown in FIGS. 3C and 3D, various determination areas can be set by combining rectangular areas.

  As still another example, for example, the determination area 800 in FIG. 8A, the determination area 810 in FIG. 8B, the determination area 820 in FIG. 8C, the determination area 900 in FIG. Each determination area such as the determination area 910 of B) and the determination area 920 of FIG. 9C may be set.

  FIG. 8A to FIG. 8C show examples of the determination region that enables calculation of linearity values for lines of various thicknesses. The determination area 800 in FIG. 8A is an example in which the ratio of the widths of the line area 801 and the background area 802 in the vertical direction is set to 1:6. The determination area 810 in FIG. 8B is an example in which the ratio of the widths of the line area 811 and the background area 812 in the vertical direction is set to 3:4. The determination area 820 of FIG. 8C is an example in which the ratio of the width in the vertical direction of the line area 821 and the background area 822 is set to 5:2. According to the determination region examples of FIGS. 8A to 8C, it is possible to detect line regions having different thicknesses. Note that the examples of FIGS. 8A to 8C cite the determination region for determining the horizontal line region, but if each line region is set in the vertical direction, linearity with respect to vertical lines of various thicknesses is set. It is also possible to calculate the value of.

  According to the examples of FIGS. 8A to 8C, even when detecting lines with various thicknesses, the image processing apparatus does not depend on the size of the determination region used for determination The amount of calculation per hit can be kept constant, and line determination in various directions and with various thicknesses can be performed at high speed.

  9(A) to 9(C) show examples of the determination region that enables calculation of linearity values for lines of various lengths. The determination area 900 in FIG. 9A is an example in which the ratio of the width of the line area 901 to the background area 902 in the vertical direction is set to 1:2, and the determination area 910 in FIG. 9A is a setting example in which the length of the line area 911 is enlarged by 5/3 times as compared with the setting example in FIG. 9A. Further, the determination area 920 in FIG. 9C is a setting example in which the length of the line area 921 is expanded by 7/3 times compared to the example in FIG. 9A. According to the determination region examples of FIGS. 9A to 9C, it is possible to detect line regions having different lengths. Note that the examples of FIGS. 9A to 9C cite the determination regions for determining horizontal line regions, but if each line region is set in the vertical direction, linearity for vertical lines of various lengths is obtained. It is also possible to calculate the value of.

  According to the examples of FIGS. 9A to 9C, even when detecting lines of various lengths, the image processing apparatus does not depend on the size of the determination area used for the determination The amount of calculation per hit can be kept constant, and line determination in various directions and various lengths can be performed at high speed.

<Modification>
In the above-described embodiment, the line area processing unit 104 exemplifies an example in which smoothing is performed by averaging the line areas. By doing so, it is possible to make the smoothing strength different for each line region. As an example, when smoothing a line area using a Gaussian filter, the strength of smoothing is changed by changing the sigma (variance value) that is a parameter of the Gaussian filter, and the degree of blurring of the line area is changed. It is possible to change it. More specifically, in some cases, it may be desirable to weaken the smoothing for a line area determined to have a large linearity value (strong linearity). In such a case, change the parameter. If so, the smoothing strength can be easily changed.

  Further, in the above-described embodiment, the noise reduction process is taken as an example, but the line region calculation process as described above can be easily applied to various applications such as calculation of a feature amount in pattern matching. Is. For example, in the calculation example of the characteristic amount, the linearity value is obtained as the characteristic amount of the image, and the pattern matching can be performed based on the characteristic amount represented by the linearity value.

  Further, in the above-described embodiment, an example in which the line area is determined has been described, but if the settings of the areas such as the line area and the background area described above are changed, the present invention is also applied to edge determination, flat area determination, and the like. It is possible. For example, in edge determination, two regions, a first region and a second region, are set for an image, the average value of the integrated values as described above is calculated for the first and second regions, and the If the difference (ratio) between the first and second areas is large, it can be determined that the area is an edge. Further, for example, with respect to the flat area, if there is almost no difference (ratio) between the first and second areas, it can be determined as the flat area.

  Furthermore, image processing such as performing the line area determination, the edge determination, the flat area determination, and the like as in the above-described embodiment, and performing different processing in each subsequent stage according to the plurality of determination results is also possible. It will be possible.

  Further, the above-described determination of the line area can be applied to, for example, detection of a scratch area or the like (including a dust area due to dust or the like) on the image pickup surface of the image pickup device that generates the input image. Then, when a flaw area or the like is detected based on the determination of the line area, a predetermined attention image (for example, a circle icon image surrounding the flaw area or the like) is input to the detected flaw area in the subsequent process. Something like superimposing it on the image. In addition, in the case of this example, the linearity value can be used as a value indicating whether or not there is a possibility of being a scratched area, and for example, according to the linearity value, the display form of a predetermined attention image is displayed. It is also possible to change. As an example, a display form example is conceivable in which the attention image is displayed in red when the linearity value is large and the possibility of a flaw is high.

  In the above-described embodiment, the example in which all the pixels are the target pixels has been described, but the target pixels may be pixels at predetermined intervals, for example. In this way, when the target pixel is a pixel at every predetermined interval, line determination is performed at intervals, which makes it possible to shorten the line determination execution time, and also to achieve necessary and sufficient detection accuracy. Since it is also possible to obtain it, the execution time and the detection accuracy can be balanced.

  In addition, although the two-dimensional image is handled in the above-described embodiment, image processing of a three-dimensional image is also possible. In this case, it is sufficient to generate a rotated three-dimensional image for each three-dimensional image, set the region as described above for the three-dimensional image, and determine, for example, a line region, an edge, a flat region, or the like.

<Other embodiments>
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. It can also be realized by the processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  The above-described embodiments are merely examples of specific embodiments for carrying out the present invention, and the technical scope of the present invention should not be limitedly interpreted by these. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

  100, 120 image processing device, 103, 130 line area determination unit, 104 line area processing unit, 105B to 105D, 105 image rotation unit, 106A to 106D, 106 integral image generation unit, 107A to 107D, 107 linearity calculation unit, 108 linearity maximum region selection unit, 109 line region smoothing unit

Claims (17)

Rotating means for generating a rotated image by rotating the input image;
Generating means for generating an integral image obtained by integrating each pixel value of the input image and an integral image obtained by integrating each pixel value of the rotated image;
Specifying means for specifying a line region based on an integral value calculated from a region of the integral image corresponding to a region near the pixel of interest of the input image;
An image processing apparatus comprising: The identifying means is
As a region of the integral image corresponding to a region near the pixel of interest in the input image, a determination region formed by a first region and a second region is set in the integral image, and the first region is set. Calculating a value representing the linear shape characteristic of the linear area based on the integral value of the area and the integral value of the second area,
The image processing apparatus according to claim 1, wherein a line region is specified in the input image based on the calculated value representing the line shape property. Rotating means for generating a rotated image by rotating the input image;
Generating means for generating an integral image obtained by integrating each pixel value of the input image and an integral image obtained by integrating each pixel value of the rotated image;
Specifying means for specifying a line area in the input image, and the specifying means,
In correspondence to the pixel of interest of the input image, and sets a determination area formed by the first and second regions in said integral image, the first area integral value and the second of Based on the integrated value of the area, calculate a value representing the linear shape that is the shape of the line region ,
An image processing apparatus, characterized in that a line region is specified in the input image based on a value representing the calculated line shape property corresponding to a pixel of interest of the input image. The rotation means generates a plurality of rotation images obtained by rotating the input image at different rotation angles,
The generating unit generates the integral image for each of the rotation images,
The identifying means is
Wherein by setting the determination region in correspondence to the pixel of interest of the input image for each of the integral image, calculates a value representative of the line shape of,
It is characterized in that the line region is specified in the input image based on a plurality of values representing the line shape characteristics calculated in association with the target pixel of the input image for each of the integrated images. The image processing device according to claim 2. The rotating means generates the plurality of rotated images in parallel,
The generation unit generates the integrated image in parallel for each of the rotation images,
The identifying means is
For each of the integrated images generated in parallel, the determination region corresponding to the target pixel of the input image is set in parallel, and the value representing the linear shape property corresponding to the target pixel of the input image is calculated. Then
The image processing apparatus according to claim 4, wherein the line region is specified in the input image based on a plurality of values representing the line shape characteristics calculated in parallel. The rotation means generates the plurality of rotation images by sequentially changing and rotating the rotation angle with respect to the input image,
The generation unit sequentially generates the integral image every time the rotation image is generated,
The identifying means is
Wherein each time the generation of the integral image is made, and sequentially calculates a value representative of the line shape of which corresponds to the pixel of interest of the input image,
The image processing apparatus according to claim 4, wherein the line region is specified in the input image based on the values of the plurality of line shapes that are sequentially calculated. The identifying means is
The value of the ratio of the average value of the integrated value of the first region average value and the second region of the integral value of, calculated as a value representing the line shape with respect to the pixel of interest of the input image,
7. The line area is specified in the input image based on a determination area in which the calculated values representing the plurality of line shape characteristics have a maximum value. The image processing device according to item 1. The specifying unit does not perform the specifying of the line region when the maximum value of the calculated values representing the plurality of linear shapes is equal to or less than a predetermined threshold value. 7. The image processing device according to 7.   The specifying means sets the value of the ratio when the average value of the integrated values of the first region is twice the average value of the integrated values of the second region to the predetermined threshold value. The image processing apparatus according to claim 8, which is characterized in that. The identifying means is
The line area is set to the first area, and the determination area in which the background area other than the line area is set to the second area is set,
10. The line area is specified in the input image based on a determination area in which the calculated values representing the plurality of line shape characteristics have the maximum values. The image processing device according to item 1.   The image processing apparatus according to claim 10, wherein the specifying unit sets a ratio of a width of the line area and a width of the background area of the determination area.   The image processing apparatus according to claim 10, wherein the specifying unit sets the lengths of the line area and the background area of the determination area. Based on the information representative of the specified the line regions, and identifying the line region in the input image, by smoothing the identified line area, the input image by a line region after the smoothing 13. The image processing apparatus according to claim 1, further comprising a smoothing unit that updates the image. The specifying means outputs, as the information indicating the specified line area, at least coordinate information of a pixel of interest of the input image and rotation angle information of the rotation image to the smoothing means,
The smoothing unit identifies the line region in the input image based on the coordinate information of the pixel of interest of the input image and the rotation angle information of the rotated image to perform the smoothing and the updating. The image processing apparatus according to claim 13, wherein: The specifying means further outputs, as the information indicating the specified line area, a value indicating a linear shape characteristic of the line area to the smoothing means,
It said smoothing means, in accordance with the value representing the line shape of the image processing apparatus according to claim 14, characterized in that to vary the strength of the smoothing. In an image processing method executed by an image processing apparatus for detecting a line area having line shape from an input image,
Rotate the input image to generate a rotated image,
Generate an integral image that integrates each pixel value of the input image and an integral image that integrates each pixel value of the rotated image,
An image processing method, characterized in that the line region is specified based on an integral value calculated from a region of the integral image corresponding to a region near a pixel of interest of the input image.   A program for causing a computer to function as each unit of the image processing apparatus according to claim 1.

Images