JP5799681B2 - Image processing apparatus, integral image creation method and program - Google Patents

Description

  The present invention relates to an image processing device, an integrated image creation method, and a program.

  An imaging device that captures a face image of a subject, an acquisition unit that acquires image data including the subject, a filter unit that executes an average difference filter on the acquired image data, and an image in which the average difference filter is executed Face area detecting means for detecting the face area of the subject from the data, and image pickup means for picking up a subject image by performing at least focusing control on the detected face area of the subject. 2. Description of the Related Art Conventionally, there has been known an imaging apparatus characterized in that a detection unit detects a face area of a subject using a feature amount created in advance from face image data of the same subject type on which an average difference filter has been executed. ing.

  In the imaging apparatus described above, the average difference filter process is performed on the pixels in the image data by calculating the absolute value of the difference between the average luminance value and the luminance value of the target pixel. The average luminance value is an average value of luminances of pixels in the vicinity of the target pixel (a certain pixel). It has been conventionally known that an integral image should be created in advance so that the average luminance value can be easily calculated. The integrated image is created by cumulatively calculating the sum of the luminance values of each pixel block (see, for example, Patent Document 1).

  However, when the integral image is stored in a storage device such as a RAM (Random Access Memory), the integral value becomes very large, and there is a problem that the storage capacity necessary for storing the integral image becomes enormous. .

  The present invention has been made in view of the above points, and an object thereof is to provide an image processing apparatus, an integrated image creation method, and a program capable of reducing the storage capacity necessary for storing an integrated image.

In order to solve the above-described problem, the image processing apparatus according to claim 1 is configured such that an area integral value of one coordinate included in the block and a vertical direction of one or more coordinates are divided in units of blocks into which the image is divided. Storage means for storing a vertical integral value obtained by cumulatively adding luminance values to the horizontal value, a horizontal integrated value obtained by cumulatively adding luminance values in the horizontal direction, and a luminance value, and area integration of all coordinates included in the block Control means for calculating a value from an area integral value of one coordinate read from the storage means, a vertical integral value of one or more coordinates, a horizontal integral value, and a luminance value.

  In addition, what applied the component, expression, or arbitrary combination of the component of this invention to a method, an apparatus, a system, a computer program, a recording medium, a data structure, etc. is also effective as an aspect of this invention.

  According to the present invention, it is possible to provide an image processing apparatus, an integrated image creation method, and a program that can reduce the storage capacity required for storing an integrated image.

It is explanatory drawing of an example of an integral image. It is explanatory drawing of an example of the calculation of the sum total of the brightness | luminance using an integral image. It is a hardware block diagram of an example of PC. It is explanatory drawing of an example of the integral image of a block unit in this Embodiment. It is an image figure of an example of the process which reads four area integral values from the data of the integral image of a block unit. It is explanatory drawing of the other example of the integral image of the block unit in this Embodiment. It is explanatory drawing of an example of the four-way integral image of the block unit in this Embodiment. It is an image figure of an example of the process which reads an area integral value from the data of a four-way integral image. It is explanatory drawing of an example of the integral image of the block unit of a 1st area | region to a 4th area | region. It is explanatory drawing of an example showing the scanning method at the time of face detection. It is a flowchart of an example of the algorithm of a face detection. It is explanatory drawing of an example showing the motion of RAM at the time of face detection. It is explanatory drawing of an example of a four-way integral image in case the number of initial lines is 224 lines and the number of update lines is 64 lines. It is explanatory drawing of an example of the process which scans 224 lines with a scanning window. It is explanatory drawing of an example of a three-dimensional integral image. It is explanatory drawing of an example of the three-dimensional integral image of a block unit in this Embodiment. It is a hardware block diagram of the other example of an image processing apparatus.

  Hereinafter, embodiments of the present invention will be described in detail. Note that the image processing apparatus in the present embodiment may be a device, apparatus, circuit, or the like that creates an integral image. The image processing apparatus according to the present embodiment is, for example, a computer, an imaging apparatus such as a multifunction peripheral or a copier, a digital camera, or a circuit that performs image processing. The image processing apparatus according to the present embodiment treats a plurality of pixels as one block, and reduces the storage capacity necessary for storing the integral image compared with the conventional method in the integral image creation method in which integration is performed in units of blocks. To reduce.

(Area integration)
Area integration (integrated image) is created by cumulatively calculating the sum of luminance values of each pixel block. FIG. 1 is an explanatory diagram of an example of an integral image.

  FIG. 1A shows an example of an input image. The input image in FIG. 1A is assumed to be VGA (vertical 480 pixels, horizontal 640 pixels) and 8 bits in luminance. The luminance value of the coordinates (i, j) is Y (i, j). The area integral value of the coordinates (i, j) is A (i, j). Note that i is a natural number of 640 or less. j is a natural number of 480 or less. The area integral value A (i, j) is defined as the following formula (1).

A (i, j) = ∫∫Y (i, j) = ΣΣY (i, j) (1)
FIG. 1C shows an example of the integral image. The integration direction of the integrated image in FIG. 1C is from upper left to lower right. That is, A (1,1) is equal to Y (1,1). A (640, 480) is equal to the sum of the luminance values Y (i, j) of all coordinates (i, j).

  FIG. 1B shows an example of an input image stored in the RAM. FIG. 1D shows an example of the integral image stored in the RAM. For example, after storing an input image as shown in FIG. 1 (b), in order to calculate as shown in FIG. 1 (d), only one RAM having the storage capacity shown in FIG. 1 (d) is required. In other words, the securing of the input image and the calculation of the integral image can be combined in the physically same RAM.

  The maximum value of the luminance value Y is 255 (8 bits). Therefore, the maximum value of the area integration value A is 78336000 (= 255 × 640 × 480). In order to store the area integration value A in the RAM, 27 bits (= 132651000) are required. Accordingly, the storage capacity of the RAM necessary for storing the area integration value A is 1012.5 KB (= 640 × 480 × 27 bits). In this embodiment, calculation is performed with 8 bits = 1 byte, 1024 bytes = 1 KB (kilobytes) unless otherwise specified.

  FIG. 2 is an explanatory diagram of an example of calculation of the total luminance using the integral image. FIG. 2A shows that the sum S of luminance values Y surrounded by an arbitrary rectangle in the integral image can be calculated from the area integral value A of the four coordinates of the arbitrary rectangle. In FIG. 2A, four coordinates of an arbitrary rectangle in the integrated image are represented by P0, P1, M0, and M1.

  FIG. 2B visually represents the sum of luminance values Y indicated by the area integral value A of four coordinates P0, P1, M0 and M1 of an arbitrary rectangle in the integral image. The sum S of luminance values Y surrounded by an arbitrary rectangle in the integral image shown in FIG. 2A can be calculated by the following equation (2).

S = A (P0) + A (P1) -A (M0) -A (M1) (2)
The number of pixels surrounded by an arbitrary rectangle can be obtained from the coordinates of the four coordinates P0, P1, M0, and M1. Therefore, the average value of the luminance of the pixels surrounded by the arbitrary rectangle is obtained by dividing the sum S of the luminance values Y surrounded by the arbitrary rectangle in the integral image by the number of pixels surrounded by the arbitrary rectangle. be able to.

  The image processing apparatus according to the present embodiment loses the characteristic that the sum S of luminance values Y surrounded by an arbitrary rectangle in the integral image can be obtained by accessing the RAM four times or less as described above. The RAM storage capacity necessary for storing the integral image is reduced as follows.

(Hardware configuration diagram)
The image processing apparatus 10 is realized by, for example, a PC (personal computer) as shown in FIG. FIG. 3 is a hardware configuration diagram of an example of a PC. The PC includes an input device 11, an output device 12, a recording medium reading device 13, an auxiliary storage device 14, a main storage device 15, an arithmetic processing device 16, and an interface device 17 that are connected to each other via a bus 19.

  The input device 11 is a keyboard or a mouse. The input device 11 is used for inputting various signals. The output device 12 is a display device or the like. The output device 12 is used for displaying various windows and data. The interface device 17 is a modem, a LAN card, or the like. The interface device 17 is used for connecting to a network such as a LAN or the Internet.

  A program for realizing the image processing apparatus 10 is provided by, for example, distribution of the recording medium 18 or downloading from a network or the like. The program that realizes the integral image creation method is at least a part of the program that realizes the image processing apparatus 10.

  The recording medium 18 is a recording medium that records information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, or a magneto-optical disk, or a semiconductor that electrically records information, such as a ROM or flash memory. Various types of recording media such as a memory can be used.

  When the recording medium 18 on which the program is recorded is set in the recording medium reader 13, the program is installed from the recording medium 18 to the auxiliary storage device 14 via the recording medium reader 13. A program downloaded from a network or the like is installed in the auxiliary storage device 14 via the interface device 17.

  The auxiliary storage device 14 stores programs, necessary files, data, and the like. The main storage device 15 reads the program from the auxiliary storage device 14 and stores it when the program is started. The arithmetic processing unit 16 implements various functions in accordance with programs stored in the main storage device 15.

  Further, the image processing apparatus 10 may be realized by hardware as shown in FIG. FIG. 17 is a hardware configuration diagram of another example of the image processing apparatus. FIG. 17 shows an example in which video data sent from an imaging device 21 such as a camera is stored in the RAM 23 and an area integral value is output to the face detection circuit 26. In the example of FIG. 17, the luminance storage circuit 22, the integration circuit 24, and the integration value calculation circuit 25 correspond to the image processing apparatus 10.

  The imaging device 21 transmits YUV422 or YUV420 video data (no color information is required) to the luminance storage circuit 22. When the video data is RGB, it is transmitted after being converted to YUV422 or the like. The luminance storage circuit 22 stores the luminance value Y in the RAM 23. At this time, all the luminance values Y in the block are stored at the same address.

  When the storage of the luminance value Y in the RAM 23 is completed, the luminance storage circuit 22 does not capture new video data until the detection result is output. The integration circuit 24 rewrites the luminance value Y stored in the RAM 23 into a combination of the luminance value Y, the horizontal integration value H, the vertical integration value V, and the area integration value A.

  The integral value calculation circuit 25 executes the processing of the calculation formula shown in FIG. The integral value calculation circuit 25 collectively reads the luminance value Y, horizontal integral value H, vertical integral value V, and area integral value A in the block including the coordinates for which the area integral value is requested from the face detection circuit 26, and is requested. The integrated area value A of the coordinates is output to the face detection circuit 26.

(architecture)
The image processing apparatus 10 divides an input image into 320 × 240 blocks by taking a square of 2 pixels vertically and 2 pixels horizontally as one block. The image processing apparatus 10 stores integral image data as described later in a RAM in units of blocks. FIG. 4 is an explanatory diagram of an example of an integral image in units of blocks in the present embodiment. FIG. 4A shows an example of integrated image data in units of blocks in the present embodiment.

  The integrated image data of the block unit includes area integrated value A (i−1, j−1), vertical integrated value V (i, j−1), horizontal integrated value H (i−1, j), And a luminance value Y (i, j). The horizontal integration value of the coordinates (i, j) is assumed to be H (i, j). The vertical integration value of the coordinates (i, j) is V (i, j). Note that i and j are multiples of 2.

  The vertical integration value V is a cumulative addition of the luminance value Y in the vertical direction. Specifically, the vertical integration value V (i, j) is a cumulative addition of luminance values Y (i, 1) to Y (i, j). The horizontal integration value H is a cumulative addition of the luminance value Y in the horizontal direction. Specifically, the horizontal integration value H (i, j) is a cumulative addition of luminance values Y (1, j) to Y (i, j).

  FIG. 4B is an example of integral image data stored at each address of the RAM in units of blocks. The integrated image data of each block stored at each address of the RAM contains only one original area integral value A.

  However, the image processing apparatus 10 can read out the four area integration values A by reading out the data of the integral image in units of blocks from one address of the RAM by the calculation formula shown by the circuit in FIG. FIG. 4C shows the following formulas (3) to (5).

A (i, j-1) = A (i-1, j-1) + V (i, j-1) (3)
A (i-1, j) = A (i-1, j-1) + H (i-1, j) (4)
A (i, j) = A (i-1, j-1) + V (i, j-1)
+ H (i-1, j) + Y (i, j) (5)
FIG. 5 is an image diagram showing an example of a process for reading out four area integral values from block unit integral image data. FIG. 5A shows the range of pixels in which the area integral value A (i−1, j−1) included in the integrated image data in block units is cumulatively calculated as the sum of the luminance values Y by hatching. It is a thing. FIG. 5B shows a range of pixels in which the vertical integration value V (i, j−1) included in the block-unit integrated image data cumulatively calculates the sum of the luminance values Y in the vertical direction. This is indicated by diagonal lines. FIG. 5C shows a pixel range in which the horizontal integration value H (i−1, j) included in the block-based integrated image data cumulatively calculates the sum of the luminance values Y in the horizontal direction. This is indicated by diagonal lines.

  The expression (3) is obtained by adding the sum of the luminance values Y shown in the shaded portion of FIG. 5A and the sum of the luminance values Y shown in the shaded portion of FIG. (I, j-1) is obtained. Equation (4) is obtained by adding the sum of the luminance values Y shown in the shaded area in FIG. 5A and the sum of the luminance values Y shown in the shaded area in FIG. (I-1, j) is obtained.

  Equation (5) is the sum of the brightness values Y shown in the shaded area in FIG. 5A, the sum of the brightness values Y shown in the shaded area in FIG. 5B, and the hatched line in FIG. 5C. The area integral value A (i, j) is obtained by adding the sum of the luminance values Y shown in the part and the luminance value Y (i, j) included in the integrated image data in block units. It is.

  As described above, the image processing apparatus 10 can read the four integrated area values A at a time for each block. The image processing apparatus 10 can obtain the area integral value A at any coordinate by one access to the RAM.

  The storage capacity of the RAM necessary for storing the integral image data in block units can be estimated as follows. The maximum value of the horizontal integration value H is 163200 (= 255 × 640). In order to store the horizontal integration value H in the RAM, 18 bits are required. The maximum value of the vertical integration value V is 122400 (= 255 × 480). In order to store the vertical integration value V in the RAM, 17 bits are required. Therefore, 70 bits (the luminance value Y is 8 bits, the horizontal integration value H is 18 bits, the vertical integration value V is 17 bits, and the area integration value A is 27 bits in order to store the integrated image data of one block in the RAM. )is necessary.

  Therefore, the storage capacity of the RAM necessary for storing the integral image data in units of blocks is 656.25 KB (= 320 × 240 × 70 bits). As described above, the image processing apparatus 10 stores the integral image data in units of blocks, rather than the storage capacity 1012.5 KB of the RAM necessary for storing the area integral value A shown in FIG. The storage capacity of the RAM required for this can be reduced.

  In the above description, a quadrangle of two vertical pixels and two horizontal pixels is described as one block. However, the number of pixels included in one block can be increased by an integral multiple. The image processing apparatus 10 can also divide the input image into 160 × 120 (= 19200) blocks, with a quadrangular pixel of 4 pixels vertically and 4 pixels horizontally as one block.

  FIG. 6 is an explanatory diagram of another example of the integral image in units of blocks in the present embodiment. The image processing apparatus 10 stores integral image data as shown in FIG. 6A in a block unit in a RAM. FIG. 6A shows an example of integrated data in units of blocks and integrated image data stored in each address of the RAM in units of blocks in the present embodiment.

  The integrated image data in units of blocks shown in FIG. 6A includes one area integral value A, three horizontal integral values V, three vertical integral values H, and nine luminance values Y, and one address of the RAM. Stored in The integrated image data of each block stored at each address of the RAM contains only one original area integral value A. However, the image processing apparatus 10 can read out the 16 area integral values A by reading out the data of the integral image in units of blocks from one address of the RAM using the calculation formula shown in FIG.

  The storage capacity of the RAM necessary for storing the integral image data in block units can be estimated as follows. Similarly to the description in which the square of 2 pixels in the vertical direction and the square of 2 pixels in the horizontal direction are made into one block, even when the square of 4 pixels in the vertical direction and 4 pixels in the horizontal direction are made into one block, in order to store the horizontal integration value H in the RAM, 18 A bit is needed. In order to store the vertical integration value V in the RAM, 17 bits are required. In order to store the luminance value Y in the RAM, 8 bits are required. In order to store the area integration value A in the RAM, 27 bits are required.

  Therefore, in order to store the block-unit integrated image data including one area integral value A, three horizontal integral values V, three vertical integral values H, and nine luminance values Y in the RAM, the area integral value A Requires 27 bits, three horizontal integration values V are 18 × 3 bits, three vertical integration values H are 17 × 3 bits, and nine luminance values Y are 8 × 9 bits, 204 bits. In addition, the storage capacity of the RAM required to store the integral image data in units of blocks is 478.125 KB (= 160 × 120 × 204 bits).

  As described above, the image processing apparatus 10 stores the integral image data in units of blocks, rather than the storage capacity 1012.5 KB of the RAM necessary for storing the area integral value A shown in FIG. The storage capacity of the RAM required for this can be reduced.

  Note that as the number of pixels included in one block increases, the storage capacity of the RAM required to store the integral image data in block units can be reduced. However, the block units as shown in FIG. A control circuit that realizes a calculation formula for reading a plurality of area integral values A from the integral image data becomes larger.

  Also, depending on the FAB (foundry), there may be no huge bit width RAM. In this case, there is a problem that the number of RAM increases and a delay occurs. When trying to reduce the entire circuit scale to the limit, the extent to which the block size is increased differs depending on the FAB, but even if the block size is increased beyond a certain level, there is almost no effect on the reduction in the storage capacity of the RAM. Therefore, in the following description, a quadrangle of 4 pixels in the vertical direction and 4 pixels in the horizontal direction are handled as one block.

  As shown in FIG. 7, the image processing apparatus 10 may divide the input image into four and perform area integration in the four corner directions from the center of the input image. FIG. 7 is an explanatory diagram of an example of a block-wise four-way integral image in the present embodiment. In addition, although it divides | segments equally in FIG. 7, it does not need to be strictly equal division | segmentation. In FIG. 7, the four divided areas are named from the first area to the fourth area, respectively.

  FIG. 8 is an image diagram of an example of a process of reading an area integral value from data of a four-way integral image. An arbitrary rectangle in the four-way integral image is a) in one region, b) in two regions, c) if it touches one boundary, d) includes all regions, e) if it touches two boundaries . In FIG. 8, the sum of the luminances in the respective rectangles in the cases a) to e) is Sa, Sb, Sc, Sd, and Se.

  In the cases of a), b), and d) that are not in contact with the boundary, the total luminances Sa, Sb, and Sd in the respective rectangles can be calculated if the four-point area integral values A are known. For example, the total luminance Sa in the rectangle in the case of a) can be calculated by P0a + P1a−M0a−M1a, where the area integral values A at the four coordinates of the rectangle are M0a, M1a, P0a, and P1a, respectively.

  In the case of c) that is in contact with one boundary, the total luminance Sc in the rectangle can be calculated if the area integral values A at two points are known. For example, the total luminance Sc in the rectangle in the case of c) can be calculated by P0c−M0c, where the area integral values A at the two coordinates of the rectangle are M0c and P0c, respectively.

  In the case of e) in contact with the two boundaries, the total luminance Se in the rectangle can be obtained if the area integral value A at one point is known. For example, in the case of e), the total luminance Se in the rectangle is P0e where the area integral value A at one coordinate of the rectangle is P0e.

  Therefore, the characteristic that the sum S of luminances in an arbitrary rectangle in the four-way integral image can be obtained by accessing the RAM four times or less as described above is not lost.

  FIG. 9 is an explanatory diagram of an example of an integral image in block units from the first region to the fourth region. The image processing apparatus 10 stores the integrated image data of the first area to the fourth area as shown in FIGS. 9A-1 to 9A-4 in the block unit in the RAM. FIG. 9B shows another example of integrated data in units of blocks and integrated image data stored in each address of the RAM in units of blocks in the present embodiment.

  As shown in FIGS. 9 (a-1) to 9 (a-4), the area integration of the coordinates in the block differs depending on each area from the first area to the fourth area, but one area This is the same in that it includes an integral value A, three horizontal integral values V, three vertical integral values H, and nine luminance values Y.

  However, since it is divided into four, the number of luminance values Y added at the corners of each region from the first region to the fourth region (the point where the area integral value A is maximum) is reduced, and the maximum value of the area integral value A is stored in the RAM. The number of bits required for storage is reduced by 2 bits to 25 bits. Further, the bit required to store the horizontal integration value H in the RAM is reduced by 1 bit to 17 bits. The number of bits required to store the vertical integration value V in the RAM is 1 bit, which is 16 bits.

  Therefore, in order to store the data of the integrated image of one block in the RAM, the area integrated value A is 25 bits, the horizontal integrated value V is 17 × 3 bits, the vertical integrated value H is 16 × 3 bits, and nine luminance values. Y needs 196 bits of 8 × 9 bits. In addition, the storage capacity of the RAM necessary for storing the integral image data in units of blocks is 459.375 KB (= 160 × 120 × 196 bits). Depending on the application, the input image may be divided into, for example, left and right to simplify the control. When the input image is divided into two, the integration direction has the shape of the third region and the fourth region without the first region and the second region. In order to store one block of integral image data in the RAM, the area integral value A is 26 bits, the horizontal integral value V is 17 × 3 bits, the vertical integral value H is 17 × 3 bits, and nine luminance values Y are 200 bits of 8 × 9 bits are required.

  The image processing apparatus 10 can also partially perform area integration in accordance with a face detection algorithm. Here, an example in which an input image is divided into two in the left and right directions will be described. Scanning at the time of face detection can be performed as described in JP 2010-28370 A, for example.

  The scanning at the time of face detection is briefly described as follows. FIG. 10 is an explanatory diagram showing an example of how to scan during face detection. FIG. 10A shows an example of scanning windows having different sizes. Scan windows are prepared according to the size of the face. FIG. 10B shows how scanning is performed in each scanning window. As shown in FIG. 10B, all the scanning windows are moved from the upper left to the lower right so that the entire input image can be scanned.

  In the face detection algorithm, the luminance value Y in the scanning window is partially read and matched with a filter prepared in advance to determine whether it is a face or a non-face. In the face detection algorithm, it is not necessary to obtain all the area integral values in one screen at a time. In the face detection algorithm, the area integral value can be partially updated as in the flowchart shown in FIG.

  FIG. 11 is a flowchart of an example of a face detection algorithm. In step S1, the image processing apparatus 10 stores the luminance value Y in the RAM for each block. In step S <b> 2, the image processing apparatus 10 performs area integration for 120 lines (an integer multiple of the number of lines of the block) that is the same number of lines as the largest scanning window.

  In step S3, the image processing apparatus 10 scans 120 lines in all the scanning windows. In step S4, the image processing apparatus 10 performs area integration of 4 lines, which is the number of lines of the next block. In step S5, the image processing apparatus 10 scans all the scanning windows until the end of the four lines.

  In step S6, the image processing apparatus 10 determines whether scanning of the entire input image has been completed. If the entire input image has not been scanned, the image processing apparatus 10 returns to step S4 and repeats the processes of steps S4 to S6. If the entire input image has been scanned, the image processing apparatus 10 ends the processing of the flowchart of FIG.

  In the flowchart of FIG. 11, when the size of the scanning window is 120 × 120 pixels, there is a condition that the moving width of one scanning window is 4 lines or more, but generally the movement is about 10% of the size of the scanning window. There is no problem because of the width.

  FIG. 12 is an explanatory diagram showing an example of the movement of the RAM during face detection. As shown in FIG. 12, the image processing apparatus 10 uses two RAMs to store the luminance value Y, 120-line area integration, and 4-line area integration update.

  FIG. 12A shows how the luminance value Y is stored using two RAMs. The first RAM “RAM-A” stores an 8-bit luminance value Y in units of blocks (vertical 4 pixels and horizontal 4 pixels). The RAM-A stores a luminance value Y of a block of 160 × 120 (= 19200).

  FIG. 12B shows a state where the area integration of 120 lines is performed using two RAMs. When the input image is divided into two, 200 bits are required to store the data of the integrated image of one block in the RAM as described above. Therefore, the image processing apparatus 10 stores the area integration value A in units of blocks in 160 × 30 × 200 bits in which the first RAM “RAM-A” and the second RAM “RAM-B” are shared. . Note that the luminance value Y of the 160 × 90 (= 14400) block remains stored in the RAM-A.

  FIG. 12C shows a state in which the area integration of 4 lines is updated using two RAMs. The image processing apparatus 10 performs area integration of four lines in the next block. The area integrated value A for four lines to be updated is overwritten in the storage area of the RAM-A in which the luminance value Y of the block is stored, and the lines 121 to 124 from the address 0 to the address 3 of the RAM-B. Is overwritten with the area integral value A.

  In FIG. 10, the reason for not dividing the image into four directions is that when the vertical integration direction is different between the first and second regions and the third and fourth regions, the input image starts from the center upward and again This is because it has to be repeated from the center. The capacity of the RAM is 342.1875 KB (= 160 × 120 × 128 bits + 160 × 30 × 72 bits), which is about one third of the 1012.5 KB required for the conventional integral image creation method. An integrated image can be created with a storage capacity.

  Here is a supplementary explanation. The number of lines for which area integration is performed first is called the initial number of lines. The number of lines for updating and integrating the area is called the number of updated lines. The number of update lines may be increased as long as the initial number of lines is sufficiently large (the required RAM storage capacity increases). For example, when the initial number of lines is 224 lines and the number of update lines is 64 lines, any moving width can be accommodated if the size of the scanning window is up to 160 × 160 pixels.

  As shown in the flowchart of FIG. 11, when the area integration value is partially updated, the delay is slightly delayed as the number of updates is larger as compared to the case where the area integration is performed all at once. . FIG. 13 is an explanatory diagram of an example of a four-way integral image when the initial number of lines is 224 and the update number of lines is 64. In FIG. 13, the integration direction is divided into four directions by setting the first region and the second region to the upper 224 lines and the third region and the fourth region to the lower 256 lines.

  If the number of lines in the first region and the second region is k, k ≦ 2 ^ 8 and 480−k ≦ 2 ^ in order to prevent the number of bits for storing the area integral value from increasing in any region. It can be seen that 8 is sufficient. X ^ y represents x to the power of y. That is, if 224 ≦ k ≦ 256 and the initial number of lines is k or more, the integration direction may be divided into four directions.

  FIG. 14 is an explanatory diagram of an example of processing for scanning 224 lines in the scanning window. For example, FIG. 14A shows a state in which area integration of 224 lines is performed, and scanning is performed from the upper left to the lower right in a scanning window having a size of 160 × 160 pixels. FIG. 14B shows that since the size of the scanning window is 160 × 160 pixels, the upper 64 lines are unnecessary areas that are not included in the scanning window.

  FIG. 14C shows that the 64 lines of the unnecessary area are discarded and the area of the next 64 lines is integrated to make an update area. FIG. 14D shows a state in which the unnecessary area is discarded and scanning is performed from the upper left to the lower right in the scanning window of 160 × 160 pixels in the new 224 lines in which the update area is added. Note that scanning is performed with a scanning window having a position and size that partially includes the lower 64 lines that are update regions.

  In some cases, such as when a face detected by a face detection algorithm is registered for face authentication, the face in the coordinates may be desired to be extracted as luminance after the face detection. In the present embodiment, the luminance value of all coordinates can be obtained from the area integral value of all coordinates as described above. This is because the storage capacity of the RAM is reduced without using lossy compression in this embodiment. Application of the existing lossy compression technique to this embodiment is possible as an application.

  In medical image diagnosis, for example, a solid (human body) may be cut into small pieces and scanned finely. In this case, since the feature amount appears also in the vertical direction, volume integration is performed. In motion recognition, continuous images may be integrated in the time axis direction. In this case, time integration is performed.

  FIG. 15 is an explanatory diagram of an example of a three-dimensional integral image. In the case of the three-dimensional integration, the total luminance I in the rectangular parallelepiped is obtained from the eight-point three-dimensional integration values P0 to P3 and M0 to M3 using the following formula (6) as shown in FIG. . FIG. 15A shows the integration direction.

I = P0 + P1 + P2 + P3-M0-M1-M2-M3 (6)
The three-dimensional integrated image can be divided into block units shown in FIG. 16, for example. FIG. 16 is an explanatory diagram of an example of a three-dimensional integral image in block units according to the present embodiment. As shown in FIG. 16B, the data of the block-unit three-dimensional integral image includes a three-dimensional integral value, three two-dimensional integral values, three one-dimensional integral values, and a luminance value y. It is. Note that when the volume integration is further time integrated, four-dimensional integration is performed as shown in FIG.

  Here, it is assumed that r and n are integers, and r is 0 or more and n or less. The zero-dimensional integral value is the luminance value y. In this embodiment, 2 ^ n pixels of an n-dimensional integrated image are handled as one block, the luminance value y is stored, integrated for each axis, and finally expressed by nCr r-dimensional integrated values. As a result, the required RAM storage capacity is reduced. The number of pixels in one block may be increased.

  The image processing apparatus 10 according to the present embodiment can be used for a face detection circuit and a face recognition circuit. Further, the image processing apparatus 10 of the present embodiment can be applied to a circuit that performs image pattern matching such as fingerprint authentication or a circuit that extracts features from an image such as three-dimensional medical image diagnosis and motion authentication. .

  The present invention is not limited to the specifically disclosed embodiments, and various modifications and changes can be made without departing from the scope of the claims. For example, the present invention is not limited to luminance values, but includes RGB values and squares of luminance values.

  The storage means described in the claims corresponds to the auxiliary storage device 14 or the main storage device 15. The control means corresponds to, for example, the arithmetic processing unit 16 that performs the processing of the calculation formula shown in FIG.

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 11 Input device 12 Output device 13 Recording medium reader 14 Auxiliary storage device 15 Main storage device 16 Arithmetic processing device 17 Interface device 18 Recording medium 19 Bus

JP 2011-19013 A

Claims (11)

In the block unit into which the image is divided, the area integral value of one coordinate included in the block, the vertical integral value obtained by cumulatively adding the luminance value in the vertical direction of one or more coordinates, and the luminance value in the horizontal direction. Storage means for storing the horizontal integrated value and luminance value cumulatively added together ;
Control means for calculating the area integral value of all coordinates included in the block by the area integral value of one coordinate read from the storage means, the vertical integral value, the horizontal integral value and the luminance value of one or more coordinates An image processing apparatus comprising: Storage means for storing an area integral value of one coordinate included in the block, a vertical integral value of one or more coordinates, a horizontal integral value, and a luminance value in units of blocks into which the image is divided;
Control means for calculating the area integral value of all coordinates included in the block by the area integral value of one coordinate read from the storage means, the vertical integral value, the horizontal integral value and the luminance value of one or more coordinates When
Have
The storage means includes a luminance value (i, j) of coordinates (i, j) included in the block,
A vertical integral value (i, j-1) of coordinates (i, j-1),
Horizontal integration value (i-1, j) of coordinates (i-1, j);
Storing the area integral value (i-1, j-1) of the coordinates (i-1, j-1);
The control means converts the area integral value (i, j-1) of the coordinates (i, j-1) included in the block into an area integral value (i-1, j-1) + vertical integral value (i, j -1),
An area integral value (i-1, j) of coordinates (i-1, j) is calculated by area integral value (i-1, j-1) + horizontal integral value (i-1, j);
The area integral value (i, j) of the coordinates (i, j) is expressed as area integral value (i-1, j-1) + vertical integral value (i, j-1) + horizontal integral value (i-1, j). ) + luminance value (i, j) images processor you and calculates by. Storage means for storing an area integral value of one coordinate included in the block, a vertical integral value of one or more coordinates, a horizontal integral value, and a luminance value in units of blocks into which the image is divided;
Control means for calculating the area integral value of all coordinates included in the block by the area integral value of one coordinate read from the storage means, the vertical integral value, the horizontal integral value and the luminance value of one or more coordinates When
Have
The storage means has a size of nxn when the block is
(N−1) × (n−1) luminance values;
A vertical integral value of (n−1),
A horizontal integral value of (n−1),
One image picture processor characterized in that it stores the area integration value. The image processing apparatus according to claim 1 to 3 any one of claims, characterized in that with the integration direction of the area integration in the direction of the n-th power of up to 2 for three-dimensional or higher-dimensional images. Said storage means, for three-dimensional or higher-dimensional images, 0-dimensional integrated value from an n-dimensional integral values claims 1 to 4 any one, characterized in that to represent the value of up (luminance values) The image processing apparatus described. An integrated image creation method executed by a computer,
In the unit of the block into which the image is divided, the area integral value of one coordinate included in the block, the vertical integral value obtained by cumulatively adding the luminance value in the vertical direction of one or more coordinates, and the luminance value in the horizontal direction The block is obtained from the area integration value of one coordinate, the vertical integration value, the horizontal integration value, and the luminance value of one or more coordinates read out from the storage means storing the accumulated horizontal integration value and luminance value. An integral image creation method characterized by calculating an area integral value of all coordinates included in. An integrated image creation method executed by a computer,
In the block unit into which the image is divided, an area integration value of one coordinate included in the block, a vertical integration value of one or more coordinates, a horizontal integration value, and a luminance value are read out from a storage unit storing one piece. The area integral value of all coordinates included in the block is calculated from the area integral value of the coordinates, the vertical integral value of one or more coordinates, the horizontal integral value, and the luminance value.
It is characterized by
The storage means is included in the block
The luminance value (i, j) of the coordinates (i, j);
A vertical integral value (i, j-1) of coordinates (i, j-1),
Horizontal integration value (i-1, j) of coordinates (i-1, j);
The area integral value (i-1, j-1) of the coordinates (i-1, j-1) and
Store
The area integral value of all coordinates included in the block is calculated as follows:
Included in the block
An area integral value (i, j-1) of coordinates (i, j-1) is calculated by area integral value (i-1, j-1) + vertical integral value (i, j-1);
An area integral value (i-1, j) of coordinates (i-1, j) is calculated by area integral value (i-1, j-1) + horizontal integral value (i-1, j);
The area integral value (i, j) of the coordinates (i, j) is expressed as area integral value (i-1, j-1) + vertical integral value (i, j-1) + horizontal integral value (i-1, j). ) + Integral image creation method calculated by luminance value (i, j) . An integrated image creation method executed by a computer,
  In the block unit into which the image is divided, an area integration value of one coordinate included in the block, a vertical integration value of one or more coordinates, a horizontal integration value, and a luminance value are read out from a storage unit storing one piece. The area integral value of all coordinates included in the block is calculated from the area integral value of the coordinates, the vertical integral value of one or more coordinates, the horizontal integral value, and the luminance value.
It is characterized by
  The storage means has a size of nxn when the block is
  (N−1) × (n−1) luminance values;
  A vertical integral value of (n−1),
  A horizontal integral value of (n−1),
  One area integral value and
To store
An integral image creation method characterized by the above. Computer
In the unit of the block into which the image is divided, the area integral value of one coordinate included in the block, the vertical integral value obtained by cumulatively adding the luminance value in the vertical direction of one or more coordinates, and the luminance value in the horizontal direction Storage means for storing the horizontal integrated value and luminance value cumulatively added together ;
Control means for calculating the area integral value of all coordinates included in the block by the area integral value of one coordinate read from the storage means, the vertical integral value, the horizontal integral value and the luminance value of one or more coordinates Program to make it function as. Computer
Storage means for storing an area integral value of one coordinate included in the block, a vertical integral value of one or more coordinates, a horizontal integral value, and a luminance value in units of blocks into which the image is divided;
Control means for calculating the area integral value of all coordinates included in the block by the area integral value of one coordinate read from the storage means, the vertical integral value, the horizontal integral value and the luminance value of one or more coordinates When
Function as
The storage means is included in the block
The luminance value (i, j) of the coordinates (i, j);
A vertical integral value (i, j-1) of coordinates (i, j-1),
Horizontal integration value (i-1, j) of coordinates (i-1, j);
The area integral value (i-1, j-1) of the coordinates (i-1, j-1) and
Store
The control means is included in the block
An area integral value (i, j-1) of coordinates (i, j-1) is calculated by area integral value (i-1, j-1) + vertical integral value (i, j-1);
An area integral value (i-1, j) of coordinates (i-1, j) is calculated by area integral value (i-1, j-1) + horizontal integral value (i-1, j);
The area integral value (i, j) of the coordinates (i, j) is expressed as area integral value (i-1, j-1) + vertical integral value (i, j-1) + horizontal integral value (i-1, j). ) + Luminance value (i, j)
A program characterized by that .   Computer
  Storage means for storing an area integral value of one coordinate included in the block, a vertical integral value of one or more coordinates, a horizontal integral value, and a luminance value in units of blocks into which the image is divided;
  Control means for calculating the area integral value of all coordinates included in the block by the area integral value of one coordinate read from the storage means, the vertical integral value, the horizontal integral value and the luminance value of one or more coordinates When
Function as
  The storage means has a size of nxn when the block is
  (N−1) × (n−1) luminance values;
  A vertical integral value of (n−1),
  A horizontal integral value of (n−1),
  One area integral value and
Is stored
A program characterized by that.

Images