JP2011186678A - Image processing apparatus and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus that implements a luminance expansion function clearly displaying both bright and dark local images present in an entire screen. <P>SOLUTION: A luminance expansion part 1 is equipped with: a block processing means for dividing an area of interest set in an input image subjected to luminance expansion into a plurality of blocks, calculating a block-specific luminance expansion parameter of each divided block from luminance values of a plurality of blocks convoluting luminance values of surrounding blocks, and creating a block-specific luminance conversion table depending on the luminance expansion parameter; an interpolation means for applying bilinear interpolation to each pixel of each divided block according to the block-specific luminance conversion table created by the block processing means by reference to the luminance conversion tables of blocks surrounding the block; and an image conversion means for creating a luminance-expanded output image relative to the input image from the pixels of each block subjected to the bilinear interpolation by the interpolation means. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus having a luminance expansion processing function for handling an image captured by a camera as an input image, and an image processing program characterized by a luminance expansion algorithm for camera video.

  In the tracking processing technology for moving objects using monocular camera images, a tracking area is used to detect moving objects (object images) from the area of interest using a partial area of the entire screen of the input image captured and input by the camera as the area of interest. There is an image recognition processing technology to perform (see Patent Document 1).

  In addition, there are various brightness conversion methods for converting the brightness of an input image photographed and input by a camera according to a predetermined brightness conversion characteristic (characteristic curve) set in advance according to the brightness of the input image, using various nonlinear curves, sigmoid function curves, etc. A luminance conversion method has been proposed and implemented (see Patent Documents 2 and 3).

  In the image processing technique described above, the luminance expansion parameter is extracted and determined from the attention area of the entire screen. For this reason, if the image to be stretched input from the camera is an image including both light and dark, the brightness stretching process is performed depending on the brightness of the entire area of interest, and therefore both the brightness and darkness of the partial image in the area of interest are sharp. There was a problem that could not be displayed.

  Various techniques have also been proposed for making an image into blocks and collecting luminance information (such as a histogram) in units of the blocks (see Patent Document 4).

  However, this type of method for collecting pixel information in units of blocks and determining parameters uses only luminance values included in the blocks to determine parameters in units of blocks. When it is applied, there is a problem that shaggy occurs at the boundary between blocks, and that when blocks having extremely different brightness are adjacent to each other, an unnatural image is formed.

Japanese Patent No. 4427052 Japanese Patent No. 4340303 JP-A-8-161485 JP-A-8-62741

  As described above, in the related art, when the image to be stretched input from the camera includes both light and dark in a partial area of the entire screen, it is not possible to clearly display both the light and dark of the partial image in the area. was there.

  The present invention has been made in view of the above circumstances, and an image processing apparatus capable of realizing a luminance expansion processing function capable of clearly displaying and outputting each bright and dark image portion even when a bright and dark image exists in a part of the entire screen. An object of the present invention is to provide an image processing program.

  The present invention divides a set attention area into a plurality of blocks for an input image to be subjected to luminance expansion processing, and the luminance values of a plurality of blocks obtained by convolving the luminance values of surrounding blocks for each of the divided blocks A block processing unit for calculating a luminance expansion parameter for each block based on the block and creating a luminance conversion table for each block based on the luminance expansion parameter, and a luminance conversion table for each block generated by the block processing unit Based on the above, for each pixel of each of the divided blocks, for each block, the interpolation processing means for performing bilinear interpolation with reference to the luminance conversion table of the surrounding block, and the bilinear interpolation performed by the interpolation processing means Image conversion means for creating a luminance expanded output image for the input image based on the pixels of each block; Wherein the image processing apparatus provided with the.

  In the image processing apparatus, the block processing means uses, for the target block A to be processed among the divided blocks, the adjacent upper, lower, left, and right blocks B and the adjacent diagonal blocks C, and The convolution processing means by weighting> B> C, and the block processing means further divides the image of the area of interest into a plurality of blocks according to a block size designated from the outside; For each block divided by the block dividing means, the luminance sum for each block is calculated, and using the calculated luminance sum for each block, the weight and convolution processing including the block and the surrounding blocks are performed for each block. Luminance total calculation means for calculating the luminance sum of the blocks, and the luminance total calculation means For each block, the luminance average calculating means for calculating the luminance average of the block using the luminance sum total, and the luminance expansion for each block based on the luminance average of the block calculated by the luminance average calculating means. A luminance expansion parameter calculating means for calculating a parameter, a luminance conversion table generating means for generating a luminance conversion table for luminance expansion for each block based on the luminance expansion parameter calculated by the luminance expansion parameter calculating means, It is characterized by comprising.

  In the present invention, an input image captured by a camera is held in an image buffer, and a processing device that performs gamma correction on the input image on the image buffer is set with respect to an input image to be subjected to luminance expansion processing. The area is divided into a plurality of blocks, and for each of the divided blocks, a luminance expansion parameter for each block is calculated based on the luminance values of the plurality of blocks obtained by convolving the luminance values of the surrounding blocks, and the luminance expansion parameter is calculated. Based on the function of creating a brightness conversion table for each block and the created brightness conversion table for each block, the brightness conversion table of the surrounding blocks for each pixel of each divided block. Based on the function of performing bilinear interpolation with reference to the pixel of each block subjected to the bilinear interpolation Wherein the image processing program for realizing a function of creating a luminance decompression output image, the relative fill power image.

  According to the present invention, it is possible to provide an image processing apparatus capable of realizing a luminance expansion processing function capable of clearly displaying and outputting each light and dark image even when a light and dark image exists in a part of the entire screen.

1 is a block diagram showing components of a main part of an image processing apparatus according to an embodiment of the present invention. Operation | movement explanatory drawing for demonstrating the brightness | luminance expansion process operation | movement of the brightness | luminance expansion | extension process part which concerns on the said embodiment. Operation | movement explanatory drawing for demonstrating the blocking luminance expansion | extension process operation | movement which concerns on the said embodiment. Operation | movement explanatory drawing for demonstrating the blocking luminance expansion | extension process operation | movement which concerns on the said embodiment. Operation | movement explanatory drawing for demonstrating the convolution process operation | movement which concerns on the said embodiment. Operation | movement explanatory drawing for demonstrating the convolution process operation | movement which concerns on the said embodiment. Operation | movement explanatory drawing for demonstrating the bilinear interpolation process operation | movement which concerns on the said embodiment. The block diagram which shows the structure of the image processing apparatus which concerns on the said embodiment. The block diagram which shows the structure of the brightness expansion process part which concerns on the said embodiment. The flowchart which shows the process sequence of the brightness expansion process part which concerns on the said embodiment. The flowchart which shows the more detailed process sequence of the brightness expansion process part which concerns on the said embodiment. The flowchart which shows the more detailed process sequence of the brightness expansion process part which concerns on the said embodiment. The flowchart which shows the more detailed process sequence of the brightness expansion process part which concerns on the said embodiment. The flowchart which shows the more detailed process sequence of the brightness expansion process part which concerns on the said embodiment. The flowchart which shows the more detailed process sequence of the brightness expansion process part which concerns on the said embodiment. The flowchart which shows the more detailed process sequence of the brightness expansion process part which concerns on the said embodiment. The flowchart which shows the more detailed process sequence of the brightness expansion process part which concerns on the said embodiment. The flowchart which shows the more detailed process sequence of the brightness expansion process part which concerns on the said embodiment.

  Embodiments of the present invention will be described below with reference to the drawings.

  The image processing apparatus according to the present invention provides a function of dividing an image into a plurality of blocks, determining a decompression parameter by combining parameters extracted from overlapping blocks, and automatically sharpening both light and dark Then, in addition to the luminance value included in the target block, the luminance value of the block in the vicinity of the surrounding 8 is also weighted to include a luminance expansion parameter, and a plurality of luminance expansion parameters calculated by the processing unit are used. Processing means b for obtaining a value after expansion of the target pixel (using parameters of the block including the target pixel and surrounding blocks).

  The processing means a obtains a luminance expansion parameter and a luminance conversion table for each image block, and the processing means b obtains a converted luminance value by bilinear interpolation using surrounding blocks.

  The luminance expansion parameter and the luminance conversion table described above depend on a luminance expansion processing method for converting the luminance of an input image captured and input by a camera according to a predetermined luminance conversion characteristic (characteristic curve) set in advance according to the brightness. For example, it is applied to luminance expansion parameters and luminance conversion tables applied to nonlinear luminance expansion processing methods using nonlinear curves (convex curves, concave curves), or luminance expansion processing using sigmoid function curves. The luminance expansion parameter and the luminance conversion table applied to any of the luminance expansion processing methods such as the luminance expansion parameter and the luminance conversion table.

The components of the main part of the image processing apparatus according to the embodiment of the present invention are shown in FIG.
The luminance expansion processing unit 1 shown in FIG. 1 uses an image captured by a monocular camera as an input image, performs luminance expansion processing (gamma correction) on the input image, and converts the input image from the luminance value distribution state of the input image. Convert to the optimum gradation. Here, the luminance expansion processing is performed on the image of the attention area set for the input image of 0 to 255 gradations of QVGA (320 pixels × 240 pixels).

  In the luminance expansion processing in the luminance expansion processing unit 1, the luminance expansion parameter for luminance expansion necessary for gamma correction is divided with reference to the theory of gamma (γ) correction, and the area of interest in the input image is divided into a plurality of blocks. There is provided means for automatically calculating based on the average luminance value of the block unit image.

  The luminance expansion processing unit 1 divides a set attention area into a plurality of blocks for an input image to be subjected to luminance expansion processing, and a plurality of blocks in which the luminance values of surrounding blocks are convoluted for each of the divided blocks A block processing unit that calculates a luminance expansion parameter for each block based on the luminance value of the block, creates a luminance conversion table for each block based on the luminance expansion parameter, and a block unit generated by the block processing unit. Based on the brightness conversion table, for each pixel of each divided block, for each block, an interpolation processing means for performing bilinear interpolation with reference to the brightness conversion table of the surrounding block, and bilinear interpolation by the interpolation processing means. An image that creates a luminance-extended output image for the input image based on the applied pixels of each block Constituted by and a switch means.

  The luminance expansion processing operation of the luminance expansion processing unit 1 shown in FIG. 1 will be described with reference to FIGS. 2 is an operation explanatory diagram for explaining the luminance expansion processing operation of the luminance expansion processing unit 1, and FIGS. 3 and 4 are operation explanatory diagrams for explaining the operation principle of the luminance expansion processing in the block processing means described above. FIGS. 5 and 6 are operation explanatory diagrams for explaining the operation principle of the convolution processing in the block processing means, and FIG. 7 is an operation description for explaining the operation principle of the bilinear interpolation processing in the interpolation processing means. FIG.

  The block processing means 4a of the luminance expansion processing unit 1 shown in FIG. 2 first divides a preset attention area 3 into a plurality of blocks for an input image (320 pixels × 240 pixels) 2 captured by a monocular camera. . As a specific example, the input image is divided into 32 × 32 pixel units, 16 × 16 pixel units, 8 × 8 pixel units, or the like, and the attention area 3 is divided into a plurality of blocks. For example, the screen of the attention area 3 is divided into 16 × 16 pixel unit blocks (20 × 15 blocks). For each block division area, the average luminance is obtained using the luminance value of the surrounding block as a variable element.

  This operation principle is shown in FIGS. Here, as shown in FIG. 3 and FIG. 4, the 16 × 16 pixel pattern obtained by dividing the screen into blocks is changed to a value of one small block from the block 4 pattern shifted by 1/2 block (8 × 8 pixels). Think. Here, for a block (8 × 8 pixels) in which four patterns of blocks are overlapped by a 1/2 block shift, the average of the luminance of the overlapping blocks, the average of the square of the luminance, etc. are obtained, and based on this value Then, a luminance conversion table for luminance expansion, which is applied to a preset luminance expansion processing method, is created for each block. Further, the block processing means 4a calculates a luminance sum for each block for each of the divided blocks, and uses the calculated luminance sum for each block to weight and convolve each block with the block and its surrounding blocks. The total luminance of all the processed blocks is calculated, and the average luminance of each block is calculated. The images of the convolution count and convolution processing at this time are shown in FIGS. Here, as shown in FIG. 5, “4” for the target block for which the average luminance is to be calculated, and “2” for the upper, lower, left, and right four blocks around the target block. With the convolution count of “1” for each block in four diagonal directions, the luminance value of each block is convolved with the data accumulation image shown in FIG. Can be requested. Based on the average luminance value for each block, a luminance expansion parameter and a luminance conversion table for each block to be applied to a preset luminance expansion processing method are created.

  Next, in the interpolation processing unit 4b of the luminance expansion processing unit 1, with respect to the pixels of each block using the luminance conversion table according to the luminance expansion parameter obtained in units of the blocks, the interpolation unit 4b Then, bilinear interpolation is performed using the luminance conversion table of the surrounding four blocks. In this bilinear interpolation, as shown in FIG. 7, the conversion value of the target pixel is determined and set by linear interpolation (bilinear interpolation) of the conversion values of the surrounding four blocks (A, B, C, D). Here, the block position where the pixel of interest is located and the positional difference between the pixel of interest and the corresponding block are calculated with a decimal point, and the value of the luminance conversion table of the block and surrounding blocks is multiplied by the coefficient of the block position difference to perform linear interpolation. The luminance conversion value is obtained by the principle of (bilinear interpolation).

  Based on the pixels of each block subjected to bilinear interpolation by the interpolation processing unit 4b, a luminance expansion output image 5 for the input image is created.

  As described above, the block processing unit 4a of the luminance expansion processing unit 1 obtains the luminance expansion parameter and the luminance conversion table for each image block, and the interpolation processing unit 4b of the luminance expansion processing unit 1 performs bilinear interpolation using surrounding blocks. By obtaining a converted luminance value and creating a luminance expansion output image 5 for the input image based on the pixels of each block subjected to bilinear interpolation (reflecting luminance expansion parameters of surrounding blocks), the block Brightness and darkness images that exist locally in a part of the attention area can be clearly displayed in gradation without causing a luminance difference at the boundary.

An example that further embodies the above embodiment is shown in FIGS.
FIG. 8 is a block diagram showing the configuration of the image processing apparatus according to the embodiment of the present invention, FIG. 9 is a block diagram showing the configuration of the luminance expansion processing unit provided in the front image processing unit according to the embodiment, and FIG. FIG. 11 to FIG. 18 are flowcharts showing the processing procedure of the luminance expansion processing unit in more detail. Here, the configuration in which the luminance expansion processing function unit as the gist of the present invention is provided in the front image processing unit of the image processing apparatus is illustrated, but the luminance expansion processing function unit is configured as shown in FIG. For example, the present invention can be applied to various image processing apparatuses such as an image processing apparatus that simply performs luminance expansion processing on camera video.

  As shown in FIG. 8, the image processing apparatus according to the embodiment of the present invention includes a camera 11, a capture unit 12, a front image processing unit 13, an image processing unit 14, and a display unit 15. Composed.

  The camera 11 includes a lens unit and an image pickup device (for example, a CCD solid-state image pickup device or a CMOS image sensor) provided at an image forming position of the lens unit, so that a subject (animal body) that moves outdoors or indoors can be used. A captured image for one screen is output to a target in a predetermined pixel unit (for example, one frame 320 × 240 pixels = QVGA).

  The capture unit 12 has a processing function for capturing an image in frame units captured by the camera 11 and holding it in the image buffer 21 in the front image processing unit 13.

  The front image processing unit 13 includes an image buffer 21 that holds an image in frame units captured by the capture unit 12 from the camera 11 as an input image to be subjected to luminance expansion processing, and the luminance expansion processing unit 1 illustrated in FIG. 1 described above. And a luminance expansion processing unit 22 having the same processing function. The internal components of the front image processing unit 13 will be described later with reference to FIG.

  The image processing unit 14 takes in the image after the luminance expansion processing held in the image buffer 21 of the front image processing unit 13, for example, a rectangular area including a difference binarization process of an image having a time difference, noise removal, and a change pixel A series of image processing such as extraction and tracking processing of the changed pixel region is performed. The display unit 15 displays and outputs an image of, for example, a moving object region processed by the image processing unit 14.

FIG. 9 shows components of the luminance expansion processing unit 22 provided in the front image processing unit 13 shown in FIG.
The luminance expansion processing unit 22 has the functions of the block processing means (4a) and the interpolation processing means (4b) described above, generates a luminance expansion parameter and a luminance conversion table for each image block by the block processing function, and performs interpolation. The converted luminance value of the pixel of each block is obtained by bilinear interpolation using surrounding blocks by the processing function.

  The configuration shown in FIG. 9 shows an example in which the block division luminance expansion processing function according to the embodiment of the present invention is applied to a nonlinear luminance expansion processing method using a nonlinear curve (convex curve, concave curve).

  In this nonlinear luminance expansion processing method, it is determined whether an average luminance value calculated by an average luminance calculation unit 222 described later is equal to or less than the central luminance value (127) of 256 gradations, and when it is equal to or less than the central luminance value A reference table with a convex curve is created as a luminance conversion table for luminance expansion for the input image. When the central luminance value is exceeded, a reference table with a concave curve is created as a luminance conversion table for luminance expansion for the input image. This nonlinear luminance expansion processing function is applied to the already-known nonlinear luminance expansion processing method according to Japanese Patent No. 4340303, and the gist thereof will be described later.

  In the front image processing unit 13 illustrated in FIG. 9, the luminance expansion processing unit 22 provided in the front image processing unit 13 includes a block reference luminance setting unit 221, an average luminance calculation unit 222, and a power parameter calculation unit 223. , A reference table determination unit 224, an image conversion unit 225, and a linear interpolation (bilinear interpolation) calculation unit 226.

  The block reference luminance setting unit 221 divides the attention area of the input image captured and input by the camera 11 into a plurality of blocks according to the block size designated from the outside, and sums the luminance value of each block for each block. calculate. Here, any block size (for example, 16 × 16 pixels) of 8 × 8 pixels, 16 × 16 pixels, 24 × 24 pixels, and 32 × 32 pixels is designated, and attention is paid to the input image according to the designated size. The area is divided into blocks. When an area corresponding to a fractional pixel smaller than the specified block size occurs in the edge portion of the input image, the area is excluded from the luminance expansion processing target or other luminance expansion processing means is applied.

  The average luminance calculation unit 222 divides the block into blocks by the block reference luminance setting unit 221 and, based on the luminance sum calculated for each block, all blocks subjected to weighting and convolution processing including the block and its surrounding blocks. Is calculated, and the average luminance value (= M) of the image in block units is calculated.

  The power parameter calculation unit 223 is applied in the above-described nonlinear luminance expansion processing method. Based on the average luminance calculated by the average luminance calculation unit 222, an input image (attention area) according to the equation (1) described later. A power parameter for the luminance expansion processing for is calculated.

  The reference table determination unit 224 determines whether the average luminance value is equal to or less than the central luminance value (127) or exceeds the central luminance value (127), and the average luminance value is the central luminance value (127). When the following is true, the table creation unit 231 creates a reference table with a convex curve according to equation (2) described later using the power parameter as a luminance conversion table (reference table) 232 for luminance expansion for the input image, and When the average luminance value exceeds the central luminance (127), the table creation unit 231 uses the above power parameter to convert a reference table with a concave curve according to the following equation (3) into a luminance conversion table 232 for luminance expansion for the input image. Create as.

  The image conversion unit 225 uses the luminance conversion table (reference table) 232 created by the reference table determination unit 225 to perform a luminance conversion process on the input image (attention area) held in the image buffer 21, and Then, the image whose output luminance is converted is written back to the image buffer 21 to create a luminance expanded image on the image buffer 21.

  As shown in FIG. 7, the linear interpolation calculation unit 226, for each block pixel using the luminance conversion table according to the luminance expansion parameter obtained in units of blocks, for each block, the luminance of the block and the surrounding four blocks Bilinear interpolation is performed using the conversion table.

  FIG. 10 shows a processing procedure of the luminance expansion processing unit 22 described above.

  The luminance expansion processing unit 22 performs the process shown in FIG. 10 every time an input image in frame units captured by the camera 11 is held in the image buffer 21 via the capture unit 12.

  When the input image to be subjected to the luminance expansion processing is held in the image buffer 21, the block reference luminance setting unit 221 sets the attention area of the input image captured and input by the camera 11 to a block size (for example, specified from the outside). 16 × 16 pixels) is divided into a plurality of blocks (20 × 15 blocks), the sum of the luminance values of each block is obtained for each block, and the luminance sum of each block is used as the luminance information of the block of interest to calculate the average luminance The data is sent to 222 (step S11).

  When the average luminance calculation unit 222 receives the luminance information of the block of interest (total luminance in units of blocks) from the block reference luminance setting unit 221, based on this luminance information, as shown in FIG. 5 and FIG. For each block, the total luminance of all blocks subjected to weighting and convolution processing including the block and its surrounding blocks is calculated to calculate the average luminance value of the block unit (target block) image, and the average luminance of the image of the target block The value is sent to the power parameter calculation unit 223 and the reference table determination unit 224 (step S12).

  The power parameter calculation unit 223 calculates a power parameter for luminance expansion processing for the input image (area of interest) according to the equation (1) described below based on the average luminance value calculated by the average luminance calculation unit 222, The parameter is sent to the reference table determination unit 224.

  The reference table determination unit 224 uses the average luminance value input from the average luminance calculation unit 222 and the power parameter input from the power parameter calculation unit 223 as a luminance expansion parameter, and creates a luminance conversion table according to the luminance expansion parameter. Here, it is determined whether the average luminance value is the central luminance value (127) or less or exceeds the central luminance value (127), and the average luminance value is the central luminance value (127) or less. The table creation unit 231 creates a reference table with a convex curve according to the equation (2) described later using the power parameter as a luminance conversion table (reference table) 232 for luminance expansion with respect to the input image, and the average luminance value is When the central luminance (127) is exceeded, the table creation unit 231 creates a reference table with a concave curve according to the later-described equation (3) using the power parameter as the luminance conversion table 232 for luminance expansion for the input image ( Step S13).

  Here, the luminance expansion processing in the nonlinear luminance expansion processing method using the convex curve and the concave curve described above will be described.

  The power parameter calculation unit 223 automatically calculates a power parameter γ for luminance expansion based on the average luminance of the area of interest using the following equation (1).

γ = (α / Bmed ² ) × (Iave−Bmed) ² +1.0 (1)
(Where α is a fixed value, Bmed is the median brightness, Iave is the average brightness of the input image area of interest)
The average luminance is compared with the central luminance of the luminance gradation range of the input image, and the following equation (2) or (3) is automatically switched using the power parameter γ, and the convex curve and the concave curve are Create a brightness conversion table.

I ′ = Bmax − ((Bmax−I) / Bmax) ^γ × Bmax: Iave ≦ Bmed (2) Convex curve
I ′ = I / Bmax ^γ × Bmax: Iave> Bmed (3) Concave curve (where I ′ is the output luminance, I is the input luminance, Bmed is the median luminance, and Bmax is the maximum luminance value) , Iave is the average brightness of the input image attention area)
The convex curve and concave curve luminance conversion table 232 created by the reference table determination unit 224 is sent to the image conversion unit 225 (step S13).

  The image conversion unit 225 uses the luminance conversion table (reference table) 232 created by the reference table determination unit 225 to perform luminance conversion processing on the input image (attention area) held in the image buffer 21. For each pixel of the target block whose luminance has been expanded in units of blocks, the linear interpolation calculation unit 226 uses the luminance conversion table of the target block and the surrounding four blocks as shown in FIG. Bilinear interpolation is performed in accordance with the distance difference, and an image whose luminance has been converted by this bilinear interpolation is written back to the image buffer 21, whereby a luminance expanded image obtained by performing bilinear interpolation between blocks is created on the image buffer 21. (Steps S14 and S15).

  A more detailed procedure of the above processing is shown in FIGS. In the figure, width is the width of the screen, and here it is 320 pixels. The height is the height of the screen, and is 240 pixels here. BRIGHTNESS_MAX is the maximum value of luminance, here, luminance value = 255. BRIGHTNESS_MIEDIAN is a median value of luminance, and here, luminance value = 128.

  In steps S111 to S116 shown in FIG. 11, securing and initializing a memory area for storing the luminance expanded image (p_TempImg), securing and initializing a memory area for storing the cumulative image and the summed image (block unit), and averaging the luminance are performed. Each processing such as initialization (to 0) of a variable to be obtained and creation and storage of a cumulative total image are executed. In steps S117 to S119 shown in FIG. 12, the sum of luminance in units of blocks is obtained and stored.

  In steps S121 to S123 shown in FIG. 13, a convolution process for the sum of blocks is executed. A loop is performed for the number of blocks, and the following processing shown in FIG. 14 is executed. However, if the block of interest is a screen edge block, the block shifted into one block is the block of interest.

  In steps S131 to S135 shown in FIGS. 14 and 15, the luminance average value tempMean of the block of interest is calculated and stored, and the gamma value of the block is obtained based on the luminance average value.

gamma =
(A / (BRIGHTNESS_MEDIAN * BRIGHTNESS_MEDIAN)) * (powpow ((tempMean-BRIGHTNESS_MEDIAN), 2)) + 1
(Where α is a fixed value)
In steps S136 to S138 shown in FIG. 16, a loop is performed for the luminance value, and a conversion table for the block is created. When the average luminance value is less than or equal to the median value of the luminance value range, the converted luminance value is obtained by the following formula.

(Luminance value after conversion) =
BRIGHTNESS_MAX− (pow ((BRIGHTNESS_MAX−input image luminance value) / BRIGHTNESS_MAX, gamma value)) × BRIGHTNESS_MAX
When the average luminance value is larger than the median value of the luminance value range, the converted luminance value is obtained by the following formula.

(Luminance value after conversion) =
(Pow ((input image brightness value) / BRIGHTNESS_MAX, gamma value)) × BRIGHTNESS_MAX
In steps S141 to S145 shown in FIG. 17, the entire screen is looped and the following processing is executed.

  The block position (decimal point) where the pixel of interest is located is obtained. The position difference of the pixel of interest from the block is obtained with a decimal point. When the block of interest is a screen end block, the block shifted into one block is set as the block of interest. The value of the luminance conversion table of the block and the surrounding blocks (4 blocks including the right, lower, and lower right blocks) is multiplied by the coefficient of the block position difference to obtain a conversion value by the principle of linear interpolation. Check the minimum and maximum of the obtained conversion value and store it as the final result.

  In steps S146 to S148 shown in FIG. 18, the memory area that is no longer needed is released, and the memory area of p_TempImg is released.

  As described above, according to the embodiment of the present invention, the luminance expansion parameter and the luminance conversion table for each image block are obtained, and the conversion luminance is obtained by bilinear interpolation using surrounding blocks in the interpolation processing means 4b of the luminance expansion processing unit 1. By obtaining a value and creating a luminance expansion output image 5 for the input image based on the pixels of each block subjected to bilinear interpolation (reflecting luminance expansion parameters of surrounding blocks) Even if a bright and dark image exists in a part of the entire screen without causing a luminance difference, it is possible to realize an image processing apparatus capable of realizing a luminance expansion processing function capable of clearly displaying and outputting each bright and dark image.

  In the above-described embodiment, the camera image of 0 to 255 gradations of QVGA (320 pixels × 240 pixels) is used as the input image to be subjected to the luminance expansion processing. However, the present invention is not limited to this, and an image such as VGA is used. The luminance expansion processing function unit according to the present invention can also be applied to an image processing apparatus to be handled. In the above embodiment, the components of the luminance expansion processing unit 22 that implements the luminance expansion processing function (block reference luminance setting unit 221, average luminance calculation unit 222, power parameter calculation unit 223, reference table determination unit 224, image conversion) Part of the unit 225, the linear interpolation calculation unit 226, etc.) can be realized by software processing.

  In the embodiment described above, the sum of the luminance values of each block is obtained for each block, and the sum of the luminance of all blocks subjected to weighting and convolution processing including the target block and its surrounding blocks is calculated to be a block unit (target block). However, the present invention is not limited to this. For example, it is also possible to calculate a luminance average for each block and perform weighting and convolution processing using the average luminance for each block. .

  In the embodiment described above, the block division luminance expansion processing function according to the embodiment of the present invention is applied to a nonlinear luminance expansion processing method using a non-linear curve (convex curve, concave curve). For example, the present invention is applicable to a luminance expansion processing method using a sigmoid function curve and other various luminance expansion processing methods.

  Further, in the luminance expansion processing according to the above-described embodiment, since the luminance expansion processing is performed on all the screens of the area of interest, the CPU performance is significantly required (the smaller the block size, the more necessary performance is However, if multiple blocks divided into blocks are defined as one area, and the table is updated by sequentially shifting the area, or if it is determined that the block has a large luminance set However, it is possible to increase the processing speed (reduce the processing load on the CPU) by updating the 8-neighboring surrounding blocks.

  DESCRIPTION OF SYMBOLS 1 ... Luminance expansion | extension processing part, 2 ... Image (input image) image | photographed with monocular camera, 3 ... Area of interest object image, 4a ... Block processing means, 4b ... Interpolation processing means, 5 ... Luminance expansion output image, 11 ... Camera, DESCRIPTION OF SYMBOLS 12 ... Capture part, 13 ... Preliminary image process part, 14 ... Image process part, 15 ... Display part, 21 ... Image buffer, 22 ... Luminance expansion | extension process part, 221 ... Block reference brightness | luminance setting part, 222 ... Average brightness | luminance calculation part 223: Power parameter calculation unit, 224: Reference table determination unit, 225 ... Image conversion unit, 226 ... Luminance expansion output image, 231 ... Table creation unit, 232 ... Luminance conversion table (reference table).

Claims (8)

For an input image that is subject to luminance expansion processing, the set area of interest is divided into a plurality of blocks, and for each divided block, the luminance values of the surrounding blocks are convolved with the luminance values of the plurality of blocks. Block processing means for calculating a luminance expansion parameter in units of blocks and creating a luminance conversion table in units of blocks based on the luminance expansion parameters;
Interpolation processing for performing bilinear interpolation with reference to the luminance conversion table of the surrounding block for each pixel for each pixel of each divided block based on the luminance conversion table in block units created by the block processing means Means,
Image conversion means for creating a luminance expansion output image for the input image based on the pixels of each block subjected to bilinear interpolation by the interpolation processing means;
An image processing apparatus comprising:   The block processing means uses the adjacent upper, lower, left, and right blocks B and the adjacent diagonal blocks C for the target block A to be processed among the divided blocks, and weights A> B> C by weighting A> B> C The image processing apparatus according to claim 1, wherein a convolution process is performed. The block processing means includes
Block dividing means for dividing the image of the area of interest into a plurality of blocks according to a block size designated from the outside;
The luminance sum for each block is calculated for each of the blocks divided by the block dividing means, and the weighting and convolution processing including the block and its surrounding blocks is performed for each block using the calculated luminance sum for each block. A luminance sum calculation means for calculating the luminance sum of all blocks;
Using the luminance sum calculated by the luminance sum calculating means, for each block, the luminance average calculating means for calculating the luminance average of the block;
Luminance expansion parameter calculation means for calculating a luminance expansion parameter for each block based on the luminance average of the block calculated by the luminance average calculation means;
Based on the luminance expansion parameter calculated by the luminance expansion parameter calculating unit, a luminance conversion table creating unit that creates a luminance conversion table for luminance expansion for each block;
The image processing apparatus according to claim 2, further comprising:   The luminance expansion parameter calculation means is based on the luminance average of the block calculated by the luminance average calculation means in accordance with the luminance conversion characteristic of a preset luminance expansion processing method for luminance conversion of the image of the area of interest divided into blocks. The image processing apparatus according to claim 3, wherein a luminance expansion parameter is calculated for each block.   The luminance expansion parameter calculation means uses a luminance expansion processing method using a convex curve and a concave curve for nonlinear luminance expansion processing of the image of the attention area, or a sigmoid function curve for sigmoid luminance expansion processing of the image of the attention area. The image processing apparatus according to claim 4, wherein the image processing apparatus is applied to a luminance expansion processing method.   The image conversion unit performs luminance conversion processing on the attention area held in the image buffer using the luminance conversion table in units of blocks subjected to bilinear interpolation by the interpolation unit, and converts output luminance to input luminance The image processing apparatus according to claim 1, wherein a process of writing the processed image back into the image buffer is performed. A processing device that holds an input image captured by a camera in an image buffer and performs gamma correction on the input image on the image buffer.
The area of interest set for the input image that is subject to luminance expansion processing is divided into a plurality of blocks, and each divided block is divided into blocks based on the luminance values of the plurality of blocks obtained by convolving the luminance values of surrounding blocks. A function for calculating a luminance expansion parameter in units and creating a luminance conversion table in block units based on the luminance expansion parameters;
Based on the created block-by-block luminance conversion table, for each pixel of each divided block, for each block, a function to perform bilinear interpolation with reference to the luminance conversion table of the surrounding blocks;
A function of creating a luminance expansion output image for the input image based on the pixels of each block subjected to the bilinear interpolation;
An image processing program for realizing The function of creating the luminance conversion table for each block is as follows:
Among the divided blocks, with respect to the target block A to be processed, the convolution processing is performed using the adjacent upper, lower, left, and right blocks B and the adjacent diagonal blocks C and weighting A>B> C. Function to
A luminance sum for each block is calculated for each of the divided blocks, and the luminance of all blocks subjected to the weighting and convolution processing including the block and its surrounding blocks is calculated for each block using the calculated luminance sum for each block. A function to calculate the sum,
A function for calculating an average luminance of the block for each block using the calculated luminance sum;
A function for calculating a luminance expansion parameter for each block based on the calculated average luminance of the block;
A function of creating a luminance conversion table for luminance expansion for each block based on the calculated luminance expansion parameter;
The image processing program according to claim 6 for realizing the above.

Images