JP4788695B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus, and can be applied to, for example, an image recognition apparatus and a data compression apparatus. The present invention makes it possible to easily execute image processing such as edge enhancement by generating difference data between image data having different resolutions.

  Conventionally, in an image processing apparatus, a method of encoding image data by hierarchical encoding such as so-called pyramid encoding has been proposed. In this encoding method, input image data is converted into image data having a different resolution and encoded.

  That is, as shown in FIG. 8, in this type of encoding processing, for example, 2 × 2 pixel blocks are sequentially formed from the input image data DA11, DA12, DA21, DA22,. Calculate the average value. As a result, in this encoding process, the image data DA11, DA12, DA21, DA22,... Of the uppermost layer of the image data DA11, DA12, DA21, DA22,. Image data DB11, DB12, DB21, DB22,... Are generated. Here, the image data with the highest resolution is the highest-order image data.

  Further, in this encoding process, for example, 2 × 2 pixel blocks are sequentially formed from the image data DB11, DB12, DB21, DB22,..., And an average value of pixel values is calculated in each block. As a result, in this encoding process, the upper layer image data of the image data DA11, DA12, DA21, DA22,... Has a lower layer resolution that is reduced to ¼ in the horizontal and vertical directions. Image data DC11, DC12, DC21, DC22,... Are generated.

  In this encoding process, the transmission efficiency is improved by selectively transmitting image data of a required hierarchy from the image data of each hierarchy generated in this way.

By the way, if the multi-layered image data generated by encoding processing in this way can be applied to processing other than image transmission, the application range of this type of encoding device can be expanded. It is considered that the versatility of the encoding device can be improved. In this case, it is conceivable to apply the multi-layer image data generated in this way to various image processing.
JP 07-264603 A Japanese Patent Application Laid-Open No. 07-222170 Japanese Patent Application Laid-Open No. 07-222157 Japanese Patent Application Laid-Open No. 09-065339

  The present invention has been made in consideration of the above points, and intends to propose an image processing apparatus capable of executing various image processing using image data having different resolutions.

  In order to solve such a problem, the first aspect of the present invention provides the second image data generated by averaging the first image data in predetermined pixel units with respect to the first image data having a predetermined sampling pitch. The moving average data generating means for calculating the moving average of the second image data and outputting the moving average data; and subtracting data sequentially obtained by subtracting the moving average data from the first image data. Arithmetic processing means for generating correction value data for correcting the first image data or the image data corresponding to the first image data by adding a predetermined DC level is provided.

  The moving average data obtained by calculating the moving average of the second image data indicates the illuminance distribution of the image and has shading information. Thus, if correction value data is generated by adding a predetermined DC level to subtraction data obtained by subtracting the moving average data from the first image data, shading is corrected in the correction value data. .

  According to the present invention, an illuminance distribution of an image can be detected by a moving average of low-resolution image data, and shading correction can be performed with a simple configuration using the illuminance distribution.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate.

(1) First Embodiment FIG. 1 is a block diagram showing an image processing apparatus according to a first embodiment of the present invention. This image processing apparatus 1 is applied to image recognition. The image processing apparatus 1 sequentially calculates the image data DA by the encoding circuits 2A to 2E, thereby generating the image data DA to DE having the hierarchical structure described above with reference to FIG. 8 and holding it for a predetermined period. In addition, this Example is an Example which shows the structure of the premise of this application.

  That is, in the first encoding circuit 2A, continuous image data DA is input and held, thereby holding the image data DA of the highest hierarchy. Further, the encoding circuit 2A blocks the image data DA by a block of 2 × 2 pixels, and averages the image data DA in each block, whereby the image data DA by the highest hierarchy is obtained. An image data DB consisting of one layer lower layer is output.

  Similarly, in the second encoding circuit 2B, by inputting and holding continuous image data DB, the image data DB of one layer lower layer is held with respect to the image data DA of the highest layer. Further, the encoding circuit 2B blocks these image data DBs by blocks of 2 × 2 pixels, and averages these image data DBs in each block, whereby the image data DA by the highest hierarchy is obtained. Image data DC consisting of two layers lower layers is output.

  Similarly, in the third and fourth encoding circuits 2C and 2D, the continuous image data DC and DD output from the preceding encoding circuits 2B and 2C are input and held, so that the highest level is obtained. The image data DA and DD of the second and third hierarchies are held for the image data DA by the hierarchies, respectively. Further, the encoding circuits 2C and 2D block the image data DC and DD into blocks of 2 × 2 pixels, and average the image data DC and DD in each block, thereby obtaining an image of the highest hierarchy. For the data DA, the image data DD and DE which are respectively in the 3rd and 4th layer lower layers are output.

  In the lowest-order encoding circuit 2E, by inputting and holding continuous image data DE, the image data DE in the lower hierarchy of 4 layers is held for the image data DA in the highest hierarchy.

  When the image data is held in this way, the encoding circuits 2A to 2E hold the image data DA to DE in a common memory. Further, as shown in FIG. 2, in each block of 2 × 2 pixels, in the memory space corresponding to a predetermined pixel, instead of the image data DA22, DB22,. The image data DB11 and DC11 of the one layer lower layer composed of average values are recorded. Accordingly, the encoding circuits 2A to 2E reduce the memory capacity and hold the image data of each layer. Further, the encoding circuits 2A to 2E output the image data DA to DE thus held in the same arrangement as that at the time of input.

  Under the control of the system control circuit 4, the interpolation circuit 3 selectively inputs the image data DB to DE from these encoding circuits 2B to 2E except the highest-order encoding circuit 2A, performs interpolation calculation processing, and performs a difference calculation circuit. 5 is output at a sampling pitch (resolution) corresponding to the processing in step 5. Here, the interpolation circuit 3 executes an interpolation calculation process by, for example, linear interpolation, spline interpolation, or the like. As a result, the interpolation circuit 3 uses the sampling pitch corresponding to the processing in the difference calculation circuit 5 to obtain the low-resolution image data D1 based on the lower layer image data for the image data in the layer processed by the difference calculation circuit 5. Generate and output.

  The difference calculation circuit 5 selectively inputs the image data DA to DD from the encoding circuits 2A to 2D excluding the lowest-order encoding circuit 2E under the control of the system control circuit 4, and the image output from the interpolation circuit 3 The difference between the data D1 is calculated and output. As a result, the difference calculation circuit 5 generates and outputs image data D2 based on high frequency components from image data of different layers. Here, the image data D2 indicates the high frequency components of the image data DA to DD input from the encoding circuits 2A to 2D. Will be shown.

  The system control circuit 4 is configured by a computer that controls the operation of the entire image processing apparatus 1. The system control circuit 4 controls the operations of the interpolation circuit 3 and the difference calculation circuit 5, interpolates the image data of one layer lower layer with respect to the image data of the layer processed in the image processing circuit 6, and performs image data processing. A hierarchy of image data to be processed by the interpolation circuit 3 is designated so as to generate D1. Corresponding to this processing, the system control circuit 4 calculates a difference so as to calculate a difference value between the image data of the hierarchy processed in the image processing circuit 6 and the image data D1 output from the interpolation circuit 3. The operation of the circuit 5 is controlled.

  The system control circuit 4 converts the output data D2 of the difference calculation circuit 5 obtained in this way into absolute values, and then averages the image data to be processed by the interpolation circuit 3 (hereinafter, this average value is referred to as the average value). Called an activity). The system control circuit 4 determines whether or not this activity is equal to or greater than a predetermined value. If the activity is equal to or less than the predetermined value, the system control circuit 4 further uses the image data DC to DE of one layer lower layer than the interpolation circuit 3. The generation of the image data D1 is instructed, and difference data D2 based on the image data D1 is input. Further, the system control circuit 4 calculates an activity based on the difference data D2 in units of 4 × 4 pixel blocks, and determines whether this activity is equal to or greater than a predetermined value.

  The system control circuit 4 sequentially switches the hierarchy of image data to be processed by the interpolation circuit 3 until the activity based on the difference data D2 reaches a predetermined value or more, and thereby the image processing circuit 6 detects a sufficiently significant motion vector. Calculate the block size that can be.

  When the block size is calculated in this manner, the system control circuit 4 instructs the calculated block size to the image processing circuit 6, and also instructs the start of the operation. As a result, the system control circuit 4 instructs the image processing circuit 6 to detect a motion vector by variable block matching.

  The image processing circuit 6 detects a motion vector by applying a block matching method to image data of a preset hierarchy according to the block size instructed from the system control circuit 4. Furthermore, a motion detection result D3 obtained by tracking the motion of a desired subject from the motion vector detected in this way is output.

  In the above configuration, the sequentially input image data DA (FIG. 1) is hierarchically encoded in the encoding circuits 2A to 2E, thereby reducing the number of pixels sequentially to the upper layer to 1/4. The averaged image data DA to DE are held in the encoding circuits 2A to 2E.

  The image data held in the encoding circuits 2A to 2B in this way is selectively output to the interpolation circuit 3 as image data of one layer lower layer than the image data processed in the image processing circuit 6 here. Here, the interpolation processing is performed, the sampling values are set equal to the image data processed in the image processing circuit 6, and the image data D1 including the low frequency component of the image data is generated.

  In the subsequent difference calculation circuit 5, a difference value between the image data processed in the image processing circuit 6 and the image data D1 is calculated, whereby a pixel of 2 × 2 pixel block of the image data processed in the image processing circuit 6 is obtained. Difference data D2 indicating the roughness of is calculated.

  In the value ΔE of the difference data D2, as shown in FIG. 3, when the difference value ΔE is small by indicating the roughness of the image data DA, it corresponds to one pixel of the image data by this one layer lower layer. In the 2 × 2 pixel block of the image data DA, the change in pixel value is small, and for example, it is determined as an area like a monochrome object. As a result, for this 2 × 2 pixel block, it is considered that a significant motion vector cannot be detected even if a motion vector is detected between successive images. In FIG. 3 and subsequent FIG. 4, for simplification of explanation, a case where difference data D <b> 2 is calculated between different image data in one layer hierarchy is described with one block set to 2 pixels.

  On the other hand, as shown in FIG. 4, when this difference value ΔE is large, the change in the pixel value is large in this block, and for this block, the motion vector is detected between consecutive images and significant. It is considered that a simple motion vector can be detected.

  In addition, even when a significant motion vector cannot be detected in this way, this comparison layer is changed, and the image data processed by the image processing circuit 6 is subjected to an interpolation calculation process using the image data of the lower two layers. If the difference value ΔE is calculated from the data D1, the difference value ΔE having a large value can be calculated by calculating the image data D1 from the image data obtained by averaging one block with a block of 4 × 4 pixels. There is a case. In this case, a significant motion vector can be detected by detecting motion using a 4 × 4 pixel block.

  Also in this case, when the difference value ΔE is small, if the image data D1 is generated from the lower layer image data and the difference value ΔE is calculated, an image obtained by averaging one block with a block of 8 × 8 pixels By calculating the image data D1 from the data, it may be possible to calculate a difference value ΔE having a large value. In this case, a significant motion vector can be detected if motion is detected using an 8 × 8 pixel block.

  As a result, in the image processing apparatus 1, the value ΔE of the difference data D2 is converted into an absolute value by the image data of the lower layer of the first layer, and then averaged in units of 2 × 2 pixel blocks to detect an activity. Further, when this activity is less than or equal to a predetermined value, under control of the system control circuit 4, the subsequent lower layer image data is output to the interpolation circuit 3, where the sampling value is set for the image data processed by the image processing circuit 6. Image data D1 that is set equal and has a lower frequency than this image data is generated. Further, difference data D2 is generated from the image data, and the difference data D2 is converted to an absolute value in units of 4 × 4 pixel blocks, and then averaged to detect an activity. Further, when this activity is less than or equal to a predetermined value, the subsequent lower layer image data is output to the interpolation circuit 3, where the sampling value is set equal to the image data processed by the image processing circuit 6, and this image Image data D1 having a much lower frequency than the data is generated.

  When these processes are repeated and an activity of a predetermined value or more is detected, the image processing circuit 6 sets a block size corresponding to this activity as a motion vector detection target, and moves the motion of preset image data. A vector is detected, and a desired motion D3 of the subject is detected based on the detected motion vector.

  As a result, the image processing apparatus 1 detects the motion vector by changing the block size by effectively using the layered image data. Thus, in this embodiment, this block size is selected in the range of 2 × 2 pixels, 4 × 4 pixels, 8 × 8 pixels, 16 × 16 pixels, and 32 × 32 pixels at the maximum. Furthermore, a motion vector is detected based on the detected block size, and the motion vector detection range is adaptively switched according to the image.

  According to the above configuration, the difference data is generated between the image data of different layers, the block size is detected on the basis of the difference data, and the motion vector detection target is set. Effectively, a significant motion vector can be detected with a simple configuration.

(2) Second Embodiment FIG. 5 is a block diagram showing an image processing apparatus according to a second embodiment of the present invention. In this image processing apparatus 10, the same configurations as those of the image processing apparatus 1 described above with reference to FIG. In addition, this Example is an Example which shows the structure of the premise of this application.

  In this image processing apparatus 10, the system control circuit 11 controls the operations of the interpolation circuit 3 and the difference calculation circuit 5 so as to process the image data set by the operator for the image data processed in the image processing circuit 12. To do. Here, the image data is, for example, the same image data DA in the difference calculation circuit 5 as compared with the image data DA processed in the image processing circuit 12, and a predetermined number of lower layer image data in the interpolation circuit 3. Is set.

  In response to this setting, the interpolation circuit 3 generates image data at the same sampling pitch as the image data DA processed in the image processing circuit 12. Further, the difference calculation circuit 5 performs an interpolation calculation process on the image data to be processed as necessary, thereby generating difference data D2 for the image data having the same sampling pitch as the image data DA processed in the image processing circuit 12.

  As a result, the system control circuit 11 forms a surfboard filter by the interpolation circuit 3 and the difference calculation circuit 5, and as shown in FIG. The image data D2 is output (FIGS. 6A to 6C).

  The image processing circuit 12 adds the image data D2 of the edge component to the image data DA to be processed, and thereby outputs image data D4 in which the edge of the image data DA is emphasized. Also, under the control of the system control circuit 4, the image data D2 of the edge component is subtracted from the image data DA to be processed, thereby outputting the image data D4 in which the edge of the image data DA is blurred. In these processes, the image processing circuit 12 weights and adds the image data D2 under the control of the system control circuit 4, thereby varying the degree of contour in various ways. Thus, the image processing circuit 12 controls the contour of the image data D4.

  Further, the edge component image data D2 is binarized by a predetermined threshold value, and is output as edge data D4 to the subsequent image processing apparatus. As a result, the image processing circuit 12 can use the image data D2 of the edge component by the key signal or the like.

  According to the configuration shown in FIG. 5, by generating difference data D2 between image data of different layers, it is possible to easily extract the edge component of the processing target image, thereby performing contour control with a simple configuration. Can do.

  Further, for the interpolation circuit 3 and the difference calculation circuit 5 used for generating the difference data D2, the level of the processing target image is changed so that the degree of contour control and the frequency components of the contour to be controlled are varied. It is possible to change the contour and thereby perform contour control with desired characteristics.

  Further, by binarizing the edge component image data D2 that can change the degree of contour control and the frequency component of the contour to be controlled in various ways, for example, for various image processing such as key signals. Reference data D5 that can be used can be generated.

(3) Third Embodiment In this embodiment, shading of image data is corrected in an image processing apparatus. In this embodiment, the configuration of the second embodiment is the same as that of the second embodiment except that the operations of the interpolation circuit 3, the difference calculation circuit 5, and the image processing circuit 12 and the corresponding system control circuit 11 are different. The configuration shown in FIG. 5 describing the second embodiment will be described.

  The interpolation circuit 3 receives the image data DE in the lowest layer as specified by the system control circuit 11, and calculates a moving average of the image data DE according to a predetermined sampling number. At this time, the interpolation circuit 3 calculates the moving average for the brightness of the image data DE by calculating the moving average by calculating the image data DE. If the sampling value by the moving average is set within a predetermined range, the image data D1 by the interpolation circuit 3 has shading information for the image data DA to be processed by the image processing circuit 12, and the image data DA The illuminance distribution is shown for the image at.

  As shown in FIG. 7, the difference calculation circuit 5 receives the image data DA to be processed by the image processing circuit 12, and subtracts the image data D1 from the image data DA (FIGS. 7A and 7B). Here, similarly to the interpolation circuit 3, the difference calculation circuit 5 performs this subtraction process on the brightness of the image data DA by arithmetic processing. Thus, the difference calculation circuit 5 generates image data D2 that sequentially indicates changes in pixel values of the image data DA with reference to the illuminance distribution by the image data D1. Thus, the image data D2 sequentially indicates changes in pixel values of the image data DA based on the illuminance distribution of the image based on the image data DA including shading.

  The image processing circuit 12 accumulates the image data DA for one screen and averages it. At this time, the image processing circuit 12 calculates an average value for the brightness of the image data DA, similarly to the difference calculation circuit 5. Thereby, the image processing circuit 12 calculates the average brightness DL of the image based on the image data DA. The image processing circuit 12 adds the average value DL to the output data D2 of the difference calculation circuit 5, thereby adding the DC level corresponding to the average brightness DL to the output data D2, thereby correcting the shading of the image data DA. A corrected image is generated (FIGS. 7C and 7D).

  The image processing circuit 12 corrects the brightness of the corrected image in various ways by giving an offset value to the brightness of the corrected image data generated in this way under the control of the system control circuit 11. Further, the brightness of the output data D2 and / or the image data DA is corrected by a desired gain under the control of the system control circuit 11 instead of or in addition to the correction of the brightness. Adjust contrast in various ways.

  According to the above configuration, the illuminance distribution of the image based on the image data can be easily calculated by detecting the moving average of the image data of the lowest layer. Accordingly, even when the illumination of the subject has uneven illumination, the shading due to the uneven illumination can be easily corrected using the image data D1 of the calculated illuminance distribution.

  Thus, by correcting the brightness level of the corrected image or the like, it is possible to effectively avoid the influence of this shading and adjust the contrast and brightness in various ways.

(4) Other Embodiments In the first embodiment described above, the case where the motion vector is detected by changing the block size of the preset image data has been described. However, the present invention is not limited to this, You may make it switch the hierarchy of the image data used as a motion vector detection object by activity. In this way, for example, a significant motion vector can be detected with a fixed block size.

  In the first embodiment described above, the case has been described in which image data of the sampling pitch corresponding to the upper layer is generated by interpolating the lower layer image data, and the difference data D2 is generated from this image data. However, the present invention is not limited to this, and when the activity can be detected with practically sufficient accuracy, the interpolation calculation process is omitted, and the lower layer image data is subtracted from the image data of the corresponding block to obtain the difference data. It may be generated. In this way, the overall configuration can be further simplified.

  Similarly, in the second embodiment, the interpolation calculation process in the interpolation circuit 3 may be omitted.

  In the third embodiment described above, the case where the image data D1 is generated by moving and averaging the image data DE of the lowest layer has been described. However, the present invention is not limited to this, and various hierarchies can be used as necessary. The present invention can be widely applied to the case where a moving average is generated from the image data.

  Further, in the third embodiment, the case where the average illuminance is calculated by averaging the image data DA of the highest layer for one screen has been described. However, the present invention is not limited to this, and the lower layer image data is calculated. May be averaged to calculate the average illuminance.

  In the third embodiment, the case where the average illuminance is calculated by averaging the image data DA for one screen and the DC level based on the average illuminance is added has been described. However, the present invention is not limited to this. The direct current level may be set to various values.

  In the first embodiment described above, the case where the motion vector is detected to detect the motion of the subject has been described. However, the present invention is not limited to this. For example, the motion vector is detected and used for data compression processing. You can also.

  In the above-described embodiment, the case has been described in which image data is sequentially averaged in units of 2 × 2 pixels and image data having a hierarchical structure is generated. The present invention can be widely applied when generating image data having a structure.

  The present invention relates to an image processing apparatus, and can be applied to, for example, an image recognition apparatus and a data compression apparatus.

1 is a block diagram illustrating an image processing apparatus according to a first embodiment of the present invention. It is a basic diagram with which it uses for description of the encoding circuit of the image processing apparatus of FIG. It is a basic diagram with which it uses for description of the difference data of the encoding processing apparatus of FIG. It is a basic diagram with which it uses for description when the value of the difference data of FIG. 3 is large. It is a block which shows the image processing apparatus which concerns on 2nd Example of this invention. FIG. 6 is a schematic diagram for explaining the operation of the image processing apparatus of FIG. 5. It is a basic diagram with which it uses for description of operation | movement of the image processing apparatus which concerns on 3rd Example of this invention. It is an approximate line figure used for explanation of hierarchical coding.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,10 ... Image processing apparatus, 2A-2E ... Coding circuit, 3 ... Interpolation circuit, 4, 11 ... System control circuit, 5 ... Difference calculation circuit, 6, 12 ... Image processing circuit

Claims (4)

With respect to the first image data having a predetermined sampling pitch, second image data generated by averaging the first image data in predetermined pixel units is received, and a moving average of the second image data is obtained. Moving average data generating means for calculating and outputting moving average data;
A predetermined DC level is added to the subtraction data sequentially obtained by subtracting the moving average data from the first image data to correct the first image data or the image data corresponding to the first image data. Arithmetic processing means for generating correction value data ;
Equipped with a,
The arithmetic processing means generates the subtraction data after weighting the first image data and / or the moving average data .   The image processing apparatus according to claim 1, wherein the DC level is an average value obtained by averaging the first or second image data for one screen.   The image processing apparatus according to claim 1, wherein the arithmetic processing unit offsets the correction value data. The image processing apparatus according to claim 1, wherein the arithmetic processing unit weights the correction value data.

Images