JPH11215400A - Image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To facilitate the image processing of edge enhancement or the like by switching the detection object of a moving vector in accordance with the value of differential data between first and second image data. SOLUTION: This image processor 1 sequentially calculates image data DA by encoding circuit 2A-2E, generates image data DA-DE by hierarchical structure and holds them for a prescribed period. An interpolation circuit 3 selectively inputs image data DB-DE from the encoding circuits 2B-2E other than the highest encoding circuit 2A and executes an interpolation operation processing and outputs data by a sampling pitch (resolution) corresponding to the processing in a differential calculation circuit 5. The differential calculation circuit 5 selectively inputs the image data DA-DD from the encoding circuits 2A-2D by the control of a system control part 4, calculates a difference with picture data D1 outputted from the interpolation circuit 3 and outputs the result. A block size is detected with the differential data as a reference, and the detection object of the moving vector is set.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to an image processing apparatus, and can be applied to, for example, an image recognition apparatus and a data compression apparatus. The present invention makes it possible to easily execute image processing such as edge enhancement by generating difference data between image data having different resolutions.

[0002]

2. Description of the Related Art Hitherto, in an image processing apparatus, there has been proposed a method of encoding image data by hierarchical encoding such as so-called pyramid encoding. In this encoding method, input image data is converted into image data with different resolutions and subjected to encoding processing.

That is, as shown in FIG. 8, in this type of encoding processing, input image data DA11,
A2, DA21, DA22,..., For example, sequentially form a block of 2 × 2 pixels, and calculate an average value of pixel values in each block. Thereby, in this encoding process, the image data DA11, DA12, DA21, D
A22,..., The highest hierarchical image data,
Lower-layer image data DB11, DB12, DB21, D in which the resolution is reduced by half in the horizontal and vertical directions
B22,... Are generated. Here, the image data with the highest resolution is set as the highest-order image data.

Further, in the encoding process, the image data DB11, DB12, DB21, DB22,.
..., for example, blocks of 2 × 2 pixels are sequentially formed, and an average value of pixel values is calculated in each block. As a result, in this encoding process, the image data DA11, D
A12, DA21, DA22,... Lower-level image data DC11, D in which the resolution is reduced to 1/4 in the horizontal and vertical directions with respect to the highest-level image data.
C12, DC21, DC22,... Are generated.

In this encoding process, the transmission efficiency is improved by selectively transmitting the image data of the required layer from the image data of each layer generated in this way.

[0006]

By the way, if the image data of a plurality of hierarchies generated by performing the encoding process as described above can be applied to processes other than the image transmission, the applicable range of this type of encoding device is increased. It is considered that the encoding apparatus can be expanded and the versatility of this type of encoding apparatus can be improved. In this case, it is conceivable to apply the image data of a plurality of layers generated in this way to various image processing.

The present invention has been made in view of the above points, and has as its object to propose an image processing apparatus which can execute various image processing using image data having different resolutions.

[0008]

According to the present invention, there is provided a first image data having a predetermined sampling pitch and a first image data generated by averaging the first image data in a predetermined pixel unit. And a control unit that switches a motion vector detection target in accordance with the value of the difference data.

[0009] Similarly, there is provided a differential data generating means for generating differential data, and a contour control means for correcting the first image data or the image data corresponding to the first image data using the differential data. .

A moving average data generating means for calculating a moving average of the second image data and outputting the moving average data, and subtracting the moving average data from the first image data to obtain successively obtained subtracted data, And an arithmetic processing means for adding the DC level to generate correction value data.

If difference data is generated between the first image data and the second image data, the difference data
It will indicate the roughness of the image. As a result, the detection target of the motion vector is switched according to the value of the difference data, and for example, the range of block matching can be adaptively switched according to the image.

[0012] In the same difference data, an edge component of the first image data is indicated. Accordingly, the contour control can be easily performed by providing the contour control means for correcting the first image data or the image data corresponding to the first image data based on the difference data.

The moving average data obtained by calculating the moving average of the second image data indicates the illuminance distribution of the image and has shading information. Accordingly, if correction value data is generated by adding a predetermined DC level to subtraction data sequentially obtained by subtracting the moving average data from the first image data, shading is corrected in the correction value data. .

[0014]

Embodiments of the present invention will be described below in detail with reference to the drawings.

(1) First Embodiment FIG. 1 is a block diagram showing an image processing apparatus according to a first embodiment of the present invention. This image processing apparatus 1 is applied to image recognition. The image processing device 1 sequentially processes the image data DA by the encoding circuits 2A to 2E, thereby generating the image data DA to DE having the hierarchical structure described above with reference to FIG.

That is, the first encoding circuit 2A holds the image data DA of the highest hierarchy by inputting and holding the continuous image data DA.
Further, the encoding circuit 2A converts the image data DA into 2 ×
By dividing the image data DA in each block into blocks by averaging the image data DA in each block,
An image data DB consisting of one lower layer is output for the image data DA of the uppermost layer.

Similarly, in the second encoding circuit 2B,
By inputting and storing the continuous image data DB, the image data DB of one lower hierarchy is held for the image data DA of the highest hierarchy. Further, the encoding circuit 2B divides the image data DB into blocks of 2 × 2 pixels, and averages the image data DB in each block. It outputs image data DC consisting of two lower layers.

Similarly, the third and fourth encoding circuits 2
In C and 2D, the preceding encoding circuits 2B and 2C
By inputting and holding the continuous image data DC and DD output from the CPU, the image data DC and DD of the second and third layers lower than the image data DA of the uppermost layer are held, respectively. Further, the encoding circuit 2C
And 2D divide the image data DC and DD into blocks of 2 × 2 pixels, and average these image data DC and DD in each block, thereby obtaining the highest level image data DA. ,
Image data DD composed of three layers and four layers lower, respectively
And DE.

The lowest encoding circuit 2E inputs and holds the continuous image data DE, and holds the image data DE of four lower layers with respect to the image data DA of the highest layer.

When holding the image data in this way, the encoding circuits 2A to 2E hold the image data DA to DE in a common memory. Further, as shown in FIG. 2, in each block of 2 × 2 pixels, in a memory space corresponding to a predetermined pixel, image data DA22, DB22,.
The image data DB11 and DC11 in the lower layer of the first hierarchy composed of the average value of this block are recorded. As a result, the encoding circuits 2A to 2E hold the image data of each layer while reducing the memory capacity. Further, the encoding circuits 2A to 2E output the image data DA to DE thus held in the same arrangement as that at the time of input.

Under the control of the system control circuit 4, the interpolation circuit 3 selectively inputs the image data DB to DE from the coding circuits 2B to 2E except for the top coding circuit 2A and performs an interpolation operation. The output is performed at a sampling pitch (resolution) corresponding to the processing in the difference calculation circuit 5.
Here, the interpolation circuit 3 executes the interpolation calculation processing by, for example, linear interpolation, spline interpolation, or the like. Accordingly, the interpolation circuit 3 converts the low-resolution image data D1 based on the image data of the lower layer to the image data of the hierarchy processed by the difference calculation circuit 5 at a sampling pitch corresponding to the processing of the difference calculation circuit 5. Generate and output.

Under the control of the system control circuit 4, the difference calculation circuit 5 selectively receives the image data DA to DD from the encoding circuits 2A to 2D except for the lowest encoding circuit 2E, and outputs the image data DA to DD from the interpolation circuit 3. The difference between the image data D1 and the calculated image data D1 is calculated and output. Thereby, the difference calculation circuit 5
Generates and outputs image data D2 based on high-frequency components from image data of different layers. Here, the image data D2 indicates the high-frequency components of the image data DA to DD input from the encoding circuits 2A to 2D, so that each of the image data D2 is 2
In the case of a block of × 2 pixels, it indicates the roughness of the pixel value in this block.

The system control circuit 4 is composed of a computer for controlling the operation of the image processing apparatus 1 as a whole. The system control circuit 4 controls the operations of the interpolation circuit 3 and the difference calculation circuit 5, and interpolates the image data of one layer lower than the image data of the layer processed by the image processing circuit 6 to perform To generate D1,
The layer of the image data to be processed by the interpolation circuit 3 is specified. Further, in response to this processing, the system control circuit 4 calculates a difference between the image data of the layer to be processed in the image processing circuit 6 and the image data D1 output from the interpolation circuit 3 so as to calculate a difference value. The operation of the circuit 5 is controlled.

The system control circuit 4 converts the output data D2 of the difference calculation circuit 5 obtained as described above into an absolute value, and then averages the image data to be processed by the interpolation circuit 3 as a unit (hereinafter, this is referred to as an average value). The average value is called the activity). The system control circuit 4 determines whether or not this activity is equal to or more than a predetermined value. If the activity is equal to or less than the predetermined value, the system control circuit 4 uses the image data DC to DE of the next lower layer to the interpolation circuit 3. Instructs generation of image data D1, and generates difference data D2 based on the image data D1.
Enter Further, the system control circuit 4 calculates an activity based on the difference data D2 for each block of 4 × 4 pixels, and determines whether the activity is equal to or more than a predetermined value.

The system control circuit 4 sequentially switches the hierarchy of the image data to be processed by the interpolation circuit 3 until the activity based on the difference data D2 exceeds a predetermined value. Calculate the block size that can detect.

When the block size is calculated in this way, the system control circuit 4 instructs the image processing circuit 6 on the calculated block size and also starts the operation. As a result, the system control circuit 4 instructs the image processing circuit 6 on the detection target of the motion vector by the variable block matching.

The image processing circuit 6 uses the block size specified by the system control circuit 4 to detect a motion vector by applying a block matching method to image data of a preset hierarchy. Further, a motion detection result D3 obtained by tracking the motion of a desired subject from the motion vector detected in this way is output.

In the above configuration, the sequentially input image data DA (FIG. 1) is subjected to hierarchical encoding processing in the encoding circuits 2A to 2E, thereby sequentially reducing the number of pixels to 1/4 for the upper layer. The reduced and averaged image data DA to DE are held in these encoding circuits 2A to 2E.

The image data held in the encoding circuits 2A to 2B in this manner is such that the image data of one layer lower than the image data processed by the image processing circuit 6 is selectively sent to the interpolation circuit 3. The image data is output, interpolated, and the sampling value is set equal to the image data to be processed in the image processing circuit 6, and image data D1 composed of a low frequency component of the image data is generated.

In the following difference calculation circuit 5, the image data processed by the image processing circuit 6 and the image data D1
Is calculated, thereby calculating difference data D2 indicating the pixel roughness for a block of 2 × 2 pixels of the image data to be processed in the image processing circuit 6.

In the value ΔE of the difference data D2,
As shown in FIG. 3, when the difference value ΔE is small by indicating the roughness of the image data DA, a block of 2 × 2 pixels of the image data DA corresponding to one pixel of the image data by the one lower layer In, the change in the pixel value is small, and the area is determined to be, for example, a monochrome object. As a result, for this 2 × 2 pixel block, it is considered that a significant motion vector cannot be detected even if a motion vector is detected between consecutive images. In FIG. 3 and FIG. 4 to be continued, the difference data D2 is calculated between image data of different levels in one layer for the sake of simplicity, and one block is set to two pixels.

On the other hand, as shown in FIG. 4, when the difference value ΔE is large, the change in pixel value is large in this block, so that a motion vector is detected between successive images in this block. Therefore, it is considered that a significant motion vector can be detected.

Even when a significant motion vector cannot be detected in this way, the level to be compared is changed and the image data processed by the image
If the difference value ΔE is calculated from the image data D1 obtained by performing the interpolation operation processing on the image data of the lower layer of the hierarchy, the image data D1 is calculated from the image data obtained by averaging one block by a block of 4 × 4 pixels. In some cases, a large difference value ΔE can be calculated. In this case, a significant motion vector can be detected by detecting motion using a block of 4 × 4 pixels.

Also in this case, when the value of the difference value ΔE is small, the image data D1
Is generated and the difference value ΔE is calculated, one block becomes 8
By calculating the image data D1 from the image data averaged by a block of × 8 pixels, a large difference value ΔE may be calculated in some cases. in this case,
A significant motion vector can be detected by detecting a motion using a block of 8 × 8 pixels.

Thus, in the image processing apparatus 1, 1
The value ΔE of the difference data D2 based on the image data of the lower layer
Is converted to an absolute value, and then averaged for each block of 2 × 2 pixels to detect an activity. Further, when the activity is equal to or less than the predetermined value, the image data of the next lower layer is output to the interpolation circuit 3 under the control of the system control circuit 4, and the sampling value of the image data processed by the image processing circuit 6 is Image data D1 which is set to be equal and has a lower frequency than this image data is generated. Further, difference data D2 is generated from the image data.
After the absolute value is converted into a unit of a block of × 4 pixels, the value is averaged and the activity is detected. Further, when the activity is equal to or less than the predetermined value, the image data of the next lower layer is output to the interpolation circuit 3, where the sampling value is set equal to the image data processed by the image processing circuit 6, and Image data D1 having a lower frequency than the data is generated.

When these activities are repeated and an activity equal to or greater than a predetermined value is detected, the image processing circuit 6 sets a block size corresponding to the activity as a motion vector detection target and sets a predetermined image size. A motion vector is detected from the data, and a desired motion D3 of the subject is calculated based on the detected motion vector.
Is detected.

As a result, in the image processing apparatus 1, a motion vector is detected by varying the block size by effectively using the hierarchized image data. Thus, in this embodiment, the maximum block size is 2 × 2 pixels, 4 × 4 pixels, 8 × 8 pixels,
The selection is made in a range of 6 × 16 pixels and 32 × 32 pixels. Further, a motion vector is detected based on the detected block size, and the motion vector detection range is adaptively switched according to the image.

According to the above arrangement, difference data is generated between image data of different hierarchies, a block size is detected based on the difference data, and a motion vector detection target is set. By using image data effectively, a significant motion vector can be detected with a simple configuration.

(2) Second Embodiment FIG. 5 is a block diagram showing an image processing apparatus according to a second embodiment of the present invention. In the image processing apparatus 10, the same components as those of the image processing apparatus 1 described above with reference to FIG. 1 are denoted by the corresponding reference numerals, and redundant description will be omitted.

In the image processing apparatus 10, the system control circuit 11 controls the interpolation circuit 3 and the difference calculation circuit 5 so that the image data processed by the image processing circuit 12 is processed by the image data set by the operator. Control behavior. Here, this image data is compared with the image data DA processed by the image processing circuit 12 by the same image data DA by the difference calculation circuit 5.
However, in the interpolation circuit 3, a predetermined number of lower-layer image data are set.

The interpolation circuit 3 responds to this setting by
Image data is generated at the same sampling pitch as the image data DA processed in the image processing circuit 12.
Further, the difference calculation circuit 5 performs interpolation calculation processing on the image data to be processed as necessary, thereby generating difference data D2 for image data having the same sampling pitch as the image data DA processed in the image processing circuit 12.

Thus, the system control circuit 11 forms a surfboard filter with the interpolation circuit 3 and the difference calculation circuit 5, and as shown in FIG. 6, the image data DA processed by the image processing circuit 12 is processed by the difference calculation circuit 5. Output image data D2 of the edge component (FIG. 6A)
-(C)).

The image processing circuit 12 adds the image data D2 of the edge component to the image data DA to be processed, thereby outputting image data D4 obtained by emphasizing the edges of the image data DA. Further, under the control of the system control circuit 4, the image data D2 of the edge component is subtracted from the image data DA to be processed, thereby outputting image data D4 obtained by blurring the edges of the image data DA. In these processes, the image processing circuit 12 weights and adds the image data D2 under the control of the system control circuit 4, thereby varying the degree of the outline in various ways. Thus, the image processing circuit 12 controls the contour of the image data D4.

The edge component image data D2 is binarized by a predetermined threshold value, and is output to the subsequent image processing device as edge data D4. Thus, the image processing circuit 12 can use the image data D2 of the edge component by a key signal or the like.

According to the configuration shown in FIG. 5, by generating difference data D2 between image data having different hierarchies, it is possible to easily extract an edge component of an image to be processed.
Thus, contour control can be performed with a simple configuration.

If necessary, for the interpolation circuit 3 and the difference calculation circuit 5 used for generating the difference data D2, the level of the image to be processed is changed to control the degree of contour control and the frequency component of the contour to be controlled. Can be changed in various ways, whereby contour control with desired characteristics can be executed.

Further, by binarizing the image data D2 of the edge component, which can change the degree of the contour control and the frequency component of the contour to be controlled variously, various kinds of signals such as a key signal can be obtained. Reference data D5 that can be used for image processing can be generated.

(3) Third Embodiment In this embodiment, shading of image data is corrected in the image processing apparatus. In this embodiment, the operations of the interpolation circuit 3, the difference calculation circuit 5, and the image processing circuit 12, and the corresponding system control circuit 1
Except for the difference in the operation of No. 1, the configuration is the same as that of the second embodiment, so that the configuration shown in FIG. 5 described in the second embodiment will be described.

The interpolation circuit 3 receives the image data DE of the lowermost layer according to the designation of the system control circuit 11, and calculates a moving average of the image data DE by a predetermined sampling number. At this time, the interpolation circuit 3 performs an arithmetic operation on the image data DE to calculate a moving average, thereby obtaining the image data D
A moving average is calculated for the lightness of E. If the sampling value based on the moving average is set within a predetermined range, the image data D1 by the interpolation circuit 3 has shading information for the image data DA processed by the image processing circuit 12, and the image data DA Shows the illuminance distribution for the image at.

As shown in FIG. 7, the difference calculation circuit 5 receives the image data DA to be processed in the image processing circuit 12, and subtracts the image data D1 from the image data DA (FIGS. 7A and 7B). ). Here, the difference calculation circuit 5
Similar to the interpolation circuit 3, this subtraction process is performed on the brightness of the image data DA by arithmetic processing. As a result, the difference calculation circuit 5 generates image data D2 that sequentially indicates a change in the pixel value of the image data DA based on the illuminance distribution based on the image data D1. Thus, the image data D2 sequentially indicates a change in the pixel value of the image data DA from the illuminance distribution of the image based on the image data DA including shading.

The image processing circuit 12 sets the image data DA to 1
It accumulates and averages for the screen. At this time, the image processing circuit 1
2 calculates an average value for the brightness of the image data DA, similarly to the difference calculation circuit 5. Thereby, the image processing circuit 12 calculates the average brightness DL of the image based on the image data DA. The image processing circuit 12 adds the average value DL to the output data D2 of the difference calculation circuit 5 to add a DC level corresponding to the average brightness DL to the output data D2, thereby correcting shading of the image data DA. A corrected image is generated (FIGS. 7C and 7D).

The image processing circuit 12 includes the system control circuit 1
By giving the offset value to the brightness of the corrected image data generated in this way under the control of 1, the brightness of the corrected image is variously corrected. Further, the system control circuit 11
, The brightness of the output data D2 and / or the image data DA is corrected by a desired gain in place of or in addition to the brightness correction, thereby variously adjusting the brightness contrast in the corrected image. I do.

According to the above configuration, the illuminance distribution of an image based on image data can be easily calculated by detecting the moving average of the image data of the lowest layer. Thus, using the calculated image data D1 of the illuminance distribution,
Even when the illumination of the subject has illumination unevenness, shading due to the illumination unevenness can be easily corrected.

Thus, by correcting the luminance level of the corrected image and the like, it is possible to effectively avoid the influence of the shading and adjust the contrast and brightness in various ways.

(4) Other Embodiments In the above-described first embodiment, the case where the motion vector is detected by changing the block size of the image data set in advance has been described. The present invention is not limited to this, and the hierarchy of the image data to be detected as the motion vector may be switched according to the activity. In this way, for example, a significant motion vector can be detected with a fixed block size.

In the above-described first embodiment,
Although a case has been described where the lower layer image data is subjected to interpolation calculation processing to generate image data having a sampling pitch corresponding to the upper layer and the difference data D2 is generated from this image data, the present invention is not limited to this, and the present invention is not limited to this. If the activity can be detected with sufficient accuracy, the interpolation calculation process may be omitted, and the difference data may be generated by subtracting the image data of the lower layer from the image data of the corresponding block.
By doing so, the overall configuration can be further simplified.

Similarly, in the second embodiment, the interpolation operation in the interpolation circuit 3 may be omitted.

In the third embodiment described above,
The moving average of the image data DE of the lowermost layer is used to calculate the image data D.
Although the case of generating 1 has been described, the present invention is not limited to this, and can be widely applied to a case where a moving average is generated from image data of various layers as needed.

Further, in the third embodiment described above, a case has been described in which the image data DA of the uppermost layer is averaged for one screen to calculate the average illuminance, but the present invention is not limited to this, and the present invention is not limited to this. The average illuminance may be calculated by averaging the image data of the layers.

In the third embodiment described above,
The case has been described where the image data DA is averaged for one screen to calculate the average illuminance and the DC level based on the average illuminance is added. However, the present invention is not limited to this, and the DC level is set to various values. You may do so.

In the first embodiment described above,
Although the case where the motion of the subject is detected by detecting the motion vector has been described, the present invention is not limited to this. For example, a motion vector can be detected and used for data compression processing.

Further, in the above-described embodiment, two
Although the case where image data having a hierarchical structure is generated by averaging image data in units of × 2 pixels has been described, the present invention is not limited to this, and is widely applied to the case where image data having a hierarchical structure is generated in various pixel units. Can be applied.

[0063]

As described above, according to the present invention, difference data is generated between image data having different resolutions, and a detection target of a motion vector is switched according to the difference data. , The detection target of the motion vector can be switched adaptively.

Further, by generating difference data between image data having different resolutions, it is possible to detect an edge component with a simple configuration, and to easily execute processing such as contour control using the edge component. Can be.

Further, the illuminance distribution of the image can be detected by the moving average of the low-resolution image data, and shading correction can be performed with a simple configuration using the illuminance distribution.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating an image processing apparatus according to a first embodiment of the present invention.

FIG. 2 is a schematic diagram used for describing an encoding circuit of the image processing apparatus in FIG. 1;

FIG. 3 is a schematic diagram for explaining difference data of the encoding processing device of FIG. 1;

FIG. 4 is a schematic diagram for explaining a case where the value of difference data in FIG. 3 is large.

FIG. 5 is a block diagram illustrating an image processing apparatus according to a second embodiment of the present invention.

FIG. 6 is a schematic diagram for explaining the operation of the image processing apparatus of FIG. 5;

FIG. 7 is a schematic diagram for explaining the operation of an image processing apparatus according to a third embodiment of the present invention.

FIG. 8 is a schematic diagram used for explaining hierarchical coding.

[Explanation of symbols]

1, 10 image processing apparatus, 2A to 2E coding circuit, 3 interpolation circuit, 4, 11 system control circuit,
5 ... Difference calculation circuit, 6, 12 ... Image processing circuit

Claims (10)

[Claims] 1. A difference generating difference data between first image data at a predetermined sampling pitch and second image data generated by averaging the first image data in predetermined pixel units. An image processing apparatus comprising: a data generation unit; and a control unit that switches a motion vector detection target according to a value of the difference data. 2. The image processing apparatus according to claim 1, wherein the control unit switches a detection range of the motion vector to switch a detection target of the motion vector. 3. The image processing apparatus according to claim 1, wherein the control unit switches a resolution of image data for detecting the motion vector, and switches a detection target of the motion vector. 4. A difference for generating difference data between first image data at a predetermined sampling pitch and second image data generated by averaging the first image data in predetermined pixel units. An image processing apparatus comprising: a data generation unit; and an outline control unit that corrects the first image data or image data corresponding to the first image data based on the difference data. 5. The image processing apparatus according to claim 4, further comprising a binarizing means for binarizing the difference data. 6. A second image data generated by averaging said first image data in predetermined pixel units with respect to first image data at a predetermined sampling pitch,
A moving average data generating means for calculating a moving average of the second image data and outputting moving average data; a predetermined direct current to subtraction data obtained by sequentially subtracting the moving average data from the first image data; An image processing apparatus comprising: an arithmetic processing unit that generates correction value data by adding levels. 7. The image processing apparatus according to claim 6, wherein the DC level is an average value obtained by averaging the first or second image data for one screen. 8. The image processing apparatus according to claim 6, wherein said arithmetic processing means offsets said correction value data. 9. The image processing apparatus according to claim 6, wherein the arithmetic processing unit generates the subtraction data after performing weighting processing on the first image data and / or the moving average data. 10. The image processing apparatus according to claim 6, wherein said arithmetic processing means weights said correction value data.