JP2012155573A - Image area division device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve image quality further by orthogonally transforming signals of a source image, dividing a space area for each band and obtaining space area division signals.SOLUTION: An orthogonal transformation part 2 executes orthogonal transformation to a source image O and extracts space high frequency signals O, O, O. A space high frequency composition part 3 composites the space high frequency signals O, O, Oand generates space high frequency composite signals S. A space area division part 5 executes the orthogonal transformation of the space high frequency composite signals S for the number of preset band decomposition number kto generate space low frequency signals S=Sto S, sets a flag 1 by a threshold computation of the spectrum power of respective element positions, generates the space area division signals BSto BSfor the respective bands, generates space high frequency signals S, S, Sby the orthogonal transformation, and generates space area division signals BS by the similar threshold computation.

Description

  The present invention relates to an apparatus and a program for orthogonally transforming a signal of an original image and performing spatial domain division for each band.

  In recent years, image super-resolution technology has been actively researched and developed. The super-resolution technique is a technique that complements a spatial high frequency exceeding the Nyquist frequency of an image and enlarges the image. A typical complementing method is a frequency reconstruction type. The frequency reconstruction type uses spatial frequency information equal to or lower than the Nyquist frequency to supplement a spatial high frequency exceeding the Nyquist frequency. In this frequency reconstruction type, in order to improve the image quality, it is required to analyze the spatial high-frequency component of the image in detail for each band.

  For example, a super-resolution technique using a frequency reconfiguration type is described in a patent application in which the applicant and the inventor are the same and the unpublished patent application (Japanese Patent Application No. 2010-105709) at the time of this application. ing. This technology uses wavelet transform to decompose an image into bands, and performs super-resolution by applying Gaussian filter processing with different dispersion values for each band to complement the spatial high frequency exceeding the Nyquist frequency. is there.

  On the other hand, for the purpose of low bit rate encoding of an image, a technique for performing image encoding while retaining an edge region is known (see Patent Document 1). This image encoding apparatus uses a Sobel filter to detect an edge region, detects an image edge based on a luminance signal and a color difference signal, and highly accurate image edge region from the plurality of image edges. And the image is divided into a texture area and an edge area.

Japanese Patent No. 3458972

  With the technique described in the above-mentioned unpublished patent application, high-quality images can be reproduced by complementing the spatial high frequency exceeding the Nyquist frequency. In this case, if the spatial region is divided using the spatial frequency information for each band, and Gaussian filter processing with an optimal dispersion value can be performed for each spatial region, further improvement in image quality can be expected.

  On the other hand, the spatial region can be divided using the Sobel filter of Patent Document 1 described above. However, the Sobel filter is a kind of simple high-pass filter, and the spatial frequency band cannot be analyzed in detail for each band. For this reason, in the spatial region division of a moving object or the like, there is a problem in the accuracy and accuracy of the spatial region division because the image edge is blurred.

  Therefore, the present invention has been made to solve the above-described problems, and the object thereof is to obtain a spatial domain division signal by orthogonally transforming an original image signal and performing spatial domain division for each band. Another object of the present invention is to provide an image area dividing device and program capable of further improving image quality.

  In order to solve the above problems, an image region segmentation apparatus according to the present invention performs orthogonal transform on an original image signal to generate a spatial high-frequency signal, and combines the spatial high-frequency signal generated by the orthogonal transform unit. The spatial high-frequency synthesis unit that generates the spatial high-frequency synthesis signal, and the spatial high-frequency synthesis signal generated by the spatial high-frequency synthesis unit are subjected to orthogonal transformation with a preset number of band resolutions, and the signal after the orthogonal transformation is obtained. A spatial region dividing unit that divides the spatial region by performing threshold processing on the element value and generates a spatial region divided signal for each band.

  Further, the image region dividing apparatus according to the present invention orthogonally transforms the spatial high-frequency synthesized signal generated by the spatial high-frequency synthesizing unit instead of the preset number of band resolutions, and the spectral power of the signal after the orthogonal transformation And a spatial resolution dividing unit that performs orthogonal transformation of the band resolution determined by the band resolution determination unit on the spatial high frequency synthesized signal. And generating a spatial domain division signal for each band.

  The image region dividing apparatus according to the present invention may further include a space region dividing signal for each band generated by the space region dividing unit, and a space region dividing signal having a one-dimensional lower rank decomposition band than the band. Then, for each corresponding position in the two spatial domain division signals, a product operation of a preset number of band resolutions is performed, and a threshold value process is performed on the element value of the signal after the product calculation to divide the edge region And an edge region dividing unit that generates an edge region dividing signal for each band.

  The image region dividing apparatus according to the present invention may further include a space region dividing signal for each band generated by the space region dividing unit and a space region dividing signal having a one-dimensional smaller band decomposition level than the band. The product of the band decomposition number determined by the band decomposition number determination unit is calculated for each corresponding position in the two spatial domain division signals, and threshold processing is performed on the element value of the signal after the product calculation An edge region dividing unit that divides the edge region and generates an edge region divided signal for each band.

  In the image region dividing device according to the present invention, the edge region dividing unit removes isolated points in the edge region divided signal by using a filter with respect to the generated edge region divided signal for each band. Features.

  The image region segmentation device according to the present invention is characterized in that the orthogonal transform is a discrete wavelet packet transform, a discrete wavelet transform, a discrete Fourier transform, or a discrete cosine transform.

  Furthermore, an image region dividing program according to the present invention causes a computer to function as the image region dividing device.

  As described above, according to the present invention, the original image can be divided into spatial regions for each band, and the image quality can be improved by using the spatial region division signal or the edge region division signal obtained as a result of the spatial region division. Can be realized. For example, when the present invention is applied to a frequency reconfiguration type super-resolution apparatus, a Gaussian filter having an optimum dispersion value is used for a region indicated by a spatial domain division signal or an edge domain division signal for each band. Super-resolution processing with high image quality can be performed.

It is a block diagram which shows the structure of the image area division | segmentation apparatus by embodiment of this invention. It is a flowchart which shows the process of an orthogonal transformation part. It is a figure explaining the processing content of an orthogonal transformation part. It is a flowchart which shows the process of a spatial high frequency synthetic | combination part. It is a figure explaining the processing content of a spatial high frequency synthetic | combination part. It is a flowchart which shows the process of a zone decomposition number determination part. It is a figure explaining the processing content of a zone decomposition number determination part. It is a flowchart which shows the process of a space area | region division part. It is a figure explaining the processing content of a space area | region division part. It is a flowchart which shows the process of an edge area | region division part. It is a figure explaining the processing content of an edge area | region division part. 3 is a flowchart showing processing of Example 1; 6 is a flowchart illustrating processing of Example 2; 10 is a flowchart illustrating processing of Example 3; 14 is a flowchart illustrating processing of Example 4; It is a block diagram which shows the structure of the image coding apparatus to which the image area division | segmentation apparatus by embodiment of this invention is applied. It is a block diagram which shows the structure of the image decoding apparatus to which the image area division | segmentation apparatus by embodiment of this invention is applied. It is a figure which shows the outline | summary of the dispersion value determination process in an image coding apparatus.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an image area dividing device according to an embodiment of the present invention. The image region dividing device 1 includes an orthogonal transform unit 2, a spatial high frequency synthesizing unit 3, a band resolution number determining unit 4, a spatial region dividing unit 5, an edge region dividing unit 6, and a selecting unit 7. The image region dividing device 1 receives an original image O, performs orthogonal transformation to extract a spatial high frequency signal, synthesizes the spatial high frequency signal, generates a spatial high frequency synthesized signal, and divides the spatial region based on the spatial high frequency synthesized signal A signal BS ^k is generated and output, and an edge region divided signal PrBS ^k is generated and output based on the spatial region divided signal BS ^k .

The orthogonal transformation unit 2 of the image region dividing device 1 receives the original image O, performs orthogonal transformation of the original image O, and performs spatial low-frequency signal O _LL ¹ and spatial high-frequency signals O _LH ¹ , O _HL ¹ , and O _HH ^1. _Are extracted, and spatial high-frequency signals O _LH ¹ , O _HL ¹ , and O _HH ¹ are extracted and output to the spatial high-frequency synthesis unit 3.

The spatial high-frequency synthesizer 3 receives the spatial high-frequency signals O _LH ¹ , O _HL ¹ , and O _HH ¹ from the orthogonal transform unit 2, synthesizes these signals, generates a spatial high-frequency synthesized signal S, and determines the number of band resolutions. Output to the unit 4 and the space region dividing unit 5.

The band resolution number determining unit 4 receives the spatial high frequency synthesized signal S from the spatial high frequency synthesizing unit 3 and orthogonally transforms the spatial high frequency synthesized signal S to obtain the spectral power of the spatial low frequency signal O _LL ¹ . Then, when the spectral power is smaller than a predetermined value, the band decomposition number determination unit 4 repeats orthogonal transformation, and determines the number of orthogonal transformations when the spectrum power is equal to or higher than the predetermined value as the band decomposition number k _max . Output to the selector 7. Thereby, it is possible to examine how much the original image O has the spectrum power by utilizing the fact that the spectrum power increases each time the orthogonal transformation is repeated.

The spatial area dividing unit 5, the space from the high-frequency synthesis unit 3 inputs the high spatial frequency combined signal S, enter the band decomposition number k _max from the selector 7 to be described later, the spatial high-frequency combined signal S to band decomposition number k _max The orthogonal low-frequency signals S _LL ^k = S _LL ^{1 to} S _LL ^kmax for each band are generated. Then, the spatial domain dividing unit 5 sets a flag 1 at an element position where the spectral power (element value) of each element position in the spatial low-frequency signal S _LL ^k = S _LL ^{1 to} S _LL ^kmax for each band is equal to or greater than a threshold value. Then, the spatial domain division signals BS _LL ^{1 to} BS _LL ^kmax for each band are generated. In addition, the spatial domain dividing unit 5 generates spatial high-frequency signals S _LH ¹ , S _HL ¹ , and S _HH ¹ by orthogonal transformation, and synthesizes these signals to generate a synthesized signal (maximum value signal) S _H ¹ . Then, the flag 1 is set at the element position where the spectrum power of each element position in the maximum value signal S _H ¹ is equal to or greater than the threshold value, and the spatial domain division signal BS is generated. Then, the spatial domain division unit 5 outputs the generated spatial domain division signals BS _LL ^{1 to} BS _LL ^kmax , BS to the outside as the spatial domain division signal BS ^k, as well as the spatial domain division signal BS ^k and the band decomposition number k _max. Is output to the edge region dividing unit 6.

The edge region division unit 6 receives the space region division signal BS ^k and the band resolution number k _max from the space region division unit 5 and outputs 4 regions of the space region division signal BS _LL ^k from k = k _max to k = 1. The product of the doubled signal and the spatial domain division signal BS _LL ^k−1 whose band decomposition floor is one floor lower is calculated to generate the edge domain division signal PrBS _LL ^k . When k = 1, the product of the signal obtained by quadrupling the area of the spatial domain division signal BS _LL ¹ and the spatial domain division signal BS is calculated. Thus, the edge region divided signal PrBS ^k can be obtained by recursively using the space region divided signal BS ^k for each band.

Further, the edge region dividing unit 6 removes isolated points from all element positions of the edge region divided signal PrBS _LL ^k by using an isolated point removal filter of 3 × 3 size. Thereby, the division | segmentation accuracy of an edge area | region can be improved.

Then, the edge region dividing unit 6 outputs the generated edge region divided signals PrBS _LL ^{1 to} PrBS _LL ^kmax (a signal generated by the product operation or a signal after isolated point removal) as an edge region divided signal PrBS ^k to the outside. Output.

The edge region dividing unit 6 inputs the band decomposition number k _max from the space region dividing unit 5, but may input it from the selecting unit 7.

The selection unit 7 inputs the band resolution number k _max determined from the band resolution number determination unit 4 and also inputs a fixed band resolution number k _max set in advance, and either one of the bands according to the setting by the user The decomposition number k _max is selected and output to the space region dividing unit 5.

(Orthogonal transform unit)
Next, the orthogonal transform unit 2 shown in FIG. 1 will be described in detail. FIG. 2 is a flowchart showing the processing of the orthogonal transformation unit 2, and FIG. 3 is a diagram for explaining the processing contents of the orthogonal transformation unit 2. Referring to FIG. 2, orthogonal transform unit 2 receives original image O (step S201), and performs orthogonal transform (DWPT calculation) on the original image O by a discrete wavelet packet (wavelet-packet) (step S202). ), A spatial low-frequency signal O _LL ^{1, a} horizontal high-frequency signal O _LH ¹ , a vertical high-frequency signal O _HL ^1, and an oblique high-frequency signal O _HH ¹ are generated. Then, the orthogonal transformation unit 2 extracts the horizontal high-frequency signal O _LH ¹ , the vertical high-frequency signal O _HL ^1, and the oblique high-frequency signal O _HH ¹ as the spatial high-frequency signals O _LH ¹ , O _HL ¹ , and O _HH ¹ (Step S203). And output to the spatial high-frequency synthesis unit 3 (step S204).

Referring to FIG. 3, the original image O on the xy axis is subjected to orthogonal transform using discrete wavelet packets, so that the spatial frequency signal on the horizontal frequency f _H axis and the vertical frequency f _v axis, that is, the spatial low frequency signal O _LL ¹ and spatial high-frequency signals O _LH ¹ , O _HL ¹ , and O _HH ¹ are generated. Then, spatial high-frequency signals O _LH ¹ , O _HL ¹ , and O _HH ¹ are extracted from the converted spatial frequency signals (shaded portions).

[Spatial high frequency synthesis unit]
Next, the spatial high frequency synthesis unit 3 shown in FIG. 1 will be described in detail. FIG. 4 is a flowchart showing the process of the spatial high frequency synthesizer 3, and FIG. 5 is a diagram for explaining the processing content of the spatial high frequency synthesizer 3. Referring to FIG. 4, spatial high frequency synthesis unit 3 inputs spatial high frequency signals O _LH ¹ , O _HL ¹ , and O _HH ¹ from orthogonal transform unit 2 (step S 401), and each spatial high frequency signal O of original image O. By performing maximum value calculation at arbitrary element positions (i, j) of _LH ¹ , O _HL ¹ , and O _HH ¹ , these signals are combined to generate a spatial high-frequency combined signal S (step S 402). Then, the spatial high frequency synthesis unit 3 outputs the spatial high frequency synthesis signal S to the band resolution number determination unit 4 and the spatial domain division unit 5 (step S403). For example, the maximum value calculation of the spatial high-frequency combined signal S (1,1) are three spatial frequency band ^{_{O LH 1, O HL 1,}} O HH 1 of the maximum value of the three elements of the same element position (1,1) calculating MAX becomes ^{_{{O LH 1 (1,1),}} O HL 1 (1,1), O HH 1 (1,1)}. That is, the spatial high frequency synthesis unit 3 generates the spatial high frequency synthesis signal S by the following equation.
S (i, j) = MAX {O LH 1 (i, j), O HL 1 (i, j), O HH 1 (i, j)}

Referring to FIG. 5, the maximum value calculation is performed on each element position of the spatial high-frequency signals O _LH ¹ , O _HL ¹ , and O _HH ¹ on the horizontal frequency f _H axis and the vertical frequency f _v axis. A spatial high-frequency synthesized signal S is generated (right hatched portion).

[Band resolution number determination unit]
Next, the band decomposition number determination unit 4 shown in FIG. 1 will be described in detail. FIG. 6 is a flowchart showing the processing of the band decomposition number determining unit 4, and FIG. 7 is a diagram for explaining the processing contents of the band decomposition number determining unit 4. Referring to FIG. 6, band decomposition number determination unit 4 receives spatial high frequency synthesis signal S from spatial high frequency synthesis unit 3 (step S601), sets k = 0 (step S602), and calculates k = k + 1. To calculate a new k (step S603). Then, the band decomposition number determination unit 4 performs orthogonal transformation (k-th order DWPT operation) on the spatial high-frequency synthesized signal S using a k-th order discrete wavelet packet to generate a spatial low-frequency signal S _LL ^k (step S604). .

Band decomposition number determination section 4, the spatial low-frequency signal S _LL ^k, the spectrum power at all elements positional element value alpha (i, j) the maximum MS _LL ^k of, calculating by the following equation (step S605).
MS _LL ^k = MAX {S _LL ^k (i, j); α (i, j)}

The band resolution number determination unit 4 determines whether or not the maximum value MS _LL ^k of all the element values for the spatial low frequency signal S _LL ^k is smaller than a predetermined threshold Thd (step S606), and the maximum value MS _LL ^k. Is smaller than the threshold Thd (step S606: Y), the process proceeds to step S603, k = k + 1 is calculated to calculate a new k, and the processes of steps S603 to S605 are performed. On the other hand, the band decomposition number determination unit 4 determines in step S606 that the maximum value MS _LL ^k is not smaller than the threshold value Thd (step S606: N), that is, the maximum value MS _LL ^k is equal to or greater than the threshold value Thd. when it is determined, to determine the k band decomposition number k _max (band decomposition number k _max = k) (step S607), and outputs the band decomposition number k _max to the selection unit 7 (step S608).

Here, according to k increases, becomes greater maximum MS _LL ^k of all element values of the spatial low-frequency signal S _LL ^k, finally, when it is determined that the maximum value MS _LL ^k is equal to or greater than the threshold value Thd Is determined to be the band resolution number k _max . Therefore, the determined band resolution number k _max indicates the extent to which the original image O has spectral power.

Referring to FIG. 7, orthogonal spatial transformation (k-th order DWPT calculation) using discrete wavelet packets is sequentially performed on the input spatial high-frequency synthesized signal S, and element values at all element positions are ^obtained for spatial low-frequency signal S _LL ^k. It shows that the maximum value MS _LL ^k of α (i, j) is calculated.

The band resolution number determination unit 4 may calculate an average value or a median value instead of calculating the maximum value MS _LL ^k of all the element values for the spatial low frequency signal O _LL ^k .

[Spatial domain division]
Next, the space area dividing unit 5 shown in FIG. 1 will be described in detail. FIG. 8 is a flowchart showing the processing of the spatial region dividing unit 5, and FIG. 9 is a diagram for explaining the processing contents of the spatial region dividing unit 5. Referring to FIG. 8, spatial region dividing unit 5 receives spatial high frequency synthesized signal S from spatial high frequency synthesizing unit 3, and also inputs band resolution number k _max from selecting unit 7 (step S801), and spatial high frequency synthesizing. The signal S is subjected to orthogonal transformation (1 to k _max order DWPT calculation) by discrete wavelet packets corresponding to the number of band resolutions k _max to obtain spatial low frequency signals S _LL ^k = S _LL ^{1 to} S _LL ^{kmax for} each band. Is generated (step S802). And it transfers to step S803 and step S804.

The spatial domain dividing unit 5 proceeds from step S802 and compares the spectral power (element value) at each element position in the spatial low-frequency signal S _LL ^k = S _LL ^{1 to} S _LL ^kmax for each band with a predetermined threshold value. Then, the flag 1 is set at the element position where the spectrum power is equal to or greater than the threshold, and the spatial domain division signals BS _LL ^{1 to} BS _LL ^kmax for each band are generated (step S803). Then, the process proceeds to step S806.

The spatial region dividing unit 5 proceeds from step S802, generates spatial high-frequency signals S _LH ¹ , S _HL ¹ , and S _HH ¹ by orthogonal transformation in step S802, and generates each spatial high-frequency signal S of the spatial high-frequency synthesized signal S. By performing maximum value calculation at arbitrary element positions (i, j) of _LH ¹ , S _HL ¹ , and S _HH ¹ , these signals are combined to generate a combined signal (maximum value signal) S _H ¹ ( Step S804). The process in step S804 is the same as the process in step S402 by the spatial high frequency synthesizer 3 shown in FIG. Then, the spatial domain dividing unit 5 compares the spectral power at each element position in the maximum value signal S _H ¹ with a predetermined threshold, sets a flag 1 at an element position where the spectral power is equal to or greater than the threshold, and spatial domain division A signal BS is generated (step S805). Then, the process proceeds to step S806.

The spatial domain dividing unit 5 shifts from step S803 and step S805, outputs the generated spatial domain divided signals BS _LL ^{1 to} BS _LL ^kmax , BS for each band to the outside as the spatial domain divided signal BS ^k , and performs edge processing The data is output to the area dividing unit 6 (step S806). Note that the spatial region dividing unit 5 also outputs the band resolution number k _max to the edge region dividing unit 6.

Referring to FIG. 9, spatial region division signal BS _LL ^{1 at} a predetermined position (portion of a square frame) of spatial low-frequency signal S _LL ¹ when k = 1 is as shown in the flag diagram, and k The resolution is higher than that of the spatial domain division signal BS _LL ^{2 at} the same predetermined position (the location of the square frame) of the spatial low frequency signal S _LL ² when = 2. Similarly, the spatial domain division signal BS _LL ^{2 at} a predetermined position of the spatial low frequency signal S _LL ² when k = ² is equal to the same predetermined position (square shape) of the spatial low frequency signal S _LL ³ when k = 3. The resolution is higher than that of the spatial domain division signal BS _LL ³ in the frame). That is, the frequency band of the original image O becomes narrower toward the center frequency between the horizontal sampling frequency f _SH and the vertical sampling frequency f _{SV of} the original image O and 0 which is the origin as k increases. Become.

[Edge region division part]
Next, the edge area dividing unit 6 shown in FIG. 1 will be described in detail. FIG. 10 is a flowchart showing processing of the edge region dividing unit 6, and FIG. 11 is a diagram for explaining processing contents of the edge region dividing unit 6. Referring to FIG. 10, edge region division unit 6 receives spatial region division signal BS ^k (BS _LL ^{1 to} BS _LL ^kmax , BS) and band resolution number k _max from spatial region division unit 5 (step S1001). Then, the band resolution number k _max is set to k (k = k _max ) (step S1002).

The edge region dividing unit 6 enlarges the region of the spatial region division signal BS _LL ^k to 4 times the area (2 times horizontal and 2 times vertical), and sets the region as 4BS _LL ^k (step S1003). And the area of the region of the spatial region division signal BS _LL ^k, is compared with the area of the spatial region division signal BS _LL ^k-1 region in the one-dimensional low k-1 floor band than this band, the spatial region division signal BS _LL ^k-1 is four times the space domain division signal BS _LL ^k . Therefore, in step S1004, which will be described later, the arbitrary position of the spatial domain division signal BS _LL ^{k-1 and} the position of the spatial domain division signal BS _LL ^k corresponding thereto correspond to a four-fold area difference.

The edge region dividing unit 6 calculates the product of 4BS _LL ^k and BS _LL ^k-1 by the following formula, and generates an edge region divided signal PrBS _LL ^k that is the product set (step S1004).
PrBS _LL ^k = 4BS _LL ^k ∩BS _LL ^k-1

The edge region dividing unit 6 removes isolated points from all element positions of the edge region divided signal PrBS _LL ^k using an isolated point removal filter of 3 × 3 size (step S1005). Specifically, the edge region dividing unit 6 uses an isolated point removal filter to determine whether or not the number of flags 1 is greater than or equal to θ in nine regions of 3 × 3 size. If it is determined, the value of the center position of 3 × 3 size is set to 1, and if it is determined that it does not exist, the value of the center position is set to 0. θ is a predetermined threshold value. Thereby, the isolated flag 1 can be removed in the neighboring region.

The edge region dividing unit 6 calculates k = k−1 to calculate a new k (step S1006), determines whether k = 1 (step S1007), and determines that k = 1 is not satisfied. If so (step S1007: N), the process proceeds to step S1003, and the processes of steps S1003 to S1006 are performed. As a result, edge region division signals PrBS _LL ^{2 to} PrBS _LL ^{kmax for} each band are generated.

On the other hand, when it is determined in step S1007 that k = 1 (step S1007: Y), the edge region dividing unit 6 calculates the product of 4BS _LL ¹ and BS by the following formula, The edge region division signal PrBS _LL ¹ is generated (step S1008).
PrBS _LL ¹ = 4BS _LL ¹ ∩BS

The edge region dividing unit 6 outputs the generated edge region divided signals PrBS _LL ^{1 to} PrBS _LL ^kmax for each band as edge region divided signals PrBS ^k to the outside (step S1009). Depending on the frequency band to be analyzed, the processing from step S1003 to step S1007 may not be repeated until k = 1.

In general, the sharp edges in the original image O contain spectral power in a wide band while being gradually reduced in the frequency band from the low frequency region to the high frequency region, and are distributed in isolated points. do not do. By utilizing such edge characteristics, the edge region division signals PrBS _LL ^{1 to} PrBS _LL ^kmax can be generated by the processing shown in FIG.

The edge region dividing unit 6 outputs the edge region divided signals PrBS _LL ^{1 to} PrBS _LL ^kmax generated by the product operation in step S1004 to the outside as the edge region divided signal PrBS ^k without using the isolated point removal filter. Alternatively, as shown in FIG. 10, using the isolated point removal filter, the edge region divided signals PrBS _LL ^{1 to} PrBS _LL ^kmax after the isolated point removal are sent to the outside as edge region divided signals PrBS ^k . You may make it output.

FIG. 11 shows the edge region split signal PrBS _LL ² (= 4BS _LL ² ∩BS _LL ¹ ) when k = 2, and the isolated points are removed by the 3 × 3 size isolated point removal filter. The edge region division signal PrBS _LL ² is generated. The area of flag 1 of the edge area division signal PrBS _LL ² corresponds to 4 × 4 times the size of the original image O.

As described above, according to the image region dividing apparatus 1 according to the embodiment of the present invention, the orthogonal transformation unit 2 performs the orthogonal transformation of the original image O to generate the spatial high-frequency signals O _LH ¹ , O _HL ¹ , and O _HH ¹ . The spatial high frequency synthesizer 3 extracts and synthesizes these signals by the maximum value calculation of the same element position in the spatial high frequency signals O _LH ¹ , O _HL ¹ , and O _HH ¹ to generate a spatial high frequency synthesized signal S. The decomposition number determination unit 4 orthogonally transforms the spatial high-frequency synthesized signal S to obtain the spectral power of the spatial low-frequency signal O _LL ¹ , and performs orthogonal transformation when the spectral power is smaller than a predetermined value by calculating the threshold of the spectral power. Repeatedly, the number of orthogonal transforms when the spectrum power is equal to or higher than a predetermined value is determined as the band resolution number k _max . Further, the spatial domain dividing unit 5 performs orthogonal transformation of the spatial high frequency synthesized signal S to generate spatial low frequency signals S _LL ^k = S _LL ^{1 to} S _LL ^kmax , and the spatial low frequency signal S _LL ^k = S _LL ^1. flags 1 to element position where spectral power is equal to or greater than the threshold value of each element position in to S _LL ^kmax, and to generate a spatial region division signal BS _LL ¹ to BS _LL ^kmax in the respective bands. Then, the spatial domain dividing unit 5 generates spatial high-frequency signals S _LH ¹ , S _HL ¹ , and S _HH ¹ by orthogonal transformation, and synthesizes these signals to generate a synthesized signal (maximum value signal) S _H ^1. The flag 1 is set at the element position where the spectrum power of each element position in the maximum value signal S _H ¹ is equal to or greater than the threshold value, and the spatial domain division signal BS is generated.

The edge area dividing unit 6, k = from k _max to k = 1, calculates the product between the signal and the spatial regions four times an area of the spatial region division signal BS _LL ^k division signal BS _LL ^k-1 , K = 1, the product of the signal obtained by quadrupling the area of the spatial domain division signal BS _LL ¹ and the spatial domain division signal BS is calculated, and the edge area division signals PrBS _LL ^{1 to} PrBS _LL ^{kmax for} each band are calculated. Was generated.

  As a result, the original image O can be divided into spatial regions for each band, and image quality can be improved by using the spatial region division signal or the edge region division signal. For example, in a frequency reconfigurable super-resolution device, high-quality super-resolution processing can be performed by using a Gaussian filter with an optimal dispersion value for the area indicated by the spatial domain division signal or edge domain division signal for each band. It can be performed. Details will be described later.

  Note that a normal computer can be used as the hardware configuration of the image area dividing device 1 according to the embodiment of the present invention. The image area dividing device 1 includes a computer having a volatile storage medium such as a CPU and a RAM, a non-volatile storage medium such as a ROM, an interface, and the like. The functions of the orthogonal transform unit 2, the spatial high frequency synthesis unit 3, the band resolution number determination unit 4, the spatial region division unit 5, the edge region division unit 6, and the selection unit 7 included in the image region division apparatus 1 are the functions described above. Each is realized by causing the CPU to execute the described program. These programs can also be stored and distributed in a storage medium such as a magnetic disk (floppy (registered trademark) disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), semiconductor memory, or the like.

[Example 1]
Next, a specific embodiment of the image area dividing device 1 shown in FIG. 1 will be described. First, Example 1 will be described. Example 1, without using the band decomposition number k _max determined by the band decomposition number determination section 4, with a band decomposition number k _max of fixed preset, the spatial region of each band division signal BS ^k (BS _LL ^{1 to} BS _LL ^kmax (BS) is generated and output. FIG. 12 is a flowchart illustrating the processing of the first embodiment.

The orthogonal transformation unit 2 of the image region dividing apparatus 1 inputs the original image O (step S1201), performs orthogonal transformation on the original image O, and extracts spatial high-frequency signals O _LH ¹ , O _HL ¹ , and O _HH ¹ . (Step S1202). Then, the spatial high frequency synthesizer 3 synthesizes the spatial high frequency signals O _LH ¹ , O _HL ¹ , and O _HH ¹ to generate a spatial high frequency synthesized signal S (step S1203).

The spatial domain dividing unit 5 performs orthogonal transformation of the spatial high-frequency synthesized signal S by a predetermined fixed number of band resolutions k _max , and spatial low-frequency signals S _LL ^k = S _LL ^{1 to} S _{LL for} each band. ^kmax is generated, flag 1 is set by calculating the threshold value of the spectral power at each element position, and spatial domain divided signals BS _LL ^{1 to} BS _LL ^kmax for each band are generated, and spatial high-frequency signal S _LH ¹ , S _HL ¹ and S _HH ¹ are generated, and a spatial domain division signal BS is generated by the same threshold calculation (step S1204). Then, the spatial domain division unit 5 outputs the spatial domain division signals BS _LL ^{1 to} BS _LL ^kmax , BS for each band as the spatial domain division signal BS ^k (step S1205).

  Thus, according to the first embodiment, the original image O can be spatially divided for each band, and the spatial domain division signal for each band is applied to, for example, a frequency reconfiguration type super-resolution apparatus. Improvement in image quality can be realized.

[Example 2]
Next, Example 2 will be described. Example 2 generates and outputs a spatial domain division signal BS ^k (BS _LL ^{1 to} BS _LL ^kmax , BS) for each band using the band decomposition number k _max determined by the band decomposition number determination unit 4. It is. FIG. 13 is a flowchart illustrating the processing of the second embodiment. In FIG. 13, steps S1301 to S1303, S1305, and S1306 are the same as steps S1201 to S1205 of the first embodiment shown in FIG.

The band decomposition number determination unit 4 of the image region dividing device 1 orthogonally transforms the spatial high frequency synthesized signal S to obtain the spectral power of the spatial low frequency signal O _LL ¹ , and the spectral power is equal to or greater than a predetermined value by calculating the threshold of the spectral power. The number of orthogonal transforms in the case of ## _EQU2 ## is determined as the band resolution number k _max (step S1304).

The spatial domain dividing unit 5 performs orthogonal transformation of the spatial high-frequency synthesized signal S by the number of band resolutions k _max determined in step S1304, and spatial low-frequency signals S _LL ^k = S _LL ^{1 to} S _{LL for} each band. _LL ^kmax is generated, and the same processing as step S1204 of the first embodiment shown in FIG. 12 is performed (step S1305).

As described above, according to the second embodiment, the image quality can be improved as in the first embodiment. In addition, the band resolution number _kmax is determined each time based on the spectrum power of the signal by the band resolution number determination unit 4, and is particularly effective for an original image that changes with time such as a moving image. .

Example 3
Next, Example 3 will be described. Example 3 without using the band decomposition number k _max determined by the band decomposition number determination section 4, with a band decomposition number k _max of fixed preset, the spatial region of each band division signal BS ^k (BS _{In this example} , edge region division signals PrBS ^k (PrBS _LL ^{1 to} PrBS _LL ^kmax ) for each band are generated from _LL ^{1 to} BS _LL ^kmax (BS) and output. FIG. 14 is a flowchart illustrating the processing of the third embodiment. 14, step S1401 to step S1404 are the same as step S1201 to step S1204 of the first embodiment shown in FIG.

The edge region dividing unit 6 of the image region dividing device 1 performs a quadrupling of the region of the spatial region division signal BS _LL ^k from k = k _max to k = 1 and the spatial region division signal BS _LL ^k−1 . When the product is calculated and k = 1, the product of the signal obtained by quadrupling the area of the spatial domain division signal BS _LL ¹ and the spatial domain division signal BS is calculated, and the edge area division signal PrBS _LL ^{1 for} each band is calculated. It generates ~PrBS _LL ^kmax (step S1405), and outputs as the edge region division signal PRBS ^k (step S1406).

  As described above, according to the third embodiment, the image quality can be improved as in the first embodiment. In particular, the image quality can be further improved for the edge component included in the original image.

Example 4
Next, Example 4 will be described. In the fourth embodiment, the band decomposition number k _max determined by the band decomposition number determination unit 4 is used, and the edge area division for each band is performed from the spatial area division signal BS ^k (BS _LL ^{1 to} BS _LL ^kmax , BS) for each band. In this example, signals PrBS ^k (PrBS _LL ^{1 to} PrBS _LL ^kmax ) are generated and output. FIG. 15 is a flowchart illustrating the processing of the fourth embodiment. In FIG. 15, steps S1501 to S1505 are the same as steps S1301 to S1305 of the second embodiment shown in FIG. 13, and steps S1506 and S1507 are the same as steps S1405 and S1505 of the third embodiment shown in FIG. 14. Since this is the same as S1406, description thereof is omitted.

As described above, according to the fourth embodiment, the image quality can be improved as in the first embodiment. In addition, the band resolution number _kmax is determined each time based on the spectrum power of the signal by the band resolution number determination unit 4, and is particularly effective for an original image that changes with time such as a moving image. . In particular, the image quality can be further improved for the edge component included in the original image.

[Example when applied to a frequency reconfigurable super-resolution device]
Next, an example in which the image region dividing device 1 shown in FIG. 1 is applied to a frequency reconstruction type super-resolution device will be described. In this example, the spatial region divided signal BS ^k (BS _LL ^{1 to} BS _LL ^kmax , BS) or the edge region divided signal PrBS ^k (PrBS _LL ^{1 to} PrBS _LL ^kmax ) output by the image region dividing apparatus 1 is used. A Gaussian filter having an optimum variance value is applied only to the region of flag 1 indicated by this signal. Since the region of flag 1 is a region where the spectral power is equal to or greater than the threshold value, high-resolution super-resolution processing can be performed.

FIG. 16 is a block diagram illustrating a configuration of an image encoding device to which the image region dividing device 1 illustrated in FIG. 1 is applied. The image encoding device 100 performs down-sampling by performing orthogonal transformation (for example, wavelet decomposition) on the original image F, and outputs a low-resolution down-sampled image CA ⁽ⁿ⁾ (n: decomposition rank), Based on the downsampling images CA ⁽ⁿ⁾ , CA ^{(n−1) and the} like obtained by orthogonal transformation in the downsampling unit 10, variance values σ _CH ⁽ⁿ⁾ , σ of a Gaussian filter used in the upsampling process described later. _CV ⁽ⁿ⁾ , σ _CD ⁽ⁿ⁾ (subscripts CH, CV, CD indicate high-frequency components in the horizontal, vertical, and diagonal directions) and orthogonal transform by the downsampling unit 10 and an encoder 12 for encoding the downsampled picture CA ⁽ⁿ⁾ obtained, encoded downsampled at the encoding unit 12 Includes a transmission unit 13 for transmitting image enca ⁽ⁿ⁾ and the variance of the Gaussian filter determined by the variance determining section ^{_{11 σ CH (n), σ}} CV (n), σ CD and ⁽ⁿ⁾ as a transmission signal, the Configured. Here, the spatial region division signal BS ^k (BS _LL ^{1 to} BS _LL ^kmax , BS) or the edge region division signal PrBS ^k (PrBS _LL ^{1 to} PrBS _LL ^kmax ) output by the image region division device 1 is determined as a variance value. Assuming that the filtering process is performed on the element value of the flag 1 region indicated by this signal, which is used in the unit 11, the variance values σ _CH ⁽ⁿ⁾ , σ _CV ⁽ⁿ⁾ , σ of the Gaussian filter _CD ⁽ⁿ⁾ is determined.

FIG. 17 is a block diagram illustrating a configuration of an image decoding device to which the image region dividing device 1 illustrated in FIG. 1 is applied. The image decoding apparatus 200 receives the down-sampled image EnCA ⁽ⁿ⁾ and the Gaussian filter variance values σ _CH ⁽ⁿ⁾ , σ _CV ⁽ⁿ⁾ , σ _CD ⁽ⁿ⁾ transmitted from the image encoding apparatus 1 described above. Receiving unit 20, decoding unit 21 decoding the downsampled image EnCA ⁽ⁿ⁾ received by the receiving unit 20, and variance values σ _CH ⁽ⁿ⁾ , σ _CV ^{(n of} the Gaussian filter received by the receiving unit 20 ⁾ , Σ _CD ⁽ⁿ⁾ , and an upsampling unit 22 for upsampling the downsampled image EnDeCA ⁽ⁿ⁾ decoded by the decoding unit 21. Here, the spatial region division signal BS ^k (BS _LL ^{1 to} BS _LL ^kmax , BS) or the edge region division signal PrBS ^k (PrBS _LL ^{1 to} PrBS _LL ^kmax ) output by the image region division device 1 is an upsampling unit. 22, filtering processing is performed on only the element values in the region of flag 1 indicated by this signal using the variance values σ _CH ⁽ⁿ⁾ , σ _CV ⁽ⁿ⁾ , σ _CD ⁽ⁿ⁾ of the Gaussian filter.

FIG. 18 is a diagram showing an overview of the variance value determination process in the variance value determination unit 11 of the image encoding device 100 shown in FIG. First, the dispersion value determining unit 11 performs first-order discrete wavelet decomposition on the low-frequency domain component CA ⁽ⁿ⁾ of the decomposition rank n, and each high-frequency domain component CH ^{(n + 1)} , horizontal, vertical, and diagonal directions in the decomposition rank n + 1. CV ^{(n + 1)} and CD ^{(n + 1)} are obtained (step S1). Each of the high-frequency region components CH ^{(n + 1)} , CV ^{(n + 1)} , and CD ^{(n + 1) in} the horizontal, vertical, and diagonal directions is used as the spatial low-frequency region component, and the zero matrix of the same size is used as the spatial high-frequency region component. ExCH ⁽ⁿ⁾ , ExCV ⁽ⁿ⁾ , and ExCD ⁽ⁿ⁾ are obtained by enlarging the horizontal and vertical directions twice by the discrete wavelet reconstruction (step S2).

The following formula is obtained by reconstructing the first-order discrete wavelet using CH ^{(n + 1)} as a spatial low frequency region component and a zero matrix of the same size as a spatial high frequency region component. 0 indicates a zero matrix.
ExCH ⁽ⁿ⁾ = IDWT ⁽¹⁾ (CH ^{(n + 1)} , 0, 0, 0, wavelet_n)

The dispersion value determination unit 11 obtains ExCH ⁽ⁿ⁾ , ExCV ⁽ⁿ⁾ , and ExCD ⁽ⁿ⁾ expanded twice in the horizontal and vertical directions, and then outputs the spatial region division signal BS ^k output from the image region division device 1. _{^{(BS LL 1 ~BS LL kmax,}} BS) or edge region division signal _{^{PrBS k (PrBS LL 1 ~PrBS LL}} kmax) is a region of a flag 1 indicating ^{ExCH (n), ExCV (n} ), to EXCD ⁽ⁿ⁾ , Filtering by the Gaussian filter using the variance values σ _CH ⁽ⁿ⁾ , σ _CV ⁽ⁿ⁾ , σ _CD ⁽ⁿ⁾ of the Gaussian filter is performed to obtain GExCH ⁽ⁿ⁾ , GExCV ⁽ⁿ⁾ , GExCD ⁽ⁿ⁾ ( Step S3). In this case, arbitrary values or empirical values are used as initial values for the variance values σ _CH ⁽ⁿ⁾ , σ _CV ⁽ⁿ⁾ , and σ _CD ⁽ⁿ⁾ of the Gaussian filter. For example, if the variance values σ _CH ⁽ⁿ⁾ , σ _CV ⁽ⁿ⁾ , and σ _CD ⁽ⁿ⁾ of the Gaussian filter are changed in increments of 0.1 to 0.1, the initial value is, for example, σ _CH ^{( n)} = 0.1, σ _CV ⁽ⁿ⁾ = 0.1, and σ _CD ⁽ⁿ⁾ = 0.1.

After performing filtering using a Gaussian filter to obtain GExCH ⁽ⁿ⁾ , GExCV ⁽ⁿ⁾ , GExCD ⁽ⁿ⁾ , the low frequency region component CA ⁽ⁿ⁾ is used as the spatial low frequency region component, and GExCH ⁽ⁿ⁾ , GExCV ^{( n)} , GExCD ⁽ⁿ⁾ is used as a spatial high-frequency region component, and doubled horizontally and vertically by first-order discrete wavelet reconstruction to obtain GExCA ^(n-1) (step S4).

After obtaining GExCA ^(n-1) , a difference value from CA ^(n-1) is calculated. If the current difference value is smaller than the previous difference value, the current difference value and the variance values σ _CH ⁽ⁿ⁾ , σ _CV ⁽ⁿ⁾ , σ _CD ⁽ⁿ⁾ of the Gaussian filter used this time are stored (step S5). ). The variance value determining unit 11 has a memory (not shown), and in this memory, the current difference value and the variance values σ _CH ⁽ⁿ⁾ , σ _CV ⁽ⁿ⁾ , σ _CD ^{(n) of} the Gaussian filter used this time. Remember. Since there is no previous difference value at the first time, the difference value calculated this time and the variance values σ _CH ⁽ⁿ⁾ , σ _CV ⁽ⁿ⁾ , σ _CD ⁽ⁿ⁾ of the Gaussian filter used this time are stored as they are. Further, as an image evaluation method, there is a mean square error (MSE), and this evaluation method may be used.

The processes in steps S1 to S5 are repeated while changing the variance values σ _CH ⁽ⁿ⁾ , σ _CV ⁽ⁿ⁾ , σ _CD ⁽ⁿ⁾ of the Gaussian filter, and GExCA ⁽ⁿ⁻¹⁾ and CA ⁽ⁿ⁻ The variance values σ _CH ⁽ⁿ⁾ , σ _CV ⁽ⁿ⁾ , and σ _CD ⁽ⁿ⁾ of the Gaussian filter when the difference value from ¹⁾ is the smallest are determined. The determined variance values σ _CH ⁽ⁿ⁾ , σ _CV ⁽ⁿ⁾ and σ _CD ⁽ⁿ⁾ of the Gaussian filter are transmitted to the image decoding apparatus 200 together with the down-sampled image EnCA ⁽ⁿ⁾ . In the image decoding apparatus 200, the variance values σ _CH ⁽ⁿ⁾ , σ _CV ⁽ⁿ⁾ , and σ _CD ⁽ⁿ⁾ of the Gaussian filter are received and used, so that the image is closest to the original image from the down-sampled image EnCA ^(n). Images can be played back.

Also in the upsampling unit 22 of the image decoding device 200 shown in FIG. 17, the spatial region division signal BS ^k (BS _LL ^{1 to} BS _LL ^kmax , BS) or the edge region division signal output from the image region division device 1. Using the received Gaussian filter variance values σ _CH ⁽ⁿ⁾ , σ _CV ⁽ⁿ⁾ , σ _CD ⁽ⁿ⁾ for the flag 1 region indicated by PrBS ^k (PrBS _LL ^{1 to} PrBS _LL ^kmax ), a Gaussian filter is used. Filtering processing is performed.

As described above, according to the image encoding device 100 and the image decoding device 200 that are frequency reconfigurable super-resolution devices, the spatial domain division signal BS ^k (BS _LL ^{1 to} BS _LL ^kmax , BS) for each band. Alternatively, high-resolution super-resolution processing is performed by applying a Gaussian filter having an optimal dispersion value to the region of flag 1 indicated by the edge region divided signal PrBS ^k (PrBS _LL ^{1 to} PrBS _LL ^kmax ) for each band. be able to.

  The present invention has been described with reference to the embodiment. However, the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the technical idea thereof. The present invention is effective for all image processing that performs spatial region division for each band. Further, the present invention can be applied to various image processing in which spatial region division for each band is effective, such as image reduction, encoding, super-resolution, and the like.

  Further, in the image region segmentation apparatus 1 shown in FIG. 1, orthogonal transformation (DWPT calculation) is performed using discrete wavelet packets (wavelet-packets), but other orthogonal transformations may be performed. For example, discrete wavelet transform, discrete Fourier transform, and discrete cosine transform may be performed. Further, by using the same type of transform as the orthogonal transform used in the image region dividing device 1 for the image encoding device 100 and the image decoding device 200 shown in FIGS. Processing can be performed.

DESCRIPTION OF SYMBOLS 1 Image area | region dividing device 2 Orthogonal transformation part 3 Spatial high frequency synthetic | combination part 4 Band decomposition | disassembly number determination part 5 Spatial area division | segmentation part 6 Edge area | region division | segmentation part 7 Selection part 10 Downsampling part 11 Variance value determination part 12 Encoding part 13 Transmission part 20 Reception unit 21 Decoding unit 22 Upsampling unit 100 Image encoding device 200 Image decoding device

Claims (7)

An orthogonal transform unit that orthogonally transforms the signal of the original image and generates a spatial high-frequency signal;
A spatial high-frequency synthesis unit that synthesizes the spatial high-frequency signal generated by the orthogonal transform unit and generates a spatial high-frequency synthesis signal;
The spatial high frequency synthesis signal generated by the spatial high frequency synthesis unit is subjected to orthogonal transform with a preset number of band resolutions, and threshold processing is performed on the element value of the signal after the orthogonal transformation to divide the spatial region. A spatial domain division unit for generating a spatial domain division signal for each band;
An image area dividing apparatus comprising: The image region dividing device according to claim 1,
Instead of the preset number of band resolutions, a band for orthogonally transforming the spatial high-frequency synthesized signal generated by the spatial high-frequency synthesis unit and determining the number of band resolutions based on the spectrum power of the signal after the orthogonal transformation It has a decomposition number determination unit,
The spatial domain division unit performs orthogonal transform of the band resolution number determined by the band resolution number determination unit on the spatial high-frequency synthesized signal to generate a spatial domain division signal for each band. Image area dividing device. The image region dividing device according to claim 1,
Furthermore, the correspondence between the two spatial domain division signals between the spatial domain division signal for each band generated by the spatial domain division unit and the spatial domain division signal whose band decomposition floor is one dimension lower than the band. For each position, a product operation of a predetermined number of band resolutions is performed, and threshold processing is performed on the element value of the signal after the product operation to divide the edge region, thereby generating an edge region divided signal for each band. An image area dividing apparatus comprising: an edge area dividing unit. The image area dividing device according to claim 2,
Furthermore, the correspondence between the two spatial domain division signals between the spatial domain division signal for each band generated by the spatial domain division unit and the spatial domain division signal having a one-dimensional smaller band decomposition level than the band. For each position, the product of the band decomposition number determined by the band decomposition number determination unit is performed, the edge value is divided by performing threshold processing on the element value of the signal after the product calculation, and the edge for each band An image area dividing apparatus comprising: an edge area dividing unit that generates an area dividing signal. The image area dividing device according to claim 3 or 4,
The image area dividing apparatus according to claim 1, wherein the edge area dividing unit removes isolated points in the edge area divided signal using a filter with respect to the generated edge area divided signal for each band. In the image area dividing device according to any one of claims 1 to 5,
An image region segmentation apparatus, wherein the orthogonal transform is discrete wavelet packet transform, discrete wavelet transform, discrete Fourier transform, or discrete cosine transform.   An image region dividing program for causing a computer to function as the image region dividing device according to any one of claims 1 to 6.

Images