JP2009527978A - Image encoding and decoding method and apparatus - Google Patents

Abstract

A method and apparatus for encoding wavelet coefficients in consideration of characteristics of the human visual system in the spatial domain and the frequency domain are provided. The product of the spatial weight generated using the local bandwidth normalized with respect to the visual axis of the human eye and the frequency weight generated using the subband error sensitivity in the wavelet domain. By calculating and generating visual weights, and encoding and transmitting wavelet coefficients according to an encoding order determined based on the generated visual weights, it provides improved image quality even at low channel capacity. sell.

Description

  The present invention relates to an image encoding and decoding method and apparatus, and more particularly, using a visual weight determined in consideration of a human time system in a frequency domain and a spatial domain, The present invention relates to an image encoding method and apparatus, and a decoding method and apparatus for encoding a wavelet transformed image.

  With increasing channel capacity of wide area wireless networks, many attempts have been made to improve the image quality of images and applications serviced within the wireless network area. However, due to the variable nature of channel capacity, sufficient bandwidth is not guaranteed to transmit any overall traffic. Therefore, various object-oriented algorithms and layered algorithms have been proposed to allocate additional coding resources to interested individuals and regions so that they can be efficiently adapted to variable channels.

  Recently, various wavelet-based image compression algorithms have been proposed. Conventional wavelet-based image compression algorithms use the correlation between coefficients in each band. Known typical wavelet coefficient compression methods include an EZW (Embedded image coding using Zerotrees of Wavelet coefficients) algorithm and an SPIHT (Set Partitioning In Hierarchical Trees) algorithm.

  The hierarchical structure of wavelet decomposition has an advantageous structure for obtaining global features from an image sequence. In other words, the wavelet domain has a hierarchical structure that can simultaneously interpret the spatial and frequency domain information, and therefore has a good structure for grasping the overall image characteristics from the information of one subband. In addition, the wavelet region basically has multi-resolution characteristics, so that it is advantageous at the time of data transmission of a gradual image encoder.

  On the other hand, the spatial distribution of the optic nerve distributed in the human retina has non-linear characteristics. That is, the optic nerve is most densely centered on the phobia, and the density of the optic nerve decreases rapidly as the distance from the phobia increases. Therefore, the local visual frequency bandwidth perceived by the optic nerve decreases sharply with increasing distance from the phobia.

  Prior art image encoders take into account the characteristics of the human visual system (HVS: human visual system) and increase the channel transmission rate of visually important information, thereby improving the subjective image quality. However, it does not present a specific reference value for selecting visually important information in consideration of human HVS frequency and spatial visual resolution.

  The present invention has been devised to solve the above problems, and sets the visual weight value of the wavelet transform coefficient in consideration of the human visual sensitivity in the spatial domain and the frequency domain, An image encoding method and apparatus capable of improving the image quality of an encoded image even when the channel capacity is small by determining a progressive encoding order of wavelet transform coefficients based on this visual weight value, An object is to provide a decoding method and apparatus.

  In order to solve the above problems, an image encoding method according to the present invention performs a wavelet transform on an input image to generate wavelet transform coefficients, and considers human visual sensitivity in the spatial domain and frequency domain. Generating a visual weight value of the wavelet transform coefficient; determining an encoding order of the wavelet transform coefficient using the generated visual weight value; and the determined encoding order And encoding the wavelet transform coefficients.

  An image encoding apparatus according to the present invention includes a transform unit that performs wavelet transform on an input image to generate a wavelet transform coefficient, and the visual sensitivity of the wavelet transform coefficient in consideration of human visual sensitivity in a spatial domain and a frequency domain. A visual weight value generation unit that generates a dynamic weight value, an encoding order determination unit that determines an encoding order of the wavelet transform coefficients using the generated visual weight value, and the determined code And a sequential wavelet coefficient encoding unit that encodes the wavelet transform coefficients according to a conversion order.

  An image decoding method according to the present invention includes a step of decoding wavelet transform coefficients encoded according to a magnitude order of visual weights generated in consideration of human visual sensitivity in a spatial domain and a frequency domain. And performing inverse wavelet transform on the decoded wavelet transform coefficients, and restoring an image using the coefficients of each subband subjected to the inverse wavelet transform.

  An image decoding apparatus according to the present invention sequentially decodes wavelet transform coefficients encoded according to the order of magnitudes of visual weights generated in consideration of human visual sensitivity in a spatial domain and a frequency domain. A wavelet coefficient decoding unit, an inverse transform unit that performs inverse wavelet transform on the decoded wavelet transform coefficient, an image restoration unit that restores an image using the coefficient of each subband subjected to the inverse wavelet transform, It is characterized by providing.

  According to the present invention, wavelet coefficients are sequentially encoded and transmitted based on a visual weight generated in consideration of human HVS in the frequency domain and the spatial domain, thereby further improving at a low channel capacity. It is possible to encode and transmit an image having a high quality.

  Hereinafter, in order to assist in understanding the visual entropy used for setting the visual weight value of the wavelet transform coefficient in consideration of the human visual sensitivity in the spatial domain and the frequency domain in the present invention, After describing the concept, visual entropy in the spatial domain, and visual entropy in the wavelet domain, the image coding and decoding method and apparatus of the present invention will be described.

( Definition of entropy)
During image coding, the scalar quantizer Q quantizes a random variable X having real values.

Is generated. The value of X is _[y -, y _+] present in the range _{of, [y} -, y _+] If Wakere a value in the range of the M-number of intervals, each _{interval, [y m-1, y} m ] (1 ≦ m ≦ M, y ₀ = y ₋ , y _M = y ₊ ). At this time, if x∈ [y _m−1 , y _m ], Q (x) = x _m . Within each interval divided into M, the probability pm of the value of the _mth range is expressed as

Assume that In this case, the quantized random variable

Entropy

Is the following formula

It is as follows. here,

Is the quantized random variable

Means the minimum value of the average number of bits required for encoding the value of.

In general, for a random variable X, if the probability density function (PDF) of the random variable X is P (x), the differential entropy H _d (x) of the random variable X is given by It is defined as equation (1).

If the quantization error generated in the scalar quantizer Q is D, the following equation

Is known to hold. When a uniform quantizer is used as the scalar quantizer Q, the equal sign holds in the above equation. That is, the quantized random variable

A uniform quantizer can be used to minimize the average number of bits required to encode the value of. When the size of each quantization bin used in a uniform quantizer and ^{Δ, D = (Δ 2/} 12) , and the minimum average bit rate R _x is

It becomes.

If the coefficient a [m] transformed for the signal A and the orthogonal normal basic function g _m are used,

Assuming that N is the number of samples of signal A in the transform domain, the quantized coefficient of a [m] is

And the entropy is

It becomes. When the total quantization error of the quantized transform coefficient is D, the optimum bit allocation is the total number of bits R required for encoding the quantized transform coefficient a [m], that is,

(R _m is, a number of bits required to encode the [m]) is intended to minimize. Average number of bits generated per sample

Then, D _m which is a quantization error of each transform coefficient a [m] using a Lagrange multiplier, ie,

Is the average number of bits generated per sample if is homologous between the transform coefficients

Has a minimum value. Mean differential entropy

Is the average differential entropy of the N sampled transform coefficients, ie,

Is defined. If the signal A is a Gaussian random variable, the variance of the wavelet coefficient a [m] is

Then, the entropy of the Gaussian random variable is as the following formula (2).

If a [m] is a Laplace random variable, the entropy of the Laplace random variable is as in the following equation (3).

( Visual entropy in the spatial domain)
As described above, since the human eye samples information through a non-linear distribution of optic nerves existing in the retina, the human eye obtains non-linear visual data. Therefore, the human eye obtains data at a non-linear rate based on the fixation point, and the image connected to the retinal optic nerve is an image in which high frequency components are removed non-linearly through a non-linear sampling process of the optic nerve. It becomes. Thus, an image recognized by the human optic nerve is defined as a forbidden image.

  In general, a human fixation point may be one point or various points, and may be one object or many objects. In addition, the human fixation point may be a specific region of the image depending on the image content and application.

  FIG. 1A and FIG. 1B show an original image a (x) and a forbidden image, respectively, in order to compare the original image and the forbidden image.

It is a figure which shows an example.

  Referring to FIGS. 1A and 1B, the region of interest of the observer is assumed to be a tennis player. In this case, the forbidden area is a peripheral area centered on the tennis player. As shown in FIG. 1B, due to the non-linear characteristics of the human optic nerve, the resolution of the image recognized by the human optic nerve has a symmetrical pattern around the retina and decays exponentially. A new coordinate system obtained with such a nonlinear mapping structure is a curved coordinate system.

It is defined as

  2A and 2B show the original image a (x) and the forbidden image of FIGS. 1A and 1B, respectively.

The curved coordinate system

Original image mapped to

And fobied

FIG. That is,

Are the original image a (x) and the forbidden image of FIGS. 1A and 1B.

Is an image obtained by performing coordinate transformation in a curved coordinate system in consideration of the structure of the human eye having a concave curved surface form.

  Comparing FIG. 2A and FIG. 2B, the mapped original image that is actually recognized by the human optic nerve

And mapped forbidden images

Is almost visually identical.

  In FIG. 1A, the spatial area of the original image

Assuming that the area corresponding to the original image in the Cartesian coordinate system is A _o , the mapped original image in the curved coordinate system shown in FIGS. 2A and 2B.

And mapped fovated images

The area of

It is. here,

From x

This is a Jacobian function that represents a coordinate transformation to. In the discrete domain,

Is the square of the local frequency

Is proportional to

Is defined as the following equation (4).

In the formula (4), c is a constant. Assuming that the conversion coefficient value of one pixel of a given image is a random variable X, H _d (x) is obtained as in the above-described equation (1). Total differential entropy for images

Is as in the following equation (5).

Similarly, a forevated image that has been transformed in a curved coordinate system

Differential entropy of

And total visual entropy

Is defined as the following equations (6) and (7).

Foveated image in which original image a (x) and curved coordinate system are mapped

Is a locally band-limited signal having a local bandwidth Ω _o , so the probability density function of the original image in the Cartesian coordinate system, the probability density function of the Fovated image in the curvilinear coordinate system, and the differential entropy Can be assumed to be the same. That is,

It is.

Thus, the original image that exists in the orthogonal coordinate system, the difference between the amount of information required to represent the human visual characteristics Fobi er Ted image converted into consideration the curvilinear coordinate system, and the area A _o of the original image curve It is determined by using the difference between the Fobi er Ted image a _c in the coordinate system. That is, when an image is encoded using a curved coordinate system, a (A _o −A _c ) H (x) (here, compared to encoding an original image in an orthogonal coordinate system). , A _o ≧ A _c ) is saved.

Theoretically, the amount of entropy saved is an upper limit that can be reduced when encoding image data without losing visual information. Therefore, the normalization gain Gm obtained by encoding the forged image in the curved coordinate system in consideration of human visual characteristics is (A _o −A _c ) / A _o .

( Differential entropy of wavelet coefficients)
First, W (X) is assumed to be a wavelet transform function. In FIG. 1A, an original image a (X) is converted into a wavelet region by W (X). At this time, the wavelet coefficient a [m] (m is an index of the wavelet coefficient) can be defined as the following equation (8).

As described above, g _m represents an orthogonal normal basic function.

  Original image mapped to curved coordinate system

Foveated image with curved coordinate system mapped

Is assumed to be a locally band-limited signal with local bandwidth Ω _o

And can be approximated.

  The wavelet coefficient b [m] of the original image mapped in the curved coordinate system can be defined as the following equation (9).

If Expression (1) and Expression (6) are used, the wavelet transform coefficient a [m] in the orthogonal coordinate system and the wavelet transform coefficient b [m] in the curved coordinate system are respectively expressed by the following Expression (10). It is expressed as follows.

( Visual entropy in wavelet domain)
A visual weight value set in consideration of human visual characteristics in the spatial domain and the frequency domain is defined as ω _m . For a given visual weight ω _m , visual entropy

Can be expressed as the following equation (11).

As described above, ω _m includes a component related to the spatial domain and a component related to the frequency domain.

The local frequency f _n described in the equation (4) can be used as a visual weight value in the spatial domain. If, assuming the f _m and the local frequency in the wavelet domain, f _m is expressed as the following equation (12).

Here, m is an index of the wavelet coefficient a [m], and r represents the display resolution of the wavelet. In Expression (12), f _c represents a critical frequency, and f _d represents a display Nyquist frequency. The critical frequency and the display Nyquist frequency will be described as follows.

  According to an experiment for measuring the sensitivity of a human HVS to light and dark as a function having the retina eccentricity as a parameter, it is known that the following relationship (13) is established.

Where f is the spatial frequency (cycles / deg), e is the retinal eccentricity (deg), CTo is the minimum light / dark critical value, α is the spatial frequency cancellation constant, e ² is the half resolution eccentricity constant, CT (F, e) represents a visually perceptible light / dark critical value of the function with f and e as parameters. The light / dark sensitivity CS (f, e) is defined as the reciprocal 1 / CT (f, e) of the light / dark critical value.

For a given eccentricity e, may calculate the critical frequency f _c by using the equation (13). Here, the critical frequency f _c is a human a value representing the limit of visually perceptible spatial frequency greater than the frequency components critical frequency f _c can not visually perceptible.

Maximum possible brightness value in a case assuming a CT (f, e) if a critical frequency f _c as the expression (13) from the following equation (14) is obtained.

FIG. 3 is a diagram for explaining the eccentricity of the retina and the visual cognitive structure. Here, it is assumed that the observed image plane 300 has a width of N pixels, and the line connecting the fixation point 310 from the phobia is perpendicular to the image plane 300. Further, it is assumed that a value obtained by normalizing the distance from the phobia to the eyes of the observer by the image size is v.

Referring to FIG. 3, the eccentricity e is an arbitrary point x that is separated from the observer's fixation point 310 by a predetermined distance u (a value measured by being normalized by the image size) from the fixation point.
320 denotes an angle difference generated by a difference in position where the retina is connected to the retina. Therefore, when observing the fixation point 310 of the image plane 300, the eccentricity e by the observer at a distance of about v in the image plane 300 is

It is.

  On the other hand, it is known that the maximum resolution that can be recognized in an actual digital image is also limited by the display resolution r. At this time,

Is defined. Due to the sampling principle, the display Nyquist frequency f _d , which is the maximum frequency that can be represented by the display device without any aliasing, is half the resolution of the display device. Therefore, the display Nyquist frequency _fd is as the following equation (15).

In the two-dimensional spatial domain, a weighted value of the square values of the normalized local frequency f _m as the following equation (16) in the spatial domain

It can be used as

FIG. 4 is a diagram illustrating a wavelet decomposition structure. Referring to FIG. 4, LL, HL, LH, and HH subbands are obtained by alternately applying horizontal and vertical wavelet decomposition processes. The LL subband can also be decomposed into smaller subbands, and the process is repeated any number of times.

Wavelet coefficients present in other subbands and locations provide variable cognitive importance to human HVS. It is necessary to determine the visual significance of each wavelet coefficient in the frequency domain in consideration of human HVS. In the present invention, the frequency domain weight value which is the frequency domain component of the visual weight value ω _m

Is determined by the subband of each wavelet. Assuming that the visually perceptible noise critical value is Y, it is known that the value of Y can be expressed by the following equation (17) through experiments.

Here, θ is an index representing a subband of a wavelet, f is a spatial frequency (cycles / degree), and g _θ , f _o , and k are constants. Using the given display resolution r and wavelet decomposition level λ, the spatial frequency f is as follows: f = r 2 ^−λ

At this time, the error detection critical value T _{λ, θ} of the wavelet coefficient at an arbitrary wavelet decomposition level λ and subband _θ is expressed by the following equation (18).

Here, A _{λ, θ} represents the basic function size. Accordingly, the error sensitivity S _ω (λ, θ) in one subband has a reciprocal of the error detection critical value T _{λ, θ} , that is, a value of 1 / T _{λ, θ} .

In the present invention, S _ω (λ, θ) normalized as in the following equation (19) is weighted in the frequency domain.

Use as

Using the above equations (16) and (19), the visual weight value ω _m set in consideration of human visual characteristics in the spatial domain and the frequency domain is finally expressed by the following equation (20). Is defined as

( Image coding / decoding method and apparatus considering visual weight)
In the following, a method for encoding and decoding an image using a visual weight obtained by calculating a product of the above-described spatial domain weight and frequency domain weight, and an image coder using the same explain.

  FIG. 5 is a block diagram showing an image encoding apparatus according to the present invention, and FIG. 6 is a flowchart showing an image encoding method according to the present invention. Referring to FIG. 5, an image encoding apparatus 500 according to the present invention includes a transform unit 510, a visual weight value generation unit 520, a region of interest determination unit 530, an encoding order determination unit 540, and a sequential wavelet coefficient encoding unit 550. Is provided.

  In step 610, the transform unit 510 performs wavelet transform on the input image, divides the input image into low frequency subbands and high frequency subbands, and obtains a wavelet transform coefficient for each pixel of the input image.

  In step 620, the visual weight generation unit 520 generates visual weights of wavelet transform coefficients in consideration of human visual sensitivity in the spatial domain and the frequency domain.

As described above, the visual weight generation unit 520 uses the local frequency f _n described in Equation (4) as the visual weight in the spatial domain or the critical frequency f _c in the wavelet domain. and display of the Nyquist frequency f _d, and selects the minimum value as the local frequency f _m in the wavelet domain, the equation normalized weight values for the square value spatial region of the local frequency f _m as (16)

It can be used as That is, the visual weight generation unit 520 performs the critical frequency in the wavelet domain.

And the display Nyquist frequency, which is the maximum frequency that can be expressed without aliasing by the display device

The smallest value is selected from the above, and this is normalized as shown in Equation (16), and the weighted value in the spatial domain

Is generated. Further, the visual weight generation unit 520 expresses an error sensitivity S _ω (λ, θ) having a value of 1 / T _{λ, θ} as a reciprocal of the error detection critical values T _{λ, θ in} the subband ( 19) weighted values in the frequency domain by normalization

Is generated. The visual weight generation unit 520 then calculates the weight in the spatial domain.

And weights in the frequency domain

Is used to generate a visual weight of a reference value that determines the encoding order of the wavelet coefficients.

  When generating the visual weight value, the region-of-interest determination unit 530 determines an area in which the human line of sight is fixed, and determines an image area recognized by the human optic nerve, that is, a forbidden area. The region-of-interest determination unit 530 detects a region having a visually high motion activity from the image through motion detection, or tracks an observer's pupil movement as used in a security camera application program. A region of interest can be detected from the image, or a region input by selection by the user can be determined as the region of interest.

  In step 630, the encoding order determination unit 540 determines the encoding order of the wavelet transform coefficients using the generated visual weights. In step 640, the sequential wavelet coefficient encoding unit 550 determines The wavelet transform coefficients are quantized and entropy-coded according to the encoded order, and a bit stream is generated. For example, the encoding order determination unit 540 uses the visual weight values generated by the visual weight value generation unit 520 to use the visual weight value order of the wavelet coefficients of each subband in one frame. The sequential wavelet coefficient encoding unit 550 encodes and transmits wavelet coefficients having a larger visual weight.

  Also, the encoding order determination unit 540 calculates the total number of wavelet coefficients that can be transmitted with the current channel capacity using the current channel capacity and the difference entropy value of the wavelet coefficient, and generates the generated visual weight value. Depending on the size order, the total number of wavelet transform coefficients can be selected.

  On the other hand, the total sum of the transmitted visual information is determined by the sum of the visual entropy of the transmitted data. In order to maximize the amount of visual transmission with limited channel capacity, it is more efficient to transmit wavelet coefficient values containing relatively high visual information first, as in the present invention. . As described above, the visual information included in one bit is a product of a spatial weight value and a visual weight value set in consideration of human visual characteristics in the frequency and space areas. It is evaluated by the dynamic weight. Using the above equation (20), the visual entropy is defined as the following equation (21).

Assuming that the given channel capacity is C, the total number M of wavelet coefficients that can be transmitted is calculated by the following equation (22).

  Let k be an index representing the order of wavelet coefficients rearranged based on visual weights according to the present invention. In this case, the transmittable visual entropy is calculated as in the following equation (23).

In Equation (23), K means the maximum number of wavelet transform coefficients that can be transmitted when C has a limited channel capacity. Thus, the visual entropy of the wavelet coefficient transmitted according to the importance of vision is as the following equation (24).

Here, _Cω represents the sum of visual entropies transmitted to a given channel capacity C. If visual weighting according to the present invention

Is used, it has a relative visual entropy gain G _t as shown in the following equation (25).

here,

Satisfied. In equation (25),

Means the total visual entropy calculated by taking the visual weights into consideration for the entropy of the wavelet coefficients. That is, when the total wavelet coefficients M ^T,

It is.

  FIG. 7 is a block diagram showing a configuration of an image decoding apparatus according to the present invention, and FIG. 8 is a flowchart showing an image decoding method according to the present invention. Referring to FIG. 7, the image decoding apparatus 700 according to the present invention includes a sequential wavelet coefficient decoding unit 710, an inverse transform unit 720, and an image restoration unit 730.

  In step 810, the sequential wavelet coefficient decoding unit 710 performs visual weighting of wavelet transform coefficients generated in consideration of human visual sensitivity in the spatial domain and the frequency domain by the above-described image coding method. The wavelet transform coefficients encoded according to the magnitude order of are decoded. That is, the sequential wavelet coefficient decoding unit 710 performs wavelet transform coefficient entropy decoding and inverse quantization on the wavelet transform coefficient provided in the bitstream, and outputs the wavelet transform coefficient.

  In step 820, the inverse transform unit 720 performs inverse wavelet transform on the decoded wavelet transform coefficients, and outputs the wavelet coefficients in each subband.

  In step 830, the image restoration unit 730 restores the image using the coefficients of the subbands subjected to the inverse wavelet transform.

  FIG. 9A is a diagram in which the image quality of an image restored after being encoded by the conventional SPIHT algorithm is measured by a target bit rate, and FIG. 9B is encoded according to the order of magnitude of visual weight values according to the present invention. FIG. 6 is a diagram in which the image quality of an image restored after being measured is measured by a target bit rate.

  As the image quality measurement method, PSNR (Peak Signal to Noise Ratio) and FWQI (Foveated Wavelet Image Quality Index) were used. The FWQI is described in detail in “A universal image quality index” (Z. Wang and AC Bovic, IEEE Signal Processing Letter), but a specific description thereof is omitted here.

  Comparing FIG. 9A and FIG. 9B, the image quality of the image restored after encoding according to the present invention with the low bit rate and based on the visual weight is restored after being encoded by the conventional SPIHT algorithm. It can be confirmed that the image quality is superior to that of the image. When the bit rate increases, the amount of wavelet coefficients that can be transmitted increases, so the difference in the image quality of the restored image is not large, but the present invention is particularly improved when the channel bandwidth is small. Can provide an image of a selected quality.

  FIG. 10 shows a comparison of the visual entropy when the wavelet coefficients are rearranged and transmitted according to the present invention according to the channel capacity according to the channel capacity, and when compared with the linear transmission method. FIG. 11 is a graph showing the case where the wavelet coefficients are rearranged and transmitted according to the present invention according to the channel capacity according to the channel capacity, and when the conventional SPIHT algorithm is used for transmission. Is a graph showing the visual entropy gain defined in (1). 10 and 11, the x axis is

Is a value obtained by normalizing the channel capacity in which the weight value is considered.

  Referring to FIG. 10, in the image coding method according to the present invention, the total amount of transmitted visual entropy increases rapidly with a low channel capacity, and converges sequentially when the channel capacity is 1. Referring to FIG. 11, it can be reconfirmed that the image coding method according to the present invention has a relatively high visual entropy gain value compared to the conventional SPIHT algorithm with a low channel capacity. Referring to FIG. 11, it can be confirmed that when the channel capacity is about 0.1, the visual entropy gain value increases rapidly to about 0.23. Such visual entropy gain has a gain of about 0.2 even greater than that of the conventional SPIHT algorithm when the channel capacity is about 0.1 to 0.45.

  On the other hand, the above-described image encoding method and decoding method can be created by a computer program. The code and code segments that make up the program are easily inferred by computer programmers in the field. In addition, the program is stored in a computer-readable information recording medium, and is read and executed by the computer, thereby realizing a method for encoding and decoding moving images. The information recording medium includes a magnetic recording medium, an optical recording medium, and a carrier wave medium.

  The present invention has been mainly described with reference to preferred embodiments. Those skilled in the art will appreciate that the present invention may be embodied in variations that do not depart from the essential characteristics of the invention. Accordingly, the disclosed embodiments should be considered in an illustrative, not a limiting sense. The scope of the present invention is shown not in the above description but in the claims, and all differences within the equivalent scope should be construed as being included in the present invention.

It is a figure which shows an example of the original image a (x). Fobied image

It is a figure which shows an example.
The original image a (x) in FIG.

Original image mapped to

FIG.
Fig. 1B forbidden image

The curved coordinate system

Foveated image mapped to

FIG.
It is a figure for demonstrating the eccentricity and visual cognitive structure of a retina. It is a figure which shows a wavelet decomposition structure. It is a block diagram which shows the image coding apparatus by this invention. 3 is a flowchart illustrating an image encoding method according to the present invention. It is a block diagram which shows the structure of the image decoding apparatus by this invention. 5 is a flowchart illustrating an image decoding method according to the present invention. It is the figure which measured the image quality of the image decompress | restored after encoding by the conventional SPIHT algorithm with the target bit rate. FIG. 5 is a diagram illustrating the image quality of a restored image after encoding according to the order of magnitudes of visual weight values according to the present invention, according to a target bit rate. 6 is a graph showing visual entropy in comparison with a linear transmission method when channellet capacity is transmitted by rearranging wavelet coefficients based on visual weight values according to the channel capacity and when transmitted by a conventional SPIHT algorithm. . When the wavelet coefficients are rearranged based on the visual weight according to the channel capacity according to the channel capacity and transmitted according to the conventional SPIHT algorithm, the graph shows the visual entropy gain defined by Equation (25). is there.

Claims (20)

In the image encoding method,
Performing wavelet transform on the input image to generate wavelet transform coefficients;
Generating visual weights of the wavelet transform coefficients in consideration of human visual sensitivity in the spatial domain and frequency domain;
Determining an encoding order of the wavelet transform coefficients using the generated visual weights;
And a step of encoding the wavelet transform coefficients in accordance with the determined encoding order. Generating a visual weight value of the wavelet transform coefficient;
Applying the normalized local bandwidth around the region of interest of the wavelet transformed input image, the spatial domain weight value of the wavelet transform coefficient
A step of determining
Frequency domain weighted value of the wavelet transform coefficient using error sensitivity in subbands of the wavelet transformed input image
A step of determining
The image encoding method according to claim 1, further comprising: calculating the product of the spatial domain weight value and the frequency domain weight value to generate the visual weight value. The spatial domain weight
Is
Human-determined by using a minimum value of the display Nyquist frequency f _d is the maximum frequency that can be represented a critical frequency f _c and the image is a value representing the limit of visually perceptible spatial frequency without aliasing The image encoding method according to claim 2. e
Where N is the number of pixels, v is the distance between the eye and the image normalized by the image size, and d is the distance between the corresponding pixel position of the wavelet transform coefficient and the oscillation point. The critical frequency f _c is expressed by the following equation, where CT ₀ is the minimum contrast critical value, α is the spatial frequency cancellation constant, and e ² is the half resolution eccentricity constant.
Defined as
The display Nyquist frequency f _d is given by
Defined as
The spatial domain weight
It is the local frequency in the wavelet domain to the minimum value of the critical frequency f _c and a display Nyquist frequency f _d
Where m is the index of the wavelet coefficient,
The image encoding method according to claim 3, wherein the image encoding method has a value defined as follows. The frequency domain weight value
Is
When the wavelet decomposition level is λ and the index representing the wavelet subband is θ, the error sensitivity S _ω (λ, θ) in the subband to which the wavelet coefficient belongs is normalized. The image encoding method according to claim 2. The error sensitivity S _ω (λ, θ) is
A _{λ, θ} is the magnitude of the basic function, f is the spatial frequency, g _θ , f _o , k is a constant, and r is the display resolution.
6. The image encoding method according to claim 5, further comprising a value obtained by normalizing an inverse value of an error detection critical value _{Tλ, θ} of the wavelet coefficient defined as follows. Determining the encoding order of the wavelet transform coefficients;
Calculating the total number of wavelet coefficients that can be transmitted with the current channel capacity using a current channel capacity and a differential entropy value of the wavelet coefficients;
The image encoding method according to claim 2, further comprising: selecting as many of the wavelet transform coefficients as the total number according to the order of the generated visual weights. The region of interest of the input image is
3. The region according to claim 2, determined by tracking movement of an area having a visually high motion activity in the image or motion of the observer's pupil through motion detection, or by user selection. Image coding method. In an image encoding device,
A transform unit that performs wavelet transform on the input image to generate wavelet transform coefficients;
In consideration of human visual sensitivity in the spatial domain and the frequency domain, a visual weight generation unit that generates a visual weight of the wavelet transform coefficient;
An encoding order determining unit that determines an encoding order of the wavelet transform coefficients using the generated visual weight values;
An image encoding apparatus comprising: a sequential wavelet coefficient encoding unit that encodes the wavelet transform coefficients in accordance with the determined encoding order. The visual weight generation unit
A spatial domain weighted value of the wavelet transform coefficient by applying a local bandwidth normalized around a region of interest of the wavelet transformed input image
A spatial domain weight determination unit for determining
Utilizing error sensitivity in subbands of the wavelet transformed input image, the frequency domain weight value of the wavelet transform coefficient
A frequency domain weight determination unit for determining
The image code of claim 9, further comprising: a multiplication unit that calculates a normalized product of the spatial domain weight value and the frequency domain weight value to generate the visual weight value. Device. The spatial domain weight
Is
Human of display Nyquist frequency f _d is the maximum frequency that can be represented a critical frequency f _c and the image is a value representing the limit of visually perceptible spatial frequency without aliasing is determined by using the minimum value The image coding apparatus according to claim 10. e
(Where N is the number of pixels, v is the distance between the eye and the image normalized by the image size, and d is the distance between the corresponding pixel position of the wavelet transform coefficient and the oscillation point) eccentricity defined, CT ₀ is minimum contrast threshold, alpha is the spatial frequency offset constant, e ^2, when the half resolution eccentric constant), the critical frequency f _c, the following equation
Defined as
The display Nyquist frequency f _d is given by
Defined as
The spatial domain weight
It is the local frequency in the wavelet domain to the minimum value of the critical frequency f _c and a display Nyquist frequency f _d
Where m is the index of the wavelet coefficient,
The image encoding device according to claim 11, wherein the image encoding device has a value defined as follows. The frequency domain weight value
Is
When the wavelet decomposition level is λ, and the index representing the sublet of the wavelet is θ, the error sensitivity S _ω (λ, θ) in the subband to which the wavelet coefficient belongs is normalized. The image encoding device according to claim 10. The error sensitivity S _ω (λ, θ) is
A _{λ, θ} is the magnitude of the basic function, f is the spatial frequency, g _θ , f _o , k is a constant, and r is the display resolution.
14. The image encoding device according to claim 13, wherein the image encoding device has a value obtained by normalizing an inverse value of an error detection critical value T _{λ, θ} of the wavelet coefficient defined as follows. The encoding order determination unit includes:
Using the current channel capacity and the difference entropy value of the wavelet coefficient, the total number of wavelet coefficients that can be transmitted with the current channel capacity is calculated, and the total number of the wavelet coefficients can be calculated according to the order of the generated visual weight values. The image encoding apparatus according to claim 9, wherein the wavelet transform coefficient is selected.   A region of interest determination unit that determines a region of interest by tracking a movement of a region having a visually high motion activity or an observer's pupil in the image through motion detection or by user selection is further provided. The image encoding device according to claim 9. In the image decoding method,
Decoding wavelet transform coefficients encoded according to the order of magnitude of visual weights generated taking into account human visual sensitivity in the spatial and frequency domains;
Performing an inverse wavelet transform on the decoded wavelet transform coefficients;
And a step of restoring an image using coefficients of each subband subjected to the inverse wavelet transform. The visual weight is
Human of display Nyquist frequency f _d is the maximum frequency that can be represented a critical frequency f _c and the image is a value representing the limit of visually perceptible spatial frequency without aliasing is determined using the minimum value Spatial region weight
And a frequency domain weight having a value obtained by normalizing the error sensitivity S _ω (λ, θ) in the subband to which the wavelet coefficient belongs, where λ is the wavelet decomposition level and θ is the index representing the wavelet subband. value
The image decoding method according to claim 17, wherein the image decoding method is calculated using a product of. In the image decoding device,
A sequential wavelet coefficient decoding unit that decodes wavelet transform coefficients encoded according to the order of magnitude of visual weights generated in consideration of human visual sensitivity in the spatial domain and the frequency domain;
An inverse transform unit for performing an inverse wavelet transform on the decoded wavelet transform coefficients;
An image decoding apparatus comprising: an image restoration unit that restores an image using a coefficient of each subband subjected to the inverse wavelet transform. The visual weight is
Human of display Nyquist frequency f _d is the maximum frequency that can be represented a critical frequency f _c and the image is a value representing the limit of visually perceptible spatial frequency without aliasing is determined using the minimum value Spatial region weight
And a frequency domain weight having a value obtained by normalizing the error sensitivity S _ω (λ, θ) in the subband to which the wavelet coefficient belongs, where λ is the wavelet decomposition level and θ is the index representing the wavelet subband. value
The image decoding apparatus according to claim 19, wherein the image decoding apparatus is calculated using a product of.