JP4094652B2 - Visual processing device, visual processing method, program, recording medium, display device, and integrated circuit - Google Patents

Description

  The present invention relates to a visual processing device, a visual processing method, a program, a recording medium, a display device, and an integrated circuit. In particular, the present invention relates to a visual processing device, a visual processing method, a program, a recording medium, a display device, and an integrated circuit that adjust the visual processing effect of an image to be different.

Conventionally, an image processing apparatus using gradation processing and an image processing apparatus using spatial processing are known as image quality improvement processing of an image signal of an original image.
The gradation processing is processing for converting pixel values using a lookup table (hereinafter referred to as “LUT”) for each target pixel regardless of the surrounding pixels of the target pixel, and is called gamma correction. There is also. For example, when contrast enhancement is performed, pixel values are converted using an LUT that enhances a gradation level having a high appearance frequency in the original image. As gradation processing using the LUT, gradation processing that determines and uses one LUT for the entire original image, and gradation processing that determines and uses an LUT for each of the image areas obtained by dividing the original image into a plurality of images are known. It has been.
Spatial processing is to convert the value of the pixel of interest using the value of the pixel of interest to which the filter is applied and the values of surrounding pixels. Using this spatially processed image signal, contrast enhancement of the original image is performed (see, for example, Patent Document 1).
US Pat. No. 4,667,304

On the other hand, as an image quality improvement process close to human vision, there is a visual process for converting the value of the pixel of interest based on the comparison between the value of the pixel of interest and the values of the pixels in the surrounding area. In such visual processing, brightness information is extracted from a wide range of areas around the position of the pixel of interest in order to further enhance the processing effect.
However, since the value of the pixel of interest is determined from the comparison between the value of the pixel of interest and the values of the surrounding pixels, if there is a steep edge region in the peripheral region, it is affected by the values of the surrounding pixels. Even in a flat region where the pixel value hardly fluctuates, the output of visual processing changes gently in the vicinity of the edge. When a large luminance change occurs in a flat region, a shadow-like contour is generated in the region adjacent to the edge (hereinafter referred to as “side effect”), resulting in an unnatural image.
The present invention solves such problems, and even when an image having a steep edge region is input, a visual processing device, a visual processing method, a program, and a recording that can suppress side effects An object is to provide a medium, a display device, and an integrated circuit.

  A first invention includes a peripheral image information extraction unit that extracts peripheral image information of an input image signal, a visual processing unit that performs visual processing on the image signal based on the image signal and the peripheral image information, The visual processing device includes a control signal generation unit that outputs an effect adjustment signal for setting an effect of processing, and an effect adjustment unit that sets the effect of visual processing according to the effect adjustment signal. The control signal generator determines an edge vicinity area and a flat area included in the image area of the image formed by the image signal, and determines that the image area is not the edge vicinity area or is determined not to be a flat area. On the other hand, an effect adjustment signal that realizes visual processing based on the image signal and the peripheral image information is generated, and the image signal and the peripheral are detected for the image region that is the edge vicinity region and is determined to be a flat region. An effect adjustment signal for realizing visual processing with an effect weaker than the effect of visual processing based on image information is generated.

  In this visual processing device, the control signal generator outputs an effect adjustment signal for setting the effect of visual processing, the peripheral image information extraction unit extracts peripheral image information of the input image signal, and effects adjustment Since the effect of the visual processing is set according to the effect adjustment signal, the effect of the visual processing can be made different according to the effect adjustment signal, and the effect of the visual processing is adjusted in the image area where the side effect occurs. Side effects can be suppressed. Further, in this visual processing device, the control signal generator determines the edge vicinity area and the flat area included in the image area of the image formed by the image signal, and not the edge vicinity area or the flat area. An image area determined to be an edge vicinity area and a flat area by generating an effect adjustment signal for realizing visual processing based on the image signal and peripheral image information for the image area determined to be not On the other hand, since the effect adjustment signal that realizes the visual processing with the effect weaker than the visual processing effect based on the image signal and the peripheral image information is generated, the effect of the visual processing on the image area where the side effect is likely to occur is weakened. And the occurrence of side effects can be suppressed. In particular, it is possible to efficiently suppress a side effect in which a so-called halo occurs on an image.

Here, “visual processing with weak effects” is a concept including no visual processing effect, that is, no visual processing. The “peripheral image information” refers to information derived from the target pixel and pixels around the target pixel. For example, the average brightness (gradation value) of an N × N pixel region centered on the peripheral pixel, etc. Is an example of this. Note that in order to acquire the peripheral image information, it is not always necessary to perform processing in units of pixels, and peripheral image information may be acquired by performing processing in units of blocks including a plurality of pixels.
2nd invention is 1st invention, Comprising: A visual processing part performs the visual processing which adjusts a local contrast with respect to an image signal based on an image signal and peripheral image information.
Accordingly, side effects can be suppressed even in a visual processing device that realizes visual processing for adjusting local contrast.

Here, “local contrast” refers to the contrast derived by contrasting the brightness of the target pixel and its surrounding pixels, and “visual processing for adjusting local contrast” refers to, for example, local In the image area, it means a process of adjusting the contrast of the local image area based on the brightness contrast between the pixel of interest and its surrounding pixels. It goes without saying that not only pixel unit processing but also block unit processing may be used. Further, “visual processing for adjusting local contrast” is a concept including processing based on contrast of color information (lightness, saturation, hue, etc.) in addition to brightness contrast.
3rd invention is 1st or 2nd invention, Comprising: A control signal generation part outputs an effect adjustment signal according to the variation | change_quantity of surrounding image information.
Thereby, the side effect which generate | occur | produces with the change of surrounding image information can be suppressed.

4th invention is 1st or 2nd invention, Comprising: A control signal generation part is based on the variation | change_quantity detected by the variation | change_quantity detection part which detects the variation | change_quantity of surrounding image information, and the variation | change_quantity detection part. And an effect adjustment signal generator for generating an effect adjustment signal.
Thereby, the side effect which generate | occur | produces with the variation | change_quantity of surrounding image information can be suppressed.
5th invention is 1st or 2nd invention, Comprising: The control signal generation | occurrence | production part detects the flatness degree of the flat area from which an intensity difference with an adjacent area becomes below a predetermined value from an image signal An edge detection unit that detects an edge amount of an edge region in which the luminance difference from the adjacent region is equal to or greater than a predetermined value from the image signal, and an edge that calculates an edge proximity degree indicating an edge vicinity degree of the image region based on the edge amount A proximity detector, and an effect adjustment signal generator that generates an effect adjustment signal based on the flatness detected by the flatness detector and the edge proximity calculated by the edge proximity detector.

According to this, even if an image having a steep edge region is input, side effects of a flat region near the edge region can be suppressed.
A sixth invention is any one of the first to fifth inventions, wherein the effect adjustment unit outputs a first synthesized signal obtained by synthesizing the image signal and the peripheral image information in accordance with the effect adjustment signal. The visual processing unit performs visual processing on the image signal based on the first composite signal and the image signal.
According to this, the visual processing unit can select different gradation conversion processing based on the first composite signal, and can visually process the image signal by the selected gradation conversion processing. The effect of visual processing can be changed (adjusted).
The seventh invention is any one of the first to fifth inventions, wherein the effect adjustment unit synthesizes the image signal and the output subjected to visual processing by the visual processing unit in accordance with the effect adjustment signal. A second synthesized signal is output.

According to this, the ratio between the image signal and the processing signal can be changed according to the effect adjustment signal, and the effect of visual processing can be varied (adjusted).
According to an eighth aspect of the invention, there is provided a peripheral image information extracting step for extracting peripheral image information of the input image signal, a visual processing step for performing visual processing on the image signal based on the image signal and the peripheral image information, A visual processing method comprising: a control signal generation step for outputting an effect adjustment signal for setting an effect of processing; and an effect adjustment step for setting an effect of visual processing according to the effect adjustment signal. In the control signal generation step, the edge vicinity area and the flat area included in the image area of the image formed by the image signal are determined, and the image area determined to be not the edge vicinity area or not the flat area. On the other hand, an effect adjustment signal that realizes visual processing based on the image signal and the peripheral image information is generated, and the image signal and the peripheral are detected for the image region that is the edge vicinity region and is determined to be a flat region. An effect adjustment signal for realizing visual processing with an effect weaker than the effect of visual processing based on image information is generated.

Thereby, it is possible to realize a visual processing method that exhibits the same effect as that of the first invention.
According to a ninth aspect of the invention, a peripheral image information extracting step for extracting peripheral image information of an input image signal to a computer, and a visual processing step for performing visual processing on the image signal based on the image signal and the peripheral image information And a control signal generation step for outputting an effect adjustment signal for setting the effect of visual processing, an effect adjustment step for setting the effect of visual processing according to the effect adjustment signal, and a program to be executed. In the control signal generation step, the edge vicinity area and the flat area included in the image area of the image formed by the image signal are determined, and the image area determined to be not the edge vicinity area or not the flat area. On the other hand, an effect adjustment signal that realizes visual processing based on the image signal and the peripheral image information is generated, and the image signal and the peripheral are detected for the image region that is the edge vicinity region and is determined to be a flat region. An effect adjustment signal for realizing visual processing with an effect weaker than the effect of visual processing based on image information is generated.

As a result, it is possible to realize a program that exhibits the same effects as those of the first invention.
A tenth aspect of the invention is a peripheral image information extracting step for extracting peripheral image information of an input image signal to a computer, and a visual processing step for performing visual processing on the image signal based on the image signal and the peripheral image information And a control signal generation step for outputting an effect adjustment signal for setting the effect of visual processing, and an effect adjustment step for setting the effect of visual processing in accordance with the effect adjustment signal. It is. In the control signal generation step, the edge vicinity area and the flat area included in the image area of the image formed by the image signal are determined, and the image area determined to be not the edge vicinity area or not the flat area. On the other hand, an effect adjustment signal that realizes visual processing based on the image signal and the peripheral image information is generated, and the image signal and the peripheral are detected for the image region that is the edge vicinity region and is determined to be a flat region. An effect adjustment signal for realizing visual processing with an effect weaker than the effect of visual processing based on image information is generated.

Thereby, it is possible to realize a recording medium having the same effect as that of the first invention.
An eleventh aspect of the invention includes a peripheral image information extraction unit that extracts peripheral image information of an input image signal, a visual processing unit that performs visual processing on the image signal based on the image signal and the peripheral image information, An integrated circuit includes a control signal generation unit that outputs an effect adjustment signal for setting an effect of processing, and an effect adjustment unit that sets an effect of visual processing according to the effect adjustment signal. The control signal generator determines an edge vicinity area and a flat area included in the image area of the image formed by the image signal, and determines that the image area is not the edge vicinity area or is determined not to be a flat area. On the other hand, an effect adjustment signal that realizes visual processing based on the image signal and the peripheral image information is generated, and the image signal and the peripheral are detected for the image region that is the edge vicinity region and is determined to be a flat region. An effect adjustment signal for realizing visual processing with an effect weaker than the effect of visual processing based on image information is generated.

Thus, an integrated circuit that exhibits the same effect as that of the first invention can be realized.
A twelfth aspect of the invention is a data receiving unit that receives image data transmitted or broadcasted, a decoding unit that decodes the received image data into video data, and outputs an output signal by visually processing the decoded video data A visual processing device according to any one of the first to seventh inventions, and a display unit that displays an output signal visually processed by the visual processing device.
As a result, visual processing that achieves the same effects as those of the first to seventh inventions can be realized in the display device.

  According to the present invention, it is possible to provide a visual processing device, a visual processing method, a program, a recording medium, and an integrated circuit capable of suppressing side effects even when an image having a steep edge region is input. it can.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(Embodiment 1)
In general, a natural image has a large number of gradations, and by performing visual processing on the natural image, a sharp image with improved local contrast can be obtained. On the other hand, if there are steep edges in the image, side effects tend to be noticeable when visual processing is performed. If the visual processing is weakened to suppress this side effect, the processing is weakened even for a natural image, resulting in an image with no sharpness.
Therefore, by weakening the visual processing only for the vicinity of the edge, it is possible to suppress side effects in the vicinity of the edge while maintaining the processing effect for the entire natural image.
The visual processing device according to the first embodiment of the present invention outputs an effect adjustment signal for varying the effect of visual processing, and varies the effect of visual processing according to the effect adjustment signal (intensity and correction amount). ) Make adjustments.

Further, in the image to be subjected to visual processing, an area adjacent to the edge or a flat area adjacent to the edge is detected, and an effect adjustment signal is generated from the amount of edge and the degree of flatness, and the visual adjustment is performed according to the effect adjustment signal. The adjustment is made so that the effect of the treatment is different.
Thereby, even when an image having a steep edge region is input to the visual processing device, side effects in the vicinity of the edge can be suppressed while obtaining the effect of visual processing.
Here, visual processing is processing that has a characteristic close to how the human eye can see, depending on the comparison between the value of the target pixel of the input image signal and the value (brightness) of its surrounding pixels. This process determines the value of the output signal. The visual processing is applied to backlight correction, knee processing, D-range compression processing, color processing, brightness adjustment (including gradation processing and contrast adjustment), and the like.
In the embodiment of the present invention, the luminance component Y or lightness component L of the YCbCr color space, YUV color space, Lab color space, Luv color space, YIQ color space, and YPbPr color space is defined as a luminance signal. Hereinafter, the luminance signal will be described as an image signal.

A visual processing device according to Embodiment 1 of the present invention will be described with reference to FIGS. FIG. 1 is a block diagram showing a configuration of a visual processing device 1 according to Embodiment 1 of the present invention.
In FIG. 1, the visual processing device 1 according to the first embodiment of the present invention responds to a spatial processing unit 10 that outputs peripheral image information (unsharp signal) US from an input image signal, and the flatness of an edge vicinity region. A control signal generator 40 that outputs an effect adjustment signal MOD, an effect adjustment unit 20 that outputs a synthesized signal MUS synthesized by changing the ratio of the image signal IS and the peripheral image information US according to the effect adjustment signal MOD, A visual processing unit 30 that visually processes the image signal IS based on the composite signal MUS and the image signal IS is provided.
Hereinafter, each functional unit of the visual processing device 1 will be described.
The spatial processing unit 10 extracts the value of the target pixel and the value of the pixel in the peripheral area of the target pixel (hereinafter referred to as “peripheral pixel”) from the image signal IS, and uses the extracted pixel value for the image signal IS. Filter processing for.

For example, the spatial processing unit 10 generates an unsharp signal US obtained by processing the image signal IS with a low-pass filter. The unsharp signal US is generated by the following calculation.
US = (Σ [W _ij ] × [A _ij ]) ÷ (Σ [W _ij ])
Here, [W _ij ] is a weighting factor of the pixel located in the i-th row and j-th column in the target pixel and the peripheral pixels, and [A _ij ] is in the i-th row and j-th column in the target pixel and the peripheral pixels. The value of the pixel located. Further, “Σ” means that the sum of the target pixel and the surrounding pixels is calculated.
Note that a smaller weight coefficient may be given as the absolute value of the pixel value difference is larger, or a smaller weight coefficient may be given as the distance from the target pixel is larger. The area size of the peripheral pixels is a size set in advance according to the effect, and the visual effect can be enhanced by making the size larger than a predetermined size. For example, if the size of the target image is 1024 pixels in length and 768 pixels in width, the unsharp signal US is generated from an area that is 80 pixels or more in length and width, respectively, and compared with a local area of about 3 pixels in length and width. This can enhance the visual effect.

Further, as the low-pass spatial filter, a FIR (Finite Impulse Response) type low-pass spatial filter or an IIR (Infinite Impulse Response) type low-pass spatial filter that is usually used for generating an unsharp signal may be used. .
Next, the effect adjusting unit 20 combines the image signal IS and the unsharp signal US by interpolation processing according to the effect adjusting signal MOD output from the control signal generating unit 40, and outputs a combined signal MUS. The composite signal MUS is divided internally as shown in (Equation 1) below, for example, according to the effect adjustment signal MOD. The control signal generator 40 will be described later.
MUS = US × MOD + IS × (1.0−MOD) (Formula 1)
Here, the value of the effect adjustment signal MOD varies in the range from “0.0” to “1.0”, the value of the effect adjustment signal MOD is “0.0”, no processing is performed, and the value of the effect adjustment signal MOD. Is “1.0”, the strength of the processing is maximized. Note that (Equation 1) can be modified as in (Equation 2), and similarly, a combined signal MUS can be generated.

MUS = (US-IS) × MOD + IS (Formula 2)
Next, the visual processing unit 30 performs gradation conversion on the image signal IS according to the composite signal MUS from the effect adjustment unit 20.
The visual processing unit 30 performs gradation conversion based on, for example, the two-dimensional gradation conversion characteristic shown in FIG. Here, the two-dimensional gradation conversion refers to gradation conversion in which an output value is determined with respect to two inputs of the composite signal MUS and the image signal IS. The visual processing unit 30 outputs a processing signal OS to the image signal IS and the composite signal MUS based on the two-dimensional gradation conversion characteristics. Various visual effects can be produced by the gradation conversion characteristics.
The two-dimensional gradation conversion characteristics will be described with reference to FIG. FIG. 2 is an explanatory diagram for explaining the two-dimensional gradation conversion characteristics. Here, the horizontal axis represents the input image signal IS, and the vertical axis represents the output of the converted processing signal OS.

  As shown in FIG. 2, the two-dimensional gradation conversion has a predetermined gradation conversion characteristic according to the signal levels (gradation values) of the combined signals MUS0 to MUSn. That is, in the two-dimensional gradation conversion, any one of the gradation conversion curves MUS0 to MUSn is selected according to the signal level (gradation value) of the composite signal MUS, and the selected gradation conversion curve This is realized by converting the input signal IS (IS gradation value) into a processing signal OS (OS gradation value). For example, when the level (gradation value) of the MUS signal is “1”, the gradation conversion curve MUS1 of FIG. 2 is selected, and when the level (gradation value) of the MUS signal is “120”, the gradation A conversion curve MUS120 is selected. However, the gradation conversion curves MUS0 to MUSn do not necessarily have to be prepared in a number corresponding to the number of gradation values of the MUS signal. For example, the gradation conversion curves MUS0 to MUSn are set to the number of gradation values of the MUS signal. A number smaller than the corresponding number is prepared, and for gradation conversion curves that are not prepared, a gradation conversion curve corresponding to the gradation value of the MUS signal is calculated from the prepared gradation conversion curve by interpolation processing. Thus, two-dimensional gradation conversion may be realized.

In the two-dimensional gradation conversion, for example, if the pixel value of the image signal IS is an 8-bit value, the pixel value of the processing signal OS with respect to the value of the image signal IS divided into 256 stages has a predetermined two-dimensional gradation conversion characteristic. Determined by. The gradation conversion characteristic is a gradation conversion curve having a predetermined gamma conversion characteristic, and the output monotonously decreases with respect to the subscript of the composite signal MUS. Note that even if there is a portion where the output is not partly monotonically decreasing, the subscript of the composite signal MUS may be substantially monotonically decreasing. Further, as shown in FIG. 2, in the two-dimensional gradation conversion characteristics, (output value in the case of MUS = MUS0) ≧ (in the case of MUS = MUS1) with respect to the brightness values of the pixels of all the image signals IS. The relationship of output value) ≧... ≧ (output value when MUS = MUSn) is satisfied.
According to the two-dimensional gradation conversion characteristics shown in FIG. 2, the visual processing unit 30 selects MUS0 when the input image signal IS has a value “a” and the brightness value of the surrounding area is small. The value of the processing signal OS is “P”. Conversely, when the brightness value of the surrounding area is large, the value of the processing signal OS becomes “Q” by selecting MUSn. As described above, even if the input image signal IS has the value “a”, the processing signal OS can be largely changed from the value “P” to the value “Q” by the change in the brightness value of the surrounding area. Thereby, the contrast of a dark part can be emphasized according to the synthetic | combination signal MUS.

On the other hand, in order to eliminate the effect of visual processing, if the composite signal MUS = image signal IS, the tone conversion characteristic of curve 2 shown in FIG. 2 can be provided. With the tone conversion characteristics of curve 2, the brightness of the entire image can be adjusted (gamma conversion), but there is no visual effect such as increasing the contrast only in the local dark area.
By changing this two-dimensional gradation conversion characteristic, various visual processing effects can be obtained, and knee processing, DR compression processing, color processing, or brightness adjustment (gradation processing, contrast adjustment can be performed). Including).
Next, the processing signal OS when the visual processing unit 30 changes the effect of visual processing based on the composite signal MUS will be described with reference to FIG. FIG. 3 is an explanatory diagram for explaining the output of the processing signal OS.

In FIG. 3A, the horizontal axis represents the pixel position to be processed, and the vertical axis represents the output of the composite signal MUS.
For example, when the value of the effect adjustment signal MOD is “0.5”, the composite signal MUS is an intermediate output between the image signal IS and the unsharp signal US.
At this time, as shown in FIG. 3B, the processing signal OS visually processed based only on the image signal IS is set to OS (IS, IS), and the visual processing based on the image signal IS and the unsharp signal US is performed. If the processing signal OS is OS (IS, US), OS (IS, MUS) which is a processing signal OS visually processed according to the image signal IS and the composite signal MUS is OS (IS, IS) and OS (IS , US).
Therefore, when the value of the effect adjustment signal MOD is “1.0”, the composite signal MUS = US, and the processing signal OS (IS, US) that has the “effect is maximum” of the visual processing is output. On the other hand, when the value of the effect adjustment signal MOD is “0.0”, the composite signal MUS = IS, and the processing signal OS (IS, IS) that is “no effect” of the visual processing is output.

As described above, the visual processing unit 30 can increase or decrease the effect of the visual processing of the dark portion contrast according to the composite signal MUS. As a result, there are various effects ranging from the effect of processing that only converts the brightness of the entire image to the effect of processing that changes (changes) the contrast in the local area according to the ambient brightness. A visual effect can be realized in the visual processing device 101.
In the visual processing device 1, knee processing, DR compression processing, color processing, and the like can be realized by changing the two-dimensional gradation conversion characteristics.
The visual processing unit 30 may include a two-dimensional lookup table (hereinafter referred to as “two-dimensional LUT”). Gray scale conversion is performed by setting characteristic data (hereinafter referred to as “profile”) shown in FIG. 2 in the two-dimensional LUT of the visual processing unit 30.

The visual processing unit 30 may perform visual processing using an arithmetic circuit. In particular, when a profile having characteristics that can be approximated by a simple straight line is set in the two-dimensional LUT of the visual processing unit 30, the table of the two-dimensional LUT can be eliminated, and the circuit scale of the visual processing device 1 is reduced. be able to.
Next, the control signal generator 40 will be described with reference to FIGS. FIG. 4 is a block diagram showing the configuration of the control signal generator 40, and FIG. 5 is an explanatory diagram for explaining the output of the effect adjustment signal MOD.
As shown in FIG. 4, the control signal generation unit 40 includes an edge detection unit 41 that detects an edge amount that is a luminance difference for each adjacent region from the image signal IS, and an edge vicinity that detects the degree of proximity of the edge region from the edge amount. A flatness detection unit 43 that detects the flatness of a flat region where the luminance difference between the detection unit 42 and the adjacent region is a predetermined value or less, and an effect adjustment that outputs an effect adjustment signal MOD according to the degree of edge proximity and the flatness And a signal generation unit 44.

The edge detection unit 41 detects an edge amount from the image signal IS for each region in a predetermined range. The edge detection unit 41 detects the edge amount EG using an edge detection filter (not shown) such as a primary differential filter such as a Sobel filter or a Prewitt filter, or a secondary differential filter such as a Laplacian filter. For example, when the image signal IS shown in FIG. 5A is input, the edge detector 41 outputs an edge amount as shown in FIG. Here, the vertical axis of FIG. 5A is the value of the image signal IS, and the horizontal axis is the pixel position of the pixel being processed. Further, the vertical axis of FIG. 5B is the edge amount EG, and the horizontal axis is the pixel position of the pixel being processed.
The edge vicinity detection unit 42 detects the vicinity area of the edge. For example, the edge vicinity detection unit 42 processes the edge amount detected for each predetermined area with a low-pass filter, and outputs a degree of proximity that becomes a large output when approaching the edge vicinity. Further, the edge proximity degree may be obtained by spatially shifting the edge amount EG in the horizontal and vertical directions or adding the averages. Further, the edge proximity degree may be obtained by processing the edge amount EG with the MAX filter. For example, as illustrated in FIG. 5C, the edge vicinity detection unit 42 outputs an edge vicinity degree that becomes a larger output as it is closer to the edge. Here, the vertical axis in FIG. 5C is the edge proximity, and the horizontal axis is the pixel position of the pixel being processed.

  In the flatness detection unit 43, it is more desirable to calculate the flatness FT using a luminance difference in a wider range than the edge detection unit 41 in order to determine that the flatness is flat. This is because when a human sees an image, he / she can recognize side effects such as feeling an unnatural image because he / she looks at a flat region having a certain large area. For example, when viewing an image spatially processed using a high-definition (HDTV) display device at a distance of 3H (H is the height of the screen), which is the optimum viewing distance, H / 20 (full HD (full spec high-definition)) (In the case of 1920 × 1080 pixels), it corresponds to 54 pixels vertically and horizontally.) It is desirable to detect the flatness FT in the above range. However, since the viewing distance is relatively shortened as the screen is enlarged, the flatness FT may be detected in a range of, for example, H / 30 or more (corresponding to 36 pixels vertically and horizontally). A certain degree of effect can be obtained by detecting the flatness FT in a range of at least H / 50 (corresponding to 22 pixels in the vertical and horizontal directions).

The flatness detection unit 43 detects the flatness (flatness FT) of the flat area where the luminance difference with the adjacent area is a predetermined value or less. For example, the area of a region where the luminance difference from the adjacent region is a predetermined value or less is obtained, and a larger flatness (flatness FT) is output as the area is larger.
Alternatively, as shown in FIG. 5D, the brightness difference from the adjacent area is detected from the output of the edge amount in FIG. 5B, and the larger flatness (flatness FT) is output as the brightness difference is smaller. May be. In FIG. 5D, the vertical axis is the flatness FT, and the horizontal axis is the pixel position of the pixel being processed.
As shown in FIG. 5 (e), the effect adjustment signal generator 44 multiplies the proximity in FIG. 5 (c) and the flatness in FIG. 5 (d) to obtain a large edge proximity and a high flatness. An effect adjustment signal MOD that weakens the visual effect is output. Here, the vertical axis of FIG. 5D is the output of the effect adjustment signal MOD, and the horizontal axis is the pixel position of the pixel being processed. In the visual processing device 1, the visual effect becomes stronger as the value of the effect adjustment signal MOD is larger.

As a result, as shown in FIG. 5E, the effect adjustment signal generating unit 44 performs an output that weakens the visual effect in the edge vicinity region, and the visual effect is applied to a region far from the edge vicinity region. The output which strengthens is generated. Further, the effect adjustment signal generation unit 44 performs an output that weakens the visual effect as the flatness degree increases in the region near the edge according to the flatness degree, and an output that strengthens the visual effect as the flatness degree decreases. .
13A shows the relationship between the pixel position and the image signal IS, FIG. 13B shows the relationship between the pixel position and the edge amount EG, and FIG. 13C shows the relationship between the pixel position and the edge proximity. FIG. 13D shows the relationship between the pixel position and the flatness FT, and FIG. 13E shows the relationship between the pixel position and the effect adjustment signal MOD. When an image signal IS as shown in FIG. 13A is input, the edge amount EG is shown in FIG. 13B, the edge proximity is shown in FIG. 13C, and the flatness FT is shown in FIG. FIG. 13 (e) shows the effect adjustment signal MOD in d). Here, a region indicated by EF in FIG. 13 is a flat portion near the edge. In the flat region EF near the edge, the effect adjustment signal MOD is small.

As a result, the visual processing device 1 can reduce side effects only in the vicinity of the edge, and can realize visual processing having an excellent visual processing effect on a natural image.
Next, the operation of the visual processing device 1 will be described with reference to FIG. FIG. 6 is a flowchart for explaining the operation of the visual processing device 1.
As shown in FIG. 6, an image is input to the visual processing device 1 (S101), and the edge detection unit 41 detects an edge amount that is a luminance difference for each adjacent region from the image signal IS (S102).
Next, the visual processing device 1 processes the edge amount with the low-pass filter by the edge vicinity detection unit 42, and detects the degree of proximity from the edge amount (S103). Further, the visual processing device 1 detects the luminance difference from the edge amount by the flatness detection unit 43, and detects the flatness degree in the vicinity of the edge (S104).

Next, in the visual processing device 1, the effect adjustment signal generation unit 44 multiplies the proximity level output from the edge vicinity detection unit 42 and the flatness level output from the flatness detection unit 43 to obtain the effect adjustment signal MOD. Generate (S105).
Next, in the visual processing device 1, the effect adjustment unit 20 generates a synthesized signal MUS that is synthesized by changing the ratio of the image signal IS and the unsharp signal US according to the effect adjustment signal MOD (S106).
Next, in the visual processing device 1, the visual processing unit 30 selects one of the curves of the two-dimensional gradation conversion characteristics shown in FIG. 2 corresponding to the composite signal MUS, and converts the image signal IS (S107). . Thereby, the visual processing device 1 executes visual processing adjusted so as to vary the effect of visual processing in accordance with the composite signal MUS.

Next, the visual processing device 1 determines whether there is a pixel to be processed next (S108). Next, when there is no pixel that needs processing, the visual processing is completed. On the other hand, if there is a pixel that needs to be processed next, the process returns to step S101, and the next image (pixel) is input. Thereafter, the steps from S101 to S108 are repeated until there are no more pixels that need to be processed.
As described above, according to the visual processing device 1 of Embodiment 1 of the present invention, side effects can be reduced only in the vicinity of edges, and visual processing having an excellent visual processing effect on natural images is realized. be able to.
The visual processing device 1 calculates the edge proximity from the edge amount and the flatness from the input image signal IS, and generates the effect adjustment signal MOD based on the edge proximity and the flatness. The effect adjustment signal MOD may be generated from the change amount of the unsharp signal US.

Hereinafter, a modified example of the control signal generating unit 40 will be described with respect to a method for detecting a flat region near the edge according to the modified example of the control signal generating unit 40.
An embodiment for generating the effect adjustment signal MOD from the amount of change in the unsharp signal US will be described with reference to FIGS. 7 and 14. FIG. 7A is a block diagram illustrating a configuration of a visual processing device 1 according to a modification example of the present embodiment. That is, a modification of the visual processing device 1 shown in FIG. 1 is shown. 1 differs from the visual processing device 1 of FIG. 1 in that the input of the control signal generator 70 (modified example of the control signal generator 40) is not the input signal IS but the unsharp signal US signal. FIG. 7B is a block diagram showing a configuration of a control signal generator 70 which is a modification of the control signal generator 40.
As shown in FIG. 7B, the control signal generation unit 70 has a change amount detection unit 71 that detects a change amount of the unsharp signal US, and an effect that outputs an effect adjustment signal MOD according to the detected change amount. And an adjustment signal generator 72.

The unsharp signal US is a signal in which a high-frequency signal component included in a natural image is deleted (reduced), but a steep edge component remains. For this reason, by extracting the edge peripheral region based on the edge component still remaining in the unsharp signal US, it is possible to detect a flat portion in the vicinity of the approximate edge.
In the present embodiment, the input of the control signal generator 70 is the unsharp signal US, and a flat portion near the edge is detected by obtaining the amount of change. For this reason, in flat detection where it is desirable to refer to a large area (image region), the image region to be referred to can be reduced, and the amount of processing required to detect a flat portion can be reduced.
The change amount detection unit 71 detects using an edge detection filter (not shown) such as a primary differential filter such as a Sobel filter or a Prewitt filter, or a secondary differential filter such as a Laplacian filter.

Further, in order to adjust the width of the edge peripheral area, it may be combined with a low-pass filter or a MAX filter.
For example, when the image signal IS shown in FIG. 14 (a) is input and an unsharp signal as shown in FIG. 14 (b) is obtained, the change amount detector 71 is shown in FIG. 14 (c). Thus, a large signal is output near the edge where the unsharp signal US changes. Here, the vertical axis of FIG. 14A is the value of the image signal IS, and the horizontal axis is the pixel position being processed. The vertical axis in FIG. 14B is the value of the unsharp signal US, and the horizontal axis is the pixel position being processed. The vertical axis of FIG. 14C is the value of the change amount of the unsharp signal US, and the horizontal axis is the pixel position being processed.
The effect adjustment signal generation unit 72 adjusts the output according to the change amount detected by the change amount detection unit 71. That is, the effect adjustment signal generator 72 outputs the effect adjustment signal MOD so that the signal level (value) of the effect adjustment signal MOD decreases as the change amount increases. For example, as shown in FIG. 8, the signal level of the effect adjustment signal MOD is changed within a range up to a predetermined value Thb by changing the signal level of the effect adjustment signal MOD when the detected amount of change is equal to or greater than a predetermined value Tha. Decrease level. Above the predetermined value Thb, the signal level of the effect adjustment signal MOD is not changed. Thereby, the signal level of the effect adjustment signal MOD can be changed when a steep edge region is input without reacting to the edge component normally included in the natural image. Here, the horizontal axis represents the amount of change, and the vertical axis represents the output (signal level) of the effect adjustment signal MOD. Note that the output range of the signal level of the output effect adjustment signal MOD is set to “0.0” to “1.0”, but “0.2” to “1.0” depending on the intensity of visual processing. You may make it adjust to. Further, the visual processing device 1 is configured such that the greater the signal level of the effect adjustment signal MOD, the stronger the visual processing effect in the visual processing device 1.

As shown in FIG. 14D, the effect adjustment signal generation unit 72 performs an output that weakens the visual effect in the flat region EF near the edge, and the visual effect is applied to a region far from the region near the edge. The output which strengthens is generated. By using the effect adjustment signal MOD generated by the effect adjustment signal generator 72, the flat area EF near the edge performs a process that weakens the visual effect, and the area far from the edge vicinity is visually It is possible to realize the visual processing device 1 that performs processing for enhancing the effect. Note that the vertical axis of FIG. 14D is the value of the effect adjustment signal MOD, and the horizontal axis is the pixel position being processed.
As described above, according to the control signal generator 70, the flat region near the edge can be detected from the amount of change in the unsharp signal US, and the effect adjustment signal MOD can be generated.
Note that a flat region near the edge is detected from a reduced image such as a thumbnail image in which the image signal is reduced, and an effect adjustment signal MOD is output based on the degree of flatness near the edge or the amount of change in the unsharp signal US. You may do it.

Further, a reduction processing unit (not shown) for reducing the image signal is provided between the image signal and the control signal generation unit 40, and the flatness degree near the edge or unsharpness from the reduced image generated by the reduction processing unit. The effect adjustment signal MOD may be output based on the change amount of the signal US.
By using the reduced image in this way, it is possible to detect a flat region near the edge while suppressing the influence of noise. In other words, the reduced image generated by the reduction method that thins out after averaging the image signal has a reduced noise component, so that the reduced image is used to detect a flat region near the edge while suppressing the influence of noise. be able to. If a reduced image is used, the number of pixels to be detected can be reduced and the amount of calculation can be reduced.
In addition, a low-pass filter or the like may be installed in front of the control signal generation unit 40 and the control signal generation unit 70 to limit the band of the image signal and detect a flat region near the edge. Thereby, the noise component can be reduced, and a flat region near the edge can be detected while suppressing the influence of noise.

(Embodiment 2)
In the first embodiment of the present invention, the composite signal MUS synthesized by changing the ratio of the image signal IS and the peripheral image information (unsharp signal) US according to the effect adjustment signal MOD is output, and the visual processing unit 30 is effective. Although the processing signal OS obtained by visually processing the image signal IS according to the composite signal MUS from the adjustment unit 20 is output, in the second embodiment of the present invention, the processing signal OS visually processed by the effect adjustment unit 21 is output. An embodiment in which a processing signal OS obtained by combining the image signal IS and the image signal IS according to the effect adjustment signal is output will be described with reference to FIG.
FIG. 9 is a block diagram showing a configuration of the visual processing device 2 according to Embodiment 2 of the present invention. Hereinafter, the same parts as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

In FIG. 9, the visual processing unit 30 outputs a processing signal OS based on the image signal IS and the output US of the spatial processing unit 10.
The effect adjustment unit 21 performs an internal calculation on the image signal IS and the processing signal OS according to the effect adjustment signal MOD, thereby changing (changing) the effect of visual processing. For example, the output OUT from the effect adjustment unit 21 is calculated by internal division calculation as shown in the following (Equation 3).
OUT = OS × MOD + IS × (1.0−MOD) (Formula 3)
(Equation 3) can also be realized by modifying it as (Equation 4).
OUT = (OS−IS) × MOD + IS (Formula 4)
As described above, according to the second embodiment of the present invention, the synthesized signal OUT synthesized by changing the ratio of the processing signal OS and the image signal IS in accordance with the effect adjustment signal MOD can be output. The effect can be changed (changed).

Note that the control signal generator 40 may be replaced with the control signal generator 70 in the first embodiment of the present invention. Also in this manner, the edge vicinity region can be detected in the same manner, and the effect adjustment signal MOD corresponding to the amount of change in the peripheral information in the vicinity of the edge can be generated.
(Embodiment 3)
In Embodiment 1 of the present invention, a composite signal MUS synthesized by changing the ratio of the image signal IS and the peripheral image information US according to the effect adjustment signal MOD is output, and the visual processing unit 30 receives the signal from the effect adjustment unit 20. Although the processing signal OS obtained by visually processing the image signal according to the composite signal MUS is output, in the third embodiment of the present invention, the effect adjustment unit 22 determines the effect of the visual processing according to the effect adjustment signal MOD. An embodiment for outputting the processed signal OS synthesized by changing the ratio of the outputs of the different visual processing units 31 and 32 will be described with reference to FIG.

FIG. 10 is a block diagram showing the configuration of the visual processing device 3 according to Embodiment 3 of the present invention. Hereinafter, the same parts as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.
The effect adjustment unit 22 outputs the output OSA of the visual processing unit 31 in which the first profile 60 having different visual processing intensity is set in the LUT according to the effect adjustment signal MOD output from the control signal generation unit 40, and the second OSA. The output OSB of the visual processing unit 32 in which the profile 61 is set to LUT is synthesized by internal division calculation, and the processing signal OS is output. Note that a composite output may be generated by external division calculation. At this time, the processing signal OS is as shown in (Equation 5).
OUT = OSA × MOD + OSB × (1.0−MOD) (Formula 5)
Note that (Equation 5) can also be realized by modifying it as (Equation 6).

OUT = (OSA−OSB) × MOD + OSB (Formula 6)
As described above, according to the third embodiment of the present invention, synthesis is performed by changing the ratio of the output of the visual processing unit 31 and the output of the visual processing unit 32 having different visual processing effects according to the effect adjustment signal MOD. By obtaining the synthesized output, visual processing with different degrees of visual effect can be performed.
Note that the control signal generator 40 may be replaced with the control signal generator 70 in the first embodiment of the present invention. Also in this manner, the edge vicinity region can be detected in the same manner, and the effect adjustment signal MOD corresponding to the amount of change in the peripheral information in the vicinity of the edge can be generated.
(Embodiment 4)
In the visual processing devices according to the first to third embodiments of the present invention, gradation conversion values based on the two-dimensional gradation conversion characteristics are output, but the fourth embodiment of the present invention. Now, a case where gradation conversion is performed using a gain signal will be described with reference to FIGS.

FIG. 11 is a block diagram showing the configuration of the gain-type visual processing system 4 according to Embodiment 4 of the present invention, and FIG. 12 is an explanatory diagram for explaining the two-dimensional gain characteristics. Hereinafter, the same parts as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.
In FIG. 11, the gain-type visual processing system 4 includes a gain-type visual processing device 5 that outputs a gain signal GAIN obtained by visually processing an image signal IS, and a multiplier 11 that multiplies the gain signal GAIN and the image signal IS. ing.
The gain-type visual processing device 5 includes a visual processing device 1 that outputs a processing signal OS obtained by visually processing the image signal IS, and a divider 12 that divides the processing signal OS by the image signal IS. Here, the visual processing device 1 outputs a gradation conversion value obtained by visually processing the output of the image signal IS, and the gain-type visual processing device 5 is obtained by dividing the gradation conversion value by the image signal IS. realizable.

The multiplier 11 multiplies the gain signal GAIN output from the gain-type visual processing device 5 and the image signal IS, and outputs a gradation conversion value obtained by visually processing the output of the image signal IS.
Note that the visual processing unit 30 may directly use a profile having a two-dimensional gain characteristic shown in FIG. Here, the vertical axis in FIG. 12 is the gain output GN, and the horizontal axis is the image signal IS. The two-dimensional gain characteristic shown in FIG. 12 is equivalent to the one obtained by dividing the output of the two-dimensional gradation characteristic profile shown in FIG. 2 by the image signal IS. The profile having the two-dimensional gain characteristic may be set in the LUT of the visual processing unit 30 of the visual processing device 1. As described above, if the profile of the two-dimensional gain characteristic is set in advance in the LUT of the visual processing unit 30, the gain signal GN and the gain signal GAIN are equivalent. Therefore, even if the divider 12 is deleted, the gain-type visual processing device 5 Can be realized.

In the gain-type visual processing device 5, since the change in the processing signal is small with respect to the change in the input image signal IS, the number of bits of the input signal can be reduced and the circuit scale can be reduced. Further, when the visual processing unit 30 is provided with a two-dimensional LUT, the memory capacity can be reduced.
As described above, according to the gain-type visual processing system 4 of the fourth embodiment of the present invention, by controlling the gain signal GAIN, gradation saturation can be easily suppressed and excellent visual processing is realized. can do.
The visual processing device 1 according to the first embodiment of the present invention may be replaced with the visual processing device 2 according to the second embodiment of the present invention. Also by this, the gain-type visual processing device 5 can be similarly realized.
The visual processing device 1 according to the first embodiment of the present invention may be replaced with the visual processing device 3 according to the third embodiment of the present invention. Also by this, the gain-type visual processing device 5 can be similarly realized.

As described above, according to the first to fourth embodiments of the present invention, even when an image having a steep edge region is input, visual processing with reduced side effects is realized. be able to.
Note that the visual processing device described in the above embodiment may be built in or connected to a device that handles moving images, and the effect adjustment signal MOD may be generated from an image for each frame or field. When the image signal is a frame image, the control signal generation unit 40 detects edge information and flatness from one (1 frame) or more previous frame image, or when the image signal is a field image, from one (1 field) or more previous field image. Information can be extracted. Thereby, the visual processing device can use the effect adjustment signal MOD corresponding to the edge information and the flatness information from the head of the frame. Further, the visual processing device can extract edge information and flatness information from the previous (one field previous) field image, and uses an effect adjustment signal MOD corresponding to the edge information and flatness information from the head of the field image. Can do. Further, the control signal generator 40 extracts the edge information and the flatness information from one or more previous (one frame) previous frame images or one (one field) or more previous field images, thereby reducing the circuit delay. It is easy to match, and the circuit scale can be reduced.

In addition, various functions such as a spatial processing function, an effect adjustment function, and a visual processing function in the visual processing device or the visual processing system according to the first to fourth embodiments of the present invention are integrated circuits. It may be implemented by hardware, or may be implemented by software that operates using a central processing unit (hereinafter referred to as “CPU”), a digital signal processing unit, or the like.
First, when various functions are implemented by hardware, each function in the embodiment of the present invention may be individually integrated circuits, or may be an integrated circuit integrated into one chip so as to include a part or all of them. Good.
The integrated circuit may be realized by a dedicated circuit or a general-purpose processor. For example, an FPGA (Field Programmable Gate Array) that can be programmed after manufacturing a semiconductor chip, or a reconfigurable processor that can reconfigure the connection and setting of cells inside the integrated circuit may be used.

Further, if integrated circuit technology comes out as a result of advances in semiconductor technology or other derived technology, it is naturally also possible to carry out function block integration using this technology. For example, biocomputers may be applied due to advances in biotechnology.
Next, a case where various functions are implemented by software will be described with reference to FIG. FIG. 15 is a block diagram showing a configuration of a computer according to the embodiment of the present invention.
In FIG. 15, a computer 6 includes a CPU 100 for executing instructions of various programs, a read-only memory 101 (hereinafter referred to as “ROM 101”) in which programs are stored, and a random access memory 102 in which data in temporary storage is stored. (Hereinafter referred to as “RAM 102”), an input unit 103 for inputting an image, an output unit 104 for outputting an image, and a storage unit 105 for storing programs and various data.

Furthermore, a communication unit 106 that performs communication with the outside and a drive 107 that appropriately connects an information storage medium may be provided.
Each functional unit transmits and receives control signals and data via the bus 110.
The CPU 100 executes various functions according to programs stored in the ROM 101.
The ROM 101 stores programs, profiles, and the like.
The RAM 102 temporarily stores data necessary for processing of various functions by the CPU 100.
The input unit 103 inputs an image. For example, a video signal is acquired by receiving a radio wave and decoding the received signal. Further, a digital image may be acquired directly via a wire.

The output unit 104 outputs an image. For example, the data is output to a display device such as a liquid crystal display device or a plasma display.
The storage unit 105 is configured with a magnetic memory or the like, and stores various programs and data.
The communication unit 106 may be connected to the network 111 and acquire the program via the network 111 or install the acquired program in the storage unit 105 as necessary. As a result, the computer 6 can download the program through the communication unit 106.
The drive 107 connects an information storage medium as appropriate, and acquires storage information stored in the information storage medium. The information storage medium is, for example, a disk 108 such as a magnetic disk, a magneto-optical disk, or an optical disk, or a memory card 109 such as a semiconductor memory.

A program for executing various functions, a profile, or the like may be stored in the disk 108 or a memory card 109 such as a semiconductor memory, and the information may be given to the computer 6.
The program may be incorporated in advance in a computer with dedicated hardware, or provided in advance in the ROM 101 and the storage unit 105.
The program can be applied to devices that handle images, such as information processing apparatuses, televisions, digital cameras, mobile phones, and PDAs. The program is incorporated in or connected to a device that handles images, and executes visual processing with reduced side effects on a flat region near the edge.
The visual processing device may be built in or connected to a device that handles moving images, and may generate the effect adjustment signal MOD from the image for each frame or field. The control signal generator 40 can extract edge information and flatness information from one or more previous frame images when the image signal is a frame image, or from one or more previous field images when the image signal is a field image. Thereby, the visual processing device can use the effect adjustment signal MOD corresponding to the edge information and the flatness information from the head of the frame. The visual processing device 1 can extract edge information and flatness information from the previous field image, and can use the effect adjustment signal MOD corresponding to the edge information and flatness information from the head of the field image. In addition, the control signal generation unit 40 extracts the edge information and the flatness information from one or more previous frame images or one or more previous field images, thereby easily adjusting the circuit delay and reducing the circuit scale. it can.

(Embodiment 5)
Next, as a fifth embodiment, an application example of the visual processing device described in the above embodiment and a system using the same will be described with reference to FIGS.
FIG. 16 is a block diagram showing an overall configuration of a content supply system ex100 that implements a content distribution service. The communication service providing area is divided into desired sizes, and base stations ex107 to ex110, which are fixed wireless stations, are installed in each cell.
This content supply system ex100 includes, for example, a computer ex111, a PDA (Personal Digital Assistant) ex112, a camera ex113, a mobile phone ex114, a camera via the Internet ex101, the Internet service provider ex102, the telephone network ex104, and the base stations ex107 to ex110. Each device such as the attached mobile phone ex115 is connected.

However, the content supply system ex100 is not limited to the combination as shown in FIG. 16, and any of the combinations may be connected. Also, each device may be directly connected to the telephone network ex104 without going through the base stations ex107 to ex110 which are fixed wireless stations.
The camera ex113 is a device capable of shooting a moving image such as a digital video camera. In addition, the mobile phone may be a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access) system, or a GSM (Global Mobile System). Alternatively, PHS (Personal Handyphone System) may be used.

  Further, the streaming server ex103 is connected from the camera ex113 through the base station ex109 and the telephone network ex104, and live distribution based on the encoded data transmitted by the user using the camera ex113 becomes possible. The encoded processing of the captured data may be performed by the camera ex113 or may be performed by a server or the like that performs data transmission processing. The moving image data shot by the camera ex116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device such as a digital camera that can shoot still images and moving images. In this case, the moving image data may be encoded by the camera ex116 or the computer ex111. The encoding process is performed in the LSI ex117 included in the computer ex111 and the camera ex116. Note that image encoding / decoding software may be incorporated into any storage medium (CD-ROM, flexible disk, hard disk, or the like) that is a recording medium readable by the computer ex111 or the like. Furthermore, you may transmit moving image data with the mobile phone ex115 with a camera. The moving image data at this time is data encoded by the LSI included in the mobile phone ex115.

  In the content supply system ex100, the content (for example, video shot of music live) captured by the user with the camera ex113, the camera ex116, and the like is encoded and transmitted to the streaming server ex103, while the streaming server ex103. Distributes the content data to the requested client. Examples of the client include a computer ex111, a PDA ex112, a camera ex113, and a mobile phone ex114 that can decode the encoded data. In this way, the content supply system ex100 can receive and play back the encoded data at the client, and can also receive a private broadcast by receiving, decoding, and playing back at the client in real time. It is a system that becomes possible.

When displaying content, the visual processing device described in the embodiment may be used. For example, the computer ex111, the PDA ex112, the camera ex113, the mobile phone ex114, and the like may be provided with the visual processing device described in the above-described embodiment, and realize a visual processing method and a visual processing program.
The streaming server ex103 may provide the two-dimensional gain data (profile) to the visual processing device via the Internet ex101. Furthermore, there may be a plurality of streaming servers ex103 that provide different two-dimensional gain data. Furthermore, the streaming server ex103 may create two-dimensional gain data. As described above, when the visual processing device can acquire the two-dimensional gain data via the Internet ex101, the visual processing device does not need to store the two-dimensional gain data used for the visual processing in advance, and stores the visual processing device. It is also possible to reduce the capacity. Further, since two-dimensional gain data can be acquired from a plurality of servers connected via the Internet ex101, different visual processing can be realized.

A mobile phone will be described as an example.
FIG. 17 is a diagram illustrating a mobile phone ex115 including the visual processing device 1 according to the embodiment. The mobile phone ex115 includes an antenna ex201 for transmitting and receiving radio waves to and from the base station ex110, a video from a CCD camera, a camera unit ex203 capable of taking a still image, a video shot by the camera unit ex203, and an antenna ex201. A display unit ex202 such as a liquid crystal display for displaying data obtained by decoding received video and the like, a main body unit composed of a group of operation keys ex204, an audio output unit ex208 such as a speaker for audio output, and audio input To store encoded data or decoded data such as a voice input unit ex205 such as a microphone, captured video or still image data, received mail data, video data or still image data, etc. Recording media ex207, mobile phone ex115 recording media ex And a slot portion ex206 that enables mounting 07. The recording medium ex207 stores a flash memory element, which is a kind of EEPROM (Electrically Erasable and Programmable Read Only Memory), which is a nonvolatile memory that can be electrically rewritten and erased, in a plastic case such as an SD card.

Further, the cellular phone ex115 will be described with reference to FIG. The mobile phone ex115 controls the main control unit ex311 which controls the respective units of the main body unit including the display unit ex202 and the operation key ex204. The power supply circuit unit ex310, the operation input control unit ex304, and the image encoding Unit ex312, camera interface unit ex303, LCD (Liquid Crystal Display) control unit ex302, image decoding unit ex309, demultiplexing unit ex308, recording / playback unit ex307, modulation / demodulation circuit unit ex306, and audio processing unit ex305 via a synchronization bus ex313 Are connected to each other.
When the end of call and the power key are turned on by a user operation, the power supply circuit ex310 activates the camera-equipped digital mobile phone ex115 by supplying power from the battery pack to each unit. .

The mobile phone ex115 converts the voice signal collected by the voice input unit ex205 in the voice call mode into digital voice data by the voice processing unit ex305 based on the control of the main control unit ex311 including a CPU, ROM, RAM, and the like. The modulation / demodulation circuit unit ex306 performs spread spectrum processing, the transmission / reception circuit unit ex301 performs digital analog conversion processing and frequency conversion processing, and then transmits the result via the antenna ex201. In addition, the cellular phone ex115 amplifies the received signal received by the antenna ex201 in the voice call mode, performs frequency conversion processing and analog-digital conversion processing, performs spectrum despreading processing by the modulation / demodulation circuit unit ex306, and analog audio by the voice processing unit ex305. After conversion into a signal, this is output via the audio output unit ex208.
Further, when an e-mail is transmitted in the data communication mode, text data of the e-mail input by operating the operation key ex204 of the main body is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spread spectrum processing on the text data in the modulation / demodulation circuit unit ex306, performs digital analog conversion processing and frequency conversion processing in the transmission / reception circuit unit ex301, and then transmits the text data to the base station ex110 via the antenna ex201.

When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image encoding unit ex312 via the camera interface unit ex303. When image data is not transmitted, the image data captured by the camera unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.
The image encoding unit ex312 converts the image data supplied from the camera unit ex203 into encoded image data by compression encoding, and sends the encoded image data to the demultiplexing unit ex308. At the same time, the cellular phone ex115 sends the sound collected by the audio input unit ex205 during imaging by the camera unit ex203 to the demultiplexing unit ex308 as digital audio data via the audio processing unit ex305.

The demultiplexing unit ex308 multiplexes the encoded image data supplied from the image encoding unit ex312 and the audio data supplied from the audio processing unit ex305 by a predetermined method, and the resulting multiplexed data is a modulation / demodulation circuit unit Spread spectrum processing is performed in ex306, digital analog conversion processing and frequency conversion processing are performed in the transmission / reception circuit unit ex301, and then transmitted through the antenna ex201.
When data of a moving image file linked to a home page or the like is received in the data communication mode, the received signal received from the base station ex110 via the antenna ex201 is subjected to spectrum despreading processing by the modulation / demodulation circuit unit ex306, and the resulting multiplexing is obtained. Data is sent to the demultiplexing unit ex308.
In addition, in order to decode the multiplexed data received via the antenna ex201, the demultiplexing unit ex308 separates the multiplexed data to generate an encoded bitstream of image data and an encoded bitstream of audio data. The encoded image data is supplied to the image decoding unit ex309 via the synchronization bus ex313, and the audio data is supplied to the audio processing unit ex305.

Next, the image decoding unit ex309 generates reproduction moving image data by decoding the encoded bit stream of the image data, and supplies this to the display unit ex202 via the LCD control unit ex302, thereby For example, moving image data included in a moving image file linked to a home page is displayed. At the same time, the audio processing unit ex305 converts the audio data into an analog audio signal and then supplies the analog audio signal to the audio output unit ex208. Thus, for example, the audio data included in the moving image file linked to the home page is reproduced. The
In the above configuration, the image decoding unit ex309 may include the visual processing device of the above embodiment.
In addition, the present invention is not limited to the above system, and recently, digital broadcasting by satellite and terrestrial has become a hot topic. As shown in FIG. 19, the visual processing device described in the above embodiment is also incorporated in the digital broadcasting system. Can do. Specifically, in the broadcasting station ex409, a coded bit stream of video information is transmitted to a communication or broadcasting satellite ex410 via radio waves. Receiving this, the broadcasting satellite ex410 transmits a radio wave for broadcasting, and receives the radio wave with a home antenna ex406 having a satellite broadcasting receiving facility, such as a television (receiver) ex401 or a set top box (STB) ex407. The device decodes the encoded bit stream and reproduces it. Here, a device such as the television (receiver) ex401 or STBex407 may include the visual processing device described in the above embodiment. Moreover, you may use the visual processing method of the said embodiment. Furthermore, a visual processing program may be provided. In addition, the visual processing device, the visual processing method, and the visual processing program described in the above embodiment are also applied to a playback device ex403 that reads and decodes an encoded bitstream recorded on a storage medium ex402 such as a CD or DVD that is a recording medium. It is possible to implement. In this case, the reproduced video signal is displayed on the monitor ex404. In addition, the visual processing device, the visual processing method, and the visual processing program described in the above embodiment are installed in the STBex 407 connected to the cable ex405 for cable television or the antenna ex406 for satellite / terrestrial broadcasting. A configuration of reproducing with ex408 is also conceivable. At this time, the visual processing device described in the above embodiment may be incorporated in the television instead of the STB. It is also possible to receive a signal from the satellite ex410 or the base station ex107 by the car ex412 having the antenna ex411 and reproduce a moving image on a display device such as the car navigation ex413 that the car ex412 has.

Furthermore, an image signal can be encoded and recorded on a recording medium. Specific examples include a recorder ex420 such as a DVD recorder that records image signals on a DVD disk ex421 and a disk recorder that records images on a hard disk. Further, it can be recorded on the SD card ex422. If the recorder ex420 includes the visual processing device of the above embodiment, the image signal recorded on the DVD disc ex421 or the SD card ex422 can be interpolated and reproduced and displayed on the monitor ex408.
The configuration of the car navigation ex413 may be a configuration excluding the camera unit ex203, the camera interface unit ex303, and the image encoding unit ex312 in the configuration illustrated in FIG. 18, for example, and the same applies to the computer ex111 and the television (reception) Machine) Ex401 can also be considered.

In addition to the transmission / reception type device having both the encoder and the decoder, the mobile phone ex114 and the like are mounted in three mounting formats: a transmitter only with an encoder and a receiver only with a decoder. Can be considered.
The specific configuration of the present invention is not limited to the above-described embodiment, and various changes and modifications can be made without departing from the scope of the invention.

  According to the visual processing device, the visual processing method, the program, the recording medium, the display device, and the integrated circuit according to the present invention, the image signal can be visually processed, particularly when an image having a steep edge region is input. Even so, since side effects can be suppressed, it is useful in the field of video / image processing devices, and the visual processing device, visual processing method, program, recording medium, display device and integrated circuit according to the present invention are: It can be implemented in the field.

The block diagram which shows the structure of the visual processing apparatus of Embodiment 1 of this invention. Explanatory drawing explaining the same two-dimensional gradation characteristics Explanatory drawing explaining the output of the processing signal OS Block diagram showing the configuration of the control signal generator Explanatory drawing explaining the output of the same effect adjustment signal Flow chart for explaining the operation of the visual processing device The block diagram which shows the structure of the control signal generation part of the modification Explanatory drawing explaining the effect adjustment signal of the modification The block diagram which shows the structure of the visual processing apparatus in Embodiment 2 of this invention. The block diagram which shows the structure of the visual processing apparatus in Embodiment 3 of this invention. The block diagram which shows the structure of the visual processing system in Embodiment 4 of this invention. Explanatory drawing explaining the same two-dimensional gain characteristic Explanatory drawing explaining the output of the effect adjustment signal of the visual processing apparatus in Embodiment 1 of this invention Explanatory drawing explaining the output of the effect adjustment signal of the visual processing apparatus in Embodiment 1 of this invention The block diagram which shows the structure of the computer in embodiment of this invention Overall configuration diagram of a content supply system in Embodiment 5 of the present invention Front view of a mobile phone equipped with the visual processing device according to the embodiment A block diagram for explaining the overall configuration of the mobile phone according to the embodiment Explanatory drawing about the whole structure of the system for digital broadcasting in the same embodiment

Explanation of symbols

1, 2, 3 Visual processing device 4 Gain-type visual processing system 5 Gain-type visual processing device 10 Spatial processing unit 11 Multiplier 12 Divider 20, 21, 22, Effect adjustment unit 30, 31, 32 Visual processing unit 40, 70 Control Signal generator 41 Edge detector 42 Edge vicinity detector 43 Flatness detector 44, 72 Effect adjustment signal generator 60 Profile A
61 Profile B
71 Change amount detection unit 100 CPU
101 ROM
102 RAM
DESCRIPTION OF SYMBOLS 103 Input part 104 Output part 105 Storage part 106 Communication part 107 Drive 108 Disk 109 Memory card 110 Bus 111 Network

Claims (11)

A spatial processing unit that outputs peripheral image information derived from pixel values of an input image signal and peripheral pixels ;
A visual processing unit that performs visual processing on the image signal based on the image signal and the peripheral image information;
A control signal generator for outputting an effect adjustment signal for setting the effect of the visual processing;
An effect adjustment unit that sets the effect of the visual processing according to the effect adjustment signal;
With
The visual processing unit performs visual processing for adjusting a local contrast on the image signal based on the image signal and the peripheral image information,
The control signal generator determines an edge vicinity region and a flat portion region included in an image region of an image formed by the image signal;
Generating the effect adjustment signal that realizes the visual processing based on the image signal and the peripheral image information for the image region that is determined not to be an edge vicinity region or a flat region;
Realizing the visual processing with an effect weaker than the effect of the visual processing based on the image signal and the peripheral image information for the image region determined to be a flat region and an edge vicinity region Generating the effect adjustment signal;
Visual processing device. The control signal generation unit outputs the effect adjustment signal according to a change amount of the peripheral image information.
The visual processing device according to claim 1 . The control signal generator is
A change amount detection unit for detecting a change amount of the peripheral image information;
An effect adjustment signal generation unit that generates the effect adjustment signal based on the change amount detected by the change amount detection unit;
Having
The visual processing device according to claim 1 . The control signal generator is
A flatness detecting unit for detecting a flatness degree of a flat area where a luminance difference with an adjacent area is a predetermined value or less from the image signal;
An edge detection unit that detects an edge amount of an edge region in which a luminance difference with an adjacent region is a predetermined value or more from the image signal;
An edge vicinity detection unit that calculates an edge vicinity degree indicating an edge vicinity degree of the image area based on the edge amount;
An effect adjustment signal generating unit that generates the effect adjustment signal based on the flatness detected by the flatness detection unit and the edge proximity calculated by the edge vicinity detection unit,
The visual processing device according to claim 1 . The effect adjustment unit outputs a first synthesized signal obtained by synthesizing the image signal and the peripheral image information according to the effect adjustment signal,
The visual processing unit performs visual processing on the image signal based on the first synthesized signal and the image signal;
The visual processing device according to any one of claims 1 to 4 . The effect adjustment unit outputs a second synthesized signal obtained by synthesizing the image signal and the output subjected to visual processing by the visual processing unit in accordance with the effect adjustment signal.
The visual processing device according to any one of claims 1 to 4 . A spatial processing step for outputting peripheral image information derived from the pixel values of the input image signal image signal and the peripheral pixels ;
A visual processing step of performing visual processing on the image signal based on the image signal and the peripheral image information;
A control signal generation step of outputting an effect adjustment signal for setting the effect of the visual processing;
An effect adjustment step of setting the effect of the visual processing according to the effect adjustment signal;
With
In the visual processing step, visual processing for adjusting a local contrast is performed on the image signal based on the image signal and the peripheral image information,
In the control signal generation step, an edge vicinity region and a flat portion region included in an image region of an image formed by the image signal are determined,
Generating the effect adjustment signal that realizes the visual processing based on the image signal and the peripheral image information for the image region that is determined not to be an edge vicinity region or a flat region;
Realizing the visual processing with an effect weaker than the effect of the visual processing based on the image signal and the peripheral image information for the image region determined to be a flat region and an edge vicinity region Generating the effect adjustment signal;
Visual processing method. On the computer,
A spatial processing step of outputting peripheral image information derived from pixel values of an input image signal and peripheral pixels ;
A visual processing step of performing visual processing on the image signal based on the image signal and the peripheral image information;
A control signal generation step of outputting an effect adjustment signal for setting the effect of the visual processing;
An effect adjustment step of setting the effect of the visual processing according to the effect adjustment signal;
A program for executing
In the visual processing step, visual processing for adjusting a local contrast is performed on the image signal based on the image signal and the peripheral image information,
In the control signal generation step, an edge vicinity region and a flat portion region included in an image region of an image formed by the image signal are determined,
Generating the effect adjustment signal that realizes the visual processing based on the image signal and the peripheral image information for the image region that is determined not to be an edge vicinity region or a flat region;
Realizing the visual processing with an effect weaker than the effect of the visual processing based on the image signal and the peripheral image information for the image region determined to be a flat region and an edge vicinity region Generating the effect adjustment signal;
program. On the computer,
A spatial processing step of outputting peripheral image information derived from pixel values of an input image signal and peripheral pixels ;
A visual processing step of performing visual processing on the image signal based on the image signal and the peripheral image information;
A control signal generation step of outputting an effect adjustment signal for setting the effect of the visual processing;
An effect adjustment step of setting the effect of the visual processing according to the effect adjustment signal;
A recording medium recording a program for executing
In the visual processing step, visual processing for adjusting a local contrast is performed on the image signal based on the image signal and the peripheral image information,
In the control signal generation step, an edge vicinity region and a flat portion region included in an image region of an image formed by the image signal are determined,
Generating the effect adjustment signal that realizes the visual processing based on the image signal and the peripheral image information for the image region that is determined not to be an edge vicinity region or a flat region;
Realizing the visual processing with an effect weaker than the effect of the visual processing based on the image signal and the peripheral image information for the image region determined to be a flat region and an edge vicinity region. Generating the effect adjustment signal;
recoding media. A processor used in an image processing apparatus,
The processor follows the instructions of the recorded program,
A spatial processing step of outputting peripheral image information derived from pixel values of input image signal peripheral pixels ;
A visual processing step of performing visual processing on the image signal based on the image signal and the peripheral image information;
A control signal generation step of outputting an effect adjustment signal for setting the effect of the visual processing;
An effect adjustment step of setting the effect of the visual processing according to the effect adjustment signal;
Run
In the visual processing step, visual processing for adjusting a local contrast is performed on the image signal based on the image signal and the peripheral image information,
In the control signal generation step, an edge vicinity region and a flat portion region included in an image region of an image formed by the image signal are determined,
Generating the effect adjustment signal that realizes the visual processing based on the image signal and the peripheral image information for the image region that is determined not to be an edge vicinity region or a flat region;
Realizing the visual processing with an effect weaker than the effect of the visual processing based on the image signal and the peripheral image information for the image region determined to be a flat region and an edge vicinity region. Generating the effect adjustment signal;
Processor. A data receiving unit for receiving image data transmitted or broadcast;
A decoding unit for decoding the received image data into video data;
The visual processing device according to any one of claims 1 to 6 , wherein the decoded video data is visually processed to output an output signal;
A display unit for displaying the output signal visually processed by the visual processing device;
A display device comprising:

Images