JP5193089B2 - Signal processing apparatus and signal processing method - Google Patents

Description

  The present invention relates to a signal processing apparatus and a signal processing method for performing signal processing on an image (video) formed by an image (video) signal. In particular, a signal that changes the effect of contour correction processing, band limiting processing, noise reduction processing, etc. in a portion (specific portion) included in a specific luminance signal region or color difference signal region in an image and a portion other than the specific portion. The present invention relates to a signal processing apparatus and a signal processing method for performing processing.

In conventional imaging devices, when imaging a subject, in order to improve the resolution and sharpness of the captured image, signal processing of the imaging device and display device emphasizes the contours of the subject and the edges of fine patterns. Contour correction processing is performed. In recent years, the resolution has been increased due to the increase in the number of pixels of the image sensor mounted on the imaging apparatus, and even a finer part of the subject has been expressed on the captured image acquired by the imaging apparatus. ing.
On the other hand, the pixel size is miniaturized as the number of pixels of the image sensor increases. For this reason, the amount of noise for the signal acquired by the image sensor increases, and as a result, the S / N ratio (Signal to Noise Ratio) of the signal acquired by the image sensor deteriorates. In a conventional imaging device, noise reduction processing for improving the deterioration of the S / N ratio (S / N ratio improvement) is also performed.

Generally, in an imaging apparatus, it is preferred that an image (video) with improved resolution is acquired by enhancing the details of the image by contour correction processing. However, when imaging an image of a person as a subject, especially when imaging a woman's face, etc., contour correction processing can enhance facial wrinkles, spots, acne, and rough skin. Therefore, there are many cases where the subject woman or the user of the imaging apparatus dislikes this. Therefore, in the conventional imaging device, for example, a portion in which the color difference signal is a skin color signal region is detected in the image, and the portion is determined to be the skin of a person, and the effect of the contour correction processing is suppressed. By enhancing the effect of the processing or the noise reduction processing, the processing of hiding the undesired portion is performed while maintaining the resolution of the other portions.
On the other hand, for example, leaves and lawns represent details in more detail, so the conventional imaging device detects a portion where the color difference signal is a green signal region, and the portion further improves the effect of the contour correction processing. There is a case where processing such as enhancement processing is performed and the effect of noise reduction processing is reduced.

The “signal region” refers to a color space (or a color plane) defined by the signal format of a signal used when detecting a region (for example, an RGB color space, a YCbCr color space, a YPbPr color space, or xvYCC). Color space, Lab color space (La * b * color space), IQ color plane, PbPr color difference plane, CbCr color difference plane, ab color plane of Lab color space (a * b * color plane of La * b * color space)) The range occupied by the video signal. In the above case, for example, a color area corresponding to skin color or green in the color difference plane represented by the Pb axis, the Pr axis, the I axis, and the Q axis corresponds to the “signal area”.
In addition, the “part where the color difference signal is a skin color or green signal area” means the part where the video signal included in the area corresponding to skin color or green on the color difference plane is located in the image space, that is, in the screen (on the image) For example, an image region in which a subject image such as a human face or a leaf is formed on an image formed by a video signal corresponds to this.

However, the boundary correction between the specific part of the image and the part adjacent to the specific part, for example, the boundary part between the face of the person and the background, the effects of the contour correction process and the noise reduction process as described above. An abrupt change will result in an unnatural image.
In order to solve this problem, a method of smoothly changing the effect of contour correction and noise reduction at the boundary between the specific part and the part adjacent to the specific part to prevent the image quality from being uncomfortable. It is done. There exists patent document 1 as such a conventional signal processing method. In the technique disclosed in Patent Document 1, whether or not a pixel of interest is included in a specific color region is indicated by a specific color determination flag, and pixels on the periphery of the pixel of interest (for example, 3 × 3) on the image plane. In the case of using a filter, whether or not 8 pixels around the pixel of interest is also included in the specific color region is similarly indicated by a specific color determination flag. Then, contour correction and noise removal are performed based on the values of the specific color determination flags of the pixel of interest and its surrounding pixels (a total of 9 pixels including the pixel of interest and the surrounding 8 pixels when a 3 × 3 filter is used). Thereby, in the technique disclosed in Patent Document 1, the effect of contour correction and noise removal is prevented from changing abruptly at the boundary portion between the specific color region and the region other than the specific color.
Japanese Patent No. 4200745

However, in the conventional signal processing method described above, it is necessary to determine whether the peripheral pixels of the target pixel in addition to the target pixel are simultaneously included in the specific color region. Since the peripheral pixels include not only horizontal pixels but also vertical pixels, for example, when a 3 × 3 filter is used, a specific color determination flag indicating whether or not a specific color region is included is set to 9 It is necessary to simultaneously acquire specific color determination flags for nine pixels at the same time (synchronization of vertical signals), that is, at a predetermined timing (time). In order to synchronize the specific color determination flag with 9 pixels, a signal processing apparatus that executes a conventional signal processing method needs to employ one of the following two configurations.
(1) The conventional signal processing apparatus includes two extra lines for the specific color determination flag (for example, includes two line memories) in the same manner as the video signal after generating the specific color determination flag. . As a result, 9 pixels are synchronized.
(2) In order to generate a specific color determination flag by simultaneously determining whether or not each of the 9 pixels is included in the specific color area from the color difference signal obtained by synchronizing the 9 pixels, the conventional signal processing device In addition to the pixel for the target pixel, eight extra specific color area detection circuits are provided (for the eight peripheral pixels of the target pixel).

Regardless of which of the above two configurations is employed, a memory or a specific color area detection circuit for synchronization is required separately, which increases the circuit scale.
The present invention solves the above-described conventional problems, without causing an increase in circuit scale, and without abruptly changing the processing effect at the boundary portion between a specific portion of the subject in the image and its adjacent portion. It is an object of the present invention to realize a signal processing device, a signal processing method, a program, and an integrated circuit that can acquire an image without a sense of incongruity by changing in stages.

A first invention is a signal processing device including a space calculation processing unit, a region detection unit, and an image processing unit.
The spatial calculation processing unit performs predetermined calculation processing on a target pixel included in an image formed by the first image signal and using a peripheral pixel that is a pixel around the target pixel on the image. Thus, the second image signal is generated from the first image signal. The region detection unit determines whether or not the target pixel of the second image signal is included in the predetermined signal region, and generates a region determination flag indicating the determination result. The image processing unit performs predetermined image processing on the first image signal based on the region determination flag generated by the region detection unit.
In this signal processing device, signal processing of a plurality of pixels adjacent to the target pixel is performed by performing arithmetic processing with the adjacent pixel before determining whether or not the target pixel of the video signal is included in the predetermined signal region. Therefore, even if the target pixel is a pixel included in the boundary portion other than the predetermined signal region and the predetermined signal region, the region determination flag does not change abruptly, and the predetermined image Processing can be performed in stages (gradually).

Therefore, the image (video) acquired by the signal processing apparatus changes in a stepwise (gradual) manner in the effect of the predetermined image processing even in a predetermined signal region and a boundary portion other than the predetermined signal region. Therefore, it becomes a natural image (video). For example, by setting the “predetermined signal area” as a “skin color area”, it is possible to effectively suppress emphasis on facial wrinkles, stains, acne, rough skin, etc. when imaging a female face or the like. In both cases, a sense of detail can be maintained in an image region other than the “skin color region”, so that a natural image (video) preferred by the woman or user of the subject can be acquired by this signal processing device.
The “first image signal” and the “second image signal” are concepts including not only a signal that can form an image (image signal) but also a signal that can form an image (video signal). .

In addition, the “signal region” refers to a color space (or a color plane) defined by a signal format of a signal used when detecting a region (for example, RGB color space, YCbCr color space, YPbPr color space, xvYCC). Color space, Lab color space (La * b * color space), IQ color plane, PbPr color difference plane, CbCr color difference plane, ab color plane of Lab color space (a * b * color plane of La * b * color space)) The range occupied by the image signal. For example, a color area corresponding to skin color or green on the color difference plane represented by the Pb axis, the Pr axis, the I axis, and the Q axis corresponds to the “signal area”.
2nd invention is 1st invention, Comprising: The space arithmetic processing part is at least one direction of the horizontal direction of the attention pixel, the vertical direction, the diagonal direction, and the time direction on the image space which a 1st image signal forms. Predetermined arithmetic processing is performed using peripheral pixels including pixels adjacent to.

In this signal processing device, pixels that are adjacent in the horizontal direction of the image if arithmetic processing is performed between pixels adjacent in the horizontal direction, and pixels that are adjacent in the oblique direction in the vertical direction of the image if arithmetic processing is performed between pixels adjacent in the vertical direction. If the calculation processing is performed between the pixels, the region determination flag at the boundary portion of the subject does not change abruptly in the oblique direction of the image, and the calculation processing is performed between pixels adjacent in the time direction, that is, between image frames. For example, even if the subject changes suddenly, such as a scene change, the region determination flag does not change rapidly, and predetermined image processing is performed stepwise (gradually).
A third invention is the first or second invention, and further includes a delay line unit that simultaneously synchronizes the target pixel of the first image signal and the peripheral pixels in the vertical direction of the target pixel. The spatial arithmetic processing unit performs predetermined arithmetic processing using the first image signal synchronized by the delay line unit. The image processing unit performs predetermined image processing on the first image signal synchronized by the delay line unit.

In this signal processing apparatus, since the delay line unit can be shared in the processing of the image processing unit and the processing of the spatial arithmetic processing unit, the circuit scale can be reduced when it is configured by hardware.
The fourth invention is any one of the first to third inventions, and the spatial calculation processing unit performs a predetermined calculation process by executing a spatial filter process on the pixel of interest.
Thereby, for example, by making the spatial filter processing LPF (low-pass filter) processing, the change in the boundary portion of the image (video) signal used when detecting the signal region can be made smoother. Since the change of the region determination flag becomes smoother, predetermined image processing can be performed stepwise (gradually).
A fifth invention is any one of the first to fourth inventions, wherein the region determination flag is composed of a plurality of bits, the signal region is expressed by a plurality of codes, and the first signal is a predetermined signal region. A region, a second signal region that is a signal region other than the first signal region, and a boundary portion signal region that is a signal region at a boundary portion between the first signal region and the second signal region are stepwise with different codes. It is a flag to represent.

Thus, for example, the area determination flag is composed of 2 bits, a predetermined signal area is expressed by a code “3”, a signal area other than the predetermined signal area is expressed by a code “0”, By expressing the signal area between them in the order of “2” and “1” in order from the closest to the predetermined signal area, the boundary portion of the predetermined signal area is compared with the case where the area determination flag is composed of 1 bit. Therefore, the predetermined image processing can be performed more finely and stepwise.
A sixth invention is any one of the first to fifth inventions, wherein the image processing unit controls in steps the effect of contour correction applied to the first image signal based on the code indicated by the region determination flag. The first image signal based on the contour correction unit, the band limiting unit that controls the band limitation effect applied to the first image signal stepwise based on the code indicated by the region determination flag, and the code indicated by the region determination flag At least one of noise reduction units for controlling the effect of noise reduction applied to the above in a stepwise manner.

Thus, when the image processing is contour correction for the boundary portion between the portion included in the predetermined signal region and the adjacent portion, the size of the contour correction signal to be added to the image (video) signal is stepped. (Gradually) can be changed. When the image processing is band limited, the frequency band of the image (video) signal itself can be changed stepwise (gradually). When the image processing is noise reduction, the noise component included in the image (video) signal can be changed stepwise (gradually).
A seventh invention is any one of the first to sixth inventions, wherein the predetermined signal region is at least one of a predetermined luminance signal region and a color difference signal region.
As a result, when the detection is performed in the luminance signal area, the effect of the image processing can be controlled according to the luminance level of the subject. When the detection is performed in the color difference signal area, the image processing is performed according to the color of the subject. The effect can be controlled.

The eighth invention is any one of the first to eighth inventions, wherein the first image signal and the second image signal are formed from a luminance signal and a color difference signal. The region detection unit determines whether or not the target pixel of the second image signal is included in the predetermined signal region, and determines the luminance region and the second image signal that are defined based on the luminance signal that forms the second image signal. This is performed using a color difference plane region defined based on the color difference signal to be formed.
Thereby, when detecting in the luminance signal area, the effect of the image processing can be controlled according to the luminance level of the target pixel. When detecting in the color difference signal area, the target pixel on the color difference plane can be controlled. The effect of the image processing can be controlled depending on whether it exists in the region.
A ninth invention is any one of the first to seventh inventions, wherein the first image signal and the second image signal are formed from an R component signal, a G component signal and a B component signal. The region detection unit determines whether or not the target pixel of the second image signal is included in the predetermined signal region.
(1) a color space or color plane defined by the R component signal, G component signal and B component signal forming the second image signal, or
(2) a color space or color plane obtained by converting a color space or color plane defined by the R component signal, G component signal and B component signal forming the second image signal;
To do.

As a result, even when signals based on the R component signal, the G component signal, and the B component signal are used, the effect of the image processing can be reduced depending on the region in the color space or on the color plane. Can be controlled.
Here, the “R component signal” is a red component signal, the “G component signal” is a green component signal, and the “B component signal” is a blue component signal.
In addition, the “color space” for determining whether or not the target pixel of the second image signal is included in the predetermined signal region may be the RGB color space, or the RGB color space is converted into another color space. (For example, YCbCr color space, Lab (La * b *) color space, xvYCC color space).
Further, the “color plane” for determining whether or not the target pixel of the second image signal is included in the predetermined signal region is a concept including a color plane acquired by converting the RGB color space. As the “color plane”, a PbPr color difference plane converted from the RGB color space, a CbCr color difference plane, an IQ color plane, an ab color plane (a * b *) color plane defined by the Lab (La * b *) color space, etc. There is. For example, an RGB color space defined by an R component signal, a G component signal, and a B component signal is converted into an IQ color plane, and the converted IQ color plane is converted (the RGB color space is converted into an IQ color plane). Whether or not the pixel of interest of the second image signal is included in the predetermined signal area may be determined based on the signal components of the I component signal and the Q component signal acquired in step S1.

A tenth aspect of the invention is a signal processing method including a space calculation processing step, a region detection step, and an image processing step.
In the spatial calculation processing step, a predetermined calculation process is performed on the target pixel included in the image formed by the first image signal and used as a processing target using peripheral pixels that are pixels around the target pixel on the image. Thus, the second image signal is generated from the first image signal. In the region detection step, it is determined whether or not the target pixel of the second image signal is included in the predetermined signal region, and a region determination flag indicating the determination result is generated. In the image processing step, predetermined image processing is performed on the first image signal based on the region determination flag generated in the region detection step.
Thereby, it is possible to realize a signal processing method having the same effect as that of the first invention.
An eleventh invention is a program for causing a computer to execute a signal processing method including a space calculation processing step, a region detection step, and an image processing step.

In the spatial calculation processing step, a predetermined calculation process is performed on the target pixel included in the image formed by the first image signal and used as a processing target using peripheral pixels that are pixels around the target pixel on the image. Thus, the second image signal is generated from the first image signal. In the region detection step, it is determined whether or not the target pixel of the second image signal is included in the predetermined signal region, and a region determination flag indicating the determination result is generated. In the image processing step, predetermined image processing is performed on the first image signal based on the region determination flag generated in the region detection step.
Thus, it is possible to realize a program that causes a computer to execute a signal processing method that exhibits the same effect as that of the first invention.
A twelfth aspect of the invention is an integrated circuit including a space calculation processing unit, a region detection unit, and an image processing unit.

The spatial calculation processing unit performs predetermined calculation processing on a target pixel included in an image formed by the first image signal and using a peripheral pixel that is a pixel around the target pixel on the image. Thus, the second image signal is generated from the first image signal. The region detection unit determines whether or not the target pixel of the second image signal is included in the predetermined signal region, and generates a region determination flag indicating the determination result. The image processing unit performs predetermined image processing on the first image signal based on the region determination flag generated by the region detection unit.
Thus, an integrated circuit that exhibits the same effect as that of the first invention can be realized.

  According to the invention, even when the signal component of the pixel of interest and the adjacent pixel are completely different in the boundary portion of the original subject, the spatial boundary portion is obtained by performing the calculation between the adjacent pixels in the image space. By detecting the signal area from the blurred brightness signal, color difference signal, etc., the signal component of the adjacent pixel can be blurred into the target pixel, so contour correction, band limitation, noise reduction, etc. according to the type of subject It is possible to perform image processing for controlling the effect without any sense of incongruity.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
<1.1: Configuration of Signal Processing Device>
FIG. 1 is a diagram showing a schematic configuration of a signal processing apparatus 100 according to the first embodiment of the present invention.
As shown in FIG. 1, the signal processing apparatus 100 includes a delay line unit 1 that synchronizes an input luminance signal and a color difference signal, and a space that performs spatial filter processing on the luminance signal output from the delay line unit 1. A filter unit 21 and a specific luminance region detection unit 31 that generates a luminance region determination flag from the output of the spatial filter unit 21 are provided. The signal processing apparatus 100 also includes a spatial filter unit 22 that performs spatial filter processing on the color difference signal output from the delay line unit 1, and a specific color region detection that generates a color region determination flag from the output of the spatial filter unit 22. Unit 32. Furthermore, the signal processing apparatus 100 includes an image processing unit 4 that performs predetermined image processing on the luminance signal and the color difference signal output from the delay line unit 1 based on the luminance region determination flag and the color region determination flag. .

The delay line unit 1 receives the luminance signal and the color difference signal, and simultaneously synchronizes the target pixel (the pixel to be processed) and the adjacent pixels in the horizontal, vertical, and diagonal directions of the target pixel. An example of the configuration of the delay line unit 1 will be described with reference to FIG. 1A.
For convenience of explanation, FIG. 1 schematically shows a luminance signal and a chrominance signal that are subject to signal processing.
FIG. 1A illustrates an example of a configuration that only processes luminance signals. Specifically, FIG. 1A is a diagram illustrating a luminance signal delay line unit 1A, a spatial filter unit 21, a specific luminance region detection unit 31, and an image processing unit 4 that are luminance signal processing units of the delay line unit 1. is there. Note that FIG. 1A is an example of a configuration in the case of synchronizing three vertical lines.
As illustrated in FIG. 1A, the luminance signal delay line unit 1 </ b> A includes a first delay unit 11 and a second delay unit 12.

The first delay unit 11 and the second delay unit 12 delay the output by 1H (one line) with respect to the input. The first delay unit 11 and the second delay unit 12 are configured by, for example, a line memory or a FIFO (First In First Out) memory.
As shown in FIG. 1A, the luminance signal delay line unit 1A includes a luminance signal Y0 that is a signal obtained by directly outputting an input luminance signal, a luminance signal Y1 that is a signal delayed by 1H with respect to the luminance signal Y0, and luminance A luminance signal Y2 that is a signal obtained by delaying the signal by 2H with respect to Y0 and three luminance signals are output.
The luminance signals Y0, Y1, and Y2 output from the luminance signal delay line unit 1A are input to the spatial filter unit 21 and the image processing unit 4 as shown in FIG. 1A.
In this way, signals are simultaneously synchronized in the vertical direction by the luminance signal delay line section 1A.

Note that the color difference signals (Pb signal (Cb signal) and Pr signal (Cr signal)) can also have the same configuration as in FIG. 1A.
In FIG. 1, portions corresponding to FIG. 1A are illustrated in a simplified manner for convenience of explanation. In FIG. 1A, the signals to be synchronized in the vertical direction are for three lines (for three vertical pixels). However, the number of signals is not limited to this, and may be other numbers (for example, five lines). Needless to say. When other numbers of lines are employed, the number of delay units (line memory or the like) corresponding to the number of employed lines may be provided.
The spatial filter unit 21 receives the luminance signal (synchronized luminance signal) output from the delay line unit 1 and inputs at least one of a horizontal direction, a vertical direction, and an oblique direction with respect to the input luminance signal. Perform filtering. Then, the spatial filter unit 21 outputs the filtered luminance signal to the specific luminance region detection unit 31.

In addition, when the spatial filter process performed by the spatial filter unit 21 is based on a 3 × 3 two-dimensional filter, an example of the configuration may be as illustrated in FIG. 1A. In this case, as shown in FIG. 1A, the spatial filter unit 21 includes a delay device (for example, a D-flip flop) that delays the luminance signals Y0, Y1, and Y2 by one pixel, a coefficient device that multiplies the filter coefficient, An adder and a FIR (Finite Impulse Response) filter can be used. Note that the configuration of the spatial filter processing unit is not limited to the configuration shown in FIG. 1A, and may be realized by combining, for example, a vertical filter and a horizontal filter. It goes without saying that the configuration of the FIR filter changes depending on the number of taps of the filter.
The spatial filter unit 22 receives the color difference signal (synchronized color difference signal) output from the delay line unit 1 and inputs at least one of a horizontal direction, a vertical direction, and an oblique direction with respect to the input color difference signal. The filter process is performed, and the spatial filter unit 22 outputs the filtered color difference signal to the specific color region detection unit 32. The configuration of the spatial filter unit 22 can also be configured in the same manner as the spatial filter unit 21 (for example, it can be configured by the same configuration as in FIG. 1A).

The specific luminance region detection unit 31 receives the output from the spatial filter unit 21 and determines whether or not the luminance signal filtered by the spatial filter unit 21 is included in a predetermined luminance signal region. Then, a luminance region determination flag with a plurality of bits indicating the determination result is generated. Then, the specific luminance area detection unit 31 outputs the generated luminance area determination flag to the image processing unit 4.
The specific color area detection unit 32 receives an output from the spatial filter unit 22 and determines whether or not the color difference signal filtered by the spatial filter unit 22 is included in a predetermined color difference signal area. Then, a color region determination flag with a plurality of bits indicating the determination result is generated. Then, the specific color region detection unit 32 outputs the generated color region determination flag to the image processing unit 4.
The image processing unit 4 receives the output from the delay line unit 1, the luminance region determination flag, and the color region determination flag. The image processing unit 4 performs image processing on the luminance signal and the color difference signal output from the delay line unit 1 based on the codes represented by the luminance region determination flag and the color region determination flag. Thereby, the processing effect can be changed for each subject image on the screen (on the image).

<1.2: Operation of Signal Processing Device>
The operation of the signal processing apparatus 100 configured as described above will be described below.
In the signal processing apparatus 100 of the present embodiment, the input luminance signal and color difference signal are synchronized by the delay line unit 1.
For example, in order to synchronize pixels adjacent to each other in a two-dimensional matrix of 3 pixels × 3 pixels centering on a pixel of interest (processing target pixel), it is configured by a FIFO (First-In First-Out) memory 2 or the like. A delay line for the line may be used. An example of the configuration in this case is the configuration shown in FIG. 1A.
The synchronized luminance signal and color difference signal are filtered by the spatial filter unit 21 and the spatial filter unit 22.

  As shown in FIG. 2, this filtering process may be performed by multiplying each of the synchronized 9-pixel data by a filter coefficient corresponding to the location. For example, the filtering process for pixels adjacent in the horizontal direction, Filter processing for pixels adjacent in the vertical direction and filter processing for pixels adjacent in the oblique direction may be performed separately. A configuration example (example) of the spatial filter unit 21 and the spatial filter unit 22 in the case of multiplying and adding each of the 9-pixel data by a filter coefficient corresponding to the location is the configuration shown in FIG. 1A. FIG. 1B illustrates an example of a configuration example in which the filter processing in the horizontal direction is performed after the filter processing in the vertical direction is performed. In addition, by performing the filter process in the horizontal direction after performing the filter process in the vertical direction, it is possible to multiply each of the 9-pixel data by the filter coefficient corresponding to the location and add the D-flip The number of filters, filter coefficient units and adders can be reduced (see FIGS. 1A and 1B).

In addition, the filter characteristics realized by the spatial filter unit 21 and the spatial filter unit 22 are usually low-pass filter (LPF) characteristics, but are sufficient for detecting a specific luminance region or a specific color region. Other characteristics (for example, BPF characteristics) may be used as long as a certain signal component can be extracted.
As described above, by the spatial filter processing executed by the spatial filter unit 21 and the spatial filter unit 22, for example, even when the signal component of the target pixel and its adjacent pixels are completely different signal region components, , The signal components of adjacent pixels are included at a predetermined ratio, and the signal components change stepwise (gradually) at the boundary portion. In other words, the spatial filter processing executed by the spatial filter unit 21 and the spatial filter unit 22 can alleviate (prevent) the signal component from changing abruptly at the boundary between the target pixel and its adjacent pixels.

In the specific luminance area detecting unit 31, the luminance level is divided into several areas in advance, for example, four areas (area 0 to area 3 in FIG. 3) as shown in FIG. Then, the specific luminance area detection unit 31 determines in which area of the four areas the filtered luminance signal is included, and expresses the determination result as four types of codes with two bits “luminance area” A “determination flag” is generated.
As shown in FIG. 4, the specific color area detection unit 32 occupies an area occupied by a specific color to be detected on the color difference plane (in FIG. 4, the PbPr color difference plane defined by the Pb axis and the Pr axis), for example, “ Area 3 ”and an area that is not a specific color are set as“ area 0 ”. Then, the boundary areas are set as “area 2” and “area 1” in order from the one closer to the specific color area, that is, the higher the degree of specific color. Then, the specific color region detection unit 32 determines in which region the filtered color difference signal is included, and generates a color region determination flag that expresses the determination result as two types of codes with 2 bits.

For example, by setting the “skin color region” in “region 3” in FIG. 4, it is possible to suppress the enhancement of wrinkles and the like of the human face portion on the image, and the skin color region portion on the image. For the background and the boundary portion, it is possible to perform processing that improves the detail feeling step by step (gradually). The “skin color region” is preferably set based on the color of the human face or the color of the human skin. For example, when “skin color region” is set in “region 3” in FIG. 4, 95% or more of the Japanese face color is centered on the average Japanese face color on the PbPr color difference plane. By setting the area of the included area on the color difference plane, the “skin color area” may be set in “area 3” in FIG. Needless to say, this is an example, and the present invention is not limited to this.
In FIG. 4, the color difference plane is represented by the Pb axis and the Pr axis. However, a color difference plane (IQ color plane) represented by the I axis and the Q axis may be used, and a plane defined by another axis (color information is represented by A plane that can be determined). The “plane defined by the other axis” is, for example, a CbCr color difference plane. There is an ab color plane in the Lab color space (a * b * color plane in the La * b * color space). In addition, the color space (for example, RGB color space, YCbCr color space, YPbPr color space, xvYCC color space, Lab color space (La * b * color space)) is divided and filtered. The specific color area detection process may be performed by determining which of the divided areas the signal belongs to.

Also, by converting the signal output from the spatial filter unit 22, another color plane (or color space) is converted, and the specific color area detection process is performed on the converted color plane (or color space). May be. For example, when the signals output from the spatial filter unit 22 are a Pb signal and a Pr signal, the specific color region detection unit performs PbPr-IQ conversion processing (Pb signal and Pr signal on the Pb signal and Pr signal). , A process of converting into an I signal and a Q signal), and a specific color region detection process may be performed using an IQ color plane (a color plane defined by the I axis and the Q axis). When the specific color is “skin color”, it is preferable to use an IQ color plane that makes it easy to determine whether or not it is a skin color region.
The image processing unit 4 performs signal processing on the luminance signal and the color difference signal output from the delay line unit 1 based on the codes indicated by the luminance region determination flag and the color region determination flag.

The luminance region determination flag and the color region determination flag are stepwise values based on a signal (low-frequency component signal) from which a component (high-frequency component) that changes rapidly by the spatial filter unit 21 and the spatial filter unit 22 is removed. Is generated as a flag for taking the image, so even in a rapidly changing portion (for example, a boundary portion or an edge portion) on the image, it is gradually increased according to the flag values of the luminance region determination flag and the color region determination flag. To) change (does not change suddenly).
Therefore, by controlling the effect of signal processing on the luminance signal and color difference signal output from the delay line unit 1 by the luminance region determination flag and the color region determination flag, the subject image included in the predetermined signal region and its spatial It is possible to effectively suppress a sudden change in the effect of the image processing at the boundary portion between the portion adjacent to the portion. For this reason, the image (video) formed by the signal output from the image processing unit 4 has no sense of incongruity and becomes a natural image (video) as a whole image (video).

(1.2.1: Outline correction processing)
Next, processing performed by the image processing unit 4 will be described.
First, the case where the processing executed by the image processing unit 4 is “contour correction processing” will be described.
FIG. 5 is a diagram illustrating a schematic configuration of the signal processing device 100A when performing “contour correction processing” as image processing. In the signal processing device 100 </ b> A, the image processing unit 4 in the signal processing device 100 is replaced with a contour correction unit 5. Other than that, the signal processing device 100 </ b> A is the same as the signal processing device 100. In the signal processing device 100A, the same parts as those of the signal processing device 100 are denoted by the same reference numerals, and description thereof is omitted.
The contour correcting unit 5 performs signal processing for emphasizing the contour portion of the subject image on the luminance signal and the color difference signal output from the delay line unit 1 based on the luminance region determination flag and the color region determination flag.

FIG. 6 is a diagram illustrating a schematic configuration of the contour correction unit 5.
As illustrated in FIG. 6, the contour correction unit 5 includes a contour correction signal generation unit 51, a coring processing unit 53, a gain adjustment unit 54, a coring amount / gain amount calculation unit 52, and an addition unit 55.
The contour correction signal generation unit 51 receives the video signal (luminance signal and color difference signal) output from the delay line unit 1 and generates a contour correction signal from the input video signal (luminance signal and color difference signal). Output to the ring processing unit 53;
The coring amount / gain amount calculation unit 52 receives the luminance region determination flag and the color region determination flag as input, and based on the codes indicated by the luminance region determination flag and the color region determination flag, the coring amount C and the gain of the contour correction signal. The amount G is calculated, and the coring amount / gain amount calculating unit 52 outputs the calculated coring amount C to the coring processing unit 53 and outputs the calculated gain amount G to the gain adjusting unit 54. .

The coring processor 53 receives the contour correction signal output from the contour correction signal generator 51 and the coring amount C output from the coring amount / gain amount calculator 52. The coring processing unit 53 performs coring processing on the contour correction signal according to the coring amount C calculated by the coring amount / gain amount calculating unit 52, and the coring processing unit 53 The contour correction signal after the ring processing is output to the gain adjustment unit 54.
The gain adjusting unit 54 receives the contour correction signal output from the coring processing unit 53 and the gain amount G output from the coring amount / gain amount calculating unit 52. The gain adjusting unit 54 adjusts the gain of the contour correction signal after coring processing according to the gain amount G calculated by the coring amount / gain amount calculating unit 52, and the gain adjusting unit 54 adjusts the gain after gain adjustment. The contour correction signal is output to the adder 55.

The adder 55 receives the video signal (luminance signal and chrominance signal) output from the delay line unit 1 and the contour correction signal output from the gain adjuster 54, and adds both to thereby correct the contour. The generated video signal (luminance signal and color difference signal) is generated. That is, the addition unit 55 adds the contour correction signal that has been subjected to coring processing and gain adjustment to the video signal (luminance signal and color difference signal).
Hereinafter, the contour correction process will be described with reference to FIGS. 6 and 6A.
The contour correction signal generation unit 51 performs a band pass filter (BPF) process on the input video signal in the horizontal direction, the vertical direction, and the diagonal direction, and extracts a signal having a desired frequency component from the video signal as a contour correction signal. . The extracted contour correction signal is output to the coring processing unit 53.

In the processing in the contour correction signal generation unit 51, the delay line (for example, line memory) of the delay line unit 1 is processed into processing of the contour correction unit 5 (main line processing) and spatial filter processing (spatial filter units 21, 22). Can be shared with). This will be described with reference to FIGS. 1A, 1B, and 6A.
FIG. 6A shows a luminance signal contour correction unit 5A, which is an example of a configuration of only luminance signal processing in the contour correction unit 5. For convenience of explanation, the following description will be made assuming that the contour correction signal generation unit 51 performs filtering on the video signal using pixels for three lines (three pixels in the vertical direction).
As shown in FIG. 6A, the contour correction signal generation unit (for luminance signal) 51 of the luminance signal contour correction unit 5A has three outputs of the luminance signal delay line unit 1A (see FIGS. 1A and 1B). Luminance signals Y0 to Y2 are input. Since the luminance signals Y0 to Y2 are signals synchronized for three lines in the vertical direction, the contour correction signal generation unit (for luminance signal) 51 uses the luminance signals Y0 to Y2 to generate the contour correction signal. Generation filter processing (for example, filter processing using a 3 × 3 two-dimensional filter) can be performed.

That is, in the signal processing device 100A, with such a configuration, the delay line (for example, line memory) of the delay line unit 1 is processed by the contour correction unit 5 (main line processing) and the spatial filter processing (spatial filter unit). 21 and 22).
In the above description, the luminance signal has been described, but the same applies to the color difference signal.
In the contour correction signal generation unit 51, it is mainly effective to perform processing on the luminance signal. Therefore, the contour correction signal generation unit 51 is configured only by the contour correction signal generation unit for luminance signal processing. You may do it.
In addition, the contour correction signal generation unit 51 may have a configuration in which a contour correction signal generation unit for luminance signal processing and a contour correction signal generation unit for color difference signal processing are separately provided.
The coring amount / gain amount calculation unit 52 determines the signal region based on the code indicated by the luminance region determination flag output from the specific luminance region detection unit 31 and the color region determination flag output from the specific color region detection unit 32. Is done. Then, based on the determination result for the signal region by the coring amount / gain amount calculation unit 52, the coring processing unit 53 performs coring processing C to be performed on the contour correction signal, and the gain adjustment unit. In 54, the gain amount G of the gain adjustment process executed on the contour correction signal after the coring process is determined. The coring amount C and the gain amount G determined by the coring amount / gain amount calculating unit 52 are output to the coring processing unit 53 and the gain adjusting unit 54, respectively.

In the coring processing unit 53, the coring process is performed on the contour correction signal output from the contour correction signal generation unit 51 according to the coring amount C calculated by the coring amount / gain amount calculation unit 52. Specifically, the coring processing unit 53 slices the contour correction signal output from the contour correction signal generation unit 51, and deletes the contour correction signal having a minute amplitude equal to or less than the coring amount C as noise. That is, a coring process is performed to obtain a signal having an amplitude of 0. Note that the coring processing unit 53 may perform a process of reducing the amplitude of a signal having a coring amount C or less, for example, instead of the coring process.
The contour correction signal subjected to coring processing by the coring processing unit 53 is output to the gain adjustment unit 54.
The gain adjustment unit 54 performs gain adjustment on the contour correction signal subjected to the coring process according to the gain amount G calculated by the coring amount / gain amount calculation unit 52. Then, the gain-adjusted contour correction signal is output to the adder 55.

The adder 55 adds the contour correction signal subjected to coring processing and gain adjustment to the original video signal (signal output from the delay line unit 1), and then performs overflow or underflow clipping processing as necessary. Is output as a video signal whose contour is corrected.
In the contour correction processing by the contour correction unit 5 shown in FIG. 6, the size (amplitude, signal component) of the contour correction signal is appropriately set by coring processing or gain adjustment based on the code indicated by the luminance region determination flag or the color region determination flag. Controlled. In other words, this contour correction process maintains the effect of the contour correction of the entire image, and for example, for a “skin color region” corresponding to a human face, the contour correction is performed by increasing the coring amount and decreasing the gain amount. By reducing the coring amount and increasing the gain amount step by step based on the code of the color region determination flag for the boundary portion with the background, the effect of contour correction is gradually increased (gradually) (B) The detail feeling can be improved step by step (gradually) for the background of the skin color region and its boundary.

On the other hand, for example, for a “green region” corresponding to a leaf, the effect of contour correction is further emphasized by reducing the coring amount and increasing the gain amount. By gradually increasing the coring amount and decreasing the gain amount based on the code, the effect of contour correction can be weakened gradually (gradually) (in the background of the green area and its boundary) On the other hand, the detail can be reduced in stages (gradually)).
Further, since the amount of noise is relatively increased with respect to the signal in the low luminance level region, the noise component is also emphasized by the contour correction processing. In that case, the effect of contour correction is suppressed by increasing the coring amount and decreasing the gain amount for the low luminance region, and the luminance region determination flag is indicated for the boundary portion where the low luminance portion and the high luminance portion are adjacent to each other. By reducing the coring amount and increasing the gain amount stepwise based on the code, the effect of contour correction can be increased stepwise (gradually).

Thereby, the signal processing apparatus 100A can perform contour correction processing without a sense of incongruity without abruptly changing the effect of contour correction at the boundary portion of the subject image in the image space. As a result, the image (video) acquired by the signal processing device 100A becomes natural.
(1.2.2: Bandwidth limiting process)
Next, a case where the process executed by the image processing unit 4 is a “band limitation process” will be described.
FIG. 7 is a diagram illustrating a schematic configuration of the signal processing device 100B when performing “band limitation processing” as image processing. In the signal processing device 100 </ b> B, the image processing unit 4 in the signal processing device 100 is replaced with the band limiting unit 6, and the output of the spatial filter unit 21 is input to the band limiting unit 6. Other than that, the signal processing device 100B is the same as the signal processing device 100. In the signal processing device 100B, the same parts as those of the signal processing devices 100 and 100A are denoted by the same reference numerals and description thereof is omitted.

The band limiting unit 6 includes a video signal output from the delay line unit 1, a low-frequency luminance signal output from the spatial filter unit 21, a luminance region determination flag output from the specific luminance region detection unit 31, and a specific color The color region determination flag output from the region detection unit 32 is input. Based on the luminance region determination flag and the color region determination flag, the band limiting unit 6 performs processing for limiting the frequency component band on the video signal (luminance signal and color difference signal) output from the delay line unit 1. .
FIG. 8 is a diagram showing a schematic configuration of the band limiting unit 6.
As shown in FIG. 8, the subtraction unit 61 receives the luminance signal output from the delay line unit 1 and the low-frequency luminance signal output from the spatial filter unit 21 and outputs from the delay line unit 1. A high-frequency luminance signal is generated by subtracting the low-frequency luminance signal from the luminance signal. Then, the generated high frequency luminance signal is output to the coring processing unit 63.

The coring amount / gain amount calculation unit 62 receives the luminance region determination flag output from the specific luminance region detection unit 31 and the color region determination flag output from the specific color region detection unit 32 as inputs. The coring amount / gain amount calculating unit 62 calculates the coring amount C and the gain amount G of the high-frequency luminance signal based on the codes indicated by the luminance region determination flag and the color region determination flag. Then, the coring amount / gain amount calculating unit 62 outputs the calculated coring amount C and gain amount G to the coring processing unit 63 and the gain adjusting unit 64, respectively.
The coring processing unit 63 receives the high-frequency luminance signal output from the subtracting unit 61 and the coring amount C calculated by the coring amount / gain amount calculating unit 62 and converts the high-frequency luminance signal according to the coring amount C. The coring process is executed for this. Then, the coring processing unit 63 outputs the high-frequency luminance signal subjected to the coring process to the gain adjustment unit 64.

The gain adjusting unit 64 receives the coring-processed high-frequency luminance signal output from the coring processing unit 63 and the gain amount G calculated by the coring amount / gain amount calculating unit 62, and inputs the coring amount / gain. According to the gain amount G calculated by the amount calculation unit 62, the gain adjustment is performed on the coring-processed high-frequency luminance signal. Then, the gain adjustment unit 64 outputs the high-frequency luminance signal after gain adjustment to the addition unit 65.
The addition unit 65 receives the high-frequency luminance signal after gain adjustment output from the gain adjustment unit 64 and the low-frequency luminance signal output from the spatial filter unit 21 as input, and coring processing and gain-adjusted high The luminance signal is added to the low luminance signal to generate a luminance signal and output.
Hereinafter, the band limiting process will be described with reference to FIG.
The subtracting unit 61 extracts a high frequency component of the luminance signal by subtracting the low frequency luminance signal from the input luminance signal to generate a high frequency luminance signal. The high frequency luminance signal generated by the subtracting unit 61 is output to the coring processing unit 63.

The coring amount / gain amount calculation unit 62 determines the signal region based on the code indicated by the luminance region determination flag output from the specific luminance region detection unit 31 and the color region determination flag output from the specific color region detection unit 32. Is done. Then, based on the determination result for the signal region by the coring amount / gain amount calculating unit 62, the coring processing unit 63 performs coring processing C to be performed on the high frequency luminance signal, and gain adjustment. The gain amount G of the gain adjustment process executed on the high frequency luminance signal after the coring process is determined in the unit 64. Then, the coring amount C and the gain amount G determined by the coring amount / gain amount calculating unit 62 are output to the coring processing unit 63 and the gain adjusting unit 64, respectively.
In the coring processing unit 63, the coring process is performed on the high frequency luminance signal output from the subtraction unit 61 according to the coring amount C calculated by the coring amount / gain amount calculating unit 62. Specifically, the coring processing unit 63 slices the high-frequency luminance signal output from the subtracting unit 61, and thereby deletes the high-frequency luminance signal having a minute amplitude equal to or smaller than the coring amount C as noise. That is, a coring process is performed to obtain a signal having an amplitude of 0. Note that the coring processing unit 63 may perform, for example, processing for reducing the amplitude of a signal having a coring amount C or less instead of the coring processing.

The high frequency luminance signal subjected to the coring process by the coring processing unit 63 is output to the gain adjusting unit 64.
The gain adjustment unit 64 performs gain adjustment on the high-frequency luminance signal subjected to the coring process according to the gain amount G calculated by the coring amount / gain amount calculation unit 62. Then, the high-frequency luminance signal after gain adjustment is output to the adding unit 65.
In the adding unit 65, the high-frequency luminance signal subjected to the coring process and gain adjustment is added to the low-frequency luminance signal output from the spatial filter unit 21, and is output as a band-limited luminance signal.
In the band limiting process by the band limiting unit 6 shown in FIG. 8, the size (amplitude, signal component) of the high frequency luminance signal is adjusted by coring processing or gain adjustment based on the code indicated by the luminance region determination flag or the color region determination flag. Appropriately controlled. In other words, with this band limiting process, while maintaining the frequency band of the entire image, for example, for a “skin color region” corresponding to a human face, a high-frequency luminance signal is increased by increasing the coring amount and decreasing the gain amount. The band-limiting effect is reduced stepwise by reducing the coring amount and increasing the gain amount step by step based on the color area determination flag code. (Details can be improved gradually (gradually) for the background of the skin color region and its boundary portion).

On the other hand, for example, for a “green region” corresponding to a leaf, a high-frequency luminance signal component is further increased by reducing the coring amount and increasing the gain amount, and a color region determination flag for the boundary portion with the background By gradually increasing the coring amount and decreasing the gain amount based on the code, the band-limiting effect can be increased stepwise (gradually) (the background of the green area and its boundary) The detail can be reduced in stages (gradually)).
In addition, this band limiting process increases the amount of noise relative to the signal in regions with low luminance levels, so that the noise component is also suppressed by further increasing the coring amount and further decreasing the gain amount. On the other hand, at the boundary portion where the low luminance portion and the high luminance portion are adjacent to each other, the coring amount can be decreased and the gain amount can be increased step by step based on the code indicated by the luminance region determination flag.

As a result, the signal processing apparatus 100B can perform the band limiting process without a sense of incongruity without abruptly changing the band limiting effect at the boundary portion of the subject image in the image space. As a result, the image (video) acquired by the signal processing device 100B becomes natural.
≪Modification of band limiter≫
Next, a band limiting unit 6A that is a modification of the band limiting unit 6 will be described.
FIG. 9 is a diagram illustrating a band limiting unit 6 </ b> A that is another schematic configuration (modified example) of the band limiting unit 6.
As illustrated in FIG. 9, the band limiting unit 6 </ b> A includes a mixing ratio calculation unit 66, gain adjustment units 67 and 68, and an addition unit 69.

The mixing ratio calculation unit 66 receives the luminance region determination flag output from the specific luminance region detection unit 31 and the color region determination flag output from the specific color region detection unit 32, and sets the luminance region determination flag and the color region determination flag. The mixing ratio k (0 ≦ k ≦ 1) of the luminance signal and the low frequency luminance signal is calculated based on the code shown, and the mixing ratio calculation unit 66 calculates the calculated mixing ratio k as a gain adjustment unit (coefficient unit). ) Output to 67 and 68.
The gain adjusting unit 67 (coefficient unit) receives the luminance signal output from the delay line unit 1 and the mixing ratio k calculated by the mixing ratio calculating unit 66 and outputs the luminance signal output from the delay line unit 1. Is adjusted with the mixing ratio k (multiply the luminance signal by k). Then, the gain adjusting unit 67 outputs the gain-adjusted luminance signal to the adding unit 69.
The gain adjustment unit 68 (coefficient unit) receives the low-frequency luminance signal output from the spatial filter unit 21 and the mixing ratio k calculated by the mixing ratio calculation unit 66 as an input, and is output from the spatial filter unit 21. The gain adjustment is performed on the low-frequency luminance signal by the mixing ratio (1-k) (the low-frequency luminance signal is multiplied by (1-k)). Then, the gain adjusting unit 68 outputs the gain-adjusted low frequency luminance signal to the adding unit 69.

The adding unit 69 adds the luminance-adjusted luminance signal output from the gain adjusting unit 67 and the gain-adjusted low-frequency luminance signal output from the gain adjusting unit 68 to generate a luminance signal.
Hereinafter, the band limiting process by the band limiting unit 6A will be described with reference to FIG.
In the mixing ratio calculation unit 66, the signal area is determined based on the codes indicated by the luminance area determination flag and the color area determination flag. Then, in each determined signal area, the mixing ratio k (internal division ratio k) of the input luminance signal and the low-frequency luminance signal is determined. In the band limiting process by the band limiting unit 6A, as the mixing ratio k (internal ratio k) is decreased, the ratio of the low-frequency luminance signal increases and the frequency band of the luminance signal to be output is limited.
The gain adjusting units 67 and 68 adjust the gain of the luminance signal and the low-frequency luminance signal according to the mixing ratio k calculated by the mixing ratio calculating unit 66, respectively.

The adder 69 adds the gain-adjusted luminance signal and the low-frequency luminance signal, and outputs the result as a band-limited luminance signal.
That is, the gain adjustment units 67 and 68 and the addition unit 69 perform internal division processing on the luminance signal and the low-frequency luminance signal. That is, assuming that the luminance signal output from the adding unit 69 is Yout, the luminance signal input to the gain adjusting unit 67 is Yin, and the low frequency luminance signal input to the gain adjusting unit 68 is Ylow,
Yout = k · Yin + (1−k) · Ylow (0 ≦ k ≦ 1)
The internal division process is executed.
In the band limiting process by the band limiting unit 6A shown in FIG. 9, the mixing ratio of the luminance signal and the low-frequency luminance signal is controlled based on the code indicated by the luminance region determination flag and the color region determination flag. While maintaining the band, for example, for the “skin color region” corresponding to the face of a person, the component of the high-frequency luminance signal is suppressed by increasing the mixing ratio of the low-frequency luminance signal (the value of the internal ratio k). For the boundary portion with the background, the mixing ratio of the low-frequency luminance signal is gradually decreased based on the code of the color region determination flag (the value of the internal division ratio k is increased). (This is equivalent to processing.) By doing so, the effect of the band limitation can be gradually reduced (gradually). Can be improved) ).

Further, since the amount of noise is relatively increased with respect to the signal in the low luminance level region, the mixing ratio of the low luminance signal is further increased (corresponding to the process of reducing the value of the internal division ratio k). Noise components are also suppressed, and the mixing ratio of the low-frequency luminance signal can be reduced stepwise based on the code indicated by the luminance region determination flag for the boundary portion where the low-luminance portion and the high-luminance portion are adjacent.
As a result, the signal processing apparatus including the band limiting unit 6A can perform the band limiting process without a sense of incongruity without abruptly changing the band limiting effect at the boundary portion of the subject image in the image space. As a result, the image (video) acquired by the signal processing device including the band limiting unit 6A is natural.
(1.2.3: Noise reduction processing)
Next, a case where the processing executed by the image processing unit 4 is “noise reduction processing” will be described.

FIG. 10 is a diagram illustrating a schematic configuration of a signal processing device 100C when performing “noise reduction processing” as image processing. In the signal processing device 100 </ b> C, the image processing unit 4 in the signal processing device 100 is replaced with a noise reduction unit 7. Other than that, the signal processing device 100 </ b> C is the same as the signal processing device 100. In the signal processing device 100C, the same parts as those of the signal processing devices 100, 100A, and 100B are denoted by the same reference numerals and description thereof is omitted.
The noise reduction unit 7 performs a process of reducing noise components included in the video signal output from the delay line unit 1 based on the luminance region determination flag and the color region determination flag.
FIG. 11 is a diagram illustrating a schematic configuration of the noise reduction unit 7.
As shown in FIG. 11, the noise reduction unit 7 includes a comparison threshold value calculation unit 71, a peripheral pixel comparison unit 72, and a pixel replacement unit 73.

The comparison threshold calculation unit 71 receives the luminance region determination flag output from the specific luminance region detection unit 31 and the color region determination flag output from the specific color region detection unit 32. The comparison threshold value calculation unit 71 calculates a threshold value TH that is a criterion for determining whether or not there is a correlation between the pixel of interest and the surrounding pixels based on the codes indicated by the luminance region determination flag and the color region determination flag, and the comparison The threshold calculation unit 71 outputs the calculated threshold TH to the surrounding pixel comparison unit 72.
Note that the threshold TH is determined based on so-called camera performance such as frequency characteristics and S / N ratio of an image (video) acquired in an apparatus (for example, a camera system) having the signal processing apparatus 100C. Is preferred. The threshold TH may be determined at the time of manufacture of the apparatus having the signal processing apparatus 100C, or an interface unit may be provided in the apparatus having the signal processing apparatus 100C so that the user can set the threshold TH. Good.

The peripheral pixel comparison unit 72 receives the video signal output from the delay line unit 1 and the threshold TH calculated by the comparison threshold calculation unit 71, and correlates the target pixel and the peripheral pixels with the threshold TH as a reference. The correlation determination flag F expressing the determination result is generated, and the surrounding pixel comparison unit 72 outputs the generated correlation determination flag F to the pixel replacement unit 73.
The pixel replacement unit 73 receives the video signal output from the delay line unit 1 and the correlation determination flag F output from the peripheral pixel comparison unit 72, and determines the target pixel as an average value of the peripheral correlation pixels according to the correlation determination flag F. Replace with
Hereinafter, the noise reduction processing will be described with reference to FIG.
The comparison threshold calculation unit 71 determines the signal area based on the codes indicated by the luminance area determination flag and the color area determination flag. Then, in each determined signal area, a threshold TH serving as a reference for determining whether or not there is a correlation between the target pixel of the video signal (luminance signal and color difference signal) and the peripheral pixel is determined.

In the peripheral pixel comparison unit 72, when the difference in level between the target pixel and the peripheral pixel is equal to or smaller than the threshold TH, the peripheral pixel of the target pixel is determined to be “correlated” with the target pixel, and the correlation determination flag F is set (for example, the value of the correlation determination flag F is set to “1”). On the other hand, when the level difference between the target pixel and the peripheral pixel exceeds the threshold value TH, it is determined that the peripheral pixel of the target pixel is “no correlation” with the target pixel, and the correlation determination flag F is not set (for example, The value of the correlation determination flag F is set to “0”).
In the pixel replacement unit 73, when the correlation determination flag F indicates that the pixel of interest and its surrounding pixels are “correlated” (for example, when the value of the correlation determination flag F is “1”), By averaging the peripheral pixels for which the correlation determination flag is set among the pixels, data with reduced noise components is generated and replaced with the pixel of interest. On the other hand, when the correlation determination flag F indicates that the target pixel and its surrounding pixels are “no correlation” (for example, when the value of the correlation determination flag F is “0”), the target pixel is used as it is. Output from the replacement unit 73 (the video signal input to the pixel replacement unit 73 is output as it is).

In the processing of the pixel replacement unit 73, the delay line (for example, line memory) of the delay line unit 1 can be shared by the spatial filter processing (processing by the spatial filter units 21 and 22). When the replacement unit 73 is configured, a delay line (for example, a line memory) is not required separately, so that an increase in hardware scale can be suppressed.
In the noise reduction processing by the above method, the noise component reduction effect increases as the number of pixels to be averaged increases. Therefore, if the threshold value TH determined by the comparison threshold value calculation unit 71 is increased, the number of surrounding pixels having correlation increases, and the noise component reduction effect increases.
In the noise reduction processing by the noise reduction unit 7 illustrated in FIG. 11, the threshold TH used when determining the correlation between the pixel of interest and the surrounding pixels based on the codes indicated by the luminance region determination flag and the color region determination flag is large. Appropriately controlled. That is, by this noise reduction processing, while maintaining the noise reduction effect of the entire image, for example, for the “skin color region” corresponding to the face of a person, the noise component is suppressed by increasing the threshold TH, For the boundary portion with the background, the noise limiting effect can be changed stepwise (gradually) by gradually reducing the threshold TH based on the code of the color region determination flag. Further, since the amount of noise is relatively increased with respect to the signal in the low luminance level region, the noise component can be further suppressed by further increasing the threshold value TH. On the other hand, the low luminance portion and the high luminance portion In the boundary portion adjacent to each other, the threshold value TH can be decreased stepwise based on the code indicated by the luminance region determination flag.

As a result, the signal processing apparatus 100C can perform noise reduction processing without a sense of incongruity without abruptly changing the noise reduction effect at the boundary portion of the subject image in the image space. As a result, the image (video) acquired by the signal processing device 100C is natural.
As described above, various types of signal processing have been described. However, the contour correction, the band limitation, and the noise reduction processing according to the present embodiment use a delay line for synchronizing video signals. Can be shared with the delay line used to synchronize the video signal for the spatial filter processing (processing of the spatial filter units 21 and 22) before the signal area detection in FIG. is there.
Note that the methods of contour correction, band limitation, and noise reduction processing are not limited to the present embodiment, and other methods may be used as long as the degree of effect can be controlled based on the region determination flag.

Further, in the present embodiment, an example in which contour correction, band limitation, and noise reduction are performed independently as image processing has been described. However, in the signal processing device, all of these processes may be performed, or the patent of the present invention. Other image processing that controls the effect of the processing step by step for each signal region may be executed in the signal processing device without departing from the scope of the claims.
In the present embodiment, the luminance signal and the color difference signal are synchronized using the delay line (by the delay line unit) and subjected to the spatial filter processing, and then the signal region is detected. The RGB signal (primary color signal) R signal, G signal, and B signal are synchronized using a delay line and subjected to spatial filter processing, and then the luminance signal and color difference are detected by the specific luminance region detection unit and the specific color region detection unit. The signal region may be detected by converting the signal. Specifically, for example, the signal region may be detected as follows.
(1) The RGB R signal, G signal, and B signal are converted into a YPbPr Y signal, a Pb signal, and a Pr signal, and a signal region is detected by the Y signal component and the PbPr color difference plane.
(2) The RGB R signal, G signal, and B signal are converted into YIQ Y signal, I signal, and Q signal, and a signal region is detected by the Y signal component and IQ color plane.

Alternatively, the signal area may be detected with the R signal, the G signal, and the B signal being RGB signals (primary color signals).
Note that the image processing such as contour correction, band limitation, and noise reduction processing may be performed on one or both of the luminance signal and the color difference signal, or may be performed on the primary color signals R, G, and B. It doesn't matter.
Further, the area set for determining the color area is not limited to that shown in FIG. 4, and the number of areas set and the shape of the area may be other. For example, the shape of the region may be a circular region, an elliptical region, or a sector region.
In the present embodiment, the case where the signal region is detected after performing the spatial filter processing in the horizontal direction, the vertical direction, and the oblique direction of the image has been described. However, the present invention is not limited to this. A signal area may be detected after the spatial filter processing is performed in the temporal direction between the pixels of the frame before and after the pixel of interest configured by a frame memory. By doing so, it is possible to suppress a rapid change in each image processing effect such as contour correction, band limitation, and noise reduction processing at the transition of a video scene.

As described above, in the signal processing device according to the present embodiment, contour correction without a sense of incongruity is performed by controlling the processing effect stepwise at the boundary portion of the subject image on the image (video) without increasing the circuit scale. Image processing such as band limitation and noise reduction processing can be realized.
[Other Embodiments]
In the signal processing apparatus described in the above embodiment, each block may be individually made into one chip by a semiconductor device such as an LSI, or may be made into one chip so as to include a part or the whole.
Here, although LSI is used, it may be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.
Further, the method of circuit integration is not limited to LSI, and implementation with a dedicated circuit or a general-purpose processor is also possible. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

Further, if integrated circuit technology comes out to replace LSI's as a result of the advancement of semiconductor technology or a derivative other technology, it is naturally also possible to carry out function block integration using this technology. Biotechnology can be applied as a possibility.
Moreover, each process of the said embodiment may be implement | achieved by hardware, and may be implement | achieved by software. Further, it may be realized by mixed processing of software and hardware. Needless to say, when the signal processing according to the above embodiment is realized by hardware, it is necessary to adjust the timing for performing each processing. In the above embodiment, for convenience of explanation, details of timing adjustment of various signals generated in actual hardware design are omitted.
The specific configuration of the present invention is not limited to the above-described embodiment, and various changes and modifications can be made without departing from the scope of the invention.

  The signal processing apparatus, signal processing method, program, and integrated circuit according to the present invention perform signal processing that can smoothly change the effect at the boundary portion of the subject image even when the effect of the image processing is desired to be changed for each subject image. Since it can be realized, it is useful in the field of video equipment related industries, and the present invention can be implemented in this field.

1 is a schematic configuration diagram of a signal processing device 100 according to a first embodiment of the present invention. Schematic configuration diagram of the luminance signal delay line unit 1A, the spatial filter unit 21, the specific luminance region detection unit 31, and the image processing unit 4 Schematic configuration diagram of the luminance signal delay line unit 1A, the spatial filter unit 21, the specific luminance region detection unit 31, and the image processing unit 4 The figure explaining the coefficient arrangement of the spatial filter The figure explaining the relationship between a luminance signal area and a luminance area determination flag The figure explaining the relationship between a color signal area and a color area determination flag The figure explaining schematic structure of 100 A of signal processing apparatuses which perform the outline correction which concerns on 1st Embodiment of this invention. Schematic configuration diagram of the contour correction unit 5 Schematic configuration diagram of the contour correction unit 5A Schematic configuration diagram of a signal processing device 100B that performs band limitation according to the first embodiment of the present invention. Schematic configuration diagram of the band limiting unit 6 Schematic configuration diagram of the band limiting unit 6A Schematic configuration diagram of a signal processing device 100C that performs noise reduction according to the first embodiment of the present invention. Schematic configuration diagram of the noise reduction unit 7

100, 100A, 100B, 100C Signal processing device 1, 1A Delay line unit 11, 12 Delay unit (first delay unit, second delay unit)
21, 22 Spatial filter unit 31 Specific luminance region detection unit 32 Specific color region detection unit 4 Image processing unit 5, 5A Contour correction unit 51 Contour correction signal generation unit 52, 62 Coring amount / gain amount calculation unit 53, 63 Coring Unit 54, 64 gain unit 55, 65, 69 addition unit 6, 6A band limiting unit 61 subtraction unit 66 mixing ratio calculation unit 67, 68 gain adjustment unit 7 noise reduction unit 71 comparison threshold calculation unit 72 peripheral pixel comparison unit 73 pixel replacement Part

Claims (11)

By performing the spatial processing on the target pixel included is a processing target image formed by the first image signal, the spatial operation that generates a target pixel of the second image signal from the pixel of interest in the first image signal A processing unit;
An area detection unit that determines whether or not the target pixel of the second image signal is included in a predetermined signal area, and generates an area determination flag indicating the determination result;
And line Cormorant image processing unit to the front Symbol first image signal to predetermined image processing corresponding to the area determining flag generated by the region detecting section,
Equipped with a,
The space calculation processing unit uses the pixels adjacent in at least one of a horizontal direction, a vertical direction, a diagonal direction, and a time direction of the pixel of interest on an image space formed by the first image signal. I do,
Signal processing device. A delay line unit that simultaneously synchronizes the target pixel of the first image signal and the pixel adjacent to the target pixel in the vertical direction;
The spatial processing unit uses the first image signal synchronized by the delay line section, performs the spatial processing,
The image processing unit performs the predetermined image processing on the first image signal synchronized by the delay line unit;
The signal processing apparatus according to claim 1. The spatial processing unit, by performing spatial filtering, performs the spatial processing with respect to the pixel of interest,
The signal processing apparatus according to claim 1 or 2 . The region determination flag has flags corresponding to at least a first signal region, a boundary signal region, and a second signal region,
The first signal area is the predetermined signal area;
The boundary partial signal area is a signal area surrounding the first signal area;
The second signal region is a signal region other than the first signal region and the boundary signal region.
The signal processing apparatus according to any one of claims 1 to 3. The image processing unit
In accordance with the code value indicated by the region determination flag, the effect of contour correction to be applied to the first image signal is adjusted in stages,
In accordance with the value of the code indicated by the region determination flag, the effect of band limitation applied to the first image signal is adjusted stepwise.
Adjusting the effect of noise reduction applied to the first image signal stepwise according to the code value indicated by the region determination flag ;
The signal processing apparatus according to any one of claims 1 to 4. The predetermined signal area is at least one of a predetermined luminance signal area and a color difference signal area.
The signal processing apparatus according to any one of claims 1 to 5. The first image signal and the second image signal are formed from a luminance signal and a color difference signal,
The region detection unit determines whether or not the target pixel of the second image signal is included in the predetermined signal region based on a luminance region defined based on a luminance signal that forms the second image signal, and Using a color difference plane region defined based on a color difference signal forming the second image signal,
The signal processing apparatus according to any one of claims 1 6. The first image signal and the second image signal are formed from an R component signal, a G component signal, and a B component signal,
The region detection unit
A color space defined by an R component signal, a G component signal, and a B component signal forming the second image signal is used to determine whether the target pixel of the second image signal is included in the predetermined signal region. Alternatively, using a color plane, or a color space or color plane obtained by converting a color space or color plane defined by the R component signal, G component signal and B component signal forming the second image signal, Do,
The signal processing apparatus according to any one of claims 1 6. By performing the spatial processing on the target pixel included is a processing target image formed by the first image signal, the spatial operation that generates a target pixel of the second image signal from the pixel of interest in the first image signal Processing steps;
A region detection step of determining whether or not the target pixel of the second image signal is included in a predetermined signal region, and generating a region determination flag indicating the determination result;
And line Cormorant image processing step to the first image signal to predetermined image processing corresponding to the area determining flag generated by the area detecting step,
Equipped with a,
In the space calculation processing step, the space calculation is performed using pixels adjacent to at least one of a horizontal direction, a vertical direction, a diagonal direction, and a time direction of the pixel of interest on an image space formed by the first image signal. Process,
Signal processing method. By performing the spatial processing on the target pixel included is a processing target image formed by the first image signal, the spatial operation that generates a target pixel of the second image signal from the pixel of interest in the first image signal Processing steps;
A region detection step of determining whether or not the target pixel of the second image signal is included in a predetermined signal region, and generating a region determination flag indicating the determination result;
And line Cormorant image processing step to the first image signal to predetermined image processing corresponding to the area determining flag generated by the area detecting step,
Equipped with a,
In the space calculation processing step, the space calculation is performed using pixels adjacent to at least one of a horizontal direction, a vertical direction, a diagonal direction, and a time direction of the pixel of interest on an image space formed by the first image signal. Process,
A program for causing a computer to execute a signal processing method. By performing the spatial processing on the target pixel included is a processing target image formed by the first image signal, the spatial operation that generates a target pixel of the second image signal from the pixel of interest in the first image signal A processing unit;
An area detection unit that determines whether or not the target pixel of the second image signal is included in a predetermined signal area, and generates an area determination flag indicating the determination result;
And line Cormorant image processing unit predetermined image processing to the first image signal corresponding to the area determining flag generated by the region detecting section,
Equipped with a,
The space calculation processing unit uses the pixels adjacent in at least one of a horizontal direction, a vertical direction, a diagonal direction, and a time direction of the pixel of interest on an image space formed by the first image signal. I do,
Integrated circuit.

Images