JP3999091B2 - Image correction processing apparatus and program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image correction processing apparatus and a program that are provided in a terminal, a server, or the like in an image distribution system and perform correction processing such as sharpness of an image to be distributed.
[0002]
[Prior art]
In recent years, in mobile terminals such as mobile phones and PDAs (personal digital assistants), models with digital cameras equipped with imaging means such as CMOS sensors and CCDs are becoming widespread. Images taken by such portable terminals are distributed by an image distribution system such as distributed as an attached file of an e-mail or image data itself via a communication means such as a mobile communication network or the Internet. It can be distributed to other mobile terminals and personal computers.
[0003]
In these image distribution systems, the data capacity that can be distributed is limited depending on the distribution facility and distribution time of the mobile communication network, the specifications determined for the portable terminal, and the like. The limitation on the data capacity varies depending on the carrier company of the mobile communication network and the terminal model. For this reason, in a portable terminal, the image quality is controlled so as not to exceed a predetermined data capacity limit by controlling the Q coefficient when JPEG compression of image data or controlling the strength of image sharpness. Need to be delivered by.
[0004]
As a conventional technique for controlling the sharpness of an image, for example, when determining image processing parameters, there is a technique for controlling the degree of sharpness enhancement of a face image according to the size of the face included in the image (for example, patents). Reference 1). In some television camera apparatuses, a contour enhancement gain of a specific hue is variable according to zoom ratio and focal length information (see, for example, Patent Document 2).
[0005]
[Patent Document 1]
Japanese Patent Laid-Open No. 11-339035 [Patent Document 2]
JP-A-8-51634 [0006]
[Problems to be solved by the invention]
In the method of controlling the data volume of image data by controlling the strength of image sharpness as described above, usually the image sharpness is weakened to reduce the data volume. This utilizes the property that the data capacity after JPEG compression becomes smaller when the image redundancy is high, that is, when the image contains a lot of low-frequency components.
[0007]
However, when the data capacity is reduced to satisfy the capacity limit by reducing the sharpness of the image, the sharpness is uniformly controlled for the entire captured image. When a person is photographed as a background, the sharpness of the person's face, which is the most important subject in the photographed image, is weakened, resulting in a problem that the image quality of the face part deteriorates.
[0008]
The present invention has been made to solve the above-described problems, and its object is to provide an image without degrading the image quality of the face portion of a photographed person when there is a limit to the data capacity that can be distributed in image distribution. An object of the present invention is to provide an image correction processing apparatus and program capable of reducing the data capacity to be within the limit range.
[0009]
[Means for Solving the Problems]
The present invention is an image correction processing apparatus that performs image correction processing including sharpness processing on a person image, and includes face extraction processing means for extracting a face image region in the person image, and non-face image regions other than a face. Sharpness parameter determination means for determining the sharpness parameters of the face image area and the non-face image area so that the sharpness is weaker than the sharpness of the face image area; and the face image based on the sharpness parameter determined by the sharpness parameter determination means Region- specific sharpness processing means for performing sharpness processing for each region by changing the strength of sharpness of the region and non-face image region, and image data subjected to sharpness processing for the face image region and non-face image region Image compression means for compressing, the sharpness parameter In the determining means, the capacity of the image data compressed by the image compressing means is caused by a restriction of an image transmitting means for distributing the image data or a receiving terminal that is a distribution destination of the image data by the image transmitting means. There is provided an image correction processing apparatus characterized in that a sharpness parameter of the non-face image area is determined so as to be within a limited capacity of data that can be distributed .
[0010]
According to the above configuration, since sharpness processing is performed individually for each image area other than the face and the face, for example, by controlling to reduce the sharpness of only the image area other than the face, the image quality of the face portion is not deteriorated. The image data capacity can be reduced. In addition, by controlling the sharpness intensity based on the limit of the data capacity that can be distributed caused by the restrictions of the image transmission means or the receiving terminal, the image data capacity can be adjusted appropriately so as to satisfy the limit capacity of the data capacity. Is possible. In this case, it is possible to reduce the image data capacity while minimizing image quality degradation without excessively weakening the sharpness of the image area other than the face.
[0011]
In addition, the sharpness parameter determination unit varies the sharpness parameter according to an area ratio of the face image region in the person image.
[0012]
According to the above configuration, it is possible to appropriately perform sharpness processing in each image area in accordance with the area ratio of the face image area, and it is possible to adjust the image data capacity to be a predetermined value.
[0013]
The present invention also includes a step of extracting a face image area in a person image, and sharpness of the face image area and the non-face image area so that the sharpness of the non-face image area other than the face is weaker than the sharpness of the face image area. Determining a parameter, changing a sharpness level of the face image area and the non-face image area based on the determined sharpness parameter, and individually performing a sharpness process for each area, and the face image area and An image correction processing program for causing a computer to execute a step of compressing image data that has undergone sharpness processing of a non-face image region, wherein in the step of performing sharpness processing, a capacity of the compressed image data is Image transmission means for distributing the image data or the image transmission To provide an image correction processing program for determining the sharpness parameter of the non-face image area to be within the limit capacity of the distributable data caused by limitations of the delivery destination is a receiving terminal of the image data by the means.
[0018]
With the above configuration, a function equivalent to the above-described image correction processing apparatus can be realized on a computer, and the image data capacity is adjusted without deteriorating the image quality of the face portion and minimizing the image quality degradation of the face other than the face. It becomes possible. For example, when there is a limit on the data capacity that can be distributed when distributing a person image, it is possible to appropriately reduce the image data capacity so as to satisfy this limited capacity.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
In this embodiment, the case where the image correction processing apparatus according to the present invention is applied to a camera-equipped mobile phone is illustrated. The image correction processing program according to the present invention implements an image correction processing method corresponding to the operation of the image correction processing apparatus using a computer. The image correction processing apparatus according to the present invention can be provided in the image delivery server in addition to being provided on the terminal side in the image delivery system.
[0020]
FIG. 1 is a block diagram showing a main configuration of an embodiment of a camera-equipped cellular phone provided with an image correction processing apparatus according to the present invention.
[0021]
A camera-equipped mobile phone 1 includes a communication unit 101 that performs modulation / demodulation and wireless transmission and reception of a radio signal, a display unit 102 including a display element such as a liquid crystal display for displaying communication information and an image, and a drive circuit thereof, a telephone number Operation unit 103 including a group of buttons for performing input and character input, image data designation / selection, and the like, a microphone for performing transmission / reception of a telephone, a speaker and a transmission / reception unit 104 including a voice signal processing circuit, a GPS satellite, A current position recognition unit 105 including a GPS receiver that acquires position information from a base station signal, a control unit 106 that includes a CPU, a memory, and the like to control each part and perform signal processing, and a photographing lens, a CMOS sensor, and a CCD An image pickup unit 107 including an image pickup device such as an image sensor performs various types of image signal processing including image correction processing according to the present invention on captured image data. Configured to have an image signal processing unit 108.
[0022]
FIG. 2 is a block diagram showing in detail the configuration of the imaging unit 107 and the image signal processing unit 108 in FIG.
[0023]
The imaging unit 107 includes a CCD 201 that is a photoelectric conversion element as an imaging element that receives and photoelectrically converts an object image incident through an optical system of a photographing lens (not shown) and generates an image signal corresponding to the received light amount. Is done. The image signal processing unit 108 includes an A / D converter 202 that converts an analog image signal level for each pixel into a digital value, a white balance circuit 203 that corrects color data for each pixel to a value corresponding to the color temperature of the light source, A linear matrix circuit 204 that performs matrix arithmetic processing on the data and performs color correction in accordance with a reproduction display system such as a liquid crystal display, and a γ conversion circuit 205 that performs gradation correction according to the gradation characteristics of the reproduction display system are provided.
[0024]
The image signal processing unit 108 converts RGB image data into YC data represented by luminance (Y) and color differences (Cr, Cb), and converts the converted Y, Cr, Cb data for each pixel. A YC conversion / synchronization processing circuit 206 to be synchronized and an image memory 207 for storing the synchronized Y, Cr, and Cb data are provided. Furthermore, the image signal processing unit 108 extracts a face image region from the captured image and separates it from a non-image region other than the face, and a sharpness parameter determination unit that determines a parameter representing the degree of image sharpness. 209, a region-specific sharpness processing unit 210 that performs sharpness processing on each of the separated face image region and non-face image region, and a color difference matrix circuit 211 that performs a matrix operation on each of the color differences Cr and Cb to minimize color reproduction change And a JPEG compression unit 212 that performs image compression processing on YCrCb data to generate output image data.
[0025]
Next, the operation of the camera-equipped cellular phone 1 configured as described above will be described by taking as an example a case where a person's subject is photographed and a photographed image is transmitted to a communication partner.
[0026]
A person's subject image captured by the CCD 201 via the taking lens is output as an image signal from the CCD 201 and converted into digital data by the A / D converter 202. The image data of the subject image converted into digital data is subjected to color correction by the white balance circuit 203 so as to achieve an appropriate white balance, and further subjected to matrix calculation processing using a predetermined color reproduction coefficient in the linear matrix circuit 204. Is done. Next, gradation conversion is performed by the γ conversion circuit 205 so that a predetermined γ characteristic corresponding to the display device or the like is obtained. The image data after γ conversion is converted from RGB data to YCrCb data in the YC conversion / synchronization processing circuit 206, and also subjected to synchronization processing for each pixel. The synchronized image data is temporarily stored in the image memory 207.
[0027]
Next, the face extraction processing unit 208 extracts the face image area of the subject from the image data. As a method for extracting a face image area, for example, the technique disclosed in Japanese Patent Laid-Open No. 9-101579 “Face Area Extraction Method and Copying Condition Determination Method” by the present applicant can be applied. This technology determines whether or not the hue of each pixel of the captured image is included in the skin color range and divides it into a skin color region and a non-skin color region, and detects edges in the image to detect each pixel in the image. The location is classified as an edge portion or a non-edge portion. Then, an area surrounded by pixels that are located in the skin color area and classified as a non-edge part and surrounded by pixels that are determined to be edge parts is extracted as a face candidate area, and the extracted face candidate area becomes a human face. It is determined whether the region corresponds to the region, and the face image region is extracted based on the determination result.
[0028]
The face image area thus extracted and the non-face image area other than the face obtained by separating the face image from the entire image are controlled for the brightness signal (Y) for each area. Sharpness processing is performed. The region-specific sharpness processing will be described in detail below with reference to FIGS.
[0029]
FIG. 3 is a block diagram showing the configuration of the region-specific sharpness processing unit according to the present embodiment, and FIG. 4 is an operation explanatory diagram for explaining the state of sharpness strength controlled in the region-specific sharpness processing unit of FIG. is there.
[0030]
The region-specific sharpness processing unit 210 includes a low-frequency separation low-pass filter 301 and a low-medium frequency extraction low-pass filter 302 that separately extract a low-frequency component and a low-medium frequency component from a luminance signal before processing, subtractors 303 and 304, A medium frequency component 305 and a high frequency amplifier 306 that amplify the medium frequency component and the high frequency component, respectively, and adders 307 and 308 are provided.
[0031]
In FIG. 3, the luminance signal Y before sharpness processing is input to a low-frequency separation low-pass filter 301 and a low-medium frequency extraction low-pass filter 302 and subjected to filtering processing, and a low-frequency component YL and a low-medium frequency component YLM are respectively obtained. Extracted. Then, the subtractor 303 subtracts the low frequency component YL from the low medium frequency component YLM to separate the medium frequency component YM, and the subtractor 304 subtracts the low medium frequency component YLM from the original luminance signal Y. Thus, the high frequency component YH is separated.
[0032]
The separated medium frequency component YM and high frequency component YH are amplified by a medium frequency amplifier 305 having a gain M and a high frequency amplifier 306 having a gain H, respectively. The amplified medium frequency component YM and high frequency component YH are added by the adder 307, and further added by the adder 308 to the low frequency component YL to become a signal in the entire frequency band, and as a luminance signal YS after sharpness processing. Is output.
[0033]
The degree of sharpness can be controlled as shown in FIG. 4 by varying the gains M and H of the medium frequency amplifier 305 and the high frequency amplifier 306, respectively. That is, the sharpness is enhanced by setting the gains M and H of the medium frequency component YM and the high frequency component YH high with respect to the low frequency component that is not amplified, and conversely, the sharpness is decreased by setting the gain low. .
[0034]
Thus, enhancing the sharpness is to increase the gains of the medium frequency component YM and the high frequency component YH, and as a result, the signal amplitude (contrast) of the medium frequency component YM and the high frequency component YH is increased. Therefore, the number of bits in a predetermined sampling period increases and the amount of information increases. Conversely, by reducing the gains of the medium frequency component YM and the high frequency component YH, the sharpness becomes weak and the amount of information decreases. In the present invention, this principle is used to control the sharpness of the image area other than the face of the person who is the main subject of photographing to be weak, thereby reducing the total amount of information and reducing the image data capacity. .
[0035]
Thereby, in an image distribution system via a network using a mobile terminal or the like, an image transmission including a communication means such as a communication path and a distribution server according to the distribution facility and distribution time of the mobile communication network or the specification of the mobile terminal. If there is a limit on the amount of data that can be distributed due to system configuration restrictions such as the configuration of the means and the configuration of the receiving terminal or operational restrictions (for example, 32 kbytes or less), this data capacity limitation is the main It is possible to appropriately reduce the image data capacity without degrading the image quality of the subject.
[0036]
Next, the sharpness control procedure according to this embodiment will be described with reference to the flowchart of FIG.
[0037]
First, in step S401, the sharpness parameter determination unit 209 determines the image sharpness parameter for each of the face image area and the non-face image area other than the face. As described above, the sharpness parameter corresponds to the gains M and H of the medium frequency amplifier 305 and the high frequency amplifier 306 in FIG. For example, in the face image area, the values of gains M and H are set to 1 or 1 or more, and the sharpness is not changed or enhanced. On the other hand, in the non-face image area, either one or both of the gains M and H is set to 1 or less to weaken the sharpness.
[0038]
Subsequently, in step S402, the region-specific sharpness processing unit 210 applies the image data of each of the face image region and the non-face image region extracted by the face extraction processing unit 208 based on the image data stored in the image memory 207. The sharpness processing is performed separately for the face image area and the non-face image area by the gains M and H of the sharpness parameter determined for each area. The sharpness processing for the entire image may be performed in advance in the YC conversion / synchronization processing circuit 206, and the sharpness processing unit 210 for each region may perform individual sharpness processing for each face image region and non-face image region.
[0039]
The processed image data subjected to the sharpness processing in this way is subjected to matrix calculation on the color difference Cr and Cb data in the color difference matrix circuit 211 in step S403, and then in step S404 the JPEG compression unit 212 performs JPEG. Performs image compression processing by the method.
[0040]
In step S405, it is determined whether or not the capacity of the image data after JPEG compression is within the limit capacity. If the capacity exceeds the limit capacity, the gain M of the medium frequency component and the gain H of the high frequency component are determined in step S406. Either or both of them are reduced.
[0041]
Thereafter, the processes in steps S402 to S405 are repeated, and the process ends when the data capacity of the JPEG-compressed image data is within the limit capacity. Next, the control unit 106 transmits the image data output from the image signal processing unit 108 to the communication partner via the communication unit 101.
[0042]
As described above, according to the configuration and procedure of the present embodiment, when taking a human image with the camera-equipped mobile phone 1 and transmitting it to the communication partner, by controlling the sharpness of the image area other than the face, The image data capacity can be reduced to a limit value that can be distributed without degrading the image quality of the face portion of the most important person in the photographed subject. In this case, by setting the sharpness parameter according to the limited capacity of the image data, it is possible to reduce the image data capacity while minimizing image quality degradation without excessively weakening the sharpness of the image area other than the face. it can.
[0043]
In this embodiment, only the face image area of the person is extracted and the sharpness of the image area other than the face is controlled. However, the person image area including the head or upper body is extracted, and the person other than the person is extracted. The sharpness of the image area may be controlled. In order to extract the person image area, a method of extracting a face image area first and then using a predetermined template corresponding to the size of the extracted face image area may be used.
[0044]
The sharpness parameter determined by the sharpness parameter determination unit 209, that is, the gains M and H of the medium frequency amplifier 305 and the high frequency amplifier 306 of the image data luminance signal are determined by the size of the face image region extracted by the face extraction processing unit 208. You may make it vary according to the area ratio which occupies for the whole image. In this case, the face extraction processing unit 208 calculates the area ratio of the face image area, and adjusts the sharpness parameter according to the area of the face image area. For example, if the extracted face image area is large, the area of the image area other than the face will be smaller and the importance will be reduced, so there will be no problem even if sharpness is reduced, and in order to secure data reduction, Further reduce the sharpness of the image area. That is, the gains M and H of the medium frequency amplifier 305 and the high frequency amplifier 306 are set smaller to reduce the amount of information.
[0045]
In the present embodiment, all the above-described processes are performed on the image of the person photographed by the camera-equipped cellular phone 1 inside the camera-equipped cellular phone 1, but a part of the process, for example, a face image is performed. Individual sharpness processing in the face image area and the non-face image area, such as region extraction, sharpness parameter determination, and sharpness processing, may be performed in the image distribution server of the image distribution system. In this case, the image distribution server is provided with an image memory, a face extraction processing unit, a sharpness parameter determination unit, and a region-specific sharpness processing unit.
[0046]
Furthermore, in the present embodiment, an example in which the image correction processing apparatus is applied to a camera-equipped mobile phone has been described, but the present invention is not limited to this. For example, the image correction processing apparatus of this embodiment can be provided in a digital camera that can be connected to a portable terminal or the like having a communication function. Alternatively, it is also possible to transfer an image photographed with a digital camera to a personal computer and execute image data processing including the sharpness processing of the present embodiment described above on the personal computer. In this case, image data processed by a personal computer can be transmitted to a server, another computer, a portable terminal, or the like via an information distribution system on the Internet, such as distribution as an attached file of an e-mail.
[0047]
According to this embodiment described above, when distributing a human image captured by a camera, the face image area of the person is extracted, and the sharpness of the image area other than the face is controlled, thereby degrading the image quality of the face portion. Therefore, it is possible to reduce the data capacity of the entire image by appropriately reducing the data capacity of the part other than the face, and it is possible to satisfy the limitation of the data capacity that can be distributed.
[0048]
【The invention's effect】
As described above, according to the present invention, when there is a limit on the data capacity that can be distributed in image distribution, the image data capacity is kept within the limited range without deteriorating the image quality of the face portion of the photographed person. It is possible to reduce the size.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of a main part of an embodiment of a camera-equipped mobile phone including an image correction processing apparatus according to the present invention.
FIG. 2 is a block diagram illustrating a configuration of an imaging unit and an image signal processing unit according to the present embodiment.
FIG. 3 is a block diagram illustrating a configuration of a region-specific sharpness processing unit according to the present embodiment;
FIG. 4 is an operation explanatory diagram for explaining a state of sharpness strength controlled in a region-specific sharpness processing unit of the present embodiment;
FIG. 5 is a flowchart showing a procedure of sharpness processing according to the present embodiment.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Mobile phone 101 with a camera 101 Communication part 102 Display part 103 Operation part 104 Transmission / reception part 106 Control part 107 Image pick-up part 108 Image signal process part 206 YC conversion and synchronization process part 207 Image memory 208 Face extraction process part 209 Sharpness parameter determination part 210 Region-specific sharpness processing unit 211 Color difference matrix unit 212 JPEG compression unit

Claims (3)

An image correction processing apparatus that performs image correction processing including sharpness processing on a human image,
Face extraction processing means for extracting a face image area in the person image;
Sharpness parameter determination means for determining the sharpness parameters of the face image area and the non-face image area so that the sharpness of the non-face image area other than the face is weaker than the sharpness of the face image area;
Region-specific sharpness processing means for performing sharpness processing individually for each area by changing the strength of the sharpness of the face image area and the non-face image area based on the sharpness parameter determined by the sharpness parameter determination means ;
Image compression means for compressing image data that has undergone sharpness processing of the face image area and the non-face image area;
The sharpness parameter determining means is an image transmitting means for distributing the image data in which the volume of the image data compressed by the image compressing means or a receiving terminal that is a distribution destination of the image data by the image transmitting means. An image correction processing apparatus , wherein the sharpness parameter of the non-face image area is determined so as to be within a limited capacity of distributable data caused by restrictions . The image correction processing apparatus according to claim 1 , wherein the sharpness parameter determination unit varies the sharpness parameter according to an area ratio of the face image region in the person image. Extracting a face image area in a person image;
Determining sharpness parameters of the face image region and the non-face image region such that the sharpness of the non-face image region other than the face is weaker than the sharpness of the face image region ;
Changing the sharpness of the face image area and the non-face image area based on the determined sharpness parameter and performing sharpness processing individually for each area;
An image correction processing program for causing a computer to execute a step of compressing image data subjected to sharpness processing of the face image region and the non-face image region,
In the step of performing the sharpness process, the capacity of the compressed image data is generated due to restrictions of an image transmitting unit for distributing the image data or a receiving terminal that is a distribution destination of the image data by the image transmitting unit. An image correction processing program for determining a sharpness parameter of the non-face image area so as to be within a limit capacity of distributable data .

Images