JP2004342003A - Image processing method and apparatus, and image processing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently obtain a desirable image with easy operation by reducing the number of times of processing required for acquisition of the desirable image when converting a portrait image to an image having a desired expression. <P>SOLUTION: A facial area and facial parts such as eyes, a nose, and a mouth are extracted from an original image photographed by a mobile phone 10 with a camera, and respective part areas are subjected to image processing in accordance with image processing parameters to obtain a desirable image. A user sets image processing parameters on an operation part such as a cross button 24 in order to obtain an image of the expression desired by the user. After the parameters are set to execute image processing once, default parameters in the next image processing are automatically set in accordance with actual results. When another image is processed thereafter, image processing is performed on the basis of the default parameters, and an image resulting from the processing is displayed on a display device 16. When the processing result is not the one desired by the user, parameters are changed furthermore to perform fine adjustment making the converted image as a reference. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing method and apparatus, and more particularly to a method, apparatus and system suitable for improving operability when processing a face image of a person photographed by a digital camera or a mobile phone with a camera into a favorite image. .
[0002]
[Prior art]
Recently, digital cameras, camera-equipped mobile phones, and the like have become widespread, and are used by many people as one of the easy photographing methods. In addition, conversion to an image of an intended expression is widely performed by performing various image processing on the original image data.
[0003]
Patent Document 1 proposes an image processing method capable of performing predetermined image processing on specific areas of a plurality of related frames.
[0004]
[Patent Document 1]
JP-A-11-275351
[0005]
[Problems to be solved by the invention]
When a favorite face image is generated by image processing of a person's face photograph, there are a wide variety of setting items such as processing contents and designation of a target part, and parameter setting operations are complicated. In particular, in a small mobile terminal such as a mobile phone, since the operation buttons are small and the number thereof is limited, a more complicated operation is forced.
[0006]
The present invention has been made in view of such circumstances, and an object of the present invention is to provide an image processing method, apparatus, and system capable of realizing complicated processing for a human image with a simple operation.
[0007]
[Means for Solving the Problems]
In order to achieve the above object, an image processing method according to the present invention is an image processing method for extracting a human face part from an original image and performing image processing on the extracted face part to generate an image of a desired expression. The image processing parameters defined for the processing target area can be changed according to the user operation, and the default parameters for the next image processing from the image processing parameters set in the image processing performed until the previous time Is automatically set, and then image processing reflecting the default parameter is performed when image processing is performed, the converted image generated by the processing is displayed on the display means, and further parameters are further set based on the default parameter. When a parameter adjustment operation for changing the parameter is received and the parameter is changed by the parameter adjustment operation, Which comprises carrying out the image processing in accordance with the modified image processing parameter.
[0008]
According to the present invention, at least one (preferably a plurality of) facial parts (facial parts) such as “eyes”, “nose”, and “mouth” are extracted from an original image including a person by image analysis, By performing image processing on the part area in accordance with image processing parameters, an image with a preferable expression can be obtained. A parameter that defines the content of image processing is defined for the region to be processed, and the user can set the image processing parameter through the user interface so that the image has the intended expression.
[0009]
Once the parameters are set and the image processing is performed, the next default parameter is automatically set based on the result, and the image processing is performed based on the default parameter after the next time. Since the converted image generated according to the default parameters is displayed on the display means, the user can confirm the image of the processing result by viewing the display. If the obtained result is not intended, a parameter adjustment operation is performed. This adjustment operation modifies the default parameter, for example, by specifying an increase / decrease value (change amount) based on the current default parameter. By adjusting the parameters based on the default parameters, desired conditions can be easily set.
[0010]
The image processing is performed again according to the parameter thus changed, and a converted image as a processing result is provided. The corrected parameter information is used for setting the next default parameter.
[0011]
As described above, according to the present invention, default parameters in the next image processing are automatically set based on the parameters of image processing performed in the past, and a desired image is obtained by processing based on the default parameters (default processing). If not, since further parameter adjustment is performed based on the parameter, the number of processes until a desired image is obtained can be greatly reduced.
[0012]
Further, according to the present invention, for example, when the processing target area covers the entire person, when the background area is distinguished from the person, or for parts such as hair, lips, torso, clothes, accessories, etc. It is also possible to store processing parameters for processing selected from the position, shape characteristics, density characteristics, color characteristics, texture characteristics, and spatial frequency characteristics and reflect them in subsequent conversions.
[0013]
In order to provide an apparatus that embodies the above method invention, an image processing apparatus according to the present invention is extracted by a facial part extracting unit that extracts a facial part of a person from an input original image, and the facial part extracting unit. Parameter changing operation means for performing an operation of changing the setting of the image processing parameter defined for the processing target area including the face part, and subjecting the target area of the original image to image processing according to the set image processing parameter Image processing means for generating an image of expression reflecting the parameters, and automatic default parameter setting for automatically setting default parameters in the next image processing from the image processing parameters set in the image processing performed until the previous time And default parameters set by the default parameter automatic setting means A display unit that displays the converted image generated by the image processing unit based on the parameter adjustment operation for further changing the parameter with reference to the default parameter, and a parameter adjustment instruction is input from the parameter change operation unit And image processing control means for performing image processing by the image processing means in accordance with the image processing parameter changed according to the instruction.
[0014]
According to one aspect of the present invention, the default parameter automatic setting means includes storage means for storing image processing parameters used in the previous image processing, and the previous image processing parameters are used as default parameters for the next image processing. It is characterized by setting to.
[0015]
In addition to setting the previous parameter as the default for the next time, set the default value for the most frequently used setting from the past processing history, or obtain the average value of the setting value from the past history and set the value. There may be a mode such as setting to default. When the default processing is determined from the accumulated history information, the processing parameters preferred by the user can be accurately reflected in the next data as the number of executions of the image processing increases.
[0016]
An image processing apparatus of the present invention includes a communication unit connected to a communication network, an imaging unit that converts an optical image of a subject into an electrical signal, an image recording unit that records an image captured through the imaging unit, There is a mode mounted on a mobile phone with a camera.
[0017]
A small mobile terminal typified by a camera-equipped mobile phone has a limited operation system such as a small operation button. Therefore, it is very beneficial to improve the operability according to the present invention. The image processing apparatus according to the present invention is not limited to a case where the image processing apparatus is realized by a single device such as a camera-equipped mobile phone, and a plurality of devices (devices) including a part of the image processing device. ) May be combined to realize the image processing apparatus of the present invention as a whole.
[0018]
Still another aspect of the present invention provides an image processing system that achieves the above object. In other words, image processing including a communication terminal including a communication unit that transmits original image data via a network, and an image processing server that processes image data transmitted from the communication terminal via the network. In the system, the image processing server includes a facial part extraction unit that extracts a facial part of a person from an original image acquired from the communication terminal, and a processing target area that includes the facial part extracted by the facial part extraction unit. Image processing means for performing image processing in accordance with set image processing parameters and generating an image of expression reflecting the parameters, and the converted image generated by the image processing means via the network via the communication terminal An image providing unit for providing the image processing parameter to the image processing parameter defined for the target area. Parameter changing operation means for performing an operation for changing the setting of the data, display means for displaying the converted image generated by the image processing means, and parameter information corresponding to an instruction from the parameter changing operation means. Parameter information providing means for providing to the processing server, and the image processing server or the communication terminal automatically receives default parameters for the next image processing from the image processing parameters set in the image processing performed until the previous time. Default parameter automatic setting means for automatically setting is provided, the converted image generated by the image processing means is displayed on the display means according to the default parameter set by the default parameter automatic setting means, and the default parameter is displayed. Further parameters on the basis of When a parameter adjustment operation to be changed is received and a parameter adjustment instruction is input from the parameter change operation means, parameter information corresponding to the instruction is provided to the image processing server, and the image changed according to the adjustment instruction Image processing by the image processing means is performed according to processing parameters.
[0019]
In the case where default parameter automatic setting means is provided on the communication terminal side, default parameter information is provided from the communication terminal to the image processing server when requesting image processing.
[0020]
Further, an aspect in which default parameter automatic setting means is provided on the image processing server side is also possible. In this case, parameter information designated by the operation system of the communication terminal is sent to the image processing server, and the parameter information is stored in the image processing server.
[0021]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.
[0022]
[First Embodiment]
FIG. 1 is an external view showing an example of a camera-equipped mobile phone according to the first embodiment of the present invention. The mobile phone 10 is provided with a camera unit 12, an antenna 14, a display 16, an earpiece 17, a mouthpiece 18, and an operation unit 20. The operation unit 20 includes a dial button 21, a call button 22, a call disconnect / power button 23, a cross button 24, an execution button 25 and other various operation means. The operation unit 20 is not limited to a push-type switch member, and may be configured as a dial member, a lever switch, or the like.
[0023]
The display 16 is composed of a liquid crystal display capable of color display. Of course, other types of display devices such as organic EL may be applied. Reference numeral 27 in FIG. 1 denotes a speaker for outputting sound during reproduction of moving images, various sound effects, and the like. Although not explicitly shown in FIG. 1, the mobile phone 10 is provided with a microphone (denoted as reference numeral 28 in FIG. 2) for capturing sound during moving image shooting.
[0024]
As shown in FIG. 1, the camera unit 12 is rotatably attached to the housing 29 of the mobile phone 10. The shooting direction can be freely changed by turning the camera unit 12, and the camera can be directed to the display surface side of the display 16 as shown, or the camera can be directed to the back side of the display 16. . Instead of the camera unit 12 with the rotation function, a mode in which a camera unit fixed at a predetermined position is mounted, and a mode in which a plurality of camera units are mounted is also possible.
[0025]
FIG. 2 is a block diagram showing the configuration of the mobile phone 10. As shown in the figure, the mobile phone 10 is generally composed of a central processing unit (CPU) 30, a memory 32, a voice processing unit 34, a transmission / reception circuit 36, a speaker 37 of the earpiece 17, and a microphone 38 of the mouthpiece 18. It has the necessary components to realize the telephone function.
[0026]
The CPU 30 functions as a control unit that controls the operation of the entire mobile phone 10 according to a predetermined program, and also functions as an arithmetic device that performs various calculations. That is, the CPU 30 controls each circuit in the mobile phone 10 based on a command signal input from the operation unit 20, and performs communication control, shooting operation control, image processing control, display 16 display control, and memory 32 read / write control. And capacity management (including detection of free capacity).
[0027]
The CPU 30 is connected to the memory 32 via the bus 39. The memory 32 is a block including a non-volatile memory area in which a program executed by the CPU 30 and various data necessary for control are stored, and a volatile memory area used as a program development area and a calculation work area of the CPU 30 It is. The non-volatile memory area is used as a data storage area for a telephone directory and a call history, and also as a storage area for voice and images. Information on processing parameters in the image processing function of the present invention is stored in a nonvolatile memory area. The volatile memory area is also used as a temporary storage area for image data.
[0028]
The audio processing unit 34 is a means for processing or converting an audio signal in cooperation with the CPU 30. The audio processing unit 34 performs decoding processing of data received from the transmission / reception circuit 36 and encoding processing of data transmitted through the transmission / reception circuit 36, output signal generation processing to the speakers 27 and 37, and output from the microphones 28 and 38. Performs encoding processing of the input audio signal.
[0029]
When a communication connection with the communication partner is established in the call mode (mode using the telephone function), the voice input from the microphone 28 is converted into an electric signal and sent to the voice processing unit 34. The audio processing unit 34 converts the audio signal input from the microphone 38 into a digital signal (A / D conversion), performs necessary signal processing, and converts it into audio data of a predetermined format. The audio data (data for transmission) processed by the audio processing unit 34 is sent to the transmission / reception circuit 36 via the bus 39, converted into a predetermined transmission signal format, and then transmitted from the antenna 14.
[0030]
The signal received by the antenna 14 is demodulated by the transmission / reception circuit 36 and sent to the audio processing unit 34. The audio processing unit 34 converts the audio signal received from the transmission / reception circuit 36 into audio data in a predetermined format. The audio data (received data) generated by the audio processing unit 34 is converted into an analog audio signal (D / A conversion) and then sent to the speaker 37. Thus, the voice of the other party is audibly output from the speaker 37. Of course, the cellular phone 10 can transmit and receive various data such as image data, document data, and program data by using not only voice data but also an electronic mail function and an Internet browser function.
[0031]
Further, the cellular phone 10 includes a photographing lens 40 and a CCD image sensor (hereinafter referred to as CCD) 42, an analog processing unit 44 for processing an image signal obtained from the CCD 42, an A / D converter 45, an image. A processing unit 46 and a compression / decompression unit 47 are provided. In this example, a CCD is used as the image pickup element. However, the device is not limited to the CCD type, and other types of devices such as a CMOS type may be used.
[0032]
The light that has passed through the photographing lens 40 is focused on the light receiving surface of the CCD 42. A large number of photosensors (light receiving elements) are arranged in a plane on the light receiving surface of the CCD 42, and red (R), green (G), and blue (B) primary color filters are arranged in a predetermined arrangement corresponding to each photosensor. Arranged in structure. A color filter such as CMY can be used instead of the RGB color filter.
[0033]
The subject image formed on the light receiving surface of the CCD 42 is converted into a signal charge of an amount corresponding to the amount of incident light by each photosensor. The CCD 42 has an electronic shutter function that controls the charge accumulation time (shutter speed) of each photosensor according to the timing of the shutter gate pulse.
[0034]
The signal charges accumulated in each photosensor of the CCD 42 correspond to the signal charges on the basis of pulses (horizontal drive pulse φH, vertical drive pulse φV, overflow drain pulse) given from the timing generator (TG) 48 in accordance with instructions from the CPU 30. It is sequentially read out as a voltage signal (image signal). The image signal output from the CCD 42 is sent to the analog processing unit 44, and after necessary processing such as correlated double sampling (CDS) processing and gain adjustment, it is converted into a digital signal by the A / D converter 45. Is done. The digitized image data is sent to the image processing unit 46.
[0035]
The image processing unit 46 includes a synchronization circuit (a processing circuit that calculates the color of each point by interpolating a spatial shift of the color signal associated with the color filter array of the single CCD), a luminance / color difference signal generation circuit, and a gamma correction. The digital image signal processing means includes a circuit, a contour correction circuit, a white balance correction circuit, and the like, and processes an image signal while utilizing the memory 32 in accordance with a command from the CPU 30.
[0036]
The image processing unit 46 has been generated by performing predetermined processing such as white balance adjustment processing, gamma conversion processing, conversion processing to luminance signals (Y signals) and color difference signals (Cr, Cb signals) (YC processing). The image data is temporarily stored in the memory 32. Here, in the case where the image being shot is displayed on the display 16, the contents of the memory 32 are read out and sent to the display circuit 50, converted into a display signal format by the display circuit 50, and then supplied to the display 16. . As a result, the real-time video (through image) captured by the CCD 42 is displayed on the display 16, and the photographer can check the shooting angle of view (composition) from the display video on the display 16.
[0037]
When a photographing button on the operation unit 20 (for example, the execution button 25 is also used) is pressed, the CPU 30 detects this and performs a recording photographing operation. That is, exposure control of the CCD 42 and charge read-out control are performed in response to pressing of the photographing button. The image data for recording thus captured is subjected to YC processing and other predetermined signal processing in the image processing unit 46, and then sent to the compression / decompression unit 47, where it is compressed according to a predetermined compression format (for example, JPEG format). Is done. The compressed image data is stored in the image storage area of the memory 32.
[0038]
When the moving image shooting mode is selected, the recording operation starts when the shooting button is pressed, and then the moving image recording operation stops when the shooting button is pressed again. Note that the recording operation may be performed while the shooting button is continuously pressed, and the recording may be stopped by releasing the pressing.
[0039]
The sound at the time of moving image shooting is detected by the microphone 28, and the detection signal (audio signal) is converted into a predetermined signal format by the audio processing unit 34. The audio data is compressed together with the image data in the compression / decompression unit 47 according to a predetermined compression format (for example, MP4 format) and then recorded in the memory 32.
[0040]
At the time of image reproduction, image file data is read from the memory 32 in response to a user file selection operation or automatically. The compressed data read from the memory 32 is decompressed by the compression / decompression unit 47, converted into a display signal via the display circuit 50, and then output to the display. Thereby, the image content of the file is displayed on the display.
[0041]
FIG. 3 is a functional block diagram showing the main part of the image processing apparatus mounted on the mobile phone 10.
[0042]
As illustrated, the image processing apparatus 52 includes a face area extraction unit 54, a face part extraction unit 55, an image conversion unit 56, and a parameter data storage unit 57. The image processing device 52 can be realized by an appropriate combination of hardware and software. In the case of this example, the face area extraction unit 54, the face part extraction unit 55, and the image conversion unit 56 are included in the image processing unit 46 of FIG. 2, and the parameter data storage unit 57 is included in the memory 32 of FIG.
[0043]
As shown in FIG. 3, the data of the original image (processed image) 60 including the face image is input to the image processing device 52 and sent to the face area extraction unit 54 and the face part extraction unit 55. The face area extraction unit 54 analyzes the input image data and extracts a face area from the image. The face part extraction unit 55 extracts face parts such as eyes, nose, mouth, and eyebrows.
[0044]
As a method for extracting a face region from the original image 60, for example, feature extraction processing is performed in order to extract face parts such as eyes, nose and mouth in the original image. In the feature extraction process, wavelet transformation is performed to extract and quantize wavelet coefficients having appropriate positions and frequencies. A matching process is performed with face part dictionary data created by performing a similar feature extraction process from a large number of sample images in advance to extract a face part.
[0045]
The position from which the facial parts are extracted is determined as a face, and division processing by color and texture is performed. This is a process of dividing similar color and texture areas together, for example, the skin color area including the eye coordinates is used as the face area, and the black and brown areas that are slightly above the eye coordinates are used as the hair area. To extract a person area. In addition, the human area dictionary data indicating the average positional relationship between the eye position and the boundary line between the person and the background is matched with the boundary line between the person area obtained from the original image and the background area, The boundary line between the background area and the background area
[0046]
Also, a filtering process for extracting a boundary line between a person and a background from a high-frequency component in the original image 60, a method for extracting a person area from the original image 60, a skin color in the original image 60, and a skin color area. The connected regions that are considered to belong to the same region are sequentially expanded from a certain point, and the face region is extracted depending on whether or not the shape of the extracted region is the shape of the face. There is a method of extracting a person region by extracting a hair region at the top of the region, a neck and a chest region at the bottom of the face, and the like. Various methods can be considered as a method for extracting a person region from the original image 60, and the method is not limited to the above method.
[0047]
The face part information extracted by the face area extraction unit 54 and the face part extraction unit 55 is sent to the image conversion unit 56. The image conversion unit 56 refers to the image processing parameters in the parameter data storage unit 57 for the extracted target regions such as the eye region, the nose region, and the mouth region, performs image processing according to the parameters, and sets the parameters. Generate an image of the reflected expression.
[0048]
The parameter data storage unit 57 stores image processing parameter data used in the previous image processing, and the stored image processing parameter is set as a default parameter. When the past image processing parameters are not stored in the parameter data storage unit 57, an operation for inputting a desired parameter from the operation unit 20 is performed.
[0049]
In this way, an image (converted image 62) reflecting the parameters is generated in the image conversion unit 56. The converted image 62 generated by the image conversion unit 56 is displayed on the display 16.
[0050]
When the displayed image is not intended, the user performs fine adjustment of the parameter from the operation unit 20. For example, an adjustment value (designation of a change amount based on a default parameter) for the current setting parameter is input by using a cross button or the like, or a parameter change value is selected from a menu screen. In this way, the image processing in the image conversion unit 56 is performed again based on the adjusted parameters, the converted image 62 reflecting the corrected parameters is generated, and the display image on the display 16 is updated. In this way, the processing result image after parameter adjustment is displayed on the display 16.
[0051]
The above operation is repeated until a favorite image is obtained. When the image of the intended expression is obtained, the image processing operation is terminated. At this time, information on the image processing parameters from which a favorite image is obtained is stored in the parameter data storage unit 57.
[0052]
When performing image processing for another image next time, the previous image processing parameter stored in the parameter data storage unit 57 is set as a default, so that the same processing result as the previous time can be easily obtained.
[0053]
FIG. 4 is a flowchart showing a procedure of image processing in the mobile phone 10, and FIG. 5 is a diagram showing an example of a screen. First, a face image is displayed on the display 16 of the mobile phone 10 (step S110 in FIG. 4). The image to be displayed may have various modes such as a preview image immediately after being captured by the camera unit 12, a playback image of a file stored in the memory 32, and an image received by mail.
[0054]
As shown in FIG. 5A, simultaneously with the display of the face image, an “image processing” command menu 71 is displayed at the lower right of the screen of the display 16. This menu display means that the function of “image processing button” is assigned to the operation button 72.
[0055]
When desiring to perform image processing on the displayed face image, the user presses an image processing button (operation button 72). When an operation for selecting an “image processing” command is performed (step S112 in FIG. 4), the CPU 30 detects this and determines the presence or absence of processing parameter data (step S114). If there is a track record of performing image processing before, information on processing parameters at that time is stored in the memory 32. When there is parameter data stored in the memory 32 (YES in step S114), image processing reflecting the setting of the processing parameter is executed, and the converted image (processing result image) is displayed on the display 16. (Step S118).
[0056]
On the other hand, if it is determined in step S114 that the process parameter data is not stored (NO determination), a process parameter setting operation is accepted (step S116).
[0057]
FIG. 6 is a chart showing an example of image processing parameters. According to FIG. 6, the face, head hair, eye part, lips, torso, clothes, accessories, nose, background, etc. are determined as the processing target areas, and the shape, position, density characteristics, color, etc. are determined for each target area. Image feature quantities such as characteristics, texture characteristics, and spatial frequency characteristics can be set.
[0058]
A step adjustment rate of ± 1, ± 2, ± 3 is defined for the “shape” of the face in the left / right and up / down directions, and the user can use the +/− buttons (also used with the up / down keys of the cross button) as desired. By operating and inputting parameters, the face size is increased or decreased. Similarly, step adjustment rates of ± 1, ± 2, and ± 3 are defined for the shapes of the eyes and lips.
[0059]
As for the shape of “hair”, a plurality of hair images (part images for pasting and combining) representing various hair styles are prepared in advance, and the user can select a favorite hair image. . FIG. 6 shows that the hair image defined for type “C” is selected. With respect to the shape of the “jewelry”, a plurality of types of clothing accessory images are prepared in advance, and in FIG. 6, a clothing accessory image defined as the type “R” is selected. As the shape of “nose”, a plurality of types such as a high nose and a low nose are defined. FIG. 6 shows that the type “B” is selected.
[0060]
In addition, with regard to shape, position, density characteristics, color characteristics, texture characteristics, and spatial frequency characteristics, adjustment values (for example, ± 1, ± 2, ± 3) for adjusting each feature amount in stages are defined. Has been. The user can designate a desired adjustment value using the cross button 24 or the like. For example, the eye position is adjusted +1 in the vertical direction, and the nose position is adjusted +2 in the vertical direction.
[0061]
In this way, image processing (including synthesis processing for pasting a part image for synthesis) is performed according to the set image processing parameters, and a converted image reflecting the parameters is displayed on the display 16 (step S118 in FIG. 4). At this time, as shown in FIG. 5B, simultaneously with the display of the image indicating the conversion result, a command menu 74 “OK” and a command menu 75 “Further processing” are displayed below the screen of the display 16. The These menu displays mean that the command function of “OK now” is assigned to the operation button of reference numeral 76 and the command function of “further processing” is assigned to the operation button of reference numeral 72.
[0062]
If the user does not like the converted image displayed on the display 16, the “further processing” button (operation button 72) is pressed to move to the parameter change input screen to adjust the parameters. On the other hand, when it is determined that the converted image displayed on the display 16 is OK, the “OK now” button (operation button 76) is pressed to end the process.
[0063]
That is, instruction input from the user is waited in a state where the converted image is displayed in step S118 of FIG. 4, and it is determined whether or not the converted image being displayed is acceptable in accordance with the input instruction (step S120). If the “further processing” button is pressed, a NO determination is made in step S120, and the process proceeds to step S122. In step S122, an operation for further adjusting the processing parameters based on the parameters used in the previous processing is accepted. For example, when the parameter value used in the previous processing is “+2”, “+1” is further added to the value to correct it to “+3”, or “−1” is adjusted from “+2”. You can go and correct it to “+1”.
[0064]
When parameter adjustment is performed in step S122, the process returns to step S118, image processing is performed based on the changed parameter, and the converted image after processing is displayed on the display 16.
[0065]
In step S120, when the “OK now” button is pressed, this sequence ends.
[0066]
According to the cellular phone 10 of this example, once image processing parameters are set for a certain face image A and image processing is performed, the image processing parameters at that time are stored and become the next default parameters. Therefore, after that, when image processing is performed on another face image B, when the “image processing” button is pressed, the default parameters are automatically referred to, and the processing reflecting the parameters applied to the face image A is the default processing. The processing result is displayed on the display 16.
[0067]
If you don't like the result of this default processing, you can change the parameters further and make fine adjustments based on the parameters related to the settings. This saves you the trouble of setting the parameters every time you convert from the original image to your favorite image. be able to.
[0068]
[Modification]
The processing target (intent) for what kind of image to finish is considered to vary depending on the characteristics, gender, age, or purpose of use of the person who is the subject. An aspect in which image processing parameters are stored for each category and for each person is also preferable. For example, it is possible to perform a whitening process only for women by switching the default process according to the gender of the subject person.
[0069]
In addition, there is an aspect in which the address book registered in the memory 32 of the mobile phone 10 is used to specify the subject person, and the image processing parameters for the person image are stored by linking with the address book. As a result, different default processing can be performed for each person.
[0070]
Furthermore, by applying facial part extraction technology, the person in the image is automatically recognized from features such as eye spacing, nose position, and other parts coordinate ratios. It is also possible to store the default parameters for each person, such as the default for personal use.
[0071]
[Second Embodiment]
FIG. 7 is a configuration diagram of an image processing system according to the second embodiment of the present invention. This system realizes a service for sending a person image (face photograph) from the client terminal 210 to the image processing center 230 via the network 220, performing image processing in the image processing center 230, and providing the image of the processing result to the user. To do.
[0072]
The client terminal 210 can take various forms such as a mobile phone, a PDA, and a mobile personal computer. The client terminal 210 includes a communication means for transmitting and receiving data by connecting to the network 220, and is preferably equipped with an electronic camera that converts an optical image of a subject into digital image data. In the case of a terminal device that does not have a photographing function, a photographing device (not shown) such as a digital camera is connected via a USB or other communication interface to acquire image data from the outside, or a memory card or By storing image data in the internal memory, it is possible to obtain an image for processing request.
[0073]
The network 220 is, for example, a wide area communication line network represented by the Internet, and includes various communication line networks such as a telephone line network, a mobile phone line network, and a dedicated line network regardless of wired or wireless.
[0074]
The image processing center 230 includes a communication unit server 231, an image processing server 232, and a data storage server 233. These servers 231 to 233 are connected to each other via a communication line 234 such as a LAN or an intranet so that they can communicate with each other. ing. The communication unit server 231 is connected to the network 220 and mainly performs communication control with the client terminal 210. The image processing server 232 performs processing for extracting a face area from an input image (image to be processed), processing for extracting a face part, transformation, replacement, synthesis, correction, and other conversion processes for the target area according to processing parameters. It is a processing part provided with a means. The storage of the data storage server 233 stores databases such as part images for synthesis, user (user) registration information, and image processing parameter setting information (see FIG. 6) managed for each user.
[0075]
The operation of the image processing system configured as described above is as follows. First, the client terminal 210 is used to access the home page of the image processing center 230 and upload a person image (original image) to be processed.
[0076]
The communication unit server 231 authenticates the user and transfers the received image to the image processing server 232. When a user who is a requester of processing is specified, the database of the data storage server 233 is referred to based on the user information. When the image processing parameter setting information is not stored for the user (for example, when this service is used for the first time), the client terminal 210 is requested to perform an operation for setting the image processing parameter. When the user performs an input for designating a desired parameter through the operation system of the client terminal 210, the parameter information is transmitted to the image processing server 232 via the communication unit server 231 and registered in the data storage server 233.
[0077]
The image processing server 232 processes the original image in accordance with the designated image processing parameter, and generates a converted image reflecting the parameter. The converted image generated by the image processing server 232 is provided to the client terminal 210 via the network 220 via the communication unit server 231 and displayed on the display 212 of the client terminal 210.
[0078]
If the user confirms the image of the processing result and does not like the image, he / she can perform a parameter adjustment operation based on the current setting parameter and request reprocessing. The image processing parameter database of the data storage server 233 is updated in accordance with the parameter adjustment operation.
[0079]
Next time, when the same user requests processing for another image, the same image processing as the previous time is performed based on the information of the image processing parameter of the user stored in the data storage server 233, and the processing result image Provided to the ant terminal 210.
[0080]
If the user does not like the result of the default process, the parameter adjustment operation is performed based on the current setting parameter (default parameter), and a re-process is requested. The image processing parameter database of the data storage server 233 is updated in accordance with the parameter adjustment operation.
[0081]
In this way, by automatically setting the default parameters for each user based on the processing results, it is possible to efficiently perform preferable image processing unique to each user with a simple operation.
[0082]
7 exemplifies a server system constructed by a plurality of servers 231 to 233 with respect to the configuration of the image processing center 230. However, in implementing the present invention, functions similar to those of the image processing center 230 are performed by a single computer. It can also be realized, or can be realized by appropriately combining a plurality of computers.
[0083]
In the example of FIG. 7, the image processing parameter information is managed for each user in the data storage server 233 on the image processing center 230 side. However, when the present invention is implemented, the image processing parameter information is stored on the client terminal 210 side. A mode of storing is also possible. In this case, the default parameter information is transmitted from the client terminal to the server side together with the image to be processed.
[0084]
[Other application examples]
The target of image processing is not limited to a still image but may be a moving image. That is, according to the present invention in which the stored image processing parameters are applied to other images, the present invention can be applied to continuous images such as a series of still images by moving images or continuous shooting. For example, by specifying image processing parameters for one frame of a moving image and deforming the image, the processing parameters can be reflected on all or some of the frames.
[0085]
【The invention's effect】
As described above, according to the present invention, face processing is performed in the past in image processing for extracting a facial part from an original image and converting the processing target area into an image of a desired expression by performing image processing according to image processing parameters. Default parameters for the next image processing are automatically set based on the image processing parameters, and if the result of processing based on the default parameters is not intended, the parameters can be further adjusted based on the parameters. Therefore, you can get your favorite image with a simple operation.
[Brief description of the drawings]
FIG. 1 is an external view showing an example of a camera-equipped mobile phone according to a first embodiment of the present invention.
FIG. 2 is a block diagram showing the configuration of the mobile phone shown in FIG.
FIG. 3 is a functional block diagram showing a main part of an image processing apparatus mounted on the mobile phone of the present example.
FIG. 4 is a flowchart showing a procedure of image processing in the mobile phone of this example.
FIG. 5 is a diagram showing a display example (a) of an original image before processing and a display example (b) of a converted image after processing;
FIG. 6 is a chart showing examples of image processing parameters.
FIG. 7 is a configuration diagram of an image processing system according to a second embodiment of the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 ... Mobile phone, 12 ... Camera part, 16 ... Display, 20 ... Operation part, 24 ... Cross button, 30 ... CPU, 32 ... Memory, 42 ... CCD, 52 ... Image processing apparatus, 54 ... Face area extraction part, 55 ... face part extraction unit, 56 ... image conversion unit, 57 ... parameter data storage unit, 210 ... client terminal, 220 ... network, 230 ... image processing center, 231 ... communication unit server, 232 ... image processing server, 233 ... data storage server

Claims (5)

An image processing method for extracting a human face part from an original image and performing image processing on the extracted face part to generate an image of a desired expression,
The image processing parameters defined for the processing target area can be changed according to user operations.
The default parameters for the next image processing are automatically set from the image processing parameters set in the image processing performed until the previous time,
After that, when image processing is performed, image processing reflecting the default parameters is performed,
While displaying the converted image generated by the processing on the display means, accepting a parameter adjustment operation to further change the parameter with reference to the default parameter,
When a parameter is changed by the parameter adjustment operation, image processing is performed according to the changed image processing parameter. A facial part extracting means for extracting a human facial part from the input original image;
A parameter changing operation means for performing an operation of changing the setting of the image processing parameter defined for the processing target region including the facial part extracted by the facial part extracting means;
Image processing means for performing image processing on a target area of the original image according to set image processing parameters, and generating an image of expression reflecting the parameters;
Default parameter automatic setting means for automatically setting default parameters in the next image processing from the image processing parameters set in the image processing performed until the previous time;
Display means for displaying the converted image generated by the image processing means based on the default parameters set by the default parameter automatic setting means;
When a parameter adjustment operation for further changing a parameter is received on the basis of the default parameter and a parameter adjustment instruction is input from the parameter change operation means, the image processing means performs the operation according to the image processing parameter changed according to the instruction. Image processing control means for performing image processing;
An image processing apparatus comprising: The default parameter automatic setting means includes storage means for storing an image processing parameter used in the previous image processing, and sets the previous image processing parameter as a default parameter for the next image processing. Item 3. The image processing apparatus according to Item 2. A camera-equipped mobile phone comprising: a communication unit connected to a communication network; an imaging unit that converts an optical image of a subject into an electrical signal; and an image recording unit that records an image captured through the imaging unit. The image processing apparatus according to claim 2, wherein the image processing apparatus is mounted. An image processing system comprising: a communication terminal having communication means for transmitting original image data via a network; and an image processing server for processing image data sent from the communication terminal via the network. There,
The image processing server includes a facial part extracting unit that extracts a facial part of a person from an original image acquired from the communication terminal;
Image processing means for performing image processing according to a set image processing parameter on a processing target region including the face part extracted by the face part extracting means, and generating an expression image reflecting the parameter;
Image providing means for providing the converted image generated by the image processing means to the communication terminal via the network;
With
The communication terminal includes a parameter changing operation means for performing an operation of changing the setting of an image processing parameter defined for the target area;
Display means for displaying the converted image generated by the image processing means;
Parameter information providing means for providing parameter information corresponding to an instruction from the parameter change operation means to the image processing server;
With
The image processing server or the communication terminal is provided with default parameter automatic setting means for automatically setting a default parameter for the next image processing from the image processing parameters set in the image processing performed until the previous time. ,
Displaying the converted image generated by the image processing unit according to the default parameter set by the default parameter automatic setting unit on the display unit, and accepting a parameter adjustment operation for further changing the parameter based on the default parameter,
When a parameter adjustment instruction is input from the parameter change operation means, parameter information corresponding to the instruction is provided to the image processing server, and according to the image processing parameter changed according to the adjustment instruction, the image processing means An image processing system, wherein image processing is performed.

Images