JP4930245B2 - Image processing device - Google Patents

Description

  The present invention relates to image processing.

  Conventionally, various image processing is performed on image data. Examples of such image processing include processing for adjusting brightness and processing for correcting the outline of a person's face. Here, a technique for executing image processing using parameters adjusted by a user is also known. Examples of such parameters include the degree of brightness adjustment and the degree of deformation. By using such a technique, it is possible to adapt the image processing to the user's preference.

JP 2004-318204 A

  In such an image processing technique, it is preferable to reflect the user's preference in advance in the initial setting used when the user does not want to change the parameter. However, in the past, the actual situation is that sufficient ideas have not been made to reflect the user's preferences in the initial settings.

  The present invention has been made to solve at least a part of the above-described problems, and an object of the present invention is to provide a technique capable of reflecting user preferences in initial settings.

  SUMMARY An advantage of some aspects of the invention is to solve at least a part of the problems described above, and the invention can be implemented as the following forms or application examples.

Application Example 1 An image processing apparatus that displays an analysis unit that analyzes image data, and a setting screen that allows setting of control parameters used for image adjustment of the image data, and settings on the setting screen A user interface control unit that determines the control parameter according to: an image adjustment unit that performs the image adjustment on the image data using the determined control parameter; and an analysis result of the image data by the analysis unit; A storage unit that stores relationship information representing a correspondence relationship between the control parameter settings, the user interface control unit, the analysis result of one or more target image data, and each of the target image data In order to correct the relationship information by using the control parameter setting set on the setting screen Image processing for determining an initial setting of the control parameter displayed on the setting screen for correct target image data using the analysis result of the new target image data and the corrected relation information apparatus.

  According to this image processing apparatus, the relationship information is corrected by using the analysis result of the target image data and the setting of the control parameter, and the initial setting of the control parameter for the new target image data is newly performed. Since the determination is made using the analysis result of the target image data and the corrected relationship information, it is possible to reflect the user's adjustment preference that can change depending on the image data in the initial setting.

Application Example 2 In the image processing apparatus according to Application Example 1, the control parameter indicates the degree of image adjustment, and the correction of the relationship information is performed by analyzing the target image data according to the relationship information. The image processing apparatus is executed such that the degree of image adjustment associated with the image adjustment degree approaches the degree of image adjustment set on the setting screen.

  According to this configuration, since the relationship information is corrected so that the degree of image adjustment approaches the degree of image adjustment set on the setting screen, the initial setting can be appropriately changed depending on the image data. It can be adapted to the user's adjustment preferences.

Application Example 3 In the image processing apparatus according to Application Example 2, the analysis result includes at least one type of feature amount obtained by analyzing the image data, and the relation information includes the feature amount and An image processing apparatus, which is a look-up table including a plurality of grid points indicating a correspondence relationship with the degree of image adjustment.

  According to this configuration, by using a plurality of grid points included in the lookup table, it is possible to easily associate the feature amount obtained by analyzing the image data with the degree of image adjustment.

Application Example 4 In the image processing apparatus according to Application Example 3, the correction of the relationship information includes a first correction mode, and the feature indicated by the analysis result of the target image data in the first correction mode. An image processing apparatus, wherein a reference point indicating a correspondence relationship between an amount and a degree of image adjustment set on the setting screen is added to the lookup table.

  According to this configuration, the degree of image adjustment set on the setting screen can be reliably reflected in the relationship information.

Application Example 5 In the image processing apparatus according to Application Example 3 or Application Example 4, the correction of the relationship information includes a second correction mode. In the second correction mode, (a) the target image data A reference point indicating a correspondence relationship between the feature amount indicated by the analysis result and the degree of image adjustment set on the setting screen, and a part of the plurality of lattice points. (B) At least a part of the grid points used for calculating the representative reference point is deleted from the lookup table, and (c) the representative reference point is stored in the lookup table. Is added to the image processing apparatus.

  According to this configuration, at least a part of the grid points used for calculating the representative reference point is deleted from the lookup table, and the representative reference point is added to the lookup table. The initial setting can be appropriately adapted to the user's adjustment preferences, which can vary depending on the image data, while preventing the image from becoming excessively large.

Application Example 6 In the image processing apparatus according to any one of Application Example 3 to Application Example 5, in the lookup table, at least a part of the at least one type of feature value is an entire range of the feature value. Including the edge grid point set to the maximum value or the minimum value, and the correction of the relationship information is such that the feature quantity indicated by the analysis result of the target image data is close to the feature quantity of the edge grid point And a third correction mode that is executed when a predetermined condition indicating is satisfied, and in the third correction mode, the feature amount indicated by the analysis result of the target image data and the image adjustment set on the setting screen The degree of image adjustment indicated by the edge lattice point is corrected by using a reference point indicating a correspondence relationship with the degree of the image and at least a part of the plurality of lattice points. Image processing apparatus.

  According to this configuration, when the feature value indicated by the analysis result is close to the feature value of the edge lattice point, the degree of image adjustment indicated by the edge lattice point is corrected. It is possible to suppress an excessively large change in degree due to a small change in.

[Application Example 7] The image processing apparatus according to any one of Application Example 3 to Application Example 6, wherein the user interface control unit further includes the image corresponding to a change in the feature amount as a result of the correction of the relation information. When a sudden change portion in which the rate of change in the degree of adjustment is larger than a predetermined threshold occurs in the lookup table, the image adjustment of the lattice point representing the sudden change portion is performed following the correction of the relationship information. An image processing apparatus for smoothing the degree of the image.

  According to this configuration, when a sudden change portion occurs in the lookup table, the degree of the reference point representing the sudden change portion is smoothed, so that the degree changes excessively greatly due to a small change in the analysis result. Can be suppressed.

Application Example 8 In an image processing method, a step of displaying a setting screen capable of setting a control parameter used for image adjustment of image data, an analysis result of one or more target image data, Correcting the relationship information indicating the correspondence between the analysis result of the image data and the setting of the control parameter by using the setting of the control parameter set on the setting screen for the target image data; Determining an initial setting of the control parameter displayed on the setting screen for new target image data using an analysis result of the new target image data and the corrected relation information A method comprising:

Application Example 9 A computer program for processing an image, which has a function of displaying a setting screen capable of setting a control parameter used for image adjustment of image data, and an analysis result of one or more target image data By using the control parameter setting set on the setting screen for each target image data, the relationship information representing the correspondence between the analysis result of the image data and the control parameter setting is corrected. And the initial setting of the control parameter displayed on the setting screen for new target image data using the analysis result of the new target image data and the corrected relation information A computer program that causes a computer to execute a function to be determined.

  The present invention can be realized in various forms, for example, an image processing method and apparatus, a computer program for realizing the function of the method or apparatus, a recording medium on which the computer program is recorded, It can be realized in the form of a data signal including the computer program and embodied in a carrier wave.

Next, embodiments of the present invention will be described in the following order based on examples.
A. First embodiment:
B. Second embodiment:
C. Image processing:
D. Variations:

A. First embodiment:
FIG. 1 is an explanatory diagram showing a printer to which an image processing apparatus according to an embodiment of the present invention is applied. The printer 100 is a printer that prints an image using image data acquired from a memory card, a digital still camera, or the like. The printer 100 includes a CPU 110, an internal memory 120, an operation unit 140, a display unit 150, a printer engine 160, a card interface (card I / F) 170, a slot 172, and a nonvolatile memory 180. ing. Each component of the printer 100 is connected to each other via a bus.

  The CPU 110 controls each component of the printer 100. The internal memory 120 stores a program executed by the CPU 110. As the internal memory 120, any memory that stores data used by the CPU 110 can be adopted. For example, various semiconductor memories including a RAM and a ROM and a hard disk drive can be adopted.

  The nonvolatile memory 180 stores parameter related information PI. The memory 180 is a rewritable nonvolatile memory. As such a memory, for example, a semiconductor memory such as a flash memory or an EEPROM, or a hard disk drive can be employed. However, the parameter relation information PI may be stored in the volatile memory.

  The operation unit 140 has a plurality of buttons. These multiple buttons accept user instructions. The operation unit 140 is not limited to a button, and any device that accepts a user instruction can be used. For example, a touch panel or a dial can be used.

  The display unit 150 displays various information. As the display unit 150, various displays such as a liquid crystal display and an organic EL display can be employed.

  The card interface 170 is an interface for exchanging data with the memory card MC inserted in the slot 172. Data stored in the memory card MC is read by the CPU 110 via the card interface 170.

  The printer engine 160 is a printing mechanism that executes printing using given print data. Various printing mechanisms such as a printing mechanism that forms an image by ejecting ink droplets onto a printing medium and a printing mechanism that forms an image by transferring and fixing toner onto the printing medium can be adopted as the printing mechanism. .

  Note that the printer 100 may further include a communication interface for performing data communication with another external device (for example, a digital still camera or a personal computer). The printer 100 may receive image data from these external devices. As such a communication interface, a wired interface or a wireless interface may be employed.

  The internal memory 120 stores an image selection unit 340, an image analysis unit 350, a user interface (UI) control unit 360, an image adjustment unit 370, and a print processing unit 380. These processing units are programs executed by the CPU 110. Each processing unit can exchange various data via the internal memory 120. Details of each processing unit will be described later.

  FIG. 2 is a flowchart showing the procedure of the image printing process. In this embodiment, the printer 100 (CPU 110) shown in FIG. 1 automatically starts this image printing process in response to the insertion of the memory card MC in the slot 172.

  In the first step S100, the image selection unit 340 (FIG. 1) selects an image to be printed in accordance with a user instruction. FIG. 3 is an explanatory diagram illustrating an example of an image selection user interface. FIG. 3 shows the operation unit 140 and the display unit 150. The operation unit 140 has four direction buttons Ub, Db, Lb, Rb, and an OK button Ob. The display unit 150 is incorporated in the operation unit 140.

  The image selection unit 340 (FIG. 1) displays on the display unit 150 a graphical user interface representing a list of images stored in the memory card MC and the selection frame SF. The selection frame SF indicates a selected image (that is, an image to be printed). The user can switch the selected image by operating the right button Rb and the left button Lb. Further, the user can confirm the selected image (hereinafter also referred to as “target image”) by pressing the OK button Ob. In the example of FIG. 3, three images IMG1, IMG2, and IMG3 are displayed, and the second image IMG2 is selected among them.

  Note that the image selection unit 340 uses the thumbnail image data included in the image data file stored in the memory card MC in order to display the interface of FIG. Instead, main image data included in the image data file may be used.

  In response to pressing of the OK button Ob (FIG. 3), the process proceeds to the next step S200 of FIG. In step S200, image processing is performed on main image data (also referred to as “target image data”) representing the selected image by the processing units 350, 360, and 370 of the printer 100 (FIG. 1). In this image adjustment, parameters for controlling the image adjustment can be adjusted by the user (details will be described later). In the next step S300, the print processing unit 380 generates print data from the target image data after the image adjustment is executed. The print data is generated by, for example, resolution conversion processing and halftone processing. The print processing unit 380 supplies print data to the printer engine 160. The printer engine 160 prints the target image subjected to image adjustment according to the received print data. As described above, the print processing unit 380 functions as a print data generation unit.

  FIG. 4 is a flowchart showing the image adjustment procedure executed in step S200 of FIG. In the first step S210, the image analysis unit 350 (FIG. 1) obtains the target image data selected by the image selection unit 340 from the memory card MC, and analyzes the target image data to obtain the characteristics of the target image. Calculate the amount.

  FIG. 5 is an explanatory diagram showing feature amounts. In the example of FIG. 5, a person's face is shown in the target image TI. The image analysis unit 350 (FIG. 1) detects the face in the target image by analyzing the target image data. Various known methods can be employed as the face detection method. For example, a method using pattern matching using a template can be employed. Next, the image analysis unit 350 calculates the average value VFave of the brightness of the area representing the detected face (hereinafter referred to as “face area”). The average face brightness VFave is used as a feature amount (hereinafter, the average face brightness VFave is also simply referred to as “face brightness VFave” or “brightness VFave”). Various areas can be used as the face area. For example, a rectangular area including a face may be adopted, or an area showing a color in a predetermined skin color range may be adopted. Further, the brightness can be calculated by various well-known methods from pixel values in the target image data (for example, gradation values of RGB color components).

  In the next step S220, the UI control unit 360 (FIG. 1) refers to the parameter relation information PI and determines the initial setting of the control parameter used for image adjustment according to the calculated feature amount. In this embodiment, the initial setting of the degree of face brightness adjustment is determined.

  FIG. 6 is a graph used for face brightness adjustment. The horizontal axis represents the input value Vin of brightness V, and the vertical axis represents the output value Vout of brightness V. In this embodiment, the entire range of brightness V is a range from 0 to 100. A graph showing the correspondence between the input value Vin and the output value Vout is also called a “tone curve”. The tone curve shown in FIG. 6 is expressed by a mathematical expression “Vout / 100 = (Vin / 100) ** k”. Here, the operation “**” indicates a power. For example, X ** 2 indicates the square of X.

  A graph GLe (k = 1) in the figure is a graph without change. The first graph GL1 (k <1) brightens the target image by setting the output value Vout to a value larger than the input value Vin. The second graph GL2 (k> 1) darkens the target image by setting the output value Vout to a value smaller than the input value Vin.

  In this embodiment, the parameter relationship information PI (FIG. 1) defines the correspondence between the face brightness VFave and the face brightness adjustment degree k. The UI control unit 360 adopts the degree k that is associated with the face brightness VFave by the parameter relation information PI as an initial setting. Details of the parameter relation information PI will be described later.

  In the next step S230 of FIG. 4, the UI control unit 360 (FIG. 1) displays on the display unit 150 a graphical user interface that allows the user to adjust the control parameters. The control parameter is a parameter for controlling image adjustment with respect to the target image data. As will be described later, in this embodiment, various image processes including brightness adjustment, contrast adjustment, vividness adjustment, hue adjustment, and deformation can be executed as image adjustment. A parameter for controlling such image processing is a control parameter. Hereinafter, the user interface that allows the user to adjust the control parameters is also referred to as an “adjustment interface”.

  FIG. 7 is an explanatory diagram illustrating an example of the adjustment interface. The display unit 150 displays five sliders SL1 to SL5 for adjusting control parameters. In the example of FIG. 7, the respective degrees (control parameters) of “Face Brightness Adjustment”, “Background Brightness Adjustment”, “Skin Color Hue Adjustment”, “Contrast Adjustment”, and “Deformation” are adjusted. Can do. The horizontal positions of the sliders SL1 to SL5 indicate control parameter settings. For example, when the first slider SL1 is moved in the right direction, the face image after image processing becomes brighter (degree k becomes smaller), and when the first slider SL1 is moved in the left direction, the face image after image processing becomes darker. (Degree k increases). The UI control unit 360 moves the slider in the right direction and the left direction (adjusts the control parameter) in response to pressing of the right button Rb and the left button Lb by the user.

  In addition, the UI control unit 360 selects one slider as an adjustment target. In the example of FIG. 7, the slider SL1 for adjusting the brightness of the face is selected as the adjustment target. The selected slider SL1 is displayed in a different color from the other sliders SL2 to SL5. Furthermore, the UI control unit 360 switches one slider to be adjusted to another slider among the five sliders SL1 to SL5 in response to the user pressing the upper button Ub or the lower button Db.

  Note that the initial setting of the first slider SL1 (degree k) is determined in step S220. As initial settings for other control parameters, predetermined settings are employed. However, as will be described later, initial settings of other control parameters may also be determined in step S220.

  Further, the UI control unit 360 (FIG. 1) causes the display unit 150 to display the target image and the user interface (slider) in a superimposed manner. Thereby, the user can adjust the control parameter while viewing the target image. Here, it is preferable that the UI control unit 360 causes the display unit 150 to display the target image after image adjustment using the control parameter set by the user. In this way, the user can adjust the control parameter while viewing the adjustment result. Note that image adjustment is performed by the image adjustment unit 370 (FIG. 1).

  The UI control unit 360 determines each control parameter in response to the user pressing the OK button Ob. That is, the UI control unit 360 sets (determines) each control parameter according to a user instruction input to the operation unit 140 in a state where the adjustment interface is displayed on the display unit 150. In response to pressing of the OK button Ob, the process proceeds to the next step S240 in FIG.

  In step S240 of FIG. 4, the image adjustment unit 370 (FIG. 1) performs image adjustment on the target image data using the control parameter. For example, the brightness of the face area described above is adjusted using the tone curve represented by the degree k set by the first slider SL1 (FIG. 7) (FIG. 6). When the OK button Ob is pressed without operating the slider SL1, the initial setting of the degree k is used as it is for image adjustment. As described above, the initial setting means a setting used for image adjustment when the user instruction indicates that the control parameter setting is maintained without change. In the other image processing, the control parameters set by the sliders SL2 to SL5 are used in the same manner (details will be described later).

  In the next step S250, the UI control unit 360 (FIG. 1) determines whether or not the control parameter has been changed from the initial setting by the user. This determination is performed with respect to the control parameter whose initial setting is determined based on the analysis result of the target image data (in this embodiment, determination is made regarding the degree k of facial brightness adjustment). If the initial settings are used as they are, step S260 is not executed, the process of FIG. 4 is completed, and the process proceeds to step S300 of FIG. On the other hand, when the control parameter is changed by the user, that is, when the setting determined by the user is different from the initial setting, the UI control unit 360 corrects the parameter relation information PI in the next step S260. The UI control unit 360 has three correction modes: an additional correction mode, an edge correction mode, and a representative correction mode. In addition, the UI control unit 360 can execute a smoothing process in step S260. Details of these processes will be described later. In response to the completion of the correction, the process in FIG. 4 is completed, and the process proceeds to step S300 in FIG.

  In step S300 of FIG. 2, the target image subjected to the image adjustment is printed. In response to the completion of printing, the image printing process of FIG. 2 is completed. When printing another target image, the process of FIG. 2 is executed again. That is, when printing a plurality of target images, the process of FIG. 2 is repeatedly executed.

Additional modification mode:
8A to 8E are explanatory diagrams of the additional correction mode. By repeatedly executing the processes of FIGS. 2 and 4, the parameter relationship information PI changes in the order of FIGS. 8 (A) to 8 (E).

  In the present embodiment, the parameter relationship information PI defines the correspondence between the face brightness VFave and the degree k. 8A to 8E show graphs showing this correspondence. The horizontal axis indicates the face brightness VFave, and the vertical axis indicates the degree k. In the present embodiment, the parameter relationship information PI is a table including a plurality of grid points representing the correspondence between the face brightness VFave and the degree k. A table that stores a plurality of grid points representing the correspondence relationship is also called a lookup table. The correspondence between the face brightness VFave and the degree k is determined by linear interpolation of these lattice points. The face brightness VFave corresponds to the “feature amount” in the claims.

  In FIG. 8A, the parameter relation information PI stores two grid points RP1 and RP2. The first lattice point RP1 indicates “VFave = 0, k = 1.0”, and the second lattice point RP2 indicates “VFave = 100, k = 1.0”. The degree k is determined by interpolation using these two grid points RP1 and RP2. The first initial line DFL1 indicates the correspondence determined by such interpolation. In the example of FIG. 8A, the degree k is “1.0” for an arbitrary face brightness VFave.

  8A shows the first analysis brightness AR1. The first analysis brightness AR1 is the face brightness VFave (feature amount) calculated in step S210 of FIG. In subsequent step S220, the first initial degree kd1 associated with the first analysis brightness AR1 by the first initial line DFL1 is used as an initial setting. In the next step S230, an adjustment interface including a slider indicating the initial setting is displayed. The user can change the degree k by operating the slider.

  FIG. 8B shows a change in the degree k by the user. In the example of FIG. 8B, the degree k is changed to the first user degree ki1 that is smaller than the first initial degree kd1 (the first user degree ki1 is smaller than 1). This means that the degree k is reduced to brighten the face. In the subsequent step S240 (FIG. 4), the first user degree ki1 is used. Then, the process proceeds to step S260.

  In step S260, the parameter relation information PI is corrected. In the example of FIG. 8C, a third lattice point RP3 indicating the correspondence between the first analysis brightness AR1 and the first user degree ki1 is added to the parameter relationship information PI. Thus, when step S220 of FIG. 4 is executed for the next target image, the initial setting of the degree k is determined by interpolation using the three grid points RP1 to RP3.

  A second initial line DFL2 shown in FIG. 8C indicates the correspondence defined by interpolation. The second analysis brightness AR2 in FIG. 8C indicates the face brightness VFave calculated in step S210 (FIG. 4) regarding the next target image. In the example of FIG. 8C, the second analysis brightness AR2 is larger (brighter) than the first analysis brightness AR1.

  In the subsequent step S220 (FIG. 4), the second initial degree kd2 associated with the second analysis brightness AR2 by the second initial line DFL2 is used as an initial setting. In the subsequent step S230, the adjustment interface is displayed.

  FIG. 8D shows a change in the degree k by the user. In the example of FIG. 8D, the degree k is changed to the second user degree ki2 that is larger than the second initial degree kd2 (the second user degree ki2 is larger than 1). This means that the degree k is increased to lower the brightness of the face. In the subsequent step S240 (FIG. 4), the second user degree ki2 is used. Then, the process proceeds to step S260.

  In step S260, the parameter relation information PI is corrected. This correction is performed in the same manner as in the example of FIG. Specifically, a fourth grid point RP4 indicating the correspondence between the second analysis brightness AR2 and the second user degree ki2 is added to the parameter relationship information PI. Thereby, when step S220 of FIG. 4 is executed for the next target image, the initial setting of the degree k is determined by interpolation using the four grid points RP1 to RP4. The third initial line DFL3 indicates the correspondence defined by such interpolation.

  By using the third initial line DFL3, the initial setting is determined as follows. When the face brightness VFave is relatively small, a degree k smaller than 1 is adopted as an initial setting as in the case where the first user degree ki1 is used. Further, when the face brightness VFave is relatively large, a degree k greater than 1 is adopted as an initial setting as in the case where the second user degree ki2 is used.

  As described above, in the additional correction mode, lattice points indicating the correspondence between the feature amount obtained by analyzing the target image data and the degree set by the user are added to the parameter relationship information PI. As a result, the degree set by the user can be reliably reflected in the parameter relationship information PI.

  Further, by repeating such correction, it is possible to suppress the initial setting determined by the parameter relationship information PI from being significantly different from the user's preference setting for various target image data.

  Hereinafter, the point representing the correspondence between the analysis result (feature amount) and the degree of image adjustment is referred to as a “reference point”. The grid points indicate reference points stored in the parameter relation information PI. A point representing the correspondence between the analysis result (feature value) and the degree k set by the user is referred to as a “user reference point”. Further, the degree k set by the user is referred to as “user degree”.

Edge correction mode:
9A to 9E are explanatory diagrams of the edge correction mode. This correction mode is executed instead of the above-described additional correction mode when the face brightness VFave is close to the minimum value (0) or the maximum value (100). Specifically, this edge correction mode is selected when the face brightness VFave is smaller than the first threshold value Vth1 and when the face brightness VFave is larger than the second threshold value Vth2. 9 (A) to 9 (E) show graphs showing the correspondence between the face brightness VFave and the degree k, similar to FIGS. 8 (A) to 8 (E). 9A to 9D show examples of correction when the face brightness VFave is smaller than the first threshold value Vth1. The parameter relation information PI changes in the order of FIGS. 9 (A) to 9 (D). The second threshold value Vth2 is larger than the first threshold value Vth1. Further, each of the threshold values Vth1 and Vth2 is larger than the minimum value (0) and smaller than the maximum value (100).

  In FIG. 9A, the parameter relation information PI stores four grid points RP1 to RP4. These lattice points RP1 to RP4 are the same as those shown in FIG. FIGS. 9A to 9E show a case where the parameter relation information PI is further modified in the state shown in FIG. FIG. 9A shows the third analysis brightness AR3. The third analysis brightness AR3 indicates the face brightness VFave calculated in step S210 of FIG. In the example of FIG. 9A, the third analysis brightness AR3 is smaller than the first threshold value Vth1. Note that the face brightness VFave of the third grid point RP3 is larger than the first threshold value Vth1.

  In the subsequent step S220 (FIG. 4), the third initial degree kd3 associated with the third analysis brightness AR3 by the third initial line DFL3 is used as an initial setting. In the subsequent step S230, the adjustment interface is displayed.

  FIG. 9B shows the change of the degree k by the user. In the example of FIG. 9B, the degree k is changed to the third user degree ki3 that is smaller than the third initial degree kd3. In the subsequent step S240 (FIG. 4), the third user degree ki3 is used. Then, the process proceeds to step S260.

  In step S260, the parameter relation information PI is corrected. In the edge correction mode of this embodiment, the degree of the first grid point RP1 is corrected instead of adding the user reference point UP3. Here, the user reference point UP3 is a reference point indicating a correspondence relationship between the third analysis brightness AR3 and the third user degree ki3. For this correction, the third grid point RP3 is used in addition to the user reference point UP3. The third lattice point RP3 has a feature amount (face brightness VFave) among the remaining lattice points excluding the first lattice point RP1 in the parameter relation information PI, and the feature amount (zero) of the first lattice point RP1. Is the closest lattice point to. The first grid point RP1 corresponds to an “edge grid point” in the claims.

  In the present embodiment, the degree k of the first grid point RP1 is determined by linear extrapolation using the user reference point UP3 and the third grid point RP3 (FIG. 9C). Thus, when step S220 of FIG. 4 is executed for the next target image, the initial setting of the degree k is determined by interpolation using the new first grid point RP1 and the three grid points RP2 to RP4. The

  A fourth initial line DFL4 shown in FIG. 9D shows the correspondence defined by interpolation using the grid points RP1 to RP4. On the other hand, FIG. 9E shows a virtual initial line DFLp that is obtained when the user reference point UP3 is added as the fifth lattice point RP5 in the same manner as in the additional correction mode shown in FIG. In a range where the brightness VFave is larger than the third analysis brightness AR3, the fourth initial line DFL4 and the virtual initial line DFLp are the same. On the other hand, in the range where the brightness VFave is smaller than the third analysis brightness AR3, the degree k of the fourth initial line DFL4 is smaller than the degree k of the virtual initial line DFLp. As described above, the fourth initial line DFL4 is appropriately adapted to the user's preference that the degree k should be reduced when the face brightness VFave is smaller than the first threshold value Vth1 as compared with the virtual initial line DFLp. Can be made.

  Further, as indicated by the fourth initial line DFL4, in the edge correction mode, the rate of change of the degree k with respect to the change of the brightness VFave becomes steep in the range where the brightness VFave is smaller than the third analysis brightness AR3. Is suppressed. That is, it is possible to suppress an excessively large change in the degree k due to a small change in the feature amount (brightness VFave) in the vicinity of the edge grid point. As a result, it is possible to prevent an unnatural setting from being adopted as the initial setting of the degree k.

  The case where the brightness VFave is smaller than the first threshold value Vth1 has been described above. When the brightness VFave is larger than the second threshold value Vth2, the degree k of the second grid point RP2 at which the brightness VFave is set to the maximum value similarly uses the user reference point and the fourth grid point RP4. And it will be corrected. In this case, the second grid point RP2 corresponds to an “edge grid point” in the claims.

Representative correction mode:
10A to 10D are explanatory diagrams of the representative correction mode. In this embodiment, this correction mode is selected by correction after the total number of grid points included in the parameter relation information PI reaches a predetermined threshold value. When the total number of grid points is smaller than a predetermined total threshold value, the above-described additional correction mode or edge correction mode is selected. In this embodiment, this total number threshold is set to 4. FIGS. 10A to 10D show graphs showing the correspondence between the brightness VFave and the degree k, similar to FIGS. 8A to 8E.

  In FIG. 10A, the parameter relation information PI stores four grid points RP1 to RP4. These lattice points RP1 to RP4 are the same as those shown in FIG. 10A to 10D show a case where the parameter relation information PI is further modified in the state shown in FIG. 9D. Here, in the example of FIG. 10A, the total number of grid points is the same as the total number threshold. Therefore, the parameter related information PI is corrected in the representative correction mode.

  FIG. 10A also shows the fourth analysis brightness AR4. The fourth analysis brightness AR4 indicates the face brightness VFave calculated in step S210 of FIG. In the example of FIG. 10A, the fourth analysis brightness AR4 indicates the brightness VFave between the second grid point RP2 and the fourth grid point RP4. Further, the lattice point indicating the brightness VFave closest to the fourth analysis brightness AR4 is the fourth lattice point RP4.

  In the subsequent step S220 (FIG. 4), the fourth initial degree kd4 associated with the fourth analysis brightness AR4 by the fourth initial line DFL4 is used as an initial setting. In the subsequent step S230, the adjustment interface is displayed.

  FIG. 10B shows a change in the degree k by the user. In the example of FIG. 10B, the degree k is changed to a fourth user degree ki4 that is larger than the fourth initial degree kd4. This means that when the brightness VFave is slightly brighter than the brightness of the fourth grid point RP4, the degree k is increased to reduce the brightness of the face. In the subsequent step S240 (FIG. 4), the fourth user degree ki4 is used. Then, the process proceeds to step S260.

  FIG. 10C shows the calculation of the representative reference point RpP. In step S260 (FIG. 4), the parameter relationship information PI is corrected by using the representative reference point RpP. In the representative correction mode of the present embodiment, first, the representative reference point RpP is calculated from the user reference point UP4 and the lattice point closest to the user reference point UP4 (the fourth lattice point RP4 in this example). Then, the lattice point (fourth lattice point RP4 in this example) used for calculating the representative reference point RpP is deleted from the parameter relationship information PI, and the representative reference point RpP is added to the parameter relationship information PI. Note that the user reference point UP4 indicates a correspondence relationship between the fourth user degree ki4 and the fourth analysis brightness AR4.

  As shown in FIG. 10C, in this embodiment, an average reference point of two reference points is calculated as a representative reference point. That is, at the representative reference point RpP, each of the brightness VFave and the degree k is set to an average value of two reference points (the fourth lattice point RP4 and the user reference point UP4). Then, the fourth grid point RP4 is deleted from the parameter relation information PI, and instead, the representative reference point RpP is added to the parameter relation information PI as a new fourth grid point RP4. Thus, when step S220 of FIG. 4 is executed for the next target image, the prescribed setting of the degree k is determined by interpolation using the three grid points RP1 to RP3 and the new fourth grid point RP4. Is done.

  A fifth initial line DFL5 shown in FIG. 10D shows the correspondence defined by such interpolation. In the range RG4 of brightness VFave in which the corrected fourth grid point RP4 is used for interpolation, the degree k (initial setting) is adapted to the fourth user degree ki4 newly set by the user. Will be corrected. As a result, the initial settings can be appropriately adapted to the user's adjustment preferences that can vary depending on the image data.

  In the representative correction mode, since the total number of grid points included in the parameter relation information PI does not increase, it is possible to suppress an excessive increase in the size of the parameter relation information PI.

Smoothing process:
11A to 11D are explanatory diagrams of the smoothing process. This smoothing process is executed by the UI control unit 360 (FIG. 1) following the correction of the parameter relation information PI. The UI control unit 360 executes the smoothing process when the following conditions are satisfied as a result of the correction of the parameter relation information PI. The condition is that the rate of change of the degree k with respect to the change in brightness VFave is greater than a predetermined threshold in at least a part of the correspondence defined by the parameter relationship information PI. A large change rate is reduced by the smoothing process. The parameter related information PI is corrected in the same manner as in each of the correction modes described above. Moreover, the graph which shows the correspondence of brightness VFave and the degree k similar to FIG. 8 (A) -8 (E) is shown by FIG. 11 (A) -11 (F). Further, the parameter relation information PI changes in the order of FIGS. 11 (A) to 11 (F).

  In FIG. 11A, the parameter relation information PI stores three lattice points RP1 to RP3. The lattice points RP1 and RP2 at both ends are the same as the lattice points RP1 and RP2 at both ends shown in FIG. The third lattice point RP3 is a lattice point between the lattice points RP1 and RP2 at both ends, and the degree k is set to a value smaller than 1. The sixth initial line DFL6 shows the correspondence defined by interpolation using these three grid points RP1 to RP3.

  FIG. 11A shows the fifth analysis brightness AR5. The fifth analysis brightness AR5 indicates the face brightness VFave calculated in step S210 of FIG. In the example of FIG. 11A, the fifth analysis brightness AR5 indicates the brightness VFave between the third grid point RP3 and the second grid point RP2.

  In the subsequent step S220 (FIG. 4), the fifth initial degree kd5 associated with the fifth analysis brightness AR5 by the sixth initial line DFL6 is used as an initial setting. In the subsequent step S230, the adjustment interface is displayed.

  FIG. 11B shows the change of the degree k by the user. In the example of FIG. 11B, the degree k is changed to a fifth user degree ki5 that is larger than the fifth initial degree kd5. This means that when the brightness VFave is brighter than the brightness of the third grid point RP3, the degree k is increased to suppress the brightness of the face. In the subsequent step S240 (FIG. 4), the fifth user degree ki5 is used. Then, the process proceeds to step S260.

  In step S260 (FIG. 4), the parameter relation information PI is corrected. Here, the correction is performed in the additional correction mode shown in FIG. FIG. 11C is an explanatory diagram showing correction of the parameter relation information PI. A fourth lattice point RP4 representing the correspondence between the fifth analysis brightness AR5 and the fifth user degree ki5 is added to the parameter relationship information PI.

  A seventh initial line DFL7 shown in FIG. 11C shows the correspondence defined by interpolation using the four grid points RP1 to RP4 including the added grid point RP4. Here, between the third grid point RP3 and the fourth grid point RP4, the rate of change sr of the degree k with respect to the change of the brightness VFave is larger than the predetermined rate threshold sr_th. Therefore, the UI control unit 360 (FIG. 1) executes a smoothing process following this correction.

  A graph AC illustrated in FIG. 11D illustrates an approximate curve. First, the UI control unit 360 determines an approximate curve AC that approximates a plurality of lattice points of the parameter relationship information PI. Various methods can be adopted as a method for determining such an approximate curve AC. In this embodiment, the approximate curve AC is determined using a smoothing spline.

  FIG. 11E shows correction of lattice points by the approximate curve AC. The UI control unit 360 corrects the degree k of each of the plurality of lattice points of the parameter relation information PI to the degree k indicated by the approximate curve AC. In the example of FIG. 11E, the degree k of the third grid point RP3 is corrected to a larger value, and the degree k of the fourth grid point RP4 is corrected to a smaller value. In this embodiment, since the curve passing through the lattice points RP1 and RP2 at both ends is adopted as the approximate curve AC, the degree k of the lattice points RP1 and RP2 at both ends is maintained. However, as an approximate curve, a curve that passes through the vicinity of the lattice points without passing through the lattice points RP1 and RP2 at both ends may be adopted. When the above correction is completed, the smoothing process ends.

  An eighth initial line DFL8 shown in FIG. 11 (F) indicates the correspondence defined by interpolation using the corrected lattice points RP1 to RP4. The change rate sr between the third grid point RP3 and the fourth grid point RP4 is smaller than the rate threshold value sr_th. When step S220 of FIG. 4 is executed for the next target image, the UI control unit 360 determines initial settings according to the eighth initial line DFL8. As indicated by the eighth initial line DFL8, the change rate sr is suppressed from being sharpened by the smoothing process. That is, it is possible to suppress an excessively large change in the degree k due to a small change in the feature amount (brightness VFave). As a result, it is possible to prevent an unnatural setting from being adopted as the initial setting of the degree k.

  Note that the UI control unit 360 (FIG. 1) preferably executes the smoothing process so that the change rate sr is equal to or less than the rate threshold value sr_th. For example, the correction of the grid points using the approximate curve AC may be repeated until the change rate sr becomes equal to or less than the rate threshold value sr_th.

  As described above, the UI control unit 360 (FIG. 1) determines the parameter relationship information PI based on the correspondence between the analysis result (brightness VFave) of the target image data and the setting of the degree k set by the user. By learning the above, the user's preference for adjustment that can change depending on the image data is learned. Thereafter, the UI control unit 360 determines the initial setting of the degree k using the analysis result of the target image data and the corrected parameter relation information PI. As a result, it is possible to reflect the user's adjustment preference that can be changed depending on the image data in the initial setting. And even if it is a case where an initial setting is employ | adopted as it is by a user, various target image data can be adjusted according to a user preference. In addition, this eliminates the need for the user to adjust the control parameter for each target image data, thereby reducing the user's burden.

  In addition, each correction of the additional correction mode shown in FIG. 8, the edge correction mode shown in FIG. 9, and the representative correction mode shown in FIG. 10 has a degree k corresponding to the analysis result (brightness VFave). It is executed so as to approach the degree k set by the user. As a result, the initial settings can be appropriately adapted to the user's adjustment preferences that can vary depending on the image data.

  In the present embodiment, the parameter relation information PI is a lookup table including a plurality of grid points. As a result, by using a plurality of grid points, it is possible to easily associate the feature amount obtained by analyzing the image data with the degree of image adjustment.

  In the present embodiment, as shown in step S250 of FIG. 4, the parameter relationship information PI is corrected when the setting determined by the user is different from the initial setting. As a result, it is possible to appropriately reflect the user's adjustment preference that can change depending on the image data in the initial setting. Further, it is possible to prevent unnecessary correction processing from being executed.

B. Second embodiment:
FIG. 12 is an explanatory diagram of the parameter relation information PIa in the second embodiment. The only difference from the parameter-related information PI of the first embodiment described above is that, in addition to the face brightness VFave, the average face hue HFave and the average background brightness VBave are used as the feature quantities. One degree k is associated with the combination of three types of feature amounts of the face brightness VFave, the average face hue HFave, and the average background brightness VBave by the parameter relation information PIa of the second embodiment. The face brightness VFave is the same as the feature amount used in the first embodiment. The average face hue HFave is an average value of hues in the face area described above. The average background brightness VBave is the average value of the brightness in the remaining area excluding the face area in the target image.

  FIG. 12 shows a feature amount space QS represented using three types of feature amounts VFave, HFave, and VBave as reference vectors. In the example of FIG. 12, the range of face brightness VFave is 0 to 100, the range of face hue HFave is 0 to 360, and the range of background brightness VBave is 0 to 100. Regarding hue, 0 and 360 indicate the same hue.

  A combination of three types of feature amounts calculated from one target image (target image data) is represented by one point in the feature amount space QS. For example, the coordinate point AR10 in the figure indicates the analysis result of one target image. Regarding the first analysis result AR10, the face brightness VFave, the face hue HFave, and the background brightness VBave are set to the first values VF10, HF10, and VB10, respectively.

  A plurality of black circles arranged in the feature amount space QS indicate lattice points RP included in the parameter relation information PIa. Each grid point RP indicates a correspondence relationship between the face brightness VFave, the face hue HFave, the background brightness VBave, and the degree k (the degree k is not shown). In FIG. 12, only a part of the plurality of lattice points RP included in the parameter relation information PIa is shown.

  The parameter relation information PIa in the second embodiment is used instead of the parameter relation information PI in each of the above-described embodiments. In step S210 of FIG. 4, the image analysis unit 350 (FIG. 1) calculates three types of feature amounts VFave, HFave, and VBave. In step S220, the UI control unit 360 refers to the parameter relationship information PIa and determines an initial setting of the degree k according to the calculated three types of feature amounts. The initial setting is determined using the grid point RP in the parameter relation information PIa, as in the first embodiment. The first initial degree kd10 shown in FIG. 12 indicates the degree k associated with the first analysis result AR10 by the parameter relation information PIa.

  Various methods can be adopted as a method for determining the initial setting in the three-dimensional space as shown in FIG. For example, in the feature amount space QS, the initial setting may be determined by using some lattice points RP that are close to the analysis result (feature amount). Hereinafter, a coordinate point representing an analysis result (feature amount) in the feature amount space QS is referred to as an analysis point. In the example of FIG. 12, the first initial degree kd10 of the first analysis point AR10 is determined using the four lattice points RPn1 to RPn4 close to the first analysis point AR10.

  Here, as a distance between two coordinate points in the feature amount space QS, various values indicating a difference in feature amount can be adopted. For example, the square root of the sum of squares of the differences between the three types of feature amounts may be employed as the distance. Here, the square of a value obtained by multiplying the difference between each feature value by a different weight for each type of feature value may be used.

  As the grid points used for determining the initial settings, various grid points that satisfy a predetermined condition indicating that they are close to the analysis point can be used. For example, a grid point whose distance to the analysis point is less than a predetermined analysis neighborhood threshold may be adopted. In this case, the total number of selected grid points is a variable value that depends on the analysis point and the parameter relation information PIa. Alternatively, a predetermined number (for example, two or five) of lattice points closest to the analysis point may be employed (that is, the distance when a plurality of lattice points RP are arranged in ascending order of the distance from the analysis point is determined. The shortest predetermined number of grid points). In any case, it is preferable to select a plurality of grid points.

  Various methods can be adopted as a method for determining the initial setting using a grid point close to the analysis point. For example, the average degree of selected grid points may be adopted as the initial setting. Moreover, you may employ | adopt a weighted average degree. Here, as the weight, it is preferable to adopt a larger value as the distance to the analysis point is shorter.

  Note that the determination of the initial setting is not limited to the above-described method using a lattice point close to the analysis point, and various other methods can be employed. For example, the initial setting may be determined by using an approximate function determined based on a plurality of lattice points of the parameter relation information PIa.

  Subsequent steps S230 and S240 (FIG. 4) are executed in the same manner as in the first embodiment. If the degree k is changed by the user, the parameter relation information PIa is corrected in step S260. As a method for correcting the parameter relation information PIa, the same method as in the first embodiment described above can be adopted. Hereinafter, various correction modes and smoothing processes will be described.

Additional modification mode:
Similar to the “additional correction mode” shown in FIGS. 8A to 8E, a new user reference point indicating the correspondence between the feature amount (analysis point) and the user degree may be added to the parameter relationship information PIa. .

Edge correction mode:
Similarly to the “edge correction mode” shown in FIGS. 9A to 9D, when the user reference point is close to one of the edge lattice points in the feature amount space QS, the degree k of the edge lattice point. May be modified. Here, the edge lattice point means a lattice point RP on the outer shell region of the feature amount space QS. The outer shell region means a region in which at least one of a plurality of types of feature values is set to the minimum value or the maximum value in the entire range of the feature values. For example, a hatched surface AVFmax shown in FIG. 12 indicates an outer shell region where the face brightness VFave is the maximum value. All the lattice points RP on this surface AVFmax correspond to edge lattice points.

  As a condition for selecting the edge correction mode, various conditions indicating that the user reference point is close to the edge lattice point can be adopted. For example, the distance between the user reference point and the edge grid point may be a predetermined edge correction threshold value or less. In this case, a plurality of edge grid points close to the user reference point may be corrected using one user reference point. Also, a predetermined number (for example, 1 or 2) of edge grid points closest to the user reference point among the plurality of edge grid points may be corrected.

  As a method for determining the new degree k of the edge grid point, various methods for determining the degree k using the user reference point and some grid points close to the edge grid point can be employed. For example, the average degree of these reference points may be adopted as the new degree k of edge grid points. Moreover, you may employ | adopt a weighted average degree. As the weight, it is preferable to adopt a larger value as the distance to the edge grid point is shorter.

  In addition, as grid points used for calculating the degree of edge grid points, various grid points that satisfy a predetermined condition indicating that the grid points are close to the edge grid points can be employed. For example, a grid point whose distance from the edge grid point is less than a predetermined edge vicinity threshold may be adopted. Further, a predetermined number (for example, two or four) of lattice points closest to the edge lattice points may be employed. In any case, it is preferable to select a plurality of grid points in addition to the user reference point. In this way, it is possible to suppress the use of an unnatural setting as the degree k of the edge lattice points.

  Various other methods can be adopted as a method for determining the new degree k. For example, the degree k of the edge lattice points may be determined by using an approximate function determined based on the user reference point and the lattice point of the parameter relation information PIa. In determining the approximate function, all the lattice points of the parameter relation information PIa may be used, or some lattice points close to the edge lattice points may be used. In any case, it is preferable to use a plurality of grid points excluding the edge grid points in addition to the user reference points. In this way, it is possible to suppress the use of an unnatural setting as the degree k of the edge lattice points.

  In the edge correction mode, the edge grid points may be corrected and the user reference points may be added to the parameter relationship information PIa. Alternatively, edge grid points may be modified without adding user reference points.

Representative correction mode:
Similar to the “representative correction mode” shown in FIGS. 10A to 10D, the representative reference point may be calculated from the user reference point and a part of the lattice points RP of the parameter relation information PIa. As the representative reference points, various representative reference points calculated by using the user reference points and some lattice points close to the user reference points can be adopted. For example, a reference point having the maximum absolute value of the degree k among these reference points may be employed as it is. Moreover, you may employ | adopt the average reference point of these reference points. A weighted average reference point may be employed. As the weight, it is preferable to adopt a larger value as it is closer to the user reference point. The average reference point is a reference point at which each of the feature amount and the degree k is represented by an average.

  In addition, as lattice points used for calculating the representative reference point, various lattice points that satisfy a predetermined condition indicating that the reference point is close to the user reference point can be employed. For example, a grid point whose distance from the user reference point is less than a predetermined representative neighborhood threshold may be employed. A predetermined number (for example, one or three) of lattice points closest to the user reference point may be employed. In any case, it is preferable to select a plurality of grid points in addition to the user reference point. In this way, it is possible to suppress the use of an unnatural reference point as the representative reference point.

  In the representative correction mode, the lattice points deleted from the parameter relation information PIa may be all or a part of the lattice points used for calculating the representative reference point. For example, a predetermined number (for example, one or two) of lattice points closest to the user reference point among the lattice points used for calculating the representative reference point may be deleted. Instead, a predetermined number (for example, one or two) of lattice points closest to the representative reference point may be deleted.

Smoothing process:
A smoothing process may be executed in the same manner as the processes shown in FIGS. In this case, the smoothing process may be executed when the rate of change of the degree k with respect to one type of feature quantity change is greater than a predetermined threshold. The comparison between the change rate and the threshold value is preferably performed for each of a plurality of types of feature amounts. Then, it is preferable that the smoothing process is performed when the change ratio regarding at least one type of feature amount is larger than the threshold value. Here, the threshold value may be different for each type of feature amount to be changed.

  As the smoothing process, various processes such as smoothing the degree k of the lattice points representing the portion where the change rate is steep can be employed. Here, the smoothing of the degree means a process of correcting the degree k indicated by the grid point so that the change rate becomes small. As such a smoothing process, the following processes can be employed. For example, an approximate function that approximates a plurality of lattice points of the parameter relationship information PIa may be calculated, and the degree k of all lattice points may be set to the degree k represented by the approximate function. Instead, a part of the lattice points including the lattice point representing the portion where the change rate is steep may be corrected using an approximate function. In this case, the lattice point representing the sudden change portion may be corrected, and in addition, the lattice point close to the sudden change portion may be corrected. As the approximate function, various functions such as a polynomial using a plurality of feature quantities as variables can be employed. In addition, as an approximate function, various functions that exhibit a small change rate compared to the change rate represented by a plurality of lattice points can be employed.

  Further, as the smoothing process, a process of re-correcting the degree k of the lattice point corrected (or added) based on the user reference point so that the change rate may be reduced may be employed. In any case, it is preferable to execute the smoothing process so that the change rate is equal to or less than the threshold value.

  The above description of the second embodiment is not limited to the case where the number of types of feature values used for determining the initial setting is three, but also when there are one type, two types, or four or more types. The same applies. Regarding the face hue HFave, 0 and 360 indicate the same hue. Therefore, the correction of the parameter relationship information PIa is preferably performed so that the degree k smoothly changes between the end where the face hue HFave is 0 and the end where the face hue HFave is 360.

C. Image processing:
In each of the above-described embodiments, the control parameter whose initial setting is determined from the analysis result of the target image data and the parameter relation information PI and PIa is not limited to the degree k of facial brightness adjustment, and various image processing can be performed. Control parameters can be adopted. In addition, initial settings of control parameters for a plurality of types of image processing may be determined by learning. In this case, the parameter relationship information PI and PIa include a plurality of lookup tables prepared for each control parameter. Here, as the image processing, for example, the following processing can be employed.

  13 and 14 are explanatory diagrams showing an outline of image processing. FIG. 13A shows a graph used for brightness adjustment. The horizontal axis represents the input value Vin of brightness V, and the vertical axis represents the output value Vout of brightness V. In this embodiment, the entire range of brightness V is a range from 0 to 100. A graph showing the correspondence between the input value Vin and the output value Vout is also called a “tone curve”. The graph GLe in the figure is a graph without change. The first graph GL1 brightens the target image by setting the output value Vout to a value larger than the input value Vin. The second graph GL2 darkens the target image by setting the output value Vout to a value smaller than the input value Vin. The user can determine whether to make the adjusted brightness V brighter or darker by adjusting the slider on the adjustment interface. The amount of change in brightness V is also adjusted by the slider. In the embodiment of FIG. 13A, the output value Vout is the same as the input value Vin for the minimum value (0) and the maximum value (100) of the brightness V.

  In step S220 of FIG. 4, the initial setting of the control parameter (brightness adjustment degree) adjusted by the slider is determined. As the feature amount (analysis result) used for determining the initial setting, for example, the average brightness of the entire target image can be adopted. In addition to the average brightness, the average saturation of the entire target image may be used.

  Note that, as in the case illustrated in FIG. 7, when the “face brightness” is adjusted, the brightness of the area representing the face in the target image is adjusted. As a method for detecting a face in the target image, various known methods can be employed. For example, a method using pattern matching using a template can be employed. Further, when adjusting the “background brightness”, the brightness of the remaining areas other than the area representing the face in the target image is adjusted.

  FIG. 13B shows a graph used for contrast adjustment. These graphs GLe, GC1, and GC2 show tone curves similar to those in FIG. The first graph GC1 increases the contrast of the target image by setting the output value Vout in the bright range to a value larger than the input value Vin and setting the output value Vout in the dark range to a value smaller than the input value Vin. . The second graph GC2 reduces the contrast of the target image by setting the output value Vout in the bright range to a value smaller than the input value Vin and setting the output value Vout in the dark range to a value larger than the input value Vin. . The user can decide whether to increase or decrease the contrast after adjustment by adjusting the slider on the adjustment interface. The contrast adjustment amount is also adjusted by the slider. In the embodiment of FIG. 13B, the output value Vout is the same as the input value Vin for the minimum value (0) and the maximum value (100) of the brightness V.

  In step S220 of FIG. 4, the initial setting of the control parameter (degree of contrast adjustment) adjusted by the slider is determined. As the feature amount (analysis result) used for the determination of the initial setting, for example, a contrast evaluation value in the entire target image can be adopted. As the contrast evaluation value, various values correlated with the contrast of the target image can be adopted. For example, the difference between the maximum value and the minimum value of the brightness of the entire target image can be employed. In addition to the contrast evaluation value, the average brightness of the entire target image may be employed.

  FIG. 13C shows a graph used for vividness adjustment. The horizontal axis represents the input value Sin of the saturation S, and the vertical axis represents the output value Sout of the saturation S. In this embodiment, the entire range of the saturation S is from 0 to 100. The graph GSe in the figure shows a graph without change. The first graph GS1 increases the saturation of the target image by setting the output value Sout to a value larger than the input value Sin. The second graph GS2 lowers the saturation of the target image by setting the output value Sout to a value smaller than the input value Sin. The user can determine whether to increase or decrease the saturation S after adjustment by adjusting the slider on the adjustment interface. Further, the amount of change in saturation S is also adjusted by the slider. In the embodiment of FIG. 13C, the output value Sout is proportional to the input value Sin.

  In step S220 of FIG. 4, the initial setting of the control parameter (degree of vividness adjustment) adjusted by the slider is determined. As the feature amount (analysis result) used for the determination of the initial setting, for example, average saturation in the entire target image can be adopted. In addition to the average saturation, the average brightness of the entire target image may be employed.

  FIG. 13D shows a graph used for skin tone adjustment. The horizontal axis indicates the input value Hin of the hue H, and the vertical axis indicates the output value Hout of the hue H. In this embodiment, the entire range of hue H is a range from 0 to 360. 0 and 360 represent the same hue. A graph GHe in the figure shows a graph without change. The first graph GH1 shifts the hue in the predetermined skin color range HR1 to the yellow side. The second graph GH2 shifts the hue in the flesh color range HR1 to the red side. The user can select the shift direction of the hue H after the adjustment from the yellow direction and the red direction by adjusting the slider on the adjustment interface. The hue shift amount is also adjusted by the slider. In the example of FIG. 13D, the output value Hout is maintained at the same value as the input value Hin for hues outside the skin color range HR1.

  In step S220 of FIG. 4, the initial setting of the control parameter (degree of hue adjustment) adjusted by the slider is determined. As the feature amount (analysis result) used for the determination of the initial setting, for example, an average hue related to pixels indicating the hue in the skin color range HR1 can be used. Further, in addition to this average hue, at least one of average brightness and average saturation relating to a pixel indicating a hue within the skin color range HR1 may be employed. In addition to this average hue, average brightness in the entire target image may be adopted.

  FIG. 14 is an explanatory diagram showing an outline of the deformation process. FIG. 14A shows a target image before deformation, and FIG. 14B shows a target image after deformation. The image adjustment unit 370 detects a person's face from the target image and determines a deformation area TA including the detected face. The deformation area TA is determined so as to include an image from the chin to the forehead with respect to the height direction of the face and to include images of left and right cheeks with respect to the width direction of the face. Then, the image adjustment unit 370 deforms the image in the deformation area TA so that the left and right cheek lines (face contour) of the face move inward. In the example of FIG. 14, the width Wd of the face (cheek) after deformation is narrower by twice the deformation amount DQ than the width Wo of the face (cheek) before deformation. By such deformation, the shape of the face in the target image becomes sharp. The deformation amount DQ is adjusted by adjusting the slider on the adjustment interface described above. In the embodiment of FIG. 14, the image outside the deformation area TA is not deformed.

  As a method for detecting a face in the target image, various known methods can be employed. For example, a method using pattern matching using a template can be employed. As a method for determining the deformation area TA, various methods for determining the area including the cheek line of the detected face image as the deformation area TA can be employed. For example, a method of determining the position, size, and inclination of the deformation area TA based on the arrangement of predetermined organs such as eyes and mouth in the detected face image can be employed. Here, various shapes can be adopted as the shape of the deformation area TA. For example, a rectangular region may be employed, a polygonal region other than a rectangle may be employed, or an elliptical region may be employed. In addition, as a method for deforming the image in the deformation area TA, various methods that reduce the width of the deformed face (cheek) can be employed.

  As a method for deforming the image in the deformation area TA, for example, the following method can be employed. First, the deformation area TA is divided into a plurality of quadrilateral blocks (for example, rectangular blocks). Next, a predetermined vertex (grid point) of the predetermined quadrilateral block is moved. The image in the quadrilateral block whose vertex is moved is deformed according to the movement of the vertex. Here, the quadrilateral block may be divided into four triangular blocks represented by the four vertices of the quadrilateral block and the barycentric point, and the image in each triangular block may be transformed by affine transformation. Here, it is preferable to move some vertices (for example, two vertices) closest to the inside of the face among the four vertices of the quadrilateral block representing the cheek in a predetermined direction toward the inside of the face. In this way, the width of the face (cheek) can be narrowed. Here, the amount of movement of the vertex may be adjusted according to the slider on the adjustment interface described above.

  In step S220 of FIG. 4, the initial setting of the control parameter (degree of deformation) adjusted by the slider is determined. As the feature amount (analysis result) used for the determination of the initial setting, for example, the width of the face area can be adopted (in the example of FIG. 14, the deformation area TA can be used as the face area). In addition to the width of the face area, the face brightness VFave may be used. The face size is easier to see as the face gets brighter. Therefore, if the face brightness VFave is employed, the user's adjustment preference that can change depending on the image data can be appropriately reflected in the initial setting of the deformation amount DQ.

  The image processing is not limited to the image processing described above, and other image processing may be employed. For example, sharpness adjustment processing may be employed. In any image processing, a curve defined by various methods is adopted as a corresponding curve for determining the correspondence between input values and output values of various gradation values (for example, brightness, hue, and saturation). Is possible. As the control parameter, various parameters for adjusting the corresponding curve can be used. For example, a spline function passing through a reference point that associates a predetermined input value with a variable output value may be used as the corresponding curve. In this case, the variable output value of the reference point can be adopted as the control parameter.

D. Variations:
In addition, elements other than the elements claimed in the independent claims among the constituent elements in the above embodiments are additional elements and can be omitted as appropriate. The present invention is not limited to the above-described examples and embodiments, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

Modification 1:
In each of the above-described embodiments, the control parameter is not limited to the parameter indicating the degree of image adjustment, and various parameters used for image adjustment can be employed. For example, a parameter indicating the image adjustment mode may be adopted. Examples of such image adjustment modes include “landscape”, “person”, “night view”, and “other” image adjustment modes.

  Note that adjusting the degree of image adjustment is often a heavy burden on the user. Therefore, as in the above-described embodiments, it is preferable to employ a control parameter indicating the degree of image adjustment as a learning target. Here, the degree of image adjustment correlates with the image adjustment amount. That is, it can be said that the greater the change in the gradation value of each pixel of the target image data, the greater the degree (for example, the sum of squares of the amount of change in the gradation value for all pixels is an evaluation indicating the magnitude of the degree. Available as a value).

Modification 2:
In each of the above-described embodiments, the relationship information indicating the correspondence between the analysis result of the image data and the setting of the control parameter is not limited to the lookup table, and information of various formats can be employed. For example, a function may be adopted. However, it is preferable to employ a lookup table. By doing this, each individual of the printer 100 (image processing apparatus) reproduces the correspondence that suits each user even if the tendency of the preference setting change with respect to the change in the analysis result varies greatly from user to user. be able to.

  In the case of employing a lookup table, it is preferable to store standard lattice points in the lookup table in advance. By doing this, it is possible to determine the initial setting of the degree for an arbitrary feature amount even before learning progresses. Such standard grid points preferably include a plurality of edge grid points arranged in the outer shell region of the feature amount space. In this way, it is possible to determine the correspondence within the range surrounded by the edge grid points. For example, in the example of FIG. 8A, the first grid point RP1 and the second grid point RP2 may be employed. In the example of FIG. 12, a plurality of lattice points RP arranged on the outer shell region of the feature amount space QS may be employed.

Modification 3:
In each of the above-described embodiments, the related information correction process is not limited to the process of combining the “additional correction mode”, the “representative correction mode”, and the “edge correction mode”, and various processes can be employed. is there. For example, the “representative correction mode” and the “edge correction mode” may be used without using the “additional correction mode”. Further, only the “representative correction mode” may be used without using the “edge correction mode”. In these cases, the correction is executed in the “representative correction mode” regardless of the total number of grid points. Further, “addition correction mode” and “edge correction mode” may be used. Further, only the “additional correction mode” may be used. In this case, in order to prevent the size of the parameter relationship information PI and PIa from becoming excessively large, it is preferable to delete some grid points when adding user reference points. As the deletion target, for example, the oldest lattice point can be adopted. Further, a combination of “additional correction mode” and “representative correction mode” may be used. Further, the present invention is not limited to these three types of correction modes, and other correction processes may be employed. For example, a process is adopted in which the degree of a part of the grid points close to the user reference point among the plurality of grid points stored in advance in the parameter relation information PI and PIa is brought close to the degree of the user reference point (user setting). Also good. In any case, it is preferable to employ a correction process in which the degree of association with the analysis result approaches the degree of the user reference point (user setting).

Modification 4:
In each of the above-described embodiments, various feature amounts can be adopted as the feature amount used for determining the initial setting of the control parameter. In any case, it is preferable to employ a feature quantity strongly related to the control parameter. Also, the adjustment of the control parameter by the user is often based on various features of the target image. Therefore, if two or more types of feature quantities are used to determine the initial setting of one control parameter, it becomes possible to predict user preferences in detail. In this case, it is preferable to employ a lookup table as the relationship information. In this way, detailed correction of the correspondence relationship is easy.

Modification 5:
In each of the embodiments described above, various user interfaces including adjustment control can be adopted as a user interface (adjustment interface) for adjusting the control parameter. Here, the adjustment control means a display area indicating the setting of the control parameter, and means a display area in which the indicated setting can be adjusted by a user instruction input to the operation unit. Such adjustment control is not limited to the slider, and various types of adjustment control can be employed. For example, a tone curve or a field that represents a set value with a number may be employed.

  In addition, the setting screen displayed by the user interface control unit is not limited to the adjustment interface described above, and various setting screens that allow the user to set the control parameter and display the initial setting of the control parameter can be employed. .

  Further, in each of the above-described embodiments, the UI control unit 360 (FIG. 1) determines the target parameter that is a control parameter to be set from among a plurality of control parameters that can be used for image adjustment according to the analysis result of the analysis unit. A user interface for adjusting the target parameter may be displayed on the display unit. At this time, it is preferable that the UI control unit 360 does not display the adjustment control for the control parameter excluded from the setting target. In this way, since control parameters other than the target parameters are suppressed from being considered by the user, it is possible to reduce the burden on the user who adjusts the parameters.

Modification 6:
In each of the above-described embodiments, any device can be adopted as the supply source of the target image data. For example, an image processing apparatus (for example, the printer 100) may acquire target image data directly from a digital still camera instead of the memory card MC. An apparatus connected to the image processing apparatus via a network may be employed. Alternatively, the image selection unit 340 (FIG. 1) may be omitted and all available image data may be used as an image adjustment target.

  The image data after image adjustment can be used not only for printing but also for various purposes. For example, the image after image adjustment may be displayed on the display device. Further, an image file for storing image data after image adjustment may be stored in a non-volatile memory (for example, a hard disk drive or a removable memory card).

  Thus, the image processing apparatus is not limited to a printer, and can be realized as various apparatuses including the image analysis unit 350, the UI control unit 360, and the image adjustment unit 370. For example, a general-purpose personal computer, a personal digital assistant (PDA), a mobile phone, or a digital still camera may be used as the image processing apparatus. Further, like a general-purpose personal computer, at least one of the display unit 150 and the operation unit 140 may be connected to the image processing apparatus as an external device.

Modification 7:
In each of the above embodiments, a part of the configuration realized by hardware may be replaced with software, and conversely, a part of the configuration realized by software may be replaced by hardware. . For example, the function of the image analysis unit 350 in FIG. 1 may be realized by a hardware circuit having a logic circuit.

  In addition, when part or all of the functions of the present invention are realized by software, the software (computer program) can be provided in a form stored in a computer-readable recording medium. In the present invention, the “computer-readable recording medium” is not limited to a portable recording medium such as a flexible disk or a CD-ROM, but an internal storage device in a computer such as various RAMs and ROMs, a hard disk, and the like. An external storage device fixed to the computer is also included.

1 is an explanatory diagram illustrating a printer to which an image processing apparatus according to an embodiment of the present invention is applied. It is a flowchart which shows the procedure of an image printing process. It is explanatory drawing which shows an example of an image selection user interface. It is a flowchart which shows the procedure of the image adjustment performed by step S200 of FIG. It is explanatory drawing which shows the feature-value. It is a graph used for face brightness adjustment. It is explanatory drawing which shows an example of an adjustment interface. It is explanatory drawing of additional correction mode. It is explanatory drawing of edge correction mode. It is explanatory drawing of a representative correction mode. It is explanatory drawing of a smoothing process. It is explanatory drawing of the parameter relationship information PIa in 2nd Example. It is explanatory drawing which shows the outline | summary of an image process. It is explanatory drawing which shows the outline | summary of a deformation | transformation process.

Explanation of symbols

100 ... Printer 110 ... CPU
DESCRIPTION OF SYMBOLS 120 ... Internal memory 140 ... Operation part 150 ... Display part 160 ... Printer engine 170 ... Card interface 172 ... Slot 180 ... Non-volatile memory 340 ... Image selection part 350 Image analysis unit 360 User interface (UI) control unit 370 Image adjustment unit 380 Print processing unit

Claims (7)

An image processing apparatus,
An analysis unit for analyzing image data;
A user interface control unit for displaying a setting screen capable of setting a control parameter used for image adjustment of the image data, and determining the control parameter according to the setting in the setting screen;
An image adjustment unit that performs the image adjustment on the image data using the determined control parameter;
A storage unit for storing relationship information representing a correspondence relationship between the analysis result of the image data by the analysis unit and the setting of the control parameter;
With
The user interface controller is
Using the analysis result of one or more target image data and the setting of the control parameter set on the setting screen for each target image data, the analysis of each target image data in the relation information Setting of the control parameter already associated with each result, and setting of the control parameter already associated with the analysis result of the image data different from each target image data in the relation information Modify the relationship information by modifying
The initial setting of the control parameter displayed on the setting screen for new target image data is determined using the analysis result of the new target image data and the corrected relationship information.
Image processing device. The image processing apparatus according to claim 1,
The control parameter indicates the degree of the image adjustment,
The correction of the relationship information is executed such that the degree of image adjustment associated with the analysis result of each target image data by the relationship information approaches the degree of image adjustment set on the setting screen.
Image processing device. The image processing apparatus according to claim 2,
The analysis result includes at least one feature amount obtained by analyzing the image data,
The relationship information is a look-up table including a plurality of lattice points indicating a correspondence relationship between the feature amount and the degree of image adjustment.
Image processing device. The image processing apparatus according to claim 3,
The correction of the relationship information includes a first correction mode,
In the first correction mode, a reference point indicating a correspondence relationship between the feature amount indicated by the analysis result of the target image data and the degree of image adjustment set on the setting screen is added to the lookup table. To be
Image processing device. The image processing apparatus according to claim 3 or 4, wherein:
The correction of the relationship information includes a second correction mode,
In the second correction mode,
(A) a reference point indicating a correspondence relationship between the feature amount indicated by the analysis result of the target image data and the degree of image adjustment set on the setting screen, and a part of the lattice points in the plurality of lattice points And a representative reference point is calculated by using
(B) From the lookup table, at least a part of the grid points used for calculating the representative reference point is deleted,
(C) the representative reference point is added to the lookup table;
Image processing device. An image processing apparatus according to any one of claims 3 to 5,
The look-up table includes edge grid points in which at least a part of the at least one type of feature amount is set to a maximum value or a minimum value in the entire range of the feature amount,
The correction of the relationship information is performed in a third correction mode that is executed when a predetermined condition indicating that the feature value indicated by the analysis result of the target image data is close to the feature value of the edge grid point is satisfied. Including
In the third correction mode, a reference point indicating a correspondence relationship between the feature amount indicated by the analysis result of the target image data and the degree of image adjustment set on the setting screen, and among the plurality of grid points The degree of image adjustment indicated by the edge grid points is corrected by using at least some grid points;
Image processing device. An image processing apparatus according to any one of claims 3 to 6,
The user interface control unit further includes:
As a result of the correction of the relationship information, when a sudden change portion in which the rate of change in the degree of image adjustment with respect to the change in the feature amount is greater than a predetermined threshold occurs in the lookup table, the correction of the relationship information Subsequently, smoothing the degree of image adjustment of the grid points representing the sudden change portion,
Image processing device.

Images