JP3991196B2 - Image processing system and image processing server - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing system that performs correction processing on an image, and an image processing apparatus server that receives a request for correction processing for such an image from an information processing terminal connected to the Internet.
[0002]
[Prior art]
In a system in which a print is obtained from a conventional film camera, the film exposure condition is generally determined by a camera photometry system. Since the exposure conditions are determined on the assumption that the subject is very average, under conditions that are different from the average conditions such as lighting conditions when shooting the subject, how light strikes the subject, and backlight conditions, There may be a problem such as uneven exposure in image data on the film. For example, the face of the person who is the main subject may appear dark due to backlight conditions. Also, due to the difference in the light source that illuminates the subject (sunlight, fluorescent light, etc.) and the outdoor sun conditions (sunset, cloudy, etc.), the color balance may shift significantly on the color film. There was a case.
[0003]
In general, these defects are often corrected when printing from a negative film to a color print. If there are defects in the printing conditions under normal printing conditions, the printing conditions (exposure time, color balance filter, etc.) are changed, and printing correction is performed for finishing. In recent years, image data on a negative film is analyzed, and corrections such as exposure conditions and color balance conditions are often performed by digital image processing.
[0004]
On the other hand, in slide film photography (positive film), the state at the time of photography is photographed as it is on the film, so that if the photographed image is defective, the slide itself remains defective. It is also possible to print from a slide film to a color print, and some correction can be performed at that time, but in general, the correction latitude is wider for printing from negative film, and general camera photography is not possible. When doing so, it is customary to use a negative film.
[0005]
In recent years, with the spread of personal computers, color printers, film scanners, reflective scanners, and the like and improvement in performance, the handling of image data as digital image data is increasing. Also, a camera system called a digital still camera has become common from a conventional film camera system. This is an RGB digital signal that allows the captured image to be displayed as it is on a monitor, for example, by converting the optical signal into an electrical signal by a photoelectric conversion element such as a CCD, instead of exposing the captured image to film as in the past. Is stored as image data. In a digital still camera body, it is now common to store captured image data as an image file on a recording medium such as a memory card or an optical disk. The user captures the image stored in the recording medium into the user's personal computer using the communication means, or takes out the recording medium itself and captures the image data from the recording medium reading means on the personal computer side, on a monitor such as a personal computer. It is possible to appreciate images. In addition, images can be viewed on the monitor of the user's personal computer, or output to a color printer or the like and printed by the user.
[0006]
However, although the captured image data can be easily viewed by the user as a digital image file, the captured data of the digital still camera is the same as that of the above-described film camera system. If there is a problem in the situation at the time of shooting, it will be recorded on the recording medium as it is. For this reason, there has been a problem that many of the above-described problems occur during viewing. In other words, when viewing images taken by a digital still camera, the exposure conditions at the time of camera shooting, the state of the subject, the backlight conditions, etc., unlike print images obtained by correcting with a conventional negative film system, etc. There are many so-called failed works that are not corrected.
[0007]
For this reason, there is an increasing need for an application that corrects a photographed image on a personal computer so that it can be enjoyed by a user and a mechanism (such as a printer driver) that corrects the image at the time of color printing. When performing correction, the recorded digital image is a technique for effectively correcting the image within a narrow range of latitude, such as slide film shooting of a film camera system. There is a need to cope with non-correction methods (related to characteristics and correction information of digital cameras, adjustment by image size, color correction for an arbitrary output destination, adjustment method that can be easily performed by general users).
[0008]
In order to automatically create effective correction methods and correction parameters from such narrow-latency original image data, the features taken from the images and scenes (landscapes, people, etc.) are analyzed. It is necessary to use a statistical correction method according to the analysis result and a correction parameter reflecting personal preference. However, in conventional applications and the like, correction parameters calculated under experimentally limited conditions have only to be used, and there has been no correction method that takes into consideration various data and personal preference characteristics.
[0009]
On the other hand, there is a problem when the user handles image data. In other words, as described above, image data correction can be performed by a PC application or a correction mechanism in a printer driver, but for digital still camera users, processing can only be performed by a PC on which the application is installed. There was a problem that it could not be processed only by a printer having a correction function.
[0010]
The corrected image cannot be output optimally unless the processing method and parameters are changed according to the characteristics of the output device (monitor, color printer, etc.) desired by the user. Therefore, when the image corrected by the user once is used in another output device, the corrected image is not used as it is, but is corrected again for another output device based on the correction information. It will be necessary. However, there has been no method for effectively managing such image data, correction information, and output information.
[0011]
Furthermore, in the case of a digital still camera, processing up to the point where RGB image data is created is generally processed on the camera side. Although it is recommended that RGB image data be stored in the RGB signal standard called sRGB (standard RGB) as a standard standard, complicated conditions and camera manufacturer photometry are required from the shooting of the subject to the creation of the RGB signal. Since the system and the image reproduction policy are intertwined, there is also a problem that the output from different camera systems is not uniform.
[0012]
[Problems to be solved by the invention]
The present invention has been made in view of the above-described circumstances, and an object thereof is to provide an image processing system capable of correcting an image preferable for the user in consideration of various photographing conditions and user-preferred characteristics. It is what. Another object of the present invention is to realize an image processing server that provides a service for correcting an image preferable for such a user via a network such as the Internet.
[0013]
[Means for Solving the Problems]
In an image processing system that performs correction processing on an image, the present invention obtains a database that obtains a plurality of feature amounts from an image to be processed and uses the obtained feature amounts to hold information relating to past correction processing. A correction method and correction parameters for correction processing to be performed on an image to be processed are referred to, and the database is composed of two types of databases for common use and individual users, and the common database Stores the correction method and the correction parameters corresponding to a plurality of classification keys including image feature amounts, and the database for individual users stores the correction method and correction parameter determination results by the determination means in the past. Is stored as information related to the correction processing of the personal information from the database for the individual user to the shared database. It is characterized in that to reflect. Then, if correction processing is performed on the image to be processed according to the determined correction method and correction parameter, the image quality that is preferable for the user, which reflects the user's preference for the feature of the image in the correction processing, is corrected.
[0014]
The image processing server of the present invention accepts uploading of an image from an information processing terminal connected to the Internet, obtains a plurality of feature amounts from the image as described above, using the uploaded image as a processing target image, A correction method and a correction parameter for correction processing to be performed on an image to be processed by referring to a database that holds information on past correction processing using the obtained feature amount, and the determined correction method and Compensation processing is performed on an image uploaded with correction parameters, and the database is composed of two types of databases for shared use and individual users, and the shared database includes a plurality of image feature amounts. The correction method and the correction parameter are stored corresponding to the classification key, and data for an individual user is stored. Base is characterized by reflecting the information about the correction process to the shared database from the determination results of the correction method and correction parameters by the determination unit stores the information about past correction processing, database for the individual user. As a result, correction processing is performed in accordance with the characteristics of the image and the characteristics desired by the user.
[0015]
The database stores correction methods and correction parameters corresponding to a plurality of classification keys including image feature amounts, and can reflect user preferences according to image features. In addition, the database is composed of two types of databases for shared use and individual users, and detailed correspondence for each user is possible. If a shared database is used, general correction processing is also possible. Furthermore, the contents of the personal database are reflected in the shared database. This makes it possible to cope with changes in general preferences from time to time.
[0016]
In addition, it is possible to provide the user with information related to the correction method and correction parameter candidates, or to present some correction images and receive a determination instruction or selection instruction from the user. As a result, the user's preference can be understood, and this can be configured to be reflected in the database.
[0017]
If the image to be processed includes information indicating the feature amount of the image or information indicating that correction processing has already been performed on the image, the feature amount is extracted in consideration of the information. can do. Such information may be added by a tag or embedded in an image, for example, and may be acquired. When correction processing is performed on an image, information indicating that can be added to, for example, a tag or embedded in the image.
[0018]
Furthermore, when an output device that outputs a corrected image is specified, image correction is performed by determining a correction method and correction parameters according to the output device, or correction is performed with a profile indicating the characteristics of the output device. It can be configured to output a later image.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 is a block diagram showing an embodiment of the image processing system of the present invention. In the figure, 11 is an inclusion information analysis / correction unit, 12 is a feature amount extraction / analysis unit, 13 is a correction method / correction amount determination unit, 14 is a user environment correction amount calculation unit, 15 is a database, 16 is a correction processing unit, 21 is a gamma correction / color space conversion processing unit, 22 is a highlight / shadow correction unit, 23 is a light / dark contrast correction unit, 24 is a hue / color balance correction unit, 25 is a saturation correction unit, 26 is a memory color correction unit, Reference numeral 27 denotes a sharpness / enhancement processing unit.
[0020]
In this example, an inclusion information analysis / correction unit 11 and a feature amount extraction / analysis unit 12 are provided as feature amount extraction means. The inclusion information analysis / correction unit 11 determines whether or not information added to the image data exists before analyzing the input image data to be corrected. For example, the information can be used as a correction parameter for reference data or a feature amount when correcting the main image data. For example, such information may be added as a tag to an image file including input image data, or may be embedded in the image data itself.
[0021]
As a specific example, there is EXIF information that is a standard established by various companies related to digital cameras, and this EXIF information can be included in a JFIF format that is a compressed image format. When shooting with a digital camera that supports this standard, some information is recorded according to the EXIF standard when the captured image is recorded in the JFIF format. For example, information such as the type of camera that has been shot, the shooting date, and the presence or absence of a flash may be stored. If there is information that can be used for these pieces of information, the information can be recognized as a feature amount. In addition, if information representing the color characteristics of an image is included as in an ICC (International Color Consortium) profile, such information is acquired for analysis in a later process, and gamma characteristics and standard analysis spaces (for example, correction to sRGB etc.).
[0022]
As the feature amount attribute of the image that can be related from the information added to the image data in this way, the specific color (for example, skin color) data using the characteristics of the camera acquired in advance from the camera model information. It is conceivable that the distribution of the above-mentioned distribution is used, and the tendency of the gradation characteristics (brightness, darkness, etc.) is converted into feature quantities and used. It is also conceivable to set an image correction method and correction parameters based on the camera shooting mode, image size, presence / absence of flash, etc., which are specific to the model. For example, when a person, landscape, or the like is recorded as the shooting mode, it is possible to specify the scene attribute of the image data, and it is possible to select a sharpness characteristic correction method from the image size.
[0023]
The feature quantity extraction / analysis unit 12 analyzes the image data itself and extracts a plurality of feature quantities in the image. Examples of feature amounts include characterizing lightness distribution, saturation distribution, and color balance in an image from an RGB histogram, HSL (Hue: Hue, Saturation, Lightness) histogram of image data, The edge degree of image sharpness can be calculated as a feature amount from the size, the edge amount and edge strength of image data, and the like. Further, in more detail, a combination of the skin color distribution in the image and the shape extraction of the facial appearance is combined to extract a human characteristic amount, and whether or not the scene is a person-oriented scene. It is possible to calculate the amount, calculate the face size of the main subject, calculate the number of persons, and the like. Furthermore, feature quantities such as a scene shooting situation (backlight situation of a main subject, size of a person, etc.) are extracted from a local brightness distribution state in the image, and a backlight state feature quantity is calculated. be able to.
[0024]
FIG. 2 is an explanatory diagram of an example of a feature quantity classification key extracted by the feature quantity extraction / analysis unit 12. For example, as shown in FIG. 2, the feature quantity extracted by the feature quantity extraction / analysis unit 12 can be associated with a large classification key, a middle classification key, and a small classification key (not shown). In this example, keys such as an input device, an image file, a scene / subject, an overall image quality, and an output device are provided as large classification keys. Further, for each major classification key, a middle classification key as shown in FIG. 2 is provided. Further, one or more small classification keys are provided for the combination of the large classification key and the middle classification key. For example, the above-mentioned “saturation histogram” or “saturation histogram median” may be provided as a small classification key for the combination of the large classification key “whole image quality” and the middle classification key “saturation”. it can. For example, when the major classification key is “Scene / Subject” and the middle classification key is “Image Scene Type”, minor classification keys such as “Person”, “Landscape”, “Article”, “Text”, etc. should be provided. Can do. The feature quantity extraction / analysis unit 12 may not be able to acquire all the feature quantities represented by the large classification key, middle classification key, and small classification key. Can be associated with a small classification key.
[0025]
Returning to FIG. 1, the correction method / correction amount determination unit 13 searches the database 15 to be described later using the feature amount extracted by the feature amount extraction / analysis unit 12 and performs correction processing to be performed on the image data. The correction method and correction parameters are determined. The database 15 can be searched using the above-described major classification key, middle classification key, minor classification key, and feature amount. In some cases, different correction parameters may be extracted from the search result even though the correction method is the same. Alternatively, there may be a case where a plurality of correction methods are extracted. In that case, arbitration processing may be performed, such as determining the priority order of feature amounts or using an average correction parameter. FIG. 3 is an explanatory diagram of an example of the arbitration process. When a plurality of correction methods and correction parameters are searched as shown in FIG. 3A, these correction methods are synthesized and optimized in FIG. 3B, and the correction parameters are linked and optimized. . Then, the optimized correction method and correction parameter may be determined as the correction method and correction parameter to be used. Alternatively, the plurality of searched correction methods and correction parameters may be displayed on a display unit (not shown), and may be selected by a user using a selection unit (not shown). Furthermore, when the correction method and the correction parameter are determined, the searched correction method and the correction parameter may be displayed on a display unit (not shown) so as to accept correction by the user.
[0026]
Returning to FIG. 1, the user environment correction amount calculation unit 14 uses the correction method / correction amount determination unit 13 as information for determining a correction method and correction parameters according to the characteristics of the output device that outputs the corrected image. To pass. Alternatively, a profile indicating the characteristics of the output device that outputs the corrected image is added to the corrected image.
[0027]
When image conversion preferable for the user is performed, the correction method or the correction parameter may vary depending on the output device with which the user views the corrected image. For example, when viewing on a monitor attached to his / her personal computer, viewing on a mobile terminal with a limited color such as 256 colors, or displaying using a projector, the inkjet color printer of company A There are various cases such as when outputting with the color laser printer of company B, and the image quality and color development may change depending on the characteristics of each output device in each case.
[0028]
An image of a digital camera is generally handled by characteristics of a standard RGB color space called sRGB. Therefore, image conversion is generally performed based on the sRGB characteristics. However, as described above, if the image output that the user wants to finally obtain is, for example, a printer owned by the user, correcting the image quality without considering the characteristics of the printer may cause color reproduction or A different correction parameter may be selected in terms of sharpness reproduction.
[0029]
Therefore, the user environment correction amount calculation unit 14 acquires the model characteristics (resolution, screen, etc.) or ICC profile (color reproduction characteristics) of the output device such as the user's color printer, and corrects the characteristics in consideration of such characteristics. The method / correction amount determination unit 13 is configured to select and determine a correction method and a correction parameter.
[0030]
The database 15 holds information regarding past correction processing. In particular, the correction method and the correction parameter can be stored in association with the large classification key, the middle classification key, the small classification key, and the feature amount as shown in FIG. FIG. 4 is an explanatory diagram of an example of information related to the correction process stored in the database 15. In the example shown in FIG. 4A, the correction method when the major classification in the classification key shown in FIG. 2 is “saturation” in the “total image quality” middle classification key and the “saturation histogram median” is in the minor classification key. An example of a correction parameter is shown. In this example, the selection of whether or not to perform saturation correction processing and the parameters (enhancement amount) of the saturation correction processing are determined based on the median value of the histogram relating to the saturation of the entire image. For example, when the saturation histogram median value X is a value between predetermined values C1 and C2 (C1 <X <C2), saturation correction is performed, and “+2” is set as a correction parameter at that time. Is retained.
[0031]
In the example shown in FIG. 4B, the correction method and correction parameters when the large classification key in the classification key shown in FIG. 2 is “whole image quality”, the middle classification key is “sharpness”, and the small classification key is “edge degree”. An example is shown. In this example, the selection of whether or not to perform smoothing and sharpness processing, and correction processing parameters (smoothing amount and enhancement amount) based on the degree of edge of the entire image (e.g., calculated using a histogram of the enhancement amount by the differential filter). ). For example, when the edge degree Y is smaller than a predetermined value E1, it is held that sharpness correction processing is performed and “+4” is set as a correction parameter at that time. When the edge degree Y is a value between the predetermined values E2 and E3 (E2 <Y <E3), smoothing processing is performed and “+1” is set as a correction parameter at that time. Is held. Further, when the edge degree Y is larger than the predetermined value E3, both the smoothing process and the sharpness correction process are performed, and “+2” is used as a correction parameter during the smoothing process, and as a correction parameter during the sharpness correction process. “−2” is held. In this way, not only the parameters of the correction method are changed according to the feature amount, but also information regarding correction processing that is applied by changing the correction method can be held.
[0032]
In the example shown in FIG. 4C, the major classification key in the classification key shown in FIG. 2 is “scene / subject”, the middle classification key is “image scene type”, the minor classification keys are “person”, “landscape”, “ An example of a correction method and correction parameters for “article” and “character” is shown. In this example, whether or not to perform brightness adjustment, contrast adjustment, sharpness processing, skin color correction processing, black character correction processing, etc. depending on each value of the image scene type, and parameter adjustment amount for brightness adjustment, contrast adjustment, and sharpness processing Is held. The parameter adjustment amount at this time indicates a relative adjustment amount, and indicates that the relative level with respect to the adjustment amount determined from other various feature amounts is adjusted. For example, when the image scene type is “person”, brightness adjustment is performed, and “+2” is set as the parameter adjustment amount at that time. This indicates that the brightness adjustment parameter determined from various other feature values is set to be brighter by two levels. At the same time, sharpness correction processing is performed, and “−1” is set as the parameter adjustment amount at that time, and the sharpness correction processing parameters determined from other various feature amounts are set one level lower. Yes. Furthermore, it is set to perform a specific skin color correction process when the image scene type is “person”. The same applies to other image scene types.
[0033]
The database 15 holds information related to the correction process in which the correction method and the correction parameter are associated with the classification key as described above, and is corrected by a search using the classification key and the feature amount from the correction method / correction amount determination unit 13. Information about processing is retrieved. The information regarding the correction process held in the database 15 is generated based on the correction method and correction parameters used in the correction process performed in the past by the user. For example, when a plurality of correction methods and correction parameters are retrieved from the database 15 by the correction method / correction amount determination unit 13 as described above, the correction method and correction parameters automatically determined by the correction method / correction amount determination unit 13; Alternatively, based on the correction method and correction parameters that are presented to the user and selected or corrected, information relating to correction processing held in the database 15 can be updated. The update may be performed every time correction processing is performed, or may be performed every certain period or number of times. In this way, by reflecting the history of correction processing performed in the past in the information related to correction processing, it is possible to correct the image to be preferable for the user in consideration of the user's favorite characteristics and the like. Note that the specific configuration method of the database 15 is arbitrary, and various known database techniques can be used.
[0034]
As the database 15, two types of databases, that is, a shared database and a database for individual users for each user can be provided. When updating the information related to the correction process based on the correction process history described above, the database for the individual user can be updated. This makes it possible to perform a correction process reflecting the user's preference, the user's environment, etc. for each user. In addition, information regarding correction processing can be appropriately reflected from the database for individual users to the shared database. This makes it possible to cope with changes in general preferences from time to time. The shared database and the personal user database may be basically the same.
[0035]
Returning to FIG. 1, the correction processing unit 16 actually performs correction processing on the input image data in accordance with the correction method and correction parameters determined by the correction method / correction amount determination unit 13. In the example shown in FIG. 1, the correction processing unit 16 includes a gamma correction / color space conversion processing unit 21, a highlight / shadow correction unit 22, a light / dark contrast correction unit 23, a hue / color balance correction unit 24, and a saturation correction unit. 25, a memory color correction unit 26, a sharpness / enhancement processing unit 27, and the like. Each processing unit can be configured to perform a respective correction process by a known method. For example, the saturation correction unit 25 multiplies a saturation coefficient when RGB image data is subjected to HSL conversion by a predetermined coefficient and performs an increase / decrease process, or a ratio according to the brightness value. Various correction methods such as a method of changing the value can be considered. Each processing unit does not always perform all processing, but only processing determined to be corrected by the correction method / correction amount determination unit 13 is performed. The content of the correction process is also changed by the correction parameter passed from the correction method / correction amount determination unit 13. Note that the user environment correction amount calculation unit 14 may add a profile indicating the characteristics of the output device that outputs the corrected image to the corrected image data.
[0036]
The operation in the above-described embodiment of the image processing system of the present invention will be briefly described. When image data to be processed is input, the inclusion information analysis / correction unit 11 can acquire the information added to the image data by analyzing the added information. Get the amount here. Further, the feature amount extraction / analysis unit 12 extracts a feature amount from the input image data. The feature amounts extracted by the inclusion information analysis / correction unit 11 and the feature amount extraction / analysis unit 12 are associated with a classification key as shown in FIG. 2, for example, and passed to the correction method / correction amount determination unit 13.
[0037]
The correction method / correction amount determination unit 13 searches the database 15 based on the classification key passed from the feature amount extraction / analysis unit 12 and the extracted feature amount, and acquires one or more correction methods and correction parameters. Further, in some cases, correction information corresponding to the output device from the user environment correction amount calculation unit 14 is also acquired. Then, the correction method and the correction parameter are determined automatically or through selection or correction by the user from these pieces of information, and among the processing units of the correction processing unit 16, the processing unit that has been determined to perform the correction processing is determined. On the other hand, a correction parameter is passed to instruct correction processing. In response to this, the processing unit that has been determined to perform the correction process in the correction processing unit 16 executes the correction process on the input image data, and outputs it.
[0038]
Further, the correction method and correction parameters actually performed are fed back to the database 15 and the database 15 (individual user database if a database for individual users exists) is updated.
[0039]
A specific example of the above-described processing will be described by qualitatively describing the situation of an image with text. For example, it is assumed that image data “an image obtained by photographing a single subject in the foreground, but the skin color of the face of the subject person appears dark because there is no flash in the backlit state” is input. A feature amount such as this sentence is extracted by the inclusion information analysis / correction unit 11 and the feature amount extraction / analysis unit 12 to search the database 15. In this case, as a process suitable for a human scene and backlight correction, a correction method and correction parameters that brightly correct the face portion of the subject person are determined by automatic selection or selection correction by the user. For example, when image data “a landscape image mainly composed of the sky and the sea but having low brightness contrast, saturation, and sharpness feature amount” is input, the inclusion information analysis / correction unit 11 and feature amount extraction / The database 15 is searched for the feature amount based on the analysis result by the analysis unit 12, and a correction method and a correction parameter for enhancing brightness contrast, saturation, and sharpness are automatically selected for the image data or determined by selection correction by the user. . Then, based on the determined correction method and correction parameters, the correction processing unit 16 performs correction processing. Since the database 15 stores information related to past correction processing, the correction method and correction parameters are determined based on correction processing performed by the user in the past. Optimal correction processing can be performed.
[0040]
FIG. 5 is a block diagram showing an example when an embodiment of the image processing system of the present invention is used as an image processing server. In the figure, 31 is an image processing server, 32 and 33 are information processing terminals, and 34 is a network. The network 34 is a communication path constituted by the Internet, an intranet, a LAN, or the like. In the following description, the network 34 is described as being the Internet, but is not limited to this.
[0041]
The image processing server 31 includes, for example, the image processing system shown in FIG. The image processing server 31 is connected to a network 34 and has a mechanism for receiving a service request from each client (for example, the information processing terminals 32 and 33) created on the Web and providing a service. For this purpose, uploading means and downloading means (not shown) are provided. The database 15 is illustrated separately from the image processing server 31 in FIG. 5, but may be included in the image processing server 31. Alternatively, the database 15 may be configured as another database server connected to the network 34.
[0042]
The information processing terminals 32 and 33 are terminals that can communicate with the image processing server 31 through at least the network 34. The information processing devices 32 and 33 are generally user's personal computers, but may be mobile terminals with built-in digital camera functions or Internet communication functions. Further, as shown in the figure, it may have a configuration capable of capturing an image from an image input device such as a scanner or a digital camera. Of course, the image input device or the like is arbitrary, and for example, an image may be acquired from another computer or server.
[0043]
FIG. 6 is a flowchart showing an operation example in an example in which one embodiment of the image processing system of the present invention is used as an image processing server. In the configuration as described above, it is assumed here that the information processing terminal 32 requests the image processing server 31 to perform image data correction processing and receives the correction processing result. Of course, the same applies to a request from the information processing terminal 33 to the image processing server 31.
[0044]
When requesting image data correction processing from the information processing terminal 32 to the image processing server 31, first, in S 61, the user logs in to the service provided by the image processing server 31 from the information processing terminal 32. At this time, identification information for identifying the user may be transmitted. In some cases, the user is authenticated by, for example, a password.
[0045]
After logging in to the image processing server 31, the user selects a desired service menu in S62. For the sake of explanation, it is assumed that the selection is made from the service menu of “automatic image correction” and “automatic image correction (multiple selection)” as an example. Of course, the service menu of the present invention is not limited to this example.
[0046]
First, it is assumed that the user selects “automatic image correction” from the service menu. After selecting the menu, in S63, the user performs an operation of uploading an image file including image data to be corrected in the information processing terminal 32 used by the user to the image processing server 31. Of course, the image file does not have to exist in the information processing terminal 32 performing the operation as long as the user can perform the upload operation. The image processing server 31 can manage the image file to be processed by the image file upload operation.
[0047]
In S64, the uploaded image file is subjected to the correction process as described above by a job for automatic correction. That is, in the automatic correction job, various feature values are extracted by analyzing image data, the database 15 is searched using the extracted feature values, and one or more correction methods and correction parameters are acquired. Is used to correct the image data in the image file. As described above, the database 15 stores correction methods and correction parameters set based on correction processing performed by the user in the past. For this reason, it is possible to select or calculate a correction method or correction parameter that is preferable for the user in view of past correction processing, and to perform correction processing using the correction method and correction parameter.
[0048]
After performing the correction process, the corrected image data is downloaded to the information processing terminal 32 of the user, and the corrected result image is displayed in S65. At this time, it is also possible to create a thumbnail of the original image before correction and display the result image together with the thumbnail of the original image on the information processing terminal 32 of the user. This makes it easier for the user to compare the images before and after correction to determine whether the correction process is appropriate.
[0049]
The user refers to the result image displayed in S65. In S66, the user determines and inputs the suitability of the correction process, and the determination result is transmitted to the image processing server 31. If the user is not satisfied with the corrected image, the process returns to S64 to select another correction method and another parameter candidate and perform the correction process again. Display and ask the user's judgment. Such processing is repeated until the user is satisfied or there are no more candidates.
[0050]
When the user is not satisfied with the corrected image, some feature amounts and parameters are displayed on the information processing terminal 32, and processing such as selecting a next candidate based on the feature amount that the user does not like is performed. The next candidate can be selected efficiently. For example, a correction parameter such as “bright” or “dark” may be designated with respect to the feature amount “overall brightness”.
[0051]
When a corrected image that the user is satisfied with is obtained, the fact is input to the image processing server 31. Since the corrected image has already been downloaded to the information processing terminal 32, the user can use the corrected image data. When the image processing server 31 receives the fact that the user is satisfied, in S67, the correction method and the correction parameters used during the correction processing that the user is satisfied are additionally recorded in the database 15 or the database 15 is updated. It is also possible to save images before and after the correction. In this way, the correction process for a series of image data is completed.
[0052]
In the above operation example, even if the user is not satisfied, the corrected image is downloaded to the information processing terminal 32 one by one. For speed improvement, for example, the image presented to the user in S65 is a thumbnail of the corrected image. After the user's consent is obtained, the actual corrected image can be downloaded to the information processing terminal 32 in S67. In this case, when a thumbnail is presented, a thumbnail image is first generated from the image data, and correction processing is performed on the thumbnail image, so that the speed can be further increased.
[0053]
In addition, when a plurality of correction methods and correction parameters are obtained by the search, correction processing can be performed on these to prepare a corrected image. FIG. 7 is an explanatory diagram of an example when a plurality of corrected images are prepared. As shown in FIG. 7, using the respective correction methods and correction parameters obtained by the search, the input image data is corrected to obtain a corrected image. And when providing to a user, for example, a thumbnail can be created and displayed on the information processing terminal 32 together with the thumbnail of the original image. For example, correction processing using one correction method and correction parameter may be performed, and correction processing using another correction method and correction parameter may be performed while presenting to the user. As a result, when the corrected image presented by the user is not satisfied, it is possible to shorten the time (turnaround time) until the next corrected image is presented and to construct a system with good operability. is there.
[0054]
In S62, it is assumed that the user selects “automatic image correction (multiple selection)” from the service menu. After selecting the menu, in S71, the user performs an operation of uploading an image file including the image data to be corrected to the image processing server 31 as in S63 described above. The uploaded image file is analyzed for image data and various feature quantities are extracted as in the case of selecting “automatic image correction” described above, and the database 15 is searched using the extracted feature quantities. One or more correction methods and correction parameters are acquired. In this “automatic image correction (multiple selection)”, one or more of a plurality of acquired correction methods and correction parameters are determined or selected at a time in S72, and correction is performed according to the respective correction methods and correction parameters. Processing is performed to obtain one or more corrected image data. The one or more corrected image data are downloaded to the information processing terminal 32, and one or more correction results are displayed in S73.
[0055]
FIG. 8 is an explanatory diagram illustrating an example of a display screen on which a plurality of correction image candidates are simultaneously displayed. In this example, the corrected image is arranged around the original image so that the original image and the corrected image can be compared. The user compares each corrected image with the original image, and can select a corrected image according to his / her preference. Of course, the display method is arbitrary, and is not limited to the example shown in FIG. 8, but it is desirable that the correction effect can be compared by arranging it so as to be compared with the original image. In this way, making it possible to select from a plurality of images makes it possible to determine image candidates that are preferable for the user at a time and select them while comparing them, which is a better service for the user.
[0056]
As described above, in “automatic image correction (multiple selection)”, a plurality of corrected images are displayed. In S74, the user may select a desired image from one or more displayed correction results. Alternatively, it is possible to input that the displayed correction result is not satisfactory. This user's intention is transmitted to the image processing server 31.
[0057]
If the user is not satisfied with the plurality of correction results, the process returns to S72, and if there are other correction methods and correction parameters in the search result, one or more of them are selected and the above described. The correction process may be performed as described above and presented to the user. In addition, when the user inputs that the user is not convinced, some feature quantities and parameters are displayed on the information processing terminal 32 as described above, and the next candidate is selected based on the feature quantities that the user does not like. It can also be configured to perform processing.
[0058]
When the corrected image accepted by the user is selected, the image processing server 31 additionally records or updates the database 15 with the correction method and correction parameters used for the corrected image accepted by the user in S75. It is also possible to save images before and after the correction. In this way, the correction process for a series of image data is completed.
[0059]
In the above-described operation example, one or more corrected image data are downloaded and displayed. However, in order to improve the speed, for example, the one or more corrected images presented to the user in S73 are converted into thumbnail images. After converting and downloading, and selecting by the user, the selected corrected image may be downloaded to the information processing terminal 32 in S75. Further, in this case, it is possible to further increase the speed by generating a thumbnail image from the first input image data and applying the respective correction processes to the thumbnail image.
[0060]
FIG. 9 is an explanatory diagram of the outline of the registration update processing of the correction result for the database. In the initial state, the database 15 stores correction methods and correction parameters set in advance according to, for example, the classification of feature amounts as shown in FIG. As explained in the above operation example, when the corrected image or its thumbnail is presented to the user and it is accepted or selected as the desired corrected image, it is used to generate the corrected image. The correction method, the correction parameter, and the feature amount of the image data before correction are added / updated to the database 15 as a history. Further, the original image and the corrected image may be recorded in the database 15.
[0061]
At this time, as shown in FIG. 9, when the database 15 is composed of a shared database and a database for individual users for each user, such history, original images, corrected images, etc. Can be recorded in the database. By managing the original image and the corrected image in the database 15, for example, the user can manage the original image and the corrected image in association with the feature amount analyzed by the inclusion information analysis / correction unit 11 and the feature amount extraction / analysis unit 12. Can also be provided.
[0062]
If the correction processing history is simply added to the database 15 as described above, the user's personal preference and the learning effect by the user's selection are not reflected, and the correction method and correction parameters are unified. Therefore, it is necessary to feed back to the database 15 the correction method and correction parameters of the image finally selected by the user.
[0063]
FIG. 10 is an explanatory diagram of a specific example of the correction method and the correction parameter when the correction method and the correction parameter selected by the user are reflected in the database. FIG. 10 shows an example in which the major classification key is “classification related to overall image quality” and the middle classification key is “saturation”. In this example, an average value (saturation correction history ave) of the saturation adjustment amount actually selected by the user is stored for the small classification of the feature amount. For example, when the saturation histogram median X is larger than a predetermined value C3 (C3 <X), the preset saturation correction parameter is “−1”, but the average value of the history so far is “ -0.2 ", indicating that the parameter with the correction level of 0 seems to be better. Accordingly, for example, for the user, the correction parameter when performing the saturation correction process can be set to “0” (or the saturation correction process is not performed). In this way, correction processing according to the user's preference can be performed.
[0064]
The correction method and the correction parameter according to the correction processing history are updated every time the user corrects the image, for example, or the history is accumulated to some extent and updated at a predetermined timing. It can be carried out. The predetermined timing may be elapse of a predetermined time or execution of a predetermined number of correction processes. In addition, the history to be referred to at the time of updating reflects the correction method and correction parameters for each user set up to that point and all the history recorded so far. The correction method and the correction parameter can be updated. For example, when the preference change is severe, the current user's preference can be better reflected by updating the correction method and the correction parameter according to the recent history rather than being dragged by the past history.
[0065]
Information related to the correction processing performed in the past by the user as described above can be recorded in the shared database shown in FIG. 9, but it reflects the preference for each user. It is desirable to record in the database. This makes it possible to determine correction methods and correction parameters with reference to correction information reflecting each user's preference. Then, a correction method based on the history and a feature quantity for which no correction parameter exists in the database for individual users may be configured to refer to the shared database. Further, for example, even when the user does not understand the correction method and the correction parameter based on the search result from the database for individual users, the common database is searched to obtain a general correction method and correction parameter. Also good. Alternatively, search results may be acquired from both databases, and a correction method and correction parameters to be used may be selected and determined from these search results according to a predetermined priority order.
[0066]
It is also possible to reflect the history, correction method, and correction parameters accumulated for each user in a shared database. In a system used by many users, it is possible to grasp the tendency of preference of the entire user, and to reflect the tendency of preference of the entire user in a shared database. As a result, it is possible to perform a correction process in time.
[0067]
FIG. 11 is an explanatory diagram of a specific example for selecting a correction method and correction parameters for each user. FIG. 11A shows an image to be corrected. From the feature amount analysis of this image data, the feature amount that is a scene that is photographed with a single person as the center, but is in a backlit state, is a state in which the main subject is reflected under. . Specifically, according to the classification key shown in FIG. 2, for example, the large classification key is “scene / subject”, the middle classification key is an image scene type, the small classification key is “person”, and similarly, “main subject presence / absence” is set. Then, “Yes”, “Main subject size” is “Medium”, “Main subject light condition” is “Backlight”, “Number of persons” is “One person”, and so on.
[0068]
For such image data, in a general correction method, the person portion that is under-focused by the backlight condition with the person as the center is corrected brightly. Therefore, a correction method and parameters for performing such correction are selected and determined from the shared database. FIG. 11B shows the result of correction using such a general correction method and correction parameters.
[0069]
However, when correction processing is performed using such a general correction method and correction parameters, the face of a person becomes bright but the background tends to fly too much. A user who dislikes such background skipping prefers an image with a slightly weaker correction as shown in FIG. 11C, for example. In such a case, if information related to such correction processing is not registered in the database for individual users, the correction image shown in FIG. 11B is first presented, and re-correction is instructed without understanding. An instruction to be accepted when the corrected image shown in 11 (C) is presented may be given. Such correction history is accumulated in the database 15 and reflected in the database for individual users.
[0070]
After the past correction information is reflected in the database for individual users, when a similar image (for example, the image shown in FIG. 11A) is input again, the database is searched for the individual user by searching the database based on the extracted feature amount. The search results obtained from the database can be selected and determined with priority. As a result, the corrected image shown in FIG. 11C can be presented to the user from the beginning.
[0071]
As described above, in the example shown in FIG. 11, the correction method and correction parameters in the shared database change the brightness / contrast of the entire image to correct the brightness of the person's face, so that the background, etc. The details of the scenery will be lost, and the atmosphere of the scene cannot often be utilized. This user understands that there is a tendency to correct the background rather than the person in a balanced manner, and as shown in FIG. 11C, the corrected image with the background of the person somewhat dark but with the background skip suppressed as the first candidate. It can be presented to the user. In this way, it is possible to perform a correction process reflecting the user's preference.
[0072]
The image processing system of the present invention and the image processing server including the image processing system of the present invention have been described above. Hereinafter, some of these modifications will be described. FIG. 12 is an explanatory diagram of an example of handling for a corrected image. When the image correction processing as described above is executed, some kind of image conversion is applied to the original image. Regardless of the converted image that has already undergone image analysis and image conversion, a situation in which the corrected image data is input again later is also assumed. In this case, in the correction process as described above, information indicating that the correction process has been performed on the image data is added to the corrected image data, and the inclusion information analysis / correction unit 11 when re-input is performed. Alternatively, it is conceivable that the feature amount extraction / analysis unit 12 acquires this information.
[0073]
For example, as shown in FIG. 12A, information indicating that correction processing has been performed can be added in the form of tag information included in the image storage format. Alternatively, as shown in FIG. 12B, for example, a method of embedding information in image data itself called “Water Marking” may be used. In this case, for example, the feature amount extraction / analysis unit 12 may perform processing for restoring the embedded information.
[0074]
When information indicating that the correction processing has already been performed is acquired from the input image data, the user is notified that the correction has been completed, and whether or not correction is performed again is determined. Can be configured to receive. It may be configured to be able to set whether or not to perform such notification, whether to permit or reject correction again.
[0075]
Another modification will be described. In the above description, the image to be presented to the user and the image to be downloaded are substantially the same although there is a difference between thumbnails or not. For example, when trying to correct the image to be output by a printer or the like, As described above, since the color development or the like is different for each output device, the corrected image selected on the screen may be different in appearance from the printed image.
[0076]
Conventionally, as a method that can be used in a specific environment, there is a color management system (hereinafter referred to as CMS) as a method for solving the problem of color difference between different devices. A system called ColorSync is known for Apple's Macintosh system, and ICM is known for Microsoft's Windows system. When displaying or printing an image, input device profiles (source profiles) and output device profiles (target profiles) are input or prepared, and color conversion is performed using them. It is a system that tries to guarantee color consistency.
[0077]
In this system (or this server), for example, with respect to color matching, when the user temporarily confirms using the above-described CMS and using the profile of the device that the user wants to output at the time of correction processing, By performing color reproduction (soft proofing) in the output device on the display screen, it is possible to present the appearance at the time of output to the user. In addition, regarding the difference in appearance between the monitor image and print image such as sharpness, the model data is stored in the database, and the sharpness correction amount is adjusted at the time of selection by the user and at the time of download, so that it can be seen as much as possible. It is also possible to resemble
[0078]
If the user does not have a printer profile, an ICC profile or the like of a device such as a color printer designated by the user can be retrieved from the Internet, obtained, and processed. Alternatively, it is possible to store a profile in advance as a database, process it, and send it to the user as necessary.
[0079]
【The invention's effect】
As is apparent from the above description, according to the present invention, various input image data such as an image taken by a user with a digital camera or the like can be automatically analyzed and corrected to a preferable image. There is an effect that a correction effect reflecting the user's preference can be obtained during the correction process. Further, by holding the correction method and the correction parameter for each user, it is possible to obtain a correction effect that more reflects the preference of the specific user. Furthermore, there is also an effect that the user's image accumulation and management can be provided to the user via the Internet as an environment used in combination with the correction process.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an embodiment of an image processing system according to the present invention.
FIG. 2 is an explanatory diagram of an example of a feature quantity classification key extracted by a feature quantity extraction / analysis unit 12;
FIG. 3 is an explanatory diagram of an example of arbitration processing;
FIG. 4 is an explanatory diagram of an example of information related to correction processing stored in a database 15;
FIG. 5 is a configuration diagram showing an example when an embodiment of the image processing system of the present invention is used as an image processing server;
FIG. 6 is a flowchart showing an operation example in an example when an embodiment of the image processing system of the present invention is used as an image processing server;
FIG. 7 is an explanatory diagram of an example when a plurality of corrected images are prepared.
FIG. 8 is an explanatory diagram illustrating an example of a display screen on which a plurality of correction image candidates are simultaneously displayed.
FIG. 9 is an explanatory diagram of an overview of registration update processing of correction results for a database.
FIG. 10 is an explanatory diagram of a specific example of a correction method and a correction parameter when the correction method and the correction parameter selected by the user are reflected in the database.
FIG. 11 is an explanatory diagram of a specific example in which a correction method and a correction parameter are selected for each user.
FIG. 12 is an explanatory diagram of an example of handling for a corrected image.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 11 ... Inclusion information analysis / correction part, 12 ... Feature quantity extraction / analysis part, 13 ... Correction method / correction quantity determination part, 14 ... User environment correction amount calculation part, 15 ... Database, 16 ... Correction processing part, 21 ... Gamma Correction / color space conversion processing unit, 22 ... highlight / shadow correction unit, 23 ... light / dark contrast correction unit, 24 ... hue / color balance correction unit, 25 ... saturation correction unit, 26 ... memory color correction unit, 27 ... sharpness Emphasis processing unit, 31: image processing server, 32, 33: information processing terminal, 34: network.

Claims (17)

  In an image processing system that performs correction processing on an image, a database that holds information on past correction processing, a feature amount extraction unit that obtains a plurality of feature amounts from an image to be processed, and the feature amount extraction unit A correction method for correction processing performed on the image by referring to the database using feature amounts and a determination unit for determining correction parameters are included, and the databases are two types of databases for shared use and individual users The shared database stores the correction method and the correction parameters corresponding to a plurality of classification keys including image feature amounts, and the database for an individual user is a correction method by the determination means And correction parameter determination results are stored as information relating to past correction processing, and the database for the individual user is stored. The image processing system characterized by reflecting the information about the correction process to the shared database from.   The determining means provides the user with information on a correction method and correction parameter candidates, and determines a correction method and a correction parameter to be used in response to an instruction for determining or selecting a correction method and a correction parameter by the user. The image processing system according to claim 1.   The feature amount extraction unit determines whether or not a tag indicating the feature amount of the image is added to the image when analyzing the feature of the image, and performs different processing depending on the determination result. The image processing system according to claim 1 or 2.   The feature amount extraction unit, when analyzing the feature of the image, determines whether or not a tag indicating that correction processing has been performed on the image, or presence of information embedded in the image, The image processing system according to any one of claims 1 to 3, wherein the processing is made different depending on the determination result.   5. The apparatus according to claim 1, further comprising a processing unit that performs correction processing on the image using the correction method and the correction parameter determined by the determination unit. Image processing system.   The determination means determines a plurality of correction methods and correction parameters, and the processing means converts the images according to the correction methods and correction parameters determined by the determination means, respectively, thereby converting a plurality of converted images. The image processing system according to claim 5, wherein the image processing system is generated.   A display unit configured to display a plurality of converted images generated by the processing unit so as to be selectable by a user; and a selection unit capable of being selected by the user from the displayed converted images. 7. The image processing system according to claim 6, wherein information relating to past correction processing corresponding to the converted image selected by the user by means is updated.   The image processing system according to claim 1, wherein the determination unit determines a correction method and a correction parameter in accordance with characteristics of an output device that outputs a corrected image. .   The image processing according to claim 5, wherein the processing unit outputs a profile indicating characteristics of an output device that outputs a corrected image together with the corrected image. system.   In an image processing apparatus server that receives an image processing request from an information processing terminal connected to the Internet, an upload unit that receives an image upload from the information processing terminal, a database that holds information relating to past correction processing, and an upload A feature amount extraction unit for obtaining a plurality of feature amounts from the image obtained, a correction method and correction for correction processing performed on the image with reference to the database using the feature amounts extracted by the feature amount extraction unit Determining means for determining parameters, and processing means for processing the image in accordance with the correction method and correction parameters determined by the determining means, and the database includes two types of databases for shared use and individual users And the shared database includes a plurality of image features including image features. The correction method and the correction parameter are stored corresponding to the classification key, and the database for the individual user stores the correction method and correction parameter determination result by the determining means as information on past correction processing, An image processing server, wherein information relating to correction processing is reflected from the database for individual users to a shared database.   The feature amount extraction unit determines whether or not a tag indicating the feature amount of the image is added to the uploaded image when analyzing the feature of the image, and performs different processing depending on the determination result. The image processing server according to claim 10.   The feature amount extraction unit, when analyzing the feature of the image, determines whether or not a tag indicating that correction processing has been performed on the image, or presence of information embedded in the image, The image processing server according to claim 10, wherein the processing is different depending on the determination result.   The determination means determines a plurality of correction methods and correction parameters, and the processing means converts the images according to the correction methods and correction parameters determined by the determination means, respectively, thereby converting a plurality of converted images. The image processing server according to claim 10, wherein the image processing server is generated.   Download means for downloading the plurality of converted images generated by the processing means to the information processing terminal, and acquisition means for acquiring information related to the converted image selected by the user from the downloaded converted images, The image processing server according to claim 13, wherein the database updates information relating to past correction processing according to information relating to the converted image selected by the user acquired by the acquisition unit.   The plurality of converted images are thumbnail images, and the determining unit determines a final correction method and a correction parameter according to information about the converted image selected by the user acquired by the acquiring unit, and the processing unit is the determining unit. 15. The image processing according to claim 13, wherein the final correction image is generated by performing correction processing according to the determined final correction method and correction parameter, and the download unit downloads the final correction image. server.   16. The image processing server according to claim 10, wherein the determination unit determines a correction method and a correction parameter in accordance with characteristics of an output device that outputs a corrected image. .   16. The image processing according to claim 10, wherein the processing unit outputs a profile indicating characteristics of an output device that outputs a corrected image together with the corrected image. server.

Images