JP2018142352A - Image processing system, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system and a method that have intuitive and simple usability, can reach a desired result with a smaller man-hours, and can improve the degree of freedom of processing.SOLUTION: The system is an image processing system that retouches an image, and comprises: an information storage part 33 that stores a plurality of pieces of case information including reference information for retouching that is referred to for specifying the details of retouching, and case identification information with which a case where retouching is performed according to the details of retouching can be identified; an image acquisition part 31 that receives an input of an image and acquires the image; a case selection part 34 that arranges and displays the plurality of pieces of case information stored in the information storage part 33 on a display part on the basis of the case identification information included in the respective pieces of case information, and receives selection of the case information from a user; and a case application part 35 that retouches the image acquired by the image acquisition part 31 according to the details of retouching specified with the reference information for retouching included in the selected case information.SELECTED DRAWING: Figure 10

Description

  The present invention relates to an image processing system for correcting an image, an image processing method, and a program for causing a computer to execute the method.

  An image obtained by taking an image with a digital camera or the like is modified (corrected) so as to eliminate the problem by touching the defective part. This modification is performed, for example, by changing image expression such as color tone and sharpness.

  For image modification, the user or device designates image processing means, parameters, etc., and designates the type to be modified accordingly (hereinafter referred to as the standard type), the target image, feature quantity, etc., There are two types of molds (hereinafter referred to as transfer molds) that are modified to achieve that goal. In addition to the classification based on such modification types, there are the following three types of image modification that are classified according to the contents specified by the user.

  The first type is a type in which the user directly designates an image modification operation (operator) and parameters (hereinafter referred to as means designation type). The second type is a type in which the user designates a target in the form of an image (hereinafter referred to as a target designation type). The third type is a type (hereinafter referred to as a result specifying type) in which a user finally specifies a modified image that matches expectations from a plurality of modified images.

  Considering the combination of the former two types and the latter three types, a total of six categories can be considered. However, since there is no method for simultaneously satisfying the transfer mold and the means designating mold, there are actually five ways.

  Examples of standard types include tone curve conversion, sharpness conversion, and edge enhancement. Since these means are well-known, detailed description is abbreviate | omitted. Since these means can change parameters freely, the degree of freedom of processing is high. However, trial and error is required to achieve the goal, and skill is required. As an example of a combination of the standard type and the means designation type, there is a technique in which the user designates the color of the designated pixel and the desired color and adjusts the color of the entire image based on them (see Patent Document 1). ).

  Also, there is a technique in which the correspondence between pixel values before and after adjustment is created as a color conversion list using the adjustment result of the image data, and this is applied to other images for correction (see Patent Document 2). In addition, there is a technique for easily creating a color conversion list including user interaction (see Patent Document 3), and a similar technique assuming a video as a processing target (see Patent Document 4).

  As an example of a combination of the standard type and the target designation type, there is a technique for presenting and selecting an effect in an icon form, which is executed by an application called Instagram. As an example of combining the standard type and the result specifying type, there is a technique in which a small number of representative processes and parameters are determined in advance, and the modified image is presented to the user and selected (Patent Document 5). reference).

  As an example of a combination of the transfer type and the target specification type, the image characteristics (highlight density, shadow density, average density, etc.) are set to the reference image image so as to align the color gradation of the finished photo print of similar scenes. There is a technique (see Patent Document 6) that matches the characteristics. In addition, in order to match the color gradation of the face of the same subject with the digital photo printer, the user is allowed to select a target face image (area), and the color gradation of the color area of the face image of the input image is set to the target face. There is also a technique for matching the color gradation of the color area of the image (see Patent Document 7). Furthermore, in order to intuitively reproduce a desired image, there is a technique for correcting an input image so that the image reproduction item (color reproduction, gradation reproduction, contour reproduction) is referred to an image reproduction item of teacher data (patent) Reference 8).

  As an example of the combination of the transfer type and the result specifying type, from the result of correction processing so that the second feature amount of the second image to be corrected is close to the first feature amount of each of the plurality of types of first images. There is a technique for selecting a desired first image (see Patent Document 9).

  The combination of the standard type and the means specifying type gives the user a simple option at the expense of the processing freedom, which is an advantage of the standard type, so that the processing freedom is limited. In addition, there is a difficulty in predicting that the result of applying color conversion on a pixel-by-pixel basis will change the impression of the entire image unless it is actually executed. In this regard, the combination of the standard type and the target designation type also has difficulty in prediction because the actual effect depends on the input image, and trial and error are necessary. The combination of the standard type and the result specification type can be selected from the modified images, so there is no problem in predictability, but the choice of processing is given to the user at the expense of the processing freedom, so the processing freedom is limited. End up.

  Since the combination of the transfer type and the target designation type cannot change means and parameters freely, the degree of processing freedom becomes extremely limited. Predictability is also reduced from the above combinations, but is not resolved. The combination of the transfer type and the result specification type can eliminate the difficulty of prediction, but like the combination of the transfer type and the target specification type, the problem that the degree of freedom of processing is limited can be solved. Can not.

  Thus, the combination of the two types can eliminate the difficulty of prediction compared to the standard type only, but the degree of freedom of processing becomes limited. If only the standard type is used in order to increase the degree of freedom of processing, it becomes difficult to predict. By the way, if the difficulty of prediction does not occur, the intuitiveness is enhanced, the usability is good, and the desired result can be reached with less man-hours. Therefore, it has been desired to provide a system and a method that are intuitive and easy to use, can reach a desired result with less man-hours, and can increase the degree of processing freedom.

  In view of the above problems, the present invention is an image processing system for correcting an image, and identifies reference information used for specifying the content of the correction and a case where the correction is performed according to the content of the correction An information storage unit that stores a plurality of case information including possible case identification information, an image acquisition unit that receives an input of an image and acquires the image, and a plurality of case information stored in the information storage unit, Based on the case identification information included in the case information and displayed on the display unit, the case selection unit that accepts the selection of case information from the user, and the modification reference information included in the selected case information An image processing system is provided that includes a case application unit that modifies an image acquired by an image acquisition unit in accordance with the content of the modification.

  According to the present invention, it is intuitive and simple to use, can achieve a desired result with less man-hours, and can increase the degree of freedom of processing.

1 is a diagram illustrating a configuration example of an image processing system according to an embodiment. 1 is a hardware configuration diagram of an image processing system according to an embodiment. The figure explaining the type | mold of image correction. The figure explaining example information. The figure which showed the example which hierarchized the case information of the standard type and the transcription | transfer type. The figure which illustrated the screen for selecting one case information from a plurality of case information. The figure which showed an example of the screen when arrange | positioning and displaying case information based on case identification information. The figure which showed another example of the screen when arrange | positioning and displaying case information based on case identification information. The figure which showed the example of a display of a target designation | designated type and a result designation | designated type. 1 is a functional block diagram illustrating a first embodiment of an image processing system according to the present embodiment. 11 is a flowchart showing a flow of image processing executed by the image processing system shown in FIG. The functional block diagram which showed 2nd Embodiment of the image processing system of this embodiment. The figure which illustrated the screen which selected the field for modification by the field selection part. The flowchart which showed the flow of the image processing performed by the image processing system shown in FIG. The functional block diagram which showed 3rd Embodiment of the image processing system of this embodiment. The flowchart which showed the flow of the image processing performed by the image processing system shown in FIG. The functional block diagram which showed 4th Embodiment of the image processing system of this embodiment. The figure which showed the example of the screen which displayed the evaluation parameter | index evaluated by a user by the image evaluation part. The flowchart which showed the flow of the image processing performed by the image processing system shown in FIG.

  FIG. 1 is a diagram illustrating a configuration example of an image processing system. The image processing system includes an image processing device 10 and a server device 11 that stores case information described later. The image processing system may be configured by only the image processing device 10 as long as the case information is not stored in the server device 11 but is stored in a storage device included in the image processing device 10. In addition, the case information can be stored in devices other than the image processing device 10 and the server device 11. In this case, the image processing system includes the image processing device 10 and the devices.

  In the example illustrated in FIG. 1, in addition to the image processing device 10 and the server device 11, an image reading device 12 that reads a document or the like and inputs an image as image data to the image processing device 10, images a subject, and the image processing device 10. And an imaging device 13 for inputting an image as image data. In addition to the above case information, the server device 11 stores image data as an image, and can read and transmit image data in response to a request from the image processing device 10.

  In the image processing system shown in FIG. 1, the image processing device 10 and the imaging device 13 are directly connected by a cable 14, and the image processing device 10, the image reading device 12, and the server device 11 are connected to each other via a network 15. Yes.

  The image processing system may be configured to include two or more of the image processing device 10, the server device 11, the image reading device 12, and the imaging device 13, or may be configured to further include other devices. In FIG. 1, the cable 14 is used for connection, but it is also possible to connect by wireless communication. Further, the image processing apparatus 10, the image reading apparatus 12, and the server apparatus 11 may be directly connected by a cable instead of the network 15.

  The network 15 can be connected by a cable or wirelessly. When connecting wirelessly, it can connect to the network 15 via a base station called an access point. The network 15 can use a WAN (Wide Area Network), the Internet, or the like.

  As the image processing apparatus 10, a PC, a tablet terminal, or the like that can correct an image can be used. The hardware configuration, functions, and processing contents of the image processing apparatus 10 will be described later.

  As the server device 11, a PC, a workstation, or the like can be used. The hardware configuration of the server apparatus 11 can be the same as the hardware configuration of the image processing apparatus 10. For this reason, explanation is omitted here. The image processing device 10 can access the server device 11, request case information and image data from the server device 11, and acquire the case information and image data requested from the server device 11.

  As the image reading device 12, a scanner device, an MFP (Multi Function Peripheral), or the like can be used. The image reading apparatus 12 includes a document table for setting a document at a predetermined position, a light source that irradiates light on the document, a mirror that guides light from the document in a predetermined direction, and a photoelectric conversion element that converts light into an electrical signal. In addition, an A / D converter that converts an electrical signal into digital data is included. The image reading device 12 outputs the converted digital data to the image processing device 10 as image data.

  As the imaging device 13, a digital camera, a video camera, or the like can be used. For example, a digital camera has an optical system including a lens and a diaphragm mechanism, a light receiving element that receives light incident through the lens and outputs it as an electrical signal, and a process that converts the output electrical signal into digital data. And an image arithmetic circuit for performing. The digital camera also includes a memory that records processed digital data as image data, a display device for displaying a captured image, a strobe, an operation switch, an interface for connecting a cable, and the like. The image processing apparatus 10 can access this memory and read out and acquire the image data recorded in this memory.

  A hardware configuration of the image processing apparatus 10 will be described with reference to FIG. Similar to a general PC, the image processing apparatus 10 includes an input I / F 20, a CPU 21, a storage device 22 such as a memory, an auxiliary storage device 23 such as an HDD, an output I / F 24, a drive 26 into which a recording medium 25 is inserted, a controller 27. The input I / F 20 connects, for example, one end of the cable 14 when the imaging device 13 is connected to the image processing device 10 by the cable 14. The other end of the cable 14 is connected to the imaging device 13.

  The CPU 21 executes a program stored in the auxiliary storage device 23, controls the image processing apparatus 10, and realizes a predetermined function. The image is corrected by realizing the predetermined function. The storage device 22 provides a work space for the CPU 21 and stores a boot program, firmware, setting parameters, and the like. When the output I / F 24 is connected to a display device (not shown) via a cable, the output I / F 24 is connected to one end of the cable. The other end of the cable is connected to a display device (not shown).

  On the recording medium 25, for example, a program for modifying the image is recorded. This program is installed in the auxiliary storage device 23 via the drive 26 that controls reading and writing of the recording medium 25. The controller 27 functions as a network I / F connected to the network 15 and controls communication with the network 15. Note that the above program may be independent program software or may be implemented in a printer driver.

  Before describing specific functions of the image processing apparatus 10 and processes executed by the functions, an outline of the present invention will be described. The disadvantage of the standard type that is corrected using the designated image processing means and parameters is that it requires skill and trial and error. The disadvantage of the combination of the standard type and the means designation type, or the combination of the transfer type and the target designation type that are modified for the image referred to as the target is that the degree of processing freedom is low. These points are correlated, and it has not been possible with conventional techniques to eliminate both drawbacks.

  In order to eliminate both of these drawbacks, the present invention introduces a mechanism in which the granularity (unit) of image modification is made larger than that of the above-described standard type and registration and reuse thereof are possible. A unit that can be registered and reused is defined as a case. The case is an actual example of modification performed by a skilled person, and from the case, it is possible to obtain case information such as the content of the modification, parameters used, and images before and after the modification. If such a case is applied, a desired image can be corrected without requiring skill or trial and error. In addition, if a desired case can be selected and applied from many cases, the degree of processing freedom can be increased.

  Therefore, it is necessary to collect cases, but when considering collecting cases efficiently, the transfer type is effective. This is because it is only necessary to collect a desired image, and it is not necessary to record an input image or a correction process in the middle. However, considering the following two points, it is not a good idea to eliminate the standard type. In other words, many professional retouchers (persons who specialize in photo editing) actually use standard-type retouching, and transfer-type retouching is currently under development.

  More specifically, the standard image modification can be properly used only by a skilled person. The expert has a lot of knowledge and experience and understands what kind of image processing means is applied to what degree and in what order, and what results are obtained. The skilled person can respond according to the pattern of the image, has an aesthetic eye that evaluates the quality of the modified image (modified image), and comprehensively possesses these skills. It is possible to fix it. However, such skills are extremely valuable and difficult to externalize easily. For this reason, the current situation is that beginners and those who are not experienced have acquired skills through repeated trial and error while being taught by skilled workers. Also, it is difficult for even an expert to express and communicate clearly in words.

  Therefore, the only way to externalize such skills is to accumulate and utilize actual examples of image modification performed by experts. In the present invention, a method selected by an expert is accumulated and used by introducing an information format called an example. If it does so, it will become possible to utilize the technique and image which were accumulate | stored in the standard type in a transfer type.

  Here, with reference to FIG. 3 to FIG. FIG. 3A is a diagram illustrating modification by the standard type. In this modification, two images before and after modification (case image) used in the case and a tone curve which is an image processing means applied in the case are used. The tone curve is a means for correcting the gradation of the image, and the gradation correction is performed so that the image A of the two case images becomes the other image B. For this reason, when an image to be modified is input as an input image, it is converted by this tone curve, and the converted image is output as a modified image. The tone curve can be expressed by a tone conversion function, and can be converted using this function. In FIG. 3A, the background, the color of the person's face, the color of the clothes, and the like are changed by this modification.

  FIG. 3B is a diagram illustrating modification by a transfer mold. In this modification, only an image (reference image) referred to as a target is used. This is because in the transfer type, since the correction is performed toward the reference image, only the reference image is required. This transfer mold can be modified using a known method such as color transfer (http://www.thegooch.org/Publications/PDFs/ColorTransfer.pdf). In FIG. 3B, the background, the face color of the person, and the color of the clothes are changed by this modification.

  FIG. 4A shows an example of standard type case information, and FIG. 4B shows an example of transfer type case information. The case information includes reference information at the time of modification that is referred to in order to specify the content of modification, and case identification information that can identify a case that has been modified according to the content of the modification. FIG. 4 shows an image processing operator and image processing parameters as reference information at the time of modification.

  The image processing operator represents a means of image processing. In FIG. 4A, the operator name “L tone curve” is shown, and in FIG. 4B, “transfer operator” is shown. The image processing parameter is a parameter used for the means. In FIG. 4A, the control point information used to create the tone curve is not shown, and in FIG. ing. In FIG. 4B, “Reference to image B” is described in parentheses following NULL, and the image B is used as a reference image.

  The case identification information may be any information as long as the case can be identified from other cases. In FIG. 4, the type that is the modification type, the case identifier (example ID), the shooting position, the shooting date and time, the image file name, the color tone information, the scene name of the shooting scene, the name of the object to be shot, and the impression representing the impression of the image Selection area information for selecting an area in an expression or image is shown. Therefore, the reference information at the time of modification can also be included in the case identification information if it can be identified from other cases.

  Since FIG. 4A is a standard type, the type is a standard type, information on latitude and longitude as the image capturing position, the name of the person in the image as the object name, and the person's name as impression information. Impression is shown. In addition, since the standard type requires two images as case images, the file names and color tone information of the images A and B are registered.

  Since FIG. 4B is a transfer mold, the type is a transfer mold, and the rest is the same as FIG. 4A. However, in the case of the transfer type, since only the reference image is required, the file name and color tone information of only the image B that is the reference image are registered. For this reason, “NULL” indicating that there is no information is input in another file name and color information column.

  Here, the case information is expressed in a table format, but in actuality, it can be realized according to a description method such as XML (eXtensible Markup Language). Since this is an example, other description methods may be adopted.

  The two case images collected in the standard case can be used as reference images for the two transfer types. The standard case information shown in FIG. 5A is the same as that shown in FIG. 4A, but the information of the image A and the information of the image B are used as reference information, and FIGS. ) Can be created. In FIG. 5, case information can be stored in a hierarchical manner with FIG. 5A as a root node and child nodes connected to the root node as FIGS. 5B and 5C. By doing so, it is possible to provide a framework capable of modifying both the standard type and the transfer type. Note that the arrows in FIGS. 5B and 5C indicate the same contents as the corresponding items in FIG.

  Next, in order to realize intuitive and simple usability, it is preferable to visually display the case in some form and give the user a choice. When the number of cases is small, the case information can be displayed side by side as they are.

  However, as the collection and registration of cases proceeds, there is a limit to the number of cases that can be displayed at one time even if they are displayed side by side as described above, so that all cases cannot be displayed at once. As shown in FIG. 6, there are several to several tens of case images and reference images as examples that can be displayed at one time, so the screen must be scrolled and displayed using a scroll bar or the like. When the display is not limited and is arranged at random, it is very difficult to find suitable case information.

  Therefore, the case identification information is not included in the case information, and the case information is appropriately displayed on the display unit, which is a display device, based on the case identification information (for example, according to the type and color tone information). It can be arranged and displayed. Thereby, the problem that it is difficult to find suitable case information can be solved.

  With reference to FIG. 7, the skin color of the face image will be described as an example. The case identification information includes selection area information for selecting a specific area such as a face area, and color tone information of the area selected by the selection area information. The color tone information is, for example, an average Lab value. L is a dimension indicating lightness, a and b are color dimensions, and are represented by coordinate values in a Lab color space such as (80, 10, 10) as illustrated in FIGS. 4 and 5. This color tone information may be calculated and registered at the time of registration, or may be calculated when necessary.

  In the transfer type, the correction is performed using the color tone information of the face image in the image B that is the case image. For this reason, by providing a rule that this color tone information is used for the standard type or allowing the user to select it, both can be arranged two-dimensionally regardless of the type of the case. The display may be an image format such as an icon or a thumbnail, or a language format such as a case ID or a shooting position. Incidentally, in FIG. 7, a list of case information is displayed in an image format.

  In addition to the list display of case information as shown in FIG. 7, the ab plane (two-dimensional) of the Lab space can be mapped as shown in FIG. 8, and each point can be displayed as each case. When the cursor is moved to an arbitrary point and stopped on the point, the case information may be displayed with an icon or the like. Further, the current input image may be shown as a star shape that can be distinguished from a point, in FIG. 8, a star shape, and it may be possible to more intuitively select a target. In this example, the skin color of the face image is taken as an example, but other scene names or object names may be used.

  When all the case information is displayed in a list, it is necessary to display only the number of cases. However, in many cases, case information is selected in the same category, so the list display is limited to that category. The number of case information to be reduced can be reduced. For example, by enabling items such as scene names and object names in the case identification information to be selected as categories, the number of case information to be displayed can be reduced. In FIG. 8, “sky”, “turf”, “flower”, and the like can be selected by tabs.

  In addition, when the type is selected, as shown in FIG. 9A, the standard type needs to display two case images for each case information, but the transfer type needs only one reference image. A lot of case information can be displayed. It is also possible to specify a target. As for the standard type, as shown in FIG. 9B, the two case images can be displayed alternately at regular time intervals instead of displaying them side by side. As a result, as in the case of the transfer mold, more case information can be displayed once.

  The image to be displayed may be only two case images for the standard type and only one reference image for the transfer type, but after being corrected according to the details of the correction in the case as shown in FIG. The modified image can also be displayed. As a result, the result can be specified. Even in this case, three images including the modified image can be displayed alternately for the standard type. In addition, regarding the transfer mold, two images including the modified image can be displayed alternately. This makes it possible to display more case information at a time.

  Although the outline has been described so far, the functional configuration for realizing the above-described modification and the processing performed by each functional unit will be described in detail below. FIG. 10 is a functional block diagram showing the first embodiment of the image processing apparatus 10. The image processing apparatus 10 includes, as function units, a control unit 30, an image acquisition unit 31, a case acquisition unit 32, an information storage unit 33, a case selection unit 34, a case application unit 35, and an image output unit 36. It is comprised including. The case application unit 35 includes a first format correction unit 37 and a second format correction unit 38.

  The control unit 30 controls the entire image processing apparatus 10. The image acquisition unit 31 receives an input of an image from the image reading device 12, the imaging device 13, the server device 11, and the like illustrated in FIG. 1, and acquires the image. Here, it is assumed that an image is acquired, but actually image data of the image is acquired. The image may be acquired by acquiring image data input from the imaging device 13 or the like, or the control unit 30 may request the server device 11 in response to an instruction from the user and acquire the read image data. it can.

  In response to the image acquisition unit 31 acquiring an image, the case acquisition unit 32 acquires a plurality of case information stored in the information storage unit 33 and passes it to the case selection unit 34. The case acquisition unit 32 can acquire all the case information stored in the information storage unit 33. When the user selects the above category, only the case information of the category can be acquired. For example, when the user selects a transfer mold as the type, only the case identification information including the transfer mold can be acquired.

  The case selection unit 34 arranges and displays the plurality of case information received from the case acquisition unit 32 on the display unit based on the case identification information included in each case information. And the case selection part 34 receives selection of the case information from a user. The case application unit 35 specifies the type of modification in the case, that is, the type, from the case identification information included in the case information selected by the user. If the type is a standard type, the first format correction unit 37 is requested for modification, and if the type is a transfer type, the second format correction unit 38 is requested for modification.

  When the first format correction unit 37 receives the request, the first format correction unit 37 uses the standard type for the image acquired by the image acquisition unit 31 according to the content of the modification specified by the modification-time reference information included in the case information selected by the user. Make corrections. In response to the request, the second format correction unit 38 uses the transfer type for the image acquired by the image acquisition unit 31 according to the content of the modification specified by the modification reference information included in the case information selected by the user. Make corrections.

  When the modification is completed and the modified image is obtained, the case application unit 35 passes it to the image output unit 36. The image output unit 36 displays the modified image on the display unit, prints and outputs the modified image, and further stores the modified image in the information storage unit 33. These outputs can be executed according to preset contents.

  These functional units are realized by the CPU 21 shown in FIG. 2 reading out and executing the image correction program stored in the auxiliary storage device 23 to the storage device 22. The information storage unit 33 can be a storage device included in the server device 11, but other devices such as the auxiliary storage device 23 included in the image processing device 10 and an external storage device connected to a cable or the network 15. It is also possible.

  With reference to the flowchart shown in FIG. 11, the process which corrects the image which the image processing apparatus 10 shown in FIG. 10 performs is demonstrated in detail. In step 1100, the image acquisition unit 31 starts receiving an image input from the imaging device 13 or the like. In step 1105, the image acquisition unit 31 acquires the input image. In step 1110, the case acquisition unit 32 acquires a plurality of case information from the information storage unit 33 in response to the image acquisition unit 31 acquiring an image. Here, description will be made on the assumption that all the case information stored in the information storage unit 33 is acquired.

  In step 1115, the case selection unit 34 arranges and displays the case information acquired by the case acquisition unit 32 on the display unit based on the case identification information included in each case information. The simplest arrangement example is an arrangement in which case information is arranged in the order of case IDs. Since the case IDs are composed of numerical values, alphabets, and the like, they can be arranged in order from the number 1 or in alphabetical order. In addition, it is also possible to arrange in order of shooting position and shooting date.

  The case selection unit 34 can specify items by user interaction, that is, dialogue with the user, and arrange the items in the order of the items. Alternatively, the case selection unit 34 can obtain and arrange the items in accordance with predetermined items such as color tone information. it can.

  The display method of the case information needs to be appropriately performed in consideration of the difference in the modification type. In the case of the standard type, two images before and after the modification (case image) are displayed side by side or alternately. In the case of the transfer type, only the reference image referred to as a target is displayed. As a result, target designation is possible.

  As for the display method of the case information, it is possible to display the modified image with respect to the image (input image) acquired by the image acquiring unit 31. In the case of the standard type, the modified image is included in the two case images. Three images are displayed side by side or alternately. In the case of the transfer type, two images including the modified image are displayed side by side or alternately. As a result, the result can be specified.

  In step 1115, the user selects suitable case information from the displayed case information, and the case selection unit 34 receives the case information selected by the user. In step 1120, in response to selection of the case information, the case application unit 35 first refers to the type in the case identification information and applies the type identification in order to apply the case information to the input image. I do. If the mold is a standard mold, the process proceeds to step 1125. If the mold is a transfer mold, the process proceeds to step 1130.

  In step 1125, the first type correction unit 37 performs correction using the standard type, and in step 1130, the second type correction unit 38 performs correction using the transfer type. The modification is performed by extracting and applying the image processing operator and the image processing parameter in the modification reference information included in the case information. For the image processing operator, tone curve, sharpness, edge enhancement and the like are specified, and for the image processing parameter, control point information used for them is specified. The first format correction unit 37 performs tone curve conversion if a tone curve is specified, and performs sharpness conversion if sharpness is specified. Since these conversions are already known, the description thereof will be omitted. Note that the number of image processing operators is not limited to one, and may be two or more.

  The above tone curve or the like is specified for the standard type, and a transfer operator is specified for the transfer type. At this time, a reference image is designated as the image processing parameter. The modification using the transfer mold can be performed using a known method such as the above-described color transfer. The first format correction unit 37 or the second format correction unit 38 passes the modified image as the modification result to the case application unit 35 when the image modification is completed.

  In step 1135, the image output unit 36 receives the modified image from the case application unit 35, displays the modified image on the display unit, and presents the modified image to the user. In step 1140, it is determined whether or not the presented modified image is acceptable. For example, if an OK button and a redo button are displayed and the user presses the OK button, it is determined that the modified image is acceptable, and if the redo button is pressed, it is determined that the modified image is not acceptable.

  If the result determined in step 1140 is a modified image, the process proceeds to step 1145, where the image output unit 36 stores the modified image in the information storage unit 33, and the process ends in step 1150. On the other hand, if the modified image is not good, the process returns to step 1115, the user selects the case information again, and the case selection unit 34 accepts the selection.

  In the embodiment described above, the modification by two molds is possible, but the modification is limited to only one mold and the modification of only that mold can be performed. In this case, the case identification information may not include the item of type, and the case application unit 35 does not need to make a determination to specify the type and only holds one of the correction units. Good.

  Next, another functional configuration of the image processing apparatus 10 and processing performed by each functional unit will be described. FIG. 12 is a functional block diagram showing the second embodiment of the image processing apparatus 10. Similar to the image processing apparatus 10 illustrated in FIG. 10, the control unit 30, the image acquisition unit 31, the case acquisition unit 32, the information storage unit 33, the case selection unit 34, the case application unit 35, and the image output unit. 36. The case application unit 35 includes a first format correction unit 37 and a second format correction unit 38. In the embodiment shown in FIG. 12, a region selection unit 40 is further provided. Since the same function unit such as the control unit 30 has already been described, only the region selection unit 40 will be described here.

  The area selection unit 40 is also realized by the CPU 21 executing a program, and selects an area to be modified. This area selection unit 40 limits the area to be modified in the input image.

  It is possible to modify the entire image based on the case information in both the standard type and the transfer type. For example, the standard type is suitable for simple correction such as bright and vivid colors, and the transfer type is suitable for a case where the entire image is similar as a scene. In the modification for the entire image, there is a case where such modification is preferable, but there is a case where it is not always possible to modify the intended image. For example, when it is desired to make only the face slightly brighter, when it is desired to suppress only the blue component, or when it is desired to emphasize only the high frequency component.

  When it is desired to correct only such a specific area, the area selection unit 40 can select the area to be corrected, and the case application unit 35 can correct the selected area. The selection of a region can be performed by using a known method of selecting by hue, a method of tracing a region boundary by a user, a method of tracing a region surface with a brush, face region extraction, fast Fourier transform (FFT), or the like. .

  More specifically, with reference to FIG. 13, an input image 41 is displayed in FIG. 13, and a tool box 42 is displayed on the left side. Various tools are displayed as icons in the tool box 42, and the automatic selection tool 43 is selected here. The automatic selection tool 43 is a tool that automatically selects a pixel at a position clicked by a mouse and an approximate color pixel adjacent thereto.

  In FIG. 13, since the pixel displaying the skin of the face is clicked, all the skin color pixels adjacent to it are extracted, and an area formed by all the extracted pixels is selected and displayed by a broken line. ing.

  The region selected by the region selection unit 40 is not limited to the region in the image space (plane). A specific channel of a color image, for example, a channel region of an image signal indicated by red, blue, or green may be selected, or a frequency region of an image signal having a specific band in a frequency space may be selected. The area selection unit 40 is not limited to selecting one of these three areas, and may select an area obtained by combining two or more of these three areas.

  With reference to the flowchart shown in FIG. 14, the process which corrects the image which the image processing apparatus 10 of this embodiment performs is demonstrated. This process starts from step 1400. Also in this case, as in FIG. 11, the image acquisition unit 31 starts receiving an image input from the imaging device 13 or the like. Steps 1405 and 1415 to 1455 are the same as steps 1105 to 1150 shown in FIG.

  In step 1410, the area selection unit 40 receives an area designation from the user and selects the designated area. The information of the selected area is sent to the case application unit 35, the area to be corrected is specified, and the correction is performed for the area.

  Still another functional configuration of the image processing apparatus 10 and processing performed by each functional unit will be described. FIG. 15 is a functional block diagram showing the third embodiment of the image processing apparatus 10. Similar to the image processing apparatus 10 illustrated in FIG. 10, the control unit 30, the image acquisition unit 31, the case acquisition unit 32, the information storage unit 33, the case selection unit 34, the case application unit 35, and the image output unit. 36. The case application unit 35 includes a first format correction unit 37 and a second format correction unit 38. In the embodiment shown in FIG. 15, a case registration unit 50 is further provided. Since the same functional unit such as the control unit 30 has already been described, only the case registration unit 50 will be described here. Note that the image processing apparatus 10 may include both the case registration unit 50 and the region selection unit 40 described above.

  The case registration unit 50 is realized by the CPU 21 executing a program, adds additional information to the modified image obtained by the image modification, and stores the new information in the information storage unit 33 for registration. The case registration unit 50 identifies the content corresponding to each item according to the expression format shown in FIG. 4 and registers it as one piece of case information. Therefore, the content corresponding to each item, excluding the modified image, is added information. As for the type, the type specified at the time of modification is inherited as it is.

  When the type is a standard type, the image processing operator and the image processing parameter referred to at the time of modification are designated as they are in the modification reference information. In the case identification information, the contents of the input image and the modified image are designated as the contents of the images A and B shown in FIG. That is, the file name and tone information of the input image and the modified image are designated. As other information, each information in the case information referred to at the time of modification is designated as a default value. For this reason, if correction is necessary, it is possible to correct the information of the portion to be corrected through user interaction.

  Even when the type is a transfer type, the image processing operator and the image processing parameter referred to at the time of modification are designated as they are in the modification reference information. In the case identification information, the content of the modified image is designated as the content of the image B. Other information is the same as the standard type.

  Thus, by creating as one new case information and adding it, the case information can be reused. In the case of the standard type, as shown in FIG. 5, since two case images are included, each of the two case images can be used as a reference image and registered as case information of two transfer types. Thereby, in addition to the standard type case information, the transfer type case information can also be added. Further, by adding the case information, it is possible to obtain a more desired modified image.

  With reference to the flowchart shown in FIG. 16, the process which corrects the image which the image processing apparatus 10 of this embodiment performs is demonstrated. This process starts from step 1600. Also in this case, as in FIG. 11, the image acquisition unit 31 starts receiving an image input from the imaging device 13 or the like. Steps 1605 to 1645 and 1655 are the same as steps 1105 to 1150 shown in FIG.

  In step 1650, the case registration unit 50 sets the modification performed on the input image as an additional case, stores the case information obtained in the case as additional case information in the information storage unit 33, and registers it. The registered case information is presented to the user and can be selected in the next image modification.

  Still another functional configuration of the image processing apparatus 10 and processing performed by each functional unit will be described. FIG. 17 is a functional block diagram showing the fourth embodiment of the image processing apparatus 10. Similar to the image processing apparatus 10 illustrated in FIG. 10, the control unit 30, the image acquisition unit 31, the case acquisition unit 32, the information storage unit 33, the case selection unit 34, the case application unit 35, and the image output unit. 36. The case application unit 35 includes a first format correction unit 37 and a second format correction unit 38. In the embodiment shown in FIG. 17, an image evaluation unit 60 is further provided. Since the same functional unit such as the control unit 30 has already been described, only the image evaluation unit 60 will be described here. Note that the image processing apparatus 10 may include an image evaluation unit 60, and one or both of the region selection unit 40 and the case registration unit 50.

  The image evaluation unit 60 is realized by the CPU 21 executing a program, and displays an index for evaluating the quality of the modified image on the display unit. If the person is not an expert, his / her eyes are not sufficient, and it is difficult to determine whether or not a desired image has been obtained simply by looking at the modified image. For this reason, it is desirable to provide the image evaluation part 60 and to display the evaluation index which can be evaluated objectively by the user.

  The image evaluation unit 60 uses at least one of image pair evaluation and single evaluation in order to provide an evaluation index. With reference to FIG. 18, the evaluation index displayed by the image evaluation unit 60 will be described. The image shown in FIG. 18 is an example of an evaluation index from which the left side is an input image, a modified image, and a Ch difference (lightness L) image, and the Ch difference image is obtained by image pair evaluation. On the lower side of the Ch difference image, ΔE (color difference) is shown as a numerical value. This ΔE is also an example of an evaluation index obtained by image pair evaluation. In short, the difference between single evaluation and image pair evaluation is the difference between obtaining an evaluation index from one image or from two images.

  As examples of evaluation indices obtained by image pair evaluation, S / N (signal to noise ratio), MSE (mean square error), ΔE, and the like are known. With these evaluation indexes, it is possible to objectively understand how much the signal level changes. In addition, it is possible to introduce an evaluation index called SSIM, which has a higher degree of matching with the subjectivity than S / N or MSE.

  As an example of an evaluation index obtained by a single evaluation, a histogram for each lightness or color channel is known. This histogram is, for example, a bar graph showing how many R, G, and B pixel values constituting an image are used. Therefore, by looking at this histogram, the user can objectively understand the tone of the entire image, and can make an appropriate determination. Factors for evaluating image quality include brightness, etc., contrast, sharpness, sharpness, graininess, and the like, and evaluation indexes for these can be introduced.

  As an example of the index obtained by the single evaluation, there can be cited a Salinity Map (attractiveness map). This is a method of expressing the degree of conspicuousness in terms of human visual characteristics in an image-wise manner using pixel values. According to this map, it is possible to obtain a guideline as to which hit should be evaluated. These examples are merely examples, and other evaluation indexes may be added.

  Examples of the image or image pair to be evaluated include a modified image, a modified image and an input image, a reference image, or a case image. The image or image pair to be evaluated may be another input image, a reference image, an input image and a reference image, or the like. Among these, the image pair evaluation of the input image and the reference image or the case image is a basis for expanding in the future to arrange more suitable case information. The evaluation of the modified image is used as objective information for judging the obtained result. By this evaluation, the user can make the evaluation with peace of mind, and at the same time, it is possible to improve the skill level necessary for image modification.

  With reference to the flowchart shown in FIG. 19, the process which corrects the image which the image processing apparatus 10 of this embodiment performs is demonstrated. This process starts from step 1900. Also in this case, as in FIG. 11, the image acquisition unit 31 starts receiving an image input from the imaging device 13 or the like. Steps 1905 to 1935 and steps 1945 to 1655 are the same as steps 1105 to 1150 shown in FIG.

  In step 1940, the image evaluation unit 60 displays an evaluation index for allowing the user to evaluate the quality of the modified image. The image evaluation unit 60 displays the quality of the modified image on the display unit in order to generate an index called the above histogram using information of the modified image, for example, RGB pixel values, and present the histogram to the user. Further, the image evaluation unit 60 uses the modified image and the input image, the reference image, or the case image information to obtain the above S / N and the like, and presents the numerical value as an index to the user. Display.

  As described above, by arranging and displaying case information based on case identification information, it is intuitive and easy to use, and it can reach the desired result with less man-hours. By enabling selection, the degree of freedom in processing can be increased. In addition, since the case identification information includes color tone information, scene name, and the like and is appropriately arranged based on the information, intuitive target designation is possible. It is possible to display cases without depending on the type, and this makes it possible to intuitively specify a target.

  By displaying the modified image, it is possible to specify an intuitive result. By selecting a region, it is possible to limit the processing target and enhance the effect of image modification. In addition, the degree of processing freedom can be further increased by enabling registration as an additional case. In addition, if a certain case is registered in the standard type, it can be used as a transfer type case from the information, and this can accelerate the collection and accumulation of the transfer type case.

  By displaying the evaluation index for image evaluation, it is possible to provide objective judgment materials, so that the user can make evaluations with more peace of mind and improve the skill level required for image modification. .

  So far, the image processing apparatus, the image processing method, and the processing executed by the program of the present invention have been described in detail with reference to the embodiments shown in the drawings, but the present invention is limited to the above-described embodiments. is not. Therefore, other embodiments, additions, changes, deletions, and the like can be changed within a range that can be conceived by those skilled in the art, and as long as the effects and advantages of the present invention are exhibited in any aspect, the present invention It is included in the range. Therefore, the present invention can also provide an image processing system including an image processing apparatus and a recording medium on which the above-described program to be executed by a computer is recorded.

DESCRIPTION OF SYMBOLS 10 ... Image processing apparatus, 11 ... Server apparatus, 12 ... Image reading apparatus, 13 ... Imaging apparatus, 14 ... Cable, 15 ... Network, 20 ... Input I / F, 21 ... CPU, 22 ... Storage device, 23 ... Auxiliary storage Device 24 ... Output I / F 25 ... Storage medium 26 ... Drive 27 ... Controller 30 ... Control unit 31 ... Image acquisition unit 32 ... Case acquisition unit 33 ... Information storage unit 34 ... Case selection unit 35 ... Case application unit, 36 ... Image output unit, 37 ... First format correction unit, 38 ... Second format correction unit, 40 ... Region selection unit, 41 ... Input image, 42 ... Tool box, 43 ... Automatic selection tool 50 ... Case registration unit, 60 ... Image evaluation unit

JP 2007-221591 A JP 2004-129226 A Japanese Patent No. 4646735 JP 2008-294969 A Japanese Patent No. 4910948 JP 2002-171408 A Japanese Patent No. 4421176 JP 2006-080746 A Japanese Patent No. 4985243

Claims (9)

An image processing system for modifying an image,
An information storage unit that stores a plurality of case information including reference information for modification that is referred to in order to identify the content of modification, and case identification information that can identify a case that has been modified according to the content of the modification,
An image acquisition unit that receives an input of an image and acquires the image;
A plurality of case information stored in the information storage unit, arranged and displayed on a display unit based on the case identification information included in each of the case information, and a case selection unit for receiving selection of case information;
A case application unit that modifies the image acquired by the image acquisition unit based on the selected case information;
The case selection unit displays one image referred to as a target for each case information, accepts designation of one target image,
The image processing system in which the case application unit corrects the specified one image.   The case identification information includes a case identifier assigned to the case, a shooting position of an image used in the case, a shooting date and time, an image file name, color tone information, a scene name of a shooting scene, a target name of a shooting target, The image processing system according to claim 1, wherein the image processing system includes at least one of an impression expression representing an impression of an image and selection area information when an area in the image is selected.   The image processing system according to claim 1, wherein the case selection unit displays the images modified by the case application unit side by side or alternately with the designated one image.   At least one specified area is received by accepting at least one designation of an area of an image space, a channel area of an image signal, and a frequency area of an image signal as an area for modifying the image acquired by the image acquisition unit. The image processing system according to claim 1, further comprising an area selection unit that selects the image.   Further included a case registration unit that stores the case information obtained in the modification as additional case information, with the modification performed on the image acquired by the image acquisition unit as an additional case. The image processing system according to claim 1.   In order to allow the user to evaluate the quality of the image modified by the case application unit, the modified image information, the modified image information and the image acquired by the image acquisition unit or the one image The image processing system according to claim 1, further comprising an image evaluation unit that generates an evaluation index using information on the image of the image and displays the evaluation index on a display unit. An image processing method executed by an image processing system that performs image modification, wherein the image processing system performs modification according to modification reference information that is referred to in order to identify the contents of modification and the contents of the modification. An information storage unit for storing a plurality of case information including:
Receiving an input of an image and obtaining the image;
Arranging and displaying a plurality of case information stored in the information storage unit on a display unit based on the case identification information included in each case information;
Receiving one of the plurality of case information displayed on the display unit;
Revising the image obtained in the obtaining step based on the selected case information,
In the step of accepting the selection, one image referred to as a target is displayed for each case information, and designation of a target image is accepted.
The image processing method, wherein in the modifying step, the image is modified toward the designated one image. At least one specified area is received by accepting at least one designation of an area of an image space, a channel area of an image signal, and a frequency area of an image signal as an area to be modified with respect to the image obtained in the obtaining step. A step of selecting
A step of modifying the image acquired in the step of acquiring as an additional case, and storing the case information obtained in the correction as additional case information in the information storage unit When,
In order to allow the user to evaluate the quality of the image modified in the modifying step, information on the modified image, or information on the modified image and information on the acquired image or the one image, The image processing method according to claim 7, further comprising: generating an evaluation index using, and displaying the evaluation index on a display unit. A program for causing a computer to execute the image processing method according to claim 7 or 8.