JP2010056628A - Color reproduction system and color reproduction processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve image processing with excellent color reproducibility without forcing a user to perform a complicated processing procedure. <P>SOLUTION: Spectral reflectance information obtained by premeasuring spectral reflectance distribution on a surface of an anticipated photographing object which is an object anticipated to be a photographing object is stored in a database 140. A processing apparatus 120 acquires the spectral reflectance information obtained by retrieval under a retrieving condition set by a user from the database 140, accepts user's setting of a reference area to be an area for referring colors of an object image included in a color image in color reproduction processing, and performs the color reproduction processing of the color image by using the spectral reflectance information acquired from the database 140. In this case, a color reproduction processing part 308 performs the color reproduction processing of the color image based on image data in the reference area and spectral reflectance information corresponding to the reference area out of the obtained spectral reflectance information. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a color reproduction system and a color reproduction processing method for reproducing the color of a subject based on a color image obtained by photographing.

  For the purpose of acquiring an accurate colorimetric value of a subject as an image, a method of performing color reproduction by estimating the spectral reflectance of a subject from an image taken by a multiband camera has been proposed (Patent Document 1). . Here, when estimating the spectral reflectance of the subject from the image obtained by the multiband camera, the estimation is performed using the spectral sensitivity characteristics of each band of the camera and the spectral statistical information of the subject. In addition, the spectrum of light that illuminates the subject at the time of photographing (hereinafter referred to as photographing illumination light) is measured at the time of photographing and is used when estimating the spectral reflectance of the subject.

In addition, a method has been proposed in which a multicolor color chart having a known spectral reflectance is photographed together with a subject and the spectral reflectance of the subject is estimated (Patent Document 2). Furthermore, there is a method for obtaining the spectral reflectance of a subject whose spectral reflectance is unknown or estimating the color by photographing a subject whose spectral reflectance is unknown and a color chart whose spectral reflectance is known under the same conditions. It has been proposed (Patent Document 3).
Japanese Patent Application Laid-Open No. 11-85952 JP 11-96333 A Japanese Patent No. 3960989

  In the technique disclosed in Patent Document 1, it is necessary to measure the spectrum of photographing illumination light. In addition, a statistical property of the spectrum of the photographed subject (hereinafter referred to as subject property) is necessary when estimating the spectral reflectance of the subject. If a spectrum measuring unit capable of measuring the spectrum of photographic illumination light is provided in the imaging device, it is inevitable to increase the size and price of the imaging device. Also, the light that illuminates the subject at the time of shooting is not necessarily one type. For example, there may be a situation in which some of the plurality of subjects existing in the shooting screen are illuminated with tungsten light and the other part is illuminated with a fluorescent lamp. In this case, it is necessary to associate the subject in the screen with the photographing illumination light, such as which subject is with which photographing illumination light, and the photographing operation becomes complicated.

  In the techniques disclosed in Patent Documents 2 and 3, it is necessary to copy the color target together with the subject, and there is a possibility of hindering rapid shooting. Also, when the subject is a distant landscape or the like, accurate color reproduction is not always possible if the light illuminating the subject differs from the light illuminating a color target near the photographing apparatus.

  Further, there is photo retouching software as software for adjusting the color tone, contrast, sharpness, etc. of an image to adjust the image to his / her preference. By using photo retouching software, the user can freely adjust the color tone, contrast, sharpness, etc. above, but much knowledge about retouching is required to adjust the image to his favorite color tone and gradation. And experience is necessary. For this reason, it is not easy for a beginner to adjust an image to his or her favorite color tone or gradation, and even if the user thinks that the image has been adjusted well, it may not always lead to accurate color reproduction. It was.

  An object of the present invention is to provide means for solving the above-described problems, and it is not necessary to photograph a color target together with a subject, and a technique that enables easy and excellent image processing with excellent color reproducibility. The purpose is to provide.

(1) According to the first aspect of the present invention, the spectral reflectance information obtained by measuring in advance the spectral reflectance distribution on the surface of the predicted imaging target that is an object that is expected to be the imaging target, respectively. Applied to a color reproduction system having a database accumulated and formed for each expected photographing object and an image data processing device for performing color reproduction processing of a color image obtained by photographing,
The image data processing device includes:
A spectral reflectance information acquisition unit that acquires spectral reflectance information obtained by searching under a search condition set by a user from the database;
A reference area setting unit for accepting the user to set a reference area that is an area for referring to the color of the subject image in the color image during the color reproduction process;
A color reproduction processing unit that performs color reproduction processing of the color image using the spectral reflectance information, the spectral reflection corresponding to the reference region in the image data in the reference region and the spectral reflectance information The above-described problem is solved by including a color reproduction processing unit that performs color reproduction processing of the color image based on rate information.
(2) The second aspect of the present invention is applied to the color reproduction processing method, and is obtained by measuring in advance the spectral reflectance distribution on the surface of an expected photographing object that is an object expected to be photographed. Obtaining spectral reflectance information obtained by searching with a search condition set by a user from a database formed by accumulating the obtained spectral reflectance information for each predicted photographing object;
Performing color reproduction processing of a color image obtained by photographing, and in the color reproduction processing, a reference area set by the user as an area for referring to a color of a subject image reflected in the color image. Performing color reproduction processing of the color image based on image data and spectral reflectance corresponding to the reference region in the spectral reflectance information.

  The present invention includes a database and an image data processing device, and the database includes a spectral spectrum obtained by measuring in advance a spectral reflectance distribution on the surface of an expected imaging target that is an object that is expected to be an imaging target. Reflectance information is accumulated for each predicted subject. An image data processing apparatus acquires spectral reflectance information obtained by searching under a search condition set by a user from a database, and refers to a color of a subject image that appears in a color image during color reproduction processing It is accepted that the user sets a reference area. Then, the image data processing device uses the spectral reflectance information acquired from the database, and based on the image data in the reference area and the spectral reflectance information corresponding to the reference area in the spectral reflectance information, the color of the color image By performing the reproduction process, it is possible to obtain a more accurate color reproduction process result with fewer work procedures without measuring the illuminating illumination light or imprinting the color target at the time of photographing.

  FIG. 1 is a diagram showing a schematic configuration of a color reproduction system 100 according to an embodiment of the present invention. The color reproduction system 100 includes an image data processing apparatus (hereinafter referred to as a processing apparatus) 120 as a client, and a server having a database system 140 therein. Hereinafter, the database 140 is simply referred to. The processing device 120 and the database 140 are connected via a network 160 such as a LAN or a WAN. Alternatively, the database 140 may be provided inside the processing device 120. In FIG. 1, one database 140 and one processing device 120 are shown, but a plurality of processing devices 120 may be connected. Similarly, a plurality of databases may exist. The processing device 120 is not limited to a personal computer to which the monitor display device 122 is connected, but may be a set-top box, an image playback device, an image recording / playback device, or the like connected to a television receiver or a monitor display device.

  In the database 140, spectral reflectance information obtained by measuring in advance the spectral reflectance distribution on the surface of the expected imaging object that is an object that is expected to be the imaging object is accumulated for each expected imaging object. Has been formed. The anticipated object to be photographed can include, for example, landscapes, buildings, remains, artworks, animals and plants, etc. existing all over the world. Alternatively, it is possible to include products, creations, samples, products, etc. in all fields such as academic, art, medicine, and industry.

  The spectral reflectance information is information obtained by measuring in advance the spectral reflectance distribution on the surface of the predicted object. As a method for measuring the spectral reflectance distribution, there is a method using a spot-type colorimeter (hereinafter referred to as a spot colorimeter) having a relatively small measurement viewing angle. In this method, the spectral reflectance at a plurality of different points on the surface of the predicted object is measured using a spot colorimeter. At this time, the information on the distribution of the spectral reflectance on the surface of the anticipated subject is obtained by dividing the surface of the subject to be photographed into a grid and measuring the spectral reflectance at a position corresponding to each intersection of the lattice. Can be obtained. The measurement interval on the surface of the expected object to be photographed may be uniform or unequal. Hereinafter, the spectral reflectance information obtained in this way is referred to as “discrete measurement spectral reflectance information” in the present specification. The discrete measurement spectral reflectance information can include spectral reflectance information of a representative part on the surface of the predicted imaging target.

  Another method for measuring the spectral reflectance distribution is to obtain two-dimensional image data. This is a method of obtaining spectral reflectance image data using a multispectral camera or the like. As an example of a multispectral camera, there is a camera having a turret switching type color filter in front of or inside a photographing lens of a camera having a monochrome imaging device. The color filter can have 16 colors, for example, and the spectral transmittance of these color filters is measured in advance.

  By repeating photographing while switching these color filters, the image data for 16 colors obtained for the same subject is processed to obtain multiband image data. At this time, the spectral characteristic of light (hereinafter, referred to as photographing illumination light) that illuminates the subject at the time of photographing is also measured. By processing the multiband image data based on the spectral characteristics of the photographing illumination light, the camera tone characteristics, the bias characteristics, and the like, it is possible to obtain spectral reflectance image data of the subject. This spectral reflectance image data is also spectral reflectance information. Hereinafter, in the present specification, the two-dimensional spectral reflectance information obtained as described above is referred to as “continuous measurement spectral reflectance information”. According to the continuous measurement spectral reflectance information, it is possible to obtain spectral reflectance information of an arbitrary part on the surface of the predicted imaging target.

  Considering the storage capacity required to accumulate spectral reflectance information in the database 140, the storage capacity required to store discrete measurement spectral reflectance information is necessary to store continuous measurement spectral reflectance information. It has the advantage that it can be made smaller than the storage capacity. On the other hand, the continuous measurement spectral reflectance information has an advantage that spectral reflectance information can be obtained at an arbitrary position on the surface of the predicted object.

  It is desirable that a plurality of sets of spectral reflectance information corresponding to one expected photographing object are included in the database 140. For example, if the object to be photographed is something like a building, different weather (humidity), surface is dry, wet with rain, measured surface (east, west, south, The spectral reflectance information obtained by measuring corresponding to the situation where the spectral reflectance changes, such as the north surface), the measurement direction (for example, the east surface is inclined 45 degrees to the right), and the like is accumulated in the database 140. It is desirable.

  In addition, if the predicted object is a natural landscape, spectral reflectance information obtained by measuring corresponding to different spectral reflectances such as different weather, different seasons, measurement directions, etc. is stored in the database 140. It is desirable to have accumulated. This is because, if the measurement conditions are different, the spectral reflectance of the surface of the object to be photographed may be different.

  As described above, assuming the season, time, weather conditions, camera angle, and the like at the time of shooting by the photographer in advance, spectral reflectance information is obtained under conditions corresponding to those shooting conditions and camera angle, and stored in the database 140. In order to make the color reproduction process, which will be described later, more accurate, it is desirable to record in the above.

  FIG. 2 is a diagram conceptually illustrating a state in which the above-described spectral reflectance information is stored together with corresponding search information in the database 140. Search information such as name 202, color name 204, spectral reflectance measurement condition 206, measurement target information 208, keyword 210, and the like is associated with the spectral reflectance information 200 of an expected photographing object and stored in the database 140. .

  As an example, the case where the spectral reflectance information 200 is obtained by measuring the eastern surface of the Arc de Triomphe in Paris, Charles de Gaulle will be described as an example. The name 202 stores information such as “East of Paris Arc de Triomphe”. The color name 204 stores color name information indicating the color of the Arc de Triomphe, such as “cream color” and “beige”. The spectral reflectance measurement condition 206 includes information related to environmental conditions when spectral reflectance is measured, such as “rainy weather”, “sunny weather”, “winter”, “summer”, “high humidity”, “low humidity”, etc. Is stored.

  When the expected object to be photographed is an old building or the like and has been repaired in the past, information on the date of measurement of the spectral reflectance may be included in the spectral reflectance measurement condition 206. By doing so, it becomes possible to know whether the spectral reflectance information is before the modification or after the modification. The measurement object information 208 stores visual information such as “building” and “gate”. The keyword 210 includes the name of the place where the object to be photographed is located, such as “France”, “Paris”, “Charles de Gaulle”, “Etoile”, “Arcade of Arcs”, and the name of the object to be photographed. Search information for allowing the user to easily find the spectral reflectance information required by the user, such as the above information, is stored.

  FIG. 3 is a block diagram schematically illustrating the internal configuration of the processing device 120. The processing device 120 includes a search condition setting unit 302, a color image storage unit 304, a spectral reflectance information acquisition unit 306, a color reproduction processing unit 308, a reference region setting unit 310, an evaluation region setting unit 312, and a standard. An information creation unit 314, an image evaluation unit 316, a processed image storage unit 318, a color reproduction information setting unit 320, an image display processing unit 322, and an evaluation result display processing unit 324 are included.

  FIG. 4 is a block diagram schematically illustrating the internal configuration of the color reproduction processing unit 308. The color reproduction processing unit 308 includes an image cutout unit 402, an average processing unit 404, a color conversion matrix creation unit 406, and a color conversion calculation unit 408.

  Hereinafter, the contents of the color reproduction process performed by the processing device 120 will be described first with reference to FIGS. 5 to 13, and then the configuration of the processing device 120 will be described with reference to FIGS. 3 and 4.

  FIG. 5 illustrates the processing device 120 when the user operates the processing device 120 to select image data to be subjected to color reproduction processing (hereinafter referred to as processing target image data) and perform a color reproduction processing start operation. An example of display on the monitor display device 122 connected to is shown. Here, the processing target image data is color image data obtained by photographing. This processing target image data may be obtained by a user using a digital camera or the like, or may be recorded on a storage medium such as an optical disk and distributed. Or what was acquired via the internet may be used. This image data is preferably multiband image data obtained by a so-called multispectral camera in order to perform more accurate color reproduction processing. Alternatively, even BGR image data generated by a conventional digital camera having a BGR primary color filter or a YCM complementary color filter can be processed image data, although the accuracy is slightly reduced. is there.

  In the following description, it is assumed that the processing target image data is multiband image data. The multi-band image data will be described as including information on the tone / bias characteristics and spectral sensitivity characteristics of the camera (hereinafter referred to as input profiles).

  As shown in FIG. 5, an image 5a based on the processing target image data (hereinafter referred to as processing target image 5a) is displayed on the screen. In this processing target image 5a, the spectral reflectance is known in addition to the image 5b of the subject whose spectral reflectance is likely to be known (spectral reflectance information is highly likely to be recorded in the database 140). It is also possible to include an image 5c of a subject that is unlikely to be.

  Also displayed on the screen are frames 5d, 5e, 5f, 5g, and 5h for accepting a user to input a search condition for searching for spectral reflectance information from the database 140. The user operates a pointing device such as a computer mouse and a keyboard to input search conditions in these frames. For example, “Triumph Arch” in frame 5d, “East Arc de Triomphe” in frame 5e, “Spring Sunny” in frame 5f, “Paris Charles de Gaulle” in frame 5g, “Beige” in frame 5h, etc. And click the “start search” button 5k.

  Although it is not always necessary to input information in all of the frames 5d to 5h, it is possible to obtain a more accurate search result of spectral reflectance information by setting search conditions in detail.

  In response to the user clicking the “search start” button 5k, a search result display window 5j is displayed on the screen. In the search result display window 5j, the spectral reflectance information search results (spectral reflectance information selection candidates) corresponding to the search conditions input in the frames 5d to 5h are displayed.

  If an appropriate selection cannot be found among the selection candidates of the spectral reflectance information displayed in the search result display window 5j, or if there are too many selection candidates, the user newly sets the search conditions and performs “start search”. The button 5k can be clicked again.

  By the way, when the image input device has a GPS sensor, a geomagnetic sensor, a posture sensor, etc., the positioning information and direction information obtained by measuring at the time of shooting and the camera direction (upward, horizontal, downward, etc.) are displayed as images. It can be stored in tag information added to the data. The processing device 120 can grasp from which position on the earth, from which direction and in which direction the image is taken by reading the information. By adding search information relating to the shooting position, shooting direction, and orientation to the spectral reflectance information stored in the database 140, it is possible to reduce the time and effort for the user to input search conditions.

  FIG. 6 shows a display immediately after the user operates the pointing device and moves the cursor over the spectral reflectance information candidate 2. As shown in FIG. 6, when the user performs a click operation with the frame 6 a surrounding the spectral reflectance information (candidate 2) displayed, the spectral reflectance information corresponding to the selected candidate 2 is acquired from the database 140. Is done. It is desirable to attach sample image data to the spectral reflectance information, and a sample image 7a based on the sample image data can be displayed as shown in FIG. The sample image 7a does not necessarily include the spectral reflectance information of the candidate 2 and is an image that allows the user to intuitively understand which surface of the predicted photographing target includes spectral reflectance information. Is possible. For example, when the spectral reflectance is measured, an image obtained by photographing with a normal BGR 3 primary color digital camera from the position where the spectral reflectance is measured may be displayed as the sample image 7a, or an illustration may be displayed. Also good. Alternatively, when the spectral reflectance information is continuous measurement spectral reflectance information, the sample image 7a can be generated and displayed based on this information.

  The user looks at the sample image 7a and determines whether or not it includes spectral reflectance information appropriate for use in color reproduction processing to be performed in the future. For example, by comparing the subject image 5b in the processing target image 5a with the sample image 7a and determining that the shooting situation is similar and the shooting direction is substantially the same, the user can select “reference area”. Click the “Set” button 7b. On the other hand, if the user views the sample image 7a and determines that it is not suitable for use in the color reproduction process, the user clicks the “return” button 7c or the “re-search” button 7d. When the “return” button 7c is clicked, the display returns to that shown in FIG. When the “re-search” button 7d is clicked, the display returns to the display shown in FIG.

  When the user clicks the “reference area setting” button 7b shown in FIG. 7, the display shown in FIG. 8 is made. The user operates a pointing device, a keyboard, or the like to set a partial area (hereinafter referred to as a reference area) 8a in the processing target image 5a that is referred to when the processing device 120 performs color reproduction processing described later. Subsequently, the user operates the pointing device, the keyboard, or the like to set the reference area 8b in the sample image 7a. Regarding the setting of the reference region 8b, instead of being performed by the user, the processing device 120 can automatically set the reference region 8b in the sample image 7a corresponding to the reference region 8a using image recognition technology. It may be configured.

  As the reference area 8a, if a portion with a relatively good illumination condition, such as a non-shadow portion in the subject image 5b, and a lightness is moderate and there is little noise, a good result of color reproduction processing can be obtained. Can do. The processing device 120 can analyze the image data in the reference area 8a as to whether or not the reference area 8a selected by the user is a preferable area. Based on the analysis result, the processing device 120 may be configured to display whether or not the reference area 8a selected by the user is a preferable area. Further, the shape and size of the reference areas 8a and 8b may be configured to be arbitrarily selectable by the user.

  By the way, when the spectral reflectance information is discrete measurement spectral reflectance information, there is a possibility that there is no spectral reflectance information of a portion corresponding to the reference region 8a selected by the user. In that case, it is possible to use the spectral reflectance information of the portion closest to the portion corresponding to the reference region 8a selected by the user. Alternatively, the processing device 120 can notify the user that there is no spectral reflectance information of a portion corresponding to the reference region 8a selected by the user. In addition, a portion where spectral reflectance information exists can be displayed in the sample image 7a. The user can find a settable reference area in the processing target image 5a by looking at the sample image 7a. When the spectral reflectance information is continuous measurement spectral reflectance information, since there is no restriction as described above, the degree of freedom in setting the reference areas 8a and 8b is increased.

  When the setting of the reference areas 8a and 8b described above is completed and the user clicks the “evaluation area setting” button 8c, the display shown in FIG. 9 is made. The user operates a pointing device, a keyboard, or the like to set a partial area (hereinafter referred to as an evaluation area) 9a in the processing target image 5a that is referred to when the processing apparatus 120 evaluates color reproduction processing described later. . Subsequently, the user operates the pointing device, the keyboard, or the like to set the evaluation area 9b in the sample image 7a.

  Regarding the setting of the evaluation region 9b, instead of being performed by the user, the processing device 120 can automatically set the evaluation region 9b in the sample image 7a corresponding to the evaluation region 9a using an image recognition technique. It may be configured. Also, the shape and size of the evaluation areas 9a and 9b may be configured to be arbitrarily selectable by the user as in the reference areas 8a and 8b.

  The evaluation regions 9a and 9b are preferably conspicuous portions in the processing target image 5a and portions that the user desires to perform more accurate color reproduction, in order to obtain better processing results.

  The user who has completed the above operation clicks the “color reproduction process start” button 9c. The processing device 120 applies the spectral reflectance of the reference region 8b as the spectral reflectance of the reference region 8a. That is, the spectral reflectance information of the part corresponding to the reference region 8b is extracted from the spectral reflectance information, and is applied as the spectral reflectance of the reference region 8a. Further, the spectral characteristic of the photographic illumination light is estimated based on the input profile included in the processing target image data. Since details of such processing are disclosed in Patent Document 3, the description thereof is omitted here.

  Subsequently, the processing device 120 performs processing for estimating the spectral reflectance of a portion other than the reference region 8a in the processing target image data based on the spectral characteristics of the photographing illumination light estimated as described above and the input profile. I do. At this time, the spectral reflectance of the subject image 5c in which the spectral reflectance information is not accumulated in the database 140 is also determined based on the spectral characteristics of the photographing illumination light estimated as described above and the input profile. Is performed. As described above, spectral reflectance estimation image data is generated from the processing target image data.

  As is clear from the above description, in the color reproduction system according to the embodiment of the present invention, it is not necessary to copy a color target with a known spectral reflectance together with the subject. In other words, by storing in the database 140 the spectral reflectance distribution on the surface of an expected object to be photographed that exists in the world, a subject image based on the data in the processing target image can be used as a color target. As a result, it is possible for a subject with a known spectral reflectance to be reproduced, and for a subject with an unknown spectral reflectance reflected together with a subject with a known spectral reflectance, more accurate color reproduction can be performed.

  By the way, the spectral reflectance estimation image data obtained as described above does not exhibit a color as it is because the influence of photographing illumination light is removed. That is, the object placed in the darkness is not seen and the color cannot be distinguished. Therefore, as if illuminating an object placed in the dark, the spectral reflectance estimation image data obtained by performing the above-described processing is performed to reproduce the color when illuminated with illumination light having a certain spectral characteristic, The processing device 120 performs a display process. In this specification, the illumination light applied to the spectral reflectance estimation image data is referred to as observation illumination light. Further, performing the process of applying the observation illumination light to the spectral reflectance estimation image data is referred to as rendering.

  The processing device 120 performs rendering processing on the spectral reflectance estimated image data, and further performs processing for applying characteristics of the monitor display device 122 connected to the processing device 120 (hereinafter referred to as monitor display characteristics). Further, the color corresponding to the specifications such as whether the monitor display device 122 is a BGR three-primary color display device, a multi-primary color display device, or a colorimetric value input display, that is, the display color space of the monitor display device 122. A reproduction process is performed and displayed on the monitor display device 122.

  The color reproduction process is performed as described above, and an image 10a after the color reproduction process (hereinafter referred to as a processed image) 10a is displayed as shown in FIG. The user can look at the processed image 10a and, if satisfied, click the “save” button 10c as necessary to save the image data after the color reproduction process. The user can also refer to the evaluation result of the processed image data by clicking the “evaluation” button 10b. This evaluation result refers to the image data of the evaluation area 9b in the spectral reflectance information (corresponding to the sample image 7a) and the image data of the evaluation area 9a in the processed image data (corresponding to the processed image 10a). It means the result of comparison.

  FIG. 11 is a diagram illustrating a state where the result of comparing the image data of the evaluation areas 9a and 9b with the spectral reflectance is displayed as the graph 11a when the user clicks the “evaluation” button 10b. The spectral reflectance graph displayed at this time is obtained by averaging the spectral reflectances corresponding to the respective pixels in the evaluation region 9a and the spectral reflectance corresponding to the respective pixels in the evaluation region 9b. It can display based on what was obtained on average.

  Intuitively how much the spectral reflectance estimated image data generated from the processing target image data matches the spectral reflectance information retrieved from the database 140 by the user referring to the evaluation result, that is, the graph 11a. It becomes possible to grasp. In particular, when the processed image 10a is not satisfactory, it is possible to know how the spectral reflectances deviate, so that it can be used as a reference when resetting the reference areas 8a and 8b. it can.

  As a display form of the evaluation results, in addition to the graph 11a for comparing the spectral reflectances shown in FIG. 11, a table 12a as shown in FIG. 12 and a radar chart 13a shown in FIG. In the example shown in FIGS. 12 and 13, the evaluation result in the L * a * b * color space is displayed. That is, a CIE standard light source (for example, D65) is applied to both the spectral reflectance estimation image data in the evaluation area 9a and the spectral reflectance information in the evaluation area 9b, and converted into image data in the CIE L * a * b * color space. Evaluation is performed.

  In the example shown in FIG. 12, the coordinate values (L1, L2, a1, a2, b1, b2) of L *, a *, b * and the difference between them (L2-L1, a1-a2, b2-b2) Are displayed in a table. When a user who has knowledge about the color space of L * a * b * looks at the evaluation results as shown in FIG. 12, the reason why the processed image 10a is not satisfactory is because the brightness is different. It is possible to know (difference between L1 and L2), because the hue and saturation are different (difference between a1 and a2 and difference between b1 and b2).

  In the example shown in FIG. 13, it is possible to intuitively grasp the color difference between colors based on the image data in each of the evaluation areas 9a and 9b.

  14 calculates the sum of the spectral reflectance differences between the processed image and the reference image shown in FIG. 11 or the sum of squares of the differences (L2-L1, a1-a2, b1-b2) shown in FIG. In this example, the size is displayed. A case where the difference value is large is displayed as “bad”, a case where the difference value is small is displayed as “good”, and the middle is displayed as “normal”. In the display form of the evaluation result shown in FIG. 12, even a user who does not have deep knowledge about image processing or a user who does not understand the concept of color space can grasp the evaluation result intuitively. It becomes.

  11 to 14 described above may be displayed by sequentially switching, for example, when the user clicks a button for switching the evaluation result display method. The user who sees the evaluation result described above can click the “reset evaluation area” button 11b or the “re-search” button 10e (FIG. 11) as necessary. Alternatively, all of the evaluation results illustrated in FIGS. 11 to 14 may be displayed on one display screen.

  When the evaluation result is not satisfactory, the user can perform the evaluation area setting operation described with reference to FIGS. 8 and 9 again by clicking the “reset evaluation area” button 11b. Alternatively, by clicking the “re-search” button 7d, it is possible to go back directly to the processing step for searching for spectral reflectance information described with reference to FIG.

  In the above, with reference to FIGS. 12 to 14, an example in which both the spectral reflectance estimation image data and the spectral reflectance information are rendered and compared in the color space of L * a * b * is described. did. When the evaluation is performed in the L * a * b * color system, the color difference can be easily evaluated (accurate). However, the present invention is not limited to this example, and it is possible to evaluate in various color systems such as the LUV color system and the XYZ color system.

  The internal configuration of the processing apparatus 120 will be described with reference to FIGS. Each element (processing block) in the processing device 120 shown in FIG. 3 may be configured by dedicated hardware logic, or a program stored in the memory of the processing device 120 is read out. It may be configured by being executed by the processing device 120. In the following description, it is assumed that each element is configured by reading a program stored in the memory of the processing device 120 and executing it by the processing device 120.

  The search condition setting unit 302 receives a search condition such as a search keyword input by the user, and performs a process of sending the search condition information to the database 140. This search condition is used for searching the database 140 for spectral reflectance information corresponding to image data to be subjected to color reproduction processing by the processing device 120. Referring to FIGS. 5 and 6, frames 5 d, 5 e,..., 5 h are displayed on monitor display device 122 connected to processing device 120, and processing device 120 accepts a search condition input by a user. As described above, this corresponds to the operation in the search condition setting unit 302.

  The color image storage unit 304 includes a hard disk device, a flash memory, and the like, and stores color image data acquired by the user of the processing device 120. As described above, this image data is obtained by a user using a digital camera, stored in a storage medium such as an optical disk, or obtained via the Internet. Any of these may be used.

  The spectral reflectance information acquisition unit 306 performs processing for acquiring the spectral reflectance information searched in the database 140 based on the search condition information described above and outputting it to the color reproduction processing unit 308. With reference to FIGS. 6 and 7, it has been described that the processing device 120 acquires the spectral reflectance information of the candidate 2 selected by the user from the database 140. This corresponds to the operation of the spectral reflectance information acquisition unit 306. To do.

  The reference area setting unit 310 accepts an operation in which a user sets a reference area in a processing target image and a sample image that are referred to in color reproduction processing. With reference to FIGS. 7 and 8, it has been described that the processing device 120 accepts the setting of the reference areas 8 a and 8 b by the user. This corresponds to the operation of the reference area setting unit 310. The reference area setting unit 310 outputs information related to the reference area 8 a to the color reproduction processing unit 308, and outputs information related to the reference area 8 b to the spectral reflectance information acquisition unit 306. Note that the spectral reflectance information acquisition unit 306 has previously described the process of outputting spectral reflectance information to the color reproduction processing unit 308, but the spectral reflectance information output here corresponds to the reference region 8b. Is.

  The evaluation area setting unit 312 accepts an operation for the user to set an evaluation area in the processing target image and the sample image that are referred to when evaluating the color reproduction processing result. With reference to FIG. 8 and FIG. 9, it has been described that the processing device 120 accepts that the evaluation areas 9 a and 9 b are set by the user. This corresponds to the operation of the evaluation area setting unit 312. The evaluation region setting unit 312 also outputs evaluation target region designation information, which is information for designating which region in the image is to be evaluated, to the image evaluation unit 316 when the image evaluation unit 316 performs image evaluation described later. .

  The color reproduction information setting unit 320 performs color reproduction processing, such as observation illumination, monitor display characteristics, and a display color space of the monitor display device, and is necessary for color reproduction information necessary for processing to display an image on the monitor display device 122. Set. Setting of color reproduction information by the color reproduction information setting unit 320 can be performed by receiving an input operation of color reproduction information by the user. Alternatively, it can be implemented by reading a file in which color reproduction information is recorded. The set color reproduction information is output from the color reproduction information setting unit 320 to the color reproduction processing unit 308 and the reference information creation unit 314.

  The reference information creation unit 314 performs color reproduction processing that applies the color reproduction information input from the color reproduction information setting unit 320 to the spectral reflectance information acquired from the database 140, and creates reference information that serves as a reference for image evaluation. To do. This reference information is output to the image evaluation unit 316. When the spectral reflectance information acquired from the database 140 is discrete measurement spectral reflectance information, the reference information is discrete reference information.

  When the spectral reflectance information acquired from the database 140 is continuous measurement spectral reflectance information, the reference information is continuous reference information. According to the continuous reference information, it is possible to obtain reference information of an arbitrary part on the surface of the predicted object to be photographed.

  Also. When the spectral reflectance information acquired from the database 140 is continuous measurement spectral reflectance information, the sample image 7a shown in FIG. 7 can be displayed using the continuous measurement spectral reflectance information. In that case, the reference information created by the reference information creation unit 314 is also output to the image display processing unit 322.

  The configuration of the color reproduction processing unit 308 will be described with reference to FIG. The color reproduction processing unit 308 includes an image cutout unit 402, an average processing unit 404, a color conversion matrix creation unit 406, and a color conversion calculation unit 408. The image cutout unit 402 includes image data corresponding to the reference region 8a set by the reference region setting unit 310 (hereinafter referred to as reference region image data) from the image data read from the color image storage unit 304. ) And output to the average processing unit 404. The average processing unit 404 performs processing to reduce the influence of noise on the reference area image data using a method such as an average filter.

  The color conversion matrix creation unit 406 includes reference area image data after noise reduction processing output from the average processing unit 404, spectral reflectance information of the reference area 8b output from the spectral reflectance information acquisition unit 306, and color reproduction. Color conversion parameters are generated based on the color reproduction information output from the information setting unit 320 and the input profile when the processing target image data includes the input profile. The color conversion matrix creation unit 406 outputs this color conversion parameter to the color conversion calculation unit 408. The color conversion calculation unit 408 applies color conversion parameters to the processing target image data, and outputs the processed image data to the image evaluation unit 316, the processed image storage unit 318, and the image display processing unit 322.

  Referring again to FIG. 3, the image display processing unit 322 performs processing for displaying the images, buttons, and the like shown in FIGS. 5 to 14 on the monitor display device 122.

  The image evaluation unit 316 receives the image data after the color reproduction process output from the color reproduction processing unit 308 and the reference information output from the reference information creation unit 314. Then, the image data and the reference information are compared and evaluated in an area designated by the evaluation area designation information output from the evaluation area setting unit 312. The image evaluation unit 316 outputs this evaluation result to the evaluation result display processing unit 324.

  The evaluation result display processing unit 324 performs processing for displaying the evaluation result display described with reference to FIGS. 11 to 14 on the monitor display device 122.

  With reference to FIGS. 15A and 15B, the color reproduction processing procedure executed by the processing device 120 will be described. The processing device 120 has a CPU and a memory, and the color reproduction processing procedure shown in FIGS. 15A and 15B is performed when the CPU executes a program stored in the memory. Alternatively, the processing device 120 may have a dedicated hardware logic, and the color reproduction processing procedure shown in FIGS. 15A and 15B may be performed by this hardware logic.

  The processing procedures shown in FIGS. 15A and 15B are started when the user issues an instruction to operate the processing device 120 to perform the color reproduction processing procedure. In the following description, it is assumed that the processing device 120 executes the color reproduction processing procedure shown in FIGS. 15A and 15B.

  In step S <b> 1500, the processing device 120 performs processing for accepting that the user selects processing target image data using a keyboard, a computer mouse (hereinafter simply referred to as a mouse), or the like. In step S <b> 1502, the processing device 120 displays an image based on the processing target image data on the monitor display device 122. This corresponds to the display of the image 5a as shown in FIG.

  In step S <b> 1504, the processing device 120 performs processing for receiving a search information input operation by the user. That is, as described with reference to FIG. 5, the processing device 120 performs a process of accepting that the user inputs search information necessary for the frames 5 d to 5 h.

  In step S <b> 1506, the processing device 120 performs processing for accepting a search start operation by the user. That is, as described with reference to FIG. 5, the processing device 120 performs a process of accepting that the user clicks the “search start” button 5 k.

  The processing device 120 sends search information to the database 140 in S1508, and receives search results from the database 140 in S1510. Based on the search result received from the database 140, the processing device 120 performs processing for displaying the spectral reflectance information candidates on the monitor display device 122 in S1512. As a result, the search result display window 5j (FIGS. 5 and 6) is displayed on the monitor display device 122.

  In S1514, the processing device 120 determines whether or not the “re-search” button 6b shown in FIG. 6 has been clicked. If this determination is affirmative, the processing device 120 returns to S1504 and performs a process of accepting a re-search operation by the user. If the determination in S1514 is negative, the processing apparatus 120 performs an operation (FIG. 6) in which the user selects a desired one from the spectral reflectance information selection candidates displayed in the search result display window 5j in S1516. Process to accept.

  In S1518, the processing device 120 performs processing for displaying the sample image 7a (FIG. 7) corresponding to the spectral reflectance information selected by the user in S1516 on the monitor display device 122. In S1520, the processing device 120 determines whether or not the “return” button 7c shown in FIG. 7 has been clicked. If this determination is affirmed, the process returns to S1516, and a process of accepting an operation for the user to select a desired one from the selection candidates of spectral reflectance information displayed in the search result display window 5j is performed again. If the determination in S1520 is negative, the process proceeds to S1522.

  In S1522, the processing apparatus 120 determines whether or not the “re-search” button 7d illustrated in FIG. 7 has been clicked. If this determination is affirmed, the process returns to S1504, and a process of accepting a search information input operation by the user is performed. If the determination in S1522 is negative, the process proceeds to S1524.

  In S1524, the processing apparatus 120 performs a process of accepting that the user clicks the “reference area setting” button 7b illustrated in FIG. 7 and subsequently performs the setting operation of the reference areas 8a and 8b illustrated in FIG. Do.

  In S <b> 1526, the processing device 120 performs processing for accepting that the user clicks the “evaluation area setting” button 8 c shown in FIG. 8 and then the user performs setting operations for the evaluation areas 9 a and 9 b shown in FIG. 9. Do. In S <b> 1528, the processing device 120 performs processing for accepting that the user clicks the “color conversion start” button 9 c shown in FIG. 9.

  In S1530, the processing device 120 performs the following color reproduction processing. That is, the processing device 120 estimates the spectrum of the imaging illumination light on the assumption that the spectral reflectance of the reference region 8a in the processing target image is equal to the spectral reflectance of the reference region 8b in the spectral reflectance information. Subsequently, the processing device 120 applies the estimated spectrum of the photographic illumination light and the input profile of the image input device that has generated the processing target image to a portion other than the reference region 8a in the processing target image to obtain spectral reflectance image data. The processing device 120 performs a process of applying the color reproduction information to the spectral reflectance image data, and obtains an image (processed image) after the color reproduction process.

  In S1532, the processing device 120 performs processing for displaying the processed image 10a on the monitor display device 122, as shown in FIG.

  In S1534, the processing device 120 determines whether or not the “return” button 10d shown in FIG. 10 has been clicked. If this determination is affirmed, the process returns to S1524, and a process of accepting an operation for setting a reference area by the user is performed. If the determination in S1534 is negative, the process proceeds to S1536.

  In S1536, the processing device 120 determines whether or not the “re-search” button 10e shown in FIG. 10 has been clicked. If this determination is affirmed, the process returns to S1504, and a process of accepting a search information input operation by the user is performed. If the determination in S1536 is negative, the process proceeds to S1538.

  In S1538, the processing device 120 determines whether or not the “save” button 10c shown in FIG. 10 has been clicked. If this determination is affirmed, the process proceeds to S1540, and a process of saving the image data after the color reproduction process is performed. If the determination in S1538 is negative, the process proceeds to S1542. Also, after the image data storage process in S1540 is performed, the process proceeds to S1542.

  In S <b> 1542, the processing device 120 performs processing for accepting that the user clicks the “evaluation” button 10 b shown in FIG. 10. In S1544, the processing device 120 performs the following evaluation process. That is, the processing device 120 performs processing for comparing the spectral reflectance image data in the evaluation region 9a with the spectral reflectance in the evaluation region 9b in the spectral reflectance information acquired from the database 140 for each wavelength.

  In S1546, the processing device 120 performs processing for displaying the result of the evaluation processing performed in S1544 on the monitor display device 122. As a result, a graph 11a is displayed as shown in FIG.

  Note that the processing executed by the processing device 120 in S1544 and S1546 described above is an example, and as described above with reference to FIGS. 11 to 14, evaluation using various methods is possible. In addition, it is possible to display the evaluation result according to it.

  In S1548, the processing device 120 determines whether or not the “save” button 10c shown in FIG. 11 has been clicked. If this determination is affirmed, the process proceeds to S1550, and a process of saving the image data after the color reproduction process is performed. This process corresponds to the process performed in response to the user clicking the “save” button 10c after determining that an image that is almost satisfactory is obtained by looking at the graph 11a.

  If the determination in S1548 is negative, the process proceeds to S1552. Also, after the image data storage process in S1550, the process proceeds to S1552.

  If the user is not satisfied with the graph 11a or the processed image 10a, the user clicks the “reset evaluation area” button 11b or the “re-search” button 10e. In response to this user operation, the processing device 120 performs the processing described below.

  In S1552, the processing device 120 determines whether or not the “reset evaluation area” button 11b illustrated in FIG. 11 has been clicked. If this determination is affirmed, the process returns to S1526, and a process of accepting an operation for setting an evaluation area by the user is performed. If the determination in S1552 is negative, the process proceeds to S1554.

  In S1554, the processing device 120 determines whether or not the “re-search” button 10e shown in FIG. 11 has been clicked. If this determination is affirmed, the process returns to S1504, and a process of accepting a search information input operation by the user is performed. If the determination in S1554 is negative, the processing device 120 completes the color reproduction process.

  As described above, in the color reproduction system according to the embodiment of the present invention, the spectral reflectance information of the predicted photographing object is accumulated in the database 140. Therefore, when an expected photographing object is reflected in the processing target image, the spectral reflectance information of the predicted photographing object can be acquired from the database. Therefore, it is possible to obtain an excellent color reproduction image without acquiring photographing illumination information when inputting an image with the image input device or copying a color target with a known spectral reflectance together with the subject. Become.

  Even if the main subject is a person or the like and the spectral reflectance information is not stored in the database 140, if the expected subject to be photographed is shown together with the main subject, it can be used as a color target. As a result, the color reproducibility of the main subject can be improved. At this time, according to the embodiment of the present invention, the user can arbitrarily set the reference areas 8a and 8b and the evaluation areas 9a and 9b described with reference to FIGS. A suitable part can be set as a reference region. In addition, any part desired by the user can be set as the evaluation area, and the evaluation area can be changed as necessary, so that the user can obtain a more accurate evaluation result of the image data after color reproduction processing. It becomes possible.

  In addition, an image that is displayed by adjusting the hue, saturation, color balance, level, contrast, etc. by repeating trial and error while viewing the image displayed on the monitor display device by the user like conventional photo retouching software Unlike the case of adjusting the color tone to the desired color tone, the user can easily obtain an image excellent in color reproducibility with a simpler processing procedure.

  For example, a case where the user travels and takes a photo at a sightseeing spot will be described as an example. The user returns home or displays an image on the display of the personal computer at the accommodation. At this time, the user who feels uncomfortable with the color reproducibility accesses the database 140 and finds the desired spectral reflectance information. Since various keywords that can be recalled by the user and spectral reflectance information are stored in the database 140 in association with each other, the user can easily find the necessary spectral reflectance information. Then, it is possible to easily obtain an image with excellent color reproducibility simply by setting the reference region and the evaluation region in both the processing target image and the spectral reflectance information.

  The color reproduction processing technology according to the present invention is applied to an information processing device such as a personal computer, an image display device, and a set top box box, video recorder, or video player connected to a monitor display device or a television receiver. Is possible.

It is a figure explaining the schematic structure of the color reproduction system which concerns on embodiment of this invention. It is a figure explaining a mode that spectral reflectance information is accumulate | stored with the search information corresponding to this spectral reflectance information. It is a block diagram explaining the schematic internal structure of information processing apparatus. It is a block diagram explaining the schematic internal structure of a color reproduction process part. It is a figure explaining a mode that the user interface for searching a process target image and the spectral reflectance information corresponding to this process target image is displayed. It is a figure explaining a mode that a user selects desired spectral reflectance information. It is a figure explaining a mode that the sample image corresponding to the spectral reflectance information selected by the user is displayed with a process target image. It is a figure explaining a mode that a user sets a reference area on both a process target image and a sample image. It is a figure explaining a mode that a user sets an evaluation area | region on both a process target image and a sample image. It is a figure explaining a mode that the image by which the color reproduction process was performed is displayed with a sample image. It is a figure explaining the example which displays the result of having performed the color reproduction process to the process target image, and evaluating it. It is a figure explaining another example at the time of displaying the result evaluated by performing color reproduction processing on a processing object picture, and is a figure explaining the example shown by the chromaticity value in a L * a * b * color system. . It is a figure explaining another example at the time of displaying the result evaluated by performing color reproduction processing on a processing target image, and explains the example which graphs and displays the chromaticity value in a L * a * b * color system. It is a figure to do. It is a figure explaining another example at the time of displaying the result evaluated by performing color reproduction processing on a processing object image, and it was called "bad", "normal", and "good" based on the magnitude of the color difference. It is a figure explaining the example which performs a sensory display. It is a schematic flowchart explaining the process sequence of the color reproduction process performed with a processing apparatus. It is a schematic flowchart explaining the process sequence of the color reproduction process performed with a processing apparatus, and is a flowchart explaining the process procedure following the process procedure shown by FIG. 15A.

Explanation of symbols

5a ... processing target image 5j ... search result display window 5k ... "search start" button 6b ... "re-search" button 7a ... sample image 7b ... "reference area setting" button 7c ... "return" button 7d ... "re-search" button 8a, 8b ... Reference area 8c ... "Evaluation area setting" button 9a, 9b ... Evaluation area 9c ... "Start color reproduction process" button 10a ... Image after color reproduction process 10b ... "Evaluation" button 10c ... "Save" button 10d ... "Return" button 10e ... "Research" button 11a ... Graph 11b ... "Reset evaluation area" button 12a ... Table 13a ... Radar chart 14a ... Evaluation result display 100 ... Color reproduction system 120 ... Processing device (information processing device)
DESCRIPTION OF SYMBOLS 122 ... Monitor display apparatus 140 ... Database 160 ... Network 200 ... Spectral reflectance information 302 ... Search condition setting part 304 ... Color image memory | storage part 306 ... Spectral reflectance information acquisition part 308 ... Color reproduction process part 310 ... Reference area setting part 312 ... evaluation area setting unit 314 ... reference information creation unit 316 ... image evaluation unit 318 ... processed image storage unit 320 ... color reproduction information setting unit 322 ... image display processing unit 324 ... evaluation result display processing unit 402 ... image cutout unit 404 ... Averaging processing unit 406 ... Color conversion matrix creation unit 408 ... Color conversion operation unit

Claims (8)

Spectral reflectance information obtained by measuring in advance the spectral reflectance distribution on the surface of the expected object to be photographed is accumulated and formed for each expected object. Database and
A color reproduction system having an image data processing device for performing color reproduction processing of a color image obtained by photographing,
The image data processing device includes:
A spectral reflectance information acquisition unit that acquires spectral reflectance information obtained by searching under a search condition set by a user from the database;
A reference area setting unit for accepting the user to set a reference area that is an area for referring to the color of the subject image in the color image during the color reproduction process;
A color reproduction processing unit that performs color reproduction processing of the color image using the spectral reflectance information, the spectral reflection corresponding to the reference region in the image data in the reference region and the spectral reflectance information A color reproduction system comprising: a color reproduction processing unit that performs color reproduction processing of the color image based on rate information.   2. The color reproduction system according to claim 1, wherein the database stores a plurality of sets of spectral reflectance information obtained by measuring under different measurement conditions with respect to one predicted photographing object.   The color reproduction system according to claim 1, wherein the image data processing device further includes an evaluation unit for evaluating the color reproduction processing result by the color reproduction processing unit. An evaluation area that accepts the user setting an evaluation area, which is an area in the subject image shown in the color image, that is referred to when the evaluation section evaluates the color reproduction processing result by the color reproduction processing section. The image data processing apparatus further includes a setting unit,
The evaluation unit is configured to perform the color reproduction process based on a comparison result between a result of the color reproduction process performed on the image data in the evaluation area and a spectral reflectance corresponding to the evaluation area in the spectral reflectance information. The color reproduction system according to claim 3, wherein the result is evaluated.   5. The color reproduction system according to claim 3, wherein the image data processing apparatus further includes an evaluation result display unit that displays an evaluation result by the evaluation unit by digitization, graphing, or both. Spectral reflectance information obtained by measuring in advance the spectral reflectance distribution on the surface of the expected object to be photographed is accumulated and formed for each expected object. Obtaining spectral reflectance information obtained by searching with a search condition set by the user from the database,
Performing color reproduction processing of a color image obtained by photographing, and in the color reproduction processing, a reference area set by the user as an area for referring to a color of a subject image reflected in the color image. A color reproduction processing method comprising: performing color reproduction processing of the color image based on image data and spectral reflectance corresponding to the reference region in the spectral reflectance information.   An evaluation area set by the user as an area in the subject image reflected in the color image, which is referred to when evaluating the result of the color reproduction process, which is to evaluate the result of the color reproduction process Color reproduction that evaluates the result of the color reproduction processing based on a comparison result between the result of the color reproduction processing performed on the image data in the image and the spectral reflectance corresponding to the evaluation region in the spectral reflectance information The color reproduction processing method according to claim 6, further comprising evaluating a result of the processing.   The color reproduction processing method according to claim 7, further comprising displaying an evaluation result obtained by evaluating the result of the color reproduction processing by quantification, graphing, or both.