JP6891189B2 - Equipment, systems, and recording media for measuring color differences - Google Patents

Description

The present invention relates to an apparatus, a system, and a recording medium for measuring a color difference.

U.S. Patent Application Publication No. 2014/0022535 discloses devices and methods for measuring optical properties to be measured, such as saturation values and total spectrum emission coefficients. In U.S. Patent Application Publication No. 2014/0022535, the spectral intensity distribution of a predetermined ambient light incident through a measurement opening is measured and stored before measuring the optical properties of the object to be measured. Next, when measuring the optical characteristics, the spectral intensity distribution of the stored ambient light is used to obtain the optical characteristics when the measured ambient light is used as an observation light source.

With objects installed, when looking at objects such as outdoor signboards, wall coverings used outside or inside structures, from a distance, grasp the overall color tendency of those objects. In order to do so, a technique of obtaining an average color value in a relatively wide area of an object and then measuring a color difference from a plurality of color values is required. If there is a pattern on the surface of the object, the color value obtained by the measurement may change depending on the position and size of the area of the object to be measured. Therefore, a spectral photometer that measures a single specific point is not suitable for measuring average color values as described herein. On the other hand, some colorimeters on the market can measure a wider area, but such devices are large and expensive and therefore not portable. This makes it difficult to measure objects such as outdoor signboards that are installed. Therefore, for example, using a commercially available mobile terminal provided with an imaging unit, it is necessary to have a function capable of easily obtaining an average color value in a measurement area and measuring a color difference between a plurality of color values.

An object of the present invention is to provide an apparatus, system, and recording medium capable of easily measuring a color difference between an average color value of an object and a color value to be compared.

An image acquisition unit that acquires image data of an image shot of an object, an area selection unit that selects a predetermined image area on the object in the image, and an average of the color values in the image area from the color values of the image data. An average value calculation unit that calculates a certain average color value, a color difference calculation unit that calculates the color difference between the average color value and the color value to be compared in the same color space as the average color value, and an output unit that outputs information about the color difference. And the equipment including.

In the above-mentioned device, the area selection unit can select an area having an area and a shape in which the average color value does not change significantly from a predetermined allowable range according to the position of the image area in the image as the image area. is there.

In the above-mentioned apparatus, the area selection unit has a specific area and shape at a position where the average color value in the area does not change much more than a predetermined allowable range when the area having the specific area and shape moves in the image. The area can be selected as the image area.

The above-mentioned device further includes an approval unit that approves an area in an image designated by a user operation, and the area selection unit can select an image area according to a user operation approved by the approval unit. ..

In the above-mentioned device, the area selection unit can select the entire object in the image as an image area.

In the above-mentioned device, the area selection unit acquires the distance between the image pickup unit that captures an image and the object from the image pickup information added to the image data, and selects an image region having a larger area as the distance is shorter. It is possible to do.

In the above-mentioned device, the area selection unit can select an image area except for a pixel area in which the brightness value in the image exceeds the brightness value limit.

The device can further include a lighting condition correction unit that calculates the lighting conditions for capturing an image based on the image data and corrects the color value of the image data according to the calculated lighting conditions, and is an average value calculation unit. Can calculate the average color value from the color value corrected by the illumination condition correction unit.

The device can further include an imaging unit that captures an image.

The device stores a plurality of reference color values, which are color values in the same color space as the average color value, and identification information of a plurality of objects related to each other, and identification information of the objects in the image. It is possible to further include an identification information acquisition unit to be acquired and a determination unit for determining whether or not the color difference is within a predetermined color difference range. Here, the color difference calculation unit calculates the color difference between the reference color value corresponding to the identification information and the average color value among the plurality of reference color values, and the output unit outputs the result of the determination by the determination unit. It is possible.

In the above-mentioned apparatus, the image acquisition unit obtains the first image data obtained by photographing the first object and the second image data obtained by photographing the second object. Acquiring as image data, the average value calculation unit calculates the first average color value of the first image data as the average color value, and the second average color value of the second image data is compared. The color difference calculation unit calculates the color difference between the first average color value and the second average color value, and the output unit outputs the color difference within a predetermined color difference range. It is possible to output information indicating whether or not.

In the above-mentioned apparatus, the first object may be a printed matter, and when the color difference is not within the color difference range, a print data correction unit that corrects the color value of the printed data of the printed matter so as to reduce the color difference is further provided. It is possible to prepare.

The above-mentioned device has a storage unit that stores a plurality of reference color values that are color values in the same color space as the average color value, and identification information of a plurality of objects related to each other, and identification information based on a color difference calculation result. It is possible to further include a reference part that refers to. Here, the color difference calculation unit calculates the color difference between each of the plurality of reference color values and the average color value, and the output unit can output the referenced identification information.

In the above-mentioned device, the storage unit converts the spatial distribution of the color values in the image obtained by photographing the objects for each of the plurality of objects into the spatial frequency spectrum region, and the frequency distribution is obtained. The device further stores the frequency distribution stored in the storage unit for each of the plurality of objects, and the spatial frequency spectrum region of the color value in the image of the image data acquired by the image acquisition unit. It is possible to further include a comparison processing unit for comparison with the frequency distribution obtained by converting within, and for the reference unit to further refer to the identification information according to the result of comparison by the comparison processing unit.

Further, a system including a terminal device and a server device that are communicably connected to each other is provided. The terminal device has an image acquisition unit that acquires image data of an image shot of an object, an area selection unit that selects a predetermined image area on the object in the image, and a color in the image area from the color value of the image data. An average value calculation unit that calculates the average color value, which is the average of the values, a color difference calculation unit that calculates the color difference between the reference color value and the average color value received from the server device, and identification information of the object in the image. Is further included in the terminal communication unit that transmits the above to the server device and receives the reference color value corresponding to the object indicated by the identification information from the server device, and the output unit that outputs the information regarding the color difference. The server device receives from the terminal device a storage unit that stores a plurality of reference color values that are color values in the same color space as the average color value, and identification information of a plurality of objects related to each other, and the identification information. It includes a server communication unit that transmits a reference color value corresponding to an object, which is indicated by identification information from a plurality of reference color values, to a terminal device.

Further, a system including a terminal device and a server device that are communicably connected to each other is provided. The terminal device includes an image acquisition unit that acquires image data of an image shot of an object, an area selection unit that selects a predetermined image area on the object in the image, and an average color that is an average of color values in the image area. An average value calculation unit that calculates the value from the color value of the image data, a terminal communication unit that sends the average color value to the server device and receives the identification information of the object from the server device, and an output unit that outputs the identification information. And, including. The server device has a storage unit that stores a plurality of reference color values that are color values in the same color space as the average color value, identification information of a plurality of objects related to each other, and an average color value received from the terminal device. A color difference calculation unit that calculates the color difference from the reference color value, a reference unit that refers to the identification information based on the color difference calculation result, and a terminal that receives the average color value from the terminal device and receives the identification information referenced by the reference unit. Includes a server communication unit that transmits to the device.

Further, a system including a terminal device and a server device that are communicably connected to each other is provided. The terminal device transmits image data to the server device, an image acquisition unit that acquires image data of an image shot of the object, an area selection unit that selects a predetermined image area on the object in the image, and the object. It includes a terminal communication unit that receives the identification information of the above from the server device, and an output unit that outputs the identification information. The server device is an average of color values in an image area from a storage unit that stores identification information of a plurality of reference color values and a plurality of objects associated with each other and color values of image data received from the terminal device. An average value calculation unit that calculates a color value, a color difference calculation unit that calculates a color difference between an average color value and a reference color value, a reference unit that refers to identification information based on the color difference calculation result, and a terminal device that outputs image data. Includes a server communication unit that receives from and transmits the identification information referenced by the reference unit to the terminal device.

Further, a system including a terminal device and a server device that are communicably connected to each other is provided. The terminal device includes an image acquisition unit that acquires a first image data obtained by photographing a first object that is a printed matter, and a predetermined image area on the first object in the first image data. Includes an area selection unit for selecting, and a terminal communication unit for transmitting image data of the image area selected by the area selection unit to the server device. The server device has a storage unit that stores the second image data obtained by photographing the second object, a first average color value of the first image data, and a second image data of the second image data. An average value calculation unit that calculates the average color value of the data, a color difference calculation unit that calculates the color difference between the first average color value and the second average color value, and a color difference calculation unit that calculates the color difference when the color difference is not within a predetermined color difference range. The print data correction unit that corrects the color value of the print data of the printed matter so as to reduce the color difference, and the server communication unit that receives the first image data from the terminal device are included.

Further, the function of acquiring the image data of the image shot of the object, the function of selecting a predetermined image area on the object in the image, and the average color value which is the average of the color values in the image area are obtained from the image data. A function to calculate from the color value, a function to calculate the color difference between the average color value and the color value to be compared, which is the average color value in the same color space, and a function to output information about the color difference are executed on the computer. A non-temporary computer-readable recording medium is provided in which the program to be recorded is recorded.

With the above-mentioned devices, systems, and recording media, the color difference between the average color value of the object and the color value of the comparison target can be easily measured.

It is a figure which shows the structure of the terminal apparatus 1 schematicly. It is a figure which shows the example of the screen displayed on the display part 13. It is a figure which shows the example which selects the image area by the area selection part 22. It is a graph which shows the fluctuation of the average color value which depends on the area of an image area. It is a figure which shows the other method of selecting an image area by the area selection part 22. It is a flow chart which shows the operation example executed by the terminal apparatus 1. It is a figure which shows roughly the structure of the terminal apparatus 1A. It is a figure which shows the example of the information stored in the storage part 14 of the terminal apparatus 1A. It is a flow chart which shows the operation example executed by the terminal apparatus 1A. It is a figure which shows roughly the structure of the terminal apparatus 1B. It is the schematic which shows the function of the terminal apparatus 1B. It is a flow chart which shows the operation example executed by the terminal apparatus 1B. It is a figure which shows roughly the structure of the terminal apparatus 1C. It is a figure which shows the example of the information stored in the storage part 14 of the terminal apparatus 1C. It is a figure which shows the example of the wall covering image, and the frequency distribution image corresponding to them. It is a flow chart which shows the operation example executed by the terminal apparatus 1C. It is a figure which shows the structure of the communication system 3D schematicly. It is a figure which shows the structure of the communication system 3E schematicly. It is a figure which shows the operation sequence example of the communication system 3E. It is a figure which shows the structure of the communication system 3F schematicly. It is a figure which shows the operation sequence example of the communication system 3F. It is a figure which shows roughly the structure of the communication system 3G. It is a figure which shows the operation sequence example of the communication system 3G.

Devices, systems, and recording media for measuring color differences will be described in detail below with reference to the accompanying figures. However, it should be noted that the technical scope of the present invention is not limited to those embodiments, and the technical scope extends to the present invention and its equivalents disclosed in the claims.

FIG. 1 is a diagram schematically showing the configuration of the terminal device 1. The terminal device 1 is realized by a mobile terminal such as a smartphone with a camera. For example, an average color value of a relatively wide range of a captured image in an object having a size of several cm to several m is obtained, and a color difference between a plurality of color values is obtained. Functions as a color difference meter to measure. The "relatively wide area" referred to herein is part or all of an object in an image, for example, corresponding to an area of approximately several millimeters square in the image. The terminal device 1 includes an imaging unit 11, an operation unit 12, a display unit 13, a storage unit 14, a terminal communication unit 15, and a control unit 20.

The image pickup unit 11 corresponds to the camera provided in the terminal device 1, captures an image of the object, and uses RAW (DNG) data, PEG (JFIF) data, sRGB data, or a similar format to measure the object. Get the image data of. Any of these data formats can be used, but in the following, an example in which the imaging unit 11 acquires sRGB data will be mainly described.

The operation unit 12 is composed of, for example, a touch panel, a keyboard, or the like, and receives a user's operation. The display unit 13 is, for example, a liquid crystal display, and may be integrated with the operation unit 12 as a touch panel display. The display unit 13 is an example of an output unit, and outputs information on the color difference calculated by the control unit 20.

FIG. 2 is a diagram showing an example of a screen displayed on the display unit 13. In the example shown here, the two images 50A and 50B captured by the imaging unit 11 are displayed in the upper range of the display unit 13. On the other hand, the average color value 61 in the rectangular image area 51 in the image 50A, the average color value 62 in the rectangular image area 52 in the image 50B, and the color difference 63 between the average color values 61 and 62 are the lower part of the display unit 13. Displayed in the range. The "average color value" referred to here is the average of the sRGB values of each pixel in the image area 51 or the image area 52.

The storage unit 14 is, for example, a semiconductor memory that stores image data acquired by the imaging unit 11, data necessary for the operation of the terminal device 1, and the like. The terminal communication unit 15 is an interface used by the terminal device 1 to transmit data to and receive data from the external device.

The control unit 20 is composed of a CPU, RAM, ROM, and the like, and controls the operation of the terminal device 1. As shown in FIG. 1, the control unit 20 includes an image acquisition unit 21, an area selection unit 22, a lighting condition correction unit 23, an average value calculation unit 24, and a color difference calculation unit 25 as functional blocks. ..

The image acquisition unit 21 acquires image data of an image of an object captured by the image pickup unit 11. However, the image acquisition unit 21 may acquire image data captured by another imaging device via, for example, the terminal communication unit 15.

The area selection unit 22 selects a predetermined image area in the image from which the average color value of the object is to be calculated. At this time, the area selection unit 22 selects an image area according to the user of the terminal device 1 using the operation unit 12, and specifies, for example, a range in the image displayed on the display unit 13. For example, when the operation unit 12 and the display unit 13 configure a touch panel display, the area selection unit 22 selects a range within the displayed image designated as an image area by the user using his / her finger. In this case, the operation unit 12 functions as an approval unit that approves the area of the image designated by the user operation.

Alternatively, the area selection unit 22 may automatically select an image area without performing a user operation. At this time, the region selection unit 22 can select a region having an area and a shape whose average color value does not change significantly from a predetermined allowable range depending on the position in the image as a predetermined image region. This selection method will be described with reference to FIGS. 3 and 4.

FIG. 3 is a figure showing an example of selecting an image area by the area selection unit 22, and FIG. 4 is a graph showing a variation in the average color value depending on the area of the image area. FIG. 3 shows a case where square image regions 51, 52, and 53 having side lengths x1, x2, and x3 (here, x1 <x2 <x3) are selected in the checkerboard pattern image 50, respectively. .. The horizontal axis of FIG. 4 represents the scale x (the length of one side thereof) of the image area, and the vertical axis of FIG. 4 represents the average sRGB value in the image area. In the example shown in FIG. 4, in the range where the scale x of the image region is 3 or less, each RGB average color value changes according to the scale x as a result of the pattern in the image 50. However, in the range where the scale x is 3 or more, each average color value is substantially constant. When the image has a checkerboard pattern, the average color value is constant regardless of the position and area of the area as long as the area is a square containing at least 4 checks in at least a 2x2 pattern. Therefore, the area selection unit 22 applies an area having a specific shape such as a rectangle or a circle to the object in the image, calculates the RGB average color value of each while gradually expanding the area of the area, and then calculates the RGB average color value. In addition, a region having an area and a shape in which the variation of each average color value depending on the area is within the permissible range may be selected.

Alternatively, when moving a region having a specific area and shape in the image, the region selection unit 22 selects the region having the area and shape at a position where the average color value in the region does not change beyond a predetermined allowable range. May be selected as a predetermined image area. This selection method will be described with reference to FIG.

FIG. 5 is a diagram showing another method for selecting an image region by the region selection unit 22. FIG. 5 shows an example in which six square image regions 51 to 56 having the same size are shown in the image 50, and the image 50 includes both a grid area and a blank area. In this example, since the image regions 51 to 53 are in the grid-like region, the average color value of each image region is substantially constant. On the other hand, in each of the image regions 54 to 56, the ratio of the area of the grid region to the blank region is different, so that the average color value of each of these image regions is different from the values of the image regions 51 to 53 and changes depending on the region. .. Therefore, the area selection unit 22 is set at a position where the obtained average color value is substantially constant even if a region having a specific area and shape in the image moves up, down, left, or right in the image by a set number of pixels or more. Regions with a particular area and shape may be selected, such as squares or circles (similar shapes based on their magnification if the image is enlarged or reduced).

Alternatively, the area selection unit 22 can select a larger image area as the distance between the image capturing unit 11 for capturing an image and the object becomes shorter. At this time, for example, the region selection unit 22 acquires a focal length value from image information such as Exif data added to image data such as RAW data and PEG data, and the image region having a larger surface surface as the focal length becomes shorter. Select. The pattern on the surface of the object appears in more detail in the image as the distance from the object is shorter. Therefore, by increasing the area of the image area, it depends on the pattern depending on how the image area is applied. Suppress fluctuations in the average color value.

Alternatively, the area selection unit 22 may select an image area having a different area depending on whether the image is taken with a telephoto lens or a wide-angle lens. Generally, when the image pickup unit 11 and the subject (object) are separated from each other, the subject is enlarged by shooting the subject with a telephoto lens, so even a detailed pattern can be seen in the image. it can. On the contrary, when the image pickup unit 11 and the subject are close to each other, the subject is made to appear smaller by photographing the subject with the wide-angle lens, so that the detailed pattern cannot be seen in the image. Therefore, even if the distance between the image pickup unit 11 and the object is the same, the region selection unit 22 has a larger area in the case of the image taken by the telephoto lens than in the case of the image taken by the wide-angle lens. It is possible to select the image area to have.

Alternatively, the area selection unit 22 may select an image area having different areas depending on the resolution of the captured image. In general, when the image resolution is low, the detailed pattern of the subject is not visible, so even if you select a relatively narrow area, it is substantially constant regardless of how that area is applied. The average color value is obtained. On the other hand, when the resolution of the image is high, the detailed pattern of the subject is visually recognized, so that the average color value changes depending on the position of the region or the like unless a relatively wide region is selected. Therefore, even if the distance between the image pickup unit 11 and the object is the same, the region selection unit 22 can select an image region having a larger area as the image resolution is higher.

Alternatively, the area selection unit 22 may select an image area excluding the pixel area in the image whose brightness value is larger than the brightness value limit. The "luminance value limit" refers to a brightness value at which the difference in contrast cannot be recognized, such as overexposure and underexposure. In the subject in the image, the white at the limit brightness of the portion where the portion brighter than the limit becomes completely white is called the upper limit luminance value, or is known as "highlight without overexposure". In general, the overexposed area is a pixel area corresponding to the reflectance of a subject such as 85% or more and 90% or more. On the other hand, in the subject in the image, the black at the limit darkness of the portion where the portion darker than the limit becomes completely black is called the lower limit luminance value, or is known as "shadow that does not collapse black". In general, the blackened area is a pixel area corresponding to the reflectance of a subject such as 3% or less and 5% or less. In this way, the area selection unit 22 may select an image area excluding the pixel area considered to be completely white or completely black.

Alternatively, the area selection unit 22 may select the entire object in the image as the image area.

The lighting condition correction unit 23 calculates the lighting condition on which the image was taken based on the image data acquired by the image acquisition unit 21, and corrects the color value of the image data according to the calculated lighting condition. Therefore, the object to be measured and a standard gray card having a reflectance of about 18% for illumination correction are photographed together, or are photographed separately in advance under the same lighting conditions, for example. Using this standard gray data, the lighting condition correction unit 23 then corrects the color value of the image data of the object according to the following principle, for example.

First, a linear sRGB value (R, G, B) _{Gray-Measured} obtained by photographing and measuring a standard gray under given lighting conditions and a color space (usually D65 or D65) in which the same standard gray is defined. The difference from the linear sRGB value (R, G, B) _{Gray-Standard} previously measured and calculated in D50) is obtained as the correction value (R, G, B) _{Direction-Factor.}

Correction value (R, G, B) Using the _{Direction-Factor} , the illumination condition correction unit 23 corrects the sRGB value (R, G, B) _Measured of the image data of the object as shown by the following equation.

On the other hand, the illumination condition correction unit 23 may ^{convert the color value of the image data captured by the imaging unit 11 into an L *} a ^* b ^* color system value, and then uses the standard gray data as described below. The color value of the image data of the object is corrected according to the principle described.

In this case, the chromaticity value (L, a, b) _{Gray-Measured} obtained by photographing and measuring the standard gray under a given lighting condition and the same standard gray as the defined color space (usually D65 or). The difference from the chromaticity value (L, a, b) _{Gray-Standard} previously measured or calculated in D50) is obtained as the correction value (L, a, b) _{Direction-Factor.}

Correction value (L, a, b) Using the _{Direction-Factor} , the illumination condition correction unit 23 corrects the chromaticity value (L, a, b) _{Measured of the image data of the object as shown by the following equation.} ..

Through this, the difference between the reference light source and the photographing light source is easily corrected, and therefore, even when the image is photographed under a light source having a color temperature different from that of the reference light source, it is measured by the terminal device 1. The color difference can always be regarded as the color difference from the gray value at the reference light source.

The average value calculation unit 24 calculates an average color value, which is the average of the color values in the image area selected by the area selection unit 22, from the color values of the image data corrected by the illumination condition correction unit 23. At this time, the average value calculation unit 24 calculates, for example, the average value of the sR value, sG value, and sB value of each pixel in the selected image area. However, the average value calculation unit 24 is not limited to handling RGB values in the target image region, and may calculate the average value of color values in any desired color space such as ^{L *} a ^* b ^{* values.}

For example, ^{when the average value of the color values in the L *} a ^* b ^* color space is calculated, the average value calculation unit 24 calculates the sRGB value of the image data corrected by the illumination condition correction unit 23 by the formula shown below. Convert via.

First, the average value calculation unit 24 takes in the gamma-corrected sRGB values sR, sG, and sB of the original image data, inverts the gamma correction according to the sRGB standard, and linear sRGB values R, G, and B as shown in the following equation. Ask for.

G and B can be calculated by the same formula as that used for R.

Next, the average value calculation unit 24 converts the acquired RGB value into an XYZ value via matrix conversion as shown by the following equation.

At this time, the average value calculation unit 24 may convert the color temperature of the XYZ value by using a color temperature conversion matrix as proposed by Bradford and Von Kries.

^{Further, the average value calculation unit 24 calculates the L *} a ^* b ^* value from the determined XYZ value via the following equation based on the standard CIE calculation.

Here, Xn, Yn, and Zn are white point coordinates. In this way, the average value calculation unit 24 calculates the average value of the L ^* value, the a ^* value, and the b ^* value of each pixel in the image area selected by the area selection unit 22, for example.

The color difference calculation unit 25 calculates the color difference between the average color value calculated by the average value calculation unit 24 and the average color value of the color values to be compared in the same color space. The color value to be compared may be, for example, a reference average color value stored in advance in the storage unit 14, or may be an average color value obtained by the average value calculation unit 24 for different images. .. For example, when obtaining the color difference for the sRGB value, the color difference calculation unit 25 calculates the color difference ΔE according to the following equation, and the average color value calculated by the average value calculation unit 24 is (R ₁ , G ₁ , B ₁ ). It is assumed that the color value to be compared is represented by (R ₂ , G ₂ , B _2).

On the other hand, when obtaining ^{the color difference for the L *} a ^* b ^* value, the color difference calculation unit 25 calculates the color difference ΔE according to the following equation, and the average color value calculated by the average value calculation unit 24 is (L ₁ , a). _It shall be represented by 1, b ₁ ), and the color value to be compared shall be represented by (L ₂ , a ₂ , b ₂ ).

Next, the color difference calculation unit 25 causes the display unit 13 to display the calculated value of the color difference ΔE, for example, as shown by the reference number 63 in FIG.

FIG. 6 is a flow chart showing an operation example executed by the terminal device 1. The processing of each step of FIG. 6 is executed by the control unit 20 in cooperation with each element of the terminal device 1 based on the program stored in the storage unit 14.

First, the image acquisition unit 21 acquires image data of an image of an object captured by the image pickup unit 11 (S1). Next, the area selection unit 22 selects an image area (S2), which is to calculate the average of the color values in the object in the image by using any one of the above methods. The illumination condition correction unit 23 calculates the illumination condition on which the image was taken based on the image data acquired in S1, and corrects the color value of the image data according to the calculated illumination condition (S3). Next, the average value calculation unit 24 calculates the average color value, which is the average of the color values in the image region selected in S2, from the color value of the image data corrected in S3 (S4). Next, the color difference calculation unit 25 calculates the color difference between the average color value calculated in S4 and the average color value of the color values to be compared in the same color space (S5), and the color difference calculated in S5. Information is displayed on the display unit 13 (S6). Then, the process of FIG. 6 ends.

It should be noted that the terminal device does not necessarily include the imaging unit 11. In other words, the terminal device may be a mobile terminal not equipped with a camera. Even in this case, the color difference and the average color value can be measured as described above by acquiring the image data taken by another imaging device.

Further, the output of the information regarding the color difference is not limited to the display on the display unit 13, and may be in another format such as audio output or data output to an external device.

FIG. 7 is a diagram schematically showing the configuration of the terminal device 1A. The terminal device 1A calculates the average color value of an object such as a wall covering used outside or inside an outdoor signboard or a structure from the image data of an image shot of the object, and sets the value as the same object. Determine if the object is at the end of its useful life by comparing it to the average color value of the new fragment or limit swatch. The hardware configuration of the terminal device 1A is substantially the same as the hardware configuration of the terminal device 1, but the identification information acquisition unit 21A and the determination unit 26A are the image acquisition unit 21, the area selection unit 22, and the lighting condition correction unit. It differs from the terminal device 1 only in that it is included as a functional block of the control unit 20A in addition to the 23, the average value calculation unit 24, and the color difference calculation unit 25. Therefore, only the part of the terminal device 1A different from the terminal device 1 will be described below, and the description of the same parts as the terminal device 1 will be omitted.

FIG. 8 is a diagram showing an example of information stored in the storage unit 14 of the terminal device 1A. The storage unit 14 of the terminal device 1A stores a plurality of reference color values, which are color values in the same color space as the average color value calculated by the average value calculation unit 24, in association with the identification information of the plurality of objects. In the example shown in FIG. 8, this identification information is the product number (No. 1, No. 2, etc.) of the object such as the wall covering material. On the other hand, the standard color values are the average color value An of the object at the time of manufacture (when new) and the average color value Bn of the limit sample in the state where the color of the object fades over time and the object is at the end of its useful life. (Here, n = 1, 2, ...). These reference color values may be, for example, an sRGB value or an L ^* a ^* b ^* value, or may be a color value in yet another color space. The stored average color values An and Bn are values obtained by correcting the lighting conditions by the same means as those used in the lighting condition correction unit 23, or are obtained from image data of an image taken under a reference light source. The value.

The identification information acquisition unit 21A acquires identification information such as a product number of the object in the image acquired by the image acquisition unit 21. The identification information acquisition unit 21A may acquire the identification information in response to, for example, an input operation by the user of the terminal device 1A, a read bar code, or the like.

The color difference calculation unit 25 of the terminal device 1A acquires the reference color value corresponding to the identification information acquired by the identification information acquisition unit 21A from the plurality of reference color values stored in the storage unit 14. Next, the color difference calculation unit 25 is calculated by the reference color value and the average value calculation unit 24 from the image data of the image region selected by the area selection unit 22 and corrected by the illumination condition correction unit 23. Calculate the color difference from the average color value.

The determination unit 26A determines whether or not the color difference ΔE calculated by the color difference calculation unit 25 is within a predetermined color difference range. At this time, when ΔE is a value outside the predetermined color difference range, the determination unit 26A causes the display unit 13 to display a display indicating that the object has faded and is at the end of its useful life. On the other hand, when ΔE is a value within a predetermined color difference range, the determination unit 26A causes the display unit 13 to display a display indicating that the object has not faded and is not at the end of its useful life. The color difference range may be a range in which the color difference between the objects cannot be recognized by the human eye when the objects are viewed at the same time (for example, ΔE <2 when using the ^{L *} a ^* b ^{* color space).} Or, when the object is viewed from a distance, the difference in the pattern on the surface of the object including the structure and color is within the range that cannot be confirmed by humans (for example, ΔE <10 when using the ^{L *} a ^* b ^{* color space).} There may be.

Even if the object does not have the reference color value stored in the storage unit 14, the determination unit 26A photographs the object and a new fragment or limit sample of the same object together, and then By obtaining the color difference between the two, it can be similarly determined whether or not the color of the object has faded.

FIG. 9 is a flow chart showing an operation example executed by the terminal device 1A. The processing of each block of FIG. 9 is executed by the control unit 20A in cooperation with each element of the terminal device 1 based on the program stored in the storage unit 14.

In S11 to S14, acquisition of image data of an object, selection of an image area, correction of color values of image data according to lighting conditions, and calculation of average color values are the same as in S1 to S4 of FIG. After S14, the identification information acquisition unit 21A acquires the identification information of the object in the image acquired in S11 (S15). Next, the color difference calculation unit 25 acquires a reference color value corresponding to the identification information acquired in S15 from the storage unit 14 (S16), and the color difference between the reference color value and the average color value calculated in S14. Is calculated (S17). Further, the determination unit 26A determines whether or not the color difference ΔE calculated in S17 is lower than the predetermined threshold value T (S18). When ΔE <T (Yes in S18), the determination unit 26A displays on the display unit 13 that the object has not faded and is not at the end of its useful life (S19), while ΔE ≥ T. In the case of (No in S18), the determination unit 26A causes the display unit 13 to display a display indicating that the object has faded and is at the end of its useful life (S20). Then, the process of FIG. 9 ends.

FIG. 10 is a diagram schematically showing the configuration of the terminal device 1B. The terminal device 1B calculates the first average color value of the first image obtained by photographing the first object which is a printed matter such as a wall covering material, and calculates the first average color value of the first image to which the printed matter is attached. The second average color value of the second image obtained by photographing the second object is calculated, and the print data of the printed matter is corrected so as to cancel the color difference between the two. The hardware configuration of the terminal device 1B is substantially the same as the hardware configuration of the terminal device 1, but the print data correction unit 26B has an image acquisition unit 21, an area selection unit 22, a lighting condition correction unit 23, and an average value. It differs from the terminal device 1 only in that it is included as a functional block of the control unit 20B in addition to the calculation unit 24 and the color difference calculation unit 25. Therefore, only the part of the terminal device 1B different from the terminal device 1 will be described below, and the description of the same parts as the terminal device 1 will be omitted.

FIG. 11 is a schematic view showing the functions of the terminal device 1B. The terminal device 1B is used for color matching, for example, when producing a wall covering material having the same color and pattern as the brick-patterned wall surface 72 by printing. Using the image pickup unit 11 of the terminal device 1B, the user photographs the printed matter 71 of the wall covering material to be attached to the wall surface 72 together with the standard gray card 73 for lighting correction (or separately under the same lighting conditions). Similarly, the wall surface 72 and the standard gray card 73 are photographed. The terminal device 1B calculates the first average color value and the second average color value. These lighting conditions are corrected for the image 50A of the printed matter 71 and the image 50B of the wall surface 72, respectively. Further, the terminal device 1B calculates the color difference ΔE between the two average color values, and corrects the print data of the printed matter 71 transmitted to the printer 4 so as to cancel the color difference ΔE. When the printed matter 71 is created so as to eliminate the color difference between the average color values, a person can check the difference between the printed matter 71 and the wall surface 72 in close proximity. However, since the detailed pattern cannot be confirmed from a distance, the printed matter 71 appears as a wall covering material having the same color and pattern as the wall surface 72 when viewed from a distance. Therefore, as a result of this color matching, a wall covering material that is inconspicuous even when attached to the wall surface 72 can be obtained as a printed matter.

The image acquisition unit 21 of the terminal device 1B is a wall surface 72 that serves as a comparison target for the colors of the first image data obtained by photographing the first object, which is the printed matter 71, and the color of the printed matter. The second image data obtained by photographing the second object is acquired as image data. These examples of image data may be captured by the imaging unit 11 or may be captured by another imaging device.

The average value calculation unit 24 is the average color value of the image area selected by the area selection unit 22 from the color values of the image data corrected by the illumination condition correction unit 23, and is the average color value of both the first and second image data. Calculate the first and second average color values.

The color difference calculation unit 25 calculates the color difference ΔE between the first average color value and the second average color value calculated by the average value calculation unit 24, and then the color difference ΔE is within a predetermined color difference range. Information indicating whether or not there is is displayed on the display unit 13.

When the color difference ΔE calculated by the color difference calculation unit 25 is not within the predetermined color difference range, the print data correction unit 26B determines the color value (YMCK) of the print data of the printed matter 71 so as to reduce the color difference according to the formula described below. ) Is corrected. The color difference range may be, for example, a range in which the difference between the pattern of the printed matter 71 and the wall surface 72 cannot be recognized from a distance (for example, ΔE <10 in the ^{L *} a ^* b ^{* color space).}

The printed data (Y, M, C, K) _Printed-YMCK of the printed matter 71 sent to the printer 4 is obtained from the linear sRGB values (R, G, B) _{Original-Photo of the image data according to the following formula.} (Converted from additive color mixing to subtractive color mixing).

The original sRGB value (R, G, B) _{Original-Photo} can be set to a value obtained by correcting the lighting condition according to the standard gray data by using the same means as that used for the lighting condition correction unit 23. Is. The printer 4 prints the printed matter 71 according to the print data (Y, M, C, K) _Printed-YMCK , and the obtained image is then taken by the image pickup unit 11.

As shown in the following equation, the difference between the linear sRGB value (R, G, B) _{1st-Print-Photo} of the captured image 50A and the original sRGB value (R, G, B) _{Original-Photo is (} R, G, B) Obtained as _{1st-Compensation-Factor.}

(R, G, B) If _{the color difference between the 1st-Print-Photo} and the sRGB value of the image 50B, which plays a role as a comparison target, does not fall within the predetermined color difference range, print as shown in the following equation. The data correction unit 26B subtracts the difference (R, G, B) _{1st-Compensation-Factor} from the original sRGB value (R, G, B) _{Original-Photo} to obtain the next print data (Y, M, C, K) Obtain the _{2nd-Printed-YMCK.} In other words, the print data correction unit 26B corrects excessive colors in additive color mixing by relatively increasing complementary colors in subtractive color mixing.

Next, the printer 4 reprints the printed matter 71 according to the print data (Y, M, C, K) _{2nd-Printed-YMCK} , and the obtained image is then taken by the imaging unit 11.

When the color difference between the linear sRGB value of the second captured image 50A and the sRGB value of the image 50B which plays a role as a comparison target does not fall within the predetermined color difference range, the print data correction unit 26B uses the following equation. _{As shown in, by subtracting the difference (R, G, B) 2nd-Compensation-Factor} obtained in the same manner as the first time from the original sRGB value (R, G, B) _{Original-Photo} , the following Print data (Y, M, C, K) _{3rd-Printed-YMCK} is obtained.

After that, similarly, the print data correction unit 26B prints data until the color difference between the linear sRGB value of the captured image 50A and the sRGB value of the image 50B which plays a role as a comparison target is within a predetermined color difference range. Is repeatedly corrected.

FIG. 12 is a flow chart showing an operation example executed by the terminal device 1B. The processing of each block of FIG. 12 is executed by the control unit 20B in cooperation with each element of the terminal device 1B based on the program stored in the storage unit 14.

First, the image acquisition unit 21 acquires the image data of the image 50A of the printed matter 71 printed according to the print data YMCK and the image data of the image 50B such as the wall surface 72 that plays the role of the comparison target (S21). Next, the area selection unit 22 selects each image area of the images 50A and 50B, which is to calculate the average of the color values, by using any one of the above methods (S22). The lighting condition correction unit 23 calculates the lighting conditions in which the images 50A and 50B are captured based on the image data acquired in S21, and corrects the color value of the image data according to the calculated lighting conditions (S23). .. _{Next, the average value calculation unit 24 sets the first average color value R 1} G ₁ B ₁ from the color values of the image data of the images 50A and 50B corrected in S23 for the image region selected in S22. And the second average color value R ₂ G ₂ B ₂ is calculated (S24).

Next, the color difference calculation unit 25 calculates the color difference between the first average color value R ₁ G ₁ B ₁ calculated in S24 and the second average color value R ₂ G ₂ B ₂ (S25). The print data correction unit 26B determines whether or not the color difference ΔE calculated in S25 is smaller than the predetermined threshold value T (S26). The process of FIG. 12 ends when ΔE <T (Yes in S26). On the other hand, when ΔE ≧ T (No in S26), the print data correction unit 26B corrects the print data YMCK of the printed matter 71 according to the above formula so as to reduce the color difference (S27). Next, the process returns to S21 and repeats the above steps.

FIG. 13 is a diagram schematically showing the configuration of the terminal device 1C. The terminal device 1C calculates, for example, the average color value of an object such as a wall covering used inside or outside a structure from the image data of an image obtained by photographing the object, and stores the storage unit. From the plurality of wall covering materials stored in 14, the wall covering material whose average color value is close to the color value of the object is referred to, and information on the referenced wall covering material is displayed on the display unit 13. The hardware configuration of the terminal device 1C is substantially the same as the hardware configuration of the terminal device 1, but the frequency conversion unit 26C, the comparison processing unit 27C, and the reference unit 28C are the image acquisition unit 21 and the area selection unit 22. The terminal device 1 is different from the terminal device 1 only in that it is included as a functional block of the control unit 20C in addition to the lighting condition correction unit 23, the average value calculation unit 24, and the color difference calculation unit 25. Therefore, only the part of the terminal device 1C different from the terminal device 1 will be described below, and the description of the same parts as the terminal device 1 will be omitted.

FIG. 14 is a diagram showing an example of information stored in the storage unit 14 of the terminal device 1C. The storage unit 14 of the terminal device 1C stores the product number (identification information), the name, the image (photograph) at the time of manufacture, the shooting condition of the image, and the shooting condition of each of the plurality of wall covering materials (objects). It stores the average color value of the wall covering of the image and the frequency distribution obtained by converting the spatial distribution of the color value in the image into the spatial frequency spectrum region, all of which are related to each other. In the example shown in FIG. 14, the stored imaging conditions include imaging magnification, resolution, and color temperature of the light source. On the other hand, when the color difference calculation unit 25 calculates the color difference, the average color value of the reference color value that plays the role of the comparison target is a color value in the same color space as the average color value calculated by the average value calculation unit 24. .. These reference color values may be, for example, an sRGB value or an L ^* a ^* b ^* value, or may be a color value in yet another color space.

FIG. 15 is a diagram showing an example of a wall covering image and a corresponding frequency distribution image. Reference numbers 80A-80C represent images of wall coverings with wood, marble, and fabric patterns, reference numbers 90A-90C represent images of frequency distribution corresponding to the images shown in reference numbers 80A-80C. Each is shown. These frequency distribution images are images obtained by Fourier transforming the spatial distribution of the luminance values in the x and y directions of the original image, and points having higher frequency components are represented by brighter points. .. Images as shown by reference numbers 90A to 90C are stored in the storage unit 14 of the terminal device 1C as a frequency distribution.

The color difference calculation unit 25 of the terminal device 1C has an average color calculated by the average calculation unit 24 from the image data of the image area selected by the area selection unit 22 for each of the plurality of reference color values stored in the storage unit 14. Calculate the color difference from the value. The lighting condition is corrected by the lighting condition correction unit 23.

The frequency conversion unit 26C converts the spatial distribution of color values in the image represented by the image data acquired by the image acquisition unit 21 into a spatial frequency spectrum region to acquire the frequency distribution. This color value spatial distribution may be a spatial distribution including a luminance value. On the other hand, although the Fourier transform is performed as a typical example of the transformation from the spatial distribution to the frequency distribution, another transform such as an integral transform, a Laplace transform, a discrete cosine transform, and a discrete wavelet transform may be used.

The comparison processing unit 27C compares the frequency distribution of the plurality of wall covering materials (objects) stored in the storage unit 14 with the frequency distribution acquired by the frequency conversion unit 26C. Specifically, the comparison processing unit 27C executes pattern matching between the frequency distribution of each wall covering material stored in the storage unit 14 and the frequency distribution acquired by the frequency conversion unit 26C, and the patterns match. Determine whether or not to do so.

Alternatively, the comparison processing unit 27C may execute pattern matching while changing the spectral intensity of each frequency distribution at a constant rate. Since the spectral intensity of the luminance value frequency distribution is likely to change under different shooting conditions, the comparison processing unit 27C determines whether or not the patterns match based on whether or not the patterns are similar to each other. You can.

Alternatively, the comparison processing unit 27C statistically processes a histogram of the frequency distribution of each wall covering material stored in the storage unit 14 and the frequency distribution acquired by the frequency conversion unit 26C, and then compares both of them. May be good. The post-statistical comparison is, for example, comparing the peak frequency of each spectrum, its half width, or both. If there are multiple peaks in each spectrum, individual peak frequency and full width at half maximum sets may be compared between the respective spectra.

The reference unit 28C refers to the identification information of each wall covering material stored in the storage unit 14 according to the calculation result executed by the color difference calculation unit 25 and the comparison result by the comparison processing unit 27C. For example, the reference unit 28C extracts the product numbers of the wall covering materials of the storage unit 14 in order from the one with the smallest color difference and frequency distribution difference obtained by pattern matching, and their serial numbers, names, and manufacturing time. Information such as the image of is displayed on the display unit 13. In other words, the reference unit 28C obtains the same wall covering material as the image taken by the imaging unit 11 of the storage unit 14, classifies the plurality of wall covering material images, and then obtains the images having the largest color difference and frequency distribution difference. Present to the user in order from the smallest image. However, the display is not limited to the order in which the color difference and the frequency distribution difference are the smallest, and only the wall covering material having the smallest color difference and the frequency distribution difference may be displayed, or the color difference and the frequency distribution difference smaller than the threshold value may be displayed. The wall covering material having the above may be displayed.

The difference between the images when pattern matching is performed between the images converted into the frequency domain tends to be larger than the difference between the original wall covering images. Therefore, the same wall covering material as the target wall covering material or a similar wall covering material can be accurately obtained through two-step image pattern matching, that is, by narrowing down the candidates by color difference and frequency distribution.

A website relating to the wall covering material having the smallest difference in the color difference calculated by the reference unit 28C by the color difference calculation unit 25 and the frequency distribution obtained through pattern matching by the comparison processing unit 27C is displayed on the display unit 13. To. Through this, the control program of the terminal device 1C functions as an application that guides the user to an online catalog of the wall covering material or an electronic commerce site where the wall covering material can be purchased when the wall covering material is photographed. ..

FIG. 16 is a flow chart showing an operation example executed by the terminal device 1C. The processing of each step of FIG. 16 is executed by the control unit 20C in cooperation with each element of the terminal device 1C based on the program stored in the storage unit 14.

In S31 to S34, acquisition of image data of an object, selection of an image area, correction of color values of image data according to lighting conditions, and calculation of average color values are the same as in S1 to S4 of FIG. After S34, the color difference calculation unit 25 calculates the color difference between each reference color value stored in the storage unit 14 and the average color value calculated in S34 (S35). The frequency conversion unit 26C converts the spatial distribution of color values in the image region selected in S32 into a frequency distribution (S36). Next, the comparison processing unit 27C compares the frequency distribution of each wall covering material stored in the storage unit 14 with the frequency distribution obtained by converting in S36 (that is, performs pattern matching) (S37). .. The reference unit 28C refers to the product number (identification information) of each wall covering material of the storage unit 14 in order from the smallest color difference in the calculation result of S35 and the smallest difference in the comparison result in S37, and refers to the product number. , The name of the wall covering material, the image, and the like are displayed on the display unit 13 (S38). Further, the reference unit 28C causes the display unit 13 to display a website relating to the wall covering material having the smallest color difference in the calculation result of S35 and the smallest difference in the comparison result of S37 (S39). Then, the process of FIG. 16 ends.

The terminal devices 1A to 1C can execute the functions of end-of-life determination, color matching of printed matter, and wall covering material search as a single terminal device, but these functions are connected to each other via a communication network. It can be realized by a system including a terminal device and a server device. Examples of cases where the above functions are realized by such a system will be described below.

FIG. 17 is a diagram schematically showing the configuration of the communication system 3D. The communication system 3D is a system for determining the end of life of a wall covering material that plays a role as an object, such as the terminal device 1A. The communication system 3D includes a terminal device 1D and a server device 2D, and these devices are communicably connected to each other via a wired or wireless communication network 5.

The hardware configuration of the terminal device 1D is the same as the hardware configuration of the terminal device 1A, and the functional block of the control unit 20D of the terminal device 1D is the same as the functional block of the control unit 20A of the terminal device 1A. The terminal communication unit 15 of the terminal device 1D transmits the identification information of the object such as the wall covering material acquired by the identification information acquisition unit 21A to the server device 2D, and the reference color value corresponding to the object indicated by the identification information. Is received from the server device 2D. The color difference calculation unit 25 of the terminal device 1D calculates the color difference between the reference color value received from the server device 2D and the average color value calculated by the average value calculation unit 24. The determination unit 26A of the terminal device 1D causes the display unit 13 to display information indicating the determination result regarding the color difference.

The server device 2D includes a storage unit 34, a server communication unit 35, and a control unit 40D. Like the storage unit 14 of the terminal device 1A, the storage unit 34 is a color value in the same color space as the average color value calculated by the average value calculation unit 24 of the terminal device 1D, and can be used as identification information of a plurality of objects. Stores multiple related reference color values. The specific details of the information stored in the storage unit 34 are the same as the information shown in FIG. The server communication unit 35 receives the identification information from the terminal device 1D, and transmits the reference color value corresponding to the object indicated by the identification information to the terminal device 1D. The control unit 40D extracts the reference color value corresponding to the object indicated by the identification information received from the terminal device 1D from the plurality of reference color values stored in the storage unit 34, including the CPU, RAM, ROM, and the like. To do.

According to the communication system 3D, the size of the database can be increased by storing the identification information and the reference color value of a plurality of wall covering materials in the server device 2D having a storage capacity larger than that of the terminal device 1D. Further, by consolidating the database in a single place, that is, in the server device 2D, operations such as updating the database are facilitated.

FIG. 18 is a diagram schematically showing the configuration of the communication system 3E. The communication system 3E is a system like the terminal device 1B that corrects the print data so that the color of the printed matter matches the target color. The communication system 3E includes a terminal device 1E and a server device 2E, and these devices are communicably connected to each other via a wired or wireless communication network 5. Further, the printer 4 is connected to the server device 2E by using a communication cable or the like.

The hardware configuration of the terminal device 1E is the same as that of the terminal device 1B, and the terminal device 1E is different from the terminal device 1B only in that the image acquisition unit 21 and the area selection unit 22 are included as functional blocks of the control unit 20E. .. The functions of the image acquisition unit 21 and the area selection unit 22 are the same as those of the terminal device 1B. The terminal communication unit 15 of the terminal device 1E transmits the image data including the image area selected by the area selection unit 22 to the server device 2E.

The hardware configuration of the server device 2E is the same as the hardware configuration of the server device 2D. The server communication unit 35 of the server device 2E receives the first image data obtained by photographing the first object which is a printed matter from the terminal device 1E. The storage unit 34 of the server device 2E stores the second image data obtained by photographing the second object such as a wall surface, which plays a role as a target for comparison with the color of the printed matter. The control unit 40E of the server device 2E includes a lighting condition correction unit 43, an average value calculation unit 44, a color difference calculation unit 45, and a print data correction unit 46E as functional blocks. The functions of these functional blocks are the same as those of the lighting condition correction unit 23, the average value calculation unit 24, the color difference calculation unit 25, and the print data correction unit 26B of the terminal device 1B.

FIG. 19 is a diagram showing an example of an operation sequence of the communication system 3E. The processing of each step of FIG. 19 is executed by the control units 20E and 40E in cooperation with the respective elements of the terminal device 1E and the server device 2E based on the programs stored in the storage units 14 and 34. ..

First, the image acquisition unit 21 of the terminal device 1E acquires the first image data which is an image of the printed matter printed according to the print data YMCK (S41). Next, using any one of the above methods, the area selection unit 22 selects an image area for which the average of the color values in the image acquired in S41 will be calculated (S42). Next, the control unit 20E of the terminal device 1E transmits the first image data of the image area selected in S42 to the server device 2E using the terminal communication unit 15 (S43).

The lighting condition correction unit 43 of the server device 2E calculates the lighting condition in which the received first image data is captured based on the image data, and corrects the color value of the image data according to the calculated lighting condition. (S44). For the image area selected in S42, the average value calculation unit 44 calculates the first average color value R ₁ G ₁ B ₁ from the color value of the first image data corrected in S 44, and the first average color value R 1 G 1 B 1 of the storage unit 34 is calculated. _{The second average color value R 2} G ₂ B ₂ is calculated from the color value of the image data of 2 (S45).

Next, the color difference calculation unit 45 calculates the color difference between the first average color value R ₁ G ₁ B ₁ and the second average color value R ₂ G ₂ B ₂ calculated in S45 (S46). ). When the color difference ΔE calculated in S46 is larger than the predetermined threshold value T, the print data correction unit 46E corrects the print data YMCK of the printed matter according to the above formula so as to reduce the color difference (S47). Next, the control unit 40E of the server device 2E outputs the print data corrected in S47 to the printer 4 using the server communication unit 35 (S48). Then, the process of FIG. 19 ends. The step of FIG. 19 is repeated a plurality of times as necessary until the color difference ΔE falls below the threshold value T at S47.

According to the communication system 3E, the print data of the printer 4 is corrected by the print data correction unit 46E of the server device 2E. By executing the correction processing of the print data by using the server device 2E having a processing speed faster than that of the terminal device 1E and a large storage capacity, the processing can be performed more quickly and more accurately.

FIG. 20 is a diagram schematically showing the configuration of the communication system 3F. The communication system 3F is a system for finding the same wall covering material as the photographed wall covering material and presenting the wall covering material to the user, like the terminal device 1C. The communication system 3F includes a terminal device 1F and a server device 2F, and these devices are communicably connected to each other via a wired or wireless communication network 5.

The hardware configuration of the terminal device 1F is the same as the hardware configuration of the terminal device 1C, and the terminal device 1F includes an image acquisition unit 21, an area selection unit 22, a lighting condition correction unit 23, and an average value calculation unit 24. , The frequency conversion unit 26F is different from the terminal device 1C only in that it is included as a functional block of the control unit 20F. The functions of these functional blocks are the same as the functions of the image acquisition unit 21, the area selection unit 22, the lighting condition correction unit 23, the average value calculation unit 24, and the frequency conversion unit 26C of the terminal device 1C. The terminal communication unit 15 of the terminal device 1F transmits the average color value calculated by the average value calculation unit 24 and the image data of the frequency distribution calculated by the frequency conversion unit 26C to the server device 2F, and is transmitted by the server device 2F. The identification information (product number of the wall covering material) of the referenced object is received from the server device 2F. The control unit 20F of the terminal device 1F causes the display unit 13 to display the identification information of the object received from the server device 2F.

The hardware configuration of the server device 2F is the same as the hardware configuration of the server device 2D. Like the storage unit 14 of the terminal device 1C, the storage unit 34 of the server device 2F is a plurality of objects having color values in the same color space as the average color value calculated by the average value calculation unit 24 of the terminal device 1F. Stores a plurality of reference color values related to the identification information of. The specific details of the information stored in the storage unit 34 are the same as the information shown in FIG. The server communication unit 35 receives the average color value and the frequency distribution image data from the terminal device 1F, and transmits the identification information referred to by the control unit 40F to the terminal device 1F.

The control unit 40F of the server device 2F includes a color difference calculation unit 45, a comparison processing unit 47F, and a reference unit 48F as functional blocks. The functions of these functional blocks are the same as the functions of the color difference calculation unit 25, the comparison processing unit 27C, and the reference unit 28C of the terminal device 1C. In particular, the color difference calculation unit 45 calculates the color difference between each of the plurality of reference color values stored in the storage unit 34 and the average color value received from the terminal device 1F.

FIG. 21 is a diagram showing an example of an operation sequence of the communication system 3F. The processing of each step in FIG. 21 is executed by the control units 20F and 40F in cooperation with the respective elements of the terminal device 1F and the server device 2F based on the programs stored in the storage units 14 and 34. ..

In S51 to S54, acquisition of image data of an object, selection of an image area, calculation of an average color value, and conversion to a frequency distribution are the same as in S31, S32, S34, and S36 of FIG. 16, respectively. Although not shown in FIG. 21, prior to S53, the illumination condition correction unit 23 of the terminal device 1F can correct the color value of the image data according to the illumination condition in the same manner as in S33 of FIG. After S54, the control unit 20F of the terminal device 1F transmits the average color value calculated in S53 and the frequency distribution image data obtained from the conversion in S54 to the server device 2F using the terminal communication unit 15 (the terminal communication unit 15 is used). S55).

The color difference calculation unit 45 of the server device 2F calculates the color difference between each of the plurality of reference color values stored in the storage unit 34 and the average color value received from the terminal device 1F (S56). Next, the comparison processing unit 47F compares the frequency distribution of each wall covering material stored in the storage unit 34 with the frequency distribution received from the terminal device 1F (that is, performs pattern matching) (S57). The reference unit 48F stores the identification information (of the wall covering material) of the object of each wall covering material stored in the storage unit 34 in order from the smallest color difference in the calculation result of S56 and the smallest difference in the comparison result in S57. (Product number) is referred to (S58). Next, the control unit 40F of the server device 2F transmits the identification information and the like referred to in S58 to the terminal device 1F using the server communication unit 35 (S59). At this time, the control unit 20F of the terminal device 1F causes the display unit 13 to display the received identification information and the like (S60). Then, the process of FIG. 21 is completed.

According to this communication system 3F, the size of the database can be increased by storing the identification information and the reference color value of a plurality of wall covering materials in the server device 2F having a storage capacity larger than that of the terminal device 1F. Further, by executing the search process using the server device 2F having a processing speed faster than that of the terminal device 1F, the process can be performed more quickly and more accurately.

FIG. 22 is a diagram schematically showing the configuration of the communication system 3G. The communication system 3G is a system like the terminal device 1C, which finds the same wall covering material as the photographed wall covering material and presents the wall covering material to the user. The communication system 3G includes a terminal device 1G and a server device 2G, and these devices are communicably connected to each other via a wired or wireless communication network 5.

The hardware configuration of the terminal device 1G is the same as the hardware configuration of the terminal device 1C, and the terminal device 1G only includes the image acquisition unit 21 and the area selection unit 22 as functional blocks of the control unit 20G. Is different. The functions of the image acquisition unit 21 and the area selection unit 22 are the same as those of the terminal device 1C. The terminal communication unit 15 of the terminal device 1G transmits the image data acquired by the image acquisition unit 21 to the server device 2G, and sends the identification information (product number of the wall covering material) of the object referred to by the server device 2G to the server. Received from device 2G. The control unit 20G of the terminal device 1F causes the display unit 13 to display the identification information of the object received from the server device 2G.

The hardware configuration of the server device 2G is the same as the hardware configuration of the server device 2D. Like the storage unit 14 of the terminal device 1C, the storage unit 34 of the server device 2G is a color value in the same color space as the average color value calculated by the control unit 40G of the server device 2G, and identifies a plurality of objects. Stores multiple reference color values related to the information. The specific details of the information stored in the storage unit 34 are the same as the information shown in FIG. The server communication unit 35 receives the image data of the object from the terminal device 1G, and transmits the identification information referenced by the control unit 40G to the terminal device 1G.

The control unit 40G of the server device 2G has, as functional blocks, a lighting condition correction unit 43, an average value calculation unit 44, a color difference calculation unit 45, a frequency conversion unit 46G, a comparison processing unit 47G, and a reference unit 48G. To be equipped. The functions of these functional blocks are the same as the functions of the lighting condition correction unit 23, the average value calculation unit 24, the color difference calculation unit 25, the frequency conversion unit 26C, the comparison processing unit 27C, and the reference unit 28C of the terminal device 1C. is there. In particular, the average value calculation unit 44 averages the color values in the image area selected by the area selection unit 22 from the color values of the image data received from the terminal device 1G and whose lighting conditions are corrected by the lighting condition correction unit 43. Calculate the average color value that is.

FIG. 23 is a diagram showing an example of an operation sequence of the communication system 3G. The processing of each step in FIG. 23 is executed by the control units 20G and 40G in cooperation with the respective elements of the terminal device 1G and the server device 2G based on the programs stored in the storage units 14 and 34. ..

The acquisition of image data of the object and the selection of the image area in S61 and S62 are the same as those in S31 and S32 in FIG. After S62, the control unit 20F of the terminal device 1F transmits the image data including the image area selected in S62 to the server device 2G using the terminal communication unit 15 (S63).

The average value calculation unit 44 of the server device 2G calculates the average color value, which is the average of the color values in the image area selected by the area selection unit 22, from the color value of the image data received from the terminal device 1G (S64). .. Although not shown in FIG. 23, prior to S64, the illumination condition correction unit 43 can correct the color value of the image data according to the illumination condition in the same manner as in S33 of FIG. Next, the color difference calculation unit 45 calculates the color difference between each reference color value stored in the storage unit 34 and the average color value calculated in S64 (S65). The frequency conversion unit 46G converts the spatial distribution of color values in the image region selected in S62 into a frequency distribution (S66). Next, the comparison processing unit 47C compares the frequency distribution of each wall covering material stored in the storage unit 34 with the frequency distribution obtained by converting in S66 (that is, performs pattern matching) (S67). .. The reference unit 48F stores the identification information (of the wall covering material) of the object of each wall covering material stored in the storage unit 34 in order from the smallest color difference in the calculation result of S65 and the smallest difference in the comparison result in S67. (Product number) is referred to (S68).

Next, the control unit 40G of the server device 2G transmits the identification information and the like referred to in S68 to the terminal device 1G using the server communication unit 35 (S69). At this time, the control unit 20G of the terminal device 1G causes the display unit 13 to display the received identification information and the like (S70). Then, the process of FIG. 23 is completed.

According to this communication system 3G, in the server device 2G having a storage capacity larger than that of the terminal device 1G, the size of the database can be increased by storing the identification information and the reference color value of a plurality of wall covering materials. Furthermore, by using the server device 2G, which has a processing speed faster than that of the terminal device 1G, to execute the conversion process to the frequency distribution and the search process, the process can be performed more quickly and more accurately than the communication system 3F. Can be done.

A computer program for realizing each of the functions of the control units 20 to 20G of the terminal devices 1 to 1G and the control units 40D to 40G of the server devices 2D to 2G on a computer is a magnetic recording medium that can be read by a computer. Alternatively, it can be provided and recorded on a recording medium such as an optical recording medium.

The above embodiments can be modified in various ways without departing from the essential spirit of the present invention. For example, FIG. 2 shows an example in which the color difference between two images 50A and 50B is obtained, but the color difference between two objects captured in one image is obtained or captured. The color difference between the image taken by the unit 11 and the image stored in advance in the storage unit 14 may be obtained in advance. On the other hand, the image to be measured is not limited to the image taken by the imaging unit 11, and may be an image taken by a terminal different from the terminal devices 1 to 1G.

Further, the information acquired from the image information by the area selection unit 22 is not limited to the focal length, and is, for example, an effective aperture value (F aperture), a shutter speed, an ISO sensitivity, a reference luminance value calculated from these values, and the like. May include.

Further, the information sent in S43 in FIG. 19, S55 in FIG. 21, and S63 in FIG. 23 may include image information regarding image capturing conditions in addition to the image data. Even if the image data examples are acquired under different shooting conditions (for example, different terminal devices, the same terminal device but different scenes), color difference calculation or frequency distribution comparison is performed in the server device that corrects the difference in shooting conditions. It can be performed. For example, when the color temperature of the light source at the time of shooting is different between the images, the comparison processing units 47F and 47G process one of the examples of the image data based on the sent image information, and control the server device. The comparison process may be performed on the parts so that the color temperatures of the parts match. Similarly, when the ISO sensitivity at the time of shooting differs between images, the comparison processing units 47F and 47G process one of the image data examples based on the transmitted image information, and the control unit of the server device. On the other hand, the comparison process may be executed so that the ISO sensitivities match.

Claims (8)

An image acquisition unit that acquires the images data of the object,
A region selection unit for selecting the constant of an image area where in the image data,
An average value calculation unit that acquires color values in each pixel in the selected image region and calculates an average color value that is the average of the acquired color values.
A color difference calculation unit that calculates the color difference between the average color value and the color value to be compared in the same color space as the average color value.
An output unit for outputting information relating to said color difference,
Only including,
The area selection unit is an apparatus that selects a region having an area and a shape in which the average color value does not change significantly from a predetermined allowable range depending on the position of the image region in the image as the predetermined image region . An image acquisition unit that acquires image data of an object,
An area selection unit that selects a predetermined image area in the image data, and
An average value calculation unit that acquires color values in each pixel in the selected image region and calculates an average color value that is the average of the acquired color values.
A color difference calculation unit that calculates the color difference between the average color value and the color value to be compared in the same color space as the average color value.
An output unit that outputs information about the color difference and
Including
The area selection unit determines the specific area and shape at a position where the average color value in the area does not change significantly from a predetermined allowable range when the area having the specific area and shape moves in the image. said region having selected as the predetermined image area, equipment. An image acquisition unit that acquires image data of an object,
An area selection unit that selects a predetermined image area in the image data, and
An average value calculation unit that acquires color values in each pixel in the selected image region and calculates an average color value that is the average of the acquired color values.
A color difference calculation unit that calculates the color difference between the average color value and the color value to be compared in the same color space as the average color value.
An output unit that outputs information about the color difference and
Including
The area selection unit acquires the distance between the image pickup unit that captures the image and the object from the imaging information added to the image data, and selects an image region having a larger area as the distance is shorter. to, equipment. An image acquisition unit that acquires image data of an object,
An area selection unit that selects a predetermined image area in the image data, and
An average value calculation unit that acquires color values in each pixel in the selected image region and calculates an average color value that is the average of the acquired color values.
A color difference calculation unit that calculates the color difference between the average color value and the color value to be compared in the same color space as the average color value.
An output unit that outputs information about the color difference and
Including
The region selection unit, the luminance value in the image to select the predetermined image area except the pixel region exceeding the luminance value limit, equipment. The apparatus according to any one of claims 1 to 4, further comprising an imaging unit that captures an image. A storage unit that stores a plurality of reference color values that are color values in the same color space as the average color value, and identification information of a plurality of objects related to each other.
An identification information acquisition unit that acquires the identification information of the object in the image, and
Further including a determination unit for determining whether or not the color difference is within a predetermined color difference range,
The color difference calculation unit calculates the color difference between the reference color value corresponding to the identification information and the average color value among the plurality of reference color values.
The device according to any one of claims 1 to 4, wherein the output unit outputs the result of the determination by the determination unit. The image acquisition unit acquires the first image data obtained by photographing the first object and the second image data obtained by photographing the second object as the image data. And
The average value calculation unit calculates the first average color value of the first image data as the average color value, and the second average color value of the second image data is the color value to be compared. Calculated as
The color difference calculation unit calculates the color difference between the first average color value and the second average color value.
The apparatus according to any one of claims 1 to 4, wherein the output unit outputs information indicating whether or not the color difference is within a predetermined color difference range. The first object is a printed matter,
The apparatus according to claim 7 , further comprising a print data correction unit that corrects the color value of the print data of the printed matter so that the color difference becomes smaller when the color difference is not within the color difference range .

Images