EP4038517A1 - Procédé et appareil pour une recherche de couleur à l'aide d'un dispositif mobile - Google Patents

Procédé et appareil pour une recherche de couleur à l'aide d'un dispositif mobile

Info

Publication number
EP4038517A1
EP4038517A1 EP19947603.7A EP19947603A EP4038517A1 EP 4038517 A1 EP4038517 A1 EP 4038517A1 EP 19947603 A EP19947603 A EP 19947603A EP 4038517 A1 EP4038517 A1 EP 4038517A1
Authority
EP
European Patent Office
Prior art keywords
color
mobile device
processing system
image
candidate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP19947603.7A
Other languages
German (de)
English (en)
Other versions
EP4038517A4 (fr
Inventor
Hong Wei
Sachin Deshpande
Taeyoung Park
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Datacolor Inc
Original Assignee
Datacolor Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Datacolor Inc filed Critical Datacolor Inc
Publication of EP4038517A1 publication Critical patent/EP4038517A1/fr
Publication of EP4038517A4 publication Critical patent/EP4038517A4/fr
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M2250/00—Details of telephonic subscriber devices
    • H04M2250/52—Details of telephonic subscriber devices including functional features of a camera

Definitions

  • the present invention generally relates to the measurement of color, and more specifically relates to color lookup using a smart phone.
  • Color matching may involve identifying a target color (e.g., from an object or from a known standard) and subsequently reproducing the target color in a color mixture, i.e., so that any visual difference between the color mixture and the target color is minimized.
  • a target color e.g., from an object or from a known standard
  • a customer may ask for a can of paint to be mixed in a color that matches the color of a wall in the customer’s home, so that touch-ups to the wall can be made in a manner that "blends in.”
  • an automobile manufacturer may ask that a coating be mixed in a color that matches the color of existing automobiles built by the manufacturer, in order to ensure color consistency across the manufacturer’s line.
  • a method performed by a processing system of a mobile device includes acquiring an image of an object of a target color, wherein the image was captured by an integrated digital camera of the mobile device, calculating a first plurality of values that describes the target color, and wherein the calculating is based on an analysis of a pixel of the image, and identifying a first candidate color from among a plurality of candidate colors in a color library, wherein each candidate color in the plurality of candidate colors is associated with a second set of values that describes the each candidate color, and wherein the second set of values describing the first candidate color matches the first set of values more closely than any second set of values associated with another candidate color of the plurality of candidate colors.
  • a non-transitory computer-readable medium stores instructions. When executed by a processing system of a mobile device, the instructions cause the processing system to perform operations.
  • the operations include acquiring an image of an object of a target color, wherein the image was captured by an integrated digital camera of the mobile device, calculating a first plurality of values that describes the target color, and wherein the calculating is based on an analysis of a pixel of the image, and identifying a first candidate color from among a plurality of candidate colors in a color library, wherein each candidate color in the plurality of candidate colors is associated with a second set of values that describes each candidate color, and wherein the second set of values describing the first candidate color matches the first set of values more closely than any second set of values associated with another candidate color of the plurality of candidate colors.
  • a method performed by a processing system in a communication network includes acquiring an image of an object of a target color, wherein the image was captured by an integrated digital camera of a mobile device that is communicatively coupled to the processing system, calculating a first plurality of values that describes the target color, and wherein the calculating is based on an analysis of a pixel of the image, identifying a first candidate color from among a plurality of candidate colors in a color library, wherein each candidate color in the plurality of candidate colors is associated with a second set of values that describes the each candidate color, and wherein the second set of values describing the first candidate color matches the first set of values more closely than any second set of values associated with another candidate color of the plurality of candidate colors, and transmitting information about the first candidate color to the mobile device.
  • FIG. 1A illustrates an example system in which examples of the present disclosure for color lookup using a mobile device may operate
  • FIG. 1 B illustrates another example system in which examples of the present disclosure for color lookup using a mobile device may operate
  • FIG. 2 is a schematic diagram illustrating one example of an array of photosensors and a corresponding array of color filters arranged in a Bayer pattern
  • FIG. 3 is a flow chart illustrating one example of a method for color lookup using a mobile device
  • FIG. 4 is a flow chart illustrating another example of a method for color lookup using a mobile device.
  • FIG. 5 is a high level block diagram of the calibration method that is implemented using a general purpose computing device.
  • the present invention includes a method, apparatus, and non-transitory computer-readable medium for color lookup using a mobile device.
  • color matching may involve identifying a target color and subsequently reproducing the target color in a color mixture, i.e., so that any visual difference between the color mixture and the target color is minimized.
  • Accurate identification of the target color also referred to as “color lookup,” is therefore vital to the color matching process.
  • Many existing systems for color lookup include colorimeters that can be paired with a mobile device, such as a smart phone or a tablet computer, to assist users in identifying and communicating colors. However, depending on the circumstances, it may not always be convenient for the user to carry or use the colorimeter.
  • Examples of the present disclosure provide a method by which a mobile device, operating on its own, can provide the same color lookup functionality as a mobile device paired with a colorimeter.
  • the mobile device’s light emitting diode (LED) flashlight/flash may be used as a light source to illuminate the target color
  • the mobile device’s rear-facing camera may be used as the color sensor.
  • the ambient light may be overflooded by maintaining a maximum intensity of light emission by the flashlight while bringing the mobile device very close to an object of a target color.
  • the mobile device may then capture an image of the object, and an application (either installed and executing locally on the mobile device or hosted on a remote device such as a server) may process the image to estimate a first set of values that describes the target color in a first color space that approximates the perception of color by the human eye (e.g., the International Commission on Illumination (CIE) 1931 XYZ, or CIEXYZ, color space).
  • the application may further convert the first set of values to a second set of values that describes the target color in a second, different color space (e.g., the CIE L * a * b * , or CIELAB, color space).
  • the second set of values may be used to search an existing color database for a match (e.g., a closest color from a palette of colors).
  • color tristimulus values are understood to be values as defined by the CIE XYZ color space, e.g., in which Y represents luminance, Z is quasi-equal to blue, and X represents a mix of response curves chosen to be non-negative.
  • CIEXYZ values are linear in light intensity and thus are amenable to matrix-based estimation from camera values.
  • L * a * b * values are understood to be values in the CIELAB color space, e.g., in which color is expressed as three values: L * for the lightness from black (0) to white (100); a * from green (-) to red (+); and b * from blue (-) to yellow (+).
  • the CIELAB color space is considered more perceptually uniform than the CIEXYZ color space and is intended to approximate the perception of color by the human eye.
  • FIG. 1A illustrates an example system 100 in which examples of the present disclosure for color lookup using a mobile device may operate.
  • FIG. 1A illustrates an example system 100 in which examples of the present disclosure for color lookup using a mobile device may operate.
  • FIG. 1A illustrates an example system 100 in which examples of the present disclosure for color lookup using a mobile device may operate.
  • FIG. 1 B illustrates another example system in which examples of the present disclosure for color lookup using a mobile device may operate.
  • the components of FIG. 1 A and FIG. 1 B are the same; however, the orientation of the mobile device relative to the object whose color is being looked up differs as discussed in further detail below.
  • the system 100 may include one or more types of communication networks, including a packet network such as an Internet Protocol (IP) network (e.g., an IP Multimedia Subsystem (IMS) network), an asynchronous transfer mode (ATM) network, a wireless network, a cellular network (e.g., 2G, 3G, and the like), a long term evolution (LTE) network, 5G and the like, related to the current disclosure.
  • IP Internet Protocol
  • IMS IP Multimedia Subsystem
  • ATM asynchronous transfer mode
  • wireless network e.g., a cellular network
  • cellular network e.g., 2G, 3G, and the like
  • LTE long term evolution
  • 5G 5G and the like
  • the system 100 may comprise a network 102, e.g., a telecommunication service provider network, a core network, an enterprise network comprising infrastructure for computing and providing communications services of a business, an educational institution, a governmental service, or other enterprises (also referred to as the/a “cloud”).
  • a network 102 e.g., a telecommunication service provider network, a core network, an enterprise network comprising infrastructure for computing and providing communications services of a business, an educational institution, a governmental service, or other enterprises (also referred to as the/a “cloud”).
  • the core network 102 may be in communication with one or more access networks, such as access network 108.
  • the access network 108 may include a wireless access network (e.g., a WiFi network and the like), a mobile or cellular access network, a PSTN access network, a cable access network, a wired access network, or the like.
  • the core network 102 and the access network 108 may be operated by different service providers, the same service provider, or a combination thereof.
  • the core network 102 may include an application server (AS) 104 and a database (DB) 106.
  • the AS 104 may comprise a computing system or server, such as the computing system 500 depicted in FIG. 5, and may be configured to provide one or more operations or functions for color lookup, as described herein.
  • the AS 104 may be configured to obtain images or a target color from a mobile device, to measure the target color, and to identify a candidate color in a color library that most closely matches the candidate color.
  • the terms “configure,” and “reconfigure” may refer to programming or loading a processing system with computer-readable/computer-executable instructions, code, and/or programs, e.g., in a distributed or non-distributed memory, which when executed by a processor, or processors, of the processing system within a same device or within distributed devices, may cause the processing system to perform various functions.
  • Such terms may also encompass providing variables, data values, tables, objects, or other data structures or the like which may cause a processing system executing computer-readable instructions, code, and/or programs to function differently depending upon the values of the variables or other data structures that are provided.
  • a “processing system” may comprise a computing device including one or more processors, or cores (e.g., as illustrated in FIG. 5 and discussed below) or multiple computing devices collectively configured to perform various steps, functions, and/or operations in accordance with the present disclosure.
  • the AS 104 may be communicatively coupled to a database (DB) 106.
  • the DB 106 may store data that is used by the AS 104 to perform operations or functions for color lookup, as described herein.
  • the DB 106 may store data including a color library.
  • the color library may include a palette, or a plurality of different colors, where each color in the plurality of different colors may be considered a candidate for matching to a target color, as discussed in further detail below.
  • the color library may include, for each color included in the color library: a color name, a numerical color identifier, L*a*b* values describing the color, and or other identifying information (e.g., a source or manufacturer of the color, whether the color belongs to a curated collection of colors, etc.).
  • the color library may be provided by a manufacturer of a software application that is used for color lookup.
  • at least a portion of the color library may be provided by one or more manufacturers of commercial items.
  • the color library may include fan decks from paint manufacturers, textile manufacturers, and the like.
  • FIGs. 1 A and 1 B Although only a single application server (AS) 104 and a single database (DB) 106 are illustrated in FIGs. 1 A and 1 B, it should be noted that any number of servers and databases may be deployed. For instance, a plurality of servers and databases may operate in a distributed and/or coordinated manner as a processing system to perform operations for color lookup, in accordance with the present disclosure. For ease of illustration, various additional elements of network 102 are omitted from FIGs. 1A and 1 B.
  • the access network 108 may be in communication with one or more user endpoint devices, such as a mobile device 110.
  • the mobile device 110 may be any type of mobile endpoint device, such as a cellular telephone, a smart phone, a tablet computer, a laptop computer, a netbook, an ultrabook, a portable media device (e.g., an MP3 player), a portable gaming device, a digital media player, and the like, or even a wearable device such as a smart watch.
  • the mobile device 110 may be configured as a computer similar to that illustrated in FIG. 5 and described in greater detail below.
  • the mobile device 110 includes an integrated digital camera 114 and a light source (e.g., a light emitting diode or LED flashlight/flash) 116.
  • a light source e.g., a light emitting diode or LED flashlight/flash
  • the integrated digital camera 114 is a red, green, blue (RGB) camera comprising a two-dimensional (2D) array of photosensors and an array of red, green, and blue color filters deposited over the photosensors.
  • the color filters may be arranged in a “Bayer” pattern, i.e., such that each photosensor is covered by one color filter, and such that fifty percent of the color filters are green, twenty-five percent of the color filters are red, and twenty-five percent of the color filters are blue.
  • the larger number of green color filters in the Bayer pattern reflects the fact that the human eye is more sensitive to green light than to red light or blue light.
  • FIG. 2 is a schematic diagram illustrating one example of an array 200 of photosensors and a corresponding array 202 of colorfilters arranged in a Bayer pattern.
  • the array 200 of photosensors comprises a plurality of photosensors arranged in a square (e.g., N x N) array.
  • the array 202 of color filters Above the array 200 of photosensors is the array 202 of color filters.
  • the array 202 of color filters includes red, green, and blue color filters as noted in the legend.
  • a first row 204i of the array 202 of color filters may comprise a first pattern of alternating red and green filters.
  • a second row 204 2 of the array 202 of color filters, which is adjacent to the first row 204i may comprise a second pattern of alternating green and blue filters.
  • the first pattern and the second pattern may be alternated, row-by-row, in the array 202 of color filters as shown.
  • no green filter of the array 202 of color filters is positioned directly adjacent to another green filter.
  • the green filters may be positioned diagonally relative to each other.
  • the broadband visible light 118 may be filtered by the color filters prior to being collected by the photosensors. That is, the color filters will allow the corresponding colors in the broadband visible light 118 to pass through to the photosensors, but will block other colors of the broadband visible light 118. For instance, the red color filters will allow red light to pass through to the photosensors; the green color filters will allow green light to pass through to the photosensors; and the blue color filters will allow blue light to pass through to the photosensors.
  • the integrated digital camera 114 may be color calibrated prior to being used to perform any sort of color lookup. To calibrate the integrated digital camera 114, the integrated digital camera 114 may be used to capture images of a set of m training colors (i.e. , colors whose color tristimulus values are already known).
  • the raw counts i.e., the numbers of pixels of each color in the image
  • the raw counts for each color filter may be averaged to create three averaged raw counts: a red channel averaged raw count d R , a green channel averaged raw count do, and a blue channel averaged raw count d B .
  • a single color measurement D made by the integrated digital camera, may be defined as a 3 x m matrix D, where:
  • a 3 x m matrix RT may be constructed to contain the known color tristimulus values of the m training colors (under a predefined standard illuminant and CIE color match functions).
  • the 3 x 3 matrix M and the offset 1 x m vector b may be estimated, where M and b map the matrix D as closely as possible to the matrix RT, such that:
  • EQN. 2 may be further expressed as a homogenous equation, such that:
  • RT M A DA (EQN. 3).
  • D’ is the transpose of the matrix D.
  • the mobile device 110 may be calibrated in the factory, e.g., by the mobile device’s manufacturer.
  • the calibration matrix may be stored either in the local memory of the mobile device 110, or at a remote server.
  • the set of m training colors may be printed on a paper card or sheet (e.g., a five inch by seven inch card or sheet, on which each training color may be printed in at least a one inch by one inch square for easy alignment during measurement).
  • a paper card or sheet of this nature may allow the calibration matrix to be easily re-generated by the end user after the mobile device 100 leaves the factory (e.g., to compensate for long-term drift of the integrated digital camera 114 and/or light source 116), or may allow the end user to generate the calibration matrix for the first time in the field (e.g., where the mobile device 100 was not calibrated in the factory).
  • the paper card or sheet may include a quick response (QR) code or other machine-readable code that, when scanned, will cause the color tristimulus values of the training colors to be automatically downloaded (e.g., from a remote server or from the local memory of the mobile device 110).
  • QR quick response
  • the mobile device 110 may skip the calibration but use an averaged color calibration matrix that is generated by the manufacturer of the phone or of the software application that performs the color lookup.
  • a plurality of mobile devices of the same make and model as the mobile device 110 may be used as master instruments.
  • the averaged color calibration matrix for each mobile device mark and model may be stored in the software application (which may be subsequently loaded onto the mobile device, as discussed below) or at a remote server.
  • the use of the m training colors for calibration comprises a linear correction.
  • an abbreviated correction of the mobile device 110 may be performed in place of or in addition to a full re-calibration.
  • the abbreviated correction may also be linear, but may be simpler than a full re-calibration, as the abbreviated correction does not require a large set of training colors.
  • the abbreviated correction may utilize a white patch and a black patch in the mobile device 110, along with a pre-loaded color calibration matrix that corresponds to the make and model of the mobile device 110.
  • the abbreviated correction may compensate for variation of the integrated digital camera’s sensitivity and/or variation of the light source’s flash intensity from mobile device to mobile device (of the same make and model).
  • a patch of black and patch of white may be printed on a small paper sheet or card, similar to the set of test colors described above.
  • a white sample and a black sample provided from a commercial fandeck or provided by a provider of the software application that performs the color lookup may be used.
  • the raw counts of the black color and the white color from the red, green, and blue channels may be measured using master instruments of the same make and model as the mobile device 110 (e.g., by the manufacturer of the mobile device 110 or by the manufacturer of the software application).
  • [d R,Wht> d G,wht> d- B,wht ⁇ may express the raw counts of the white color as measured by the master instruments, while [d R blk , d G k , d B blk ] may express the raw counts of the black color as measured by the master instruments. These raw counts may be saved in the local memory of the mobile device 110 or on a remote server. [0041] An end user may subsequently measure the same black and white samples using the mobile device 110.
  • [d R wht , d G wht , d B wht ] may express the raw counts of the white color as measured by the mobile device 110, while [d R,bik ,d G,bik ,d B,bi ⁇ may express the raw counts of the black color as measured by the mobile device 110.
  • a linear mapping that will bring the raw counts of the red channel from the mobile device 110 closer to the raw counts of the red channel from the master instruments may be expressed as:
  • the full set of linear correction coefficients may be saved in the local memory of the mobile device 110 or on a remote server.
  • the raw counts generated from an arbitrary measurement may first be corrected according to:
  • the corrected raw counts may subsequently be converted into the color tristimulus values in accordance with EQN. 9, which is discussed in further detail in connection with FIG. 3.
  • the averaged color conversion matrix that is generated using the plurality of master instruments may be used to convert the corrected raw counts into the color tristimulus values.
  • the linear correction (using m training colors) and the abbreviated correction (using black and white samples) may be performed on the same system, i.e., the linear correction and the abbreviated correction are not mutually exclusive, but may work together.
  • the linear correction using m training colors may be applied to a master instrument (e.g., mobile device), while the abbreviated correction using the black and white samples may be applied at least once to each individual device of the same make and model as the master instrument.
  • a master instrument e.g., mobile device
  • the abbreviated correction using the black and white samples may be applied at least once to each individual device of the same make and model as the master instrument.
  • the mobile device 110 may be configured to host an application that communicates with the AS 104 and/or DB for performing color lookup.
  • the application may guide a user of the mobile device 110 through a process whereby the integrated digital camera 114 is used to capture an image of an object 112 of a target color (where the light source 116 is used to illuminate the object 112 during image capture).
  • the application may further guide the user through steps by which the mobile device 110 is used to measure the target color (e.g., identify color tristimulus values or L * a * b * values of the target color) in the image of the object 112.
  • the application may further guide the user through steps whereby the mobile device 110 is used to perform a search in a color library connected to the mobile device 110 (e.g., database 106), where the search identifies a candidate color in the color library that most closely matches the target color.
  • a color library connected to the mobile device 110 (e.g., database 106)
  • the search identifies a candidate color in the color library that most closely matches the target color.
  • the application that is hosted on the mobile device 100 may guide the user through the steps of capturing the image of the object 112. The application may then guide the user through steps whereby the image is transmitted from the mobile device 110 to a remote server (e.g., AS 104), where the remote server measures the target color in the image, performs the search of the color library, and transmits a result of the search back to the mobile device 110.
  • a remote server e.g., AS 104
  • system 100 has been simplified. Thus, it should be noted that the system 100 may be implemented in a different form than that which is illustrated in FIGs. 1 A and 1 B without departing from the scope of the present disclosure.
  • FIG. 3 is a flow chart illustrating one example of a method 300 for color lookup using a mobile device.
  • the method 300 may be performed, for instance, by the mobile device 110 of FIGs. 1A and 1 B and/or by another computing device that is communicatively coupled to the mobile device 110.
  • the method 500 may be performed by a processor of a computing device, such as the processor 502 illustrated in FIG. 5.
  • the method 300 is described below as being performed by a processing system.
  • the method 300 begins in step 302.
  • the processing system may receive a request to perform a color lookup.
  • the request may be received from an application that is installed on a user’s mobile device.
  • GUI graphical user interface
  • the processing system may adjust the settings of an integrated digital camera of the mobile device for color lookup.
  • the settings that are adjusted may include shutter speed, sensitivity of the image sensor to light (e.g., ISO), and image format, among others. These settings may be adjusted through the mobile device’s application programming interface (API).
  • API application programming interface
  • the optimal settings for color lookup may vary depending upon the make and model of the mobile device.
  • the ISO may be set to the smallest value possible (e.g., fifteen) in order to minimize system noise.
  • the shutter speed may be set so that the raw counts obtained from a white color sample are approximately two thirds of the maximum possible count (e.g., 1/800 seconds), in order to minimize saturation of the integrated digital camera.
  • Autofocus functions may be disabled or set to zero.
  • the processing system may generate a first signal to activate a light source of the mobile device.
  • the mobile device may be a smart phone or a tablet computer, as discussed above, and the light source may be an LED flash or flashlight of the smart phone or tablet computer.
  • the first signal activates the light source at the light source’s highest intensity setting, such that the light source continues to emit light at the highest intensity setting until a subsequent signal is received to deactivate the light source.
  • the first signal to may include a signal to deactivate the auto-tuning features.
  • the processing system may provide instructions to the user to position the mobile device to acquire an image of a target color (e.g., a color that the user may be trying to identify via the color lookup). For instance, the processing system may track the position of the mobile device relative to an object of the target color via a “live image,” using the integrated digital camera. Based on the live image, the processing system may provide instructions to the user (e.g., text instructions, audible instructions, an image overlay or box within which to position an image of the object, etc.) to adjust the position of the mobile device for optimal image quality.
  • a target color e.g., a color that the user may be trying to identify via the color lookup.
  • the processing system may track the position of the mobile device relative to an object of the target color via a “live image,” using the integrated digital camera. Based on the live image, the processing system may provide instructions to the user (e.g., text instructions, audible instructions, an image overlay or box within which to position an image of the object, etc.) to adjust the
  • the optimal position ensures that a distance between the mobile device and the object (e.g., a distance between the integrated digital camera 114 of the mobile device 110 and the object 112, as indicated by l_2 in FIG. 1A) is approximately equal to the distance between the integrated digital camera and a light source (e.g., an LED flashlight) of the mobile device (indicated as Li in FIG. 1 A).
  • This positioning forms a 45/0 illumination geometry (e.g., where Q in FIG. 1A equals forty-five degrees). Flowever, in other examples, the positioning is not defined by a 45/0 geometry.
  • L2 may be determined by detecting a specular “hot spot” in the live image.
  • the hot spot is an artifact created by the specular reflection of the mobile device’s light source from a glossy object. Specifically, a portion of the light that is emitted by the light source will be directly reflected by the object, will enter the integrated digital camera, and will show as a very bright spot (or “hot spot”) in the resulting image.
  • the location and the size of the area on the object that would directly reflect the emitted light to form the hot spot does not tend to change as the object is moved in a direction that is normal to the integrated digital camera’s lens. Flowever, as the object is brought closer to the mobile device, the integrated digital camera’s field of view shrinks. As a result, the hot spot may appear to move toward the edge of the image and may appear larger.
  • the hot spot’s size and location may be the same for mobile devices of the same make and model. Conversely, the hot spot’s size and location may vary among different makes and models of mobile devices. Thus, determining the optimal location of the hot spot may allow one to define the optimal distance between the mobile device and the object. In other words, for a mobile device of a given make and model, the size and location of the hot spot is closely correlated with the distance between the mobile device and the object. Moreover, this correlation is invariant to the color of the object.
  • the processing system may instruct the user to hold the mobile device parallel to the object and to move the mobile device closer to the object (while still maintaining the parallel relative orientation as shown in FIG. 1 A) while the processing system observes the resulting live image.
  • the hot spot if shown, will move toward the edge of the image as the distance between the mobile device and the object shrinks, as discussed above.
  • the hot spot reaches the known optimal location (which may be stored for the particular make and model of mobile device in application software, the mobile device local memory, or on a remote server), this may indicate that the distance between the mobile device and the object is optimal for color lookup.
  • the mobile device may be equipped with a specially designed case (e.g., a mobile phone case) that holds the mobile device in a parallel orientation relative to the object of the target dolor, while also defining a distance between the camera and the object of the target color that is sufficient to achieve the optimal position.
  • the case may have a thickness that is sufficient to create a gap between the integrated digital camera of the mobile device and the object of the target color when the mobile device is resting on the object of the target color. The size of the gap may be sufficient to create the 45/0 illumination geometry shown in FIG. 1A.
  • the optimal position may not be defined by a 45/0 geometry.
  • the optimal distance between the mobile device and the object may be determined by placing the mobile device on the target at a predefined angle a as shown in FIG. 1 B (where the predefined angle a may vary according to the particular make and model of the mobile device). This allow the optimum distance between the mobile device and the object to be maintained indirectly.
  • the predefined angle a for the make and model of the mobile device may be looked up (e.g., in application software in, the mobile device local memory, or on a remote server).
  • the processing system may instruct the user to hold the mobile device near the object (e.g., rest the end of the mobile device closest to the integrated digital camera against the object at a small angle such as between zero to twenty degrees, with the camera pointing at the object), and may observe the angle between the mobile device and the object using an angle-detecting sensor (e.g., an accelerometer) API of the mobile device.
  • an angle-detecting sensor e.g., an accelerometer
  • the processing system may instruct the user to adjust the angle (e.g., move the edge of the mobile device furthest from the integrated digital camera up or down, while keeping the top rear edge of the mobile device firmly against the object) until the predefined angle a is achieved (e.g., to within a threshold tolerance, such as plus or minus three degrees).
  • the angle e.g., move the edge of the mobile device furthest from the integrated digital camera up or down, while keeping the top rear edge of the mobile device firmly against the object
  • the predefined angle a e.g., to within a threshold tolerance, such as plus or minus three degrees.
  • the processing system may generate a second signal to capture and save an image (e.g., a still image) of the object of the target color, using the integrated digital camera of the mobile device.
  • the integrated digital camera is a rear-facing camera (e.g., a camera whose lens is facing in an opposite direction from the display of the mobile device).
  • the processing system may automatically generate the second signal in response to the processing system detecting (e.g., based on communication with the angle-detecting sensor) that the angle between the mobile device and the object is equal to (or within a predefined tolerance with respect to) the predefined angle a that indicates a specified distance for image capture.
  • the image is saved in an open standard raw image file format to maintain the linearity and consistency of the image signal.
  • File formats that involve heavy post-image processing may disturb the linearity and consistency of the image signal.
  • the raw image file format may be any proprietary or non proprietary raw image file format.
  • One example of a suitable raw image file format is the Adobe Digital Negative Raw Image (DNG) file format.
  • DNG Adobe Digital Negative Raw Image
  • the processing system may generate a third signal to deactivate (i.e. , turn off) the light source.
  • the light source is activated (i.e., turned on) before the image capture of the target color in step 312.
  • the light source remains activated throughout image capture and is not deactivated until image capture is complete.
  • the ambient light surrounding the object of the target color can be overflooded. Overflooding the ambient light may lessen the effects of the ambient light on the performance of the method 300 and may also reduce the amount of time required to accurately measure the target color.
  • step 316 the processing system may extract the raw counts from the red, green, and blue channels of the image. If the integrated digital camera is configured as described above (e.g., in connection with FIGs. 1 and 2), then the image captured in step 312 will comprise a 2D image in which each pixel of the image represents exactly one of the three colors (i.e., red, green, or blue).
  • the raw counts from the red, green, and blue channels are extracted in step 316 as follows.
  • the middle one third of the image is selected or cropped to produce a region of interest in which the effective illumination angle distribution is reduced.
  • selecting the middle one third of the image as the region of interest may reduce the effective illumination angle distribution from a first range of approximately thirty-six to fifty-seven degrees to a second range of approximately forty to fifty degrees. Reducing the effective illumination angle distribution may collimate the effective illumination beam, which may facilitate more accurate color measurement of the target color.
  • excluding the pixels at the edges of the image from the region of interest may also reduce light scattering and hazing introduced by the lens of the integrated digital camera.
  • the raw counts i.e. , the numbers of pixels of each color in the region of interest
  • the raw counts may be averaged to create three averaged raw counts: a red channel averaged raw count d R , a green channel averaged raw count dc, and a blue channel averaged raw count de.
  • a single color measurement D made by the integrated digital camera, may be defined as:
  • the processing system may calculate a first color tristimulus value, a second color tristimulus value, and a third color tristimulus value for the red channel averaged raw count d R , the green channel averaged raw count dc, and the blue channel averaged raw count d B , respectively, thereby generating a plurality of color tristimulus values.
  • M A is a 3 x4 color calibration matrix that converts the averaged raw counts (3 x 1 ) to the corresponding color tristimulus values (3 x 1 ).
  • the color calibration matrix MA may be generated from the process by which the integrated digital camera is calibrated, discussed above. In this case, the averaged raw counts for all of the training colors can be combined into a 3 x m matrix, D.
  • the 3 x m matrix RT contains the known color tristimulus values of the training colors (under a predefined standard illuminant and CIE color match function).
  • the processing system may calculate a plurality of L * a * b * values for the target color from the plurality of color tristimulus values, using a predefined illuminant.
  • the predefined illuminant may be the CIE Standard Illuminant D65.
  • the processing system may identify a candidate color in a color library whose L * a * b * values most closely match the L*a*b* values of the target color.
  • a predefined number of the closest matching candidate colors may be identified (e.g., where the predefined number may equal three).
  • the color library may be stored locally on the same device as the processing system. In another example, the color library may be stored on a remote database that is accessible to the processing system via a network.
  • Steps 316-322 are optional, because in one example, steps 316-322 may be performed by another device. For instance, if the processing system is part of the mobile device, steps 316-322 could be performed locally on the mobile device or could be performed remotely, e.g., by a server (as discussed in connection with FIG. 4).
  • the processing system may display the closest matching candidate color (or the predefined number of the closest matching candidate colors) on a display of the mobile device. For instance, the display may show patches of the matching candidate color(s) or may identify the name(s), code(s), or other unique identifier(s) of the matching candidate color(s).
  • the closest matching candidate color(s) may also be saved (either locally on the mobile device or on a remote server) for future reference.
  • the method 300 may end in step 326.
  • FIG. 4 is a flow chart illustrating another example of a method 400 for color lookup using a mobile device.
  • the method 400 may be performed, for instance, by the application server 104 of FIGs. 1A and 1 B.
  • the method 400 may be performed by a processor of a computing device, such as the processor 502 illustrated in FIG. 5.
  • the method 400 is described below as being performed by a processing system.
  • the method 400 begins in step 402.
  • the processing system may acquire an image of an object of a target color, where the image was captured by an integrated digital camera of a remote mobile device that is in communication (e.g., over a network) with the processing system.
  • the integrated digital camera is configured as described above (e.g., in connection with FIGs. 1 and 2), then the image acquired in step 402 will comprise a 2D image in which each pixel of the image represents exactly one of the three colors (i.e., red, green, or blue).
  • the processing system may extract the raw counts from the red, green, and blue channels of the image of the object.
  • the raw counts from the red, green, and blue channels are extracted in step 406 as follows.
  • the middle one third of the image is selected or cropped to produce a region of interest in which the effective illumination angle distribution is reduced. For instance, selecting the middle one third of the image as the region of interest may reduce the effective illumination angle distribution from a first range of approximately thirty-six to fifty-seven degrees to a second range of approximately forty to fifty degrees. Reducing the effective illumination angle distribution may collimate the effective illumination beam, which may facilitate more accurate color measurement of the target color.
  • excluding the pixels at the edges of the image from the region of interest may also reduce light scattering and hazing introduced by the lens of the integrated digital camera.
  • the raw counts i.e., the numbers of pixels of each color in the region of interest
  • the raw counts may be averaged to create three averaged raw counts: a red channel averaged raw count d R , a green channel averaged raw count do, and a blue channel averaged raw count d B .
  • a single color measurement D made by the integrated digital camera, may be defined according to EQN. 8 (above).
  • the processing system may calculate a respective color tristimulus value for each of the red channel averaged raw count d R , the green channel averaged raw count do, and the blue channel averaged raw count d B , thereby generating a plurality of color tristimulus values.
  • the processing system may calculate a plurality of L*a*b* values for the target color from the plurality of color tristimulus values, using a predefined illuminant.
  • the predefined illuminant may be the International Commission of Illumination (CIE) Standard Illuminant D65.
  • the processing system may identify a candidate color in a color library whose L * a * b * values most closely match the L * a * b * values of the target color.
  • a predefined number of the closest matching candidate colors may be identified (e.g., where the predefined number may equal three).
  • the processing system may transmit data about the closest matching candidate color (or the predefined number of the closest matching candidate colors) to the mobile device.
  • the data may include patches of the matching candidate color(s) or may identify the name(s), code(s), or other unique identifier(s) of the matching candidate color(s).
  • the method 400 may end in step 416.
  • the method 300 represents an example in which the color lookup process may be performed entirely by a single device (e.g., the mobile device).
  • the mobile device may acquire the image of the object of the target color, measure the target color and make any necessary conversions, and perform the lookup in the color library (which may be stored locally or remotely) to identify the closest matching candidate colors.
  • the method 400 represents an example in which the color process may be performed in a distributed manner by two or more devices (e.g., the mobile device and a remote server).
  • the mobile device may acquire the image of the object of the target color, while the remote server may measure the target color and make any necessary conversions, and may perform the lookup in the color library (which may be stored locally or remotely) to identify the closest matching candidate colors.
  • the method 400 may allow the more processing intensive operations to be performed by a device having greater processing capabilities than the mobile device. This may allow results to be obtained more quickly and may also free up memory and processing on the mobile device for other applications.
  • the method 300 or 400 may be expanded to include additional steps or may be modified to include additional operations with respect to the steps outlined above.
  • one or more steps, functions, or operations of the method 300 or 400 may include a storing, displaying, and/or outputting step as required for a particular application.
  • any data, records, fields, and/or intermediate results discussed in the method can be stored, displayed, and/or outputted either on the device executing the method or to another device, as required for a particular application.
  • steps, blocks, functions or operations in FIG. 3 or 4 that recite a determining operation or involve a decision do not necessarily require that both branches of the determining operation be practiced.
  • one of the branches of the determining operation can be deemed as an optional step.
  • steps, blocks, functions or operations of the above described method can be combined, separated, and/or performed in a different order from that described above, without departing from the examples of the present disclosure.
  • FIG. 5 is a high level block diagram of the color lookup method that is implemented using a computing device 500.
  • a general purpose computing device 500 comprises a processor 502, a memory 504, a color lookup module 505 and various input/output (I/O) devices 506 such as a display, a keyboard, a mouse, a modem, a network connection and the like.
  • I/O devices 506 such as a display, a keyboard, a mouse, a modem, a network connection and the like.
  • at least one I/O device is a storage device (e.g., a disk drive, an optical disk drive, a floppy disk drive).
  • the color lookup module 505 can be implemented as a physical device or subsystem that is coupled to a processor through a communication channel.
  • the color lookup module 505 can be represented by one or more software applications (or even a combination of software and hardware, e.g., using Application Specific Integrated Circuits (ASIC)), where the software is loaded from a storage medium (e.g., I/O devices 506) and operated by the processor 502 in the memory 504 of the general purpose computing device 500. Additionally, the software may run in a distributed or partitioned fashion on two or more computing devices similar to the general purpose computing device 500.
  • ASIC Application Specific Integrated Circuits
  • the color lookup module 505 for calibrating a multi-channel color measurement instrument in the field described herein with reference to the preceding Figures can be stored on a computer readable medium or carrier (e.g., RAM, magnetic or optical drive or diskette, and the like).
  • a computer readable medium or carrier e.g., RAM, magnetic or optical drive or diskette, and the like.
  • one or more steps of the methods described herein may include a storing, displaying and/or outputting step as required for a particular application.
  • any data, records, fields, and/or intermediate results discussed in the methods can be stored, displayed, and/or outputted to another device as required for a particular application.
  • steps or blocks in the accompanying Figures that recite a determining operation or involve a decision do not necessarily require that both branches of the determining operation be practiced. In other words, one of the branches of the determining operation can be deemed as an optional step.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Evolutionary Computation (AREA)
  • Signal Processing (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Image Processing (AREA)

Abstract

Selon l'invention un système de traitement d'un dispositif mobile acquiert une image d'un objet d'une couleur cible, l'image ayant été capturée par une caméra numérique intégrée du dispositif mobile, calcule une première pluralité de valeurs qui décrivent la couleur cible, et le calcul étant fondé sur une analyse d'un pixel de l'image, et identifie une première couleur candidate parmi une pluralité de couleurs candidates, chaque couleur candidate dans la pluralité de couleurs candidates étant associée à un second ensemble de valeurs qui décrit chaque couleur candidate, et le second ensemble de valeurs décrivant la première couleur candidate correspond au premier ensemble de valeurs de manière plus proche que n'importe quel second ensemble de valeurs associées à une autre couleur candidate de la pluralité de couleurs candidates.
EP19947603.7A 2019-10-02 2019-10-02 Procédé et appareil pour une recherche de couleur à l'aide d'un dispositif mobile Pending EP4038517A4 (fr)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2019/054306 WO2021066822A1 (fr) 2019-10-02 2019-10-02 Procédé et appareil pour une recherche de couleur à l'aide d'un dispositif mobile

Publications (2)

Publication Number Publication Date
EP4038517A1 true EP4038517A1 (fr) 2022-08-10
EP4038517A4 EP4038517A4 (fr) 2023-07-05

Family

ID=75338495

Family Applications (1)

Application Number Title Priority Date Filing Date
EP19947603.7A Pending EP4038517A4 (fr) 2019-10-02 2019-10-02 Procédé et appareil pour une recherche de couleur à l'aide d'un dispositif mobile

Country Status (3)

Country Link
EP (1) EP4038517A4 (fr)
CN (1) CN114503097A (fr)
WO (1) WO2021066822A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117369546A (zh) * 2023-11-14 2024-01-09 广东交通职业技术学院 一种墙漆颜色采集配色系统、方法及智能涂料刷

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7382405B2 (en) * 2001-12-03 2008-06-03 Nikon Corporation Electronic apparatus having a user identification function and user identification method
US7248284B2 (en) * 2002-08-12 2007-07-24 Edward Alan Pierce Calibration targets for digital cameras and methods of using same
ES2323129T3 (es) * 2003-09-10 2009-07-07 Qualcomm Incorporated Interfaz de alta velocidad de datos.
EP2116896B1 (fr) * 2008-05-09 2013-05-01 Research In Motion Limited Procédé et système de fonctionnement d'un flash de caméra sur un dispositif mobile
US8488055B2 (en) 2010-09-30 2013-07-16 Apple Inc. Flash synchronization using image sensor interface timing signal
US10210369B2 (en) 2010-12-23 2019-02-19 Cognex Corporation Mark reader with reduced trigger-to-decode response time
US9436704B2 (en) 2012-02-07 2016-09-06 Zencolor Corporation System for normalizing, codifying and categorizing color-based product and data based on a universal digital color language
CN111557023A (zh) * 2017-12-29 2020-08-18 Pcms控股公司 使用公共对象维持颜色校准的方法和系统

Also Published As

Publication number Publication date
WO2021066822A1 (fr) 2021-04-08
CN114503097A (zh) 2022-05-13
EP4038517A4 (fr) 2023-07-05

Similar Documents

Publication Publication Date Title
US10902643B2 (en) Color detection algorithm
RU2567500C2 (ru) Способ и система для определения цвета из изображения
US11740132B2 (en) Method and apparatus for color lookup using a mobile device
EP2227898B1 (fr) Appareil de détection d'image et procédé d'estimation de la source d'éclairage d'une scène
US8313030B2 (en) System and method for barcode scanning with color image sensors
EP3888345B1 (fr) Procédé de génération de données d'image pour des algorithmes d'imagerie basés sur l'apprentissage machine
US20080212874A1 (en) Method for Spectral Integrated Calibration of an Image Sensor by Means of a Monochromatic Light Source
KR20220049582A (ko) 주변 조명을 특징짓기 위한 시스템들
US20120118969A1 (en) System and method for barcode scanning using image calibration
Charrière et al. Color calibration of an RGB camera mounted in front of a microscope with strong color distortion
CN107247384B (zh) 亮度补偿数据获取系统及方法、图像亮度调整方法及装置
JP4967490B2 (ja) プロファイル作成システム
US10194035B2 (en) Imager calibration via modeled responses to importance-weighted color sample data
EP4038517A1 (fr) Procédé et appareil pour une recherche de couleur à l'aide d'un dispositif mobile
EP3993382B1 (fr) Étalonnage de la couleur d'un dispositif d'imagerie
Nikkanen et al. Subjective effects of white-balancing errors in digital photography
Nyström Colorimetric and multispectral image acquisition
US20170082492A1 (en) Device for measuring colour properties
Nikkanen Color constancy by characterization of illumination chromaticity
US11573748B2 (en) Information processing apparatus and information processing system to control emitting light based on illumination information
JP2011015298A (ja) 色変換方法、プログラム、及び画像処理装置

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20220427

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230523

A4 Supplementary search report drawn up and despatched

Effective date: 20230602

RIC1 Information provided on ipc code assigned before grant

Ipc: G06V 10/75 20220101ALI20230526BHEP

Ipc: H04M 1/72403 20210101ALI20230526BHEP

Ipc: G06Q 30/06 20120101ALI20230526BHEP

Ipc: G06F 17/40 20060101ALI20230526BHEP

Ipc: G06F 16/58 20190101ALI20230526BHEP

Ipc: G06F 16/50 20190101AFI20230526BHEP