WO2024073869A1

WO2024073869A1 - Apparatus and method for acquiring surface multi-point chromaticity coordinate values of photographed object

Info

Publication number: WO2024073869A1
Application number: PCT/CN2022/123695
Authority: WO
Inventors: 施霖; 刘天屹
Original assignee: 施霖
Priority date: 2022-10-04
Filing date: 2022-10-04
Publication date: 2024-04-11

Abstract

The present application mainly relates to the fields of color vision, colorimetry, color communication, image processing, computer control, deep learning, applied mathematics, etc. Provided are an apparatus and method for acquiring surface multi-point chromaticity coordinate values of a photographed object. By means of the apparatus and method for acquiring an image of a photographed object together with a color block group under irradiation of light sources in different spectral distribution and then calculating surface multi-point chromaticity coordinate values of the photographed object, the problems of the existing means for carrying out measurement of surface multi-point chromaticity coordinate values of an photographed object being difficult and inaccurate and having a high cost are solved.

Description

Device and method for obtaining multi-point chromaticity coordinate values on the surface of a photographed object

Technical Field

The present invention relates to the fields of color vision, chromaticity, color communication, image processing, computer control, deep learning, and applied mathematics, and in particular to a device and method for obtaining multi-point chromaticity coordinate values on the surface of a photographed object.

Background technique

Accurately, quickly and at low cost, obtaining multi-point chromaticity coordinate values on the surface of the object is an important basis for color communication. Currently, colorimeters or spectrophotometers are mainly used to obtain the chromaticity coordinate values of the object surface. Each time, only a single-point chromaticity coordinate value can be obtained, and the measurement is easily affected by the three-dimensional shape of the surface of the object, resulting in inaccurate measurement results. Although a hyperspectral camera can be used to obtain the multi-point spectral distribution of the surface of the object to be measured, and then calculate the chromaticity coordinate value of each point, the high price of hyperspectral cameras limits their application. Using an ordinary color camera can only obtain a three-channel color image of the object to be measured, but cannot accurately obtain the multi-point chromaticity coordinate values of the surface of the object to be measured.

Summary of the invention

The present invention provides: a device and a method for using an image sensor to capture a group image of a subject and a color block group under the illumination of a multi-spectral distribution light source, and then using the group image to calculate the chromaticity coordinate values of multiple points on the surface of the subject, thereby solving the problems of difficulty in measuring the chromaticity coordinate values of multiple points on the surface of the subject, inaccurate measurement, and high measurement cost.

The present invention relates to various aspects of devices and methods. Embodiments of the present invention may include any combination of one or more of the various aspects described herein.

In a first aspect, a device for obtaining chromaticity coordinate values of multiple points on the surface of a photographed object is provided, comprising: a light source group, a color block group, a control unit, an imaging unit, an image segmentation and alignment unit, and a chromaticity coordinate value calculation unit, wherein the light source group includes multiple light sources with different spectral distributions, and the color block group includes multiple color blocks with different reflection spectral distributions, the control unit controls the light source group to use light sources with different spectral distributions to illuminate the photographed object and the color block group in different time periods, and controls the imaging unit to synchronously obtain multiple group images of the photographed object and the color block group under the illumination of light sources with different spectral distributions in corresponding time periods, the image segmentation and alignment unit segments an image containing a single photographed object and an image containing a single color block from the group image, and aligns the images of the same subject and the same color block obtained under different light source illumination conditions, respectively, and the chromaticity coordinate value calculation unit calculates the chromaticity coordinate values of multiple points on the surface of the photographed object using a regression model.

In a second aspect, a method for obtaining multi-point chromaticity coordinate values on the surface of a photographed object is provided, wherein a group image of the object and a color block group including multiple color blocks with different reflective spectral distributions under illumination by light source groups with different spectral distributions is obtained, an image including a single subject and an image including a single color block are segmented from the group image, images of the same subject and the same color block obtained under illumination by different light sources are respectively aligned, and the chromaticity coordinate value calculation unit calculates the multi-point chromaticity coordinate values on the surface of the object using a regression model.

The half-width range near the peak wavelength of the light source spectrum distribution of at least part of the light source groups overlaps with the half-width range near the sensitive peak wavelength of three types of human cone cells, namely, S-type, M-type and L-type.

The color block group includes a plurality of color blocks, and these color blocks have different reflective spectral distributions in the half-width range near the sensitive peak wavelength of three types of human cone cells, namely, S-type, M-type and L-type.

The calculation steps of the chromaticity coordinate value calculation unit include:

First, a regression model is established, using the color channel component values of the color block images obtained under the illumination of light sources with different spectral distributions as model input data, and the chromaticity coordinate values of the corresponding color blocks as model output annotation data, and using the color channel component values of the subject images obtained under the illumination of light sources with different spectral distributions as model input data, and the chromaticity coordinate values of the corresponding positions of the corresponding subjects obtained by other means as model output annotation data, training the regression model, and determining the regression model parameters;

Then, the trained regression model is used to input the color component values of the image of the object obtained under the illumination of light sources with different spectral distributions, and the chromaticity coordinate values corresponding to the object are output.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention may be better understood through the following detailed description of preferred but non-limiting embodiments in conjunction with the accompanying drawings, in which:

FIG1 is a schematic diagram showing a group image of a subject and a group of color blocks taken under the illumination of a multi-spectral light source group;

FIG2 is a schematic diagram showing an image segmentation and alignment unit segmenting and aligning a composite image of a photographed object and a color block group;

FIG3 is a schematic diagram of a chromaticity coordinate calculation unit;

FIG4 shows the spectral distribution curve of the multi-spectral light source group and the spectral sensitivity curves of three types of human cone cells: S-type, M-type, and L-type;

FIG5 shows the reflectance spectrum distribution curve of the color block group and the spectral sensitivity curves of three types of human cone cells: S-type, M-type, and L-type;

FIG6 is a schematic diagram of the structure of a neural network regression model.

Detailed ways

In order to make the technical problems and technical solutions solved by the present invention clearer, the technical solutions in the embodiments of the present invention are further clearly and completely described below in conjunction with the drawings and embodiments of the specification. The described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The description herein is only used to explain the present invention and is not intended to limit the present invention. For the sake of clarity, some features that are not particularly important for understanding the present invention or are obvious to those skilled in the relevant art may not be shown.

This embodiment uses multiple teeth as the photographing objects to illustrate an embodiment of the invention.

The light source groups in the embodiments are all implemented using light emitting diodes. In the embodiments, nine LEDs emitting different colors of light are used, namely: red, cold white, medium white, purple, warm white, green, yellow, white, and blue LEDs. In the embodiments, taking the example of photographing multiple tooth objects to obtain multi-point chromaticity coordinate values on the tooth surface is used for illustration. In the embodiment, the color block group includes 24 color blocks. In the embodiment, the image captured by the image sensor is a three-channel color image.

1 , the light source group 101 includes a plurality of light sources with different spectral distributions, and the control unit 102 controls the light source group 101 to use light 107 emitted by light sources with different spectral distributions to illuminate a subject 105 and a color block group 104 in different time periods. The color block group 104 includes a plurality of color blocks 106 with different wavelength spectral absorption rates, and the control unit 102 controls the imaging unit 103 to synchronously acquire a plurality of group images of the subject and the color block group under the illumination of light sources with different spectral distributions in corresponding time periods.

The control unit may be implemented by a general microcontroller, the imaging unit may be implemented by a general color CMOS image sensor and an optical lens, and the control unit may use a general relay to control the on and off of the LED light source.

2 , the group image 201 captured by the imaging unit under the illumination of light sources with different spectral distributions is processed by the image segmentation and alignment unit 202, which first segments the color block group image 203 and the object image 205, and then further segments the image 206 containing a single object and the image 204 containing a single color block, and aligns the images of the same object and the same color block obtained under different light source illumination conditions.

Currently known image instance segmentation methods based on deep neural networks, such as Mask RCNN, can be used to segment the group image into a color block group image and a subject image and further segment the group image into an image containing a single subject and an image containing a single color block.

First, take a group photo of 50 subjects' full-mouth teeth and color blocks from the front, and use a known image annotation tool to annotate the outline of a single tooth and the number of the tooth in the group photo, and also annotate the outline of a single color block and the number of the color block. Based on the industry-known pre-trained Mask RCNN network, use the acquired single tooth image, single color block image, and corresponding annotation data, and continue to train the image instance segmentation deep neural network Mask RCNN using a known method to obtain the trained neural network. Use the trained neural network to perform image instance segmentation on the group photo of teeth and color blocks, and obtain a single tooth image and a single color block image.

After segmentation, images of the same instance under different lighting conditions can be aligned using their geometric centers as reference points under different lighting conditions.

3 , the chromaticity coordinate value calculation unit 301 first uses the color channel component values of the color block image 303 obtained under the illumination of light sources with different spectral distributions as model input data, the chromaticity coordinate value 302 of the corresponding color block as model output annotation data, and uses the color channel component values of the subject image 304 obtained under the illumination of light sources with different spectral distributions as model input data, and the chromaticity coordinate value 305 of the corresponding position of the corresponding subject obtained by other means as model output annotation data, trains the regression model 306, and determines the parameters of the regression model 306; then, the color components of the subject image 304 obtained under the illumination of light sources with different spectral distributions are input into the regression model 306, and the chromaticity coordinate value 307 corresponding to the subject is output after calculation by the regression model 306.

The chromaticity coordinate values of the photographed object used as regression model training data can be obtained by other devices that measure chromaticity coordinates, such as using a hyperspectral camera to obtain the chromaticity coordinate values of the photographed object. In this case, a tooth colorimetric film can be used as the photographed object instead of real teeth to facilitate the acquisition of the chromaticity coordinate values of the tooth colorimetric spectrum.

4, the half-width range 405 near the peak wavelength of the light source spectrum distribution 404 of at least part of the light source group intersects with the half-width range 406 near the sensitive peak wavelength of three types of human cone cells: S-Cone, M-Cone, and L-Cone. The spectral sensitivity curve 401 of the S-type human cone cell, the spectral sensitivity curve 402 of the M-type human cone cell, and the spectral sensitivity curve 403 of the L-type human cone cell.

Figure 4 lists the spectral distribution curves corresponding to red, cool white, medium white, purple, warm white, green, yellow, white, and blue LED light sources, as shown by the thin solid lines in the figure. The thick dashed lines in the figure show the spectral sensitivity curves of three types of human cone cells, S-type, M-type, and L-type. The horizontal axis corresponding to the curve in the figure is the wavelength, the unit is nanometers, and the vertical axis is the standardized value, dimensionless. The light source corresponds to the intensity, and the S-type, M-type, and L-type cone cells correspond to the sensitivity.

5 , the color block group includes a plurality of color blocks, and these color blocks have different reflective spectral distributions in the half-width range 505 near the sensitive peak wavelength of three types of human cone cells, namely, S-type, M-type, and L-type. The spectral sensitivity curve 501 of the S-type human cone cell, the spectral sensitivity curve 502 of the M-type human cone cell, the spectral sensitivity curve 503 of the L-type human cone cell, and the reflective spectral distribution curve 504 of the color block.

Figure 5 lists the reflectance spectrum distribution curves corresponding to 24 color blocks, as shown by the thin solid lines in the figure. The thick dashed lines in the figure show the spectral sensitivity curves of three types of human cone cells: S-type, M-type, and L-type. The horizontal axis corresponding to the curve in the figure is the wavelength, the unit is nanometers, and the vertical axis is the value after standardization, which is dimensionless. The color blocks correspond to the intensity, and the S-type, M-type, and L-type cone cells correspond to the sensitivity.

Referring to FIG6 , the regression model is implemented using a 9-layer fully connected neural network, wherein the input layer 601 includes 27 nodes corresponding to the 3 color channel component values of the color image obtained under 9 light sources, the output layer includes 3 nodes corresponding to the chromaticity coordinate values, the middle layer 603 includes the 2nd to 8th layers, respectively including 81, 243, 729, 729, 243, 81, 81 nodes, and each layer uses a sigmoid activation function. The nodes 605 of each layer are fully connected 602.

It should be noted that in order to more clearly understand the present invention, the drawings and descriptions in the present invention have been simplified, and elements well-known in the art have been eliminated for the sake of clarity. Moreover, the eliminated elements themselves cannot promote a better understanding of the present invention, so they will not be described in detail. The omitted details and modified or alternative implementations are within the knowledge of ordinary technicians in the field. The present invention is not limited to the above embodiments, and technicians in the field can still modify the technical solutions recorded in the above embodiments, or make equivalent replacements for some of the technical features therein; however, these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

The device for obtaining chromaticity coordinate values of multiple points on the surface of a photographed object is characterized in that it includes: a light source group, a color block group, a control unit, an imaging unit, an image segmentation and alignment unit, and a chromaticity coordinate value calculation unit, wherein the light source group includes multiple light sources with different spectral distributions, and the color block group includes multiple color blocks with different reflection spectral distributions, the control unit controls the light source group to use light sources with different spectral distributions to illuminate the photographed object and the color block group in different time periods, and controls the imaging unit to synchronously obtain multiple group images of the photographed object and the color block group under the illumination of light sources with different spectral distributions in corresponding time periods, the image segmentation and alignment unit segments an image containing a single photographed object and an image containing a single color block from the group image, and aligns the images of the same subject and the same color block obtained under different light source illumination conditions, respectively, and the chromaticity coordinate value calculation unit calculates the chromaticity coordinate values of multiple points on the surface of the photographed object using a regression model.
The device as described in claim 1 is characterized in that the half-width range near the peak wavelength of the light source spectral distribution of at least part of the light source group intersects with the half-width range near the sensitive peak wavelength of three types of human cone cells, namely, S-type, M-type and L-type.
The device as claimed in claim 1 is characterized in that the color block group includes a plurality of color blocks, and these color blocks have different reflective spectral distributions in the half-width range near the sensitive peak wavelength of three types of human cone cells, namely, S-type, M-type, and L-type.
The device according to claim 1, characterized in that the calculation step of the chromaticity coordinate value calculation unit comprises:

First, a regression model is established, using the color channel component values of the color block images obtained under the illumination of light sources with different spectral distributions as model input data, and the chromaticity coordinate values of the corresponding color blocks as model output annotation data, and using the color channel component values of the subject images obtained under the illumination of light sources with different spectral distributions as model input data, and the chromaticity coordinate values of the corresponding positions of the corresponding subjects obtained by other means as model output annotation data, training the regression model, and determining the regression model parameters;

Then, the trained regression model is used to input the color component values of the image of the object obtained under the illumination of light sources with different spectral distributions, and the chromaticity coordinate values corresponding to the object are output.
The method for obtaining multi-point chromaticity coordinate values on the surface of a photographed object is characterized by obtaining a group image of the photographed object and a color block group containing multiple color blocks with different reflective spectral distributions under the illumination of light source groups with different spectral distributions, segmenting an image containing a single photographed object and an image containing a single color block from the group image, aligning images of the same subject and the same color block obtained under different light source illumination conditions, and using a regression model to calculate the multi-point chromaticity coordinate values on the surface of the photographed object.
The method as claimed in claim 5 is characterized in that the half-width range near the peak wavelength of the light source spectral distribution of at least part of the light source group intersects with the half-width range near the sensitive peak wavelength of three types of human cone cells: S-type, M-type, and L-type.
The method as claimed in claim 5 is characterized in that the color block group comprises a plurality of color blocks, and these color blocks have different reflective spectral distributions in the half-width range near the sensitive peak wavelength of three types of human cone cells, namely, S-type, M-type, and L-type.
The method according to claim 5, characterized in that the step of calculating the chromaticity coordinate value comprises:

First, a regression model is established, using the color channel component values of the color block images obtained under the illumination of light sources with different spectral distributions as model input data, and the chromaticity coordinate values of the corresponding color blocks as model output annotation data, and using the color channel component values of the subject images obtained under the illumination of light sources with different spectral distributions as model input data, and the chromaticity coordinate values of the corresponding positions of the corresponding subjects obtained by other means as model output annotation data, training the regression model, and determining the regression model parameters;

Then, the trained regression model is used to input the color component values of the image of the object obtained under the illumination of light sources with different spectral distributions, and the chromaticity coordinate values corresponding to the object are output.