JP7335283B2 - Information processing terminal, program, information processing system, and color correction method - Google Patents

Description

The present invention relates to an information processing terminal, a program, an information processing system, and a color correction method.

When an image is taken with an imaging device such as a camera, the color of the captured image changes depending on the surrounding lighting environment, the settings of the imaging device, and the like. Therefore, it is necessary to perform color correction in order to correctly reproduce colors from captured images.

2. Description of the Related Art Conventionally, there has been known a technique for correcting colors by preparing a color chart and photographing the color chart. (Patent document 1)

JP-A-2005-045438

However, in the prior art, a color chart must be prepared in order to perform color correction. In addition, since it is necessary to include the color chart in the photographing scene, it is necessary to prevent other objects from being photographed in the color chart portion, which limits the photographable range. In addition, the color chart itself is reflected in the captured image.

An object of one embodiment of the present invention is to provide an information processing terminal capable of performing color correction of a photographed image without using a color chart.

The disclosed information processing terminal is an information processing terminal that includes a display unit that displays a display image, and a photographing unit that can photograph the face of a user looking at the display unit, wherein the display unit has one color. display control means for displaying a display image including the above colors; photographing control means for causing the photographing unit to photograph the display image while the display image is displayed; and display image extraction for extracting the reflected light of the display image from the photographed image. and a color correction means for performing color correction of the captured image based on the reflected light of the extracted display image.

It is possible to perform color correction of an image captured by an information processing terminal without using a color chart.

1 is a front view of an example of an information processing terminal according to an embodiment; FIG. 1 is a hardware configuration diagram of an example of an information processing terminal according to an embodiment; FIG. It is a processing block diagram of an example of an information processing terminal according to the present embodiment. 5 is a flowchart of an example of overall processing of a color correction method using an information processing terminal according to the present embodiment; 6 is a flowchart of color-corrected image extraction processing in the information processing terminal according to the embodiment; 7 is a flowchart of correction coefficient calculation processing in the information processing terminal according to the embodiment; 4 is a flowchart of color correction processing in the information processing terminal according to the embodiment; It is a figure which shows an example of the display image displayed on the display part of the information processing terminal which concerns on this embodiment. It is a figure explaining extraction of the color corrected image of the information processing terminal which concerns on this embodiment. It is a figure explaining the processing example of the color correction process by the information processing terminal which concerns on this embodiment. 1 is a configuration diagram of an example of an information processing terminal and a server device according to an embodiment; FIG. It is a figure which shows the modification of the display image of the information processing terminal which concerns on this embodiment. It is a figure which shows the modification of the display image of the information processing terminal which concerns on this embodiment. It is a figure which shows the modification of the display image of the information processing terminal which concerns on this embodiment. It is a figure which shows the modification of the display image of the information processing terminal which concerns on this embodiment.

This embodiment will be described below with reference to the drawings.

<Hardware configuration>
<<Information processing terminal 1>>
FIG. 1 is a front view of an information processing terminal 1 according to this embodiment. The information processing terminal 1 may be a communication terminal such as a mobile phone or a smart phone, or an information terminal such as a portable personal computer or a tablet computer. Here, a smart phone will be described as an example of the information processing terminal 1 of the present embodiment. A display unit 100 and a photographing unit 200 are provided on the front side of the information processing terminal 1 . The display unit 100 is a so-called touch panel. The photographing unit 200 is a camera provided on the display unit 100 side of the information processing terminal 1, a so-called in-camera.

The information processing terminal 1 is implemented by, for example, an information processing terminal having a hardware configuration shown in FIG. FIG. 2 is a hardware configuration diagram of an example of an information processing terminal according to this embodiment.

Information processing terminal 1 of FIG. , an audio processing unit 60 , a display unit 100 , and an imaging unit 200 . Each is connected by a bus B1.

The CPU 10 is an arithmetic device that reads a program (application) from a storage device such as the ROM 30 or the recording medium 45 onto the RAM 20 and executes processing.

The RAM 20 is an example of a volatile semiconductor memory (storage device) that temporarily holds programs (applications) and the like.

The ROM 30 is an example of a non-volatile semiconductor memory (storage device) that can retain programs (applications) and the like even when the power is turned off. The ROM 30 stores programs such as BIOS (Basic Input/Output System) that are executed at startup, and various settings such as OS (Operating System) settings and network settings.

The external I/F 40 is an interface with an external device such as the recording medium 45 . The information processing terminal 1 of FIG. 2 can use the external I/F 40 to read from and/or write to the recording medium 45 .

The recording medium 45 includes an SD (Secure Digital) memory card, a USB (Universal Serial Bus) memory, and the like.

The wireless I/F 50 is an interface that wirelessly connects to an external network such as a wireless LAN (Local Area Network) or the Internet. The wireless I/F 50 has an antenna and connects to an external network using a mobile communication system such as LTE (Long Term Evolution) or a wireless communication system such as wireless LAN (Local Area Network) or Bluetooth (registered trademark). . The information processing terminal 1 can use the wireless I/F 50 to communicate with an external server device or the like.

The audio processing unit 60 is a device that performs input/output processing of audio and the like. A microphone 62 and a speaker 64 are connected to the audio processing unit 60 , and audio is input from the microphone 62 and output from the speaker 64 .

The display unit 100 is a display with a touch panel. The display unit 100 displays a display image, a UI (User Interface), information, etc., which will be described later. Further, by detecting a touch on the screen, an input such as an operation is performed.

The photographing unit 200 is a camera provided on the display unit 100 side of the information processing terminal 1, a so-called in-camera. Since the imaging unit 200 is provided on the display unit 100 side of the information processing terminal 1 as shown in FIG.

The information processing terminal 1 according to the present embodiment can implement various processes described later by executing a program (application) in the hardware configuration described above.

The information processing terminal 1 of the present embodiment uses the image of the user's face captured by the imaging unit 200 to correct the color of the captured image of the user's face. Specifically, information processing terminal 1 displays a predetermined display image on display unit 100 . The imaging unit 200 of the information processing terminal 1 photographs the user's face while the user looks at the display unit 100 . The information processing terminal 1 calculates a color correction coefficient using the display image of the display unit 100 that is reflected in the captured image of the user's face. Color correction of the photographed image of the user's face is performed using the correction coefficient.

<Software configuration>
Taking the information processing terminal 1 as an example, the software configuration of this embodiment will be described. FIG. 3 is a processing block diagram of an example of the information processing terminal 1 according to this embodiment. The information processing terminal 1 includes display control means 310 , photographing control means 320 , display image extraction means 330 , correction coefficient calculation means 340 , color correction means 350 , and control means 400 .

The display control means 310 controls the display unit 100 of the information processing terminal 1 to display a predetermined image (for example, a display image to be described later). The display unit 100 displays a predetermined image under the control of the display control means 310 .

The photographing control unit 320 controls the photographing unit 200 of the information processing terminal 1 to photograph while a predetermined image is displayed on the display unit 100 of the information processing terminal 1 . The imaging unit 200 performs imaging under the control of the imaging control means 320 .

The display image extracting means 330 extracts an image (display image) displayed by the display unit 100 from the image (captured image) photographed by the photographing unit 200, which is reflected in the user's eye, particularly the so-called iris and pupil portion of the eye. ).

The correction coefficient calculator 340 obtains the chromaticity value of the image extracted by the display image extractor 330 . A color correction coefficient is calculated based on the obtained chromaticity value and the chromaticity value of the image displayed by the display unit 100 (display image).

The color correction means 350 corrects the color of the image captured by the imaging section 200 using the correction coefficients calculated by the correction coefficient calculation means 340 .

The control means 400 controls the entire information processing terminal 1 .

<Processing>
《Overall processing》
The information processing terminal 1 according to this embodiment performs processing, for example, in a procedure as shown in FIG. FIG. 4 is a flowchart showing an example of overall processing of the information processing terminal 1 according to this embodiment. In the present embodiment, the display image 500 is displayed and captured by the imaging unit 200, correction coefficients are calculated using the captured image, and color correction of the captured image is performed.

(Step S10) The control unit 400 of the information processing terminal 1 controls to activate the photographing unit 200 when the program (application) is executed by the user. The imaging unit 200 is activated under the control of the control means 400 .

(Step S20) The display control means 310 of the information processing terminal 1 controls the display unit 100 to display the display image. The display unit 100 displays a display image under the control of the display control means 310 .

An example of a display image displayed by the display unit 100 will be described. FIG. 8 is a diagram showing an example of a display image displayed on the display unit 100 of the information processing terminal 1 according to this embodiment. The display image 500 is a five-color pattern in which each place is filled with white, red, green, blue, and gray. For red, green, and blue, display pure colors. In FIG. 8, white, red, green, blue, and gray are represented by W, R, G, B, and GY, respectively. (The same applies to FIGS. 12A, 12B, 12C, and 12D.)

The display image 500 has white and red in the first row, green and blue in the second row, and gray in the third row. In the display image 500 of this embodiment, the height of the third row is higher than that of the first and second rows.

(Step S<b>30 ) The information processing terminal 1 instructs the user to face the display unit 100 . Specifically, the control unit 400 of the information processing terminal 1 instructs the voice processing unit 60 to say, “Please face the screen” from the speaker 64 by voice.

Information processing terminal 1 may, if necessary, instruct the user to remove glasses or contact lenses when instructing the user to face display unit 100 .

(Step S40) The photographing control means 320 of the information processing terminal 1 controls the photographing section 200 to photograph. The photographing unit 200 performs photographing under the control of the photographing control means 320 . When the imaging unit 200 performs imaging, the imaging control unit 320 causes the display image 500 to be displayed on the display unit 100 .

As for the timing of shooting, for example, shooting may be performed when the user taps the display unit 100 . Alternatively, when the user operates a button such as a shutter button, the image may be captured. Further, the display of the display unit 100 or the sound from the speaker 64 by the sound processing unit 60 may be counted down, and when the countdown is completed, the photograph may be taken. In any case, when the imaging unit 200 performs imaging, the imaging control unit 320 causes the display unit 100 to display the display image 500 .

Note that the order of steps S20 and S30 may be reversed. That is, the information processing terminal 1 first instructs the user to face the display unit 100 . After that, the display control means 310 of the information processing terminal 1 controls the display unit 100 to display the display image.

At that time, instead of giving a voice instruction from the speaker 64 by the voice processing unit 60, "Please turn your face to the screen." may be displayed. Alternatively, an illustration, an icon, or the like may be displayed on the display unit 100 to instruct the user to face the display unit 100 . An image captured by the imaging unit 200 may be displayed.

Further, when the order of steps S20 and S30 is reversed, the photographing control means 320 causes the display control means 310 to display the display image 500 on the display unit 100 immediately before photographing, and displays the display image 500 immediately after photographing. The image 500 may be prevented from being displayed. That is, the display unit 100 may display the display image 500 only for a short period of time (for a moment) at the time of photographing, like a flash of a camera.

(Step S50) The display image extracting means 330 of the information processing terminal 1 extracts the image (display image) displayed by the display unit 100 that is seen by the user from the image (captured image) captured by the imaging unit 200. Corrected image extraction processing is performed. The image (display image) displayed by the display unit 100 that is seen by the user is an image obtained by photographing the light reflected by the eyes of the display image. The color-corrected image extraction process will be described in detail in a later paragraph.

(Step S60) The correction coefficient calculation means 340 of the information processing terminal 1 obtains the chromaticity value of the image extracted by the display image extraction means 330. FIG. Based on the calculated chromaticity value and the chromaticity value of the image displayed by the display unit 100 (display image), a correction coefficient calculation process for calculating a color correction coefficient is performed. The correction coefficient calculation process will be described in detail in a later paragraph.

(Step S<b>70 ) The color correction means 350 of the information processing terminal 1 uses the correction coefficients calculated by the correction coefficient calculation means 340 to perform color correction processing for correcting the colors of the image captured by the imaging unit 200 . In this embodiment, color correction is performed on the captured image. The color correction process will be described in detail in a later paragraph.

(Step S80) The control unit 400 of the information processing terminal 1 saves the image that has undergone the color correction process.

<<S50: Color correction image extraction process>>
The color-corrected image extraction process in step S50 will be described. FIG. 5 is a flowchart of an example of color-corrected image extraction processing. In the color-corrected image extraction process, a portion of the displayed image reflected by the eye is extracted from the photographed image. The process of extracting the portion of the display image reflected by the eye in the color correction image extraction process is an example of the process of extracting the reflected light of the display image.

(Step S110) The display image extraction unit 330 of the information processing terminal 1 performs face recognition processing on the image (captured image) captured by the imaging unit 200, and recognizes the face included in the captured image. A known technique can be applied to the face recognition processing.

(Step S120) The display image extracting means 330 of the information processing terminal 1 extracts the face portion from the face recognition result in step S110. Furthermore, facial feature points are extracted. The display image extracting means 330 of this embodiment extracts the so-called iris and pupil portion of the eye as a feature point.

In addition, when the display image extracting means 330 of the information processing terminal 1 extracts the feature points of the face, there may be a case where the so-called iris and pupil portions of the eyes cannot be extracted. For example, it is conceivable that the user blinks his/her eyes and wears glasses or contacts when photographing. In this way, when the so-called iris and pupil portion of the eye cannot be extracted, the process may return to step S20 and photographing may be performed again.

(Step S130) The display image extraction means 330 of the information processing terminal 1 extracts the display image 500 displayed on the display unit 100 from the so-called iris and pupil portions of the eyes extracted in step S120. The displayed display image 505 is extracted. FIG. 9 shows a displayed image 505 reflected from the so-called iris-pupil portion of the right eye. In step S130, the display image 505 reflected by the so-called iris portion (iris and pupil portion) of the right eye is extracted.

In the information processing terminal 1 of the present embodiment, the display image 505 is extracted from the so-called iris and pupil portion of the eye. ) is not limited to If the color can be extracted, the display image 505 reflected by the so-called white portion of the eye may be extracted.

<<S60: Correction coefficient calculation process>>
The correction coefficient calculation process in step S60 will be described. FIG. 6 is a flowchart of an example of correction coefficient calculation processing. The correction coefficient calculation process calculates correction coefficients necessary for color correction.

(Step S210) The correction coefficient calculation means 340 of the information processing terminal 1 acquires the RGB values of the portions corresponding to the respective colors of the display image 505 reflected by the eye, which is extracted by the display image extraction means 330 . The display image 500 is composed of five colors of white, red, green, blue and gray. Here, white, red, green, blue, and gray are defined as color 1, color 2, color 3, color 4, and color 5, respectively. Obtain the RGB values (R _i , G _i , B _i ) of the portion of the displayed image 505 corresponding to the portion of the displayed image 500 with color i (i=1, . . . , 5).

The RGB value of the portion of the display image 505 corresponding to the portion of the color i of the display image 500 may be, for example, the average value of the RGB values of the portion of the display image 505 corresponding to the portion of the color i of the display image 500. . Alternatively, for example, the RGB value of the central pixel of the portion of the display image 505 corresponding to the color i portion of the display image 500 or the average value of the RGB values of the pixels near the center may be used.

(Step S220) The correction coefficient calculation means 340 of the information processing terminal 1 calculates the RGB values (R _i , G _i , B _i ), the XYZ values of color i are calculated based on Equation 1. Let the XYZ values calculated from the RGB values (R _i , G _i , B _i ) be (X _i , Y _i , Z _i ).

(Step S230) The _{correction coefficient} calculation _means 340 of the information processing terminal 1 calculates the xy value (chromaticity value ). Let x i and _{y i} be the xy values corresponding to X _i , Y _i and Z _i of the XYZ values.

(Step S240) The correction coefficient calculation means 340 of the information processing terminal 1 obtains the correction coefficients Δx _i and Δy _i based on Equation 3 using the xy values of each color calculated in step S230. Note that x0 _i and y0 _i are the xy values of the color i of the display image 500 .

In the information processing terminal 1 of the present embodiment, as the display image 500, pure colors of red, green, and blue are displayed. That is, the red color of the displayed image 500 is only the red component, that is, (R, G, B)=(1, 0, 0). Similarly, the green color of the displayed image 500 has only green components, i.e., (R, G, B)=(0, 1, 0), and the blue color of the displayed image 500 has only blue components, i.e., (R, G,B)=(0,0,1). Therefore, the xy values (x0 _i , y0 _i ) of red, green, and blue of the display image 500 are chromaticity values of pure colors.

<<S70: Color Correction Process>>
The color correction processing in step S70 will be described. FIG. 7 is a flowchart of an example of color correction processing. The color correction process corrects the colors of the image using the correction coefficients calculated in the correction coefficient calculation process.

(Step S310) The color corrector 350 of the information processing terminal 1 acquires an image to be corrected. In the information processing terminal 1 of the present embodiment, this is the image captured in step S30. That is, it is an image used when obtaining the color correction coefficients.

(Step S320) The color correction means 350 of the information processing terminal 1 divides the image to be corrected into a red image, a green image, and a blue image, each divided by RGB components. Here, the red image will be described in detail. For each pixel of the red image, that is, the R value of each pixel of the image to be corrected, the Y value (Y _r ) is obtained based on Equation (1). Let y _r0 be the y value of pure red. Let Δx _r and Δy _r be the correction values obtained for the red color of the display image 500 . Based on Equation 4, the correction values ΔX _r and ΔZ _r in the XYZ value space are obtained. Next, the correction values ΔR _r , ΔG _r , and _ΔBr of each pixel in the RGB space are obtained based on Equation 5. The obtained correction values ΔR _r , ΔG _r , and _ΔBr of each pixel in the RGB space are subtracted from each pixel value of the red image. Each pixel of the red image is corrected by the above processing. Similar processing is performed for the green image and the blue image. Then, the corrected red image, blue image, and green image are combined by, for example, addition to create a final corrected image.

In this embodiment, color correction is performed using correction coefficients obtained for each color. good too. Alternatively, only some colors may be corrected. In this embodiment, correction is performed by converting the correction coefficients Δx and Δy into correction values in the RGB space. Correction may be performed using Δy.

The display image reflected by the eye is an image in which the light from the display unit is reflected by the cornea and lens of the eye. As a result, the intensity varies greatly depending on the reflectance of the cornea and lens of the eye. In this embodiment, by using chromaticity values, it is possible to reduce the influence of intensity fluctuations and perform color correction.

With the information processing terminal 1 of this embodiment, color correction can be performed using the information processing terminal 1 without preparing a color chart. Moreover, since no color chart is used, there is no need to secure a space for photographing the color chart, and the color chart itself will not appear in the photographed image.

As for the image after the color correction, since the display image 505 that appears to the eye is small, there is usually no problem even if the display image 505 remains in the image. However, if the displayed image 505 is of concern, it is possible to correct the displayed image 505 to have the same color as the surrounding color.

<<Processing example>>
The results of actual processing will be described. FIG. 10 is a diagram illustrating an example of color correction processing by the information processing terminal 1 according to this embodiment.

FIG. 10 is a diagram plotting pure colors (red (R), green (G), blue (B)), colors before correction (unprocessed), and colors after correction on CIE 1976 a*b* coordinates. . Three colors (pure colors) of red (R), green (G), and blue (B) were plotted with a circular marker R0 for red, a square marker G0 for green, and an inverted triangular marker B0 for blue. A plurality of pure colors are plotted with different intensities. It should be noted that the intensity of each pure color increases with increasing distance from the origin.

Values calculated from the RGB values before correction are plotted with diamond-shaped markers MR, MG, and MR, and values calculated from the RGB values after correction are plotted with triangular markers CR, CG, and CB.

In FIG. 10, when the RGB values of the captured image are correct, for example, for red, the RGB value markers MR before correction are plotted on or between the red pure color markers R0. In the case of red, on the a*b* coordinates, if it is above the line connecting the pure red marker R0, it means that it contains a green component, and if it is below it, it means that it contains a blue component. there is

Similarly, for green, for example, if the RGB values of the captured image are correct, the uncorrected RGB value markers MG are plotted on or between the green full color markers G0. In addition, in the case of green, on the a*b* coordinates, if it is above the line connecting the green pure color marker G0, it means that it contains a red component, and if it is below it, it means that it contains a blue component. there is

Similarly, for blue, for example, if the RGB values of the captured image are correct, the uncorrected RGB value marker MB is plotted on or between the blue full color marker B0. In addition, in the case of blue, on the a*b* coordinates, if it is above the line connecting the pure blue marker B0, it means that it contains a red component, and if it is below it, it means that it contains a green component. there is

Consider the diamond-shaped markers MR, MG, and MB before correction in FIG. For blue, the marker MB is on a line connecting the nearly blue solid color marker B0. Therefore, it is considered that the color of blue is reproduced. For red, the marker MR is plotted to the right compared to the solid red marker R0. Therefore, it can be judged to be slightly bluish. Green is plotted lower compared to the solid color marker G0. Therefore, it can be judged to be slightly bluish.

Consider the corrected triangular markers CR, CG, and CB in FIG. It can be seen that the corrected results are plotted on or between the markers R0, G0 and B0, which are almost pure colors, respectively. From this, it can be seen that each color has been corrected to a correct color.

<Modification>
In the present embodiment, processing was performed by the information processing terminal 1 alone, but as shown in FIG. processing device) may be combined to perform processing as an information processing system. For example, it may be as follows. First, the information processing terminal 1 performs the processes of steps S10 to S40 in FIG. Next, the image captured by the imaging unit 200 of the information processing terminal 1 is transmitted to the server device 5 . The server device 5 performs the processing of steps S50 to S60. The correction coefficient calculated by the server device 5 is transmitted to the information processing terminal 1 . Finally, the information processing terminal 1 performs the processing of steps S70 to S80.

Alternatively, the server device 5 may perform the processing from steps S50 to S70 to transmit the color-corrected image to the information processing terminal 1. FIG. In that case, the image may be transmitted not only to the information processing terminal 1 that captured the image, but also to another information processing terminal or server device.

A display image displayed on the display unit 100 is not limited to the display image 500 . For example, the arrangement of colors in the display image 500 may be exchanged. For example, the display image 500 may have green and blue in the first row, white and red in the second row, and gray in the third row. As described above, in the display image 500 of the present embodiment, since the third row is higher than the first and second rows, gray is arranged in the third row. is preferred, but other colors may be used.

The number of colors of the display image is not limited to five colors like the display image 500 shown in FIG. The display image may contain one or more colors.

The color arrangement itself may be changed as shown in FIGS. 12A, 12B, 12C, and 12D. For example, like the display image 510 in FIG. 12A, only one color is displayed at a time, and for example, the display image 510 of white (W), gray (GY), . It may be repeated. In that case, correction coefficients may be obtained for each of the displayed images 510 to perform color correction on any or all of the captured images.

Alternatively, the display image 510 of each color may be displayed in order and photographed using the continuous shooting function of the camera. The order of colors to be displayed may also be changed as appropriate.

As in the display image 520 of FIG. 12B, one color may be displayed in one stage. Note that the arrangement of colors is not limited to the example in FIG. 12B and may be changed.

As in the display image 530 of FIG. 12C, one color may be displayed in a plurality of gradations. For example, change the gradation of achromatic colors such as light gray (GY1) in the first step, gray (GY2) darker than the first step in the second step, and even darker gray (GY3) in the third step. You may make it display. Note that the color is not limited to an achromatic color such as gray (GY) as in the example of FIG. 12C, and may be changed to another color.

As in the display image 540 of FIG. 12D , the color area may not be displayed in the entire horizontal direction of the display unit 100, but may be displayed in an elliptical shape in the middle. Note that the shape is not limited to an ellipse and may be polygonal. Also, the background color may be gray (GY) or white (W). Also, it may be black, that is, display no color.

Colors are not limited to white, red, green, blue, and gray, and cyan, magenta, yellow, and the like may be used. Color combinations may also include cyan, magenta, yellow, and the like.

White balance can be adjusted by using an achromatic image as the display image. By using an image of pure colors such as red, green, and blue as a display image, each color can be corrected. Therefore, it is desirable to combine achromatic colors with each color. As for combinations, all colors may be provided in the same display image, or may be displayed in order. For example, after first displaying white (achromatic color) and adjusting the white balance, a pure color such as red may be displayed and the correction coefficient for each color may be calculated.

As the information processing terminal 1, any device having a camera on the display screen side, that is, a so-called in-camera, can be applied. For example, a mobile phone with an in-camera, a portable personal computer with an in-camera (so-called notebook computer or laptop computer), or a tablet computer may be applied. As for the camera, an external camera may be used as long as it is capable of photographing the display screen side.

In commercially available smartphones, the color development of the display part may differ from the pure color. In that case, when calculating the correction coefficient in the present embodiment, the influence of the color development of the display unit may be corrected in post-processing based on the model information of the smartphone or the like.

In the information processing terminal 1 of the present embodiment, correction is performed using a still image, but it may be performed using a moving image. For example, when the user operates a button such as a shutter button, a moving image may be captured before and after the operation.

In the information processing terminal 1 of the present embodiment, a process of calculating a correction coefficient and performing color correction of the captured image is performed using a captured image. may be used for color correction.

In the information processing terminal 1 of the present embodiment, the display image reflected by the eye is used, but the location where the display image is reflected is not limited to the eye as long as the reflected display image is captured. For example, an image reflected by a mirror, glasses, or the like may be used.

In the information processing terminal 1 of the present embodiment, a display image reflected by the eyes of the user (human) is used, but a display image reflected by the eyes of an animal (for example, a dog or a cat) may be used.

Although the embodiment has been described in detail above, the above embodiment is an example in all respects and is not limited to the above embodiment. Various modifications and changes can be made to the above embodiment within the scope of the embodiments of the present invention described in the claims.

This application claims priority from Basic Application No. 2019-024573 filed on February 14, 2019 with the Japan Patent Office, the entire contents of which are incorporated herein by reference.

1 Information Processing Terminal 100 Display Unit 200 Photographing Unit 310 Display Control Means 320 Photographing Control Means 330 Display Image Extracting Means 340 Correction Coefficient Calculating Means 350 Color Correcting Means

Claims (8)

An information processing terminal having a display unit for displaying a display image and a photographing unit capable of photographing the face of a user looking at the display unit,
display control means for displaying a display image including one or more colors on the display unit;
a photographing control means for causing the photographing unit to photograph while the display image is displayed;
display image extracting means for extracting the reflected light of the display image from the captured image;
color correction means for performing color correction of the photographed image based on the reflected light of the extracted display image;
Information processing terminal having 2. The information processing terminal according to claim 1, wherein said display image extracting means extracts said display image seen by said user from said photographed image as reflected light of said display image. The display control means displays the display image including red, green and blue,
The color correcting means corrects the color displayed by the display unit based on the chromaticity values of red, green, and blue contained in the reflected light of the extracted display image and the chromaticity values of pure colors red, green, and blue. 3. The information processing terminal according to claim 1, wherein color correction of the displayed image is performed. The display control means displays the display image including an achromatic color,
2. The color correction means performs color correction of the display image displayed by the display unit based on the chromaticity value of the display image color included in the extracted reflected light of the display image. 4. The information processing terminal according to any one of items 1 to 3. The information processing terminal according to any one of claims 1 to 4, wherein the display control means causes the display unit to display the display image including patterns of white, red, green, blue and gray. . An information processing terminal having a display unit for displaying a display image and a photographing unit capable of photographing the face of a user looking at the display unit,
display control means for displaying a display image containing one or more colors on the display unit;
shooting control means for causing the shooting unit to shoot while the display image is displayed;
display image extracting means for extracting the reflected light of the display image from the captured image;
color correction means for performing color correction of the photographed image based on the reflected light of the extracted display image;
A program to function as an information processing terminal having a display unit for displaying a display image and a photographing unit capable of photographing the face of a user looking at the display unit; An information processing system comprising an information processing device,
display control means for displaying a display image including one or more colors on the display unit;
a photographing control means for causing the photographing unit to photograph while the display image is displayed;
display image extracting means for extracting the reflected light of the display image from the captured image;
a color correcting means for performing color correction of the photographed image based on the reflected light of the extracted display image;
An information processing system having A color correction method executed by an information processing terminal having a display unit for displaying a display image and a photographing unit capable of photographing the face of a user looking at the display unit,
a display control step of causing the display unit to display a display image containing one or more colors;
a photographing control step of causing the photographing unit to photograph while the display image is displayed;
a display image extracting step of extracting the reflected light of the display image from the captured image;
a color correction step of performing color correction of the photographed image based on the reflected light of the extracted display image;
A color correction method comprising:

Images