JP7345126B1 - Image processing device, image processing method and program - Google Patents

Abstract

The present invention provides an image processing device, an image processing method, or a program that can perform color correction according to age and reduce the proportion of overexposure in an image after color correction. SOLUTION: A computing device 5 serving as an image processing device detects a region of interest in an image photographed by a photographing section 2, performs color correction on the detected region of interest according to age, and performs color correction on the detected region of interest in accordance with the age of the user. A locally corrected image including the area and other areas is output. The image is, for example, an eye surgery image. The region of interest is, for example, a circular region at the center of the eyeball. [Selection diagram] Figure 1

Description

Application of Article 30, Paragraph 2 of the Patent Law Japanese Society of Ophthalmological Surgery, Ophthalmic Surgery, 2023, Volume 36, Extraordinary First Issue, December 28, 2022 46th Academic Meeting of the Japanese Society of Ophthalmological Surgery, January 29, 2023 Day

The present disclosure relates to an image processing device, an image processing method, or a program that performs color correction on an image according to the age of a person viewing the image.

Human visual characteristics change with age. Specifically, it is thought that the older one gets, the darker or yellower the color becomes. Regarding changes in visual characteristics due to aging, Patent Document 1 listed below describes correcting the image to compensate for the deterioration of visual characteristics that changes with human age.

Patent No. 3552413

However, in the method of Patent Document 1, the older the person is, the larger the amount of image correction becomes, and as a result, blown-out highlights are more likely to occur in the corrected image.

The present disclosure has been made in view of the above circumstances, and an object of the present disclosure is to provide an image processing device, an image processing method, or a program that can perform color correction according to age and reduce the proportion of overexposure in images after color correction. shall be.

The image processing device of the present disclosure includes:
an image acquisition unit that acquires an image;
a detection unit that detects a region of interest in the image;
a correction unit that performs color correction on the attention area detected by the detection unit according to the age of a subject viewing the image;
an output unit that outputs a locally corrected image including the color-corrected region of interest and other regions;
Equipped with

The image processing method of the present disclosure includes:
an image acquisition step of acquiring an image;
a detection step of detecting a region of interest in the image;
a correction step of performing color correction on the region of interest detected in the detection step according to the age of the subject viewing the image;
outputting a locally corrected image including the color-corrected region of interest and other regions;
Equipped with

The program of this disclosure is
an image acquisition step of acquiring an image;
a detection step of detecting a region of interest in the image;
a correction step of performing color correction on the region of interest detected in the detection step according to the age of the subject viewing the image;
outputting a locally corrected image including the color-corrected region of interest and other regions;
have the computer execute it.

According to the image processing device, image processing method, and program of the present disclosure, color correction is performed on the attention area of the image without performing color correction on the entire image. The percentage of overexposure can be reduced.

FIG. 1 is a configuration diagram of a display system. 3 is a flowchart illustrating an example of image correction processing executed by the arithmetic device. 3 is a detailed flowchart of step S11 in FIG. 2. FIG. This is an example of an original image acquired in step S1 of FIG. 2. This is an example of an image after mask processing obtained in step S2 of FIG. 2. This is an example of an image after pixel value conversion obtained in step S3 of FIG. 2. This is an example of a grayscale image obtained in step S4 of FIG. 2. This is an example of a binarized image obtained in step S5 of FIG. 2. It is an example of the binarized image obtained in step S6 of FIG. 2 after the first noise removal process. It is an example of the binarized image obtained in step S7 of FIG. 2 after the second noise removal process. This is an example of a mask obtained in step S9 of FIG. 2. This is an example of an extracted image of the region of interest obtained in step S10 of FIG. 2. This is an example of the attention area image after color correction obtained in step S11 of FIG. 2. This is an example of a locally corrected image including a color-corrected region of interest and other regions obtained in step S12 of FIG. 2. FIG. 2 is a diagram illustrating a density histogram (luminance distribution) of an image in order to explain the general binarization method. 3 is a diagram illustrating a luminance distribution of an original image acquired in step S1 of FIG. 2. FIG. 17 is a diagram illustrating a brightness distribution when only the attention area is color-corrected for the image showing the brightness distribution of FIG. 16. FIG. 17 is a diagram illustrating a brightness distribution when the entire image is color-corrected for the image showing the brightness distribution of FIG. 16. FIG.

Embodiments of the present disclosure will be described below with reference to the drawings. FIG. 1 shows a display system 1 of this embodiment. The display system 1 is configured as a system that displays a video of an ophthalmic surgery such as cataract surgery (a video of a patient's eye undergoing ophthalmic surgery) in real time during the surgery. Note that cataract surgery is a surgery in which part or all of the crystalline lens of an eye suffering from cataract is removed, and an intraocular lens is inserted into the eye in place of the removed crystalline lens.

The display system 1 includes a photographing section 2, an input section 3, a display section 4, and a calculation device 5. The photographing unit 2 sequentially photographs images of the patient's eye at each point in time during ophthalmic surgery. That is, in photographic diagram 2, a moving image of the patient's eye is photographed during an ophthalmic surgery. The photographing unit 2 photographs a moving image of the eye to be treated viewed from the front. The moving image photographed by the photographing section 2 is sent to the arithmetic device 5. The color of each pixel constituting the image photographed by the photographing unit 2 is represented by, for example, an RGB display system color. In the RGB display system, colors are expressed by a combination of three primary colors: R (red), G (green), and B (blue). The number of gradations of each primary color of RGB or the number of gradations of brightness may be, for example, 256 gradations. In this case, the gradation value is indicated, for example, from 0 to 255 for each primary color. The larger the gradation value, the higher the brightness. That is, the minimum gradation value "0" indicates that the brightness is the lowest. On the other hand, the maximum gradation value of "255" indicates that the brightness is the highest. Note that the number of gradations other than 256 gradations may be used. Further, when the brightness is expressed in 256 gradations, the brightness of "0" indicates that the color of the pixel is "black". The brightness of "255" indicates that the color of the pixel is "white".

The input unit 3 has a known structure such as buttons, a numeric keypad, and a mouse, and is a part that receives input from the user. Specifically, the input unit 3 receives input such as the age of the person viewing the image. The input contents input to the input section 3 are sent to the arithmetic device 5.

The display unit 4 is a part that displays images. The display unit 4 is, for example, a liquid crystal display, but may be a display of another type such as a plasma display or an organic EL display. Further, the display unit 4 may be a head-mounted display. The display unit 4 is configured to be able to display color images. The color display system on the display unit 4 is, for example, an RGB display system.

The arithmetic unit 5 is a part that controls the entire display system 1. The arithmetic unit 5 has the same configuration as a normal computer, that is, it is composed of a CPU 6, RAM, ROM, and the like. Note that the arithmetic device 5 corresponds to the image processing device of the present disclosure.

The arithmetic device 5 includes a storage section 7 . The storage unit 7 is a non-volatile storage unit, and is a non-transitory tangible storage medium that non-temporarily stores computer-readable programs and data. A non-transient physical storage medium is realized by a semiconductor memory, a magnetic disk, or the like. Note that the storage section 7 may be configured as a built-in storage section built into the arithmetic device 5, or may be configured as an external storage section externally attached to the arithmetic device 5. The storage unit 7 stores various data such as a program 8 for processing executed by the CPU 6.

Next, details of the process executed by the CPU 6 of the arithmetic device 5 based on the program 8 will be explained. FIGS. 2 and 3 are flowcharts showing an example of the process. Note that the processes in FIGS. 2 and 3 are executed, for example, during an ophthalmologic surgery. When the process of FIG. 2 is started, the CPU 6 acquires the current image photographed by the photographing unit 2 (S1). FIG. 4 illustrates an example of an image acquired in step S1. The image in Figure 4 is a frontal image of the patient's eye during cataract surgery. Specifically, a surgical instrument is inserted into the central area of the eyeball (pupil, iris, and cornea area) to crush the crystalline lens. This is an image in the middle. Note that step S1 corresponds to the image acquisition step of the present disclosure. The CPU 6 that executes step S1 corresponds to an image acquisition section.

In step S2, the attention area, which is the area in the image that the subject viewing the image (specifically, the cataract surgeon) focuses on, is set as the central circular area (pupil, iris, and cornea area) of the eye to be operated on. In subsequent processing, the region of interest is detected (extracted) from the image acquired in step S1. The region of interest is a region (operation site, treatment region) of the eyeball (operated eye) where the surgeon performs the surgical treatment. Note that the circle of the attention area is, for example, a circle outside the iris.

Specifically, first, a process (mask process) is performed to hide areas other than the center of the image acquired in step S1 (S2). FIG. 5 exemplifies the image after the masking process in step S2. The masked image includes at least the central circular area of the eyeball (pupil, iris, and cornea area).

Next, each pixel value of the image after mask processing is set to the minimum pixel value (“0” in the case of 256 gradations) or the maximum pixel value (“255” in the case of 256 gradations) with a predetermined threshold as the boundary. (S3). Specifically, the RGB values of each pixel are set to the maximum pixel value (="255") if they are smaller than the threshold, and to the minimum pixel value (="0") when they are greater than or equal to the threshold. ). For example, if the RGB value of the pixel = (125, 200, 50) and the threshold value = 130, it is converted to the RGB value = (0, 255, 0).

In this way, in step S3, brightness inversion processing is performed to brighten dark areas (low brightness areas) in the image and darken bright areas (high brightness areas). This is because the central region of the eyeball (region of interest) is generally darker than the peripheral region (sclera region), so the central region of the eyeball is made brighter than the peripheral region. Note that the threshold value may be a predetermined value so that when the pixel values are converted, the central area of the eyeball becomes brighter and the peripheral area becomes darker. FIG. 6 shows an image after performing pixel value conversion in step S3 on the image in FIG.

Next, the image after step S3 is converted into a grayscale image (S4). FIG. 7 is an image obtained by converting the image in FIG. 6 to gray scale.

Next, a binarization process is performed to change the color of each pixel of the grayscale image obtained in step S4 to either white or black (S5). Specifically, each pixel value of a grayscale image is set to a value indicating white (for example, "255") when it is equal to or greater than the threshold value, and a value indicating black when it is smaller than the threshold value (for example, "0"). ”). Any known method may be used for the binarization process, and for example, the Grand Law may be used. The grand law is one of the discriminant analysis methods, and is a method for automatically determining the optimal threshold value based on the density histogram of an image. For example, as shown in FIG. 15, when the density histogram of an image is divided into class 0 and class 1 based on a certain threshold value T, the following method is used to obtain the threshold value T.
・Class 0 and class 1 are far apart.
・Data groups within each class are grouped together.

In a frontal image of an eyeball, a density difference (brightness difference) is likely to occur between the central region of the eyeball (pupil, iris, and circular region of the cornea) and the peripheral region (sclera region). Therefore, by using the grand law, it is easy to obtain a threshold suitable for dividing the central region and the peripheral region. FIG. 8 shows an image after the image in FIG. 7 has been binarized. As shown in FIG. 8, a binarized image is obtained in which the central area of the eyeball is white and the peripheral area is black.

Note that steps S3 to S5 correspond to the binarization processing step of the present disclosure. The CPU 6 that executes steps S3 to S5 corresponds to a binarization processing section.

Next, noise areas in the binarized image obtained in step S5 are removed (S6, S7). Specifically, first, as a first noise removal process, for example, a known morphological transformation is applied to the binarized image (S6). For example, noise in the binarized image can be removed by applying opening processing of morphological transformation to the binarized image. Note that the above-mentioned opening process is a process in which expansion is performed after contraction. FIG. 9 shows the image after applying morphological transformation to the image of FIG.

Next, as a second noise removal process, noise is removed based on the area of each region in the binarized image after step S6 (S7). Specifically, the outline of an area with a pixel value of "255" (white) in the binarized image is obtained, and the area of each area surrounded by the outline is determined. At this time, even if the white area is the same, each non-contiguous area is treated as a separate area and its area is calculated. The area may be, for example, the number of pixels included in the area. Then, the white region with the largest area is left, and the other white regions are removed as noise. Specifically, the color of the white area having an area other than the maximum area is inverted to the background color (black). FIG. 10 shows an image after performing area-based noise removal on the binarized image of FIG. Note that steps S6 and S7 correspond to a noise removal step. The CPU 6 that executes steps S6 and S7 corresponds to a noise removal section.

Next, a circle (shape corresponding to the region of interest) in the binarized image obtained in step S7 is detected (S8). In other words, elements having circular features in the binarized image are detected. For example, the detection of circles may be performed based on the Hough transform. Note that step S8 corresponds to the shape detection step of the present disclosure. The CPU 6 that executes step S8 corresponds to the shape detection section.

Next, a mask is generated that hides the area other than the circle detected in step S8 (in other words, the area outside the circle) (S9). Here, a circle with a diameter and center position corresponding to the circular element detected in step S8 is set, and a mask is generated in which the inner region of the circle is opened and the outer region of the circle is hidden. FIG. 11 illustrates a mask generated based on the image of FIG. 10. Note that step S9 corresponds to the generation step of the present disclosure. The CPU 6 that executes step S9 corresponds to a generation unit.

Next, the mask generated in step S9 is applied to the original image acquired in step S1 (S10). Note that step S10 corresponds to a mask processing step. The CPU 6 that executes step S10 corresponds to a mask processing section.

FIG. 12 shows an image after applying the mask of FIG. 11 to the image of FIG. 4. Through steps S2 to S10 described above, a region of interest is detected (extracted) from the original image acquired in step S1. Detection of the region of interest in steps S2 to S10 is performed automatically. Note that steps S2 to S10 correspond to the detection step of the present disclosure. The CPU 6 that executes steps S2 to S10 corresponds to the detection section.

Next, the region of interest extracted in steps S2 to S10 is subjected to color correction according to the age of the person viewing the image (S11). Here, the attention area is set so that when the subject views the image displayed on the display unit 4, the image is perceived in the same way as the color (including brightness) perceived by a young person (for example, a 20-year-old person). Correct each pixel value. Further, in step S11, color correction is not performed on areas other than the attention area of the original image. Note that step S11 corresponds to a correction step. The CPU 6 that executes step S11 corresponds to a correction section.

FIG. 3 illustrates a detailed flowchart of the color correction in step S11. Shifting to the process of FIG. 3, the CPU 6 sets the age of the person viewing the image (S20). Specifically, the CPU 6 acquires the age information of the target person input from the input unit 3, and sets the age indicated by the acquired age information as the target age. The age information may be the exact age of the subject, the approximate age (eg, 60's), or the year the subject was born (eg, according to the Western calendar). Further, for example, when a person under the age of 20 is defined as a young person, and a person older than 20 years old is defined as an elderly person, the target person may be an elderly person. In this case, in step S20, an age higher than 20 years old is set.

Next, the color display system of each pixel in the region of interest extracted in steps S2 to S10 is converted from the RGB display system to the XYZ display system (S21). The values of each pixel (R, G, B) are converted into tristimulus values (X, Y, Z) in the XYZ display system. Here, conversion from R (red), G (green), and B (blue) to X, Y, and Z is expressed by the following equations (E1) to (E3).
X=a ₁₁ R+a ₁₂ G+a ₁₃ B (E1)
Y=a ₂₁ R+a ₂₂ G+a ₂₃ B (E2)
Z=a ₃₁ R+a ₃₂ G+a ₃₃ B (E3)

The coefficients a ₁₁ , a ₁₂ , a ₁₃ , a ₂₁ , a ₂₂ , a ₂₃ , a ₃₁ , a ₃₂ , and a ₃₃ are the contribution of R to X, the contribution of G to X, and the contribution of B to X, respectively. Contribution, contribution of R to Y, contribution of G to Y, contribution of B to Y, contribution of R to Z, contribution of G to Z, contribution of B to Z is considered to be a quantity that represents These nine parameters a ₁₁ to a ₃₃ are known constants, for example, a ₁₁ =0.4124, a ₁₂ =0.3576, a ₁₃ =0.1805, a ₂₁ =0.2126, a ₂₂ =0.7152, a ₂₃ =0.0722, a ₃₁ =0.0193, a ₃₂ =0.1192, and a ₃₃ =0.9505.

Here, X _R , X _G , X _B , Y _R , Y _G , Y _B , Z _R , Z _G , and Z _B are defined by the following formulas (E4) to (E6). Furthermore, by using these, formulas (E7) to (E9) can be obtained.
X _R = a ₁₁ R, X _G = a ₁₂ G, X _B = a ₁₃ B (E4)
Y _R = a ₂₁ R, Y _G = a ₂₂ G, Y _B = a ₂₃ B (E5)
Z _R = a ₃₁ R, Z _G = a ₃₂ G, Z _B = a ₃₃ B (E6)
X=X _R +X _G +X _B (E7)
Y= _YR + _YG + _YB (E8)
Z=Z _R +Z _G +Z _B (E9)

Next, as a color correction, the tristimulus values X, Y, Z obtained by the conversion in step S21 are used to reduce the influence of changes (decrease) in light transmittance of the crystalline lens due to yellowing of the crystalline lens due to aging. is corrected (S22). Here, based on the light transmittance of the crystalline lens at the target age obtained in step S20 and the light transmittance of the crystalline lens at a predetermined reference age (for example, 20 years old) as the age of a young person, the target person is The tristimulus values X, Y, and Z are corrected so that the color vision when viewing the image approaches that of a person of a reference age. Specifically, the tristimulus values _, Z, respectively, are _individually adjusted ( _corrected ). More specifically, the corrected tristimulus values X', Y', and Z' are determined by the following equations (E10), (E11), and (E12).
X'=X _R /K _R ^(X) +X _G /K _G ^(X) +X _B /K _B ^(X) (E10)
Y'=Y _R /K _R ^(Y) +Y _G /K _G ^(Y) +Y _B /K _B ^(Y) (E11)
Z'=Z _R /K _R ^(Z) +Z _G /K _G ^(Z) +Z _B /K _B ^(Z) (E12)

Here, K _R ^(X) , K _G ^(X) , K _B ^(X) , K _R ^{(Y), K G (Y)} _, ^K _B ^(Y) , K _R ^(Z) , K _G ^(Z) , K _B ^(Z) is a value defined as an effective component ratio, and is determined as follows.

When the decline in color vision due to aging is not considered, the above-mentioned X _n , Y _n , Z _n (n=R, G, B) are expressed by the following equations (E13) to (E15) depending on the spectral distribution of the display section 4. be done.
X _n = kΣS (λ) x (λ) f _n (λ) (E13)
Y _n = kΣS (λ) y (λ) f _n (λ) (E14)
Z _n = kΣS (λ) z (λ) f _n (λ) (E15)

In the above equations (E13) to (E15), λ is the wavelength of visible light. S(λ) is the spectral distribution of the D65 white point with respect to the wavelength λ. x(λ), y(λ), and z(λ) are color matching functions of X, Y, and Z at wavelength λ, respectively. f _n (λ) (n=R, G, B), that is, f _R (λ), f _G (λ), and f _B (λ) are the spectral distributions of red, green, and blue in the display section 4, respectively. . k is a suitable constant. The range of the sum in Σ is determined to include the entire visible light wavelength range, and specifically may be, for example, from λ=380 to λ=780.

The above is a case of a young healthy person, but this changes with age as follows. According to the well-known research by Pokorny et al., A: the lens transmission ratio, which is the ratio of the light transmittance of the crystalline lens of a _{2 -} year-old ( _A2 may be greater than or equal to _A1 ) to the light transmittance of the crystalline lens of a 1-year-old. F is described by the following equation (E16). Note that ^ is a power, and L is the optical density of the crystalline lens.
F(λ, A ₂ , A ₁ )=10＾{-L(λ, A ₂ )}/10＾{-L(λ, A ₁ )}=10＾{L(λ, A ₁ )-L( λ, A ₂ )} (E16)

The optical density L of the crystalline lens is expressed by the following formula (E17) when A is greater than 20 and 60 or less, and by formula (E18) when A is greater than 60, where A is the age. Note that T _L1 and T _L2 are appropriate constants.
L(λ, A)=T _L1 (1+0.02(A-32))+T _L2 (E17)
L(λ, A)=T _L1 (1.56+0.0667(A-60))+T _L2 (E18)

When the above F is used, equations (E13) to (E15) change to the following equations (E19) to (E21) due to aging.
X _n ^* = kΣS (λ) x (λ) f _n (λ) F (λ, A ₂ , A ₁ ) (E19)
Y _n ^* = kΣS (λ) y (λ) f _n (λ) F (λ, A ₂ , A ₁ ) (E20)
Z _n ^* = kΣS (λ) z (λ) f _n (λ) F (λ, A ₂ , A ₁ ) (E21)

X _n ^* , Y _n ^* , Z _n ^* shown in equations (E19) to (E21) are the same colors as the colors (tristimulus values X _n , Y _n , Z _n ) perceived by a _1- year-old person. A shows the tristimulus values of colors perceived by a ₂ -year-old (in other words, the reproduction of colors perceived by a _2- year-old). Here, A ₁ is the reference age, and A ₂ is the target age obtained in step S10. In this case, X _n ^* , Y _n ^* , and Z _n ^* represent colors that reproduce the color vision of a person of target age A ₂ when the color vision of a person of reference age A ₁ is used as a reference. Further, it is assumed that the colors (X _n , Y _n , Z _n ) shown by the above equations (E13) to (E15) are colors perceived by a person of the standard age.

Then _, X _{n * ,} ^Y _{n * ,} Z _n ^{* (} n=R , G, B). That is, the effective component ratio is defined by the following equations (E22) to (E24).
K _n ^(X) = X _n ^* /X _n (E22)
K _n ^(Y) = Y _n ^* / Y _n (E23)
K _n ^(Z) = Z _n ^* / Z _n (E24)

The effective component ratios K _n ^(X) , K _n ^(Y) , and K _n ^(Z) are yellow, which indicates a change in color vision (deterioration of color vision) due to yellowing of the lens due to aging (age difference between reference age and target age). It is a strange filter. These effective component ratios K _n ^(X) , K _n ^(Y) , K _n ^(Z) (n=R, G, B) are used in the above equations (E10) to (E12). _{In the} yellowing correction ^of formulas (E10) ^to (E12 ₎ , ^the tristimulus value , Y _, and Z respectively, the effective component ratios K _n ⁽ X ₎ , _{K n (} ^Y ), K _n ⁽ The reciprocal of ^Z) is multiplied as a correction coefficient. In this way, the correction coefficients, which are the reciprocals of the effective component ratios K _n ^(X) , K _n ^(Y) , and K _n ^(Z) , are the R, G, and B components of each stimulus value X, Y, and _Z. It is set for each Y _n and Z _n .

The yellowing correction coefficient (the reciprocal of the effective component ratio K _n ^(X) , K _n ^(Y) , K _n ^(Z) ) is calculated in advance for each combination of reference age A ₁ and target age A ₂ , and is shown in FIG. The information may be stored in advance in the storage unit 7 shown in FIG. Alternatively, during the process of step S12, the effective component ratios K _n ^(X) , K _n ^(Y) , K _n ^(Z) and their inverse yellowing correction coefficients are calculated based on equations (E22) to (E24). It's okay. Note that the reference age A ₁ may be set to an arbitrary age through an input operation using the input unit 3 .

Referring to equations (E4) to (E9), the nine numerical values indicating the contribution of R, G, and B to the tristimulus values X, Y, and Z before yellowing correction are a ₁₁ , a ₁₂ , a ₁₃ , a ₂₁ , a ₂₂ , a ₂₃ , a ₃₁ , a ₃₂ , and a ₃₃ . On the other hand, the nine numerical values representing the contribution of R, G, and B in the tristimulus values X', Y', and Z' after yellowing correction shown by equations (E10) to (E12) are a ₁₁ /K _R ^(X) , a ₁₂ /K _G ^(X) , a ₁₃ /K _B ^(X) , a ₂₁ /K _R ^(Y) , a ₂₂ /K _G ^(Y) , a ₂₃ /K _B ^{(Y )} , a ₃₁ /K _R ^(Z) , a ₃₂ /K _G ^(Z) , and a ₃₃ /K _B ^(Z) . That is, yellowing correction is synonymous with individually adjusting (correcting) nine numerical values indicating the contribution of R, G, and B of the RGB display system in each of the tristimulus values X, Y, and Z.

The yellowing correction in step S12 is performed as described above. Next, using the following equations (E25) to (E27), the value of each pixel after yellowing correction is calculated from the tristimulus values X', Y', Z' to the RGB display system values (R', G', B ') (S23).
R'=b ₁₁ X'+b ₁₂ Y'+b ₁₃ Z' (E25)
G'=b ₂₁ X'+b ₂₂ Y'+b ₂₃ Z' (E26)
B'=b ₃₁ X'+b ₃₂ Y'+b ₃₃ Z' (E27)

The coefficients b ₁₁ , b ₁₂ , b ₁₃ , b ₂₁ , b ₂₂ , b ₂₃ , b ₃₁ , b ₃₂ , and b ₃₃ are the contribution of X to R, the contribution of Y to R, and the contribution of Z to R, respectively. Contribution, contribution of X to G, contribution of Y to G, contribution of Z to G, contribution of X to B, contribution of Y to B, contribution of Z to B is considered to be a quantity that represents These nine parameters from b ₁₁ to b ₃₃ are known constants, for example, b ₁₁ = 3.2406, b ₁₂ = -1.5372, b ₁₃ = -0.4986, b ₂₁ = -0.9689. , b ₂₂ =1.8758, b ₂₃ =0.0415, b ₃₁ =0.0557, b ₃₂ =-0.2040, b ₃₃ =1.0570.

FIG. 13 shows an image after performing color correction in step S11 on the image of the region of interest in FIG. Returning to FIG. 2, next, the image of the region of interest after color correction in step S11 is combined with the original image acquired in step S1 (S12). Specifically, the region of interest in the original image is replaced with the region of interest after color correction. FIG. 14 shows an image (locally corrected image) obtained by combining the color-corrected region of interest with the original image.

Next, the locally corrected image obtained in step S12 is output to the display unit 4 for display (S13). Thereby, the locally corrected image is presented to the subject (operator). Note that steps S12 and S13 correspond to output steps. The CPU 6 that executes steps S12 and S13 corresponds to an output section.

After that, the process returns to step S1, and the above-described steps S1 to S13 are performed on the image at the next point in time. In this way, steps S1 to S13 are repeated for images at each point in time constituting the moving image shot by the shooting unit 2.

As described above, in this embodiment, since color correction is performed only on the region of interest, it is possible to reduce the proportion of overexposure in the image after color correction. Here, Table 1 below shows the difference between overexposed areas for the entire corrected image when color correction is applied to the entire image and when color correction is performed only to the area of interest for the same original image. The percentages are shown for each age group. Note that the "whiteout reduction rate" in Table 1 is the difference between the whiteout rate (overall correction) and the whiteout rate (local correction). As shown in Table 1, local correction of only the region of interest reduces the overexposure ratio compared to overall correction, and the reduction ratio is particularly large as the age increases.

Moreover, FIG. 16 illustrates the brightness distribution of the original image acquired in step S1. The reference numeral "101" in FIG. 16 indicates the brightness distribution of a circular region (region of interest) at the center of the eyeball in the original image. Reference numeral "102" indicates the brightness distribution of an area other than the attention area in the original image.

FIG. 17 exemplifies the brightness distribution when only the corrected image obtained in steps S1 to S12, that is, the region of interest of the original image, is color corrected. The reference numeral "201" in FIG. 17 indicates the brightness distribution of a circular region (region of interest) at the center of the eyeball in the locally corrected image. The code “202” indicates the brightness distribution of an area other than the attention area in the locally corrected image. The luminance distribution of symbol "102" in FIG. 16 and the luminance distribution of symbol "202" in FIG. 17 are the same.

FIG. 18 shows the brightness distribution when the same color correction as in step S11 is performed on the entire original image (that is, both the attention area and other areas) shown by the brightness distribution in FIG. Illustrated. Reference numeral "301" in FIG. 18 indicates the brightness distribution of a circular region (region of interest) at the center of the eyeball in the entire corrected image. Reference numeral "302" indicates the brightness distribution of an area other than the attention area in the entire corrected image. The luminance distribution of symbol "201" in FIG. 17 and the luminance distribution of symbol "301" in FIG. 18 are the same. Note that FIGS. 17 and 18 show the luminance distribution when color correction is performed with the age of the subject being 70 years old.

As shown in FIG. 18, when color correction is performed on the entire image, the difference in brightness between the region of interest and other regions (particularly the difference in brightness on the high brightness (high pixel value) side) is large. On the other hand, when color correction is performed only on the region of interest in the image, the difference in brightness between the region of interest and other regions is smaller than that in FIG. 18, as shown in FIG.

In this manner, the local correction according to the present embodiment can reduce the unnaturalness of a large brightness difference between the region of interest and other regions.

Furthermore, in this embodiment, since color correction is performed on the region of interest in the ophthalmologic surgery image, it is possible to facilitate the ophthalmologic surgery even if the operator is an elderly person. Since the circular region at the center of the eyeball is color-corrected as the region of interest, it is easier for the surgeon to perform cataract surgery (removal of crystalline lens, insertion of intraocular lens, etc.).

Furthermore, in the embodiment described above, since the region of interest is automatically detected, it is possible to perform color correction on the moving image photographed by the photographing unit 2 in real time. Further, since the shape corresponding to the region of interest is detected after removing the noise region in the binarized image, the detection accuracy can be increased.

Note that the present disclosure is not limited to the above embodiments, and various changes are possible. For example, in the embodiment described above, an example in which an ophthalmic surgery image is color-corrected is shown, but a surgical image of a site other than the eye may be color-corrected. Further, for example, an eye image obtained during a treatment other than surgery (for example, during an eye examination) may be color-corrected. Furthermore, color correction may be performed on images other than those for surgery or examination.

In the above embodiment, yellowing correction is exemplified as color correction, but the color correction method is not limited to the method of this embodiment. Further, in addition to or in place of the yellowing correction, a correction (miosis correction) may be performed to cancel the effect of reduction in pupil diameter due to aging. In this case, the γ correction described in Japanese Patent No. 7120556 may be performed.

Further, in the above embodiment, an example was shown in which the locally corrected image is output to the display unit 4, but the locally corrected image may be output to an image projection device (projector), or the locally corrected image may be printed on a paper medium or the like. The image may be output to an image printing device.

1 Display system 2 Photographing section 3 Input section 4 Display section 5 Arithmetic device (image processing device)
6 CPU
7 Storage section 8 Program

Claims (10)

an image acquisition unit that acquires an image;
a detection unit that detects a region of interest in the image;
a correction unit that performs color correction on the attention area detected by the detection unit according to the age of a subject viewing the image;
an output unit that outputs a locally corrected image including the color-corrected region of interest and other regions;
An image processing device comprising: The image processing device according to claim 1, wherein the image is a surgical image. The image processing device according to claim 1 or 2, wherein the image is an image of an eye. The image processing device according to claim 3, wherein the detection unit detects a circular area in the image as the region of interest. the image is a video;
The image processing device according to claim 1, wherein the correction unit performs the color correction on images at each time point of the moving image. 1 . The detection unit includes a binarization processing unit that performs a binarization process on the image, and detects the region of interest based on a binarized image that is the image that has been subjected to the binarization process. The image processing device described in . The detection unit includes:
a shape detection unit that detects a shape corresponding to the region of interest in the binarized image;
a generation unit that generates a mask that hides a region other than the shape detected by the shape detection unit;
a mask processing unit that applies the mask to the image before performing the binarization process;
The image processing device according to claim 6, comprising: The detection unit includes a noise removal unit that removes a noise region in the binarized image,
The image processing device according to claim 7, wherein the shape detection unit detects a shape corresponding to the region of interest based on the binarized image after removing the noise region. an image acquisition step of acquiring an image;
a detection step of detecting a region of interest in the image;
a correction step of performing color correction on the region of interest detected in the detection step according to the age of the subject viewing the image;
outputting a locally corrected image including the color-corrected region of interest and other regions;
An image processing method comprising: an image acquisition step of acquiring an image;
a detection step of detecting a region of interest in the image;
a correction step of performing color correction on the region of interest detected in the detection step according to the age of the subject viewing the image;
outputting a locally corrected image including the color-corrected region of interest and other regions;
A program that causes a computer to execute.

Images