CN106446859B - Utilize the method for stain and the trace of blood in mobile phone front camera automatic identification human eye - Google Patents

Abstract

The invention discloses a method for automatically identifying black spots and blood streaks in human eyes by using a front camera of a mobile phone. The solution is: 1) identify the distance d between the portrait and the camera through the distance sensor on the smartphone; 2) keep the original resolution of the image formed by the mobile phone unchanged within the distance d greater than the threshold D, and keep the original resolution of the image formed by the mobile phone unchanged when the distance d is less than Interpolation is performed when the threshold is D, and the resolution of the resulting image is increased so that the human eye image can be displayed more accurately in the camera; 3) The interpolated human eye image is sequentially processed by image grayscale, sobel edge detection, image segmentation, and image binarization ; 4) Cut out the monocular image from the processed human eye image; 5) Identify the bloodshot and black spots that appear in the monocular image; 6) And automatically give the user a prompt. The invention combines the smart phone with healthy life, and can be used to increase the functions of the mobile phone to provide automatic identification and prompts for tiny bloodshots and black spots appearing in human eyes.

Description

Method for automatically identifying black spots and bloodshot eyes in human eyes by using the front camera of a mobile phone

technical field

The invention belongs to the technical field of image processing, and in particular relates to a method for improving the resolution of an image formed by a front camera and automatically identifying black spots and bloodshot eyes in human eyes, which can be used to increase the functions of mobile phones.

With the advancement of science and technology, owning a smart phone has become a common phenomenon, and people's requirements for a healthy life are getting higher and higher. In addition to using mobile phones for basic communication, people often use the front camera of the mobile phone to take selfies, but when adjusting the focus to obtain more detailed information, the definition is reduced and a very clear image cannot be obtained. In addition, people often use the front camera as a mirror, hoping to see both the overall situation and the local area, and when looking at the local area, they hope that the clearer the better, so they adjust the focal length of the camera, but they can’t get a very clear image, so they can’t It is very good to observe some more detailed information of my face.

Most of the mainstream high-end mobile phones on the market now have a front camera with much lower pixels than the rear camera. For example, the front camera of Huawei P9 has 8 million pixels, and the front camera of ZTE ZTE AKON 7 has 8 million pixels. Pixels, iphone7 front camera pixels are 7 million pixels, OPPO R9 front camera pixels are 16 million pixels, these mainstream high-end mobile phone front cameras will have small pits when zooming in close, especially when people use mobile phones The front camera shooting function acts as a mirror to observe the details of your eyes, but you can’t see a clear image by adjusting the focal length of the camera, and there is no automatic recognition function for details such as bloodshot eyes and black spots. It is far from meeting people's growing demand for health.

Contents of the invention

The purpose of the present invention is to address the above-mentioned deficiencies in the prior art, and provide a method for automatically identifying black spots and bloodshot eyes in people's eyes by using the front camera of a mobile phone, so as to meet people's growing needs for health.

To achieve the above object, technical solutions of the present invention include as follows:

(1) recognize the distance d between the portrait and the camera by the distance sensor on the smart phone;

(2) Set the distance threshold R=1m, and compare the distance d between the portrait and the camera with the set threshold R: if d>R, keep the original resolution of the image formed by the mobile phone unchanged; if d<R , then execute step (3);

(3) Use the bilinear interpolation algorithm to interpolate the image formed by the front camera, and the interpolation number N is Integer multiples of , so that the human eye image can be displayed more accurately in the camera;

(4) Carry out grayscale and sobel operator edge detection on the interpolated human eye image, and find the area of the human eye in the x and y directions according to the edge detection result, discard the points that are not the human eye area, and complete the human eye clipping, and then Performing binarization processing on the cropped human eye image to obtain a binarized image;

(5) Perform horizontal and vertical calculations on the binarized image, scan and segment the monocular image, and use the adjacent interpolation algorithm to normalize the monocular image k into a 32*16 monocular image h;

(6) Select a normal human monocular binarized image with a size of 32*16 and no bloodshot and black spots as the template image H, and use the template image H to make a difference with the normalized monocular image h to obtain the human eye difference image h' ;

(7) Cut the difference image h' to obtain the abnormal point image h", and normalize the abnormal point image h" into a 32*16 size abnormal point image h"' through the adjacent interpolation algorithm;

(8) Calculate the number C of white points in the abnormal point image h"' after normalization, and calculate the information ratio g between the number C of white points and 32*16;

(9) Set the threshold G=0.6835 for judging black spots and bloodshot spots, compare the size of g and G, if g=<G, it is judged that the eyes have black spots, and if 1>g>G, then it is judged that the eyes have bloodshot eyes point, if g=1, it is judged that the eyes are normal;

(10) Display the judgment result to the user on the camera interface of the front camera.

The present invention has the following advantages:

1. On the basis of the photographing function of the front camera of the mobile phone, the present invention first utilizes interpolation technology to improve the resolution of the photographed image, and automatically recognizes bloodshot eyes and Black dots can not only improve the performance of the front camera of the existing mobile phone, but also increase the user's camera experience and make people's life more intelligent and convenient;

2. The present invention can dynamically adjust the resolution of the image taken by the front camera of the mobile phone according to the distance between the portrait and the mobile phone. The closer the distance, the higher the resolution of the image, so that people can see the small features of the eye more clearly;

3. The present invention can automatically identify tiny blood streaks and black spots in human eyes, which provides convenience for users to know their own eye conditions in time.

Description of drawings

Fig. 1 is the realization flowchart of the present invention;

Fig. 2 is a schematic diagram of bilinear interpolation in the present invention;

Fig. 3 is a relationship diagram between camera resolution and distance d in the present invention;

Detailed ways

The present invention is described in detail below in conjunction with accompanying drawing:

With reference to Fig. 1, the realization steps of the present invention are as follows:

Step 1, obtain the distance between the portrait and the camera, that is, the distance d between the portrait and the camera is measured by reading the distance sensor of the mobile phone.

Step 2, judge whether to change the resolution of the image formed by the front camera according to the distance d.

(2a) Set the distance threshold R=1m, compare the distance d between the portrait and the camera with the set threshold R: if d>R, keep the image resolution unchanged; if d<R, then Execute step (2b);

(2b) Use the bilinear interpolation algorithm to interpolate the portrait formed by the front camera, and the interpolation number N is Integer multiples of , so that the human eye image can be displayed more accurately in the camera;

(2c) Starting from the coordinates (0,0) of the portrait image, scan all the pixels in the image in sequence, let the scanned pixel coordinates be (x, y), and then get based on (x, y) Four scanning pixels, namely A(x,y), B(x+1,y), C(x,y+1), D(x+1,y+1), let these scanning pixels The sizes are f(A), f(B), f(C), f(D);

(2d) According to the distance d between the portrait and the camera, A(x,y), B(x+1,y), C(x,y+1), D(x+1,y+1) four The number of interpolation points in the cube area enclosed by the points is: where the constant δ=10;

(2e) Use the bilinear interpolation algorithm to determine the interpolation value of the four points A(x,y), B(x+1,y), C(x,y+1), and D(x+1,y+1) Coordinates of points and size in pixels of interpolated points:

(2e1) Let the coordinates of the final interpolation point be (x', y'), as shown in Figure 2.

(2e2) Calculate the interpolation point in the x-axis direction:

First calculate the intermediate interpolation point P1(x’,y) of A(x,y), B(x+1,y), the pixel size of the interpolation point P1 is:

f(P1)=(x+1-x’)*f(A)+(x’-x)*f(B);

Then calculate the intermediate interpolation point P2(x’,y) of C(x,y+1), D(x+1,y+1), the size of the interpolation point pixel is:

f(P2)=(x+1-x')*f(C)+(x'-x)*f(D);

(2e3) Use M(x',y') to represent the interpolation point between P1(x',y) and P2(x',y), the size of the interpolation point pixel is:

It can be known from the above formula that when (x, y) is determined, the coordinate size of the interpolation point x’ takes a value between (x, x+1), and y’ takes a value between (y, y+1);

(2e4) Further determine the coordinates of the interpolation points in A(x, y), B(x+1, y), C(x, y+1), D(x+1, y+1) four points are The pixel size of the interpolation point is:

where n=1,2...N;

(2e5) The relationship between the image resolution and the distance d after the interpolation is completed is shown in FIG. 3 .

Step 3, preprocessing the interpolated human eye image.

(3a) grayscale human eye image, that is, the tool function rgb2gray function of image processing is used to make the color image into a grayscale image;

(3b) Detect image edges

Operators that can be used to detect image edges include robert operator, sobel operator, prewitt operator, krisch operator, laplacian operator, gauss-laplacian operator, etc. This step uses sobel edge detection.

Step 4, human eye cropping.

(4a) corrode the detected human eye image, that is, use the image processing basic function imerode to remove the small object area in the human eye, and remove the segmentation interference item;

(4b) Scan to find the area of the human eye in the x and y directions, and discard the points that are not in the human eye area to complete the human eye clipping.

Step 5, binarize the cropped human eye image

(5a) Obtain the optimal threshold T for binarization:

The algorithms for obtaining the threshold include bimodal method, P parameter method, Otsu method, maximum threshold method, optimal threshold method, etc. In this example, the optimal threshold method is used to obtain the optimal threshold T for binarization;

(5b) Starting from the coordinates (0,0) of the cropped human eye image, scan all the pixels in the image in sequence, so that the scanned pixel coordinates are (x, y), and the pixel value is f( x,y);

(5c) Compare the pixel value f(x,y) with the threshold T, if the pixel value f(x,y) is less than the threshold T, then f(x,y) is 0, if the pixel value f(x,y) Greater than or equal to the threshold T, then f(x,y) is 1.

Step 6, segment the monocular image.

Starting from the coordinate (0,0) point of the human eye image after binarization, scan all the pixels in the image in sequence, and discard the points where the sum of pixels in the x direction is 0, and the points where the sum of pixels in the y direction is 0 , segment two single-eye images, that is, the left-eye image k1 and the right-eye image k2.

Step 7, normalize the left-eye image, and use the adjacent interpolation algorithm to normalize the left-eye image k1 into a left-eye image h1 with a size of 32*16.

(7a) Perform a geometric transformation on the left-eye image k1 to obtain a transformed image k' with a size of 32*16;

(7b) Starting from the coordinate (0,0) of the image k' obtained after transformation, scan all the pixels (x', y') in the image in turn, and find the distance (x', y) in the left-eye image k1 ') the nearest point (x, y), let the pixel value k'(x', y') of the (x', y') point be equal to the pixel value k(x, y) of the (x, y) point, this The image k' obtained when is the left eye image h1;

Use the same method to process the right-eye image to obtain the right-eye image h2 normalized to a size of 32*16.

Step 8, obtaining the human eye difference image.

(8a) Select a binarized image of the left eye of a normal person with a size of 32*16 and no bloodshot and black spots as the left eye template image H1, and use the difference between the left eye template image H1 and the normalized left eye image h1 to obtain the left eye difference value image h1';

(8b) Select a binarized image of the right eye of a normal person with a size of 32*16 and no bloodshot and black spots as the right eye template image H2, and use the right eye template image H2 to make a difference with the normalized right eye image h2 to obtain the right eye difference value image h2';

Step 9, obtain the outlier point image.

(9a) Cut the left-eye difference image h1' to obtain the left-eye abnormal point image h1", and use the adjacent interpolation algorithm to normalize the left-eye abnormal point image h1" into a left-eye abnormal point image h1"' with a size of 32*16 ;

(9a) Cut the right-eye difference image h2' to obtain the right-eye abnormal point image h2", and normalize the right-eye abnormal point image h2" into a right-eye abnormal point image h2"' with a size of 32*16 through the adjacent interpolation algorithm.

Step 10, calculate the information ratio.

(10a) Calculate the number C1 of white points in the left-eye abnormal point image h1"' after normalization, that is, start from the coordinate (0,0) point of the left-eye abnormal point image h1"', scan all the pixels of the image, Sum the points with a pixel value of 1 to obtain the white point C1 in the left eye, and calculate the information ratio g1 between the white point C1 in the left eye and 32*16;

(10b) Calculate the number of white points C2 in the right-eye abnormal point image h2"' after normalization, that is, start from the coordinate (0,0) point of the right-eye abnormal point image h2"', scan all the pixels of the image, Sum the points with a pixel value of 1 to obtain the white point C2 in the right eye, and calculate the information ratio g2 between the white point C2 in the right eye and 32*16;

Step 11, determine whether there are blood streaks and black spots.

(11a) Setting the threshold G=0.6835 for judging black spots and bloodshot spots;

(11b) Compare the size of g1 and G. If g1=<G, it is judged as black spots in the left eye; if 1>g1>G, it is judged as bloodshot in the left eye; if g1=1, it is judged as left eye normal;

(11b) Compare the size of g2 and G. If g2=<G, it is judged that there is a black spot in the right eye; if 1>g2>G, it is judged that there is a bloodshot spot in the right eye; Eyes are normal.

Step 12, prompting the user for the determination result on the camera interface of the front camera.

The user can go to the hospital for treatment in time according to the prompt of the mobile phone, so as not to delay the illness and protect the health of the eyes.

The above description is only a specific example of the present invention. Obviously, for those skilled in the art, after understanding the content and principle of the present invention, it is possible to carry out the form and details without departing from the principle and structure of the present invention. Various amendments and changes, but the amendments and changes of these basic inventive concepts are still within the protection scope of the claims of the present invention.

Claims (5)

1. A system that utilizes the front camera of a mobile phone to automatically identify black spots and bloodshot eyes, the system performs the following functional steps: (1) recognize the distance d between the portrait and the camera by the distance sensor on the smart phone; (2) Set the distance threshold R=1m, and compare the distance d between the portrait and the camera with the set threshold R: if d>R, keep the original resolution of the image formed by the mobile phone unchanged; if d<R , then execute step (3); (3) Use the bilinear interpolation algorithm to interpolate the image formed by the front camera, and the interpolation number N is an integer multiple of (1-d _R ), so that the human eye image can be displayed more accurately in the camera; (4) Carry out grayscale and sobel operator edge detection on the interpolated human eye image, and find the area of the human eye in the x and y directions according to the edge detection result, discard the points that are not the human eye area, and complete the human eye clipping, and then Performing binarization processing on the cropped human eye image to obtain a binarized image; (5) Perform horizontal and vertical calculations on the binarized image, scan and segment the monocular image, and use the adjacent interpolation algorithm to normalize the monocular image k into a 32*16 monocular image h; (6) Select a normal human monocular binarized image with a size of 32*16 and no bloodshot and black spots as the template image H, and use the template image H to make a difference with the normalized monocular image h to obtain the human eye difference image h' ; (7) Cut the difference image h' to obtain the abnormal point image h", and normalize the abnormal point image h" into a 32*16 size abnormal point image h"' through the adjacent interpolation algorithm; (8) Calculate the number C of white points in the abnormal point image h"' after normalization, and calculate the information ratio g between the number C of white points and 32*16; (9) Set the threshold G=0.6835 for judging black spots and bloodshot spots, compare the size of g and G, if g=<G, it is judged that the eyes have black spots, and if 1>g>G, then it is judged that the eyes have bloodshot eyes point, if g=1, it is judged that the eyes are normal; (10) Display the judgment result to the user on the camera interface of the front camera. 2. system according to claim 1, wherein utilize bilinear interpolation algorithm to carry out interpolation to the image that front camera is formed in the step (3), carry out as follows: (3a) Starting from the coordinates (0,0) of the portrait image, scan all the pixels in the image in sequence, let the scanned pixel coordinates be (x, y), and then use (x, y) as the basis to obtain Four scanning pixels, namely A(x,y), B(x+1,y), C(x,y+1), D(x+1,y+1), let these scanning pixels The sizes are f(A), f(B), f(C), f(D); (3b) According to the distance d between the portrait and the camera, A(x,y), B(x+1,y), C(x,y+1), D(x+1,y+1) four The number of interpolation points in the square area enclosed by the points is: Where constant δ=10, distance threshold R=1m; (3c) Determine the coordinates of the interpolation points among the four points A(x,y), B(x+1,y), C(x,y+1), and D(x+1,y+1) as Calculate the pixel size of the interpolation point as: where n=1,2...N. 3. system according to claim 1, wherein in the step (4), carry out binarization processing to the human eye image after cropping, carry out as follows: (4a) using the maximum variance threshold segmentation method to obtain the optimal threshold T of binarization; (4b) Starting from the coordinates (0,0) of the cropped human eye image, scan all the pixels in the image in sequence, so that the scanned pixel coordinates are (x, y), and the pixel value is f( x,y); (4c) Compare the pixel value f(x,y) with the threshold T, if the pixel value f(x,y) is less than the threshold T, then f(x,y) is 0, if the pixel value f(x,y) Greater than or equal to the threshold T, then f(x,y) is 1. 4. The system according to claim 1, wherein in the step (5), the monocular image is normalized to the monocular image h of 32*16 size with the adjacent interpolation algorithm, as follows: (5a) Geometrically transform the monocular image k to obtain a transformed image k' with a size of 32*16; (5b) Starting from the coordinate (0,0) point of the transformed image k', scan all the pixels (x', y') in the image in turn, and find the closest distance (x', y') in the monocular image k The point (x, y) of the point (x', y'), let the pixel value k'(x', y') of the point (x', y') be equal to the pixel value k(x, y) of the point (x, y), and the obtained Image k' is the monocular image h. 5. The system according to claim 1, wherein the white point number C in the abnormal point image h"' after calculating normalization in the step (8) is from the coordinate (0,0) point of the abnormal point image h"' At the beginning, scan all the pixels of the image, sum the points whose pixel value is 1, and obtain the number C of white points.