CN117495864A - Imaging direction computing system and diopter estimating system based on image processing - Google Patents

Abstract

The invention relates to the technical field of medical image analysis, and discloses an imaging direction calculation system and a diopter estimation system based on image processing, wherein the diopter estimation system is used for acquiring an optometry video of an eye area of a patient; dividing pupil area images from each frame of images; performing super-resolution processing on the pupil region image, and performing threshold segmentation on the super-resolution image to segment out a light reflecting region; edge detection is carried out on the mapping area, mapping edge outlines are obtained, and left and right boundary abscissas of the mapping edge outlines are obtained; transforming the left and right boundary abscissas to obtain transformed left and right boundary abscissas; obtaining left and right boundary abscissas after all frame images in the optometry video are transformed; performing linear fitting on the transformed left and right boundary abscissa sequences to obtain the moving speed of the left and right boundaries; taking the average value of the moving speed as the mapping speed; based on the reflected light velocity, diopters are calculated. The invention reduces the interference of subjective factors and accelerates the optometry process.

Description

Image motion direction calculation system and diopter estimation system based on image processing

Technical field

The present invention relates to the technical field of medical image analysis, and in particular to an image motion direction calculation system and a diopter estimation system based on image processing.

Background technique

The statements in this section merely mention the background technology related to the present invention and do not necessarily constitute prior art.

Refractive errors are the most common eye disorder and a key cause behind correctable vision impairment. Refractive errors can be diagnosed through a variety of methods, including subjective refraction, objective refraction, etc. Retinoscopy is one of the traditional objective refraction methods. It analyzes the direction and brightness of the patient's fundus image and uses different lenses to determine the patient's refractive error. The advantage of retinoscopy is that its results are objective and reliable, do not require the subject's subjective cooperation, and have strong applicability. However, the retinoscopy method also has some problems. It usually requires a long time and professional intervention, which limits its use in large-scale vision screening.

Contents of the invention

In order to solve the shortcomings of the existing technology, the present invention provides an image motion direction calculation system and a diopter estimation system based on image processing; based on the retinoscopy video, image processing technology is used, and through mathematical modeling and corresponding algorithms, the image motion is calculated direction and refractive error. The main advantages of this technology are to improve the accuracy of refractive error calculation, reduce the interference of subjective factors, and speed up the refraction process.

On the one hand, a diopter estimation system is provided, including: a first acquisition module, which is configured to: collect a refraction video of the patient's eye area; a light reflection speed calculation module, which is configured to: calculate each frame of the image in the refraction video Segment the pupil area image; perform super-resolution processing on the segmented pupil area image to obtain a super-resolution pupil area image; perform threshold segmentation on the super-resolution pupil area image to segment the light-reflecting area; perform morphology on the light-reflecting area Learn to process and perform edge detection to obtain the light-reflecting edge contour, and then obtain the left and right boundary abscissas of the light-reflecting edge contour; transform the left and right boundary abscissas to obtain the transformed left and right boundary abscissas; for the optometry video For all frame images in , the transformed left and right boundary abscissas are obtained; linear fitting is performed on the transformed left and right boundary abscissa sequences to obtain the moving speed of the left and right boundaries; the average value of the moving speed of the left and right boundaries is used as the reflection Speed; the diopter calculation module is configured to calculate the diopter based on the reflected light speed and the diopter fitting formula.

On the other hand, an image processing-based shadow movement direction calculation system is provided, including: a second acquisition module configured to: collect the optometry video of the patient's eye area; and a light movement direction calculation module configured to: Each frame of the image in the optometry video is segmented into a pupil area image; the segmented pupil area image is subjected to super-resolution processing to obtain a super-resolution pupil area image; a threshold segmentation is performed on the super-resolution pupil area image to segment the reflected light area; perform morphological processing on the light-reflecting area and perform edge detection to obtain the light-reflecting edge contour, and then obtain the left and right boundary abscissas of the light-reflecting edge contour; transform the left and right boundary abscissas to obtain the transformed The left and right boundary abscissas; for all frame images in the optometry video, the transformed left and right boundary abscissas are obtained; linear fitting is performed on the transformed left and right boundary abscissa sequences to obtain the moving speed of the left and right boundaries; the left and right boundaries are The average moving speed is used as the reflected light speed; the reflected light moving direction is calculated according to the reflected light speed; the light belt moving direction calculation module is configured to: calculate the light belt moving direction; the shadow moving direction calculation module is configured as: Based on the moving direction of the reflected light and the moving direction of the light strip, the moving direction of the shadow is calculated.

The above technical solution has the following advantages or beneficial effects: the present invention uses image processing to calculate the moving direction and diopter based on the optometry video, improves the accuracy of refractive error calculation, reduces the interference of subjective factors, and accelerates The optometry process helps to realize the automation and intelligence of the optometry process. The invention belongs to the technical field of medical image analysis, and specifically relates to a method for estimating the direction of motion and refractive error based on image processing. Among them, the method includes obtaining the optometry video, extracting the boundary of the reflection and calculating the parameters related to the reflection, extracting the boundary of the light band and calculating the parameters related to the light band, and calculating the moving direction and diopter. The invention can realize accurate identification and calculation of the moving direction and diopter, reduces the interference of subjective factors, accelerates the optometry process, and helps to realize the automation and intelligence of the optometry process.

Description of drawings

The description and drawings that constitute a part of the present invention are used to provide a further understanding of the present invention. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention.

Figure 1 is a functional module diagram of the diopter estimation system of Embodiment 1;

Figure 2 is a functional module diagram of the motion direction calculation system based on image processing in Embodiment 1.

Detailed ways

It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

Embodiment 1

As shown in Figure 1, the diopter estimation system includes: a first acquisition module, which is configured to: collect the refraction video of the patient's eye area; a light reflection speed calculation module, which is configured to: calculate each frame in the refraction video The image is segmented to obtain the pupil area image; the segmented pupil area image is subjected to super-resolution processing to obtain a super-resolution pupil area image; the super-resolution pupil area image is thresholded to segment the light-reflecting area; the light-reflecting area is processed Morphological processing and edge detection are performed to obtain the light-reflecting edge contour, and then the left and right boundary abscissas of the light-reflecting edge contour are obtained; the left and right boundary abscissas are transformed to obtain the transformed left and right boundary abscissas; for optometry For all frame images in the video, the transformed left and right boundary abscissas are obtained; linear fitting is performed on the transformed left and right boundary abscissa sequences to obtain the moving speed of the left and right boundaries; the average value of the moving speed of the left and right boundaries is used as the map. The speed of light; the diopter calculation module is configured to calculate the diopter based on the reflected light speed and the diopter fitting formula.

Further, the optometry video of the patient's eye area is collected using an image acquisition device, and the retinoscope light belt scans the optometry video at a constant speed around the patient's eye area; and during the image collection process, the patient wears A standard spectacle frame with a white background plate and two rectangular openings to expose the eye area. The white background plate can be set on the glasses frame without lenses to facilitate the patient to wear it, but it is not limited to this form. The white background plate can also be attached to the patient's face. The white background plate is equipped with a rectangular opening, and the rectangular opening can capture the patient. eye area.

When collecting images, the lens of the camera is close to the observation hole of the retinoscope, and the image is obtained through the observation hole of the retinoscope. The image captured by the camera is the image seen by the optometrist during retinoscopy. The camera is the equivalent of the optometrist’s eyes.

Further, the segmenting the pupil area image from each frame of the image in the optometry video includes: preprocessing each frame of the image in the optometry video; performing benchmark detection on the preprocessed image to obtain the eye rectangular area ; In the rectangular area, detect the pupil and obtain the minimum circumscribed rectangular area of the pupil; segment the minimum circumscribed rectangular area of the pupil to obtain the pupil area image.

Further, the preprocessing of each frame of the image in the optometry video specifically includes: performing grayscale processing on each frame of the image, and performing filtering, brightness adjustment, and gamma correction on the grayscaled image. , get the preprocessed image.

Further, the preprocessed image is subjected to benchmark detection to obtain the eye rectangular area. The benchmark detection includes: taking the center point of the eye rectangular area as the reference point F ₁ .

It should be understood that the camera moves left and right relative to the patient, so in the captured images, the position of the same object also changes. Benchmark detection is to find a reference point in the image that is fixed in the world coordinate system. This reference point is fixed in the world coordinate system, but moves in the image coordinate system. The center point of the rectangular area of the eye will be used as the reference point, which is the base point.

Further, in the rectangular area, the pupil is detected, and the minimum circumscribed rectangular area of the pupil is obtained. The circular detection method or the trained convolutional neural network is used to identify the pupil, and the minimum circumscribed rectangular area of the pupil is obtained, and the pupil is recorded. The minimum circumscribed rectangular area is the closest endpoint coordinate F ₂ to the coordinate origin.

Further, the step of performing super-resolution processing on the segmented pupil area image to obtain the super-resolution pupil area image is to input the segmented pupil area image into the super-resolution network SRCNN (Super-Resolution Convolutional Neural Network). , perform four times super-resolution processing to obtain a super-resolution pupil area image.

Further, performing threshold segmentation on the super-resolution pupil area image to segment the light-reflecting area specifically includes: setting a threshold, segmenting points with pixel values higher than the set threshold in the super-resolution pupil area image, and dividing the Points lower than the set threshold are discarded to obtain the light reflection area.

Further, performing morphological processing on the light-reflecting area and performing edge detection to obtain the light-reflecting edge contour, and then obtaining the left and right boundary abscissas of the light-reflecting edge contour, includes: performing morphology on the light-reflecting area Process, eliminate discontinuous parts and protruding parts, and perform edge detection, and then obtain the light-reflecting edge contour. Take the point with the smallest abscissa in the outline as the left boundary abscissa of the light-reflecting edge contour, and take the point with the largest abscissa in the outline. The point serves as the right boundary of the light-reflecting edge contour; records the abscissa of the left boundary of the light-reflecting edge contour. , the right boundary abscissa of the light-reflecting edge contour .

The reflected light is the reflected light from the retina of the human eye under the illumination of the strip beam of the retinoscope.

Further, the transformation of the left and right boundary abscissas to obtain the transformed left and right boundary abscissas includes: transforming the light-reflecting boundary abscissas into ,/> Perform transformation processing to obtain the transformed abscissa of the light-reflecting boundary/> ,/> , the transformation formula is as follows:/> ;/> .

Further, for all frame images in the optometry video, the transformed left and right boundary abscissas are obtained, including: recording the transformed abscissas L ₂ and R ₂ of the reflected light boundary frame by frame, and obtaining the transformed left and right boundary coordinates The boundary coordinate sequence S ₁ and the transformed light-reflecting right boundary coordinate sequence S ₂ .

Further, linear fitting is performed on the transformed left and right boundary abscissa sequences respectively to obtain the moving speed of the left and right boundaries; the average of the moving speeds of the left and right boundaries is used as the light reflection speed, which specifically includes: converting the transformed light reflection Perform piecewise linear fitting on the left boundary coordinate sequence S1 to obtain two straight line slopes. Remove the slope that is close to zero and use the unremoved straight line slope as the moving speed V _rl of the left boundary; similarly, obtain the moving speed of the right boundary. V _rr ; then average the calculated left boundary moving speed V _rl and right boundary moving speed V _rr as the reflected light speed V _r .

Further, the diopter is calculated based on the fitting formula of reflected light speed and diopter, including: converting the reflected light speed to Enter the diopter fitting formula to calculate the diopter/> , the fitting formula form is as follows, where/> ,/> ,/> is the fitting parameter:/> .

A group of refraction videos of patients with different refraction are collected, and then the ophthalmologist annotates the refraction data, and combines it with the light reflection speed calculated in Example 1 to obtain the fitting parameters.

Embodiment 2

As shown in Figure 2, the image processing-based shadow movement direction calculation system includes: a second acquisition module, which is configured to: collect the optometry video of the patient's eye area; and a light movement direction calculation module, which is configured to: Each frame of the image in the optometry video is segmented into a pupil area image; the segmented pupil area image is subjected to super-resolution processing to obtain a super-resolution pupil area image; a threshold segmentation is performed on the super-resolution pupil area image to segment the reflected light area; perform morphological processing on the light-reflecting area and perform edge detection to obtain the light-reflecting edge contour, and then obtain the left and right boundary abscissas of the light-reflecting edge contour; transform the left and right boundary abscissas to obtain the transformed The left and right boundary abscissas; for all frame images in the optometry video, the transformed left and right boundary abscissas are obtained; linear fitting is performed on the transformed left and right boundary abscissa sequences to obtain the moving speed of the left and right boundaries; the left and right boundaries are The average moving speed is used as the reflected light speed; the reflected light moving direction is calculated according to the reflected light speed; the light belt moving direction calculation module is configured to: calculate the light belt moving direction; the shadow moving direction calculation module is configured as: Based on the moving direction of the reflected light and the moving direction of the light strip, the moving direction of the shadow is calculated.

Further, calculating the moving direction of the reflected light based on the reflected light speed includes: obtaining the moving direction of the reflected light based on the positive or negative value of the reflected light speed V _r . If the value of the reflected light speed V _r is positive, then the reflected light speed V r is positive. The moving direction is to the right; if the value of the reflected light speed V _r is negative, the moving direction of the reflected light is to the left.

The left-right direction is consistent with the left-right direction in the camera's perspective when recording the optometry video. During optometry, the optometrist needs to sit directly opposite the patient. The direction to the optometrist's left hand side is left, and the direction to the optometrist's right hand side is right. The perspective of the camera in this system is consistent with that of the optometrist.

Further, the calculation of the moving direction of the light belt includes: obtaining the optometry video, preprocessing each frame of the optometry video; performing benchmark detection on the preprocessed image to obtain a rectangular area, and setting the center point of the rectangular area As the reference point F ₁ ; set the area above the rectangular area to perform threshold segmentation and edge detection, then perform straight line detection, screen the detected straight lines, find the left and right boundaries of the light strip, and record the horizontal direction of the left boundary of the light strip. coordinate L ₃ and the abscissa R ₃ of the right boundary of the light strip; perform coordinate transformation on the abscissa L ₃ of the left boundary of the light strip and the abscissa R ₃ of the right boundary of the light strip respectively to obtain the transformed abscissa L of the left boundary of the light strip ₄ and the transformed abscissa R ₄ of the right boundary of the light band; for each frame of the optometry video, the transformed abscissa L ₄ of the left boundary of the light band and the transformed abscissa R ₄ of the right boundary of the light band are obtained, and we get The left boundary coordinate sequence S ₃ of the light belt and the right boundary coordinate sequence S ₄ of the light belt; perform piecewise linear fitting of the left boundary coordinate sequence S ₃ to obtain the slopes of two straight lines, remove the slope that is close to zero, and remove the slope that has not been removed The slope is used as the moving speed V _bl of the left boundary; similarly, the moving speed V _br of the right boundary is obtained; then the calculated moving speed V _bl of the left boundary and the moving speed V _br of the right boundary are averaged as the moving speed V of the light strip _b ; According to the sign of the moving speed of the light belt, the moving direction of the light belt is obtained.

If the value of the light belt speed V _b is positive, the moving direction of the light belt is to the right; if the value of the light belt speed V _b is negative, the moving direction of the light belt is to the left.

The angle of view for judging the left and right direction is the same as the angle of view for judging the left and right direction of the reflected light.

Further, the detected straight lines are screened to find the left and right boundaries of the light band, and the abscissa L ₃ of the left boundary of the light band and the abscissa R ₃ of the right boundary of the light band are recorded, specifically including: setting a threshold, according to Line slope selects vertical straight lines in the image and removes straight lines whose length is less than the set threshold; then among the filtered straight lines, find the leftmost and rightmost straight lines based on the abscissa of the line segment center point as light strips The left and right boundaries of boundary, and take the abscissa coordinate of the center point of the straight line as the abscissa R3 of the right boundary of the light strip.

It should be understood that since the patient wears the reference eye frame and the facial area around the eyes is blocked by the white background plate of the reference eye frame, part of the strip beam emitted by the retinoscope is projected onto the white background plate around the eyes. The other part is projected to the human eye area, and the projection of the retinoscope strip beam shown on the white background board is the light strip.

Further, the coordinate transformation is performed on the abscissa L ₃ of the left boundary of the light strip and the abscissa R ₃ of the right boundary of the light strip. The transformation formula is as follows: ;/> .

Further, calculating the shadow moving direction based on the moving direction of the reflected light and the moving direction of the light strip includes: comparing the calculated moving direction of the reflected light and the moving direction of the light strip. If the directions are consistent, the shadow moving direction is smooth. If the direction is inconsistent, the moving direction is reverse.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims (10)

1. Diopter estimation system, characterized by: The first acquisition module is configured to: acquire the optometry video of the patient's eye area; The reflected light speed calculation module is configured to: segment each frame of the image in the optometry video to obtain a pupil area image; perform super-resolution processing on the segmented pupil area image to obtain a super-resolution pupil area image; perform super-resolution Perform threshold segmentation on the pupil area image to segment the light-reflecting area; perform morphological processing and edge detection on the light-reflecting area to obtain the light-reflecting edge contour, and then obtain the left and right boundary abscissas of the light-reflecting edge contour; Transform the left and right boundary abscissas to obtain the transformed left and right boundary abscissas; for all frame images in the optometry video, obtain the transformed left and right boundary abscissas; linearly fit the transformed left and right boundary abscissa sequences respectively , get the moving speed of the left and right boundaries; take the average of the moving speeds of the left and right boundaries as the reflected light speed; The diopter calculation module is configured to calculate the diopter based on the reflected light speed and the diopter fitting formula. 2. The diopter estimation system according to claim 1, wherein the segmenting the pupil area image from each frame of the image in the optometry video includes: preprocessing each frame of the image in the optometry video; The preprocessed image is subjected to baseline detection to obtain the rectangular area of the eye; within the rectangular area, the pupil is detected and the minimum circumscribed rectangular area of the pupil is obtained; the minimum circumscribed rectangular area of the pupil is segmented to obtain the pupil area image. 3. The diopter estimation system according to claim 2, wherein the pupil is detected in the rectangular area, and the smallest circumscribed rectangular area of the pupil is obtained, using a circular detection method or a trained convolutional neural network to identify the pupil. Exit the pupil, obtain the minimum circumscribed rectangular area of the pupil, and record the coordinate F ₂ of the endpoint closest to the coordinate origin of the minimum circumscribed rectangular area of the pupil. 4. The diopter estimation system according to claim 1, wherein the step of performing super-resolution processing on the segmented pupil area image to obtain the super-resolution pupil area image is to input the segmented pupil area image. Go to the super-resolution network and perform four times super-resolution processing to obtain a super-resolution pupil area image. 5. The diopter estimation system according to claim 1, characterized in that the light-reflecting area is subjected to morphological processing and edge detection to obtain the light-reflecting edge contour, and then the left boundary abscissa and the light-reflecting edge contour are obtained. The abscissa of the right boundary, including: Perform morphological processing on the light-reflecting area, eliminate discontinuous parts and protruding parts, and perform edge detection, and then obtain the light-reflecting edge contour. Take the point with the smallest abscissa in the outline as the left boundary abscissa of the light-reflecting edge contour. Take the point with the largest abscissa in the outline as the right boundary of the light-reflecting edge contour; record the left-boundary abscissa L ₁ of the light-reflecting edge contour and the right-boundary abscissa L ₂ of the light-reflecting edge contour. 6. The diopter estimation system according to claim 1, characterized in that said transforming the left and right boundary abscissas to obtain transformed left and right boundary abscissas includes: Transform the light-reflecting boundary abscissas L ₁ and R ₁ to obtain the transformed light-reflecting boundary abscissas L ₂ and R ₂ . The transformation formula is as follows: ; . 7. The diopter estimation system according to claim 1, wherein the diopter is calculated based on the reflected light speed and the diopter fitting formula, including: Enter the reflected light speed V _r into the diopter fitting formula to calculate the diopter. , the fitting formula form is as follows: , where,/> ,/> ,/> are the fitting parameters. 8. The moving direction calculation system based on image processing is characterized by: The second acquisition module is configured to: acquire the optometry video of the patient's eye area; The reflected light movement direction calculation module is configured to: segment each frame of the image in the optometry video to obtain a pupil area image; perform super-resolution processing on the segmented pupil area image to obtain a super-resolution pupil area image; The resolution pupil area image is thresholded to segment the light-reflecting area; the light-reflecting area is subjected to morphological processing and edge detection to obtain the light-reflecting edge contour, and then the left and right boundary abscissas of the light-reflecting edge contour are obtained ; Transform the left and right boundary abscissas to obtain the transformed left and right boundary abscissas; for all frame images in the optometry video, obtain the transformed left and right boundary abscissas; linearly approximate the transformed left and right boundary abscissa sequences respectively. Combined, the moving speed of the left and right boundaries is obtained; the average of the moving speeds of the left and right boundaries is used as the reflected light speed; the reflected light moving direction is calculated based on the reflected light speed; The light strip moving direction calculation module is configured to: calculate the light strip moving direction; The shadow moving direction calculation module is configured to calculate the shadow moving direction based on the moving direction of the reflected light and the moving direction of the light strip. 9. The system for calculating the moving direction of light based on image processing as claimed in claim 8, characterized in that the calculation of the moving direction of the light strip includes: Obtain the optometry video and preprocess each frame of the optometry video; Perform benchmark detection on the preprocessed image to obtain a rectangular area, and use the center point of the rectangular area as the benchmark point F ₁ ; Perform threshold segmentation and edge detection in the upper area of the rectangular area, then perform straight line detection, filter the detected straight lines, find the left and right boundaries of the light strip, record the abscissa L ₃ of the left boundary of the light strip and the right boundary of the light strip The abscissa R ₃ ; Perform coordinate transformation on the abscissa L ₃ of the left boundary of the light band and the abscissa R ₃ of the right boundary of the light band respectively to obtain the transformed abscissa L ₄ of the left boundary of the light band and the transformed abscissa R ₄ of the right boundary of the light band; For each frame of the optometry video, the transformed light band left boundary abscissa L ₄ and the transformed light band right boundary abscissa R ₄ are obtained, and the light band left boundary coordinate sequence S ₃ and the light band right boundary coordinate are obtained Sequence S ₄ ; Perform piecewise linear fitting of the left boundary coordinate sequence S ₃ to obtain the slope of the two straight lines. Remove the slope close to zero to obtain the moving speed V _bl of the left boundary. In the same way, the moving speed V _br of the right boundary is obtained; and then calculate The obtained left boundary moving speed V _bl and right boundary moving speed V _br are averaged as the moving speed V _b of the light strip; According to the sign of the moving speed of the light belt, the moving direction of the light belt is obtained. 10. The system for calculating the direction of shadow movement based on image processing as claimed in claim 8, wherein calculating the direction of shadow movement based on the direction of movement of the reflected light and the direction of movement of the light strip includes: converting the calculated direction of the reflected light. Compare the moving direction with the moving direction of the light strip. If the directions are consistent, the moving direction is forward. If the directions are inconsistent, the moving direction is reverse.