CN113139962B

CN113139962B - System and method for scoliosis probability assessment

Info

Publication number: CN113139962B
Application number: CN202110606040.0A
Authority: CN
Inventors: 马真胜; 许可; 张建华; 杜晓刚; 李景阳
Original assignee: Beijing Ouying Information Technology Co Ltd
Current assignee: Beijing Allin Technology Co ltd
Priority date: 2021-05-26
Filing date: 2021-05-26
Publication date: 2021-11-30
Anticipated expiration: 2041-05-26
Also published as: CN113139962A

Abstract

The application discloses a system for scoliosis probability assessment, comprising: a capture device configured to obtain a back picture; and a processing device configured to communicatively connect with the capture device and receive the back picture, the processing device comprising: a detection module configured to detect a back region in a back picture; a symmetry calculation module configured to calculate back symmetry of the back region; a balance calculation module configured to detect joint points in the back region and calculate a back balance based on the detected joint points; and an evaluation module configured to evaluate a probability of scoliosis based on the back symmetry and the back balance. The present application also discloses a method for scoliosis probability assessment and a machine-readable storage medium.

Description

System and method for scoliosis probability assessment

Technical Field

The present application relates to a system and method for scoliosis probability assessment, and a corresponding machine-readable storage medium.

Background

Adolescent Idiopathic Scoliosis (AIS) is the most common three-dimensional spinal deformity in scoliosis, accounting for about 80% of the total number of idiopathic scoliosis, and scoliosis is often clinically defined as a scoliosis with a Cobb angle greater than 10 ° on a standing positive X-ray slice. AIS has a prevalence of 1% to 3% in adolescents between 10 and 16 years of age. AIS not only seriously affects the physical appearance of teenagers, but also can impair their respiratory function, motor function, mental state, and overall quality of life.

The early symptoms of scoliosis are not obvious and can be easily ignored. Untimely treatment can seriously affect the body type of a patient and cause the psychological problems of lack of confidence, depression tendency, suicide concept and the like. In severe cases, the psychological impact on the patients can be severe, the life quality of the patients can be greatly reduced, and the economic burden of families and society can be increased.

In recent years, the incidence rate of scoliosis is rising, scoliosis can be found early by screening, scientific control and treatment means can be provided in time, and the health management of the spinal deformity is enhanced. The screening has important significance for keeping physical and psychological health of teenagers and relieving social and family burdens caused by scoliosis. The early screening can discover diseases as early as possible to a great extent, and give medical professional treatment as early as possible, so that psychological, physical and economic burdens of patients after adult are reduced.

At present, the sites for screening the scoliosis at home and abroad are mainly schools. Development of school screening is facilitated in community hospitals and regional central hospitals. In the "clinical practice guidelines and path guidance for scoliosis screening of adolescents in China", it is proposed to jointly use three or more screening means as scoliosis screening methods, such as visual method, Adams' forward bending test, trunk rotation angle measurement (including scoliosis measuring ruler or portable electronic scoliosis screening tool).

Visual inspection is the most readily available form of screening without the aid of any tools. Visual observation means that a professional doctor judges and conjectures through observation of the appearance of shoulders, the appearance of waist lines and the levelness of pelvis. However, the method is too dependent on the experience and subjective judgment of a clinician, and has certain inspection significance, but the method is difficult to popularize widely due to low accuracy and incapability of quantification.

The Adams anteflexion test is an accepted primary scoliosis screening method in the industry, and has the advantages of simple operation, no wound and low cost. However, the Adams anteflexion test also has its clear disadvantages, being overly dependent on the clinician's experience, and the high false negative rate of this approach, especially for mild patients, is prone to missed diagnosis, and therefore is not suggested as a screening modality for scoliosis alone.

The torso rotation angle measurement is evaluated with the aid of a scoliosis measuring ruler or a portable electronic scoliosis screening tool. The positioning of the measuring points needs to be finished by a professional doctor and depends on professional equipment. The screening of a single patient is time-consuming and inconvenient to use in large-scale screening. The related detection indexes are less researched, and the sensitivity and the specificity of the related detection indexes are difficult to evaluate and measure.

X-ray is taken as a standard of scoliosis diagnosis gold, and is not recommended to be used as a screening means due to the existence of radiation, particularly for adolescents, and is not recommended to be used as a screening means, so that the X-ray is only currently taken as a final diagnosis means for screening positive patients.

In this regard, the guidelines suggest the combined use of three or more screening modalities as a screening method for scoliosis. However, the more methods used in combination, the less operational and practical the screening can be performed extensively in the population, and the less efficient the screening.

Artificial intelligence is a discipline that studies computers to simulate certain mental processes and intelligent behaviors of humans. As the research of the artificial intelligence technology in the tasks of detection, identification, segmentation, etc. gradually matures, the artificial intelligence technology is also widely used for screening, diagnosis and treatment of diseases in the field of medical imaging. Theoretically, successful development and application of image segmentation, detection and image classification systems based on artificial intelligence will probably assist doctors in screening to a great extent, and provide a quick, objective and accurate solution to the above problems.

Disclosure of Invention

To solve the problems in the prior art, the present application proposes a system for scoliosis probability assessment, comprising: a capture device configured to obtain a back picture; and a processing device configured to communicatively connect with the capture device and receive the back picture, the processing device comprising: a detection module configured to detect a back region in the back picture; a symmetry calculation module configured to calculate a back symmetry of the back region; a balance calculation module configured to detect articulation points in the back region and calculate a back balance based on the detected articulation points; and an evaluation module configured to evaluate a probability of scoliosis based on the back symmetry and the back balance.

According to an alternative embodiment, the acquisition device is configured to acquire the back picture based on a predefined sharpness.

According to an alternative embodiment, the artificial intelligence detection module is configured to issue a first signal after detecting the presence of the back region in the back picture.

According to an alternative embodiment, the symmetry calculation module is configured to perform the calculation of the back symmetry of the back region upon reception of a first signal and to output the calculation result of the back symmetry.

According to an alternative embodiment, the balance calculation module is configured to perform the detection of the articulation point in the back region and to perform the calculation of the back balance if a first signal is received, and to output the calculation of the back balance.

According to an alternative embodiment, the acquisition device is configured to acquire a back picture of the bare back in a plurality of postures including at least: a standing posture; a front 90 degree stoop posture; and a rear 90 bow position.

According to an alternative embodiment, the detection module is configured to mark out the back region in the back picture that is relevant for assessing the probability of scoliosis by means of machine deep learning.

According to an alternative embodiment, the symmetry calculation module is configured to outline a body contour by means of artificial intelligence, intersect the body contour with the back region, and calculate the back symmetry by means of the proportion of the back region to the back region of the overlap between the back region and the left-right flipped back region.

According to an optional embodiment, the balance calculation module is configured to: performing two-dimensional keypoint location estimation on the back region by means of a pose estimation technique, the two-dimensional keypoint location estimation comprising estimating the location of a left shoulder, a right shoulder, a left hip, and a right hip in the back region, and calculating the back balance using an angle between a line connecting the left shoulder and the right shoulder and a relative horizontal line, and an angle between a line connecting the left hip and the right hip and a relative horizontal line, wherein the relative horizontal line is a perpendicular line to a line connecting a midpoint of a line connecting the left shoulder and the right shoulder and a midpoint of a line connecting the left hip and the right hip.

According to an alternative embodiment, the evaluation module is configured to evaluate the probability of scoliosis by means of machine deep learning of back pictures with a scoliosis degree of 0 to 10 °, a scoliosis degree of 10 ° to 20 °, a scoliosis degree of 20 ° to 45 ° and a scoliosis degree greater than 45 °.

The present application also discloses a method for scoliosis probability assessment, optionally performed by the system described above, comprising the steps of: s101: acquiring a back picture; s102: detecting a back region in the back picture; s103: calculating a back symmetry of the back region; s104: detecting articulation points in the back region and calculating back balance; and S105: assessing the probability of scoliosis based on the back symmetry and the back balance.

The application also discloses a machine-readable storage medium having executable instructions stored thereon, wherein the executable instructions, when executed, cause a machine to perform the method as described above.

By means of the equipment for scoliosis probability assessment, artificial intelligence-based image segmentation, detection and image classification can assist screening to a great extent, and a quick, objective and accurate solution is provided for the scoliosis probability assessment problem.

Although the present application has been described with reference to preferred embodiments, it is not intended to limit the present application, and modifications and improvements can be made by those skilled in the art without departing from the spirit and scope of the present application.

Drawings

FIG. 1 is a schematic diagram of a system for scoliosis probability assessment according to the present application;

FIG. 2 is a flow chart of a method for scoliosis probability assessment according to the present application;

FIG. 3 is a schematic illustration of a back region detected by means of artificial intelligence;

FIG. 4 is a schematic illustration of a back region segmented by means of artificial intelligence;

fig. 5 is a schematic diagram of joint detection.

Detailed Description

The information of the naked back when the patient stands at the whole body position and at the 90-degree lumbar position is an important area and a target which need to be concerned when the adolescent carries out scoliosis screening. Doctors and professionals can access some features of the back, such as: whether the shoulders are equal in height or not, whether the scapulae incline or not, whether the waist is wrinkled or not, whether the pelvis inclines or not, whether the back is abnormally bulged or not and other external characteristics judge whether scoliosis exists or not. The invention aims to quantify the conditions of being equal in height, inclined or symmetrical through advanced technologies such as artificial intelligence deep learning, evaluate and classify the characteristics of the back, evaluate the possible inclined angle and possibility of the spine and provide a simple and easy-to-use method for large-scale screening.

FIG. 1 is a schematic diagram of a system for scoliosis probability assessment according to the present application. The system 1 comprises: a capture device 10 configured to acquire a back picture; and a processing device 20 configured to be communicatively connected to the capture device 10 and to receive the back picture. The processing apparatus 20 includes: a detection module 200 configured to detect a back region in a back picture; a symmetry calculation module 210 configured to calculate a back symmetry of the back region; a balance calculation module 220 configured to detect articulation points in the back region and calculate a back balance based on the detected articulation points; and an evaluation module 230 configured to evaluate the probability of scoliosis based on back symmetry and back balance. The capture device 10 may include a camera, a webcam, or any other imaging device that can take a picture of the back. The processing device 20 may comprise a computer, a microprocessor or any other electronic means by which the probability of scoliosis can be evaluated from a back picture.

The capturing device 10 is configured to acquire a back picture based on a predefined sharpness. The artificial intelligence detection module 200 is configured to issue a first signal after detecting the presence of a back region in the back picture. The symmetry calculation module 210 is configured to perform calculation of back symmetry of the back region upon receiving the first signal and output a calculation result of the back symmetry. The balance calculation module 220 is configured to perform detection of a joint point in the back region and perform calculation of back balance if the first signal is received, and output a calculation result of back balance. The acquisition device 10 is configured to acquire back pictures of the bare back in a plurality of postures, the plurality of postures comprising at least: a standing position, a front 90 ° stooping position, and a rear 90 ° stooping position. The detection module 200 is configured to mark out a back region in the back picture that is relevant for assessing the probability of scoliosis by means of machine deep learning. The symmetry-calculating module 210 is configured to outline the body contour by means of artificial intelligence, intersect the body contour with the back region, and calculate the back symmetry by means of the proportion of the back region to the overlapping region between the back region and the left-right flipped back region. The balance calculation module 220 is configured to: performing two-dimensional keypoint location estimation on the back region by means of a pose estimation technique, the two-dimensional keypoint location estimation comprising estimating the location of the left shoulder, the right shoulder, the left hip, and the right hip in the back region, and calculating back balance using the angle between the line connecting the left shoulder and the right shoulder and the relative horizon, and the angle between the line connecting the left hip and the right hip and the relative horizon, wherein the relative horizon is the perpendicular line of the midpoint of the line connecting the left shoulder and the right shoulder and the midpoint of the line connecting the left hip and the right hip. The evaluation module 50 is configured to evaluate the probability of scoliosis by means of machine deep learning from back pictures with scoliosis degree of 0 to 10 °, scoliosis degree of 10 ° to 20 °, scoliosis degree of 20 ° to 45 ° and scoliosis degree greater than 45 °.

FIG. 2 is a flow chart of a method for scoliosis probability assessment according to the present application. The method according to the present application comprises the following steps.

S101: and acquiring a back picture. During acquisition, the subject keeps the back (upper body) bare. The acquisition device 10 acquires at least three pictures of the subject in two positions (standing and 90 ° stooping), respectively: standing for whole body, and bending the waist at 90 degrees at the front and 90 degrees at the back.

The above manner and idea are highly consistent with the clinical screening idea. Typically at the time of clinical screening, examination will be performed using one or more of visual methods, Adams anteflexion tests and trunk rotation angle measurements. Therefore, the method of the application does not put new requirements on the past screening, is fast, and only needs the object to be kept still in the acquisition process, but does not need the object to be kept in a certain posture for a long time.

S102: the back area is detected by means of artificial intelligence. Fig. 3 is a schematic illustration of a back region detected by means of artificial intelligence. And based on the acquired patient picture data, detecting the body back area in the picture by utilizing an artificial intelligence deep learning technology. The detected region is the smallest outer envelope rectangle of the back of the subject. The smallest outer envelope rectangle is up to the uppermost edge of the shoulder, down to the uppermost edge of the hip, left to the outermost edge of the left arm, right to the outermost edge of the right arm.

The back detection module extracts image features based on a deep learning Convolutional Neural Network (CNN). The extracted features are respectively input into a bounding box (bounding boxes) coordinate regression and a classification branch evaluation back region in a multi-scale feature fusion mode through a Feature Pyramid Network (FPN). The pyramid network achieves the evaluation effect by fusing the features of different layers by utilizing the high-resolution of the low-layer features and the high semantic information of the high-layer features. Evaluation is performed on each fused feature layer separately, unlike conventional feature fusion approaches.

The purpose of detecting the back area is: positioning the back area by combining the result of artificial intelligence segmentation, thereby calculating the symmetry of the back; and evaluating the probability and risk of scoliosis by deeply learning the dermatoglyph characteristics, the characteristics of shoulders, the symmetry characteristics of the space between the two side arms and the body and the like of the region through a machine.

S103: the back region is segmented and the back symmetry is calculated by means of artificial intelligence. Fig. 4 is a schematic illustration of a back region segmented by means of artificial intelligence. In step 3, the artificial intelligence technique is applied to semantically segment the body (delineating the body contour). Next, the intersection of the body contour delineation result and the rectangle of the back region in step 2 is obtained, and the intersection is the segmentation region of the back.

Body semantic segmentation is based on a deep learning full convolution network, achieving body segmentation with pixel-level classification. In order to perform multi-scale (multiple scales) segmentation on a body, the method utilizes serial and parallel convolution modules with holes to acquire multi-scale content information, excavates convolution features of different scales, and encodes image layer features of global content information to improve segmentation effect. The segmentation basic network (backbone) is based on the xception network, and can sufficiently extract the features of the image. In the network structure of the serial module and the Spatial Pyramid Pooling (SPP) module, the reception field of the filter can be effectively increased by the porous convolution, and multi-scale information is integrated.

When the symmetry of the back area is calculated, the calculation method is provided, namely the proportion of the overlapping area of the two pictures (back pictures) before and after the turning is calculated through left-right turning (flip), and the proportion is the back symmetry. The back symmetry ranges from 0 to 1.

The formula for calculating the back symmetry is:

where F is the back symmetry, a1 is the split back region, and a2 is the flipped back region.

S104: the joint points are detected and the balance of the back is calculated. In this step, body two-dimensional (2D) keypoint location assessment is performed based on artificial intelligence visual body pose estimation techniques to assess the location of the left shoulder, the right shoulder, the left hip, and the right hip. The evaluation results are X, Y coordinates of the respective points in the image.

And estimating the position of the two-dimensional key points of the body based on an artificial intelligence visual body posture estimation technology. All body key points in the picture are firstly detected based on the idea of point-to-face from bottom-up (bottom-up), and then the key points are corresponding to different person individuals. The method comprises evaluating output positions of a multi-state convolution network of two branches, wherein one branch is used for evaluating (scoring) a reliability map (confidence maps), and the other branch is used for evaluating component contact fields (par affinity fields) and also corresponds to a heat map (heat map) and a vector map (vector map). Each body joint outputs a part label and a coordinate (X, Y).

Fig. 5 is a schematic diagram of joint detection. Since it is not completely guaranteed that the subject is exactly at the center of the picture when the subject is photographed and stands vertically, the bottom edge of the picture cannot be used as a horizontal line. When the horizontal line of the object is estimated, the method respectively takes the middle point of the connecting line of the left shoulder and the right shoulder of the object and the middle point of the connecting line of the left hip and the right hip, and takes the vertical line of the connecting line of the two middle points as the relative horizontal line of the object.

Then, the included angle between the connecting line of the left shoulder and the right shoulder and the included angle between the connecting line of the left hip and the right hip and the horizontal line are respectively obtained as the balance degree of the shoulders and the hips. In addition, the included angle between the connecting line of the left shoulder and the right shoulder and the connecting line of the left hip and the right hip can be obtained, and whether the shoulder and the hip of the object are parallel or not can be estimated according to the included angle.

S105: the probability of scoliosis is evaluated. In the process of detecting the back region link based on artificial intelligence, after the result of the evaluated back rectangular region is obtained, the section, in which the scoliosis degree is 0-10 degrees, the scoliosis degree is 10-20 degrees, the scoliosis degree is 20-45 degrees and the scoliosis degree is more than 45 degrees, of the picture most possibly falls is judged through a machine, and corresponding probability is obtained.

The distinction between scoliosis levels of 0 to 10 °, scoliosis levels of 10 ° to 20 °, scoliosis levels of 20 ° to 45 ° and scoliosis levels greater than 45 ° is mainly based on the clinical practice of taking different therapeutic measures against scoliosis in different intervals.

In the case where the degree of scoliosis is 0 to 10 °, the degree of scoliosis can be prevented from deepening without any treatment, or intervention of posture and behavior correction training and exercise therapy as appropriate.

In the case where the degree of scoliosis is 10 ° to 20 °, posture and behavior correction training and exercise therapy are performed, and follow-up observation is performed.

In the case where the degree of scoliosis is 20 ° to 45 °, posture and behavior correction training and exercise therapy are performed, and tracking follow-up is performed in conjunction with corrective brace treatment.

In the case where the degree of scoliosis is more than 45 °, active treatment by medical means, if necessary, surgical treatment is performed.

According to the method, by utilizing a machine deep learning technology, on the basis of learning a plurality of pictures (marked with back rectangular frames and corresponding scoliosis angle intervals) with basically balanced data distribution of four classification intervals, scoliosis degree angle intervals and corresponding probabilities of unknown pictures can be evaluated according to back characteristics learned by a machine.

Classifying the back region includes extracting features based on an xception network, and then performing four classification evaluations based on softmax. The xception network improves the feature expression capability and the running speed of the network based on a depth-wise partial convolution (depth-wise partial convolution) module.

The inventors carried out the following experiments: after the machine was trained, the scoliosis screening model was tested and evaluated for effectiveness in 1211 test data. The results of the evaluation were 95% sensitivity (recall) and 76% specificity. The accuracy of the four classifications was 81.3%. The above evaluations verify the feasibility of this method for scoliosis screening from body pictures.

The data required for the apparatus and method according to the present application when processing images using deep learning include a standing whole body photograph, a front 90 ° lumbar position, and a rear 90 ° lumbar position. Under the condition that the input image is a bare back photo, the efficiency of judgment through artificial intelligence is higher. The whole treatment time can be controlled within 1 second.

The apparatus and method according to the present application reduces reliance on the experience and subjective judgment of the clinician when performing the evaluation and calculation. The patient does not need to hold a particular posture for a long time to be evaluated.

According to the equipment and the screening process of the method, the advantages of the computer are furthest exerted, accurate calculation is carried out, related indexes are quantized, and the method has stronger persuasion compared with the subjective judgment of doctors.

The device and the method according to the application are based on the back picture, and compared with the current traditional early screening technology in clinic, the device and the method have the characteristics of high feasibility, simplicity and easiness in operation. In addition, the device and the method avoid X-ray detection as much as possible and reduce the risk of the object to be irradiated.

The device and the method do not need professional instruments and equipment or too long time, and are suitable for large-scale screening in schools and other occasions where teenagers are dense.

The equipment and the method can be used in combination with the existing screening method, and can make up the defect that the existing method has few quantitative indexes; in addition, the method can be used for evaluating a scoliosis interval, and has great guiding significance for subsequent treatment and rehabilitation.

Claims

1. A system (1) for scoliosis probability assessment, comprising:

an acquisition device (10) configured to acquire a back picture; and

a processing device (20) configured to be communicatively connected with the capturing device (10) and to receive the back picture, the processing device (20) comprising:

a detection module (200) configured to detect a back region in the back picture;

a symmetry calculation module (210) configured to calculate a back symmetry of the back region;

a balance calculation module (220) configured to detect articulation points in the back region and calculate a back balance based on the detected articulation points; and

an evaluation module (230) configured to evaluate a probability of scoliosis based on the back symmetry and the back balance,

wherein the balance calculation module (220) is configured to:

performing a two-dimensional keypoint location estimation of the back region by means of an attitude estimation technique, the two-dimensional keypoint location estimation comprising estimating the location of the left shoulder, the right shoulder, the left hip, the right hip in the back region, and

and calculating the back balance by using the included angle between the connecting line of the left shoulder and the right shoulder and a relative horizontal line and the included angle between the connecting line of the left hip and the right hip and the relative horizontal line, wherein the relative horizontal line is the perpendicular line of the connecting line of the midpoint of the connecting line of the left shoulder and the right shoulder and the midpoint of the connecting line of the left hip and the right hip.

2. The system according to claim 1, wherein the acquisition device (10) is configured to acquire the back picture based on a predefined sharpness.

3. The system according to claim 1 or 2, wherein the artificial intelligence detection module (200) is configured to issue a first signal after detecting the presence of the back region in the back picture.

4. The system according to claim 3, wherein the symmetry calculation module (210) is configured to perform the calculation of the back symmetry of the back region upon reception of a first signal and to output a calculation result of the back symmetry.

5. The system according to claim 3, wherein the balance calculation module (220) is configured to perform the detection of the articulation point in the back region and to perform the calculation of the back balance if a first signal is received, and to output a calculation result of the back balance.

6. The system according to claim 1 or 2, wherein the acquisition device (10) is configured to:

obtaining a picture of the back of the bare back in a standing position, an

The back pictures of the bare back in the 90 ° stooped posture were taken from the front and back.

7. The system according to claim 1 or 2, wherein the detection module (200) is configured to mark out the back region in the back picture that is relevant for assessing the probability of scoliosis by means of machine deep learning.

8. The system according to claim 1 or 2, wherein the symmetry calculation module (210) is configured to outline a body contour by means of artificial intelligence, intersect the body contour with the back region, and calculate the back symmetry by means of a proportion of the back region to the left-right flipped back region of an overlapping region between the back region and the back region.

9. A method for scoliosis probability assessment, the method being performed by the system of any one of claims 1-8, the method comprising the steps of:

s101: acquiring a back picture;

s102: detecting a back region in the back picture;

s103: calculating a back symmetry of the back region;

s104: detecting articulation points in the back region and calculating back balance; and

s105: assessing the probability of scoliosis based on the back symmetry and the back balance.

10. A machine-readable storage medium having stored thereon executable instructions, wherein the executable instructions, when executed, cause a machine to perform the method of claim 9.