WO2023179218A1 - Quantitative evaluation method and system for static tremor of dyskinesia diseases - Google Patents

Quantitative evaluation method and system for static tremor of dyskinesia diseases Download PDF

Info

Publication number
WO2023179218A1
WO2023179218A1 PCT/CN2023/074823 CN2023074823W WO2023179218A1 WO 2023179218 A1 WO2023179218 A1 WO 2023179218A1 CN 2023074823 W CN2023074823 W CN 2023074823W WO 2023179218 A1 WO2023179218 A1 WO 2023179218A1
Authority
WO
WIPO (PCT)
Prior art keywords
tremor
frequency
patient
amplitude
curve
Prior art date
Application number
PCT/CN2023/074823
Other languages
French (fr)
Chinese (zh)
Inventor
付思超
董博雅
何麒
肖郧峰
易典
程耿
Original Assignee
凝动万生医疗科技(武汉)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 凝动万生医疗科技(武汉)有限公司 filed Critical 凝动万生医疗科技(武汉)有限公司
Publication of WO2023179218A1 publication Critical patent/WO2023179218A1/en

Links

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/40Detecting, measuring or recording for evaluating the nervous system
    • A61B5/4076Diagnosing or monitoring particular conditions of the nervous system
    • A61B5/4082Diagnosing or monitoring movement diseases, e.g. Parkinson, Huntington or Tourette
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1101Detecting tremor
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235Details of waveform analysis
    • A61B5/725Details of waveform analysis using specific filters therefor, e.g. Kalman or adaptive filters
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235Details of waveform analysis
    • A61B5/7253Details of waveform analysis characterised by using transforms
    • A61B5/7257Details of waveform analysis characterised by using transforms using Fourier transforms
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics

Definitions

  • This application relates to the field of quantitative assessment of movement disorders, and in particular to methods and systems for quantitative assessment of resting tremor in movement disorders.
  • Resting tremor is one of the main clinical symptoms of movement disorders such as Parkinson's disease and essential tremor. Quantitative indicators such as whether tremor occurs, the amplitude and frequency of tremor are important references for the clinical diagnosis, evaluation and grading of such diseases. significance.
  • the diagnostic evaluation of resting tremor is mainly carried out through doctors' visual scale scoring, electromyography examination, wearable sensors and other methods.
  • the visual scale scoring mechanism relies heavily on doctors' clinical experience and is highly subjective and differentiated. Therefore, in clinical work, it is often seen that different doctors have large differences in their diagnosis, treatment, and prognosis judgments for the same patient.
  • Electromyography examinations often use needle electrodes and apply electrical stimulation technology. The subjects will suffer certain pain and injury during the examination.
  • Wearable sensors require a series of operating procedures such as putting on, taking off, charging, and disinfecting during use, and the operation is not convenient enough.
  • Chinese patent CN105701806B provides a method for detecting Parkinson's tremor motion characteristics based on depth images, but the use requires the patient to wear solid-colored gloves and use a specific device (depth camera) to take pictures. The operation is not convenient enough, and it can only capture the stillness of the hand. It cannot be used to evaluate sexual tremors and cannot be applied to other parts of the body such as the face, legs, etc.
  • the present invention has been proposed in view of the above and other concepts.
  • a method for quantitative assessment of resting tremor in movement disorders which can objectively and quantitatively assess resting tremor in a convenient and non-wearable manner.
  • the method includes the following steps:
  • S1 Collect video images of resting tremor in the patient's tremor-prone parts through imaging equipment;
  • the imaging device needs to be kept stationary during the acquisition process in step S1, and the distance between the imaging device and the patient remains fixed.
  • the coordinate position curve in step S2 is a curve formed by the coordinates of the same key point of the human body in all the frames of the video image, wherein the coordinate position curve includes the following directional dimension curves: At least one of: x-direction curve, y-direction curve, and z-direction curve.
  • step S3 includes the following steps:
  • S3-1 Perform discrete Fourier transform or fast Fourier transform on each direction dimension of the coordinate position curve to obtain a frequency domain sequence of each direction dimension;
  • S3-3 Determine the frequency point with the largest absolute value of the frequency domain amplitude curve in each direction dimension in a frequency band of not less than 4 Hz, as the directional tremor frequency of that direction dimension;
  • the step S4 specifically includes the following steps:
  • S4-1 Select a key frequency segment, select a filtering method and a band-pass filter, filter the frequency domain sequences in each direction dimension of the coordinate position curve, and obtain filtered curves in each direction dimension;
  • S4-2 Calculate comprehensive tremor amplitude curves in multiple direction dimensions based on the filtered curves in each direction dimension;
  • S4-3 Determine the maximum value in the comprehensive tremor amplitude curve, where the maximum value corresponds to the maximum pixel tremor amplitude of the patient.
  • the selection of key frequency segments in step S4-1 includes: a) selecting a frequency segment with the patient's tremor frequency as the center and a reasonable threshold as the radius; b) selecting a frequency segment related to the patient's diagnosed disease frequency range.
  • step S5 includes the following steps:
  • S5-1 According to the tremor-prone part captured in the video image, measure the actual length between two key points on the human body whose distances basically do not change with the tremor movement on this part;
  • S5-3 Convert the maximum actual tremor amplitude of the patient in proportion according to the maximum pixel tremor amplitude, the actual human body characteristic length and the pixel distance.
  • a resting tremor quantitative assessment system for movement disorders includes:
  • a video image collection module configured to collect video images of resting tremor in tremor-prone parts of the patient through an imaging device
  • a coordinate position curve forming module configured to extract the coordinates of key points of the human body in each frame of the video image and form a coordinate position curve of the key points of the human body
  • a tremor frequency extraction module configured to perform frequency domain transformation on the coordinate position curve and extract the tremor frequency of the patient's resting tremor
  • a pixel tremor amplitude extraction module configured to perform band-pass filtering on the coordinate position curve after frequency domain transformation, and extract the maximum pixel tremor amplitude of the patient's resting tremor;
  • the actual tremor amplitude conversion module is configured to obtain the patient's actual human body feature length and the pixel length in the video image corresponding to the actual human body feature length, and convert the patient's maximum real body feature length in proportion. tremor amplitude.
  • the imaging device needs to be kept stationary, and the distance between the imaging device and the patient remains fixed.
  • the coordinate position curve is a curve formed by the coordinates of the same key point of the human body in all frames of the video image, wherein the coordinate position curve includes at least one of the following directional and dimension curves: x direction curve, y-direction curve, z-direction curve.
  • the tremor frequency extraction module is configured to also be used for:
  • the maximum value of the tremor frequency in the stated direction As the frequency domain tremor amplitude in this direction dimension, the directional tremor frequency corresponding to the direction dimension with the largest frequency domain tremor amplitude is used as the patient's tremor frequency.
  • the pixel tremor amplitude extraction module is configured to also be used for:
  • the maximum value in the comprehensive tremor amplitude curve is determined, and the maximum value corresponds to the maximum pixel tremor amplitude of the patient.
  • the selection of the key frequency segment includes: a) selecting a frequency segment with the patient's tremor frequency as the center and a reasonable threshold as the radius; b) selecting a frequency segment related to the patient's diagnosed disease.
  • the actual tremor amplitude conversion module is configured to also be used for:
  • the tremor-prone part captured in the video image measure the actual length between two key points on the human body whose distance basically does not change with the tremor movement on this part;
  • the maximum pixel tremor amplitude the actual human body feature length and the pixel distance, Scale the patient's stated maximum actual tremor amplitude.
  • the embodiments of the present application have the advantages of convenience, accuracy, and non-contact, provide a new and convenient solution for the quantitative assessment of resting tremor symptoms in current movement disorders, and are easy to use in remote consultation.
  • More embodiments of the present invention can also achieve other advantageous technical effects not listed one by one. These other technical effects may be partially described below, and for those skilled in the art after reading the present invention, they can expected and understood.
  • Figure 1 is a schematic diagram of the flow of a quantitative assessment method for resting tremor in movement disorders according to an embodiment of the present application
  • Figure 2 is a schematic diagram of the coordinate position curve of the mandibular tip on the face of a typical Parkinson's disease patient according to an embodiment of the present application;
  • Figure 3 is a schematic diagram of a frequency domain amplitude curve in Hz according to an embodiment of the present application.
  • Figure 4 is a schematic diagram of a coordinate position curve after band-pass filtering according to an embodiment of the present application.
  • Figure 5 is a schematic diagram of a comprehensive tremor amplitude curve according to an embodiment of the present application.
  • Figure 6 is a schematic structural diagram of a resting tremor quantitative assessment system for movement disorders according to an embodiment of the present application.
  • FIG. 1 shows a schematic flowchart of a quantitative assessment method for resting tremor in movement disorders according to an embodiment of the present application.
  • the quantitative assessment method for resting tremor according to this embodiment includes steps S1 to S5, specifically as follows.
  • Step S1 Use imaging equipment to collect video images of resting tremor in the patient's tremor-prone parts.
  • Imaging equipment that can be used includes, but is not limited to, ordinary monocular cameras, binocular and multi-eye cameras, depth cameras, infrared cameras and other imaging equipment commonly used in this field.
  • Specific parts that can be quantitatively assessed for resting tremor that is, parts that are the subject of photography or imaging, are generally tremor-prone parts, including but not limited to the patient's lips, mandible, entire face, palms, soles, entire upper limbs, and entire lower limbs and other parts of the body.
  • the types of video images collected include but are not limited to ordinary RGB color videos, grayscale videos, color or grayscale videos with depth information, binocular and multi-eye color or grayscale videos, etc.
  • imaging equipment such as binocular cameras and depth cameras can be used to collect video images of multiple tremor-prone parts of patients while they are sitting still, and obtain videos with depth information. images to facilitate more precise processing and analysis.
  • an ordinary mobile phone can be simply fixed and then the mobile phone's own camera can be used to collect whole-body video images of the patient while he is sitting quietly. This is more convenient for the accompanying staff to use ordinary imaging equipment to collect. video images.
  • Step S2 Extract the coordinates of key points of the human body in each frame of the collected video image to form a coordinate position curve.
  • specific methods for extracting coordinates include but are not limited to template matching, target tracking, target recognition and other methods, as well as the comprehensive application of these methods.
  • Key points on the human body are generally specific points selected on parts prone to tremor, including but not limited to the midpoint of the upper lip, midpoint of the lower lip, tip of the mandible, wrist joints, fingertips, ankle joints, toe tips, etc. where tremor is prone to occur .
  • the coordinate position curve is a curve formed by tracking the coordinates of the same human body key points in all video frames.
  • the curve usually has multiple direction dimensions, including at least one of an x-direction curve, a y-direction curve, and a z-direction curve.
  • a template matching method can be used to use preset models of key parts of the human body, such as facial contour models, mouth contour models, hand contour models, etc., to match the video images of the corresponding parts, and extract the corresponding parts of the video images.
  • the coordinates of the point with the highest degree of matching of the corresponding human body key points in the human body key part model are used as the coordinates of the human body key points.
  • the target tracking method can also be used to manually mark the coordinate positions of key points of the human body in the first frame of the video image, extract the image features of the key points of the human body and their neighborhoods in the first frame of the image, and subsequently The corresponding image features are extracted from each frame of image, and the target tracking algorithm is applied to calculate the coordinates of the human body key points marked in each subsequent frame of image.
  • the image features include but are not limited to image templates, frequency domain information, Hessian matrices, edge contour information, grayscale histograms, integral histograms, Harris corner points, SIFT feature points, SURF feature points, etc.
  • the target tracking algorithm includes but is not limited to sliding window template matching, mean shift, particle filter, SIFT algorithm, SURF algorithm, etc.
  • the target recognition method can also be used to train the artificial neural network through sample images pre-labeled with the coordinates of certain human key points, so as to obtain the trained artificial neural network model to process the video images frame by frame and identify each The coordinates of the human body key points in the frame.
  • the coordinate positions of human body key points extracted based on template matching methods or target recognition methods may fluctuate slightly between different image frames, while the coordinate positions of human body key points extracted based on target tracking methods are more accurate between consecutive frames. and stable, but tracking drift or loss may occur due to excessive local deformation. Therefore, in practice, the above extraction methods often need to be applied comprehensively.
  • the key point of the patient's mandibular tip is identified through the target recognition method in the first frame of the video image, and then the key point of the patient's mandibular tip is identified through the target tracking method in subsequent frames. Key points are tracked.
  • the target recognition method is applied to re-identify the key point, and then the tracking process is re-entered, and so on. In this way, a more accurate and stable coordinate position curve can be obtained without manual annotation.
  • Figure 2 shows a schematic diagram of the coordinate position curve of the mandibular tip on the face of a typical Parkinson's disease patient according to an embodiment of the present application.
  • this embodiment extracts the coordinate position curve of the mandibular tip of a typical Parkinson's disease patient's face from ordinary RGB video images. Since this embodiment does not introduce depth information, there are only two dimensions of x and y. If an imaging device with depth information is used for video image collection in another embodiment of the present application, an additional z direction can be provided. Dimension information.
  • Step S3 Perform frequency domain transformation on the coordinate position curve obtained in step S2 to extract the tremor frequency of the patient's resting tremor.
  • Step S3 specifically includes the following steps:
  • Step S3-1 Perform Discrete Fourier Transform (DFT) or Fast Fourier Transform (FFT) on each direction dimension of the coordinate position curve to obtain frequency domain sequences of each direction dimension.
  • DFT Discrete Fourier Transform
  • FFT Fast Fourier Transform
  • x(t), y(t) and z(t) respectively represent the three directional dimensions of the coordinate position curve
  • N represents the total number of frames of the video image
  • DFT( ⁇ ) represents discrete Fourier transform or fast Fourier transform
  • F x (n), F y (n) and F z (n) represent three directions respectively.
  • Step S3-2 Convert the frequency domain sequence in each direction dimension into a frequency domain amplitude curve in Hz.
  • the conversion formula is as follows (2):
  • a one-dimensional frequency domain sequence Represents the frequency domain amplitude curve.
  • Step S3-3 Determine the frequency domain amplitude curve of each direction dimension in a frequency band of no less than 4Hz.
  • the frequency point with the largest absolute value within is used as the directional tremor frequency in that direction dimension.
  • the calculation method is as follows: formula (3):
  • f max represents the directional tremor frequency
  • Step S3-4 use the maximum value of the tremor frequency in the direction As the frequency domain tremor amplitude in this direction dimension, the directional tremor frequency corresponding to the direction dimension with the largest frequency domain tremor amplitude is used as the tremor frequency of the patient's resting tremor.
  • Figure 3 shows a schematic diagram of a frequency domain amplitude curve in Hz according to an embodiment of the present application.
  • the frequency domain amplitude curve shown in Figure 3 is obtained from the coordinate position curve in Figure 2 through steps S3-1 and S3-2.
  • there is an obvious peak at 5Hz after removing the low-frequency component below 4Hz (the low-frequency component below 4Hz is removed because the typical tremor frequencies of movement disorders such as Parkinson's disease and essential tremor are uniform. above 4Hz), so the directional tremor frequencies in the x and y directions extracted through step S3-3 are both 5Hz.
  • the frequency domain tremor amplitude in the y direction is greater than the x direction, so after steps S3-3 and S3- 4The patient's tremor frequency obtained is 5Hz.
  • Step S4 Band-pass filter the coordinate position curve after frequency domain transformation to extract the maximum pixel tremor amplitude of the patient's resting tremor.
  • Step S4 specifically includes the following steps:
  • Step S4-1 select the key frequency range, select the filtering method and bandpass filter, filter the frequency domain sequence of each direction dimension of the coordinate position curve obtained in step S3-1, and obtain the filtered curve of each direction dimension.
  • the selection of the key frequency segment includes but is not limited to: a) selecting a frequency segment with the patient's tremor frequency as the center and a reasonable threshold (such as 1 Hz) as the radius; b) selecting a frequency segment related to the patient's diagnosed disease .
  • a frequency segment with the patient's tremor frequency as the center and a reasonable threshold (such as 1 Hz) as the radius
  • a frequency segment related to the patient's diagnosed disease For example, according to the "Guidelines for Primary Diagnosis and Treatment of Parkinson's Disease (2019)", the typical tremor frequency range for patients with Parkinson's disease is 4 to 6 Hz. In one embodiment of the present application, for patients with confirmed Parkinson's disease, the key frequency range is selected to be 4 to 6 Hz.
  • the filtering method may include frequency domain filtering and time domain filtering methods.
  • the bandpass filter includes but is not limited to: a frequency domain bandpass filter and a time domain finite impulse response (Finite Impulse Response, FIR) filter.
  • FIR Finite Impulse Response
  • Step S4-2 Based on the filtered curves of each direction dimension, calculate the comprehensive value of multiple direction dimensions. Tremor amplitude curve. More specifically, the filtered curves in each direction dimension can be squared point by point, summed, then squared, and then multiplied by 2 to obtain a comprehensive tremor amplitude curve in multiple direction dimensions.
  • the calculation formula is as follows (4):
  • step S2 the filtered curves of the uncalculated directional dimensions will be filled with all 0s.
  • Step S4-3 Determine the maximum value in the comprehensive tremor amplitude curve, where the maximum value corresponds to the maximum pixel tremor amplitude of the patient.
  • Figure 4 shows a schematic diagram of a coordinate position curve after band-pass filtering according to an embodiment of the present application.
  • the frequency domain sequence of each direction dimension obtained after frequency domain transformation in Figure 2 is passed through a frequency band-pass filter with a passband of 4 to 6 Hz, and then subjected to inverse discrete Fourier transform (IDFT) or inverse fast Fourier transform (IFFT), and then perform the absolute value operation to obtain the filtered curve in the time domain as shown in Figure 4.
  • IDFT inverse discrete Fourier transform
  • IFFT inverse fast Fourier transform
  • Figure 5 shows a schematic diagram of a comprehensive tremor amplitude curve according to an embodiment of the present application.
  • the comprehensive tremor amplitude curve obtained after processing the filtered curve shown in Figure 4 in step S4-2 is shown in Figure 5.
  • the maximum value point marked with a circle in Figure 5 corresponds to The maximum pixel tremor amplitude determined through step S4-3.
  • Step S5 Obtain the patient's actual human body feature length and the pixel length corresponding to the actual human body feature length, and calculate the patient's maximum actual tremor amplitude in proportion.
  • Step S5 specifically includes the following steps:
  • Step S5-1 According to the tremor-prone part captured by the video image, measure the actual length between two key points on the human body that is approximately the distance from the part to the imaging device, where the distance between the two key points on the human body is measured. Basically does not change with tremor movement.
  • the tremor-prone part captured by the video image is the face, and the tremor on the face is mainly concentrated in the lower jaw or For the lower lip, when the patient's face is facing the imaging device, the distance from the patient's pupil to the imaging device is basically the same as the distance from the mandible or lower lip to the imaging device, and the interpupillary distance basically does not change with tremor movement, so the patient can be measured
  • the interpupillary distance is used as the characteristic length of the actual human body.
  • Step S5-2 Obtain the pixel distance between the two corresponding human body key points in the video image.
  • the actual human body characteristic length measured in step S5-1 is the interpupillary distance
  • the pixel distance between the two pupil centers needs to be obtained in the video image.
  • the acquisition method includes manual calibration in the picture or automatic detection of the pixel coordinate position of the pupil through a deep learning algorithm, and then calculating the pixel distance between the pixel coordinate positions;
  • Step S5-3 Convert the patient's maximum actual tremor amplitude proportionally.
  • the conversion formula is as follows (5):
  • step S4 is the maximum pixel tremor amplitude obtained in step S4, L act is the actual human body feature length, L pix is the pixel distance in the video image, is the maximum actual tremor amplitude.
  • the patient's interpupillary distance was measured to be 60 mm
  • the pixel distance of the pupils in the video image was 50 pixels
  • the patient's maximum pixel tremor amplitude was 8.2 pixels as shown in Figure 5. Then the maximum actual tremor amplitude calculated by formula (5) It is 8.2 ⁇ 60/50 ⁇ 9.8 mm.
  • tremor frequency of movement disorders such as Parkinson's disease is usually 4-8 times/second, which is generally slightly slower and slightly larger than simple tremor, but faster and slightly smaller than action tremor. This feature can also help us distinguish other diseases, such as those caused by chorea, cerebellar disorders, and hyperthyroidism.
  • Figure 6 is a schematic structural diagram of a resting tremor quantitative assessment system for movement disorders according to an embodiment of the present application.
  • the system may include: a video image collection module configured to collect the patient's tremor-prone parts through an imaging device.
  • a video image of static tremor configured to extract the coordinates of key points of the human body in each frame of the video image to form all The coordinate position curve of the key points of the human body;
  • the tremor frequency extraction module is configured to perform frequency domain transformation on the coordinate position curve to extract the tremor frequency of the patient's resting tremor;
  • the pixel tremor amplitude extraction module is configured to perform frequency domain transformation in pairs
  • the coordinate position curve is band-pass filtered to extract the maximum pixel tremor amplitude of the patient's resting tremor; and an actual tremor amplitude conversion module is configured to obtain the patient's actual human body feature length and the video image corresponding to the actual human body feature length. Pixel length, proportionally converted to the patient's maximum actual tremor amplitude.
  • the imaging device needs to be kept stationary, and the distance between the imaging device and the patient remains fixed.
  • the coordinate position curve is a curve formed by the coordinates of the same key point of the human body in all frames of the video image, wherein the coordinate position curve includes at least one of the following directional and dimension curves: x direction curve, y-direction curve, z-direction curve.
  • the tremor frequency extraction module is configured to also be used for:
  • the maximum value of the tremor frequency in the stated direction As the frequency domain tremor amplitude in this direction dimension, the directional tremor frequency corresponding to the direction dimension with the largest frequency domain tremor amplitude is used as the patient's tremor frequency.
  • the pixel tremor amplitude extraction module is configured to also be used for:
  • the maximum value in the comprehensive tremor amplitude curve is determined, and the maximum value corresponds to the maximum pixel tremor amplitude of the patient.
  • the selection of the key frequency segment includes: a) selecting a frequency segment with the patient's tremor frequency as the center and a reasonable threshold as the radius; b) selecting a frequency segment related to the patient's diagnosed disease.
  • the actual tremor amplitude conversion module is configured to also be used for:
  • the tremor-prone part captured in the video image measure the actual length between two key points on the human body whose distance basically does not change with the tremor movement on this part;
  • the maximum actual tremor amplitude of the patient is converted in proportion according to the maximum pixel tremor amplitude, the actual human body characteristic length and the pixel distance.
  • the embodiments of the present application have the following beneficial effects: the technical solution of the present application has the advantages of convenience, accuracy, and non-contact, and provides a quantitative assessment of resting tremor symptoms in current movement disorders. A new, convenient solution that is easy to use in remote consultations.

Abstract

A quantitative evaluation method and system for static tremor of dyskinesia diseases. The evaluation method comprises the following steps: S1, acquiring, by means of an imaging device, a video image of static tremor at a tremor-prone part of a patient; S2, extracting coordinates of key points of the human body from each frame of the video image to form a coordinate position curve of the key points of the human body; S3, performing frequency domain transformation on the coordinate position curve, and extracting a tremor frequency of static tremor of the patient; S4, performing band-pass filtering on the coordinate position curve which is already subjected to frequency domain transformation, and extracting a maximum-pixel tremor amplitude of static tremor of the patient; and S5, obtaining an actual human body feature length of the patient and a pixel length corresponding to the actual human body feature length, and converting, according to the proportion, the actual human body feature length and the pixel length corresponding to the same into a maximum actual tremor amplitude of the patient. The method has the advantages of convenience, accuracy, and non-contact property, and therefore is particularly convenient to use in remote inquiry.

Description

运动障碍病的静止性震颤量化评估方法和系统Method and system for quantitative assessment of resting tremor in movement disorders
本申请要求于2022年03月24日递交的第202220667737.9号中国专利申请的优先权,在此全文引用上述中国专利申请公开的内容以作为本申请的一部分。This application claims priority to Chinese Patent Application No. 202220667737.9 submitted on March 24, 2022. The disclosure of the above-mentioned Chinese Patent Application is hereby cited in its entirety as part of this application.
技术领域Technical field
本申请涉及运动障碍病的量化评估领域,尤其涉及运动障碍病的静止性震颤量化评估方法和评估系统。This application relates to the field of quantitative assessment of movement disorders, and in particular to methods and systems for quantitative assessment of resting tremor in movement disorders.
背景技术Background technique
静止性震颤是帕金森病、特发性震颤等运动障碍类疾病的主要临床症状之一,震颤是否发生、震颤的幅度和频率等量化指标对该类疾病的临床诊断、评估和分级具有重要参考意义。Resting tremor is one of the main clinical symptoms of movement disorders such as Parkinson's disease and essential tremor. Quantitative indicators such as whether tremor occurs, the amplitude and frequency of tremor are important references for the clinical diagnosis, evaluation and grading of such diseases. significance.
目前静止性震颤的诊断评估主要通过医生目测量表打分、肌电图检查、穿戴式传感器等方式进行。目测量表打分机制严重依赖医生的临床经验,其主观性和差异性较强,因此在临床工作中经常可见到不同医生对同一患者的诊断、治疗以及愈后的判断存在较大差异的现象。肌电图检查多使用针电极并应用电刺激技术,被检者在检查过程中有一定的痛苦和损伤。穿戴式传感器在使用中需要穿戴、脱卸、充电、消毒等一系列操作流程,操作不够便捷。中国专利CN105701806B提供了一种基于深度图像的帕金森震颤运动特征检测方法,但在使用中需要患者佩戴纯色手套并使用特定设备(深度相机)进行拍摄,操作不够便捷,并且只能对手部的静止性震颤进行评估,无法应用于面部、腿部等身体其他部位。At present, the diagnostic evaluation of resting tremor is mainly carried out through doctors' visual scale scoring, electromyography examination, wearable sensors and other methods. The visual scale scoring mechanism relies heavily on doctors' clinical experience and is highly subjective and differentiated. Therefore, in clinical work, it is often seen that different doctors have large differences in their diagnosis, treatment, and prognosis judgments for the same patient. Electromyography examinations often use needle electrodes and apply electrical stimulation technology. The subjects will suffer certain pain and injury during the examination. Wearable sensors require a series of operating procedures such as putting on, taking off, charging, and disinfecting during use, and the operation is not convenient enough. Chinese patent CN105701806B provides a method for detecting Parkinson's tremor motion characteristics based on depth images, but the use requires the patient to wear solid-colored gloves and use a specific device (depth camera) to take pictures. The operation is not convenient enough, and it can only capture the stillness of the hand. It cannot be used to evaluate sexual tremors and cannot be applied to other parts of the body such as the face, legs, etc.
总之,现有技术难以通过便捷的、非接触的方式对静止性震颤症状进行客观量化评估。In short, it is difficult to objectively and quantitatively assess resting tremor symptoms in a convenient, non-contact manner with existing technology.
本发明说明书的此背景技术部分中所包括的信息,包括本文中所引用的任何参考文献及其任何描述或讨论,仅出于技术参考的目的而被包括在内,并且不被认为是将限制本发明范围的主题。 The information included in this Background section of the specification, including any references cited herein and any descriptions or discussions thereof, is included for technical reference purposes only and is not to be considered limiting. subject matter within the scope of the invention.
发明内容Contents of the invention
鉴于以上所述以及其它更多的构思而提出了本发明。The present invention has been proposed in view of the above and other concepts.
鉴于现有技术的不足,根据本发明的一方面的构思,旨在提供一种操作便捷的、非接触式的、结果客观的静止性震颤症状的量化评估方法。In view of the shortcomings of the existing technology, according to one aspect of the present invention, it is intended to provide a quantitative assessment method for resting tremor symptoms that is easy to operate, non-contact, and has objective results.
根据本申请的一个方面,提供一种运动障碍病的静止性震颤量化评估方法,能够通过便捷和非穿戴的方式对静止性震颤进行客观的量化评估,所述方法包括以下步骤:According to one aspect of the present application, a method for quantitative assessment of resting tremor in movement disorders is provided, which can objectively and quantitatively assess resting tremor in a convenient and non-wearable manner. The method includes the following steps:
S1:通过成像设备采集患者的震颤易发部位的静止性震颤的视频影像;S1: Collect video images of resting tremor in the patient's tremor-prone parts through imaging equipment;
S2:在所述视频影像的每一帧中提取人体关键点的坐标,形成所述人体关键点的坐标位置曲线;S2: Extract the coordinates of key points of the human body in each frame of the video image to form a coordinate position curve of the key points of the human body;
S3:对所述坐标位置曲线进行频域变换,提取患者的静止性震颤的震颤频率;S3: Perform frequency domain transformation on the coordinate position curve to extract the tremor frequency of the patient's resting tremor;
S4:对频域变换后的坐标位置曲线进行带通滤波,提取患者的静止性震颤的最大像素震颤幅度;S4: Perform band-pass filtering on the coordinate position curve after frequency domain transformation, and extract the maximum pixel tremor amplitude of the patient's resting tremor;
S5:获取患者的实际人体特征长度及所述实际人体特征长度对应的像素长度,按比例换算出患者的最大实际震颤幅度。S5: Obtain the patient's actual human body feature length and the pixel length corresponding to the actual human body feature length, and convert the patient's maximum actual tremor amplitude in proportion.
优选的是,在所述步骤S1中的采集过程中需保持所述成像设备静止,并且所述成像设备与患者的距离保持固定。Preferably, the imaging device needs to be kept stationary during the acquisition process in step S1, and the distance between the imaging device and the patient remains fixed.
优选的是,所述步骤S2中的所述坐标位置曲线为同一所述人体关键点在全部所述视频影像的帧中的坐标形成的曲线,其中所述坐标位置曲线包括下列方向维度的曲线中的至少一者:x方向曲线、y方向曲线、z方向曲线。Preferably, the coordinate position curve in step S2 is a curve formed by the coordinates of the same key point of the human body in all the frames of the video image, wherein the coordinate position curve includes the following directional dimension curves: At least one of: x-direction curve, y-direction curve, and z-direction curve.
更具体而言,所述步骤S3包括以下步骤:More specifically, the step S3 includes the following steps:
S3-1:对所述坐标位置曲线的各个方向维度分别进行离散傅里叶变换或快速傅里叶变换,得到各个方向维度的频域序列;S3-1: Perform discrete Fourier transform or fast Fourier transform on each direction dimension of the coordinate position curve to obtain a frequency domain sequence of each direction dimension;
S3-2:将各个方向维度的所述频域序列换算为频域幅度曲线;S3-2: Convert the frequency domain sequence in each direction dimension into a frequency domain amplitude curve;
S3-3:分别确定各个方向维度的所述频域幅度曲线在不小于4Hz的频段内绝对值最大的频率点,作为该方向维度的方向震颤频率;S3-3: Determine the frequency point with the largest absolute value of the frequency domain amplitude curve in each direction dimension in a frequency band of not less than 4 Hz, as the directional tremor frequency of that direction dimension;
S3-4:以所述方向震颤频率的最大值作为该方向维度下的频域震颤幅度,以所述频域震颤幅度最大的方向维度对应的所述方向震颤频率作为 患者的所述震颤频率。S3-4: Maximum value of tremor frequency in the stated direction As the frequency domain tremor amplitude in this direction dimension, the directional tremor frequency corresponding to the direction dimension with the largest frequency domain tremor amplitude is used as The patient's tremor frequency.
优选的是,所述步骤S4具体包括以下步骤:Preferably, the step S4 specifically includes the following steps:
S4-1:选取关键频率段,选取滤波方法和带通滤波器,对所述坐标位置曲线的各个方向维度的频域序列分别进行滤波,得到各个方向维度的滤波后曲线;S4-1: Select a key frequency segment, select a filtering method and a band-pass filter, filter the frequency domain sequences in each direction dimension of the coordinate position curve, and obtain filtered curves in each direction dimension;
S4-2:根据各个方向维度的所述滤波后曲线,计算多个方向维度的综合震颤幅度曲线;S4-2: Calculate comprehensive tremor amplitude curves in multiple direction dimensions based on the filtered curves in each direction dimension;
S4-3,确定所述综合震颤幅度曲线中的最大值,所述最大值对应患者的所述最大像素震颤幅度。S4-3: Determine the maximum value in the comprehensive tremor amplitude curve, where the maximum value corresponds to the maximum pixel tremor amplitude of the patient.
优选的是,所述步骤S4-1中的所述选取关键频率段包括:a)选取以患者的所述震颤频率为中心、合理阈值为半径的频率段;b)选取与患者已确诊病症有关的频率段。Preferably, the selection of key frequency segments in step S4-1 includes: a) selecting a frequency segment with the patient's tremor frequency as the center and a reasonable threshold as the radius; b) selecting a frequency segment related to the patient's diagnosed disease frequency range.
更具体而言,所述步骤S5包括以下步骤:More specifically, the step S5 includes the following steps:
S5-1:根据所述视频影像中所拍摄的所述震颤易发部位,测量该部位上两个距离基本不随震颤运动发生变化的人体关键点之间的实际长度;S5-1: According to the tremor-prone part captured in the video image, measure the actual length between two key points on the human body whose distances basically do not change with the tremor movement on this part;
S5-2:获取所述视频影像中两个所述人体关键点之间的像素距离;S5-2: Obtain the pixel distance between the two key points of the human body in the video image;
S5-3:根据所述最大像素震颤幅度、所述实际人体特征长度和所述像素距离,按比例换算患者的所述最大实际震颤幅度。S5-3: Convert the maximum actual tremor amplitude of the patient in proportion according to the maximum pixel tremor amplitude, the actual human body characteristic length and the pixel distance.
根据本申请的另一个方面,提供一种运动障碍病的静止性震颤量化评估系统,所述系统包括:According to another aspect of the present application, a resting tremor quantitative assessment system for movement disorders is provided, and the system includes:
视频影像采集模块,配置成通过成像设备采集患者的震颤易发部位的静止性震颤的视频影像;A video image collection module configured to collect video images of resting tremor in tremor-prone parts of the patient through an imaging device;
坐标位置曲线形成模块,配置成在所述视频影像的每一帧中提取人体关键点的坐标,形成所述人体关键点的坐标位置曲线;A coordinate position curve forming module configured to extract the coordinates of key points of the human body in each frame of the video image and form a coordinate position curve of the key points of the human body;
震颤频率提取模块,配置成对所述坐标位置曲线进行频域变换,提取患者的静止性震颤的震颤频率;A tremor frequency extraction module configured to perform frequency domain transformation on the coordinate position curve and extract the tremor frequency of the patient's resting tremor;
像素震颤幅度提取模块,配置成对频域变换后的坐标位置曲线进行带通滤波,提取患者的静止性震颤的最大像素震颤幅度;以及a pixel tremor amplitude extraction module configured to perform band-pass filtering on the coordinate position curve after frequency domain transformation, and extract the maximum pixel tremor amplitude of the patient's resting tremor; and
实际震颤幅度换算模块,配置成获取患者的实际人体特征长度及所述实际人体特征长度对应的视频影像中的像素长度,按比例换算出患者的最大实 际震颤幅度。The actual tremor amplitude conversion module is configured to obtain the patient's actual human body feature length and the pixel length in the video image corresponding to the actual human body feature length, and convert the patient's maximum real body feature length in proportion. tremor amplitude.
优选的是,在视频影像采集过程中,需保持所述成像设备静止,并且所述成像设备与患者的距离保持固定。Preferably, during the video image collection process, the imaging device needs to be kept stationary, and the distance between the imaging device and the patient remains fixed.
优选的是,所述坐标位置曲线为同一所述人体关键点在全部所述视频影像的帧中的坐标形成的曲线,其中所述坐标位置曲线包括下列方向维度的曲线中的至少一者:x方向曲线、y方向曲线、z方向曲线。Preferably, the coordinate position curve is a curve formed by the coordinates of the same key point of the human body in all frames of the video image, wherein the coordinate position curve includes at least one of the following directional and dimension curves: x direction curve, y-direction curve, z-direction curve.
优选的是,所述震颤频率提取模块配置成还用于:Preferably, the tremor frequency extraction module is configured to also be used for:
对所述坐标位置曲线的各个方向维度分别进行离散傅里叶变换或快速傅里叶变换,得到各个方向维度的频域序列;Perform discrete Fourier transform or fast Fourier transform on each direction dimension of the coordinate position curve, respectively, to obtain a frequency domain sequence of each direction dimension;
将各个方向维度的所述频域序列换算为频域幅度曲线;Convert the frequency domain sequence in each direction dimension into a frequency domain amplitude curve;
分别确定各个方向维度的所述频域幅度曲线在不小于4Hz的频段内绝对值最大的频率点,作为该方向维度的方向震颤频率;Determine the frequency point with the largest absolute value of the frequency domain amplitude curve in each direction dimension in a frequency band of not less than 4 Hz, respectively, as the directional tremor frequency of that direction dimension;
以所述方向震颤频率的最大值作为该方向维度下的频域震颤幅度,以所述频域震颤幅度最大的方向维度对应的所述方向震颤频率作为患者的所述震颤频率。The maximum value of the tremor frequency in the stated direction As the frequency domain tremor amplitude in this direction dimension, the directional tremor frequency corresponding to the direction dimension with the largest frequency domain tremor amplitude is used as the patient's tremor frequency.
优选的是,所述像素震颤幅度提取模块配置成还用于:Preferably, the pixel tremor amplitude extraction module is configured to also be used for:
选取关键频率段,选取滤波方法和带通滤波器,对所述坐标位置曲线的各个方向维度的频域序列分别进行滤波,得到各个方向维度的滤波后曲线;Select a key frequency segment, select a filtering method and a band-pass filter, filter the frequency domain sequences of each direction dimension of the coordinate position curve respectively, and obtain filtered curves of each direction dimension;
根据各个方向维度的所述滤波后曲线,计算多个方向维度的综合震颤幅度曲线;Calculate comprehensive tremor amplitude curves in multiple direction dimensions according to the filtered curves in each direction dimension;
确定所述综合震颤幅度曲线中的最大值,所述最大值对应患者的所述最大像素震颤幅度。The maximum value in the comprehensive tremor amplitude curve is determined, and the maximum value corresponds to the maximum pixel tremor amplitude of the patient.
优选的是,所述选取关键频率段包括:a)选取以患者的所述震颤频率为中心、合理阈值为半径的频率段;b)选取与患者已确诊病症有关的频率段。Preferably, the selection of the key frequency segment includes: a) selecting a frequency segment with the patient's tremor frequency as the center and a reasonable threshold as the radius; b) selecting a frequency segment related to the patient's diagnosed disease.
优选的是,所述实际震颤幅度换算模块配置成还用于:Preferably, the actual tremor amplitude conversion module is configured to also be used for:
根据所述视频影像中所拍摄的所述震颤易发部位,测量该部位上两个距离基本不随震颤运动发生变化的人体关键点之间的实际长度;According to the tremor-prone part captured in the video image, measure the actual length between two key points on the human body whose distance basically does not change with the tremor movement on this part;
获取所述视频影像中两个所述人体关键点之间的像素距离;Obtain the pixel distance between two key points of the human body in the video image;
根据所述最大像素震颤幅度、所述实际人体特征长度和所述像素距离, 按比例换算患者的所述最大实际震颤幅度。According to the maximum pixel tremor amplitude, the actual human body feature length and the pixel distance, Scale the patient's stated maximum actual tremor amplitude.
与现有技术相比,本申请的实施例具有如下的有益效果:Compared with the prior art, the embodiments of the present application have the following beneficial effects:
本申请的实施例具有便捷、准确、非接触性的优点,针对目前运动障碍病静止性震颤症状的量化评估,提供了一个全新、便捷的解决方案,并便于在远程问诊中使用。The embodiments of the present application have the advantages of convenience, accuracy, and non-contact, provide a new and convenient solution for the quantitative assessment of resting tremor symptoms in current movement disorders, and are easy to use in remote consultation.
本发明的更多实施例还能够实现其它未一一列出的有利技术效果,这些其它的技术效果在下文中可能有部分描述,并且对于本领域的技术人员而言在阅读了本发明后是可以预期和理解的。More embodiments of the present invention can also achieve other advantageous technical effects not listed one by one. These other technical effects may be partially described below, and for those skilled in the art after reading the present invention, they can expected and understood.
附图说明Description of the drawings
通过参考下文的描述连同附图,这些实施例的上述特征和优点及其他特征和优点以及实现它们的方式将更显而易见,并且可以更好地理解本发明的实施例,在附图中:The above features and advantages and other features and advantages of these embodiments, as well as the manner of achieving them, will be more apparent, and embodiments of the invention may be better understood, by referring to the following description taken in conjunction with the accompanying drawings, in which:
图1是根据本申请的一实施例的运动障碍病的静止性震颤量化评估方法的流程的示意图;Figure 1 is a schematic diagram of the flow of a quantitative assessment method for resting tremor in movement disorders according to an embodiment of the present application;
图2是根据本申请的一实施例的典型的帕金森病患者面部下颌尖的坐标位置曲线的示意图;Figure 2 is a schematic diagram of the coordinate position curve of the mandibular tip on the face of a typical Parkinson's disease patient according to an embodiment of the present application;
图3是根据本申请的一实施例的以Hz为单位的频域幅度曲线的示意图;Figure 3 is a schematic diagram of a frequency domain amplitude curve in Hz according to an embodiment of the present application;
图4是根据本申请的一实施例的经过带通滤波后的坐标位置曲线的示意图;Figure 4 is a schematic diagram of a coordinate position curve after band-pass filtering according to an embodiment of the present application;
图5是根据本申请的一实施例的综合震颤幅度曲线的示意图;和Figure 5 is a schematic diagram of a comprehensive tremor amplitude curve according to an embodiment of the present application; and
图6是根据本申请实施例的运动障碍病的静止性震颤量化评估系统的结构示意图。Figure 6 is a schematic structural diagram of a resting tremor quantitative assessment system for movement disorders according to an embodiment of the present application.
具体实施方式Detailed ways
在以下对附图和具体实施方式的描述中,将阐述本发明的一个或多个实施例的细节。从这些描述、附图以及权利要求中,可以清楚本发明的其它特征、目的和优点。The details of one or more embodiments of the invention are set forth in the following description of the drawings and the detailed description. Other features, objects and advantages of the invention will be apparent from the description, drawings and claims.
应当理解,所图示和描述的实施例在应用中不限于在以下描述中阐明或在附图中图示的构件的构造和布置的细节。所图示的实施例可以是其它的实 施例,并且能够以各种方式来实施或执行。各示例通过对所公开的实施例进行解释而非限制的方式来提供。实际上,将对本领域技术人员显而易见的是,在不背离本发明公开的范围或实质的情况下,可以对本发明的各实施例作出各种修改和变型。例如,作为一个实施例的一部分而图示或描述的特征,可以与另一实施例一起使用,以仍然产生另外的实施例。因此,本发明公开涵盖属于所附权利要求及其等同要素范围内的这样的修改和变型。It is to be understood that the illustrated and described embodiments are not limited in application to the details of construction and arrangement of components set forth in the following description or illustrated in the drawings. The illustrated embodiments may be other implementations Embodiments and can be implemented or performed in various ways. Each example is provided by way of explanation of the disclosed embodiments, not limitation. In fact, it will be apparent to those skilled in the art that various modifications and variations can be made in the various embodiments of the present invention without departing from the scope or spirit of the invention as disclosed. For example, features illustrated or described as part of one embodiment can be used with another embodiment to still produce further embodiments. Thus, it is intended that the present disclosure cover such modifications and variations as come within the scope of the appended claims and their equivalents.
同样,可以理解,本文中所使用的词组和用语是出于描述的目的,而不应当被认为是限制性的。本文中的“包括”、“包含”或“具有”及其变型的使用,旨在开放式地包括其后列出的项及其等同项以及附加的项。Likewise, it is to be understood that the phrases and terminology used herein are for the purpose of description and should not be regarded as limiting. The use of "including," "including," or "having" and variations thereof herein is intended to include the items listed thereafter and their equivalents as well as additional items in an open-ended manner.
下面将参考本发明的具体实施例对本发明进行更详细的描述。The present invention will be described in more detail below with reference to specific embodiments of the invention.
图1示出了根据本申请一实施例的运动障碍病的静止性震颤量化评估方法的流程示意图。根据本实施例的静止性震颤量化评估方法包括步骤S1~S5,具体如下。Figure 1 shows a schematic flowchart of a quantitative assessment method for resting tremor in movement disorders according to an embodiment of the present application. The quantitative assessment method for resting tremor according to this embodiment includes steps S1 to S5, specifically as follows.
步骤S1,通过成像设备采集患者的震颤易发部位的静止性震颤的视频影像。Step S1: Use imaging equipment to collect video images of resting tremor in the patient's tremor-prone parts.
可以使用的成像设备包括但不限于普通的单目摄像机、双目及多目摄像机、深度相机、红外摄像机等本领域常用的成像设备。能够进行静止性震颤量化评估的特定部位,也就是作为摄影或者成像对象的部位,一般是震颤易发部位,包括但不限于患者的嘴唇、下颌、整个面部、手掌、脚掌、整个上肢、整个下肢和全身等部位。所采集的视频影像的类型包括但不限于普通RGB彩色视频、灰度视频、带深度信息的彩色或灰度视频、双目及多目彩色或灰度视频等。Imaging equipment that can be used includes, but is not limited to, ordinary monocular cameras, binocular and multi-eye cameras, depth cameras, infrared cameras and other imaging equipment commonly used in this field. Specific parts that can be quantitatively assessed for resting tremor, that is, parts that are the subject of photography or imaging, are generally tremor-prone parts, including but not limited to the patient's lips, mandible, entire face, palms, soles, entire upper limbs, and entire lower limbs and other parts of the body. The types of video images collected include but are not limited to ordinary RGB color videos, grayscale videos, color or grayscale videos with depth information, binocular and multi-eye color or grayscale videos, etc.
例如在医院等专业的诊疗机构中,可采用双目摄像机、深度相机等较为复杂精准的成像设备,对患者静坐状态下的多个震颤易发部位分别进行视频影像采集,获取带深度信息的视频影像,便于进行更精准的处理分析。而在远程问诊的过程中,可通过普通手机经简单固定后,用手机自带的摄像头对患者静坐状态下的全身视频影像进行采集,这更方便于陪护人员操作使用普通的成像设备来采集视频影像。For example, in professional diagnosis and treatment institutions such as hospitals, more complex and accurate imaging equipment such as binocular cameras and depth cameras can be used to collect video images of multiple tremor-prone parts of patients while they are sitting still, and obtain videos with depth information. images to facilitate more precise processing and analysis. In the process of remote consultation, an ordinary mobile phone can be simply fixed and then the mobile phone's own camera can be used to collect whole-body video images of the patient while he is sitting quietly. This is more convenient for the accompanying staff to use ordinary imaging equipment to collect. video images.
另外,在视频影像的采集过程中,为了后续进行准确的量化评估,一般需保持成像设备静止,并且要求成像设备与患者的距离保持固定。 In addition, during the acquisition process of video images, in order to conduct accurate quantitative evaluation later, it is generally necessary to keep the imaging equipment stationary, and the distance between the imaging equipment and the patient is required to remain fixed.
步骤S2,在所采集的视频影像的每一帧中提取人体关键点的坐标,形成坐标位置曲线。Step S2: Extract the coordinates of key points of the human body in each frame of the collected video image to form a coordinate position curve.
其中,提取坐标的具体方法包括但不限于模板匹配、目标跟踪和目标识别等方法及这些方法的综合应用。人体关键点一般是在震颤易发部位上选取的特定点,包括但不限于容易出现震颤的上嘴唇中点、下嘴唇中点、下颌尖、腕关节、手指指尖、踝关节、趾尖等。Among them, specific methods for extracting coordinates include but are not limited to template matching, target tracking, target recognition and other methods, as well as the comprehensive application of these methods. Key points on the human body are generally specific points selected on parts prone to tremor, including but not limited to the midpoint of the upper lip, midpoint of the lower lip, tip of the mandible, wrist joints, fingertips, ankle joints, toe tips, etc. where tremor is prone to occur .
坐标位置曲线为追踪同一人体关键点在全部视频帧中的坐标而形成的曲线。该曲线通常具有多个方向维度,包括x方向曲线、y方向曲线、z方向曲线中的至少一者。The coordinate position curve is a curve formed by tracking the coordinates of the same human body key points in all video frames. The curve usually has multiple direction dimensions, including at least one of an x-direction curve, a y-direction curve, and a z-direction curve.
例如,可以采用模板匹配方法,利用预设的人体关键部位模型如面部轮廓模型、嘴部轮廓模型、手部轮廓模型等来与对应部位的视频影像进行匹配,并提取所述视频影像中与所述人体关键部位模型中对应的人体关键点匹配程度最高的点的坐标,作为所述人体关键点的坐标。For example, a template matching method can be used to use preset models of key parts of the human body, such as facial contour models, mouth contour models, hand contour models, etc., to match the video images of the corresponding parts, and extract the corresponding parts of the video images. The coordinates of the point with the highest degree of matching of the corresponding human body key points in the human body key part model are used as the coordinates of the human body key points.
还可以采用目标跟踪方法,通过人工标注人体关键点在视频影像的第一帧图像中的坐标位置,提取所述人体关键点及其邻域在该第一帧图像中的图像特征,并在后续的每一帧图像中提取相应的图像特征,应用目标跟踪算法计算出后续的每一帧图像中所标注的人体关键点的坐标。所述图像特征包括但不限于图像模板、频域信息、Hessian矩阵、边缘轮廓信息、灰度直方图、积分直方图、Harris角点、SIFT特征点、SURF特征点等。所述目标跟踪算法包括但不限于滑动窗口模板匹配、均值漂移、粒子滤波、SIFT算法、SURF算法等。The target tracking method can also be used to manually mark the coordinate positions of key points of the human body in the first frame of the video image, extract the image features of the key points of the human body and their neighborhoods in the first frame of the image, and subsequently The corresponding image features are extracted from each frame of image, and the target tracking algorithm is applied to calculate the coordinates of the human body key points marked in each subsequent frame of image. The image features include but are not limited to image templates, frequency domain information, Hessian matrices, edge contour information, grayscale histograms, integral histograms, Harris corner points, SIFT feature points, SURF feature points, etc. The target tracking algorithm includes but is not limited to sliding window template matching, mean shift, particle filter, SIFT algorithm, SURF algorithm, etc.
还可以采用目标识别方法,通过预先标注有某些人体关键点坐标的样本图像来对人工神经网络进行训练,以获得训练好的人工神经网络模型来对视频影像进行逐帧处理,识别出每一帧中的所述人体关键点坐标。The target recognition method can also be used to train the artificial neural network through sample images pre-labeled with the coordinates of certain human key points, so as to obtain the trained artificial neural network model to process the video images frame by frame and identify each The coordinates of the human body key points in the frame.
通常,基于模板匹配方法或目标识别方法提取的人体关键点的坐标位置在不同图像帧之间可能存在轻微波动,而基于目标跟踪方法提取的人体关键点的坐标位置在连续帧之间更为准确和稳定,但可能因局部形变过大而产生跟踪漂移或丢失,因此在实际中上述几种提取方法常需要综合应用。Generally, the coordinate positions of human body key points extracted based on template matching methods or target recognition methods may fluctuate slightly between different image frames, while the coordinate positions of human body key points extracted based on target tracking methods are more accurate between consecutive frames. and stable, but tracking drift or loss may occur due to excessive local deformation. Therefore, in practice, the above extraction methods often need to be applied comprehensively.
例如在本申请的一个实施例中,在视频影像的第一帧图像中通过目标识别方法识别出患者的下颌尖关键点,然后在后续帧中通过目标跟踪方法对该 关键点进行跟踪。当跟踪过程中跟随算法置信度低于设定阈值时,再应用目标识别方法对该关键点进行重新识别,然后再重新转入跟踪流程,如此循环。这样可以在无需人工标注的情况下获取更为准确和稳定的坐标位置曲线。For example, in one embodiment of the present application, the key point of the patient's mandibular tip is identified through the target recognition method in the first frame of the video image, and then the key point of the patient's mandibular tip is identified through the target tracking method in subsequent frames. Key points are tracked. When the confidence of the following algorithm is lower than the set threshold during the tracking process, the target recognition method is applied to re-identify the key point, and then the tracking process is re-entered, and so on. In this way, a more accurate and stable coordinate position curve can be obtained without manual annotation.
图2示出了根据本申请一实施例的典型的帕金森病患者面部下颌尖的坐标位置曲线示意图。如图2所示,本实施例在普通RGB视频影像中提取出一个典型的帕金森病患者面部下颌尖的坐标位置曲线。由于本实施例没有引入深度信息,所以只存在x和y两个方向维度,若在本申请的另一实施例中采用了带深度信息的成像设备进行视频影像采集,则可提供额外的z方向维度信息。从图2中可以看出,由于患者的轻微宏观运动,并不能简单直观地通过坐标位置曲线的最大值和最小值之差来获取患者的震颤幅度;另外,震颤频率也难以简单直观地算出,因此需要本申请实施例所提供的后续的处理步骤来进行更为精确的计算。Figure 2 shows a schematic diagram of the coordinate position curve of the mandibular tip on the face of a typical Parkinson's disease patient according to an embodiment of the present application. As shown in Figure 2, this embodiment extracts the coordinate position curve of the mandibular tip of a typical Parkinson's disease patient's face from ordinary RGB video images. Since this embodiment does not introduce depth information, there are only two dimensions of x and y. If an imaging device with depth information is used for video image collection in another embodiment of the present application, an additional z direction can be provided. Dimension information. As can be seen from Figure 2, due to the slight macroscopic movement of the patient, the patient's tremor amplitude cannot be obtained simply and intuitively through the difference between the maximum and minimum values of the coordinate position curve; in addition, the tremor frequency is also difficult to calculate simply and intuitively. Therefore, subsequent processing steps provided by the embodiments of the present application are needed to perform more accurate calculations.
步骤S3,对步骤S2中获得的坐标位置曲线进行频域变换,以提取患者的静止性震颤的震颤频率。步骤S3具体包括如下步骤:Step S3: Perform frequency domain transformation on the coordinate position curve obtained in step S2 to extract the tremor frequency of the patient's resting tremor. Step S3 specifically includes the following steps:
步骤S3-1,对坐标位置曲线的各个方向维度分别进行离散傅里叶变换(Discrete Fourier Transform,DFT)或快速傅里叶变换(Fast Fourier Transform,FFT),得到各个方向维度的频域序列,变换公式如下公式(1):
Fx(n)=DFT(x(t)),Fy(n)=DFT(y(t)),Fz(n)=DFT(z(t))    (1)
Step S3-1: Perform Discrete Fourier Transform (DFT) or Fast Fourier Transform (FFT) on each direction dimension of the coordinate position curve to obtain frequency domain sequences of each direction dimension. The transformation formula is as follows (1):
F x (n) = DFT (x (t)), F y (n) = DFT (y (t)), F z (n) = DFT (z (t)) (1)
其中,x(t)、y(t)和z(t)分别代表坐标位置曲线的三个方向维度,t=1,…,N代表时间采样点,n=1,…,N代表频域采样点,N代表视频影像的总帧数,DFT(□)代表离散傅里叶变换或快速傅里叶变换,Fx(n)、Fy(n)和Fz(n)分别代表三个方向维度的频域序列。Among them, x(t), y(t) and z(t) respectively represent the three directional dimensions of the coordinate position curve, t=1,...,N represents the time sampling point, n=1,...,N represents the frequency domain sampling point, N represents the total number of frames of the video image, DFT(□) represents discrete Fourier transform or fast Fourier transform, F x (n), F y (n) and F z (n) represent three directions respectively. Dimensional frequency domain sequence.
步骤S3-2,将各个方向维度的频域序列换算为以Hz为单位的频域幅度曲线,换算公式如下公式(2):
Step S3-2: Convert the frequency domain sequence in each direction dimension into a frequency domain amplitude curve in Hz. The conversion formula is as follows (2):
其中f代表频率,且满足f=n·fps/N,n≤N/2,fps代表视频影像的帧率,abs(□)代表取绝对值操作,F代表公式(1)中的任一方向维度的频域序列,代表频域幅度曲线。where f represents the frequency, and satisfies f=n·f ps /N, n≤N/2, f ps represents the frame rate of the video image, abs(□) represents the absolute value operation, and F represents any value in formula (1) A one-dimensional frequency domain sequence, Represents the frequency domain amplitude curve.
步骤S3-3,分别确定各个方向维度的频域幅度曲线在不小于4Hz的频段 内的绝对值最大的频率点,作为该方向维度的方向震颤频率,计算方法如下公式(3):
Step S3-3: Determine the frequency domain amplitude curve of each direction dimension in a frequency band of no less than 4Hz. The frequency point with the largest absolute value within is used as the directional tremor frequency in that direction dimension. The calculation method is as follows: formula (3):
其中fmax代表方向震颤频率。where f max represents the directional tremor frequency.
步骤S3-4,以方向震颤频率的最大值作为该方向维度下的频域震颤幅度,以频域震颤幅度最大的方向维度对应的方向震颤频率作为患者的静止性震颤的震颤频率。Step S3-4, use the maximum value of the tremor frequency in the direction As the frequency domain tremor amplitude in this direction dimension, the directional tremor frequency corresponding to the direction dimension with the largest frequency domain tremor amplitude is used as the tremor frequency of the patient's resting tremor.
图3示出了根据本申请一实施例的以Hz为单位的频域幅度曲线示意图。如图3所示的频域幅度曲线由图2中的坐标位置曲线经过步骤S3-1和S3-2得到。从图3中可以看出,除去4Hz以下的低频部分之后在5Hz处有明显的峰值(除去4Hz以下的低频分量是因为运动障碍病如帕金森病和特发性震颤病等的典型震颤频率均在4Hz以上),所以经过步骤S3-3提取得到的x、y两个方向的方向震颤频率均为5Hz,同时,y方向的频域震颤幅度大于x方向,于是经过步骤S3-3和S3-4得到的患者的震颤频率为5Hz。Figure 3 shows a schematic diagram of a frequency domain amplitude curve in Hz according to an embodiment of the present application. The frequency domain amplitude curve shown in Figure 3 is obtained from the coordinate position curve in Figure 2 through steps S3-1 and S3-2. As can be seen from Figure 3, there is an obvious peak at 5Hz after removing the low-frequency component below 4Hz (the low-frequency component below 4Hz is removed because the typical tremor frequencies of movement disorders such as Parkinson's disease and essential tremor are uniform. above 4Hz), so the directional tremor frequencies in the x and y directions extracted through step S3-3 are both 5Hz. At the same time, the frequency domain tremor amplitude in the y direction is greater than the x direction, so after steps S3-3 and S3- 4The patient's tremor frequency obtained is 5Hz.
步骤S4,对频域变换后的坐标位置曲线进行带通滤波,提取患者的静止性震颤的最大像素震颤幅度。步骤S4具体包括如下步骤:Step S4: Band-pass filter the coordinate position curve after frequency domain transformation to extract the maximum pixel tremor amplitude of the patient's resting tremor. Step S4 specifically includes the following steps:
步骤S4-1,选取关键频率段,选取滤波方法和带通滤波器,对步骤S3-1中得到的坐标位置曲线的各个方向维度的频域序列分别进行滤波,得到各个方向维度的滤波后曲线。Step S4-1, select the key frequency range, select the filtering method and bandpass filter, filter the frequency domain sequence of each direction dimension of the coordinate position curve obtained in step S3-1, and obtain the filtered curve of each direction dimension. .
其中,所述关键频率段的选取包括但不限于:a)选取以患者的震颤频率为中心、合理阈值(如1Hz)为半径的频率段;b)选取与患者的已确诊病症有关的频率段。例如,根据《帕金森病基层诊疗指南(2019年)》,帕金森病患者的典型震颤频率段为4至6Hz。在本申请的一个实施例中,对于已确诊的帕金森病患者,选择所述关键频率段为4至6Hz。Among them, the selection of the key frequency segment includes but is not limited to: a) selecting a frequency segment with the patient's tremor frequency as the center and a reasonable threshold (such as 1 Hz) as the radius; b) selecting a frequency segment related to the patient's diagnosed disease . For example, according to the "Guidelines for Primary Diagnosis and Treatment of Parkinson's Disease (2019)", the typical tremor frequency range for patients with Parkinson's disease is 4 to 6 Hz. In one embodiment of the present application, for patients with confirmed Parkinson's disease, the key frequency range is selected to be 4 to 6 Hz.
所述滤波方法可以包括频域滤波和时域滤波方法。所述带通滤波器包括但不限于:频域带通滤波器和时域有限冲击响应(Finite Impulse Response,FIR)滤波器。The filtering method may include frequency domain filtering and time domain filtering methods. The bandpass filter includes but is not limited to: a frequency domain bandpass filter and a time domain finite impulse response (Finite Impulse Response, FIR) filter.
步骤S4-2,根据各个方向维度的滤波后曲线,计算多个方向维度的综合 震颤幅度曲线。更具体而言,可以将各个方向维度的滤波后曲线进行逐点平方后求和、然后再开方,再乘以2,得到多个方向维度的综合震颤幅度曲线。计算公式如下公式(4):
Step S4-2: Based on the filtered curves of each direction dimension, calculate the comprehensive value of multiple direction dimensions. Tremor amplitude curve. More specifically, the filtered curves in each direction dimension can be squared point by point, summed, then squared, and then multiplied by 2 to obtain a comprehensive tremor amplitude curve in multiple direction dimensions. The calculation formula is as follows (4):
其中,分别为三个方向维度的滤波后曲线,若在步骤S2中只计算了其中一个或两个方向维度,则未计算的方向维度的滤波后曲线则以全0填充。in, and They are the filtered curves of the three directional dimensions respectively. If only one or two of the directional dimensions are calculated in step S2, the filtered curves of the uncalculated directional dimensions will be filled with all 0s.
步骤S4-3,确定综合震颤幅度曲线中的最大值,所述最大值对应患者的所述最大像素震颤幅度。Step S4-3: Determine the maximum value in the comprehensive tremor amplitude curve, where the maximum value corresponds to the maximum pixel tremor amplitude of the patient.
图4示出了根据本申请一实施例的经过带通滤波后的坐标位置曲线示意图。Figure 4 shows a schematic diagram of a coordinate position curve after band-pass filtering according to an embodiment of the present application.
例如在本申请的一个实施例中,将图2经过频域变换之后得到的各个方向维度的频域序列经过一个通带为4至6Hz的频率带通滤波器,然后经过离散傅里叶反变换(IDFT)或快速傅里叶反变换(IFFT),再进行取绝对值操作后,得到时域的滤波后曲线如图4所示。可见,经过滤波处理之后,所述坐标位置曲线中对应患者缓慢宏观运动的低频分量得到了滤除,得到了图4中震颤特征更加明显的曲线。同时,由于患者在各个方向维度上出现最大幅度的时刻点可能不同,不能简单地通过各个方向维度上计算最大振幅之后进行叠加,需要通过步骤S4-2和S4-3进行逐帧计算。For example, in one embodiment of the present application, the frequency domain sequence of each direction dimension obtained after frequency domain transformation in Figure 2 is passed through a frequency band-pass filter with a passband of 4 to 6 Hz, and then subjected to inverse discrete Fourier transform (IDFT) or inverse fast Fourier transform (IFFT), and then perform the absolute value operation to obtain the filtered curve in the time domain as shown in Figure 4. It can be seen that after filtering, the low-frequency component corresponding to the patient's slow macroscopic motion in the coordinate position curve is filtered out, and a curve with more obvious tremor characteristics in Figure 4 is obtained. At the same time, since the patient may have different moments of maximum amplitude in each direction dimension, we cannot simply calculate the maximum amplitude in each direction dimension and then superimpose it. Instead, we need to perform frame-by-frame calculations through steps S4-2 and S4-3.
图5示出了根据本申请一实施例的综合震颤幅度曲线示意图。Figure 5 shows a schematic diagram of a comprehensive tremor amplitude curve according to an embodiment of the present application.
在本申请的一个实施例中,将图4所示的滤波后曲线经过步骤S4-2处理之后得到的综合震颤幅度曲线如图5所示,图5中以圆圈标出的最大值点即对应经过步骤S4-3确定的最大像素震颤幅度。In one embodiment of the present application, the comprehensive tremor amplitude curve obtained after processing the filtered curve shown in Figure 4 in step S4-2 is shown in Figure 5. The maximum value point marked with a circle in Figure 5 corresponds to The maximum pixel tremor amplitude determined through step S4-3.
步骤S5,获取患者的实际人体特征长度及实际人体特征长度对应的像素长度,按比例换算出患者的最大实际震颤幅度。步骤S5具体包括如下步骤:Step S5: Obtain the patient's actual human body feature length and the pixel length corresponding to the actual human body feature length, and calculate the patient's maximum actual tremor amplitude in proportion. Step S5 specifically includes the following steps:
步骤S5-1,根据视频影像所拍摄的震颤易发部位,测量与该部位距离成像设备的距离近似的两个人体关键点之间的实际长度,其中所述两个人体关键点之间的距离基本不随震颤运动发生变化。例如,在本申请的一个实施例中,视频影像所拍摄的震颤易发部位为面部,面部的震颤主要集中在下颌或 下唇,在患者面部正对成像设备拍摄的情况下,患者的瞳孔到成像设备的距离与下颌或下唇到成像设备的距离基本相同,且瞳距基本不随震颤运动发生变化,因此可测量患者的瞳距作为所述实际人体特征长度。Step S5-1: According to the tremor-prone part captured by the video image, measure the actual length between two key points on the human body that is approximately the distance from the part to the imaging device, where the distance between the two key points on the human body is measured. Basically does not change with tremor movement. For example, in one embodiment of the present application, the tremor-prone part captured by the video image is the face, and the tremor on the face is mainly concentrated in the lower jaw or For the lower lip, when the patient's face is facing the imaging device, the distance from the patient's pupil to the imaging device is basically the same as the distance from the mandible or lower lip to the imaging device, and the interpupillary distance basically does not change with tremor movement, so the patient can be measured The interpupillary distance is used as the characteristic length of the actual human body.
步骤S5-2,获取视频影像中对应的两个人体关键点之间的像素距离。例如,在本申请的一个实施例中,若步骤S5-1中测量的实际人体特征长度为瞳距,则需在视频影像中获取两个瞳孔中心之间的像素距离。获取方法包括在画面中人工标定或通过深度学习算法自动检测出瞳孔的像素坐标位置,然后计算像素坐标位置之间的像素距离;Step S5-2: Obtain the pixel distance between the two corresponding human body key points in the video image. For example, in one embodiment of the present application, if the actual human body characteristic length measured in step S5-1 is the interpupillary distance, the pixel distance between the two pupil centers needs to be obtained in the video image. The acquisition method includes manual calibration in the picture or automatic detection of the pixel coordinate position of the pupil through a deep learning algorithm, and then calculating the pixel distance between the pixel coordinate positions;
步骤S5-3,按比例换算患者的所述最大实际震颤幅度,换算公式如下公式(5):
Step S5-3: Convert the patient's maximum actual tremor amplitude proportionally. The conversion formula is as follows (5):
其中,为在步骤S4得到的最大像素震颤幅度,Lact为实际人体特征长度,Lpix为视频影像中的像素距离,为最大实际震颤幅度。in, is the maximum pixel tremor amplitude obtained in step S4, L act is the actual human body feature length, L pix is the pixel distance in the video image, is the maximum actual tremor amplitude.
例如在本申请的一个实施例中,患者的瞳距经测量为60毫米,所述视频影像中瞳孔的像素距离为50个像素,患者的最大像素震颤幅度如图5所示为8.2个像素,则经过公式(5)计算得到的最大实际震颤幅度为8.2×60/50≈9.8毫米。For example, in one embodiment of the present application, the patient's interpupillary distance was measured to be 60 mm, the pixel distance of the pupils in the video image was 50 pixels, and the patient's maximum pixel tremor amplitude was 8.2 pixels as shown in Figure 5. Then the maximum actual tremor amplitude calculated by formula (5) It is 8.2×60/50≈9.8 mm.
通过以上的步骤S1~S5,可以得到震颤的幅度和频率等量化指标。这些指标可用于运动障碍病的静止性震颤的量化评估。运动障碍病例如帕金森病的震颤频率多为4-8次/秒,一般要比单纯性震颤稍慢些,幅度也稍大,而比动作性震颤的频率快,幅度也略小。这个特征也可以帮助我们区别其它的疾病,如因舞蹈病、小脑疾患、还有甲状腺机能亢进等引起的疾病。Through the above steps S1 to S5, quantitative indicators such as the amplitude and frequency of tremor can be obtained. These indicators can be used to quantitatively evaluate resting tremor in movement disorders. The tremor frequency of movement disorders such as Parkinson's disease is usually 4-8 times/second, which is generally slightly slower and slightly larger than simple tremor, but faster and slightly smaller than action tremor. This feature can also help us distinguish other diseases, such as those caused by chorea, cerebellar disorders, and hyperthyroidism.
上述详细阐述了根据本申请实施例的运动障碍病的静止性震颤量化评估方法,为了便于更好地实施本申请实施例的上述方案,相应地,下面提供了本申请实施例的系统装置。The above describes in detail the quantitative assessment method of resting tremor in movement disorders according to the embodiments of the present application. In order to facilitate better implementation of the above solution of the embodiments of the present application, accordingly, the system device of the embodiments of the present application is provided below.
参见图6,图6是根据本申请实施例的运动障碍病的静止性震颤量化评估系统的结构示意图,该系统可以包括:视频影像采集模块,配置成通过成像设备采集患者的震颤易发部位的静止性震颤的视频影像;坐标位置曲线形成模块,配置成在所述视频影像的每一帧中提取人体关键点的坐标,形成所 述人体关键点的坐标位置曲线;震颤频率提取模块,配置成对所述坐标位置曲线进行频域变换,提取患者的静止性震颤的震颤频率;像素震颤幅度提取模块,配置成对频域变换后的坐标位置曲线进行带通滤波,提取患者的静止性震颤的最大像素震颤幅度;以及实际震颤幅度换算模块,配置成获取患者的实际人体特征长度及所述实际人体特征长度对应的视频影像中的像素长度,按比例换算出患者的最大实际震颤幅度。Referring to Figure 6, Figure 6 is a schematic structural diagram of a resting tremor quantitative assessment system for movement disorders according to an embodiment of the present application. The system may include: a video image collection module configured to collect the patient's tremor-prone parts through an imaging device. A video image of static tremor; a coordinate position curve forming module configured to extract the coordinates of key points of the human body in each frame of the video image to form all The coordinate position curve of the key points of the human body; the tremor frequency extraction module is configured to perform frequency domain transformation on the coordinate position curve to extract the tremor frequency of the patient's resting tremor; the pixel tremor amplitude extraction module is configured to perform frequency domain transformation in pairs The coordinate position curve is band-pass filtered to extract the maximum pixel tremor amplitude of the patient's resting tremor; and an actual tremor amplitude conversion module is configured to obtain the patient's actual human body feature length and the video image corresponding to the actual human body feature length. Pixel length, proportionally converted to the patient's maximum actual tremor amplitude.
优选的是,在视频影像采集过程中,需保持所述成像设备静止,并且所述成像设备与患者的距离保持固定。Preferably, during the video image collection process, the imaging device needs to be kept stationary, and the distance between the imaging device and the patient remains fixed.
优选的是,所述坐标位置曲线为同一所述人体关键点在全部所述视频影像的帧中的坐标形成的曲线,其中所述坐标位置曲线包括下列方向维度的曲线中的至少一者:x方向曲线、y方向曲线、z方向曲线。Preferably, the coordinate position curve is a curve formed by the coordinates of the same key point of the human body in all frames of the video image, wherein the coordinate position curve includes at least one of the following directional and dimension curves: x direction curve, y-direction curve, z-direction curve.
优选的是,所述震颤频率提取模块配置成还用于:Preferably, the tremor frequency extraction module is configured to also be used for:
对所述坐标位置曲线的各个方向维度分别进行离散傅里叶变换或快速傅里叶变换,得到各个方向维度的频域序列;Perform discrete Fourier transform or fast Fourier transform on each direction dimension of the coordinate position curve, respectively, to obtain a frequency domain sequence of each direction dimension;
将各个方向维度的所述频域序列换算为频域幅度曲线;Convert the frequency domain sequence in each direction dimension into a frequency domain amplitude curve;
分别确定各个方向维度的所述频域幅度曲线在不小于4Hz的频段内绝对值最大的频率点,作为该方向维度的方向震颤频率;Determine the frequency point with the largest absolute value of the frequency domain amplitude curve in each direction dimension in a frequency band of not less than 4 Hz, respectively, as the directional tremor frequency of that direction dimension;
以所述方向震颤频率的最大值作为该方向维度下的频域震颤幅度,以所述频域震颤幅度最大的方向维度对应的所述方向震颤频率作为患者的所述震颤频率。The maximum value of the tremor frequency in the stated direction As the frequency domain tremor amplitude in this direction dimension, the directional tremor frequency corresponding to the direction dimension with the largest frequency domain tremor amplitude is used as the patient's tremor frequency.
优选的是,所述像素震颤幅度提取模块配置成还用于:Preferably, the pixel tremor amplitude extraction module is configured to also be used for:
选取关键频率段,选取滤波方法和带通滤波器,对所述坐标位置曲线的各个方向维度的频域序列分别进行滤波,得到各个方向维度的滤波后曲线;Select a key frequency segment, select a filtering method and a band-pass filter, filter the frequency domain sequences of each direction dimension of the coordinate position curve respectively, and obtain filtered curves of each direction dimension;
根据各个方向维度的所述滤波后曲线,计算多个方向维度的综合震颤幅度曲线;Calculate comprehensive tremor amplitude curves in multiple direction dimensions according to the filtered curves in each direction dimension;
确定所述综合震颤幅度曲线中的最大值,所述最大值对应患者的所述最大像素震颤幅度。The maximum value in the comprehensive tremor amplitude curve is determined, and the maximum value corresponds to the maximum pixel tremor amplitude of the patient.
优选的是,所述选取关键频率段包括:a)选取以患者的所述震颤频率为中心、合理阈值为半径的频率段;b)选取与患者已确诊病症有关的频率段。 Preferably, the selection of the key frequency segment includes: a) selecting a frequency segment with the patient's tremor frequency as the center and a reasonable threshold as the radius; b) selecting a frequency segment related to the patient's diagnosed disease.
优选的是,所述实际震颤幅度换算模块配置成还用于:Preferably, the actual tremor amplitude conversion module is configured to also be used for:
根据所述视频影像中所拍摄的所述震颤易发部位,测量该部位上两个距离基本不随震颤运动发生变化的人体关键点之间的实际长度;According to the tremor-prone part captured in the video image, measure the actual length between two key points on the human body whose distance basically does not change with the tremor movement on this part;
获取所述视频影像中两个所述人体关键点之间的像素距离;Obtain the pixel distance between two key points of the human body in the video image;
根据所述最大像素震颤幅度、所述实际人体特征长度和所述像素距离,按比例换算患者的所述最大实际震颤幅度。The maximum actual tremor amplitude of the patient is converted in proportion according to the maximum pixel tremor amplitude, the actual human body characteristic length and the pixel distance.
与现有技术相比,本申请的实施例具有如下的有益效果:本申请的技术方案具有便捷、准确、非接触性的优点,针对目前运动障碍病静止性震颤症状的量化评估,提供了一个全新、便捷的解决方案,并便于在远程问诊中使用。Compared with the existing technology, the embodiments of the present application have the following beneficial effects: the technical solution of the present application has the advantages of convenience, accuracy, and non-contact, and provides a quantitative assessment of resting tremor symptoms in current movement disorders. A new, convenient solution that is easy to use in remote consultations.
出于说明的目的而提出了对本发明的对实施例的前文描述。所述前文描述并非意图是穷举的,也并非将本发明限于所公开的精确步骤和/或形式,显然,根据上文的教导,本领域技术人员可以在权利要求的范围内做出许多修改和变型。本发明的范围和所有的等同者旨在由所附权利要求限定。 The foregoing description of embodiments of the invention has been presented for purposes of illustration. The foregoing description is not intended to be exhaustive or to limit the invention to the precise steps and/or forms disclosed. It will be apparent that, in light of the above teachings, one skilled in the art will be able to make many modifications within the scope of the claims. and variants. The scope of the invention and all equivalents thereto are intended to be defined by the appended claims.

Claims (14)

  1. 一种运动障碍病的静止性震颤量化评估方法,其特征在于,所述方法包括以下步骤:A quantitative assessment method for resting tremor in movement disorders, characterized in that the method includes the following steps:
    S1:通过成像设备采集患者的震颤易发部位的静止性震颤的视频影像;S1: Collect video images of resting tremor in the patient's tremor-prone parts through imaging equipment;
    S2:在所述视频影像的每一帧中提取人体关键点的坐标,形成所述人体关键点的坐标位置曲线;S2: Extract the coordinates of key points of the human body in each frame of the video image to form a coordinate position curve of the key points of the human body;
    S3:对所述坐标位置曲线进行频域变换,提取所述患者的静止性震颤的震颤频率;S3: Perform frequency domain transformation on the coordinate position curve, and extract the tremor frequency of the patient's resting tremor;
    S4:对所述频域变换后的坐标位置曲线进行带通滤波,提取患者的静止性震颤的最大像素震颤幅度;和S4: Perform band-pass filtering on the coordinate position curve after frequency domain transformation, and extract the maximum pixel tremor amplitude of the patient's resting tremor; and
    S5:获取患者的实际人体特征长度及所述实际人体特征长度对应的像素长度,按比例换算出患者的最大实际震颤幅度。S5: Obtain the patient's actual human body feature length and the pixel length corresponding to the actual human body feature length, and convert the patient's maximum actual tremor amplitude in proportion.
  2. 根据权利要求1所述的运动障碍病的静止性震颤量化评估方法,其特征在于,在所述步骤S1中的采集过程中需保持所述成像设备静止,并且所述成像设备与患者的距离保持固定。The quantitative assessment method for resting tremor in movement disorders according to claim 1, wherein the imaging device needs to be kept stationary during the acquisition process in step S1, and the distance between the imaging device and the patient should be kept fixed.
  3. 根据权利要求1所述的运动障碍病的静止性震颤量化评估方法,其特征在于,所述步骤S2中的所述坐标位置曲线为同一所述人体关键点在全部所述视频影像的帧中的坐标形成的曲线,其中所述坐标位置曲线包括下列方向维度的曲线中的至少一者:x方向曲线、y方向曲线、z方向曲线。The quantitative assessment method for resting tremor in movement disorders according to claim 1, wherein the coordinate position curve in step S2 is the same key point of the human body in all the frames of the video image. A curve formed by coordinates, wherein the coordinate position curve includes at least one of the following directional dimension curves: x-direction curve, y-direction curve, and z-direction curve.
  4. 根据权利要求1所述的运动障碍病的静止性震颤量化评估方法,其特征在于,所述步骤S3具体包括以下步骤:The quantitative assessment method for resting tremor in movement disorders according to claim 1, wherein step S3 specifically includes the following steps:
    S3-1:对所述坐标位置曲线的各个方向维度分别进行离散傅里叶变换或快速傅里叶变换,得到各个方向维度的频域序列;S3-1: Perform discrete Fourier transform or fast Fourier transform on each direction dimension of the coordinate position curve to obtain a frequency domain sequence of each direction dimension;
    S3-2:将各个方向维度的所述频域序列换算为频域幅度曲线;S3-2: Convert the frequency domain sequence in each direction dimension into a frequency domain amplitude curve;
    S3-3:分别确定各个方向维度的所述频域幅度曲线在不小于4Hz的频段内绝对值最大的频率点,作为该方向维度的方向震颤频率;S3-3: Determine the frequency point with the largest absolute value of the frequency domain amplitude curve in each direction dimension in a frequency band of not less than 4 Hz, as the directional tremor frequency of that direction dimension;
    S3-4:以所述方向震颤频率的最大值作为该方向维度下的频域震颤幅度,以所述频域震颤幅度最大的方向维度对应的所述方向震颤频率作为患者的所述震颤频率。S3-4: Maximum value of tremor frequency in the stated direction As the frequency domain tremor amplitude in this direction dimension, the directional tremor frequency corresponding to the direction dimension with the largest frequency domain tremor amplitude is used as the patient's tremor frequency.
  5. 根据权利要求1所述的运动障碍病的静止性震颤量化评估方法,其特征在于,所述步骤S4具体包括以下步骤: The quantitative assessment method for resting tremor in movement disorders according to claim 1, wherein step S4 specifically includes the following steps:
    S4-1:选取关键频率段,选取滤波方法和带通滤波器,对所述坐标位置曲线的各个方向维度的频域序列分别进行滤波,得到各个方向维度的滤波后曲线;S4-1: Select a key frequency segment, select a filtering method and a band-pass filter, filter the frequency domain sequences in each direction dimension of the coordinate position curve, and obtain filtered curves in each direction dimension;
    S4-2:根据各个方向维度的所述滤波后曲线,计算多个方向维度的综合震颤幅度曲线;S4-2: Calculate comprehensive tremor amplitude curves in multiple direction dimensions based on the filtered curves in each direction dimension;
    S4-3,确定所述综合震颤幅度曲线中的最大值,所述最大值对应患者的所述最大像素震颤幅度。S4-3: Determine the maximum value in the comprehensive tremor amplitude curve, where the maximum value corresponds to the maximum pixel tremor amplitude of the patient.
  6. 根据权利要求5所述的运动障碍病的静止性震颤量化评估方法,其特征在于,所述步骤S4-1中的所述选取关键频率段包括:a)选取以患者的所述震颤频率为中心、合理阈值为半径的频率段;b)选取与患者已确诊病症有关的频率段。The quantitative assessment method for resting tremor in movement disorders according to claim 5, wherein the selecting the key frequency segment in step S4-1 includes: a) selecting the patient's tremor frequency as the center , the reasonable threshold is the frequency segment of the radius; b) Select the frequency segment related to the patient's diagnosed disease.
  7. 根据权利要求1所述的运动障碍病的静止性震颤量化评估方法,其特征在于,所述步骤S5具体包括以下步骤:The quantitative assessment method for resting tremor in movement disorders according to claim 1, wherein step S5 specifically includes the following steps:
    S5-1:根据所述视频影像中所拍摄的所述震颤易发部位,测量该部位上两个距离基本不随震颤运动发生变化的人体关键点之间的实际长度;S5-1: According to the tremor-prone part captured in the video image, measure the actual length between two key points on the human body whose distances basically do not change with the tremor movement on this part;
    S5-2:获取所述视频影像中两个所述人体关键点之间的像素距离;和S5-2: Obtain the pixel distance between the two key points of the human body in the video image; and
    S5-3:根据所述最大像素震颤幅度、所述实际人体特征长度和所述像素距离,按比例换算患者的所述最大实际震颤幅度。S5-3: Convert the maximum actual tremor amplitude of the patient in proportion according to the maximum pixel tremor amplitude, the actual human body characteristic length and the pixel distance.
  8. 一种运动障碍病的静止性震颤量化评估系统,其特征在于,所述系统包括:A resting tremor quantitative assessment system for movement disorders, characterized in that the system includes:
    视频影像采集模块,配置成通过成像设备采集患者的震颤易发部位的静止性震颤的视频影像;A video image collection module configured to collect video images of resting tremor in tremor-prone parts of the patient through an imaging device;
    坐标位置曲线形成模块,配置成在所述视频影像的每一帧中提取人体关键点的坐标,形成所述人体关键点的坐标位置曲线;A coordinate position curve forming module configured to extract the coordinates of key points of the human body in each frame of the video image and form a coordinate position curve of the key points of the human body;
    震颤频率提取模块,配置成对所述坐标位置曲线进行频域变换,提取患者的静止性震颤的震颤频率;A tremor frequency extraction module configured to perform frequency domain transformation on the coordinate position curve and extract the tremor frequency of the patient's resting tremor;
    像素震颤幅度提取模块,配置成对频域变换后的坐标位置曲线进行带通滤波,提取患者的静止性震颤的最大像素震颤幅度;和a pixel tremor amplitude extraction module configured to perform band-pass filtering on the coordinate position curve after frequency domain transformation, and extract the maximum pixel tremor amplitude of the patient's resting tremor; and
    实际震颤幅度换算模块,配置成获取患者的实际人体特征长度及所述实际人体特征长度对应的视频影像中的像素长度,按比例换算出患者的最大实际震颤幅度。The actual tremor amplitude conversion module is configured to obtain the patient's actual human body characteristic length and the pixel length in the video image corresponding to the actual human body characteristic length, and convert the patient's maximum actual tremor amplitude in proportion.
  9. 根据权利要求8所述的运动障碍病的静止性震颤量化评估系统,其特征在于,在视频影像采集过程中,需保持所述成像设备静止,并且所述成像 设备与患者的距离保持固定。The resting tremor quantitative assessment system for movement disorders according to claim 8, characterized in that during the video image collection process, the imaging device needs to be kept stationary, and the imaging device The distance between the device and the patient remains fixed.
  10. 根据权利要求8所述的运动障碍病的静止性震颤量化评估系统,其特征在于,所述坐标位置曲线为同一所述人体关键点在全部所述视频影像的帧中的坐标形成的曲线,其中,所述坐标位置曲线包括下列方向维度的曲线中的至少一者:x方向曲线、y方向曲线、z方向曲线。The resting tremor quantitative assessment system for movement disorders according to claim 8, wherein the coordinate position curve is a curve formed by the coordinates of the same key point of the human body in all the frames of the video image, wherein , the coordinate position curve includes at least one of the following directional dimension curves: x-direction curve, y-direction curve, and z-direction curve.
  11. 根据权利要求8所述的运动障碍病的静止性震颤量化评估系统,其特征在于,所述震颤频率提取模块配置成还用于:The quantitative assessment system for resting tremor in movement disorders according to claim 8, wherein the tremor frequency extraction module is configured to:
    对所述坐标位置曲线的各个方向维度分别进行离散傅里叶变换或快速傅里叶变换,得到各个方向维度的频域序列;Perform discrete Fourier transform or fast Fourier transform on each direction dimension of the coordinate position curve, respectively, to obtain a frequency domain sequence of each direction dimension;
    将各个方向维度的所述频域序列换算为频域幅度曲线;Convert the frequency domain sequence in each direction dimension into a frequency domain amplitude curve;
    分别确定各个方向维度的所述频域幅度曲线在不小于4Hz的频段内绝对值最大的频率点,作为该方向维度的方向震颤频率;Determine the frequency point with the largest absolute value of the frequency domain amplitude curve in each direction dimension in a frequency band of not less than 4 Hz, respectively, as the directional tremor frequency of that direction dimension;
    以所述方向震颤频率的最大值作为该方向维度下的频域震颤幅度,以所述频域震颤幅度最大的方向维度对应的所述方向震颤频率作为患者的所述震颤频率。The maximum value of the tremor frequency in the stated direction As the frequency domain tremor amplitude in this direction dimension, the directional tremor frequency corresponding to the direction dimension with the largest frequency domain tremor amplitude is used as the patient's tremor frequency.
  12. 根据权利要求8所述的运动障碍病的静止性震颤量化评估系统,其特征在于,所述像素震颤幅度提取模块配置成还用于:The quantitative assessment system for resting tremor in movement disorders according to claim 8, wherein the pixel tremor amplitude extraction module is configured to also be used for:
    选取关键频率段,选取滤波方法和带通滤波器,对所述坐标位置曲线的各个方向维度的频域序列分别进行滤波,得到各个方向维度的滤波后曲线;Select a key frequency segment, select a filtering method and a band-pass filter, filter the frequency domain sequences of each direction dimension of the coordinate position curve respectively, and obtain filtered curves of each direction dimension;
    根据各个方向维度的所述滤波后曲线,计算多个方向维度的综合震颤幅度曲线;Calculate comprehensive tremor amplitude curves in multiple direction dimensions according to the filtered curves in each direction dimension;
    确定所述综合震颤幅度曲线中的最大值,所述最大值对应患者的所述最大像素震颤幅度。The maximum value in the comprehensive tremor amplitude curve is determined, and the maximum value corresponds to the maximum pixel tremor amplitude of the patient.
  13. 根据权利要求12所述的运动障碍病的静止性震颤量化评估系统,其特征在于,所述选取关键频率段包括:a)选取以患者的所述震颤频率为中心、合理阈值为半径的频率段;b)选取与患者已确诊病症有关的频率段。The resting tremor quantitative assessment system for movement disorders according to claim 12, wherein the selecting a key frequency segment includes: a) selecting a frequency segment with the patient's tremor frequency as the center and a reasonable threshold as the radius. ; b) Select frequency segments related to the patient's diagnosed disease.
  14. 根据权利要求8所述的运动障碍病的静止性震颤量化评估系统,其特征在于,所述实际震颤幅度换算模块配置成还用于:The quantitative assessment system for resting tremor in movement disorders according to claim 8, wherein the actual tremor amplitude conversion module is configured to also be used for:
    根据所述视频影像中所拍摄的所述震颤易发部位,测量该部位上两个距离基本不随震颤运动发生变化的人体关键点之间的实际长度;According to the tremor-prone part captured in the video image, measure the actual length between two key points on the human body whose distance basically does not change with the tremor movement on this part;
    获取所述视频影像中两个所述人体关键点之间的像素距离;Obtain the pixel distance between two key points of the human body in the video image;
    根据所述最大像素震颤幅度、所述实际人体特征长度和所述像素距离, 按比例换算患者的所述最大实际震颤幅度。 According to the maximum pixel tremor amplitude, the actual human body feature length and the pixel distance, Scale the patient's stated maximum actual tremor amplitude.
PCT/CN2023/074823 2022-03-24 2023-02-07 Quantitative evaluation method and system for static tremor of dyskinesia diseases WO2023179218A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202220667737.9 2022-03-24
CN202220667737 2022-03-24

Publications (1)

Publication Number Publication Date
WO2023179218A1 true WO2023179218A1 (en) 2023-09-28

Family

ID=88099802

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/074823 WO2023179218A1 (en) 2022-03-24 2023-02-07 Quantitative evaluation method and system for static tremor of dyskinesia diseases

Country Status (2)

Country Link
CN (1) CN116831519A (en)
WO (1) WO2023179218A1 (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5293879A (en) * 1991-09-23 1994-03-15 Vitatron Medical, B.V. System an method for detecting tremors such as those which result from parkinson's disease
US20130338539A1 (en) * 2012-06-14 2013-12-19 International Business Machines Corporation Software program for monitoring a hand tremor of an end-user via a computer mouse input device
CN112674762A (en) * 2020-12-28 2021-04-20 江苏省省级机关医院 Parkinson tremble evaluation device based on wearable inertial sensor
US20210244316A1 (en) * 2020-02-11 2021-08-12 Prince Mohammad Bin Fahd University Smart watch for tremor monitoring and control
RU2764568C1 (en) * 2021-04-05 2022-01-18 Автономная некоммерческая образовательная организация высшего образования «Сколковский институт науки и технологий» Method for diagnosing parkinson's disease based on video data analysis using machine learning
US20220031194A1 (en) * 2020-08-03 2022-02-03 Gyrogear Limited Systems and methods for automated tremor management, tracking and recommendations

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5293879A (en) * 1991-09-23 1994-03-15 Vitatron Medical, B.V. System an method for detecting tremors such as those which result from parkinson's disease
US20130338539A1 (en) * 2012-06-14 2013-12-19 International Business Machines Corporation Software program for monitoring a hand tremor of an end-user via a computer mouse input device
US20210244316A1 (en) * 2020-02-11 2021-08-12 Prince Mohammad Bin Fahd University Smart watch for tremor monitoring and control
US20220031194A1 (en) * 2020-08-03 2022-02-03 Gyrogear Limited Systems and methods for automated tremor management, tracking and recommendations
CN112674762A (en) * 2020-12-28 2021-04-20 江苏省省级机关医院 Parkinson tremble evaluation device based on wearable inertial sensor
RU2764568C1 (en) * 2021-04-05 2022-01-18 Автономная некоммерческая образовательная организация высшего образования «Сколковский институт науки и технологий» Method for diagnosing parkinson's disease based on video data analysis using machine learning

Also Published As

Publication number Publication date
CN116831519A (en) 2023-10-03

Similar Documents

Publication Publication Date Title
US9996739B2 (en) System and method for automatic gait cycle segmentation
Wang et al. Automatic evaluation of the degree of facial nerve paralysis
EP2960862B1 (en) A method for stabilizing vital sign measurements using parametric facial appearance models via remote sensors
US11663845B2 (en) Method and apparatus for privacy protected assessment of movement disorder video recordings
Jiang et al. Automatic facial paralysis assessment via computational image analysis
CN110338759B (en) Facial pain expression data acquisition method
CN111933275A (en) Depression evaluation system based on eye movement and facial expression
WO2012171405A1 (en) Eyeball movement monitoring method and device
Ganfure Using video stream for continuous monitoring of breathing rate for general setting
CN114999646B (en) Newborn exercise development assessment system, method, device and storage medium
CN110660454A (en) Cancer pain real-time assessment instrument and assessment method thereof
Fiorin et al. Computerized analysis of narrow-field ROP images for the assessment of vessel caliber and tortuosity
CN115054250A (en) Image overflow motion detection analysis method and system
KR101996027B1 (en) Method and system for extracting Heart Information of Frequency domain by using pupil size variation
Zhang et al. Computer identification and quantification of fissured tongue diagnosis
WO2023179218A1 (en) Quantitative evaluation method and system for static tremor of dyskinesia diseases
EP3949832A1 (en) Illness aggravation estimation system
Hou et al. A computerized system of nail-fold capillaroscopy for dry eye disease diagnosis
CN110403631A (en) A kind of Noninvasive intracranial pressure measurement method based on continuous ultrasound image
Boccanfuso et al. Collecting heart rate using a high precision, non-contact, single-point infrared temperature sensor
CN113807323A (en) Accurate hand function evaluation system and method based on image recognition
JP5981113B2 (en) Evaluation method for sagging of jaw
Mengash et al. Methodology for Detecting Strabismus through Video Analysis and Intelligent Mining Techniques.
CN112022131B (en) Long-time continuous non-contact heart rate measurement method and system
Schneider et al. Preliminary investigation of walking motion using a combination of image and signal processing

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23773469

Country of ref document: EP

Kind code of ref document: A1