CN116012568A - System for acquiring cardiac rhythm information through photographing electrocardiogram - Google Patents
System for acquiring cardiac rhythm information through photographing electrocardiogram Download PDFInfo
- Publication number
- CN116012568A CN116012568A CN202310006237.XA CN202310006237A CN116012568A CN 116012568 A CN116012568 A CN 116012568A CN 202310006237 A CN202310006237 A CN 202310006237A CN 116012568 A CN116012568 A CN 116012568A
- Authority
- CN
- China
- Prior art keywords
- image
- electrocardiographic
- electrocardio
- rhythm
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention discloses a system for acquiring electrocardiographic rhythm information by photographing an electrocardiogram, wherein an electrocardiographic region extraction unit is responsible for acquiring a 12-lead paper electrocardiogram report photographed by a mobile phone camera, removing irrelevant additional information and extracting an electrocardiographic region based on a target detection neural network; the image processing unit adopts algorithms such as image denoising, inclination correction, shadow removal, data enhancement and the like to preprocess the image; the electrocardiographic rhythm information acquisition and abnormality classification unit acquires electrocardiographic rhythm information based on a convolutional neural network and classifies rhythm abnormality of the electrocardiographic rhythm information; and the android mobile phone application program functional unit is used for feeding the classification result of the electrocardiographic rhythm information of the user back to the result feedback module of the visiting user. The invention can acquire and identify and classify the electrocardiographic rhythm information by taking the paper electrocardiogram report photo, is a manageable, portable and low-cost tool, and is helpful for improving electrocardiogram interpretation in doctor aided diagnosis.
Description
Technical Field
The invention relates to the field of electrocardiograph intelligent diagnosis, in particular to a system for acquiring electrocardiograph rhythm information by taking a photo of an electrocardiogram.
Background
Electrocardiography (ECG) is a widely used non-invasive medical test that measures heart conditions by tracking the electrical activity of the heart. An electrocardiogram contains a large amount of information that directly reflects the physiology of the heart, as its morphological and temporal features are produced by the electrical and structural changes of the heart. In clinical diagnosis, an electrocardiogram is a main method for monitoring the heart electrical activity of a patient, so that the electrocardiogram is widely applied to aspects of disease diagnosis, treatment planning and the like.
Existing methods can be further divided into two types. The first is a feature-based method, in which morphological features such as frequency domain and wavelet transform are extracted first. Based on these features, various types of machine learning algorithms, such as support vector machines, decision trees and neural networks are used to train anomaly detection models. Another recently proposed approach is based on deep neural networks. These methods perform anomaly detection in an end-to-end process by extracting advanced features from electrocardiographic data. They learn the underlying representation directly from the data through some feature learning architecture.
While these methods can achieve competitive results on some common data sets, they are very challenging to apply in a practical clinical setting. First, most existing methods rely on electrocardiographic data signals. However, electrocardiographic data in the real world is typically collected and stored as images, which is a key source that algorithms need to consider.
Unlike an electrocardiogram signal, which consists of a plurality of clean and well-separated lead signals, the paper centroid image is blurred. There is an overlap between waveforms from different leads and closely-overlaid auxiliary axes (e.g., time and voltage axes) in the image, which presents challenges to accurately extracting information. Furthermore, the data sampling rate drops from hundreds of hertz in the digital signal to below ten hertz in the image data, resulting in serious information loss.
Thus, the large gap between the electrocardiographic signals and the images will fundamentally affect the performance of the feature-based and deep neural network-based general methods. For feature-based methods, it is actually difficult to extract the desired features. While one potential solution is to digitize the image first, it is difficult to extract the waveform directly from the paper ecg, and the complex noise and interference, patient information unrelated to the ecg, and manual labeling done by the physician on the ecg drawing all affect the segmentation of the ecg and the extraction of the clean waveform. Therefore, the related art usually needs to manually cut the electrocardiograph with manpower and time resources, and the digitizing efficiency and quality are low.
Disclosure of Invention
Aiming at the defects or improvement demands of the prior art, the invention provides a system for acquiring the electrocardiographic rhythm information by photographing an electrocardiogram, and aims at the defect that manual cutting of a paper electrocardiogram needs to consume a large amount of manpower and material resources, and the invention adopts a target detection technology to extract an electrocardiographic region in a 12-lead electrocardiogram report; obtaining rhythm information by adopting a deep convolutional neural network model and carrying out abnormal classification; the android application program with a concise interface and convenient operation is developed, and the requirement of providing auxiliary diagnosis information for doctors is met.
The invention provides a system for acquiring electrocardio rhythm information through photographing an electrocardiogram, which comprises an electrocardio area extraction unit, an image processing unit, an electrocardio rhythm information acquisition and abnormality classification unit and an android mobile phone application program functional unit, wherein the android mobile phone application program functional unit comprises an image reading module, an image uploading module and a result feedback module; the output end of the image reading module in the android application program functional unit is connected with the input end of the electrocardio region extraction unit, the output end of the electrocardio region extraction unit is connected with the input end of the image processing unit, the output end of the image processing unit is connected with the input end of the electrocardio rhythm information acquisition and abnormality classification unit, and the output end of the electrocardio rhythm information acquisition and abnormality classification unit is connected with the output end of the result feedback module in the android application program functional unit.
Specifically, the electrocardiographic region extraction unit acquires a 12-lead electrocardiographic report shot by a user through an image reading module in the android mobile phone application program functional unit, removes irrelevant information such as personal information of a patient and diagnosis marks of doctors, extracts an electrocardiographic waveform region based on a target detection network and transmits the electrocardiographic waveform region to the image processing unit;
specifically, the image processing unit adopts algorithms such as image denoising, image inclination correction, shadow removal, data enhancement and the like to preprocess the extracted electrocardio region, and the preprocessed image is used as input of an electrocardio rhythm abnormality recognition and classification unit;
specifically, the electrocardiographic rhythm information acquisition and abnormality classification unit acquires electrocardiographic rhythm information based on a convolutional neural network and classifies rhythm abnormality of the electrocardiographic rhythm information;
specifically, the android mobile phone application program functional unit comprises an image reading module, an image uploading module and a result feedback module; the image reading module is responsible for shooting, receiving and reading an electrocardio report of a user; the image uploading module is responsible for automatically transmitting the image file to the back-end server; the result feedback module is responsible for feeding back the classification result of the electrocardiographic rhythm information to the access user.
Specifically, the electrocardiographic region extraction unit includes the following steps:
(1) Performing image preprocessing on the original electrocardio report picture to improve the image quality; manually marking the left upper coordinate and the right lower coordinate of the electrocardiographic region by using a marking tool to the preprocessed electrocardiographic report graph to obtain a position label; randomly dividing the collected electrocardio report pictures into a training set, a verification set and a test set according to a specific proportion;
(2) Training and optimizing a YOLO target detection network based on the training set and the verification set by utilizing the preprocessed electrocardio report image and the position label, and setting a confidence threshold of the spatial position of an electrocardio area to obtain spatial position information; then slicing based on the spatial position information of the electrocardio region to obtain a downsampling feature map and an optimized electrocardio region extraction model;
(3) Predicting the electrocardiographic report picture in the test set by using the electrocardiographic region extraction model based on the YOLO target detection network;
(4) And carrying out precision evaluation on the prediction result of the test set.
Specifically, the training optimization process for the YOLO target detection network by using the training set and the verification set in the step (2) is as follows:
the method comprises the steps of carrying out normalization processing on an electrocardiographic report image in a training set, inputting the electrocardiographic report image into a backstone part of a network, obtaining three feature images with different scales, namely a minimum receptive field with a maximum scale, a moderate receptive field with a medium scale and a maximum receptive field with a minimum scale, inputting the feature images into a head part of the network, carrying out up-sampling and feature fusion on the feature images, and obtaining tensor data under three different scales through confidence threshold filtering and non-maximum suppression filtering. And calculating a loss value between the tensor data and the label value, calculating a network gradient through back propagation, updating a weight value, finally optimizing a network by using a verification set, obtaining a group of hyper-parameter values with minimum loss, and finally obtaining an electrocardiographic region extraction model based on the YOLO target detection network.
Specifically, the evaluation index of the extraction precision of the evaluation area in the step (4) is an average precision mAP, and the calculation process is as follows:
(1) And calculating the intersection ratio of the predicted frame area and the actual frame area of the target detection network, wherein the value of the intersection ratio can reflect the fitting degree of the predicted frame and the actual frame. When the cross ratio is larger than the threshold value, judging that the predicted result is correct;
(2) Calculating the average precision of all the electrocardio report images in the test set;
(3) mAP, the sum of the average accuracy of the test set divided by the total number of test set categories, is calculated.
Specifically, the image processing unit comprises image denoising, inclination correction, shadow removal and data enhancement; specifically, the image inclination correction adopts a Hough algorithm to identify a background grid line segment of an electrocardiographic region, and then calculates an image inclination angle to correct the image inclination angle.
Specifically, the electrocardiographic rhythm information obtaining and abnormality classifying unit obtains electrocardiographic rhythm information based on a convolutional neural network, and identifies and classifies the electrocardiographic rhythm information, including:
s1: constructing an electrocardiogram data set under each characteristic classification label according to the electrocardio region image and the corresponding rhythm abnormality classification label output by the image processing unit;
s2: preprocessing an electrocardiograph image in the electrocardiograph data set, and dividing the electrocardiograph data set into a training set, a verification set and a test set through a random classification method;
s3: constructing a deep convolutional neural network model, and training and optimizing the deep convolutional neural network model based on the training set and the verification set to obtain an initial electrocardiographic rhythm abnormality classification neural network model; the electrocardiographic rhythm abnormality classification neural network model takes an electrocardiographic region image in a paper centroid electricity report as input and takes the probability that the image features accord with each rhythm feature classification label as output;
s4: and carrying out model test on the initial electrocardiographic rhythm abnormal classification neural network model based on the test set, judging whether an evaluation index meets a preset evaluation standard, and if so, obtaining a target electrocardiographic image rhythm abnormal classification model.
Specifically, in the step S1, the cardiac rhythm classification tag includes seven, normal sinus rhythm, atrial fibrillation, one degree atrioventricular block, left bundle branch block, right bundle branch block, atrial premature beat, ventricular premature beat.
Specifically, the convolutional neural network model mentioned in the step S3 can accurately classify the electrocardiographic rhythm abnormalities. The model input is a vector of dimensions 1 x 7, corresponding to the particular type of cardiac rhythm therein. Because patients may have multiple heart rhythm abnormalities, each electrocardiographic report may correspond to multiple labels, and thus the output vector of the model may contain more than one non-zero element.
Specifically, the convolutional neural network model is divided into an input end, a backbone network and an output end; the input dimension is a tensor of 675×1450×3, the backbone network has eight layers, the first seven layers are convolution layers, in each convolution layer, the convolution kernel is fixed to be 5×5, and the step size is 2. After each convolution layer, an activation function (ReLu) with batch normalization is added. Then two full connection blocks, each with a ReLu nonlinear activation function and two full connection layers. To prevent the occurrence of overfitting, a discard probability of 0.6 is set in the full connection layer. The output layer contains 7 Sigmoid functions that can return the probability distribution of the corresponding 7 output categories, namely normal sinus rhythm and six rhythm anomaly types.
Specifically, the training optimization process in step S3 specifically includes:
s31: taking an electrocardiographic region image obtained by an image processing unit as input, and taking the prediction probability that the electrocardiographic image feature accords with each rhythm feature classification label as output to construct a convolutional neural network model;
s32: the characteristics of the electrocardiogram training set are obtained by a forward propagation algorithm, and a prediction result is obtained through the convolution network model;
s33: calculating a loss function, and obtaining a first loss value by the difference between a result obtained by the electrocardiogram training set through the neural network and a real label value;
s34: calculating gradients in the convolutional neural network model through a back propagation algorithm and an optimization algorithm, and updating weights;
s35: the verification set is used for optimizing the convolutional neural network, the super parameters are adjusted, which group of super parameters have the best performance according to the performance of the verification set, and a second loss value is obtained by the difference between the result obtained by the electrocardiogram verification set through the neural network and the real label;
s36: repeatedly executing the five steps until the model converges, and simultaneously drawing a change curve of the loss value by using a visualization method; the first loss value is used for updating the weight parameters of the convolutional neural network, and the second loss value is used for saving a final network model for classifying the electrocardiographic rhythm types.
Specifically, the loss function in step S33 is a multi-class cross entropy loss function, and the ratio of the maximum number of samples in all the cardiac rhythm types to the number of samples is used as the weight value in the calculation. An adaptive moment estimate (Adam) is selected as an optimizer to update the weight values of the convolutional neural network model during back propagation. For the final convolutional neural network model, the weights of the final convolutional neural network model are updated continuously, and finally the maximum iteration times and the smaller loss value are achieved.
Specifically, the convolutional neural network model mentioned in the step S3 obtains random weights using a conventional initialization technique. The initial value of the learning rate is 0.001 and the momentum parameter is initialized to 0.9. 100 training rounds are iterated, and the number of instances of the model is 64 in each training round. When the resulting error of the validation set exceeds six rounds of training without reduction, the learning rate is divided by 10.
Specifically, the evaluation indexes in the step S4 include accuracy (precision), F1 value (F1-score), recall rate (recovery), hamming loss (HammingLoss). Meanwhile, the accuracy, recall and the like of each category are respectively calculated through a metric module in the Sklearn machine learning library.
Specifically, the android mobile phone application program functional unit comprises an image reading module, an image uploading module and a result feedback module;
specifically, the image reading module is responsible for shooting, receiving and reading an electrocardiographic report of a user; the user shoots by using a mobile phone or other mobile equipment to acquire an image of a paper electrocardiograph report, firstly, the user places the electrocardiograph report on a flat desktop with sufficient illumination, and then the shooting interface of the application program is placed at a positive position to shoot; then previewing on the mobile phone, if the display is fuzzy, indicating that the photographing cannot be correctly focused, and deleting the re-photographing; the user can also select from the album that an electrocardiogram photo has been taken; finally uploading the shot electrocardio report image to an application program;
specifically, the image uploading module is responsible for automatically transmitting an electrocardiographic report image sent by a user side to a rear-end server to obtain electrocardiographic rhythm information and classification results;
specifically, the result feedback module is responsible for feeding back the electrocardiographic rhythm information and the classification result to the access user, and displaying the diagnosis result on the android client interface.
Compared with the prior art, the application program for classifying the abnormal electrocardiographic rhythm based on the deep learning has the beneficial effects that:
(1) A method for segmenting an electrocardiographic region from a paper centroid electrical report image of a full-automatic end-to-end depth convolutional neural network is provided.
(2) An image-based auxiliary diagnosis technology for acquiring the electrocardiographic rhythm information and classifying abnormal multi-labels is realized.
(3) The method is used for assisting diagnosis, reduces the pressure of doctors, only presents the electrocardiographic report containing suspected rhythm abnormality to the doctors, ensures that the diagnosis is more targeted, eliminates repeated, monotonous and time-consuming matters, and improves the clinical diagnosis efficiency.
(4) The individual variability and subjectivity of clinical diagnosis are reduced, the diagnosis result is more objective, and the missing diagnosis rate and the misdiagnosis rate are reduced, so that the medical diagnosis level is improved. Since the diagnosis of a doctor is a subjective judgment process, the diagnosis is easy to be limited and influenced by the experience and knowledge level of the doctor, so that misdiagnosis is caused or certain image details are omitted, and the computer has great advantages in avoiding the errors and the defects.
Drawings
Fig. 1 is a block diagram of the overall system.
FIG. 2 is a flow chart of the system operation.
Fig. 3 is a flowchart of the electrocardiographic region extraction unit.
FIG. 4 is a diagram of a user functionality interface for an android application.
Detailed description of the preferred embodiments
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments.
Referring to fig. 1, fig. 1 is a block diagram of a system total module for acquiring cardiac rhythm information by taking a photo of an electrocardiogram, and the block diagram comprises an cardiac electric region extraction unit, an image processing unit, an cardiac rhythm information acquisition and abnormality classification unit and an android mobile phone application program functional unit, wherein the android mobile phone application program functional unit comprises an image reading module, an image uploading module and a result feedback module; the output end of the image reading module in the android application program functional unit is connected with the input end of the electrocardio region extraction unit, the output end of the electrocardio region extraction unit is connected with the input end of the image processing unit, the output end of the image processing unit is connected with the input end of the electrocardio rhythm information acquisition and abnormality classification unit, and the output end of the electrocardio rhythm information acquisition and abnormality classification unit is connected with the output end of the result feedback module in the android application program functional unit.
As shown in fig. 1, the image training set is derived from two parts, the first part of image is a data set of a 2018 physiological signal challenge race, electrocardiosignals of known related arrhythmia class labels are randomly selected, and are converted into electrocardiographic images with grids as the background, and after data enhancement operations such as rotation, scaling and the like are performed, the electrocardiographic images are superimposed in random positions of a plurality of real background images, so that the authenticity of image data is similar to that of electrocardiography shot by a mobile phone; the second partial image is an electrocardiogram report of real shooting, and the actual sample size of the data set is increased to improve the generalization capability of the model.
As shown in fig. 2, fig. 2 is a schematic diagram of a system operation flow for acquiring cardiac rhythm information by taking a photo of an electrocardiogram, a 12-lead electrocardiogram report shot by a user is acquired by an image reading module in an android mobile phone application program functional unit, irrelevant information such as personal information of a patient and diagnosis marks of a doctor is removed, and an electrocardiogram waveform area is extracted based on a target detection network and then transmitted to an image processing unit; preprocessing the extracted electrocardio region by adopting algorithms such as image denoising, image inclination correction, shadow removal, data enhancement and the like, and taking the preprocessed image as electrocardio rhythm information to acquire and input an abnormal classification unit; based on the convolutional neural network, acquiring electrocardiogram rhythm information, and identifying and classifying the electrocardiogram with abnormal rhythm; and determining an optimal model through the evaluation index and deploying the optimal model to a back-end server. When the user uses the android application to acquire the electrocardio rhythm information and diagnose abnormality, the mobile phone can upload the shot electrocardio report image to a server to perform abnormality classification, and one or more of seven classifications (normal sinus rhythm, atrial fibrillation, primary atrioventricular block, left bundle branch block, right bundle branch block, atrial premature beat and ventricular premature beat) are fed back to the mobile phone end of the user as diagnosis results.
As shown in fig. 2, the classification network model takes an electrocardiographic region image in a paper centroid electrical report as input and takes the probability that the image features conform to classification labels of each rhythm feature as output; and carrying out model test on the initial electrocardiographic rhythm abnormal classification neural network model based on the test set, judging whether an evaluation index meets a preset evaluation standard, and if so, obtaining a target electrocardiographic image rhythm abnormal classification model.
As shown in fig. 3, fig. 3 is a flowchart of an electrocardiographic region extraction unit. The electrocardiographic region extraction unit comprises the following steps: performing image preprocessing on the original electrocardio report picture to improve the image quality; manually marking the left upper coordinate and the right lower coordinate of the electrocardiographic region by using a marking tool to the preprocessed electrocardiographic report graph to obtain a position label; randomly dividing the collected electrocardio report pictures into a training set, a verification set and a test set according to a specific proportion; training and optimizing a YOLO target detection network based on the training set and the verification set by utilizing the preprocessed electrocardio report image and the position label, and setting a confidence threshold of the spatial position of an electrocardio area to obtain spatial position information; then slicing processing is carried out based on the spatial position information of the electrocardiographic region, and a downsampling feature map is obtained; obtaining an optimized electrocardio region extraction model; predicting the electrocardiographic report picture in the test set by using the electrocardiographic region extraction model based on the YOLO target detection network; and carrying out precision evaluation on the prediction result of the test set.
As shown in fig. 3, in the verification model accuracy, the evaluation index of the extraction accuracy of the evaluation area is the average accuracy mAP, and the calculation process is as follows: and calculating the intersection ratio of the predicted frame area and the actual frame area of the target detection network, wherein the value of the intersection ratio can reflect the fitting degree of the predicted frame and the actual frame. When the cross ratio is larger than the threshold value, judging that the predicted result is correct; calculating the average precision of all the electrocardio report images in the test set; mAP, the sum of the average accuracy of the test set divided by the total number of test set categories, is calculated.
As shown in fig. 4, fig. 4 is a diagram of an android application user function interface for acquiring cardiac rhythm information by taking an electrocardiogram. Firstly, a user firstly places an electrocardiograph report on a flat desktop with sufficient illumination, then places a photographing interface of the application program in a positive position for photographing, and needs to remove the influence of redundant information such as name, gender, age, report completion time or background sundries so as to highlight key information; then previewing on the mobile phone, if the display is fuzzy, indicating that the photographing cannot be correctly focused, and deleting the re-photographing; the user can also select from the album that an electrocardiogram photo has been taken; and finally uploading the shot electrocardio report image to an application program. And the application program respectively passes the electrocardio report through the electrocardio region extraction unit, the image processing unit and the electrocardio rhythm information acquisition and abnormality classification unit through the image uploading module, finally obtains the probability of the rhythm classification result, and feeds back the final diagnosis result to the user through an android application program interface.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
It should be noted that, the term "first\second\third" related to the embodiment of the present invention is merely to distinguish similar objects, and does not represent a specific order for the objects, it is to be understood that "first\second\third" may interchange a specific order or sequence where allowed. It is to be understood that the "first\second\third" distinguishing aspects may be interchanged where appropriate to enable embodiments of the invention described herein to be implemented in sequences other than those illustrated or described.
The terms "comprising" and "having" and any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, apparatus, article, or device that comprises a list of steps or modules is not limited to the particular steps or modules listed and may optionally include additional steps or modules not listed or inherent to such process, method, article, or device.
The above examples illustrate only a few embodiments of the invention, which are described in detail and are not to be construed as limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.
Claims (9)
1. The system for acquiring the electrocardio rhythm information through photographing an electrocardiogram is characterized by comprising an electrocardio area extraction unit, an image processing unit, an electrocardio rhythm information acquisition and abnormality classification unit and an android mobile phone application program functional unit, wherein the android mobile phone application program functional unit comprises an image reading module, an image uploading module and a result feedback module; the output end of the image reading module in the android application program functional unit is connected with the input end of the electrocardio region extraction unit, the output end of the electrocardio region extraction unit is connected with the input end of the image processing unit, the output end of the image processing unit is connected with the input end of the electrocardio rhythm information acquisition and abnormality classification unit, and the output end of the electrocardio rhythm information acquisition and abnormality classification unit is connected with the output end of the result feedback module in the android application program functional unit;
the electrocardio region extraction unit acquires a 12-lead electrocardio report shot by a user through an image reading module in the android mobile phone application program functional unit, removes personal information of a patient and irrelevant information of a doctor diagnosis mark, extracts an electrocardio waveform region based on a target detection network and transmits the electrocardio waveform region to the image processing unit;
the image processing unit adopts an image denoising, image inclination correction, shadow removal and data enhancement algorithm to preprocess the extracted electrocardio region, and the preprocessed image is used as electrocardio rhythm information to be acquired and input by the abnormality classification unit;
the electrocardiographic rhythm information acquisition and abnormality classification unit acquires electrocardiographic rhythm information based on a convolutional neural network and classifies rhythm abnormality of the electrocardiographic rhythm information;
the android mobile phone application program functional unit comprises an image reading module, an image uploading module and a result feedback module; the image reading module is responsible for shooting, receiving and reading an electrocardio report of a user; the image uploading module is responsible for automatically transmitting the image file to the back-end server; the result feedback module is responsible for feeding back the classification result of the electrocardiographic rhythm information to the access user.
2. A system for acquiring cardiac rhythm information by taking a picture of an electrocardiogram according to claim 1 wherein said cardiac region extraction unit comprises the steps of:
(1) Performing image preprocessing on the original electrocardio report picture to improve the image quality; manually marking the left upper coordinate and the right lower coordinate of the electrocardiographic region by using a marking tool to the preprocessed electrocardiographic report graph to obtain a position label; randomly dividing the collected electrocardio report pictures into a training set, a verification set and a test set according to a specific proportion;
(2) Training and optimizing a YOLO target detection network based on the training set and the verification set by utilizing the preprocessed electrocardio report image and the position label, and setting a confidence threshold of the spatial position of an electrocardio area to obtain spatial position information; then slicing based on the spatial position information of the electrocardio region to obtain a downsampling feature map and an optimized electrocardio region extraction model;
(3) Predicting the electrocardiographic report picture in the test set by using the electrocardiographic region extraction model based on the YOLO target detection network;
(4) And carrying out precision evaluation on the prediction result of the test set.
3. The system for acquiring cardiac rhythm information by taking a picture of an electrocardiogram according to claim 2, wherein the training optimization of the YOLO target detection network using the training set and the verification set in step (2) comprises:
carrying out normalization processing on an electrocardiographic report image in a training set, inputting the electrocardiographic report image into a backstone part of a network to obtain three feature images with different scales, namely a feature image with the smallest receptive field and the largest scale, a feature image with the medium scale and the largest receptive field and the smallest scale, inputting the feature image into a head part of the network, carrying out up-sampling and feature fusion on the feature image, and simultaneously, obtaining tensor data under three different scales through confidence threshold filtering and non-maximum suppression filtering; and calculating a loss value between the tensor data and the label value, calculating a network gradient through back propagation, updating a weight value, finally optimizing a network by using a verification set, obtaining a group of hyper-parameter values with minimum loss, and finally obtaining an electrocardiographic region extraction model based on the YOLO target detection network.
4. The system for obtaining cardiac rhythm information by taking a picture of an electrocardiogram according to claim 2 wherein the accuracy evaluation index in said step (4) is an average accuracy mAP, which is calculated as follows:
(1) Calculating the intersection ratio of the predicted frame area and the actual frame area of the target detection network, wherein the value reflects the fitting degree of the predicted frame and the actual frame, and judging that the predicted result is correct when the intersection ratio is larger than a threshold value;
(2) Calculating the average precision of all the electrocardio report images in the test set;
(3) mAP, the sum of the average accuracy of the test set divided by the total number of test set categories, is calculated.
5. The system for acquiring cardiac rhythm information by taking a picture of an electrocardiogram according to claim 1 wherein said image processing unit comprises image denoising, tilt correction, shadow removal, data enhancement; the image inclination correction adopts a Hough algorithm to identify a background grid line segment of an electrocardio region, and then calculates an image inclination angle to correct the image inclination angle.
6. The system for acquiring cardiac rhythm information by photographing an electrocardiogram according to claim 1 wherein said cardiac rhythm information acquisition and anomaly classification unit acquires and identifies and classifies cardiac rhythm information based on a convolutional neural network, comprising:
s1: constructing an electrocardiogram data set under different rhythm classification labels according to the electrocardio area images and the corresponding labels output by the image processing unit;
s2: preprocessing an electrocardiographic image in the electrocardiographic data set, and dividing the electrocardiographic data set obtained by the image processing unit into a training set, a verification set and a test set according to a preset proportion;
s3: constructing a convolutional neural network model, and training and optimizing the convolutional neural network model based on the training set and the verification set to obtain an initial electrocardiographic rhythm abnormality classification neural network model; the electrocardiographic rhythm abnormality classification neural network model takes an electrocardiographic region image in a paper centroid electricity report as input and takes the probability that the image features accord with each rhythm feature classification label as output;
s4: and performing model test on the initial electrocardiographic rhythm abnormal classification neural network model based on the test set, and obtaining a target electrocardiographic image rhythm abnormal classification model passing the test by combining with an evaluation index.
7. The system for acquiring cardiac rhythm information by taking a picture of an electrocardiogram according to claim 6 wherein said convolutional neural network model mentioned in step S3 is capable of accurately classifying cardiac rhythm abnormalities, the model input is a vector of dimension 1 x 7 corresponding to a specific type of cardiac rhythm therein, each cardiac report may correspond to a plurality of labels because the patient may have a plurality of cardiac rhythm abnormalities, and the output vector of the model may contain more than one non-zero element;
the convolutional neural network model is divided into an input end, a backbone network and an output end; inputting tensors with the dimension of 675 multiplied by 1450 multiplied by 3, wherein the backbone network has eight layers, the first seven layers are convolution layers, and in each convolution layer, the size of a convolution kernel is fixed to be 5 multiplied by 5, and the step length is 2; after each convolution layer, adding a nonlinear activation function with batch normalization; then two full connection blocks, wherein each full connection block is internally provided with a ReLu nonlinear activation function and two full connection layers; in order to prevent the occurrence of overfitting, setting the discarding probability to be 0.6 in the full connection layer; the output layer contains 7 Sigmoid functions, returning the probability distribution of the corresponding 7 output categories, namely normal sinus rhythm and six rhythm anomaly types.
8. The system for acquiring cardiac rhythm information by taking a picture of an electrocardiogram according to claim 6 wherein the training optimization procedure of step S3 specifically comprises:
s31: taking an electrocardiographic region image obtained by an image processing unit as input, and taking the prediction probability that the electrocardiographic image feature accords with each rhythm feature classification label as output to construct a convolutional neural network model;
s32: the characteristics of the electrocardiogram training set are obtained by a forward propagation algorithm, and a prediction result is obtained through the convolution network model;
s33: calculating a loss function, and obtaining a first loss value by the difference between a result obtained by the electrocardiogram training set through the neural network and a real label value;
s34: calculating gradients in the convolutional neural network model through a back propagation algorithm and an optimization algorithm, and updating weights;
s35: the verification set is used for optimizing the convolutional neural network, the super parameters are adjusted, which group of super parameters have the best performance according to the performance of the verification set, and a second loss value is obtained by the difference between the result obtained by the electrocardiogram verification set through the neural network and the real label;
s36: repeatedly executing the five steps until the model converges, and simultaneously drawing a change curve of the loss value by using a visualization method; the first loss value is used for updating the weight parameter of the convolutional neural network, and the second loss value is used for storing a final network model to classify the electrocardiographic rhythm type;
the loss function in the step S33 is specifically a multi-class cross entropy loss function, and the ratio of the maximum number of samples in all the electrocardiographic rhythm types to the number of samples is used as the weight value in the calculation; selecting the self-adaptive moment estimation as an optimizer, and updating the weight value of the convolutional neural network model in the back propagation process; for the final convolutional neural network model, the weights of the final convolutional neural network model are updated continuously, and finally the maximum iteration times and the smaller loss value are achieved.
9. The system for acquiring the cardiac rhythm information by taking a picture of an electrocardiogram according to claim 1, wherein the android mobile phone application program functional unit comprises an image reading module, an image uploading module and a result feedback module;
the image reading module is responsible for shooting, receiving and reading an electrocardio report of a user; the user shoots by using a mobile phone or other mobile equipment to acquire an image of a paper electrocardiograph report, firstly, the user places the electrocardiograph report on a flat desktop with sufficient illumination, and then the shooting interface of the application program is placed at a positive position to shoot; then previewing on the mobile phone, if the display is fuzzy, indicating that the photographing cannot be correctly focused, and deleting the re-photographing; the user can also select from the album that an electrocardiogram photo has been taken; finally uploading the shot electrocardio report image to an application program;
the image uploading module is responsible for automatically transmitting an electrocardiographic report image sent by a user side to a rear-end server to obtain electrocardiographic rhythm information and classification results;
the result feedback module is responsible for feeding back the electrocardio rhythm information and the classification result to the access user and displaying the diagnosis result on the android client interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310006237.XA CN116012568A (en) | 2023-01-04 | 2023-01-04 | System for acquiring cardiac rhythm information through photographing electrocardiogram |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310006237.XA CN116012568A (en) | 2023-01-04 | 2023-01-04 | System for acquiring cardiac rhythm information through photographing electrocardiogram |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116012568A true CN116012568A (en) | 2023-04-25 |
Family
ID=86035209
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310006237.XA Pending CN116012568A (en) | 2023-01-04 | 2023-01-04 | System for acquiring cardiac rhythm information through photographing electrocardiogram |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116012568A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116456184A (en) * | 2023-06-19 | 2023-07-18 | 北京博点智合科技有限公司 | Method, device, equipment and storage medium for adjusting camera mounting point positions |
| CN116864140A (en) * | 2023-09-05 | 2023-10-10 | 天津市胸科医院 | Intracardiac branch of academic or vocational study postoperative care monitoring data processing method and system thereof |
-
2023
- 2023-01-04 CN CN202310006237.XA patent/CN116012568A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116456184A (en) * | 2023-06-19 | 2023-07-18 | 北京博点智合科技有限公司 | Method, device, equipment and storage medium for adjusting camera mounting point positions |
| CN116456184B (en) * | 2023-06-19 | 2023-09-08 | 北京博点智合科技有限公司 | Method, device, equipment and storage medium for adjusting camera mounting point positions |
| CN116864140A (en) * | 2023-09-05 | 2023-10-10 | 天津市胸科医院 | Intracardiac branch of academic or vocational study postoperative care monitoring data processing method and system thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109886273B (en) | CMR image segmentation and classification system | |
| US11564612B2 (en) | Automatic recognition and classification method for electrocardiogram heartbeat based on artificial intelligence | |
| Li et al. | Accurate retinal vessel segmentation in color fundus images via fully attention-based networks | |
| CN116012568A (en) | System for acquiring cardiac rhythm information through photographing electrocardiogram | |
| Liu et al. | A framework of wound segmentation based on deep convolutional networks | |
| CN109009102B (en) | Electroencephalogram deep learning-based auxiliary diagnosis method and system | |
| WO2021071688A1 (en) | Systems and methods for reduced lead electrocardiogram diagnosis using deep neural networks and rule-based systems | |
| CN117457229B (en) | Anesthesia depth monitoring system and method based on artificial intelligence | |
| CN111553892A (en) | Lung nodule segmentation calculation method, device and system based on deep learning | |
| CN113080996B (en) | Electrocardiogram analysis method and device based on target detection | |
| CN111126350B (en) | Method and device for generating heart beat classification result | |
| CN113128585B (en) | Deep neural network based multi-size convolution kernel method for realizing electrocardiographic abnormality detection and classification | |
| Khan et al. | Electrocardiogram heartbeat classification using convolutional neural networks for the detection of cardiac Arrhythmia | |
| CN116322479A (en) | Electrocardiogram processing system for detecting and/or predicting cardiac events | |
| CN110459303A (en) | Medical imaging abnormal detector based on depth migration | |
| CN113288157A (en) | Arrhythmia classification method based on depth separable convolution and improved loss function | |
| CN115530788A (en) | Arrhythmia classification method based on self-attention mechanism | |
| CN112869753A (en) | Analysis method, equipment, medium and electrocardiograph for QRST waveform of electrocardiogram | |
| WO2021071646A1 (en) | Systems and methods for electrocardiogram diagnosis using deep neural networks and rule-based systems | |
| Ghorakavi | TBNet: pulmonary tuberculosis diagnosing system using deep neural networks | |
| CN116864140A (en) | Intracardiac branch of academic or vocational study postoperative care monitoring data processing method and system thereof | |
| CN112634221B (en) | Cornea hierarchy identification and lesion positioning method and system based on images and depth | |
| Rahman et al. | Deep learning-based left ventricular ejection fraction estimation from echocardiographic videos | |
| CN116230172A (en) | System for acquiring electrocardio ST segment information through shooting electrocardiogram | |
| CN113643263B (en) | Identification method and system for upper limb bone positioning and forearm bone fusion deformity |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |