CN113537407A - Image data evaluation processing method and device based on machine learning - Google Patents
Image data evaluation processing method and device based on machine learning Download PDFInfo
- Publication number
- CN113537407A CN113537407A CN202111017522.9A CN202111017522A CN113537407A CN 113537407 A CN113537407 A CN 113537407A CN 202111017522 A CN202111017522 A CN 202111017522A CN 113537407 A CN113537407 A CN 113537407A
- Authority
- CN
- China
- Prior art keywords
- image data
- processing
- image
- evaluation model
- target image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Epidemiology (AREA)
- Primary Health Care (AREA)
- Public Health (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The application discloses an image data evaluation processing method and device based on machine learning, which relate to the technical field of artificial intelligence and data medical treatment and mainly aim to improve the problem that when the identification of the processed image data is difficult, the processing error or the original image distortion cannot be judged; meanwhile, in the process of subjective evaluation, the image data cannot be effectively evaluated due to poor stability and portability. The method comprises the following steps: acquiring target image data for which image data processing has been completed in response to an image output instruction; performing prediction processing on target image data according to the trained image evaluation model to obtain a prediction evaluation result; if the prediction evaluation result is an inferior image, acquiring a data processing tracking path of the target image data, and extracting a processing node in the data processing tracking path; outputting the target image data, the processing node, and the original image data matched with the target image data.
Description
Technical Field
The present application relates to the field of artificial intelligence and data medical technology, and in particular, to a method and an apparatus for evaluating and processing image data based on machine learning.
Background
With the rapid development of digital medical technology, the intelligent medical system has occupied a place in the medical field through the support of an artificial intelligence algorithm which is too hard per se, and particularly needs a large amount of image data for data support in the medical diagnosis process, so that a doctor can judge based on images, even if the doctor subjectively evaluates the content in the image data and the processing result of the image data.
At present, a single machine learning algorithm such as image segmentation or classification is usually adopted for image processing of image data in an existing intelligent medical system, and the obtained image processing result is difficult for a doctor to identify the image data in the subjective evaluation process, that is, the situation that image distortion and the like are processing errors generated in the processing process or the situation that the image data is distorted and the like cannot be accurately identified.
Disclosure of Invention
In view of the above, the present application provides a method and an apparatus for evaluating and processing image data based on machine learning, and mainly aims to improve the problem that it is impossible to determine whether a processing error or an original image is distorted when the identification of the processed image data is difficult; meanwhile, in the process of subjective evaluation, the image data cannot be effectively evaluated due to poor stability and portability.
According to one aspect of the application, an image data evaluation processing method based on machine learning is provided, and comprises the following steps:
acquiring target image data for which image data processing has been completed in response to an image output instruction;
performing prediction processing on the target image data according to the trained image evaluation model to obtain a prediction evaluation result, wherein the image evaluation model is obtained by performing combined training according to a weight ratio based on an image impurity evaluation model and an image quality evaluation model;
if the prediction evaluation result is an inferior image, acquiring a data processing tracking path of the target image data, and extracting a processing node in the data processing tracking path, wherein the data processing tracking path is a path for recording a storage processing flow when the original image data is processed by image data;
and outputting the target image data, the processing node and the original image data matched with the target image data.
Preferably, before the target image data is subjected to prediction processing according to the trained image evaluation model to obtain a prediction evaluation result, the method further includes:
constructing an image impurity evaluation model based on a preset unified network, and training the image impurity evaluation model based on an impurity image training sample set;
constructing an image quality evaluation model based on a preset depth neural network, and training the image instruction evaluation model based on a quality image training sample set, wherein a convolution kernel structure in the image quality evaluation model is replaced by a residual error network structure, and a full connection layer is replaced by a full convolution layer;
and distributing model weights for the trained image impurity evaluation model and the trained image quality evaluation model, and combining the image impurity evaluation model and the image quality evaluation model according to the model weights to obtain an image evaluation model.
Preferably, the method further comprises:
acquiring step information that the target image data has finished image data processing;
analyzing the processing parameters in the step information, wherein the processing parameters are used for representing the abnormal probability of each image data processing step;
and determining two model weights matched with the processing parameters according to the corresponding relation of preset processing weights, and distributing the image impurity evaluation model and the image quality evaluation model based on the two model weights.
Preferably, after acquiring the target image data for which the image data processing has been completed, the method further includes:
normalizing the target image data, and carrying out zooming and filtering operations on the target image data after normalization according to the image size of the original image data;
and subtracting the original image data from the target image data subjected to the zooming and filtering operations to obtain the target image data of the high-frequency component.
Preferably, before the acquiring target image data for which the image data processing has been completed, the method further includes:
when detecting image data processing of the original image data, marking processing nodes for each processing flow in the image data processing;
and recording a storage path and a calling path generated in the processing flow, and generating a data processing tracking path according to the processing node, the storage path and the calling path.
Preferably, the method further comprises:
if the medical auxiliary operation on the original image data is detected, reporting to the processing node to indicate that the image data processing step is updated;
and if the medical auxiliary operation on the original image data is not detected, updating the original image data to a model training sample set.
Preferably, the method further comprises:
if the prediction evaluation result is a high-quality image, counting a processing mode, a processing parameter and a processing characteristic in the image data processing;
and reporting the counted processing mode, processing parameters and processing characteristics according to a preset time interval, and processing the image data belonging to the poor-quality image based on the processing mode, the processing parameters and the processing characteristics.
According to another aspect of the present application, there is provided an image data evaluation processing apparatus based on machine learning, including:
a first acquisition module for acquiring target image data for which image data processing has been completed in response to an image output instruction;
the prediction module is used for performing prediction processing on the target image data according to the trained image evaluation model to obtain a prediction evaluation result, and the image evaluation model is obtained by performing combined training according to a weight proportion based on an image impurity evaluation model and an image quality evaluation model;
the extraction module is used for acquiring a data processing tracking path of the target image data and extracting a processing node in the data processing tracking path if the prediction evaluation result is an inferior image, wherein the data processing tracking path is a path for recording a storage processing flow when original image data is processed by image data;
and the output module is used for outputting the target image data, the processing node and the original image data matched with the target image data.
Preferably, before the prediction module, the apparatus further includes:
the construction module is used for constructing an image impurity evaluation model based on a preset unified network and training the image impurity evaluation model based on an impurity image training sample set;
the construction module is further used for constructing an image quality evaluation model based on a preset deep neural network and training the image instruction evaluation model based on a quality image training sample set, wherein a convolution kernel structure in the image quality evaluation model is replaced by a residual error network structure, and a full connection layer is replaced by a full convolution layer;
and the distribution module is used for distributing model weights for the trained image impurity evaluation model and the trained image quality evaluation model, and combining the image impurity evaluation model and the image quality evaluation model according to the model weights to obtain the image evaluation model.
Preferably, the apparatus further comprises:
the second acquisition module is used for acquiring step information that the target image data completes image data processing;
the analysis module is used for analyzing the processing parameters in the step information, and the processing parameters are used for representing the abnormal probability of each image data processing step;
and the determining module is used for determining two model weights matched with the processing parameters according to the corresponding relation of preset processing weights, and distributing the image impurity evaluation model and the image quality evaluation model based on the two model weights.
Preferably, after the first obtaining module, the apparatus further comprises:
the normalization module is used for performing normalization processing on the target image data and performing scaling and filtering operations on the target image data after normalization processing according to the image size of the original image data;
and the subtraction module is used for subtracting the original image data from the target image data after the zooming and filtering operations are finished to obtain the target image data of the high-frequency component.
Preferably, before the first obtaining module, the apparatus further includes:
the marking module is used for marking processing nodes for each processing flow in the image data processing when the image data processing of the original image data is detected;
and the recording module is used for recording the storage path and the calling path generated in the processing flow and generating a data processing tracking path according to the processing node, the storage path and the calling path.
Preferably, the apparatus further comprises:
the first reporting module is used for reporting the processing node to indicate to update the image data processing step if medical auxiliary operation on the original image data is detected;
and the updating module is used for updating the original image data to a model training sample set if medical auxiliary operation on the original image data is not detected.
Preferably, the apparatus further comprises:
the statistical module is used for counting the processing mode, the processing parameters and the processing characteristics in the image data processing if the prediction evaluation result is a high-quality image;
and the second reporting module is used for reporting the counted processing mode, processing parameters and processing characteristics according to a preset time interval so as to perform image data processing on the image data belonging to the poor-quality image based on the processing mode, the processing parameters and the processing characteristics.
According to still another aspect of the present application, a storage medium is provided, and the storage medium stores at least one executable instruction, which causes a processor to execute operations corresponding to the image data evaluation processing method based on machine learning as described above.
According to yet another aspect of the present application, there is provided a computer device including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the image data evaluation processing method based on machine learning.
By means of the technical scheme, the technical scheme provided by the embodiment of the application at least has the following advantages:
the application provides an image data evaluation processing method and device based on machine learning, and the method comprises the steps of firstly responding to an image output instruction, and acquiring target image data which is processed by the image data; performing prediction processing on the target image data according to the trained image evaluation model to obtain a prediction evaluation result, wherein the image evaluation model is obtained by performing combined training according to a weight ratio based on an image impurity evaluation model and an image quality evaluation model; if the prediction evaluation result is an inferior image, acquiring a data processing tracking path of the target image data, and extracting a processing node in the data processing tracking path, wherein the data processing tracking path is a path for recording a storage processing flow when the original image data is processed by image data; and outputting the target image data, the processing node and the original image data matched with the target image data. Compared with the prior art, the target image data is subjected to prediction processing through the image evaluation model obtained by performing combined training on the image impurity evaluation model and the image quality evaluation model according to the weight proportion, and the tracking path of data processing is recorded, so that when the image data is difficult to identify, the reason of the problem can be inquired according to the tracking path, the processing error or the image data distortion can be accurately identified, and meanwhile, the effectiveness of image data evaluation is improved.
The foregoing description is only an overview of the technical solutions of the present application, and the present application can be implemented according to the content of the description in order to make the technical means of the present application more clearly understood, and the following detailed description of the present application is given in order to make the above and other objects, features, and advantages of the present application more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the application. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 is a flowchart illustrating an image data evaluation processing method based on machine learning according to an embodiment of the present application;
FIG. 2 is a diagram illustrating a DIQA network structure provided by an embodiment of the present application;
FIG. 3 shows a schematic diagram of a residual structure provided by an embodiment of the present application;
FIG. 4 is a block diagram illustrating a machine learning-based image data evaluation processing apparatus according to an embodiment of the present application;
fig. 5 shows a schematic structural diagram of a computer device provided in an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The embodiment of the application can acquire and process related data based on an artificial intelligence technology. Among them, Artificial Intelligence (AI) is a theory, method, technique and application system that simulates, extends and expands human Intelligence using a digital computer or a machine controlled by a digital computer, senses the environment, acquires knowledge and uses the knowledge to obtain the best result.
The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.
Based on this, in an embodiment, as shown in fig. 1, a method for evaluating and processing image data based on machine learning is provided, which is described by taking the method applied to computer devices such as a server as an example, where the server may be an independent server, or may be a cloud server that provides basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, Network service, cloud communication, middleware service, domain name service, security service, Content Delivery Network (CDN), and big data and artificial intelligence platform, such as an intelligent medical system and a digital medical platform. The method comprises the following steps:
101. in response to the image output instruction, target image data for which image data processing has been completed is acquired.
In the embodiment of the application, the image output instruction is an instruction triggered by a viewing user (such as a doctor) based on the result of calling image data by a detection user (such as a patient) who has finished shooting or detection through an intelligent medical system, and the output image data is original image data and target image data processed by the image data. Therefore, the smart medical system as the current execution end first acquires the original image data captured by the detection user, and then performs the processing steps of the image data, including but not limited to image labeling, image segmentation, and the like. In order to ensure the accuracy of output image data, when an output instruction of image data is received, original image data of a detected user and target image data subjected to image data processing are acquired.
Since the processing procedure of the image data affects the content of the original image data to a different degree, it is necessary to evaluate the target image data processing.
102. And performing prediction processing on the target image data according to the trained image evaluation model to obtain a prediction evaluation result.
In the embodiment of the application, the image evaluation model is obtained by performing combined training according to a weight ratio based on the image impurity evaluation model and the image quality evaluation model, namely the image evaluation model is formed by combining the image impurity evaluation model and the image quality evaluation model. The image impurity evaluation model is used for detecting whether non-human tissue objects, such as CT beds, clothes buttons and the like, exist in the image data, and if the non-human tissue objects exist, the image impurity evaluation model indicates that certain operation problems exist in the image data. The image quality evaluation model is used for evaluating the quality of the image data, for example, whether the definition is suitable for being checked by a user, and when the two models obtain the prediction rating result, a final result can be determined according to the weight ratio to serve as a total prediction evaluation result.
The weighting ratio may be determined according to actual evaluation results of the two models, for example, the evaluation result obtained by the image impurity evaluation model has a high abnormal probability, and the weighting of the image impurity evaluation result may be reduced accordingly in order to ensure the validity of the image data evaluation result.
103. And if the prediction evaluation result is an inferior image, acquiring a data processing tracking path of the target image data, and extracting a processing node in the data processing tracking path.
In the embodiment of the present application, the data processing trace path is a path for recording a processing flow stored when the original image data is subjected to image data processing, that is, a path for marking contents such as a code called by each image data processing step and image data processing when the original image data is subjected to image preprocessing, image segmentation processing, and the like, so as to output when the result is determined to be inferior image data. The processing nodes are nodes of each processing mode adopted for processing the original image data so as to inform a user or a developer of which processing nodes the inferior image data is obtained. For example, the original image data a is subjected to normalization processing, segmentation processing and label marking processing to obtain target image data B, a series of processing performed above is a data processing tracking path of the target image data B, and the normalization processing, the segmentation processing and the label marking processing are processing nodes.
104. Outputting the target image data, the processing node, and the original image data matched with the target image data.
In the embodiment of the present application, in order to avoid misdiagnosis and the like caused by diagnosis based on poor target image data, after the target image data is evaluated as poor image data, original image data, processing nodes and the target image data are output together for the convenience of users or developers, and each output content is marked.
It should be noted that, the user can trace the source of the image data based on the output result, and the developer can find out the steps that are liable to cause the processing error of the image data based on the output result, and perform the program optimization based on the steps.
For further explanation and limitation, in this embodiment of the present application, before performing prediction processing on target image data according to a trained image evaluation model to obtain a prediction evaluation result, the method of this embodiment further includes: constructing an image impurity evaluation model based on a preset unified network, and training the image impurity evaluation model based on an impurity image training sample set; constructing an image quality evaluation model based on a preset depth neural network, and training an image instruction evaluation model based on a quality image training sample set, wherein a convolution kernel structure in the image quality evaluation model is replaced by a residual error network structure, and a full connection layer is replaced by a full convolution layer; and distributing model weights for the trained image impurity evaluation model and the trained image quality evaluation model, and combining the image impurity evaluation model and the image quality evaluation model according to the model weights to obtain the image evaluation model.
Illustratively, an Image Quality evaluation model is constructed Based on a preset Deep neural network, such as a natural Image evaluation model DIQA (Deep CNN-Based blank Image Quality Predictor), the network structure is as shown in fig. 2, a convolution kernel structure in the Image Quality evaluation model is replaced by a residual network structure, and a full connection layer is replaced by a full convolution layer. Specifically, (1) the convolution kernel structure of DIQA is replaced: the convolution kernel structure of 3 × 3 is replaced with residual structure, as shown in fig. 3, each residual structure contains 2 convolution kernel structures of 3 × 3, each convolution kernel structure is 64 feature maps (feature maps), and the second convolution kernel structure employs a dilated convolution (dilated convolution) with a dilation rate of 2. (2) In order to adapt the network to input images of different sizes, the fully connected layers (FC1 and FC2) are replaced with fully convolutional layers, the most one layer of the active function is replaced with the Linear active function Linear. (3) And the characteristics of a deeper layer are extracted by utilizing a residual error structure and an expansion convolution without two-stage training. The output of the error map end can be omitted, and only the objective score end is reserved. The loss function is replaced with a mean-square error (MSE).
Further, training a model by using a quality image training sample set according to input image data and outputting a score (between 0 and 0.5) with a label of subjective evaluation; and verifying the model by using the quality image verification sample set according to the input image and the output result of the quality image verification sample set as the evaluation score (between 0 and 0.5), and finally obtaining the trained image quality evaluation model.
It should be noted that the training mode of the image impurity evaluation model is the same as the training mode of the image quality evaluation model, and details are not repeated here. And verifying the image impurity evaluation model by using the impurity image verification sample set. Specifically, (1) inputting a sample image of a user into a preset unified network (such as a YOLO V5 model) one by one or one by one image block (selected according to specific needs, for example, if the image is large, the image block can be cut into image blocks for detection); (2) the unified network model is operated (different types of models need to be trained by adopting different image data) to carry out image data impurity evaluation processing, the types of all impurities are obtained, the specific type quantity and the type can be set according to specific requirements, and four coordinates are sequentially as follows: upper left, upper right, lower left, information for indicating the position of the impurity; (3) if one or more impurities exist in the image data, the image is divided into 0 points, and if no impurities exist, the evaluation score can be 0.5 points.
Further, in order to implement effective and accurate weighted fusion of the two models, model weights may be assigned to the trained image impurity evaluation model and the trained image quality evaluation model, and the image impurity evaluation model and the trained image quality evaluation model are combined according to the model weights to obtain an image evaluation model, which specifically includes: acquiring step information that the target image data has finished image data processing; analyzing the processing parameters in the step information, determining two model weights matched with the processing parameters according to the corresponding relation of the preset processing weights, and distributing the model weights based on the two model weights as an image impurity evaluation model and an image quality evaluation model.
The step information of image data processing includes, but is not limited to, image segmentation, image labels, etc., and different image data processing steps are pre-configured with corresponding processing parameters as identifiers of unique corresponding processing steps. The processing parameters are used to characterize the anomaly probability of each image data processing step. In the embodiment of the present application, a processing weight corresponding relationship may also be preconfigured, where the relationship describes that weights corresponding to different processing parameters are configured for two models, and the two weights may be the same or different, and may specifically be determined according to actual evaluation results of the two models. For example, the processing parameter is determined to be 2 according to the image segmentation processing step, and based on the processing weight value corresponding relation, the weight of 2 matching is determined to be 0.3 of the image impurity evaluation model and 0.7 of the image quality evaluation model, so that the weight summation between the two models is performed to obtain the final result.
It should be noted that, in the embodiment of the present application, this step may be placed in the model training process, or may be placed after the model training, so that the two models are subjected to weighted summation. For example, the final evaluation rule may be: more than 0.8 point is a high-quality image, which can be directly output to a user, exemplarily used as a basis for disease diagnosis; less than 0.2 points are low quality images, and the detection user (such as a patient) needs to redo the related examination; the user (such as a doctor) needs to check to determine whether the diagnosis standard is met or not, and the division can be performed more finely according to the requirement, which is not specifically limited in the embodiment of the present application.
In this embodiment of the present application, further, after acquiring target image data for which image data processing has been completed, the method of this embodiment further includes: normalizing the target image data, and carrying out scaling and filtering operations on the target image data after normalization according to the image size of the original image data; and subtracting the original image data from the target image data subjected to the zooming and filtering operations to obtain the target image data of the high-frequency component.
The purpose of normalizing the target image data is to obtain a standard image with the same form so as to perform other operations such as scaling, filtering, subtracting and the like on the image subsequently, for example, the target image data is subjected to 0-1 normalization processing, and the range of image pixel values is limited between 0 and 1, so that the purpose of accelerating model training and reasoning can be achieved. In the embodiment of the present application, preferably, the target image data is reduced to one fourth of the original image and then enlarged to the original size, so that during the process of reducing the target image data, the purpose of enhancing the smoothness and definition of the image can be achieved, and then the target image data is enlarged, so that the visibility of the pixels constituting the image will become higher. Further, the original image data is subtracted after the gaussian low-pass filtering to extract a high-frequency component of the target image data.
It should be noted that the low frequency component represents a region of the image data where the brightness or gray value changes slowly, i.e. a large flat region of the image, and describes a main portion of the image, which is mainly used for a comprehensive measure of the intensity of the whole image data. The high frequency components correspond to portions of the image data where the change is severe, that is, edges (contours) or noise and detailed portions of the image data, and are mainly used for measurement of the edges and contours of the image data. It can be understood that the image deformation does not substantially affect the low frequency components thereof, and the human visual system is less sensitive to changes of the low frequency components than the high frequency components, and extracting the high frequency components of the target image data helps to improve the noise sensitivity of the scoring module.
In order to determine whether target image data is usable or not by judging the cause of the poor quality when the target image data is evaluated as a poor quality image. Preferably, the reason may be queried by tracking each step processing operation performed on the original image, and in the embodiment of the present application, before acquiring the target image data of which the image data processing is completed, the method of the embodiment further includes: when detecting image data processing of original image data, marking processing nodes for each processing flow in the image data processing; and recording a storage path and a calling path generated in the processing flow, and generating a data processing tracking path according to the processing node, the storage path and the calling path.
The processing nodes are processing operations performed on the image data, such as normalization processing, segmentation processing and the like; the storage path is used for characterizing and tracking a path which is traced to the processing flow; the call path is used for characterizing the path which is passed by the call processing flow. Specifically, in the process of image data processing of an original image, each processing node in the processing flow is marked, a storage path and a calling path generated in the processing flow are recorded, and a data processing tracking path is generated according to the storage path and the calling path, so that after the image is determined to be poor, the tracking path and the processing node are obtained and output, and developers or users can judge whether the image is available. It is understood that if the prediction evaluation result is high-quality image data, the prediction evaluation result can be directly output for the user to refer to.
Further, the method of the embodiment of the present application further includes: if the medical auxiliary operation on the original image data is detected, reporting to a processing node to indicate that the image data processing step is updated; and if the medical auxiliary operation on the original image data is not detected, updating the original image data to the model training sample set.
The medical auxiliary operation may be an operation of calling original image data or calculating the image scale size of different image areas in an image, and the like, and the operation may be triggered by a user to determine the size of an organ, or may be performed by a developer to perform a secondary operation to detect whether the prediction result is accurate, and perform program optimization based on the detection result. If the medical auxiliary operation is detected, the user selects the original image data as the auxiliary image for the patient, so that errors occur in the image processing steps of the original image, and each processing node is reported for updating detection. In contrast, if the medical assistance operation on the original image data is not detected, it is indicated that the target image data is not selected by the user because the original image data itself has poor quality and is not related to the image processing step, and therefore, the target image data can be trained as a poor image label in the model training sample set.
In order to improve the processing quality of the image data, the method according to the embodiment of the present application further includes: if the prediction evaluation result is a high-quality image, counting the processing mode, the processing parameters and the processing characteristics in the image data processing; and reporting the counted processing mode, processing parameter and processing characteristic according to a preset time interval, and processing the image data belonging to the poor-quality image based on the processing mode, the processing parameter and the processing characteristic.
The processing mode includes but is not limited to normalization processing, segmentation processing, label labeling processing and other operations; the processing parameters are used for representing the abnormal probability of each image data processing step; processing characteristics include, but are not limited to, interval time, image type, etc. In order to improve the processing quality of the image data, the processing mode, the processing parameter and the processing characteristic of the high-quality image after statistics can be reported according to a preset time interval, and the image data of the poor-quality image is subjected to data processing according to the processing mode, the processing parameter and the processing characteristic of the high-quality image.
The application provides an image data evaluation processing method based on machine learning, which comprises the steps of responding to an image output instruction, and acquiring target image data which is processed by image data; performing prediction processing on the target image data according to the trained image evaluation model to obtain a prediction evaluation result, wherein the image evaluation model is obtained by performing combined training according to a weight ratio based on an image impurity evaluation model and an image quality evaluation model; if the prediction evaluation result is an inferior image, acquiring a data processing tracking path of the target image data, and extracting a processing node in the data processing tracking path, wherein the data processing tracking path is a path for recording a storage processing flow when the original image data is processed by image data; and outputting the target image data, the processing node and the original image data matched with the target image data. Compared with the prior art, the target image data is subjected to prediction processing through the image evaluation model obtained by performing combined training on the image impurity evaluation model and the image quality evaluation model according to the weight proportion, and the tracking path of data processing is recorded, so that when the image data is difficult to identify, the reason of the problem can be inquired according to the tracking path, the processing error or the image data distortion can be accurately identified, and meanwhile, the effectiveness of image data evaluation is improved.
Further, as an implementation of the method shown in fig. 1, an embodiment of the present application provides an image data evaluation processing apparatus based on machine learning, as shown in fig. 4, the apparatus includes:
the device comprises a first obtaining module 21, a prediction module 22, an extraction module 23 and an output module 24.
A first acquiring module 21 configured to acquire target image data for which image data processing has been completed, in response to an image output instruction;
the prediction module 22 is configured to perform prediction processing on the target image data according to a trained image evaluation model to obtain a prediction evaluation result, where the image evaluation model is obtained by performing combined training according to a weight ratio based on an image impurity evaluation model and an image quality evaluation model;
the extracting module 23 is configured to, if the prediction evaluation result is an inferior image, obtain a data processing tracking path of the target image data, and extract a processing node in the data processing tracking path, where the data processing tracking path is a path that records a storage processing flow when the original image data is processed by image data;
and an output module 24, configured to output the target image data, the processing node, and raw image data matched with the target image data.
In a specific application scenario, before the prediction module 22, the apparatus further includes:
the construction module is used for constructing an image impurity evaluation model based on a preset unified network and training the image impurity evaluation model based on an impurity image training sample set;
the construction module is further used for constructing an image quality evaluation model based on a preset deep neural network and training the image instruction evaluation model based on a quality image training sample set, wherein a convolution kernel structure in the image quality evaluation model is replaced by a residual error network structure, and a full connection layer is replaced by a full convolution layer;
and the distribution module is used for distributing model weights for the trained image impurity evaluation model and the trained image quality evaluation model, and combining the image impurity evaluation model and the image quality evaluation model according to the model weights to obtain the image evaluation model.
In a specific application scenario, the apparatus further includes:
the second acquisition module is used for acquiring step information that the target image data completes image data processing;
the analysis module is used for analyzing the processing parameters in the step information, and the processing parameters are used for representing the abnormal probability of each image data processing step;
and the determining module is used for determining two model weights matched with the processing parameters according to the corresponding relation of preset processing weights, and distributing the image impurity evaluation model and the image quality evaluation model based on the two model weights.
In a specific application scenario, after the first obtaining module 21, the apparatus further includes:
the normalization module is used for performing normalization processing on the target image data and performing scaling and filtering operations on the target image data after normalization processing according to the image size of the original image data;
and the subtraction module is used for subtracting the original image data from the target image data after the zooming and filtering operations are finished to obtain the target image data of the high-frequency component.
In a specific application scenario, before the first obtaining module 21, the apparatus further includes:
the marking module is used for marking processing nodes for each processing flow in the image data processing when the image data processing of the original image data is detected;
and the recording module is used for recording the storage path and the calling path generated in the processing flow and generating a data processing tracking path according to the processing node, the storage path and the calling path.
In a specific application scenario, the apparatus further includes:
the first reporting module is used for reporting the processing node to indicate to update the image data processing step if medical auxiliary operation on the original image data is detected;
and the updating module is used for updating the original image data to a model training sample set if medical auxiliary operation on the original image data is not detected.
In a specific application scenario, the apparatus further includes:
the statistical module is used for counting the processing mode, the processing parameters and the processing characteristics in the image data processing if the prediction evaluation result is a high-quality image;
and the second reporting module is used for reporting the counted processing mode, processing parameters and processing characteristics according to a preset time interval so as to perform image data processing on the image data belonging to the poor-quality image based on the processing mode, the processing parameters and the processing characteristics.
The application provides an image data evaluation processing device based on machine learning, which first responds to an image output instruction to obtain target image data which is processed by image data; performing prediction processing on the target image data according to the trained image evaluation model to obtain a prediction evaluation result, wherein the image evaluation model is obtained by performing combined training according to a weight ratio based on an image impurity evaluation model and an image quality evaluation model; if the prediction evaluation result is an inferior image, acquiring a data processing tracking path of the target image data, and extracting a processing node in the data processing tracking path, wherein the data processing tracking path is a path for recording a storage processing flow when the original image data is processed by image data; and outputting the target image data, the processing node and the original image data matched with the target image data. Compared with the prior art, the target image data is subjected to prediction processing through the image evaluation model obtained by performing combined training on the image impurity evaluation model and the image quality evaluation model according to the weight proportion, and the tracking path of data processing is recorded, so that when the image data is difficult to identify, the reason of the problem can be inquired according to the tracking path, the processing error or the image data distortion can be accurately identified, and meanwhile, the effectiveness of image data evaluation is improved.
According to an embodiment of the present application, a storage medium is provided, where the storage medium stores at least one executable instruction, and the computer executable instruction can execute the image data evaluation processing method based on machine learning in any of the above method embodiments.
Based on such understanding, the technical solution of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.), and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the implementation scenarios of the present application.
Fig. 5 is a schematic structural diagram of a computer device according to an embodiment of the present application, and the specific embodiment of the present application does not limit a specific implementation of the computer device.
As shown in fig. 5, the computer apparatus may include: a processor (processor)302, a communication Interface 304, a memory 306, and a communication bus 308.
Wherein: the processor 302, communication interface 304, and memory 306 communicate with each other via a communication bus 308.
A communication interface 304 for communicating with network elements of other devices, such as clients or other servers.
The processor 302 is configured to execute the program 310, and may specifically execute relevant steps in the above-described image data evaluation processing method based on machine learning.
In particular, program 310 may include program code comprising computer operating instructions.
The processor 302 may be a central processing unit CPU, or an application Specific Integrated circuit asic, or one or more Integrated circuits configured to implement embodiments of the present application. The computer device includes one or more processors, which may be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.
And a memory 306 for storing a program 310. Memory 306 may comprise high-speed RAM memory and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
The program 310 may specifically be configured to cause the processor 302 to perform the following operations:
acquiring target image data for which image data processing has been completed in response to an image output instruction;
performing prediction processing on the target image data according to the trained image evaluation model to obtain a prediction evaluation result, wherein the image evaluation model is obtained by performing combined training according to a weight ratio based on an image impurity evaluation model and an image quality evaluation model;
if the prediction evaluation result is an inferior image, acquiring a data processing tracking path of the target image data, and extracting a processing node in the data processing tracking path, wherein the data processing tracking path is a path for recording a storage processing flow when the original image data is processed by image data;
and outputting the target image data, the processing node and the original image data matched with the target image data.
The storage medium may further include an operating system and a network communication module. The operating system is a program for managing hardware and software resources of the entity device for processing the business data based on the multi-modal hybrid model, and supports the operation of an information processing program and other software and/or programs. The network communication module is used for realizing communication among components in the storage medium and communication with other hardware and software in the information processing entity device.
It will be apparent to those skilled in the art that the modules or steps of the present application described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present application is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. An image data evaluation processing method based on machine learning is characterized by comprising the following steps:
acquiring target image data for which image data processing has been completed in response to an image output instruction;
performing prediction processing on the target image data according to the trained image evaluation model to obtain a prediction evaluation result, wherein the image evaluation model is obtained by performing combined training according to a weight ratio based on an image impurity evaluation model and an image quality evaluation model;
if the prediction evaluation result is an inferior image, acquiring a data processing tracking path of the target image data, and extracting a processing node in the data processing tracking path, wherein the data processing tracking path is a path for recording a storage processing flow when the original image data is processed by image data;
and outputting the target image data, the processing node and the original image data matched with the target image data.
2. The method according to claim 1, wherein before performing prediction processing on the target image data according to the trained image evaluation model to obtain a prediction evaluation result, the method further comprises:
constructing an image impurity evaluation model based on a preset unified network, and training the image impurity evaluation model based on an impurity image training sample set;
constructing an image quality evaluation model based on a preset depth neural network, and training the image instruction evaluation model based on a quality image training sample set, wherein a convolution kernel structure in the image quality evaluation model is replaced by a residual error network structure, and a full connection layer is replaced by a full convolution layer;
and distributing model weights for the trained image impurity evaluation model and the trained image quality evaluation model, and combining the image impurity evaluation model and the image quality evaluation model according to the model weights to obtain an image evaluation model.
3. The method of claim 2, further comprising:
acquiring step information that the target image data has finished image data processing;
analyzing the processing parameters in the step information, wherein the processing parameters are used for representing the abnormal probability of each image data processing step;
and determining two model weights matched with the processing parameters according to the corresponding relation of preset processing weights, and distributing the image impurity evaluation model and the image quality evaluation model based on the two model weights.
4. The method according to claim 1, wherein after the target image data for which image data processing has been completed is acquired, the method further comprises:
normalizing the target image data, and carrying out zooming and filtering operations on the target image data after normalization according to the image size of the original image data;
and subtracting the original image data from the target image data subjected to the zooming and filtering operations to obtain the target image data of the high-frequency component.
5. The method according to claim 1, wherein before the acquiring target image data for which image data processing has been completed, the method further comprises:
when detecting image data processing of the original image data, marking processing nodes for each processing flow in the image data processing;
and recording a storage path and a calling path generated in the processing flow, and generating a data processing tracking path according to the processing node, the storage path and the calling path.
6. The method of claim 1, further comprising:
if the medical auxiliary operation on the original image data is detected, reporting to the processing node to indicate that the image data processing step is updated;
and if the medical auxiliary operation on the original image data is not detected, updating the original image data to a model training sample set.
7. The method according to any one of claims 1-6, further comprising:
if the prediction evaluation result is a high-quality image, counting a processing mode, a processing parameter and a processing characteristic in the image data processing;
and reporting the counted processing mode, processing parameters and processing characteristics according to a preset time interval, and processing the image data belonging to the poor-quality image based on the processing mode, the processing parameters and the processing characteristics.
8. An image data evaluation processing apparatus based on machine learning, comprising:
a first acquisition module for acquiring target image data for which image data processing has been completed in response to an image output instruction;
the prediction module is used for performing prediction processing on the target image data according to the trained image evaluation model to obtain a prediction evaluation result, and the image evaluation model is obtained by performing combined training according to a weight proportion based on an image impurity evaluation model and an image quality evaluation model;
the extraction module is used for acquiring a data processing tracking path of the target image data and extracting a processing node in the data processing tracking path if the prediction evaluation result is an inferior image, wherein the data processing tracking path is a path for recording a storage processing flow when original image data is processed by image data;
and the output module is used for outputting the target image data, the processing node and the original image data matched with the target image data.
9. A storage medium having stored therein at least one executable instruction that causes a processor to perform operations corresponding to the machine learning based image data evaluation processing method according to any one of claims 1 to 7.
10. A computer device, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the operation corresponding to the image data evaluation processing method based on machine learning in any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111017522.9A CN113537407B (en) | 2021-08-31 | 2021-08-31 | Image data evaluation processing method and device based on machine learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111017522.9A CN113537407B (en) | 2021-08-31 | 2021-08-31 | Image data evaluation processing method and device based on machine learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113537407A true CN113537407A (en) | 2021-10-22 |
| CN113537407B CN113537407B (en) | 2022-05-17 |
Family
ID=78122895
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111017522.9A Active CN113537407B (en) | 2021-08-31 | 2021-08-31 | Image data evaluation processing method and device based on machine learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113537407B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114240843A (en) * | 2021-11-18 | 2022-03-25 | 支付宝(杭州)信息技术有限公司 | Image detection method and device and electronic equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107610123A (en) * | 2017-10-11 | 2018-01-19 | 中共中央办公厅电子科技学院 | A kind of image aesthetic quality evaluation method based on depth convolutional neural networks |
| US20180343192A1 (en) * | 2017-05-25 | 2018-11-29 | Zycada Networks, Inc. | Context-aware path computation and selection |
| CN110728656A (en) * | 2019-09-06 | 2020-01-24 | 西安电子科技大学 | Meta-learning-based no-reference image quality data processing method and intelligent terminal |
| US20200184627A1 (en) * | 2019-02-14 | 2020-06-11 | Sharif University Of Technology | Quality assessment of an image |
| CN111696083A (en) * | 2020-05-20 | 2020-09-22 | 平安科技（深圳）有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN111985574A (en) * | 2020-08-31 | 2020-11-24 | 平安医疗健康管理股份有限公司 | Medical image recognition method, device, equipment and storage medium |
| CN112116567A (en) * | 2020-09-10 | 2020-12-22 | 广东电网有限责任公司广州供电局 | No-reference image quality evaluation method and device and storage medium |
-
2021
- 2021-08-31 CN CN202111017522.9A patent/CN113537407B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180343192A1 (en) * | 2017-05-25 | 2018-11-29 | Zycada Networks, Inc. | Context-aware path computation and selection |
| CN107610123A (en) * | 2017-10-11 | 2018-01-19 | 中共中央办公厅电子科技学院 | A kind of image aesthetic quality evaluation method based on depth convolutional neural networks |
| US20200184627A1 (en) * | 2019-02-14 | 2020-06-11 | Sharif University Of Technology | Quality assessment of an image |
| CN110728656A (en) * | 2019-09-06 | 2020-01-24 | 西安电子科技大学 | Meta-learning-based no-reference image quality data processing method and intelligent terminal |
| CN111696083A (en) * | 2020-05-20 | 2020-09-22 | 平安科技（深圳）有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN111985574A (en) * | 2020-08-31 | 2020-11-24 | 平安医疗健康管理股份有限公司 | Medical image recognition method, device, equipment and storage medium |
| CN112116567A (en) * | 2020-09-10 | 2020-12-22 | 广东电网有限责任公司广州供电局 | No-reference image quality evaluation method and device and storage medium |
Non-Patent Citations (4)
| Title |
|---|
| E.A. PUKHOVA 等: "The Application of Histogram Correction for Elimination of Tonal Gradation Distortions in Digital Image Processing", 《PRINT ING ENGINEERING》 * |
| JIACHEN YANG 等: "A Blind Stereoscopic Image Quality Evaluator With Segmented Stacked Autoencoders Considering the Whole Visual Perception Route", 《IEEE TRANSACTIONS ON IMAGE PROCESSING》 * |
| JONGYOO KIM 等: "Deep CNN-Based Blind Image Quality Predictor", 《IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS》 * |
| JONGYOO KIM 等: "Deep Convolutional Neural Models for Picture-Quality Prediction", 《IEEE SIGNAL PROCESING MAGAZINE》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114240843A (en) * | 2021-11-18 | 2022-03-25 | 支付宝(杭州)信息技术有限公司 | Image detection method and device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113537407B (en) | 2022-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108464840B (en) | Automatic detection method and system for breast lumps | |
| CN111862044B (en) | Ultrasonic image processing method, ultrasonic image processing device, computer equipment and storage medium | |
| CN109191451B (en) | Abnormality detection method, apparatus, device, and medium | |
| CN109168052B (en) | Method and device for determining service satisfaction degree and computing equipment | |
| CN111612756B (en) | Coronary artery specificity calcification detection method and device | |
| US20220383661A1 (en) | Method and device for retinal image recognition, electronic equipment, and storage medium | |
| CN113576508B (en) | Cerebral hemorrhage auxiliary diagnosis system based on neural network | |
| US10832410B2 (en) | Computer system, method, and program for diagnosing subject | |
| US11373309B2 (en) | Image analysis in pathology | |
| CN113763348A (en) | Image quality determination method and device, electronic equipment and storage medium | |
| CN113298780A (en) | Child bone age assessment method and system based on deep learning | |
| KR102232880B1 (en) | Method for evaluating inspector of crowdsourcing based projects for collecting image or video for artificial intelligence training data generation | |
| CN111028940B (en) | Multi-scale lung nodule detection method, device, equipment and medium | |
| CN113724185B (en) | Model processing method, device and storage medium for image classification | |
| CN112801940B (en) | Model evaluation method, device, equipment and medium | |
| CN112233161B (en) | Hand image depth determination method and device, electronic equipment and storage medium | |
| CN113707304A (en) | Triage data processing method, device, equipment and storage medium | |
| CN117333667A (en) | Automatic brain CT image bleeding area segmentation method and system | |
| CN113537407B (en) | Image data evaluation processing method and device based on machine learning | |
| CN115294505B (en) | Risk object detection and training method and device for model thereof and electronic equipment | |
| CN113349792B (en) | Method, apparatus, device and medium for classifying multi-lead electrocardiosignal | |
| CN114119588A (en) | Method, device and system for training fundus macular lesion region detection model | |
| CN114283114A (en) | Image processing method, device, equipment and storage medium | |
| KR102400568B1 (en) | Method and apparatus for identifying anomaly area of image using encoder | |
| CN114764948A (en) | Living body detection method, living body detection device, living body detection equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220526 Address after: 518000 China Aviation Center 2901, No. 1018, Huafu Road, Huahang community, Huaqiang North Street, Futian District, Shenzhen, Guangdong Province Patentee after: Shenzhen Ping An medical and Health Technology Service Co.,Ltd. Address before: Room 12G, Area H, 666 Beijing East Road, Huangpu District, Shanghai 200001 Patentee before: PING AN MEDICAL AND HEALTHCARE MANAGEMENT Co.,Ltd. |