CN117558397B - Report generating system for analysis of deterioration condition of renal patients - Google Patents
Report generating system for analysis of deterioration condition of renal patients Download PDFInfo
- Publication number
- CN117558397B CN117558397B CN202410044429.4A CN202410044429A CN117558397B CN 117558397 B CN117558397 B CN 117558397B CN 202410044429 A CN202410044429 A CN 202410044429A CN 117558397 B CN117558397 B CN 117558397B
- Authority
- CN
- China
- Prior art keywords
- image
- kidney biopsy
- sequence
- training
- semantic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 75
- 230000006866 deterioration Effects 0.000 title claims abstract description 33
- 238000011862 kidney biopsy Methods 0.000 claims abstract description 258
- 239000013598 vector Substances 0.000 claims abstract description 229
- 230000002787 reinforcement Effects 0.000 claims abstract description 25
- 208000017169 kidney disease Diseases 0.000 claims abstract description 11
- 238000012549 training Methods 0.000 claims description 109
- 238000006243 chemical reaction Methods 0.000 claims description 17
- 238000000605 extraction Methods 0.000 claims description 12
- 238000005728 strengthening Methods 0.000 claims description 12
- 239000011159 matrix material Substances 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000012937 correction Methods 0.000 claims description 8
- 238000010606 normalization Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 5
- 238000010276 construction Methods 0.000 claims description 3
- 230000005713 exacerbation Effects 0.000 claims description 3
- 201000010099 disease Diseases 0.000 abstract description 21
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 abstract description 21
- 238000011161 development Methods 0.000 abstract description 13
- 230000003902 lesion Effects 0.000 abstract description 5
- 230000009466 transformation Effects 0.000 abstract description 4
- 238000000034 method Methods 0.000 description 16
- 208000010159 IgA glomerulonephritis Diseases 0.000 description 11
- 206010021263 IgA nephropathy Diseases 0.000 description 11
- 238000010586 diagram Methods 0.000 description 6
- 238000011282 treatment Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 206010018364 Glomerulonephritis Diseases 0.000 description 2
- 238000012300 Sequence Analysis Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 229940099472 immunoglobulin a Drugs 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000008506 pathogenesis Effects 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 208000037157 Azotemia Diseases 0.000 description 1
- 206010061818 Disease progression Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000003416 augmentation Effects 0.000 description 1
- 208000020832 chronic kidney disease Diseases 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000000205 computational method Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000001627 detrimental effect Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000005750 disease progression Effects 0.000 description 1
- 230000009433 disease-worsening effect Effects 0.000 description 1
- 208000028208 end stage renal disease Diseases 0.000 description 1
- 201000000523 end stage renal failure Diseases 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 210000000987 immune system Anatomy 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 210000003734 kidney Anatomy 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000474 nursing effect Effects 0.000 description 1
- 210000005084 renal tissue Anatomy 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
- 238000012731 temporal analysis Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000001225 therapeutic effect Effects 0.000 description 1
- 238000000700 time series analysis Methods 0.000 description 1
- 238000011269 treatment regimen Methods 0.000 description 1
- 208000009852 uremia Diseases 0.000 description 1
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H15/00—ICT specially adapted for medical reports, e.g. generation or transmission thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
- G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
- G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
- G16H10/60—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for patient-specific data, e.g. for electronic patient records
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30084—Kidney; Renal
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Primary Health Care (AREA)
- Evolutionary Computation (AREA)
- Public Health (AREA)
- Epidemiology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biodiversity & Conservation Biology (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
Abstract
The invention discloses a report generation system for analyzing the deterioration condition of a kidney disease patient, which relates to the technical field of intelligent report generation and comprises the steps of acquiring kidney biopsy images of a patient at a plurality of preset time points within a preset time period to obtain a sequence of the kidney biopsy images; performing modal transformation on the sequence of kidney biopsy images to obtain a sequence of kidney biopsy image encoding vectors; carrying out semantic analysis and reinforcement update based on the image semantic attention weight on the sequence of the kidney biopsy image coding vector to obtain a sequence of reinforced kidney biopsy image context semantic association feature vectors; generating a generated deterioration condition analysis report of the patient based on the sequence of the post-reinforcement kidney biopsy image context semantic association feature vectors. Thus, through analyzing the sequence of the kidney biopsy images, the trend and mode of the lesion can be found, and the medical staff is assisted to know the disease development trend and the worsening risk of the patient.
Description
Technical Field
The invention relates to the technical field of intelligent report generation, in particular to a report generation system for analyzing the deterioration condition of a kidney disease patient.
Background
IgA is known as Immunoglobulin A, immunoglobulin A. IgA nephropathy is the most common immune glomerulonephritis worldwide. So far, the pathogenesis is not clear, and disease prediction still depends on invasive procedures of renal biopsy, although up to 20% -30% of patients may be aggravated by end stage renal disease (uremia) at present, through active treatment.
Since the probability of deterioration of IgA nephropathy gradually increases with time. However, the conventional method does not consider the time-series characteristics of IgA nephropathy, and the analysis result of the worsening condition of IgA nephropathy patients is inaccurate. Thus, an optimized solution is desired.
Disclosure of Invention
The present invention has been made to overcome the above-mentioned drawbacks, and provides a report generating system for analyzing the deterioration condition of a patient suffering from renal disease.
The present invention provides a report generating system for analysis of deterioration condition of a renal patient, comprising:
an examination image acquisition module for acquiring a plurality of kidney biopsy images of a patient at predetermined time points within a predetermined period of time to obtain a sequence of kidney biopsy images;
the mode conversion module is used for carrying out mode conversion on the sequence of the kidney biopsy images so as to obtain a sequence of kidney biopsy image coding vectors;
The strengthening updating module is used for carrying out semantic analysis on the sequence of the kidney biopsy image coding vector and strengthening updating based on the image semantic attention weight so as to obtain a sequence of the context semantic association feature vector of the kidney biopsy image after strengthening;
the worsening condition analysis report generation module is used for generating a worsening condition analysis report of the patient based on the sequence of the reinforced kidney biopsy image context semantic association feature vector;
the reinforcement update module comprises:
a context semantic association feature extraction unit, configured to extract image context semantic association features of the sequence of kidney biopsy image encoding vectors to obtain a sequence of kidney biopsy image context semantic association feature vectors;
the image semantic attention degree weight calculation unit is used for calculating the image semantic attention degree weight of the whole feature distribution of each kidney biopsy image context semantic association feature vector in the sequence of kidney biopsy image context semantic association feature vectors relative to the sequence of kidney biopsy image context semantic association feature vectors so as to obtain a plurality of image semantic attention degree weights;
the weight strengthening updating unit is used for strengthening updating the sequence of the context semantic association feature vectors of the kidney biopsy image by taking the plurality of image semantic attention weights as weight application values so as to obtain the sequence of the context semantic association feature vectors of the kidney biopsy image after strengthening;
The image semantic attention weight calculating unit is used for:
calculating the image semantic attention weight of the whole feature distribution of each kidney biopsy image context semantic association feature vector in the sequence of kidney biopsy image context semantic association feature vectors relative to the sequence of kidney biopsy image context semantic association feature vectors according to the following image semantic attention formula so as to obtain a plurality of image semantic attention weights; the image semantic attention formula is as follows:
；
wherein,and->Is 1 x->Matrix of->Is the dimension of the semantic association feature vector of the context of each of the kidney biopsy images, +.>Is the number of the context semantic association feature vectors of the kidney biopsy image, +.>Is the +.f in the sequence of the kidney biopsy image context semantic association feature vector>The context semantic association feature vector of the individual kidney biopsy image,is a Sigmoid function->Is->And (5) weighting the semantic attention of the image.
Further, the mode conversion module is used for:
passing the sequence of kidney biopsy images through an image-to-vector modality converter to obtain a sequence of kidney biopsy image encoded vectors.
Further, the context semantic association feature extraction unit is configured to:
Passing the sequence of kidney biopsy image encoding vectors through a transducer-based image semantic context correlation feature capturer to obtain the sequence of kidney biopsy image context semantic correlation feature vectors.
Further, the context semantic association feature extraction unit includes:
a query vector construction subunit, configured to perform one-dimensional arrangement on the sequence of the kidney biopsy image encoding vectors to obtain kidney biopsy image feature vectors;
a self-attention subunit, configured to calculate a product between the kidney biopsy image feature vector and a transpose vector of each kidney biopsy image encoding vector in the sequence of kidney biopsy image encoding vectors to obtain a plurality of self-attention correlation matrices;
the normalization subunit is used for respectively performing normalization processing on each self-attention correlation matrix in the plurality of self-attention correlation matrices to obtain a plurality of normalized self-attention correlation matrices;
the attention calculating subunit is used for obtaining a plurality of probability values through a Softmax classification function by each normalized self-attention correlation matrix in the normalized self-attention correlation matrices;
and the attention applying subunit is used for weighting each kidney biopsy image coding vector in the sequence of kidney biopsy image coding vectors by taking each probability value in the plurality of probability values as a weight so as to obtain the sequence of kidney biopsy image context semantic association feature vectors.
Further, the worsening condition analysis report generation module includes:
the cascading unit is used for cascading the sequence of the context semantic association feature vectors of the reinforced kidney biopsy image into a global context semantic association feature vector of the kidney biopsy image;
and a report generation unit, configured to pass the global context semantic association feature vector of the kidney biopsy image through an AIGC-based report generator to obtain a generated worsening condition analysis report of the patient.
Further, a training module for training the image-vector modality converter, the converter-based image semantic context correlation feature capturer, and the AIGC-based report generator is also included.
Further, the training module includes:
a training data acquisition unit for acquiring training data including a sequence of training kidney biopsy images of a patient, and a true value of the patient generating a worsening condition analysis report;
a training mode conversion unit, configured to pass the sequence of training kidney biopsy images through the image-vector mode converter to obtain a sequence of training kidney biopsy image encoding vectors;
The training context associated feature capturing unit is used for enabling the sequence of the training kidney biopsy image coding vectors to pass through the converter-based image semantic context associated feature capturer to obtain a sequence of training kidney biopsy image context semantic associated feature vectors;
the training image semantic concern weight calculation unit is used for calculating the image semantic concern weight of the whole feature distribution of each training kidney biopsy image context semantic association feature vector in the training kidney biopsy image context semantic association feature vector sequence relative to the training kidney biopsy image context semantic association feature vector sequence so as to obtain a plurality of training image semantic concern weights;
the training reinforcement updating unit is used for taking the semantic attention weights of the training images as weight application values and performing reinforcement updating on the sequences of the context semantic association feature vectors of the training kidney biopsy images so as to obtain the sequences of the context semantic association feature vectors of the training-reinforced kidney biopsy images;
the training cascade unit is used for cascading the sequence of the training reinforced kidney biopsy image context semantic association feature vector into a training kidney biopsy image global context semantic association feature vector;
The training feature distribution correction unit is used for carrying out feature distribution correction on the global context semantic association feature vector of the training kidney biopsy image so as to obtain a corrected global context semantic association feature vector of the kidney biopsy image;
a training report generation unit for passing the corrected kidney biopsy image global context semantic association feature vector through the AIGC-based report generator to obtain a generated loss function value;
a training unit for training the image-vector modality converter, the converter-based image semantic context correlation feature capturer and the AIGC-based report generator with the generated loss function values.
The invention acquires a plurality of kidney biopsy images of a patient at a predetermined time point within a predetermined time period to obtain a sequence of kidney biopsy images; performing modal transformation on the sequence of kidney biopsy images to obtain a sequence of kidney biopsy image encoding vectors; carrying out semantic analysis and reinforcement update based on the image semantic attention weight on the sequence of the kidney biopsy image coding vector to obtain a sequence of reinforced kidney biopsy image context semantic association feature vectors; generating a generated deterioration condition analysis report of the patient based on the sequence of the post-reinforcement kidney biopsy image context semantic association feature vectors. Thus, through analyzing the sequence of the kidney biopsy images, the trend and mode of the lesion can be found, thereby assisting medical staff in knowing the disease development trend and the worsening risk of the patient
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. In the drawings:
FIG. 1 is a block diagram of a report generating system for analysis of the deterioration condition of a renal patient, provided in an embodiment of the present invention.
FIG. 2 is a flow chart of a report generation method for analysis of a patient suffering from renal disease according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of a system architecture of a report generating method for analyzing a worsening condition of a renal patient according to an embodiment of the present invention.
Fig. 4 is an application scenario diagram of a report generating system for analyzing a worsening condition of a renal patient according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be described in further detail with reference to the accompanying drawings. The exemplary embodiments of the present invention and their descriptions herein are for the purpose of explaining the present invention, but are not to be construed as limiting the invention.
Unless defined otherwise, all technical and scientific terms used in the embodiments of the invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the present invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of the present invention.
In describing embodiments of the present invention, unless otherwise indicated and limited thereto, the term "connected" should be construed broadly, for example, it may be an electrical connection, or may be a communication between two elements, or may be a direct connection, or may be an indirect connection via an intermediate medium, and it will be understood by those skilled in the art that the specific meaning of the term may be interpreted according to circumstances.
It should be noted that, the term "first\second\third" related to the embodiment of the present invention merely distinguishes similar objects, and does not represent a specific order for the objects, and it is understood that "first\second\third" may interchange a specific order or sequence where allowed. It is to be understood that the "first\second\third" distinguishing objects may be interchanged where appropriate such that embodiments of the invention described herein may be practiced in sequences other than those illustrated or described herein.
IgA nephropathy is an immune glomerulonephritis and is mainly characterized by accumulation of a large amount of IgA immune complexes in the glomerulus, which is one of the most common kidney diseases worldwide. The pathogenesis of IgA nephropathy is not completely known at present, but may be related to abnormal immune system reaction, in general, the development process of IgA nephropathy is slow and gradual, and a large number of patients may develop obvious symptoms after years or decades.
Diagnosis of IgA nephropathy typically requires a kidney biopsy, an invasive procedure, to determine the condition by taking a sample of kidney tissue. However, this approach has certain risks and inconveniences, and it is therefore important to find a non-invasive way to predict the progression and exacerbation of the disease. Current studies indicate that the probability of deterioration of IgA nephropathy gradually increases over time. However, existing methods often do not take into account the nature of such time series, resulting in inaccurate analysis of the patient's worsening condition. Accordingly, there is a need for further research and development of new methods to better predict and monitor the worsening of IgA nephropathy.
In one embodiment of the present invention, FIG. 1 is a block diagram of a report generating system for analysis of the exacerbation of a patient suffering from kidney disease provided in an embodiment of the present invention. As shown in fig. 1, a report generating system 100 for analysis of deterioration condition of a renal patient according to an embodiment of the present invention includes: an examination image acquisition module 110 for acquiring a plurality of renal biopsy images of a patient at predetermined time points within a predetermined time period to obtain a sequence of renal biopsy images; a modality conversion module 120 for performing modality conversion on the sequence of kidney biopsy images to obtain a sequence of kidney biopsy image encoding vectors; the reinforcement updating module 130 is configured to perform semantic analysis on the sequence of the kidney biopsy image encoding vectors and perform reinforcement updating based on the image semantic attention weight to obtain a sequence of reinforced kidney biopsy image context semantic association feature vectors; and a worsening condition analysis report generation module 140, configured to generate a worsening condition analysis report of the patient based on the sequence of the post-reinforcement kidney biopsy image context semantic association feature vectors.
In the examination image acquisition module 110, it is ensured that the renal biopsy images of the patient at a plurality of predetermined time points within a predetermined period of time can be accurately acquired, and the accuracy and stability of the image acquisition apparatus, and the reasonable arrangement and coordination of the patients are ensured. By acquiring a plurality of kidney biopsy images at predetermined time points, a sequence of kidney biopsy images can be obtained, thereby better understanding the change and development trend of the disease over time.
In the mode conversion module 120, the sequence of the kidney biopsy images is subjected to mode conversion to obtain a sequence of kidney biopsy image encoding vectors, and the mode conversion ensures that the converted image encoding vectors can accurately express the characteristics of the original images. Through mode conversion, the kidney biopsy images at different time points can be converted into uniform coding vectors, so that subsequent semantic analysis and feature extraction are facilitated.
In the reinforcement update module 130, semantic analysis and reinforcement update based on the image semantic attention weight are performed on the sequence of the kidney biopsy image encoding vectors to obtain a sequence of the context semantic association feature vectors of the reinforced kidney biopsy image, the semantic analysis accurately understands the structure and the features in the image, and the reinforcement update needs to update the features in a weighted manner according to the semantic attention weight. By means of the reinforcement updating, the context semantic association characteristics of the kidney biopsy image can be extracted, and therefore the change and deterioration trend of diseases can be captured better.
In the worsening condition analysis report generation module 140, a worsening condition analysis report of the patient is generated based on the sequence of post-reinforcement kidney biopsy image context semantic association feature vectors, and the report generation accurately analyzes and interprets the feature vector sequence to draw a conclusion of the worsening condition. By generating a report of the deterioration condition analysis, important reference information can be provided to the clinician, helping them to better assess the patient's condition and to formulate a treatment regimen.
Aiming at the technical problems, the technical concept of the invention is to perform time sequence analysis on the kidney biopsy images of the patient at a plurality of different preset time points so as to intelligently generate an analysis report containing the global context semantic association characteristic information of the kidney biopsy images, thereby assisting medical staff in knowing the disease development trend and the deterioration risk of the patient.
By carrying out time sequence analysis on the kidney biopsy images of the patient at a plurality of different preset time points, the development trend and change of the disease can be better known, the medical staff can know the evolution process of the disease, and the disease condition of the patient can be more comprehensively known. Through analysis of the global context semantic association characteristic information, more comprehensive disease information including the position, degree, change trend and the like of the lesions can be provided, so that medical staff can know the disease condition of the patient more comprehensively, and personalized treatment and nursing can be provided for the medical staff.
The analysis reports containing the global context semantic association characteristic information of the kidney biopsy images are intelligently generated, and the reports can provide detailed disease descriptions and analysis so as to help medical staff to better understand the disease development trend and the worsening risk of patients. By analyzing the global context semantic association characteristic information provided in the report, medical staff can evaluate the illness state and the worsening risk of the patient more accurately, thereby being beneficial to formulating more reasonable treatment schemes and monitoring strategies and improving the treatment effect and the life quality of the patient.
In view of this, in the technical solution of the present invention, first, a sequence of renal biopsy images of a patient at a plurality of predetermined time points within a predetermined period of time is acquired to obtain the renal biopsy images. Here, the kidney biopsy images of the patient at a plurality of predetermined time points within a predetermined period may reflect the development trend and deterioration of kidney disease. Specifically, the worsening probability of IgA nephropathy gradually increases along with time, so that the development track of the disease can be known by acquiring the kidney biopsy images of the patient at different time points and observing the change process of the kidney disease. By analyzing the sequence of kidney biopsy images, the trend and pattern of lesions can be found, and further the deterioration of the patient can be predicted.
The sequence of kidney biopsy images is then passed through an image-to-vector modality converter to obtain a sequence of kidney biopsy image encoded vectors. That is, the image data is converted into a vector representation by the image-to-vector modality converter. More specifically, the renal biopsy images are typically presented in the form of images, but the images have a high complexity and dimension, while there is a lot of background information and redundant information that is detrimental to the processing and analysis of the subsequent model. In an aspect of the invention, the image-vector modality converter is capable of converting each of the kidney biopsy images into a vector representation having a fixed length. In this process, the image-vector modality converter is able to capture and characterize implicit semantic information expressed by each of the kidney biopsy images using a recurrent neural network (recurrent neural networks, RNN). In addition, the transformation of the kidney biopsy image into vector representation can reduce the dimension of data, improve the calculation efficiency, and more conveniently carry out the subsequent feature extraction and the capturing of the context-associated features.
In a specific embodiment of the present invention, the mode conversion module is configured to: passing the sequence of kidney biopsy images through an image-to-vector modality converter to obtain a sequence of kidney biopsy image encoded vectors.
Further, in one embodiment of the present invention, the augmentation update module includes: a context semantic association feature extraction unit, configured to extract image context semantic association features of the sequence of kidney biopsy image encoding vectors to obtain a sequence of kidney biopsy image context semantic association feature vectors; the image semantic attention degree weight calculation unit is used for calculating the image semantic attention degree weight of the whole feature distribution of each kidney biopsy image context semantic association feature vector in the sequence of kidney biopsy image context semantic association feature vectors relative to the sequence of kidney biopsy image context semantic association feature vectors so as to obtain a plurality of image semantic attention degree weights; and the weight reinforcement updating unit is used for performing reinforcement updating on the sequence of the context semantic association feature vectors of the kidney biopsy image by taking the plurality of image semantic attention weights as weight application values so as to obtain the sequence of the context semantic association feature vectors of the kidney biopsy image after reinforcement.
The context semantic association feature extraction unit is used for extracting image context semantic association features of the kidney biopsy image coding vector sequence to obtain a kidney biopsy image context semantic association feature vector sequence. In this way, the contextual information in the images can be extracted to help understand the relevance between the images, thereby analyzing the evolution trend and the worsening risk of the disease more accurately.
The image semantic attention weight calculation unit is used for calculating the image semantic attention weight of each vector in the kidney biopsy image context semantic association feature vector sequence relative to the overall feature distribution. By calculating the semantic attention weight of the images, the method can more accurately determine which images are more important for analyzing and predicting the worsening condition of the disease, and is beneficial to improving the prediction accuracy and reliability of the system.
The weight reinforcement updating unit uses a plurality of image semantic attention weights as weight application values to carry out reinforcement updating on the context semantic association feature vector sequence of the kidney biopsy image. The feature vector is weighted and updated according to the semantic attention weights of different images, so that the feature information of important images can be better captured, and the prediction capability of the system on the worsening condition is improved.
The units are helpful for extracting the context semantic association features of the images, calculating the semantic attention weight of the images and carrying out enhanced updating on the feature vectors so as to improve the analysis and prediction capabilities of the system on the patient kidney biopsy images, thereby better assisting medical staff in knowing the disease development trend and the worsening risk of the patient.
It should be appreciated that there is a certain context and semantic association between each of the kidney biopsy image encoding vectors in the sequence of individual kidney biopsy image encoding vectors. For example, adjacent kidney biopsy images may have some similarity or variation at the same or similar locations. In order to capture the association, in the technical scheme of the invention, the sequence of the kidney biopsy image coding vectors passes through a converter-based image semantic context association feature capturer to obtain the sequence of the kidney biopsy image context semantic association feature vectors. That is, the context information and semantic association between the individual kidney biopsy image encoding vectors are captured using the converter-based image semantic context association feature capturer, thereby more fully understanding the process and development trend of kidney lesions.
In a specific embodiment of the present invention, the context semantic association feature extraction unit is configured to: passing the sequence of kidney biopsy image encoding vectors through a transducer-based image semantic context correlation feature capturer to obtain the sequence of kidney biopsy image context semantic correlation feature vectors.
More specifically, the context semantic association feature extraction unit includes: a query vector construction subunit, configured to perform one-dimensional arrangement on the sequence of the kidney biopsy image encoding vectors to obtain kidney biopsy image feature vectors; a self-attention subunit, configured to calculate a product between the kidney biopsy image feature vector and a transpose vector of each kidney biopsy image encoding vector in the sequence of kidney biopsy image encoding vectors to obtain a plurality of self-attention correlation matrices; the normalization subunit is used for respectively performing normalization processing on each self-attention correlation matrix in the plurality of self-attention correlation matrices to obtain a plurality of normalized self-attention correlation matrices; the attention calculating subunit is used for obtaining a plurality of probability values through a Softmax classification function by each normalized self-attention correlation matrix in the normalized self-attention correlation matrices; and an attention applying subunit, configured to weight each of the kidney biopsy image encoding vectors in the sequence of kidney biopsy image encoding vectors with each of the plurality of probability values as a weight to obtain the sequence of kidney biopsy image context semantic association feature vectors.
The degree of contribution of the examination condition information expressed by each kidney biopsy image context semantic association feature vector in the sequence of kidney biopsy image context semantic association feature vectors to the worsening condition analysis may be considered different. For example, the image features of some key temporal nodes may be more important for prediction of a worsening condition, while other features may contribute less to the prediction. Therefore, in the technical scheme of the invention, the image semantic attention weight of the whole feature distribution of each kidney biopsy image context semantic association feature vector in the sequence of kidney biopsy image context semantic association feature vectors is calculated relative to the sequence of kidney biopsy image context semantic association feature vectors so as to obtain a plurality of image semantic attention weights; and taking the plurality of image semantic attention weights as weight application values, and performing reinforcement update on the sequence of the context semantic association feature vectors of the kidney biopsy image to obtain the sequence of the context semantic association feature vectors of the kidney biopsy image after reinforcement. In this way, the importance and contribution of the semantic association feature vectors of the individual kidney biopsy image contexts are quantified.
In a specific embodiment of the present invention, the image semantic attention weight calculating unit is configured to: calculating the image semantic attention weight of the whole feature distribution of each kidney biopsy image context semantic association feature vector in the sequence of kidney biopsy image context semantic association feature vectors relative to the sequence of kidney biopsy image context semantic association feature vectors according to the following image semantic attention formula so as to obtain a plurality of image semantic attention weights; the image semantic attention formula is as follows:
；
wherein,and->Is 1 x->Matrix of->Is the dimension of the semantic association feature vector of the context of each of the kidney biopsy images, +.>Is the number of the context semantic association feature vectors of the kidney biopsy image, +.>Is the +.f in the sequence of the kidney biopsy image context semantic association feature vector>The context semantic association feature vector of the individual kidney biopsy image,is a Sigmoid function->Is->And (5) weighting the semantic attention of the image.
And then cascading the sequence of the reinforced kidney biopsy image context semantic association feature vectors into kidney biopsy image global context semantic association feature vectors, and enabling the kidney biopsy image global context semantic association feature vectors to pass through an AIGC-based report generator to obtain a generated worsening condition analysis report of the patient.
Through the feature vector sequence after cascade reinforcement, the global context semantic association feature vector of the kidney biopsy image can be obtained, and the global context semantic association feature vector of the kidney biopsy image contains global information of the whole image sequence, so that the illness state and the illness state development trend of a patient can be more comprehensively described. By means of the AIGC (Artificial Intelligence and Generative Computing) -based report generator, a patient's deterioration analysis report can be generated using the kidney biopsy image global context semantic association feature vector. AIGC technology combines artificial intelligence with computational methods of generation, and is capable of generating reports with semantics and logic to provide detailed disease analysis and prediction. By integrating global context features with the AIGC report generator, the generated worsening condition analysis report can provide more accurate disease analysis and prediction, helping healthcare personnel to better understand the disease progression and worsening risk of the patient, thereby adopting corresponding treatment and monitoring strategies. The generated worsening condition analysis report can provide important reference information for medical staff, and help them to better evaluate the patient's condition and formulate a treatment plan. This helps to improve the therapeutic effect and the quality of life of the patient.
In a specific embodiment of the present invention, the worsening condition analysis report generation module includes: the cascading unit is used for cascading the sequence of the context semantic association feature vectors of the reinforced kidney biopsy image into a global context semantic association feature vector of the kidney biopsy image; and a report generation unit, configured to pass the global context semantic association feature vector of the kidney biopsy image through an AIGC-based report generator to obtain a generated worsening condition analysis report of the patient.
In one embodiment of the invention, the report generating system for renal patient deterioration condition analysis further comprises a training module for training the image-vector modality converter, the converter-based image semantic context correlation feature capturer and the AIGC-based report generator. The training module comprises: a training data acquisition unit for acquiring training data including a sequence of training kidney biopsy images of a patient, and a true value of the patient generating a worsening condition analysis report; a training mode conversion unit, configured to pass the sequence of training kidney biopsy images through the image-vector mode converter to obtain a sequence of training kidney biopsy image encoding vectors; the training context associated feature capturing unit is used for enabling the sequence of the training kidney biopsy image coding vectors to pass through the converter-based image semantic context associated feature capturer to obtain a sequence of training kidney biopsy image context semantic associated feature vectors; the training image semantic concern weight calculation unit is used for calculating the image semantic concern weight of the whole feature distribution of each training kidney biopsy image context semantic association feature vector in the training kidney biopsy image context semantic association feature vector sequence relative to the training kidney biopsy image context semantic association feature vector sequence so as to obtain a plurality of training image semantic concern weights; the training reinforcement updating unit is used for taking the semantic attention weights of the training images as weight application values and performing reinforcement updating on the sequences of the context semantic association feature vectors of the training kidney biopsy images so as to obtain the sequences of the context semantic association feature vectors of the training-reinforced kidney biopsy images; the training cascade unit is used for cascading the sequence of the training reinforced kidney biopsy image context semantic association feature vector into a training kidney biopsy image global context semantic association feature vector; the training feature distribution correction unit is used for carrying out feature distribution correction on the global context semantic association feature vector of the training kidney biopsy image so as to obtain a corrected global context semantic association feature vector of the kidney biopsy image; a training report generation unit for passing the corrected kidney biopsy image global context semantic association feature vector through the AIGC-based report generator to obtain a generated loss function value; a training unit for training the image-vector modality converter, the converter-based image semantic context correlation feature capturer and the AIGC-based report generator with the generated loss function values.
In the technical scheme of the invention, each training kidney biopsy image context semantic association feature vector in the sequence of training kidney biopsy image context semantic association feature vectors respectively expresses coding image semantic features based on inter-image source semantic context association of the sequence of training kidney biopsy images, so that image semantic feature information game discretization is considered in consideration of source semantic distribution imbalance of each training kidney biopsy image under the sequence of training kidney biopsy images and image semantic feature weight difference of the corresponding each training kidney biopsy image context semantic association feature vector relative to the feature distribution overall of the sequence of training kidney biopsy image context semantic association feature vectors, and after the sequence of training kidney biopsy image context semantic association feature vectors is weighted by taking the plurality of training image semantic feature interest weights as weight application values, the obtained sequence of training kidney biopsy image context semantic association feature vectors can have image semantic feature information game discretization, thereby influencing the training of the training kidney biopsy image context feature-enhanced sequence through classifier.
Based on this, the applicant of the present invention preferably optimizes the training kidney biopsy image global context semantic association feature vector when the training kidney biopsy image global context semantic association feature vector obtained by cascading the training enhanced kidney biopsy image context semantic association feature vector sequence is trained by an AIGC-based report generator, specifically expressed as: performing feature distribution correction on the global context semantic association feature vector of the training kidney biopsy image by using the following optimization formula to obtain a global context semantic association feature vector of the corrected kidney biopsy image; wherein, the optimization formula is:
；
wherein,is the +.f. of the training kidney biopsy image global context semantic association feature vector>Characteristic value of individual position->Is the +.f. of the training kidney biopsy image global context semantic association feature vector>Characteristic value of individual position, and->Is a scale superparameter,/->Is the +.f. of the global context semantically related feature vector of the modified kidney biopsy image>Characteristic value of individual position->Representing a logarithmic function with a base of 2 calculated.
Specifically, when the training kidney biopsy image global context semantic association feature vector is trained, the text semantic probability density generation of the report generator based on the AIGC during training acts on the training kidney biopsy image global context semantic association feature vector, due to the compact nature of the text semantic probability density generation, image semantic feature information game discretization among feature values of each position of the training kidney biopsy image global context semantic association feature vector can generate a large-scale information game, so that a solution can not be generated to converge to Nash equilibrium on the game basis, especially in the case of large-scale imperfect game discretization information taking the training enhanced kidney biopsy image context semantic association feature vector as a unit, and in this way, the equivalent convergence of information equalization of the training kidney biopsy image global context semantic association feature vector is realized through vector information self-control equalization neighborhood based on the training kidney biopsy image global context semantic association feature vector, and the convergence can be promoted through self-game feature value in the local neighborhood, thereby improving the effect of the correction kidney biopsy image global context correlation feature vector based on the AIGC.
In summary, a report generating system 100 for analyzing the worsening condition of a renal patient according to an embodiment of the present invention is illustrated, which performs time-series analysis on the renal biopsy images of the patient at a plurality of different predetermined time points, so as to intelligently generate an analysis report including the global context semantic association characteristic information of the renal biopsy images, so as to assist medical staff in knowing the disease development trend and the worsening risk of the patient.
As described above, the report generating system 100 for analysis of a worsening condition of a renal patient according to an embodiment of the present invention can be implemented in various terminal devices, such as a server or the like for report generation for analysis of a worsening condition of a renal patient. In one example, the report generating system 100 for renal patient deterioration analysis according to embodiments of the present invention may be integrated into the terminal device as a software module and/or hardware module. For example, the report generating system 100 for renal patient deterioration analysis may be a software module in the operating system of the terminal device, or may be an application developed for the terminal device; of course, the report generating system 100 for analysis of the deterioration condition of a renal patient may also be one of the plurality of hardware modules of the terminal device.
Alternatively, in another example, the report generating system 100 for renal patient deterioration analysis and the terminal device may be separate devices, and the report generating system 100 for renal patient deterioration analysis may be connected to the terminal device through a wired and/or wireless network and transmit interactive information in a contracted data format.
FIG. 2 is a flow chart of a report generation method for analysis of a patient suffering from renal disease according to an embodiment of the present invention. Fig. 3 is a schematic diagram of a system architecture of a report generating method for analyzing a worsening condition of a renal patient according to an embodiment of the present invention. As shown in fig. 2 and 3, a report generation method for analysis of deterioration condition of a renal patient, comprising: 210 acquiring a sequence of renal biopsy images of the patient at a plurality of predetermined time points over a predetermined period of time to obtain a renal biopsy image; 220, performing modal transformation on the sequence of kidney biopsy images to obtain a sequence of kidney biopsy image encoding vectors; 230, performing semantic analysis and reinforcement update based on the image semantic attention weight on the sequence of the kidney biopsy image coding vector to obtain a sequence of reinforced kidney biopsy image context semantic association feature vectors; 240, generating a generated deterioration condition analysis report of the patient based on the sequence of post-reinforcement kidney biopsy image context semantic association feature vectors.
It will be appreciated by those skilled in the art that the specific operations of the respective steps in the above report generation method for analysis of the worsening condition of a renal patient have been described in detail above with reference to the description of the report generation system for analysis of the worsening condition of a renal patient of fig. 1, and thus, repetitive descriptions thereof will be omitted.
Fig. 4 is an application scenario diagram of a report generating system for analyzing a worsening condition of a renal patient according to an embodiment of the present invention. As shown in fig. 4, in this application scenario, first, a renal biopsy image of a patient at a plurality of predetermined time points within a predetermined period of time is acquired (C as illustrated in fig. 4); the acquired kidney biopsy image is then input into a server (S as illustrated in fig. 4) deployed with a report generation algorithm for renal patient deterioration condition analysis, wherein the server is capable of processing the kidney biopsy image based on the report generation algorithm for renal patient deterioration condition analysis to generate a generated deterioration condition analysis report for the patient.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (7)
1. A report generating system for analysis of a patient suffering from renal disease, comprising:
an examination image acquisition module for acquiring a plurality of kidney biopsy images of a patient at predetermined time points within a predetermined period of time to obtain a sequence of kidney biopsy images;
the mode conversion module is used for carrying out mode conversion on the sequence of the kidney biopsy images so as to obtain a sequence of kidney biopsy image coding vectors;
the strengthening updating module is used for carrying out semantic analysis on the sequence of the kidney biopsy image coding vector and strengthening updating based on the image semantic attention weight so as to obtain a sequence of the context semantic association feature vector of the kidney biopsy image after strengthening;
the worsening condition analysis report generation module is used for generating a worsening condition analysis report of the patient based on the sequence of the reinforced kidney biopsy image context semantic association feature vector;
the reinforcement update module comprises:
a context semantic association feature extraction unit, configured to extract image context semantic association features of the sequence of kidney biopsy image encoding vectors to obtain a sequence of kidney biopsy image context semantic association feature vectors;
the image semantic attention degree weight calculation unit is used for calculating the image semantic attention degree weight of the whole feature distribution of each kidney biopsy image context semantic association feature vector in the sequence of kidney biopsy image context semantic association feature vectors relative to the sequence of kidney biopsy image context semantic association feature vectors so as to obtain a plurality of image semantic attention degree weights;
The weight strengthening updating unit is used for strengthening updating the sequence of the context semantic association feature vectors of the kidney biopsy image by taking the plurality of image semantic attention weights as weight application values so as to obtain the sequence of the context semantic association feature vectors of the kidney biopsy image after strengthening;
the image semantic attention weight calculating unit is used for:
calculating the image semantic attention weight of the whole feature distribution of each kidney biopsy image context semantic association feature vector in the sequence of kidney biopsy image context semantic association feature vectors relative to the sequence of kidney biopsy image context semantic association feature vectors according to the following image semantic attention formula so as to obtain a plurality of image semantic attention weights; the image semantic attention formula is as follows:
；
wherein,and->Is 1 x->Matrix of->Is the dimension of the semantic association feature vector of the context of each of the kidney biopsy images, +.>Is the number of the context semantic association feature vectors of the kidney biopsy image, +.>Is the +.f in the sequence of the kidney biopsy image context semantic association feature vector>Context semantic association feature vector of individual kidney biopsy image,/- >Is a Sigmoid function->Is->And (5) weighting the semantic attention of the image.
2. The report generating system for analysis of a patient's exacerbation condition of kidney disease of claim 1, wherein said modality conversion module is configured to:
passing the sequence of kidney biopsy images through an image-to-vector modality converter to obtain a sequence of kidney biopsy image encoded vectors.
3. The report generating system for analysis of deterioration condition of renal patients of claim 2, wherein the contextual semantic association feature extraction unit is configured to:
passing the sequence of kidney biopsy image encoding vectors through a transducer-based image semantic context correlation feature capturer to obtain the sequence of kidney biopsy image context semantic correlation feature vectors.
4. The report generating system for analysis of deterioration condition of renal patients of claim 3, wherein the contextual semantic association feature extraction unit comprises:
a query vector construction subunit, configured to perform one-dimensional arrangement on the sequence of the kidney biopsy image encoding vectors to obtain kidney biopsy image feature vectors;
a self-attention subunit, configured to calculate a product between the kidney biopsy image feature vector and a transpose vector of each kidney biopsy image encoding vector in the sequence of kidney biopsy image encoding vectors to obtain a plurality of self-attention correlation matrices;
The normalization subunit is used for respectively performing normalization processing on each self-attention correlation matrix in the plurality of self-attention correlation matrices to obtain a plurality of normalized self-attention correlation matrices;
the attention calculating subunit is used for obtaining a plurality of probability values through a Softmax classification function by each normalized self-attention correlation matrix in the normalized self-attention correlation matrices;
and the attention applying subunit is used for weighting each kidney biopsy image coding vector in the sequence of kidney biopsy image coding vectors by taking each probability value in the plurality of probability values as a weight so as to obtain the sequence of kidney biopsy image context semantic association feature vectors.
5. The report generating system for analysis of a worsening condition of a renal patient of claim 4 wherein the worsening condition analysis report generating module comprises:
the cascading unit is used for cascading the sequence of the context semantic association feature vectors of the reinforced kidney biopsy image into a global context semantic association feature vector of the kidney biopsy image;
and a report generation unit, configured to pass the global context semantic association feature vector of the kidney biopsy image through an AIGC-based report generator to obtain a generated worsening condition analysis report of the patient.
6. The report generating system for renal patient deterioration condition analysis of claim 5, further comprising a training module for training the image-vector modality converter, the converter-based image semantic context correlation feature capturer, and the AIGC-based report generator.
7. The report generating system for renal patient deterioration condition analysis of claim 6, wherein the training module comprises:
a training data acquisition unit for acquiring training data including a sequence of training kidney biopsy images of a patient, and a true value of the patient generating a worsening condition analysis report;
a training mode conversion unit, configured to pass the sequence of training kidney biopsy images through the image-vector mode converter to obtain a sequence of training kidney biopsy image encoding vectors;
the training context associated feature capturing unit is used for enabling the sequence of the training kidney biopsy image coding vectors to pass through the converter-based image semantic context associated feature capturer to obtain a sequence of training kidney biopsy image context semantic associated feature vectors;
The training image semantic concern weight calculation unit is used for calculating the image semantic concern weight of the whole feature distribution of each training kidney biopsy image context semantic association feature vector in the training kidney biopsy image context semantic association feature vector sequence relative to the training kidney biopsy image context semantic association feature vector sequence so as to obtain a plurality of training image semantic concern weights;
the training reinforcement updating unit is used for taking the semantic attention weights of the training images as weight application values and performing reinforcement updating on the sequences of the context semantic association feature vectors of the training kidney biopsy images so as to obtain the sequences of the context semantic association feature vectors of the training-reinforced kidney biopsy images;
the training cascade unit is used for cascading the sequence of the training reinforced kidney biopsy image context semantic association feature vector into a training kidney biopsy image global context semantic association feature vector;
the training feature distribution correction unit is used for carrying out feature distribution correction on the global context semantic association feature vector of the training kidney biopsy image so as to obtain a corrected global context semantic association feature vector of the kidney biopsy image;
A training report generation unit for passing the corrected kidney biopsy image global context semantic association feature vector through the AIGC-based report generator to obtain a generated loss function value;
a training unit for training the image-vector modality converter, the converter-based image semantic context correlation feature capturer and the AIGC-based report generator with the generated loss function values.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410044429.4A CN117558397B (en) | 2024-01-12 | 2024-01-12 | Report generating system for analysis of deterioration condition of renal patients |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410044429.4A CN117558397B (en) | 2024-01-12 | 2024-01-12 | Report generating system for analysis of deterioration condition of renal patients |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117558397A CN117558397A (en) | 2024-02-13 |
| CN117558397B true CN117558397B (en) | 2024-03-12 |
Family
ID=89817024
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410044429.4A Active CN117558397B (en) | 2024-01-12 | 2024-01-12 | Report generating system for analysis of deterioration condition of renal patients |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117558397B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117764995B (en) * | 2024-02-22 | 2024-05-07 | 浙江首鼎视介科技有限公司 | Biliary pancreas imaging system and method based on deep neural network algorithm |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112366003A (en) * | 2020-10-20 | 2021-02-12 | 武汉华大基因科技有限公司 | Adverse reaction monitoring method based on clinical real world time series elements |
| CN113641819A (en) * | 2021-08-10 | 2021-11-12 | 福州大学 | Multi-task sparse sharing learning-based argument mining system and method |
| CN115274110A (en) * | 2022-08-02 | 2022-11-01 | 电子科技大学(深圳)高等研究院 | IgA nephropathy deterioration prediction analysis report generation system based on time series |
| CN116825293A (en) * | 2023-08-25 | 2023-09-29 | 青岛市胶州中心医院 | Visual obstetrical image examination processing method |
| CN117043871A (en) * | 2021-01-22 | 2023-11-10 | 西拉格国际有限公司 | Preoperative and surgical processing for surgical data scenarios |
-
2024
- 2024-01-12 CN CN202410044429.4A patent/CN117558397B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112366003A (en) * | 2020-10-20 | 2021-02-12 | 武汉华大基因科技有限公司 | Adverse reaction monitoring method based on clinical real world time series elements |
| CN117043871A (en) * | 2021-01-22 | 2023-11-10 | 西拉格国际有限公司 | Preoperative and surgical processing for surgical data scenarios |
| CN113641819A (en) * | 2021-08-10 | 2021-11-12 | 福州大学 | Multi-task sparse sharing learning-based argument mining system and method |
| CN115274110A (en) * | 2022-08-02 | 2022-11-01 | 电子科技大学(深圳)高等研究院 | IgA nephropathy deterioration prediction analysis report generation system based on time series |
| CN116825293A (en) * | 2023-08-25 | 2023-09-29 | 青岛市胶州中心医院 | Visual obstetrical image examination processing method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117558397A (en) | 2024-02-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ukwuoma et al. | A hybrid explainable ensemble transformer encoder for pneumonia identification from chest X-ray images | |
| CN117558397B (en) | Report generating system for analysis of deterioration condition of renal patients | |
| Wang et al. | Metransformer: Radiology report generation by transformer with multiple learnable expert tokens | |
| Jamil et al. | An efficient and robust phonocardiography (pcg)-based valvular heart diseases (vhd) detection framework using vision transformer (vit) | |
| CN118098482B (en) | Intelligent medical management system and method based on 5G technology | |
| Rehman et al. | Notice of Retraction: Infrared Sensing Based Non-Invasive Initial Diagnosis of Chronic Liver Disease Using Ensemble Learning | |
| WO2023110477A1 (en) | A computer implemented method and a system | |
| Liu et al. | Structural attention graph neural network for diagnosis and prediction of COVID-19 severity | |
| Waweru et al. | Deep learning in skin lesion analysis towards cancer detection | |
| Manocha et al. | A novel deep fusion strategy for COVID-19 prediction using multimodality approach | |
| CN117524501A (en) | Multi-mode medical data analysis system and method based on feature mining | |
| Tsung et al. | Recognizing edge-based diseases of vocal cords by using convolutional neural networks | |
| Luo et al. | Prediction of negative conversion days of childhood nephrotic syndrome based on PCA and BP-AdaBoost neural network | |
| Al Smadi et al. | Information systems in medical settings: A covid-19 detection system using x-ray scans | |
| Xing et al. | An Enhanced Vision Transformer Model in Digital Twins Powered Internet of Medical Things for Pneumonia Diagnosis | |
| Junayed et al. | A transformer-based versatile network for acne vulgaris segmentation | |
| Chawla et al. | Musculoskeletal Abnormality Detection in Humerus Radiographs Using Deep Learning. | |
| Guo et al. | Multiple abnormality classification in wireless capsule endoscopy images based on EfficientNet using attention mechanism | |
| Zhan et al. | Unidcp: Unifying multiple medical vision-language tasks via dynamic cross-modal learnable prompts | |
| Jamil et al. | Robust pcg-based vhd detection model using d-cnns, nature-inspired algorithms, and vision transformer | |
| Aslan | Automatic Detection of Knee Osteoarthritis Disease with the Developed CNN, NCA and SVM Based Hybrid Model. | |
| CN117786590B (en) | Intelligent traditional Chinese medicine system taking large language model as priori | |
| Wu et al. | MAKEN: Improving Medical Report Generation with Adapter Tuning and Knowledge Enhancement in Vision-Language Foundation Models | |
| Peng et al. | MMGPL: Multimodal Medical Data Analysis with Graph Prompt Learning | |
| Shaw et al. | An efficient ensemble-based deep learning model for the diagnosis of cervical cancer |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |