CN117174240B - Medical image report generation method based on large model field migration - Google Patents
Medical image report generation method based on large model field migration Download PDFInfo
- Publication number
- CN117174240B CN117174240B CN202311401131.6A CN202311401131A CN117174240B CN 117174240 B CN117174240 B CN 117174240B CN 202311401131 A CN202311401131 A CN 202311401131A CN 117174240 B CN117174240 B CN 117174240B
- Authority
- CN
- China
- Prior art keywords
- text
- image
- report
- visual
- instance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000013508 migration Methods 0.000 title claims abstract description 37
- 230000005012 migration Effects 0.000 title claims abstract description 37
- 230000000007 visual effect Effects 0.000 claims abstract description 52
- 238000012549 training Methods 0.000 claims abstract description 37
- 230000002285 radioactive effect Effects 0.000 claims description 15
- 238000013507 mapping Methods 0.000 claims description 12
- 238000005070 sampling Methods 0.000 claims description 12
- 238000012512 characterization method Methods 0.000 claims description 3
- 239000000203 mixture Substances 0.000 claims description 3
- 230000006870 function Effects 0.000 description 4
- 238000002372 labelling Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
Landscapes
- Medical Treatment And Welfare Office Work (AREA)
Abstract
The invention relates to the technical field of image report generation and discloses a medical image report generation method based on migration in the field of large models; the training process for generating the model comprises the following steps: encoding the radiological image into a visual vector using a visual encoding module; inputting the visual vector and the generated prompt into a large model field instance migration module to obtain an intermediate report and instance migration loss; inputting the visual vector, the refined prompt and the intermediate report into a large model fine decoding module to obtain a final image report and cross entropy loss; and calculating the total loss, and updating parameters of the large model domain instance migration module and the large model fine decoding module by using a back propagation algorithm. According to the method, through the intra-domain instance ordering process, on the premise of a small number of data samples, the alignment of a large language model to specific task information in a special domain can be rapidly realized; the text generation capability of the large language model on the medical image report generation task can be further improved.
Description
Technical Field
The invention relates to the technical field of image report generation, in particular to a medical image report generation method based on migration in the field of large models.
Background
Medical image report generation aims to automatically generate a corresponding medical image report for a specific radiological image. The method solves two technical problems:
existing methods tend to complete the task of medical image generation by training a specific model from scratch on an existing public radiological medical image dataset, which makes it difficult to align the image features of the medical image with the text features in the report, and thus often achieves poor performance when dealing with the task of medical image report generation. The method of the invention is based on the strong generation capacity of the large language model (namely the large model) obtained from huge training data, and can effectively solve the problem of insufficient generation capacity of the small model in the field of medical image report generation.
The existing large language model method can obtain more excellent application in the text generation task of the general field by pre-training on huge general corpus, but when the method is directly applied to the medical image report generation task of the specific medical field, the method is not effective any more due to the knowledge gap between the general field knowledge and the medical field knowledge. The method of the invention provides the design of intra-domain instance migration and coarse-to-fine decoding, and by comparing and sequencing the task-specific instance information and the task-independent instance information in the medical field and further carrying out the fine processing on the generated characters, the model is enabled to be aligned with the task-specific information in the field rapidly, and more accurate and detailed medical image report content can be further generated.
Disclosure of Invention
In order to solve the technical problems, the invention provides a medical image report generation method based on large model field migration.
In order to solve the technical problems, the invention adopts the following technical scheme:
a medical image report generation method based on large model field migration inputs a radioactive image into a generation model, and the generation model generates an image report according to a generation prompt and a refinement prompt; the generation model comprises a visual coding module, a large model field instance migration module and a large model fine decoding module; a training process for generating a model, comprising the steps of:
step one, using a visual coding module to input a radioactive imageCoding as visual vector +.>；
Step two, the vision vector is calculatedGenerating a prompt +.>Inputting the large model domain instance migration module to obtain an intermediate report +.>Instance migration penalty->The method comprises the steps of carrying out a first treatment on the surface of the The method specifically comprises the following steps:
s21, utilizing the radioactive imageTraining data, public medical corpus, search tool, obtaining a related instance list using a related instance search process +.>；
S22, utilizing the vision vectorGenerating prompts->List of related examples->Intermediate report +.>Instance migration penalty->；
Step three, the vision vector is calculatedRefined prompt->And intermediate report->Inputting into a large model fine decoding module to obtain a final image report +.>Cross entropy loss->The method specifically comprises the following steps:
s31, visual vector is calculatedRefined prompt->And intermediate report->Inputting the text generator to obtain final image report +.>；
S32, reporting the image predicted by the text generatorEach word->Individual words of the image report with manual labeling +.>Contrast, and pass the loss function->Calculate loss->；/>The loss function is a cross entropy loss function;
step four, calculating the total lossAnd updating parameters of the large model domain instance migration module and the large model fine decoding module by using a back propagation algorithm.
Further, the visual coding module comprises a visual coding layer, a semantic space alignment layer and a linear mapping layer, and the visual coding module is used for inputting the radioactive imageCoding as visual vector +.>When in use, the method specifically comprises the following steps:
s11, taking the radioactive image asInputting visual coding layer to extract visual characteristic +.>；
S12, extracting the visual characteristicsThe input semantic space alignment layer is aligned with the text semantic space to obtain visual representation +.>；
S13, visual characterizationInput linear mapping layer, will->Is aligned with the hidden space dimension of the text generator to obtain a visual vector +.>。
Further, step S21, a related instance list is obtained using a related instance retrieval processWhen in use, the method specifically comprises the following steps:
s211, randomly sampling from training dataText, composing a training data text sequence +.>Random sampling from public medical corpus->Individual texts, composing a medical corpus text sequence +.>Wherein->Text sequence representing training data->First->Personal text->Representing a medical corpus text sequence->First->A text;
s212, for sampling from training dataImage encoder and text encoder using search tool for inserting radiological image +.>And the sampled training data text sequence +.>Mapping to the same feature space to obtain image feature +.>And training data text feature sequence +.>，/>Text feature sequence representing training data->The%>Training data text features; calculate->And->Cosine similarity->Fetch before by ordering>A related instance;
s213 for sampling from the public medical corpusImage encoder and text encoder using search tool for inserting radiological image +.>And the sampled medical corpus text sequence +.>Mapping to the same feature space to obtain image feature +.>And medical corpus text feature sequence->，/>Text feature sequence representing medical corpus->The%>Individual medical corpus text features; calculate->And->Cosine similarity->Fetch before by ordering>A related instance;
s214, sorting out from the training dataThe related instances are ordered from the public medical corpus +.>The related examples are combined to obtain a related example list +.>；/>Representation->The%>The elements.
Further, in step S22, an intermediate report is obtained using a contrast semantic ordering processInstance migration lossWhen in use, the method specifically comprises the following steps:
s221, visual vectorAnd generate prompt->Inputting a text generator to obtain an intermediate report +.>，Representation->The%>A personal word;
s222, reporting the middleAnd->、/>Inputting the text generator again, obtaining the intermediate report token from the last layer of the text generator +.>；
S223, obtaining relevant instance in the retrieval processExamples, combined into->Example pair(s)>The set of example pair compositions is denoted +.>；/>Indicate->Pairs of instances;
s224, the firstExamples of the example pair->And->、/>Inputting the text generator, obtaining the +.>Example characterization->The method comprises the steps of carrying out a first treatment on the surface of the First->Examples of the example pair->And->、/>Inputting the text generator, obtaining the +.>Example characterization->；
S225, characterizing the intermediate reportFirst->Representation of the correspondence of the examples->、/>Performing pair-by-pair comparison, and calculating to obtain instance migration loss->：
；
To control parameters of the edge size.
Further, by cross entropy loss functionCalculate loss->When (1):。
compared with the prior art, the invention has the beneficial technical effects that:
through the intra-domain instance ordering process, the invention can rapidly align the large language model to specific task information in the proprietary domain on the premise of a small number of data samples, well solves the domain adaptation problem between the general domain and the proprietary domain of the conventional large language model, and further uses the strong generating capacity of the large language model to better complete the task of medical image report generation.
Through the rough-to-fine generation process, the text generation capability of the large language model on the medical image report generation task can be further improved, and further the cross-modal alignment capability on the more effective example level is constructed, so that more corresponding and more accurate symptom description of pictures and texts can be generated, and the quality of the generated report is improved.
Drawings
Fig. 1 is a flowchart of a medical image report generation method according to the present invention.
Detailed Description
A preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.
The generation model in the invention comprises three modules, namely a visual coding module, a large model field instance migration module and a large model fine decoding module, and the flow of the medical image report generation method based on large model field migration is shown in figure 1.
A training process for generating a model, comprising the steps of:
step one, using a visual coding module to input a radioactive imageCoding as visual vector +.>。
The visual coding module comprises a visual coding layer, a semantic space alignment layer and a linear mapping layer, and is used for inputting radioactive imagesCoding as visual vector +.>When in use, the method specifically comprises the following steps:
s11, taking the radioactive image asInputting visual coding layer to extract visual characteristic +.>. In the invention, the visual coding layer adoptstransformer。
S12, extracting the visual characteristicsThe input semantic space alignment layer is aligned with the text semantic space to obtain visual representation +.>. In the invention, the semantic space alignment layer adopts Q-force.
S13, visual characterizationInput linear mapping layer, will->Is aligned with the hidden space dimension of the text generator to obtain a visual vector +.>. The text generator in the invention adopts a Vicurna text generator.
Step two, the vision vector is calculatedGenerating a prompt +.>Inputting the large model domain instance migration module to obtain an intermediate report +.>Instance migration penalty->The method comprises the steps of carrying out a first treatment on the surface of the The method specifically comprises the following steps:
s21, utilizing the radioactive imageTraining data, public medical corpus, search tool, obtaining a related instance list using a related instance search process +.>. The method specifically comprises the following steps:
s211, randomly sampling from training dataText, composing a training data text sequence +.>Random sampling from public medical corpus->Individual texts, composing a medical corpus text sequence +.>Wherein->Text sequence representing training data->First->Personal text->Representing a medical corpus text sequence->First->A text;
s212, for sampling from training dataImage encoder and text encoder using search tool for inserting radiological image +.>And the sampled training data text sequenceColumn->Mapping to the same feature space to obtain image feature +.>And training data text feature sequence +.>，/>Text feature sequence representing training data->The%>Training data text features; calculate->And->Cosine similarity->Fetch before by ordering>A related instance;
s213 for sampling from the public medical corpusImage encoder and text encoder using search tool for inserting radiological image +.>And the sampled medical corpus text sequence +.>Mapping to the sameFeature space, respectively obtain image features->And medical corpus text feature sequence->，/>Text feature sequence representing medical corpus->The%>Individual medical corpus text features; calculate->And->Cosine similarity->Fetch before by ordering>A related instance;
s214, sorting out from the training dataThe related instances are ordered from the public medical corpus +.>The related examples are combined to obtain a related example list +.>；/>Representation->The%>The elements.
S22, utilizing the vision vectorGenerating prompts->List of related examples->Intermediate report +.>Instance migration penalty->. The method specifically comprises the following steps:
s221, visual vectorAnd generate prompt->Inputting a text generator to obtain an intermediate report +.>，Representation->The%>A personal word;
s222, reporting the middleAnd->、/>Inputting the text generator again, obtaining the intermediate report token from the last layer of the text generator +.>；
S223, obtaining relevant instance in the retrieval processExamples, combined into->Example pair(s)>The set of example pair compositions is denoted +.>；/>Indicate->Pairs of instances;
s224, the firstExamples of the example pair->And->、/>Inputting the text generator, obtaining the +.>Example characterization->The method comprises the steps of carrying out a first treatment on the surface of the First->Examples of the example pair->And->、/>Inputting the text generator, obtaining the +.>Example characterization->；
S225, characterizing the intermediate reportFirst->Representation of the correspondence of the examples->、/>Performing pair-by-pair comparison, and calculating to obtain instance migration loss->：
；
To control parameters of the edge size.
Step three, the vision vector is calculatedRefined prompt->Intermediate report->Inputting into a large model fine decoding module to obtain a final image report +.>Cross entropy loss->The method specifically comprises the following steps:
s31, visual vector is calculatedRefined prompt->Intermediate report->Inputting the text generator to obtain final image report +.>。
S32, reporting the image predicted by the text generatorAll words->All words of the image report with manual labeling +.>In contrast to this, the number of the cells,and is->Calculate loss->：/>；
The loss function is the cross entropy loss function.
Step four, calculating the total lossAnd updating parameters of the large model domain instance migration module and the large model fine decoding module by using a back propagation algorithm.
A predictive process for generating a model, comprising the steps of:
s51, using a visual coding module, the radioactive image is displayedCoding as visual vector +.>。
S52, visual vectorGenerating a prompt +.>Inputting the large model domain instance migration module to obtain an intermediate report +.>。
S53, visual vector is addedRefined prompt->Intermediate report->Inputting into a large model fine decoding module to obtain a final image report +.>。
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.
Furthermore, it should be understood that although the present disclosure describes embodiments, not every embodiment is provided with a single embodiment, and that this description is provided for clarity only, and that the disclosure is not limited to specific embodiments, and that the embodiments may be combined appropriately to form other embodiments that will be understood by those skilled in the art.
Claims (5)
1. A medical image report generation method based on large model field migration inputs a radioactive image into a generation model, and the generation model generates an image report according to a generation prompt and a refinement prompt; the generation model comprises a visual coding module, a large model field instance migration module and a large model fine decoding module; a training process for generating a model, comprising the steps of:
step one, using a visual coding module to input a radioactive imageCoding as visual vector +.>；
Step two, the vision vector is calculatedGenerating a prompt +.>Inputting the large model field instance migration module to obtain an intermediate reportInstance migration penalty->The method comprises the steps of carrying out a first treatment on the surface of the The method specifically comprises the following steps:
s21, utilizing the radioactive imageTraining data, public medical corpus, search tool, obtaining a related instance list using a related instance search process +.>；
S22, utilizing the vision vectorGenerating prompts->List of related examples->Intermediate report +.>Instance migration penalty->；
Step three, the vision vector is calculatedRefined prompt->And intermediate report->Inputting into a large model fine decoding module to obtain a final image report +.>Cross entropy loss->The method specifically comprises the following steps:
s31, visual vector is calculatedRefined prompt->And intermediate report->Inputting the text generator to obtain the final image report；
S32, reporting the image predicted by the text generatorEach word->Words of image report with manual annotationContrast, and pass the loss function->Calculate loss->；/>The loss function is a cross entropy loss function;
step four, calculating the total lossAnd updating parameters of the large model domain instance migration module and the large model fine decoding module by using a back propagation algorithm.
2. The medical image report generating method based on large model area migration of claim 1, wherein: the visual coding module comprises a visual coding layer, a semantic space alignment layer and a linear mapping layer, and is used for inputting radioactive imagesCoding as visual vector +.>When in use, the method specifically comprises the following steps:
s11, taking the radioactive image asInputting visual coding layer to extract visual characteristic +.>；
S12, extracting the visual characteristicsThe input semantic space alignment layer is aligned with the text semantic space to obtain visual representation；
S13, visual characterizationInput linear mapping layer, will->Is aligned with the hidden space dimension of the text generator to obtain a visual vector +.>。
3. The method for generating medical image report based on large model area migration of claim 1, wherein step S21, a related instance list is obtained using a related instance search processWhen in use, the method specifically comprises the following steps:
s211, randomly sampling from training dataText, composing a training data text sequence +.>Random sampling from public medical corpus->Individual texts, composing a medical corpus text sequence +.>Wherein->Text sequence representing training data->First->Personal text->Representing a medical corpus text sequence->First->A text;
s212, for sampling from training dataImage encoder and text encoder using search tool for inserting radiological image +.>And the sampled training data text sequence +.>Mapping to the same feature space to obtain image feature +.>And training data text feature sequence +.>，/>Text feature sequence representing training data->The%>Training data text features; calculate->And->Cosine similarity of (2)Fetch before by ordering>A related instance;
s213 for sampling from the public medical corpusImage encoder and text encoder using search tool for inserting radiological image +.>And the sampled medical corpus text sequence +.>Mapping to the same feature space to obtain image feature +.>And medical corpus text feature sequence->，/>Text feature sequence representing medical corpus->The%>Individual medical corpus text features; calculate->And->Cosine similarity->Fetch before by ordering>A related instance;
s214, sorting out from the training dataThe related instances are ordered from the public medical corpus +.>The related examples are combined to obtain a related example list +.>；/>Representation->The%>The elements.
4. The medical image report generation method based on large model area migration of claim 3, wherein in step S22, an intermediate report is obtained using a contrast semantic ordering processInstance migration penalty->When in use, the method specifically comprises the following steps:
s221, visual vectorAnd generate prompt->Inputting a text generator to obtain an intermediate report +.>，/>Representation->The%>A personal word;
s222, reporting the middleAnd->、/>Inputting the text generator again from the last of the text generatorLayer gets intermediate report characterization->；
S223, obtaining relevant instance in the retrieval processExamples, combined into->Example pair(s)>The set of example pair compositions is denoted +.>；/>Indicate->Pairs of instances;
s224, the firstExamples of the example pair->And->、/>Inputting the text generator, obtaining the +.>Example characterization->The method comprises the steps of carrying out a first treatment on the surface of the First->Examples of the example pair->And->、/>Inputting the text generator, obtaining the +.>Example characterization->；
S225, characterizing the intermediate reportFirst->Representation of the correspondence of the examples->、/>Performing pair-by-pair comparison, and calculating to obtain instance migration loss->：
；
To control parameters of the edge size.
5. The method for generating medical image report based on large model area migration of claim 3, wherein the image report is generated by cross entropy loss functionCalculate loss->When (1): />。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311401131.6A CN117174240B (en) | 2023-10-26 | 2023-10-26 | Medical image report generation method based on large model field migration |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311401131.6A CN117174240B (en) | 2023-10-26 | 2023-10-26 | Medical image report generation method based on large model field migration |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117174240A CN117174240A (en) | 2023-12-05 |
| CN117174240B true CN117174240B (en) | 2024-02-09 |
Family
ID=88945276
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311401131.6A Active CN117174240B (en) | 2023-10-26 | 2023-10-26 | Medical image report generation method based on large model field migration |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117174240B (en) |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101881862B1 (en) * | 2017-02-10 | 2018-07-25 | 한국외국어대학교 연구산학협력단 | Apparatus and Method for Generating Interpretation Text of Medical Image |
| CN111338760A (en) * | 2020-02-27 | 2020-06-26 | 长沙市源本信息科技有限公司 | Service instance cross-node scaling method and device for edge computing |
| WO2021022752A1 (en) * | 2019-08-07 | 2021-02-11 | 深圳先进技术研究院 | Multimodal three-dimensional medical image fusion method and system, and electronic device |
| CN112614561A (en) * | 2020-12-24 | 2021-04-06 | 北京工业大学 | Brain CT medical report generation method based on hierarchical self-attention sequence coding |
| CN113159186A (en) * | 2021-04-23 | 2021-07-23 | 杭州电子科技大学 | Medical image automatic labeling method based on semi-supervised learning |
| CN114092439A (en) * | 2021-11-18 | 2022-02-25 | 深圳大学 | Multi-organ instance segmentation method and system |
| CN115131386A (en) * | 2022-07-14 | 2022-09-30 | 中山大学 | Contour extraction and detection method and system for thoracic cavity focus image |
| CN115293128A (en) * | 2022-08-04 | 2022-11-04 | 上海大学 | Model training method and system based on multi-modal contrast learning radiology report generation |
| CN115618022A (en) * | 2022-12-19 | 2023-01-17 | 中国科学技术大学 | Low-resource relation extraction method based on data synthesis and two-stage self-training |
| EP4120286A1 (en) * | 2021-07-16 | 2023-01-18 | Novoic Ltd. | Method of evaluating text similarity for diagnosis or monitoring of a health condition |
| WO2023029817A1 (en) * | 2021-08-31 | 2023-03-09 | 北京字节跳动网络技术有限公司 | Medical report generation method and apparatus, model training method and apparatus, and device |
| CN116525052A (en) * | 2023-03-29 | 2023-08-01 | 清华大学 | Hierarchical image report generation method and device combined with sentence level contrast learning |
| WO2023147308A1 (en) * | 2022-01-25 | 2023-08-03 | Northwestern Memorial Healthcare | Image analysis and insight generation |
| CN116759074A (en) * | 2023-07-03 | 2023-09-15 | 浙江一山智慧医疗研究有限公司 | Training method and application of multi-round conversational medical image analysis model |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11423538B2 (en) * | 2019-04-16 | 2022-08-23 | Covera Health | Computer-implemented machine learning for detection and statistical analysis of errors by healthcare providers |
| CN112992308B (en) * | 2021-03-25 | 2023-05-16 | 腾讯科技（深圳）有限公司 | Training method of medical image report generation model and image report generation method |
| US20230102428A1 (en) * | 2021-09-24 | 2023-03-30 | Microsoft Technology Licensing, Llc | Generating reports from scanned images |
-
2023
- 2023-10-26 CN CN202311401131.6A patent/CN117174240B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101881862B1 (en) * | 2017-02-10 | 2018-07-25 | 한국외국어대학교 연구산학협력단 | Apparatus and Method for Generating Interpretation Text of Medical Image |
| WO2021022752A1 (en) * | 2019-08-07 | 2021-02-11 | 深圳先进技术研究院 | Multimodal three-dimensional medical image fusion method and system, and electronic device |
| CN111338760A (en) * | 2020-02-27 | 2020-06-26 | 长沙市源本信息科技有限公司 | Service instance cross-node scaling method and device for edge computing |
| CN112614561A (en) * | 2020-12-24 | 2021-04-06 | 北京工业大学 | Brain CT medical report generation method based on hierarchical self-attention sequence coding |
| CN113159186A (en) * | 2021-04-23 | 2021-07-23 | 杭州电子科技大学 | Medical image automatic labeling method based on semi-supervised learning |
| EP4120286A1 (en) * | 2021-07-16 | 2023-01-18 | Novoic Ltd. | Method of evaluating text similarity for diagnosis or monitoring of a health condition |
| WO2023029817A1 (en) * | 2021-08-31 | 2023-03-09 | 北京字节跳动网络技术有限公司 | Medical report generation method and apparatus, model training method and apparatus, and device |
| CN114092439A (en) * | 2021-11-18 | 2022-02-25 | 深圳大学 | Multi-organ instance segmentation method and system |
| WO2023147308A1 (en) * | 2022-01-25 | 2023-08-03 | Northwestern Memorial Healthcare | Image analysis and insight generation |
| CN115131386A (en) * | 2022-07-14 | 2022-09-30 | 中山大学 | Contour extraction and detection method and system for thoracic cavity focus image |
| CN115293128A (en) * | 2022-08-04 | 2022-11-04 | 上海大学 | Model training method and system based on multi-modal contrast learning radiology report generation |
| CN115618022A (en) * | 2022-12-19 | 2023-01-17 | 中国科学技术大学 | Low-resource relation extraction method based on data synthesis and two-stage self-training |
| CN116525052A (en) * | 2023-03-29 | 2023-08-01 | 清华大学 | Hierarchical image report generation method and device combined with sentence level contrast learning |
| CN116759074A (en) * | 2023-07-03 | 2023-09-15 | 浙江一山智慧医疗研究有限公司 | Training method and application of multi-round conversational medical image analysis model |
Non-Patent Citations (4)
| Title |
|---|
| LIU J ; YUAN R ; ZHANG Y.TX-CNN_Detecting_tuberculosis_in_chest_X-ray_images_using_convolutional_neural_network.《IEEE International Conference on Image Processing ICIP》.2017,全文. * |
| 刘畅 ; 计海霞 ; 田仰华 ; 候唯姝 ; 音大为.三维动脉自旋标记在足月新生儿缺氧缺血性脑病的早期诊断和预后分组中的价值.《磁共振成像》.2023,全文. * |
| 基于深度迁移学习的大鼠肝纤维化诊断;余文林;陈振洲;范冰冰;黄穗;;计算机系统应用(第05期);全文 * |
| 梅周俊森 ; 孙水发 ; 李小龙.基于深度学习的医学影像报告自动生成研究综述.《长江信息通信》.2023,全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117174240A (en) | 2023-12-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108363743B (en) | Intelligent problem generation method and device and computer readable storage medium | |
| CN113254599B (en) | Multi-label microblog text classification method based on semi-supervised learning | |
| CN111259631B (en) | Referee document structuring method and referee document structuring device | |
| Kang et al. | Generative text steganography based on LSTM network and attention mechanism with keywords | |
| CN110619043A (en) | Automatic text abstract generation method based on dynamic word vector | |
| CN110263340B (en) | Comment generation method, comment generation device, server and storage medium | |
| CN113051368B (en) | Double-tower model training method, retrieval device and electronic equipment | |
| Fang et al. | Image captioning with word level attention | |
| CN110866129A (en) | Cross-media retrieval method based on cross-media uniform characterization model | |
| CN112561718A (en) | Case microblog evaluation object emotion tendency analysis method based on BilSTM weight sharing | |
| CN110147442B (en) | Text abstract generation system and method with controllable length | |
| CN114372475A (en) | Network public opinion emotion analysis method and system based on RoBERTA model | |
| CN113032601A (en) | Zero sample sketch retrieval method based on discriminant improvement | |
| EP4060548A1 (en) | Method and device for presenting prompt information and storage medium | |
| CN113761377B (en) | False information detection method and device based on attention mechanism multi-feature fusion, electronic equipment and storage medium | |
| CN115759119B (en) | Financial text emotion analysis method, system, medium and equipment | |
| CN117174240B (en) | Medical image report generation method based on large model field migration | |
| CN115795026A (en) | Chinese text abstract generation method based on comparative learning | |
| CN115496134A (en) | Traffic scene video description generation method and device based on multi-modal feature fusion | |
| CN116266268A (en) | Semantic analysis method and device based on contrast learning and semantic perception | |
| CN113919355B (en) | Semi-supervised named entity recognition method suitable for small training corpus scene | |
| Patankar et al. | Image Captioning with Audio Reinforcement using RNN and CNN | |
| CN111402012B (en) | E-commerce defective product identification method based on transfer learning | |
| CN110427615B (en) | Method for analyzing modification tense of financial event based on attention mechanism | |
| CN114818644B (en) | Text template generation method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |