CN114119352A - Image training set migration generation method and system - Google Patents
Image training set migration generation method and system Download PDFInfo
- Publication number
- CN114119352A CN114119352A CN202111320697.7A CN202111320697A CN114119352A CN 114119352 A CN114119352 A CN 114119352A CN 202111320697 A CN202111320697 A CN 202111320697A CN 114119352 A CN114119352 A CN 114119352A
- Authority
- CN
- China
- Prior art keywords
- picture
- training set
- image training
- style
- converted
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Processing (AREA)
Abstract
The invention provides a method and a system for generating image training set migration, which comprises the following steps: acquiring an original picture to be converted; inputting the original picture to be converted into a pre-trained image training set migration generation model to obtain an image training set; the image training set migration generation model is obtained by training a style migration network based on a preset style picture and a picture set to be converted. The method aims at obtaining the pictures with uniform styles through the pictures collected from the open source database or other equipment by the style migration algorithm under the condition of less training data when the image training set is constructed, thereby ensuring the data quality and the network training effect of the image training set and reducing the workload of artificially synthesizing the pictures.
Description
Technical Field
The invention relates to the technical field of medical image processing, in particular to a method and a system for generating image training set migration.
Background
In the medical field, the capsule endoscopy has the advantages of no pain, no injury, large information amount of shot images and the like, and has wide application value.
In the prior art, an original picture shot through a capsule endoscope is identified in a manual mode and classified, a model needs to be built for identifying the original picture more accurately and efficiently, but the model usually needs to be trained before use, and the model can be used for identifying the original picture after the training is finished. The training data is the most important in the current machine learning network training, and the data volume and the data quality are more key in the network training. Resulting in a small number of certain categories being collected, which affects the final training result.
The first method is to perform over-sampling, picture-augmentation or weighting processing on a small number of pictures of different types to increase the number of training pictures or the weight of the pictures; the second approach is to use some other open source data or some data collected by other medical devices to increase the number of training pictures.
The two methods can improve the number of the pictures with small number and categories in the training set to a certain extent, but for the first method, only oversampling processing, picture augmentation processing or weight division processing is carried out on the pictures, although the number and diversity of the pictures are improved to a certain extent, because the number of the original pictures is limited, a promotion space of the methods is limited to a certain extent, and for scenes with small number or more diverse actual scenes. The method has little help effect; for the second method, although the training set may be extended by using open source data or data collected by other medical devices, generally, due to different devices, exposure, gain, white balance, subsequent image processing, and the like during image capturing may cause different effects of the taken pictures, and there is a certain difference between the pictures. How to unify these differences is a big problem faced at present.
Therefore, how to avoid the above-mentioned defects, improve the processing efficiency of the training pictures, and further improve the accuracy of constructing the image training set becomes a problem to be solved urgently.
Disclosure of Invention
The invention provides a method and a system for generating image training set migration, which are used for overcoming the defects in the prior art.
In a first aspect, the present invention provides a method for generating a migration of an image training set, including:
acquiring an original picture to be converted;
inputting the original picture to be converted into a pre-trained image training set migration generation model to obtain an image training set; the image training set migration generation model is obtained by training a style migration network based on a preset style picture and a picture set to be converted.
In one embodiment, the image training set migration generation model is obtained by the following steps:
acquiring an image training set for training a preset model; the preset model is a classification network model used for identifying an original picture;
determining the preset style pictures and the picture set to be converted in the image training set;
and training the style migration network based on the preset style picture and the picture set to be converted to obtain the image training set migration generation model.
In one embodiment, the acquiring a training set of images for training a preset model includes:
acquiring an original picture shot by preset detection equipment and a sample picture having preset associated characteristics with the original picture;
and combining the original picture and the sample picture to obtain the image training set.
In one embodiment, the determining the preset style picture and the to-be-converted picture set in the image training set includes:
determining a picture source style corresponding to a result to be detected, and determining the preset style picture based on the picture source style;
and determining the other pictures except the preset style picture in the image training set as the picture set to be converted.
In an embodiment, the training the style migration network based on the preset style picture and the set of pictures to be converted to obtain the image training set migration generation model includes:
inputting the preset style picture and the picture set to be converted into the style migration network;
performing style conversion on the picture set to be converted based on the preset style picture to obtain a converted picture set;
and adjusting the style weight and the content weight of the converted picture set to obtain the image training set migration generation model.
In one embodiment, the preset detection device comprises a capsule endoscope.
In a second aspect, the present invention further provides an image training set migration generation system, including:
the acquisition module is used for acquiring an original picture to be converted;
the conversion module is used for inputting the original picture to be converted into a pre-trained image training set migration generation model to obtain an image training set; the image training set migration generation model is obtained by training a style migration network based on a preset style picture and a picture set to be converted.
In a third aspect, the present invention further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the program, the steps of the image training set migration generation method as described in any one of the above are implemented.
In a fourth aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the image training set migration generation method according to any one of the above.
In a fifth aspect, the present invention further provides a computer program product comprising a computer program, which when executed by a processor, implements the steps of the image training set migration generation method according to any one of the above.
According to the image training set migration generation method and system, when an image training set is constructed, under the condition that training data are few, pictures with unified styles are obtained through pictures collected from a starting database or other equipment through a style migration algorithm, the data quality and the network training effect of the image training set are guaranteed, and the workload of artificially synthesizing the pictures is reduced.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a method for generating a migration of an image training set according to the present invention;
FIG. 2 is an exemplary diagram of a to-be-converted image of a capsule endoscope according to the present invention;
FIG. 3 is an exemplary diagram of a capsule endoscopic style picture provided by the present invention;
FIG. 4 is an exemplary diagram of a converted image of the capsule endoscope according to the present invention;
FIG. 5 is a schematic structural diagram of an image training set migration generation system provided by the present invention;
fig. 6 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Aiming at the defects in the prior art, the invention provides an image training set migration generation method, fig. 1 is a schematic flow chart of the image training set migration generation method provided by the invention, and as shown in fig. 1, the method comprises the following steps:
s1, acquiring an original picture to be converted;
s2, inputting the original picture to be converted into a pre-trained image training set migration generation model to obtain an image training set; the image training set migration generation model is obtained by training a style migration network based on a preset style picture and a picture set to be converted.
Specifically, the invention provides a training set for training a preset model, and it should be noted that an original picture in the training set is taken through a capsule endoscope, and the working process of the capsule endoscope is described as follows:
the capsule endoscope enters the digestive tract from the oral cavity and is naturally discharged out of the body from the anus.
The capsule endoscope has limited battery endurance, and the effective working space is one part of oral cavity, esophagus, stomach, duodenum, small intestine and large intestine.
Each activity of the capsule endoscope produces an in-domain exam picture and an out-of-domain exam picture.
The intra-domain examination picture is a result of taking a certain section of the digestive tract.
The out-of-field inspection picture is a picture taken by the capsule endoscope in addition to the in-field inspection picture.
All pictures can be automatically identified without any human intervention (including image pre-processing).
After the images are identified, the images shot by the capsule endoscopy are divided into six major categories (125 minor categories) and automatically stored in 125 image folders, wherein the six major categories can be:
the first major category: class outside classification labels (10 classes);
the second major category: class two out-of-domain classification labels (13 classes);
the third major category: classifying labels (14 classes) based on a first target picture of local structural features;
the fourth major category: hole-structured first target picture classification labels (8 categories);
the fifth main category: a first target picture classification label (24 classes) based on global structural features;
the sixth major class: the second target picture category label (56 categories).
Different parts of the digestive tract such as the oral cavity, the esophagus, the stomach, the duodenum, the small intestine, the large intestine and the like can be automatically identified through the capsule endoscope.
The number of the original pictures which can be shot by each capsule endoscope at each time can be 2000-3000, namely the number of the pictures which are acquired by the capsule endoscopes and concentrated.
Raw pictures taken of the capsule endoscopy (JPG format) can be derived from the hospital information system without any processing.
In order to obtain an image training set with uniform style and enough quantity, the invention needs to train to obtain an image training set migration generation model, a large quantity of original pictures to be converted are input into the trained image training set migration generation model to obtain style uniform pictures, and the obtained certain quantity of style uniform pictures are constructed to obtain the image training set for training the preset model.
The method aims at obtaining the pictures with uniform styles through the pictures collected from the open source database or other equipment by the style migration algorithm under the condition of less training data when the image training set is constructed, thereby ensuring the data quality and the network training effect of the image training set and reducing the workload of artificially synthesizing the pictures.
Based on the above embodiment, the image training set migration generation model provided by the present invention is obtained through the following steps:
acquiring an image training set for training a preset model; the preset model is a classification network model used for identifying an original picture;
determining the preset style pictures and the picture set to be converted in the image training set;
and training the style migration network based on the preset style picture and the picture set to be converted to obtain the image training set migration generation model.
Specifically, a large number of image training sets are obtained first, and the image training sets are used for training preset models, which are generally used for obtaining final image processing results, such as image classification results, image recognition results, image detection results, and the like.
And selecting a preset style picture serving as a reference from the image training set, taking the rest pictures as a picture set to be converted, selecting a style migration network as a reference model, and training the style migration network by using the preset style picture and the picture set to be converted to obtain an image training set migration generation model.
According to the invention, by constructing and training the image training set migration generation model specially used for obtaining the image training set, the problem of low accuracy of image training data can be effectively solved, automatic construction of generated pictures is realized, and the workload of artificial picture synthesis is reduced.
Based on any one of the embodiments, the acquiring an image training set for training a preset model includes:
acquiring an original picture shot by preset detection equipment and a sample picture having preset associated characteristics with the original picture;
and combining the original picture and the sample picture to obtain the image training set.
Specifically, the image training set provided by the invention mainly comprises two parts: firstly, the picture that obtains is directly taken to current capsule scope, secondly, through the scope picture database of the multiple capsule scope brand of other brands and open source, or the picture that traditional gastroscope was taken and is obtained.
In addition, the required training set data can be quickly obtained by transferring other open source data or other data collected by other medical equipment, and the obtained picture style does not differ from the picture data of the self training set too much due to the change of the model and the shooting parameters of the collector.
And combining a plurality of image sample data obtained from the two channels to obtain an image training set.
Under the condition of less training data, the required foreground picture is acquired by using some open-source databases or pictures acquired by other equipment, so that the problem of insufficient training data is solved.
Based on any of the above embodiments, the determining the preset style picture and the to-be-converted picture set in the image training set includes:
determining a picture source style corresponding to a result to be detected, and determining the preset style picture based on the picture source style;
and determining the other pictures except the preset style picture in the image training set as the picture set to be converted.
Specifically, one of the picture sources is selected in the image training set as a unified style, that is, a reference picture of the style needs to be converted, the converted picture refers to a picture of a plurality of capsule endoscopy brands and open source endoscopy pictures used in hospitals and a picture of a traditional gastroscope, except for the style picture, the rest pictures are pictures of the required conversion style, and the converted pictures become converted pictures.
The method selects the style picture to train and fuse other converted pictures, can quickly unify the picture style, saves the picture training time and improves the network training effect.
Based on any of the above embodiments, the training the style migration network based on the preset style picture and the to-be-converted picture set to obtain the image training set migration generation model includes:
inputting the preset style picture and the picture set to be converted into the style migration network;
performing style conversion on the picture set to be converted based on the preset style picture to obtain a converted picture set;
and adjusting the style weight and the content weight of the converted picture set to obtain the image training set migration generation model.
Specifically, the present invention trains a Style migration network using Style pictures and conversion pictures, where the Style migration network uses Linear Style Transfer (Learning Linear Transformations for Fast Image and Video Style Transfer).
And performing style conversion on the converted picture through a style migration network, and simultaneously adjusting the style and the content weight so as to achieve a balanced effect on the style and the content of the converted picture.
Compared with other algorithms for performing the migration work, the data migration can be performed only by performing new model training or parameter adjustment on different data sources according to the actual picture condition. The style migration algorithm provided by the invention can be directly applied to all other untrained open source data or data collected by other medical equipment because the style of the style picture is directly extracted and the picture content of the migrated picture is fused, and a large amount of network training or parameter adjustment work is not needed.
Fig. 2 is a diagram of a picture to be converted acquired through a capsule endoscope, and the picture is converted according to the style set in fig. 3, and the final converted picture shown in fig. 4 is obtained after the conversion is completed.
According to the method, the style is unified to the style of the picture acquired by the equipment of the user through the style migration algorithm of the acquired open source database or the pictures acquired by other equipment, so that the quality of training data is better ensured, and the training effect of the network is ensured.
The image training set migration generation system provided by the present invention is described below, and the image training set migration generation system described below and the image training set migration generation method described above may be referred to in correspondence with each other.
Fig. 5 is a schematic structural diagram of an image training set migration generation system provided by the present invention, as shown in fig. 5, including: an acquisition module 51 and a conversion module 52, wherein:
the obtaining module 51 is configured to obtain an original picture to be converted; the conversion module 52 is configured to input the original picture to be converted into a pre-trained image training set migration generation model to obtain an image training set; the image training set migration generation model is obtained by training a style migration network based on a preset style picture and a picture set to be converted.
The method aims at obtaining the pictures with uniform styles through the pictures collected from the open source database or other equipment by the style migration algorithm under the condition of less training data when the image training set is constructed, thereby ensuring the data quality and the network training effect of the image training set and reducing the workload of artificially synthesizing the pictures.
Fig. 6 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 6: a processor (processor)610, a communication Interface (Communications Interface)620, a memory (memory)630 and a communication bus 640, wherein the processor 610, the communication Interface 620 and the memory 630 communicate with each other via the communication bus 640. The processor 610 may invoke logic instructions in the memory 630 to perform an image training set migration generation method comprising: acquiring an original picture to be converted; inputting the original picture to be converted into a pre-trained image training set migration generation model to obtain an image training set; the image training set migration generation model is obtained by training a style migration network based on a preset style picture and a picture set to be converted.
In addition, the logic instructions in the memory 630 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being storable on a non-transitory computer-readable storage medium, and the computer program being capable of executing, when executed by a processor, the image training set migration generation method provided by the above methods, the method including: acquiring an original picture to be converted; inputting the original picture to be converted into a pre-trained image training set migration generation model to obtain an image training set; the image training set migration generation model is obtained by training a style migration network based on a preset style picture and a picture set to be converted.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, the computer program, when being executed by a processor, is implemented to perform the image training set migration generation method provided by the above methods, the method including: acquiring an original picture to be converted; inputting the original picture to be converted into a pre-trained image training set migration generation model to obtain an image training set; the image training set migration generation model is obtained by training a style migration network based on a preset style picture and a picture set to be converted.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. An image training set migration generation method is characterized by comprising the following steps:
acquiring an original picture to be converted;
inputting the original picture to be converted into a pre-trained image training set migration generation model to obtain an image training set; the image training set migration generation model is obtained by training a style migration network based on a preset style picture and a picture set to be converted.
2. The image training set migration generation method according to claim 1, wherein the image training set migration generation model is obtained by:
acquiring an image training set for training a preset model; the preset model is a classification network model used for identifying an original picture;
determining the preset style pictures and the picture set to be converted in the image training set;
and training the style migration network based on the preset style picture and the picture set to be converted to obtain the image training set migration generation model.
3. The image training set migration generation method according to claim 2, wherein the obtaining of the image training set for training the preset model includes:
acquiring an original picture shot by preset detection equipment and a sample picture having preset associated characteristics with the original picture;
and combining the original picture and the sample picture to obtain the image training set.
4. The image training set migration generation method according to claim 2, wherein the determining the preset style picture and the to-be-converted picture set in the image training set includes:
determining a picture source style corresponding to a result to be detected, and determining the preset style picture based on the picture source style;
and determining the other pictures except the preset style picture in the image training set as the picture set to be converted.
5. The image training set migration generation method according to claim 2, wherein the training the style migration network based on the preset style picture and the to-be-converted picture set to obtain the image training set migration generation model includes:
inputting the preset style picture and the picture set to be converted into the style migration network;
performing style conversion on the picture set to be converted based on the preset style picture to obtain a converted picture set;
and adjusting the style weight and the content weight of the converted picture set to obtain the image training set migration generation model.
6. The image training set migration generation method according to claim 3, wherein the preset detection device comprises a capsule endoscope.
7. An image training set migration generation system, comprising:
the acquisition module is used for acquiring an original picture to be converted;
the conversion module is used for inputting the original picture to be converted into a pre-trained image training set migration generation model to obtain an image training set; the image training set migration generation model is obtained by training a style migration network based on a preset style picture and a picture set to be converted.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the image training set migration generation method according to any one of claims 1 to 6 when executing the program.
9. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the image training set migration generation method according to any one of claims 1 to 6.
10. A computer program product comprising a computer program, wherein the computer program, when being executed by a processor, is adapted to carry out the steps of the image training set migration generation method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111320697.7A CN114119352A (en) | 2021-11-09 | 2021-11-09 | Image training set migration generation method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111320697.7A CN114119352A (en) | 2021-11-09 | 2021-11-09 | Image training set migration generation method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114119352A true CN114119352A (en) | 2022-03-01 |
Family
ID=80377826
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111320697.7A Pending CN114119352A (en) | 2021-11-09 | 2021-11-09 | Image training set migration generation method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114119352A (en) |
-
2021
- 2021-11-09 CN CN202111320697.7A patent/CN114119352A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110232383B (en) | Focus image recognition method and focus image recognition system based on deep learning model | |
| US11900647B2 (en) | Image classification method, apparatus, and device, storage medium, and medical electronic device | |
| KR101875468B1 (en) | Method and apparatus for providing medical information service based on diesease model | |
| CN113743384B (en) | Stomach picture identification method and device | |
| JP2021182169A (en) | Information processing device, model learning method, data recognition method and trained model | |
| CN111597946A (en) | Processing method of image generator, image generation method and device | |
| US20190117167A1 (en) | Image processing apparatus, learning device, image processing method, method of creating classification criterion, learning method, and computer readable recording medium | |
| CN113762422B (en) | Image training set composition method and system | |
| JP2019200671A (en) | Learning device, learning method, program, data generation method, and identification device | |
| CN115719334A (en) | Medical image evaluation method, device, equipment and medium based on artificial intelligence | |
| US11138785B2 (en) | Method and system for generating 3D image of character | |
| JP7349425B2 (en) | Diagnosis support system, diagnosis support method, and diagnosis support program | |
| CN117524402A (en) | Method for analyzing endoscope image and automatically generating diagnostic report | |
| CN113705311A (en) | Image processing method and apparatus, storage medium, and electronic apparatus | |
| CN114119352A (en) | Image training set migration generation method and system | |
| CN110110750B (en) | Original picture classification method and device | |
| US20220110505A1 (en) | Information processing apparatus, control method, and non-transitory storage medium | |
| CN110772210A (en) | Diagnosis interaction system and method | |
| JP2022521136A (en) | A recording medium recording devices, methods and instructions for determining tooth bone age | |
| CN109886916B (en) | Capsule mirror image screening method and device | |
| CN111539940B (en) | Super wide angle fundus image generation method and equipment | |
| CN112188234B (en) | Image processing and live broadcasting method and related devices | |
| CN112967246A (en) | X-ray image auxiliary device and method for clinical decision support system | |
| CN113409323B (en) | Coronary artery segmentation method and device, computer-readable storage medium and electronic equipment | |
| WO2023109319A1 (en) | Systems and methods for crop disease diagnosis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |