CN116152497B - Semantic segmentation model optimization method and system - Google Patents
Semantic segmentation model optimization method and system Download PDFInfo
- Publication number
- CN116152497B CN116152497B CN202310165980.XA CN202310165980A CN116152497B CN 116152497 B CN116152497 B CN 116152497B CN 202310165980 A CN202310165980 A CN 202310165980A CN 116152497 B CN116152497 B CN 116152497B
- Authority
- CN
- China
- Prior art keywords
- result
- weight
- category
- semantic segmentation
- segmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 238
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000005457 optimization Methods 0.000 title claims abstract description 33
- 238000012795 verification Methods 0.000 claims abstract description 68
- 238000012545 processing Methods 0.000 claims abstract description 48
- 238000012549 training Methods 0.000 claims abstract description 43
- 238000010276 construction Methods 0.000 claims description 10
- 238000010606 normalization Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 description 3
- 239000003086 colorant Substances 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000012805 post-processing Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000000429 assembly Methods 0.000 description 1
- 230000000712 assembly Effects 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a semantic segmentation model optimization method and a semantic segmentation model optimization system, wherein a first semantic segmentation model and a processing module are constructed; initializing the first space weight and the first category weight; updating the first space weight according to the output result of the processing module in each training iteration process to obtain a second space weight; acquiring a second category weight according to the verification result and the first category weight; finally, an optimized second semantic segmentation model is obtained according to the second space weight, the second category weight and the preset condition; the method can solve the problems of image imbalance and multi-scale targets in semantic segmentation in the training process without changing the network structure of the semantic segmentation network model and increasing the operation workload in the reasoning process.
Description
Technical Field
The application relates to the field of image semantic segmentation, in particular to a semantic segmentation model optimization method and system.
Background
Image semantic segmentation is an important direction in computer vision, and is characterized in that an image is subdivided into subareas which are not overlapped with each other and are integrated into an initial image, and the pixel distribution in each subarea accords with a preset rule; the traditional method for realizing semantic segmentation based on image processing comprises threshold segmentation, region growth, watershed algorithm, graph segmentation and other algorithms; with the development of deep learning technology, image semantic segmentation algorithms based on a deep neural network are developed vigorously, and verification results and robustness are far superior to those of a traditional method, so that the method is widely applied to the fields of automatic driving scene recognition, augmented reality, medical image analysis and the like.
Some image semantic segmentation algorithms improve receptive fields by changing network structures, such as PSPNet introduces a cavity convolution and pyramid structure, parseNet refers to a residual structure of ResNet, but the method is only aimed at a specific network, and is difficult to be effectively applied to all semantic segmentation algorithms; some algorithms realize optimization of segmentation results through priori knowledge, such as a method for realizing post-segmentation processing by means of Conditional Random Field (CRF) with pixel positions and colors as priori knowledge; the post-processing method can be compatible with most semantic segmentation networks in a post-processing mode, but the operand in the reasoning process is increased; although the CRFasRNN-like method achieves the aim of integrating CRF into a semantic segmentation network, the existing network needs to be modified and retrained, and the workload is relatively large; compared with image classification and target detection tasks based on deep learning as well, sample labeling of image semantic segmentation requires accurate classification of each pixel, and labeling difficulty is quite high; in addition, another factor influencing the semantic segmentation of the image is that the real receptive field of the network is not large enough, and the multi-scale target cannot be considered.
In view of this, it is a urgent need for a person skilled in the art to provide a semantic segmentation model optimization method and system that can alleviate image imbalance in the training process and multi-scale targets in semantic segmentation by not changing the network structure of the semantic segmentation network model and not increasing the operation workload in the reasoning process.
Disclosure of Invention
The invention aims to provide a semantic segmentation model optimization method and a semantic segmentation model optimization system, which can effectively relieve the problems of unbalance of training images and multi-scale targets in semantic segmentation without changing the existing semantic segmentation network model and increasing the operand in the reasoning process.
In order to solve the technical problems, the technical scheme provided by the invention is as follows:
a semantic segmentation model optimization method comprises the following steps:
s1, constructing a first semantic segmentation model and a processing module;
s2, initializing a first space weight and a first category weight;
s3, updating the first space weight according to the output result of the processing module so as to acquire a second space weight;
s4, verifying the first semantic segmentation model according to a preset round to obtain a verification result, and obtaining a second category weight according to the verification result and the first category weight;
s5, acquiring a second semantic segmentation model according to the second spatial weight, the second class weight and a preset condition.
Preferably, before the step S3, the method further includes:
inputting image data to the first semantic segmentation model to obtain a first segmentation result;
and inputting the image data and the first segmentation result to the processing module to obtain a second segmentation result.
Preferably, the processing module is configured to optimize the first segmentation result according to the first segmentation result and the correlation between two adjacent pixels in the image data;
the optimized first segmentation result is the second segmentation result.
Preferably, the step S3 includes the following steps:
B1. respectively carrying out normalization processing on the first segmentation result and the second segmentation result to obtain a third segmentation result corresponding to the first segmentation result and a fourth segmentation result corresponding to the second segmentation result;
B2. acquiring the category of the image data at each spatial position according to the fourth segmentation result;
B3. acquiring a probability value under the category corresponding to the spatial position corresponding to the third segmentation result according to the category;
B4. and updating the first space weight according to the probability value to acquire the second space weight.
Preferably, in the step S4, the verifying the first semantic segmentation model according to a preset round to obtain a verification result includes the following steps:
C1. dividing the image data into a training set and a verification set;
C2. inputting the training set to the first semantic segmentation model for training to obtain an intermediate result;
C3. inputting the verification set to the first semantic segmentation model to verify the intermediate result so as to obtain a verification result.
Preferably, in the step S4, according to the verification result and the first class weight, a second class weight is obtained, specifically:
and updating the first category weight according to the verification result, the preset verification result range and the preset updating scale factor after each round of iteration is completed, so as to obtain the second category weight.
Preferably, the preset condition includes:
the iteration round reaches the upper limit of the preset round;
or, the verification result is larger than or equal to a preset threshold value.
Preferably, the step S5 includes the following steps:
when any one of the preset conditions is met, optimizing the first semantic segmentation model according to the second space weight and the second category weight to obtain a second semantic segmentation model;
and when any one of the preset conditions is not met, returning to the steps S3 to S5.
An optimization system for a semantic segmentation model, comprising: the system comprises a construction module, an initialization module, a space weight module, a category weight module and an optimization module;
the construction module is used for constructing a first semantic segmentation model and the processing module;
the processing module is used for optimizing the segmentation result according to the first segmentation result and the relevance between two adjacent pixels in the image data;
the initialization module is used for initializing a first space weight and a first category weight in the first semantic segmentation model;
the spatial weight module is used for updating the first spatial weight according to the output result of the processing module so as to acquire a second spatial weight;
the category weight module is used for iterating according to a preset round, verifying the first semantic segmentation model to obtain a verification result, and updating the first category weight according to the verification result to obtain a second category weight;
the optimization module is configured to obtain a second semantic segmentation model according to the second spatial weight, the second class weight and a preset condition.
The invention provides a semantic segmentation model optimization method, which comprises the steps of constructing a first semantic segmentation model and a processing module; initializing a first space weight and a first category weight in a first semantic segmentation model; weighting the first space weight according to the output result of the processing module to obtain a second space weight; iterating according to a preset round, verifying the first semantic segmentation model to obtain a verification result, and obtaining a second category weight according to the verification result and the first category weight; finally, an optimized second semantic segmentation model is obtained according to the second space weight, the second category weight and the preset condition; according to the method, through the optimized first semantic segmentation model, the problems of image imbalance and multi-scale targets in semantic segmentation in the training process can be solved by not changing the network structure of the semantic segmentation network model and not increasing the operation workload in the reasoning process.
The invention also provides a semantic segmentation model optimization system, which solves the same technical problems as the semantic segmentation model optimization method, belongs to the same technical conception, and has the same beneficial effects, and is not repeated here.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a semantic segmentation model optimization method according to an embodiment of the present invention;
FIG. 2 is a flowchart of step S3 in an embodiment of the present invention;
FIG. 3 is a flowchart of the step S4 of obtaining the verification result according to the embodiment of the present invention;
fig. 4 is a schematic structural diagram of a semantic segmentation model optimization system according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
As shown in fig. 1, an embodiment of the present invention provides a semantic segmentation model optimization method, which includes the following steps:
s1, constructing a first semantic segmentation model and a processing module;
s2, initializing a first space weight and a first category weight;
s3, updating the first space weight according to the output result of the processing module so as to acquire a second space weight;
s4, verifying the first semantic segmentation model to obtain a verification result, and obtaining a second category weight according to the verification result and the first category weight;
s5, acquiring a second semantic segmentation model according to the second space weight, the second class weight and the preset condition.
In step S1, the first semantic segmentation model is conveniently optimized by constructing the first semantic segmentation model, and the first semantic segmentation model constructed in this embodiment may be any image processing model that needs to perform semantic segmentation on an image; the constructed processing module takes the semantic segmentation result as input, optimizes the segmentation result by combining the color and the pixel space position, and outputs the segmentation result with higher precision.
In step S2, the spatial weight is obtained by comparing the difference between the output of the first semantic segmentation model and the output of the processing module in the training process of each piece of image data; the category weight is obtained through each category accuracy obtained through each verification in the training process.
It should be noted that, the spatial weight size is consistent with the training image size, realizing punishment of pixel loss in the training process, and the spatial weight is initialized to be a matrix of all 1 at the beginning of training; the class weight length is consistent with the number of classes contained in the training sample set, penalty of class loss in the training process is realized, and the class weight is initialized to be a vector of all 1 at the beginning of training.
In step S3, after updating the first spatial weight through the output result of the processing module, a second spatial weight is obtained, so that the first semantic segmentation model is more accurate in semantic segmentation of the image data, and meanwhile, other types of image semantic segmentation models can be promoted to improve the receptive field by not changing the network structure.
In step S4, iterative round setting may be performed for verification of the first semantic segmentation model, e.g. verification of the first semantic model is performed every k rounds; then verifying the first semantic segmentation model according to the intermediate result of the first semantic segmentation model to obtain a verification result, processing the first class weight according to the verification result and combining the initialized first class weight to obtain a second class weight, and effectively improving the verification result of the first semantic segmentation model after semantic segmentation of the image through repeated training and the second class weight;
it should be noted that if the iteration round is set to 10 times, the first semantic model is verified in step S3 to step S5 until the iteration times reach 10 times, a verification result is obtained, and the first class weight is processed according to the verification result and combined with the initialized first class weight to obtain the second class weight; and if the iteration round is not reached, taking the first category weight initialized in the step S2 as the second category weight.
In step S5, by presetting a judgment condition for the first semantic segmentation model, it can be effectively judged whether the first semantic segmentation model reaches an ideal state, if so, the first semantic segmentation model is optimized according to the second spatial weight and the second class weight to obtain a second semantic segmentation model, wherein the second semantic segmentation model is the optimal semantic segmentation model.
Preferably, before step S3, the method further includes:
A1. inputting image data to a first semantic segmentation model to obtain a first segmentation result;
A2. the image data and the first segmentation result are input to a processing module to obtain a second segmentation result.
In step A1, the initial image data is input to the first semantic segmentation model to obtain a first segmentation result, and the first segmentation result is sent to step A2 for processing as input data, so that the accuracy of the first semantic segmentation model can be effectively improved.
In step A2, the image data is input to the processing module, the effective data in the image data is extracted, and then the first segmentation result obtained in step A1 is input to the processing module to obtain a second segmentation result.
Preferably, the processing module is configured to optimize the first segmentation result according to the first segmentation result and the correlation between two adjacent pixels in the image data;
the optimized first segmentation result is the second segmentation result.
In actual application, the first segmentation result is used as input of a processing module, and the correlation between two adjacent pixels in the image data is combined, so that the first segmentation result is optimized through a minimization formula, and a second segmentation result is obtained;
the minimization formula is:
wherein the method comprises the steps of，ψ u (x i ) Is a unitary potential function representing a first segmentation result of the first semantic segmentation model; psi phi type p (x i ,x j ) Is a binary potential function representing two adjacent pixels x in the image data i And x j Correlation between them; e (x) represents the second segmentation result.
As shown in fig. 2, preferably, step S3 includes the steps of:
B1. respectively carrying out normalization processing on the first segmentation result and the second segmentation result to obtain a third segmentation result corresponding to the first segmentation result and a fourth segmentation result corresponding to the second segmentation result;
B2. acquiring the category of the image data at each spatial position according to the fourth segmentation result;
B3. acquiring a probability value under the category corresponding to the spatial position corresponding to the third segmentation result according to the category;
B4. and updating the first space weight according to the probability value to acquire the second space weight.
In step B1, normalization processing is performed on the first segmentation result and the second segmentation result to obtain a third segmentation result corresponding to the first segmentation result and a fourth segmentation result corresponding to the second segmentation result, where the specific formula is as follows:
wherein,representing the spatial position [ i, j ]]Probability value for category c; />Representing [ i, j ] in the first segmentation result and the second segmentation result]The output value of the category c is subjected to exponential operation based on a natural number e; />Representing the first segmentation resultAnd [ i, j ] in the second segmentation result]The index results for all C categories are accumulated.
In step B2, the correlation is calculated according to the spatial positions and colors of the two pixels, and it can be found that the pixels with adjacent positions and similar colors belong to the same category, so in this embodiment, the category to which the fourth segmentation result belongs is obtained through the spatial positions of the image data;
in step B3, the category to which the segmentation result belongs may be obtained by the correlation between two adjacent pixels, so in this embodiment, the probability value of the third segmentation result in the corresponding category of the corresponding spatial position is calculated according to the category of the fourth segmentation result obtained in step B2;
in step B4, the first space weight is weighted and updated according to the probability value calculated in the step B3 to obtain a second space weight, and training loss is calculated according to a formula;
the second spatial weight calculation formula is:
γ 2 [h 2 ,w 2 ]＝1-c 1 ，
wherein [ h ] 2 ,w 2 ]Representing spatial position, gamma 2 Representing a second spatial weight, c 1 A probability value representing a class for which the third segmentation result is in the fourth segmentation result;
the training loss calculation formula is:
wherein d c A value representing class c in d; c is the sample lumped class number;is the class weight corresponding to class c;
the training loss is obtained by calculating the difference d between the predicted value and the true value at each spatial position, accumulating the differences, weighting the d according to the updated spatial weight, and accumulating the differences to obtain a weighted loss w 。
As shown in fig. 3, preferably, in step S4, the first semantic segmentation model is verified according to a preset round to obtain a verification result, which includes the following steps:
C1. dividing the image data into a training set and a verification set;
C2. inputting a training set to a first semantic segmentation model for training to obtain an intermediate result;
C3. inputting the verification set to the first semantic segmentation model to verify the intermediate result so as to obtain a verification result.
In step C1, a comparison experiment group is formed by dividing the image data into a training set and a verification set, so that more accurate training data can be conveniently obtained, and the training set is used for training in the first semantic segmentation model to obtain a training intermediate result; the verification set is used for verifying the intermediate result of training to obtain a final verification result;
in the step C2, the training set is input into the first semantic segmentation model for training to obtain an intermediate result, and then the intermediate result is sent into the step C3 for result verification, so that the accuracy of the first semantic segmentation model can be effectively improved;
in step C3, inputting the verification set into the first semantic segmentation model to verify the intermediate result obtained in step C2 to obtain a verification result.
Preferably, in step S4, according to the verification result and the first category weight, a second category weight is obtained, specifically:
and updating the first category weight according to the verification result, the preset verification result range and the preset updating scale factor to acquire the second category weight.
In actual application, a calculation formula is constructed by combining the verification result obtained in the step C3 with a preset verification result range and a preset updating scale factor to update the first class weight to obtain a second class weight, and in the embodiment, the verification result range is preset to be 1; the specific formula is as follows:
γ c ＝λ(1-p c )
wherein lambda is an updated scale factor, and the scale of the control weight can be generally 10; p is p c Each representation isVerification results after the round of iteration is completed;
when verifying result p c The updated class weight is lower when the class weight is higher (e.g. close to 1), and the sensitivity to class misclassification is reduced in the next training round; on the contrary, when verifying the result p c The updated class weight is higher at lower (e.g., near 0) and the sensitivity to class misclassification at the next training round is increased.
Preferably, the preset conditions include:
the iteration round reaches the upper limit of the preset round;
or, the verification result is greater than or equal to a preset threshold.
In practical application, by setting two types of conditions, the problem that different results appear in the training process but no condition is judged is avoided, so in the embodiment, one of the two conditions is always met, for example, the preset upper limit of iteration times is 100 times, and even if the iteration times reach 100 times, the verification result does not reach the preset threshold value, the iteration is terminated, and the training is completed.
Preferably, according to the second spatial weight and the second class weight and the preset condition, a second semantic segmentation model is obtained, which comprises the following steps:
when any preset condition is met, optimizing the first semantic segmentation model according to the second space weight and the second category weight to obtain a second semantic segmentation model;
and when any preset condition is not met, returning to the steps S3 to S5.
In actual application, when the iteration round or the verification result meets any preset condition, the first semantic segmentation model can be optimized through the obtained second space weight and the second category weight, and an optimized second semantic segmentation model is obtained; the optimized second semantic segmentation model can solve the problems of image imbalance in the training process and multi-scale targets in semantic segmentation by not changing the network structure of the semantic segmentation network model and not increasing the operation workload in the reasoning process; if any preset condition is not satisfied, the steps S3 to S5 need to be executed again until any preset condition is satisfied, and then the iteration is terminated, thus completing the training.
As shown in fig. 4, the embodiment of the present invention further provides a semantic segmentation model optimization system, which includes: the system comprises a construction module, an initialization module, a space weight module, a category weight module and an optimization module;
the construction module is used for constructing a first semantic segmentation model and the processing module;
the processing module is used for optimizing the segmentation result according to the first segmentation result and the relevance between two adjacent pixels in the image data;
the initialization module is used for initializing a first space weight and a first category weight in the first semantic segmentation model;
the space weight module is used for updating the first space weight according to the output result of the processing module so as to acquire the second space weight;
the category weight module is used for iterating according to a preset round, verifying the first semantic segmentation model to obtain a verification result, and updating the first category weight according to the verification result to obtain the second category weight;
and the optimization module is used for acquiring a second semantic segmentation model according to the second spatial weight, the second class weight and the preset condition.
In the actual application process, a construction module, an initialization module, a space weight module, a category weight module and an optimization module are arranged in an optimization system of the semantic segmentation model; the construction module and the processing module are in parallel relation; the space weight module and the category weight module are in parallel relation; the initialization module is connected with the construction module; one end of the space weight module is connected with the initialization module, and the other end of the space weight module is connected with the processing module; the category weight module is connected with the initialization module; one end of the optimizing module is connected with the space weight, and the other end of the optimizing module is connected with the category weight; the construction module constructs the first semantic segmentation module and then enters the initialization module; the initialization module is used for initializing the first space weight in the first semantic segmentation model and then sending the initialized first space weight to the space weight module; the processing module optimizes the segmentation result and then sends the optimized segmentation result to the space weight module; the space weight module obtains a second space weight according to the initialized first space weight and the optimized segmentation result, and the second space weight is sent to the optimization module; the initialization module initializes the first category weights in the first semantic segmentation model and then sends the initialized first category weights to the category weight module; the class weight module processes the first class weight to obtain a second class weight, and sends the second class weight to the optimization module; and the optimization module optimizes the first semantic segmentation model according to the second space weight and the second category weight to obtain a second semantic segmentation model.
In the embodiments provided in the present application, it should be understood that the disclosed method and system may be implemented in other manners. The system embodiment described above is merely illustrative, for example, the division of modules is merely a logical function division, and there may be other division manners in actual implementation, such as: multiple modules or components may be combined, or may be integrated into another system, or some features may be omitted, or not performed. In addition, the various components shown or discussed may be coupled or directly coupled or communicatively coupled to each other via some interface, whether indirectly coupled or communicatively coupled to devices or modules, whether electrically, mechanically, or otherwise.
In addition, each functional module in each embodiment of the present invention may be integrated in one processor, or each module may be separately used as one device, or two or more modules may be integrated in one device; the functional modules in the embodiments of the present invention may be implemented in hardware, or may be implemented in a form of hardware plus a software functional unit.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the above method embodiments may be implemented by program instructions and associated hardware, where the program instructions may be stored in a computer readable storage medium, and where the program instructions, when executed, perform steps comprising the above method embodiments; and the aforementioned storage medium includes: a mobile storage device, a Read Only Memory (ROM), a magnetic disk or an optical disk, or the like, which can store program codes.
It should be appreciated that the terms "method," "apparatus," "unit," and/or "module," if used herein, are merely one way to distinguish between different components, elements, parts, portions, or assemblies at different levels. However, if other words can achieve the same purpose, the word can be replaced by other expressions.
As used in this application and in the claims, the terms "a," "an," "the," and/or "the" are not specific to the singular, but may include the plural, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that the steps and elements are explicitly identified, and they do not constitute an exclusive list, as other steps or elements may be included in a method or apparatus. The inclusion of an element defined by the phrase "comprising one … …" does not exclude the presence of additional identical elements in a process, method, article, or apparatus that comprises an element.
The terms "first" and "second" are used below for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature.
If a flowchart is used in the present application, the flowchart is used to describe the operations performed by the system according to embodiments of the present application. It should be appreciated that the preceding or following operations are not necessarily performed in order precisely. Rather, the steps may be processed in reverse order or simultaneously. Also, other operations may be added to or removed from these processes.
The method and the system for optimizing the semantic segmentation model provided by the invention are described in detail. The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (4)
1. The semantic segmentation model optimization method is characterized by comprising the following steps of:
s1, constructing a first semantic segmentation model and a processing module;
s2, initializing a first space weight and a first category weight;
s3, updating the first space weight according to the output result of the processing module so as to acquire a second space weight;
s4, verifying the first semantic segmentation model according to a preset round to obtain a verification result, and obtaining a second category weight according to the verification result and the first category weight;
s5, acquiring a second semantic segmentation model according to the second space weight, the second category weight and a preset condition;
wherein, before the step S3, the method further includes:
A1. inputting image data to the first semantic segmentation model to obtain a first segmentation result;
A2. inputting the image data and the first segmentation result to the processing module to obtain a second segmentation result;
wherein, the step S3 includes the following steps:
B1. respectively carrying out normalization processing on the first segmentation result and the second segmentation result to obtain a third segmentation result corresponding to the first segmentation result and a fourth segmentation result corresponding to the second segmentation result;
B2. acquiring the category of the image data at each spatial position according to the fourth segmentation result;
B3. acquiring a probability value under the category corresponding to the spatial position corresponding to the third segmentation result according to the category;
B4. updating the first space weight according to the probability value to acquire the second space weight;
the step S4 of verifying the first semantic segmentation model according to a preset round to obtain a verification result includes the following steps:
C1. dividing the image data into a training set and a verification set;
C2. inputting the training set to the first semantic segmentation model for training to obtain an intermediate result;
C3. inputting the verification set to the first semantic segmentation model to verify the intermediate result so as to obtain a verification result;
wherein, in the step S4, according to the verification result and the first category weight, a second category weight is obtained, specifically:
updating the first category weights according to the verification result, a preset verification result range and a preset updating scale factor to obtain the second category weights;
the processing module is used for optimizing the first segmentation result according to the first segmentation result and the relevance between two adjacent pixels in the image data; the optimized first segmentation result is the second segmentation result;
taking the first segmentation result as the input of the processing module, and combining the relevance between two adjacent pixels in the image data, and optimizing the first segmentation result through a minimization formula to obtain a second segmentation result;
the minimization formula is:
wherein, psi is u (x i ) Is a unitary potential function representing a first segmentation result of the first semantic segmentation model; psi phi type p (x i ,x j ) Is a binary potential function representing two adjacent pixels x in the image data i And x j Correlation between them; e (x) represents the second segmentation result.
2. The semantic segmentation model optimization method according to claim 1, wherein the preset conditions include:
the iteration round reaches the upper limit of the preset round;
or, the verification result is larger than or equal to a preset threshold value.
3. The semantic segmentation model optimization method according to claim 2, wherein the step S5 comprises the steps of:
when any one of the preset conditions is met, optimizing the first semantic segmentation model according to the second space weight and the second category weight to obtain a second semantic segmentation model;
and when any one of the preset conditions is not met, returning to the steps S3 to S5.
4. A semantic segmentation model optimization system, comprising: the system comprises a construction module, an initialization module, a space weight module, a category weight module and an optimization module;
the construction module is used for constructing a first semantic segmentation model and the processing module;
the processing module is used for optimizing the segmentation result according to the first segmentation result and the relevance between two adjacent pixels in the image data;
the optimized first segmentation result is a second segmentation result;
taking the first segmentation result as the input of the processing module, and combining the relevance between two adjacent pixels in the image data, and optimizing the first segmentation result through a minimization formula to obtain a second segmentation result;
the minimization formula is:
wherein, psi is u (x i ) Is a unitary potential function representing a first segmentation result of the first semantic segmentation model; psi phi type p (x i ,x j ) Is a binary potential function representing two adjacent pixels x in the image data i And x j Correlation between them; e (x) represents a second segmentation result;
the initialization module is used for initializing a first space weight and a first category weight in the first semantic segmentation model;
the spatial weight module is used for updating the first spatial weight according to the output result of the processing module so as to acquire a second spatial weight;
the category weight module is used for iterating according to a preset round, verifying the first semantic segmentation model to obtain a verification result, and updating the first category weight according to the verification result to obtain a second category weight;
the optimization module is used for acquiring a second semantic segmentation model according to the second spatial weight, the second class weight and a preset condition;
wherein, the space weight module is further configured to:
inputting image data to the first semantic segmentation model to obtain a first segmentation result;
inputting the image data and the first segmentation result to the processing module to obtain a second segmentation result;
the spatial weight module is configured to update the first spatial weight according to an output result of the processing module, so as to obtain a second spatial weight when the first spatial weight acts, and is specifically configured to:
respectively carrying out normalization processing on the first segmentation result and the second segmentation result to obtain a third segmentation result corresponding to the first segmentation result and a fourth segmentation result corresponding to the second segmentation result;
acquiring the category of the image data at each spatial position according to the fourth segmentation result;
acquiring a probability value under the category corresponding to the spatial position corresponding to the third segmentation result according to the category;
updating the first space weight according to the probability value to acquire the second space weight;
the category weight module is specifically configured to, when performing verification on the first semantic segmentation model according to a preset round to obtain a verification result action:
dividing the image data into a training set and a verification set;
inputting the training set to the first semantic segmentation model for training to obtain an intermediate result;
inputting the verification set to the first semantic segmentation model to verify the intermediate result so as to obtain a verification result;
the category weight module executes and according to the verification result and the first category weight, when obtaining a second category weight action, the category weight module is specifically configured to:
and updating the first category weight according to the verification result, a preset verification result range and a preset updating scale factor to acquire the second category weight.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310165980.XA CN116152497B (en) | 2023-02-24 | 2023-02-24 | Semantic segmentation model optimization method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310165980.XA CN116152497B (en) | 2023-02-24 | 2023-02-24 | Semantic segmentation model optimization method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116152497A CN116152497A (en) | 2023-05-23 |
| CN116152497B true CN116152497B (en) | 2024-02-27 |
Family
ID=86340508
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310165980.XA Active CN116152497B (en) | 2023-02-24 | 2023-02-24 | Semantic segmentation model optimization method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116152497B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019237646A1 (en) * | 2018-06-14 | 2019-12-19 | 清华大学深圳研究生院 | Image retrieval method based on deep learning and semantic segmentation |
| CN111353498A (en) * | 2018-12-21 | 2020-06-30 | 三星电子株式会社 | System and method for providing dominant scene classification through semantic segmentation |
| CN111563508A (en) * | 2020-04-20 | 2020-08-21 | 华南理工大学 | Semantic segmentation method based on spatial information fusion |
| WO2021097055A1 (en) * | 2019-11-14 | 2021-05-20 | Nec Laboratories America, Inc. | Domain adaptation for semantic segmentation via exploiting weak labels |
| CN114972753A (en) * | 2022-05-20 | 2022-08-30 | 暨南大学 | Lightweight semantic segmentation method and system based on context information aggregation and assisted learning |
-
2023
- 2023-02-24 CN CN202310165980.XA patent/CN116152497B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019237646A1 (en) * | 2018-06-14 | 2019-12-19 | 清华大学深圳研究生院 | Image retrieval method based on deep learning and semantic segmentation |
| CN111353498A (en) * | 2018-12-21 | 2020-06-30 | 三星电子株式会社 | System and method for providing dominant scene classification through semantic segmentation |
| WO2021097055A1 (en) * | 2019-11-14 | 2021-05-20 | Nec Laboratories America, Inc. | Domain adaptation for semantic segmentation via exploiting weak labels |
| CN111563508A (en) * | 2020-04-20 | 2020-08-21 | 华南理工大学 | Semantic segmentation method based on spatial information fusion |
| CN114972753A (en) * | 2022-05-20 | 2022-08-30 | 暨南大学 | Lightweight semantic segmentation method and system based on context information aggregation and assisted learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116152497A (en) | 2023-05-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109359608B (en) | Face recognition method based on deep learning model | |
| CN112800876B (en) | Super-spherical feature embedding method and system for re-identification | |
| CN112215119B (en) | Small target identification method, device and medium based on super-resolution reconstruction | |
| CN113128671B (en) | Service demand dynamic prediction method and system based on multi-mode machine learning | |
| CN111950453A (en) | Optional-shape text recognition method based on selective attention mechanism | |
| CN111539480B (en) | Multi-category medical image recognition method and equipment | |
| CN112784929B (en) | Small sample image classification method and device based on double-element group expansion | |
| CN111428557A (en) | Method and device for automatically checking handwritten signature based on neural network model | |
| CN110647916B (en) | Pornographic picture identification method and device based on convolutional neural network | |
| CN117237733A (en) | Breast cancer full-slice image classification method combining self-supervision and weak supervision learning | |
| CN113673482B (en) | Cell antinuclear antibody fluorescence recognition method and system based on dynamic label distribution | |
| CN110197213B (en) | Image matching method, device and equipment based on neural network | |
| CN111079930A (en) | Method and device for determining quality parameters of data set and electronic equipment | |
| CN114170654A (en) | Training method of age identification model, face age identification method and related device | |
| CN114332075A (en) | Rapid structural defect identification and classification method based on lightweight deep learning model | |
| CN113011532A (en) | Classification model training method and device, computing equipment and storage medium | |
| CN115080749B (en) | Weak supervision text classification method, system and device based on self-supervision training | |
| CN112749737A (en) | Image classification method and device, electronic equipment and storage medium | |
| CN115019102B (en) | Construction method and application of countermeasure sample generation model | |
| CN114137967A (en) | Driving behavior decision method based on multi-network joint learning | |
| CN116152497B (en) | Semantic segmentation model optimization method and system | |
| CN114170446A (en) | Temperature and brightness characteristic extraction method based on deep fusion neural network | |
| CN112446428B (en) | Image data processing method and device | |
| CN111783688A (en) | Remote sensing image scene classification method based on convolutional neural network | |
| CN115018884B (en) | Visible light infrared visual tracking method based on multi-strategy fusion tree |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: No. 205, Building B1, Huigu Science and Technology Industrial Park, No. 336 Bachelor Road, Bachelor Street, Yuelu District, Changsha City, Hunan Province, 410000 Patentee after: Wisdom Eye Technology Co.,Ltd. Country or region after: China Address before: Building 14, Phase I, Changsha Zhongdian Software Park, No. 39 Jianshan Road, Changsha High tech Development Zone, Changsha City, Hunan Province, 410205 Patentee before: Wisdom Eye Technology Co.,Ltd. Country or region before: China |
|
| CP03 | Change of name, title or address |