CN114612663B - Domain self-adaptive instance segmentation method and device based on weak supervision learning - Google Patents

Abstract

The invention discloses a field self-adaptive instance segmentation method and device based on weak supervision learning, which comprises the steps of firstly training an initial instance segmentation model on a source field, outputting backbone network characteristics and semantic score tensors, constructing a semantic tree by using hierarchical aggregation clustering, sampling leaf nodes of the semantic tree, rapidly judging whether an instance segmentation mask is accurate, carrying out mask correction on an instance with inaccurate prediction according to labeling information, and finely adjusting the initial instance segmentation model according to a target field mask correction result, thereby improving the effectiveness of the instance segmentation model. According to the method, the accurate samples are rapidly judged by using limited verification signals, the adaptability of the initial instance segmentation model is improved by spreading the accurate samples, partial noise in the inaccurate samples is processed, and the problems that in the aspect of domain self-adaption, the segmentation model is improved by introducing a supervision signal from a target domain data set, manual labeling is tedious and time-consuming, and self-training contains too much noise in a pseudo tag are solved.

Description

Domain adaptive instance segmentation method and device based on weakly supervised learning

Technical Field

The present invention relates to the technical field of instance segmentation, and in particular to a domain adaptive instance segmentation method and device based on weakly supervised learning.

Background Art

Instance segmentation has been an active area of research and engineering over the past decade. It is used in multiple signal processing fields, such as image editing, scene understanding, autonomous driving, and human-computer interaction. With the rapid development of deep convolutional neural networks (DCNNs), current instance segmentation methods have achieved satisfactory results and efficiency. However, when it comes to domain adaptation, when segmentation models learned from the source domain are applied to the target domain, they encounter the embarrassment of rapidly degrading performance due to the data drift between the source and target domains. Pixel-level annotation of a large number of images on the target domain requires a huge time cost, while unsupervised methods minimize the task-specific loss and domain adversarial loss of the source domain, which is limited by the distribution overlap between the source and target domains. Other methods adopt a self-training strategy to fine-tune the segmentation model by using target-specific pseudo-labels, which can only achieve limited improvements due to the noise in the pseudo-labels or the introduction of strong assumptions.

Summary of the invention

The purpose of the present invention is to provide a domain adaptive instance segmentation method and device based on weakly supervised learning in view of the deficiencies in the prior art.

The objective of the present invention is achieved through the following technical solutions:

According to a first aspect of the present specification, a domain adaptive instance segmentation method based on weakly supervised learning is provided, the method comprising the following steps:

(1) Train an initial instance segmentation model on the source domain, output backbone network features and a semantic score tensor, where the semantic score tensor includes the probability that each pixel belongs to a different instance;

(2) Perform instance segmentation on the target domain using the initial instance segmentation model trained in the source domain, and output the backbone network features and semantic score tensor corresponding to each image;

(3) Taking the maximum value on the instance dimension of the semantic score tensor obtained in step (2) to obtain the instance segmentation mask of each image in the target domain; multiplying the instance segmentation mask of each image in the target domain with the target domain backbone network features and the target domain semantic score tensor respectively to obtain the mask features and mask semantic score tensor of each instance in the target domain;

(4) Concatenate the mask feature f t of instance _t obtained in step (3) and the mask semantic score tensor s _t to obtain the enhanced mask feature f _t ⁺ of instance t;

(5) Hierarchical Agglomerative Clustering (HAC) is used to construct a semantic tree corresponding to each category. The enhanced mask feature of each instance belonging to the category is regarded as a leaf node. Each time the two child nodes with the smallest Euclidean distance between the enhanced mask features of the instances are selected for merging to obtain a merged node. The child nodes include leaf nodes and intermediate nodes. The enhanced mask feature and mask semantic score tensor of the merged node are respectively linear combinations of the enhanced mask features and mask semantic score tensors corresponding to the child nodes.

(6) For each semantic tree, sample the leaf nodes of the semantic tree based on the set sampling rate, quickly determine whether the instance segmentation mask is accurate, and mark the judgment result;

(7) Compare the statistical value of the annotation results of all sampled instances on the semantic tree corresponding to category k, such as the mean, with the set threshold: if the statistical value is greater than the threshold, it means that the prediction of category k is accurate, and the inaccurate sampled instances will be masked and corrected using the accurate sampled instances; if the statistical value is less than or equal to the threshold, it means that the prediction of category k is inaccurate, and the corresponding semantic tree is split into two subtrees, and each subtree resamples instances to calculate the statistical value of the annotation results, and then compares it with the designed threshold, and repeats the split-comparison process until the subtree cannot be split or the subtree does not contain any accurate sampled instances;

(8) Fine-tune the initial instance segmentation model based on the target domain mask correction results to improve the effectiveness of the instance segmentation model.

Furthermore, step (5) is specifically as follows:

The child nodes corresponding to instance t and the child nodes corresponding to instance o are merged to obtain the merged node n _j . The enhanced mask feature f _j ⁺ and mask semantic score tensor s _j of the merged node n _j are linear combinations of the enhanced mask features and mask semantic score tensors corresponding to the child nodes:

s _j = w _t s _t + w _o s _o

Among them, the weights w _t and w _o are related to the size of the child nodes:

Where _Pt and _P0 are the number of instances contained in the corresponding child nodes; for leaf nodes, _wt = _w0 = 1/2;

By merging nodes multiple times, a semantic tree corresponding to each category is finally constructed; the root node of the semantic tree is represented as n ₀ , and the remaining intermediate nodes are represented as where _Jk is the number of intermediate nodes of category k.

Furthermore, in step (7), the calculation formula of the statistical value Q _k of the annotation results of all sampled instances on the semantic tree corresponding to category k is:

Where N is the number of instances sampled from the semantic tree of category k, the instances are numbered k ₁ ,…,k _N , and l _t is the judgment result of the instance segmentation mask in step (6).

Furthermore, the backbone network of the initial instance segmentation model is composed of a swin transformer; in the source domain, the training data set includes an image set {X _source } and a corresponding instance mask image set {Y _source }; in the target domain, the test data set only includes an image set {X _tatget }.

Furthermore, the initial instance segmentation model performs data enhancement during the training phase, and the data enhancement includes horizontal/vertical flipping, translation, and scale scaling. The initial instance segmentation model is trained using the AdamW optimizer with an initial learning rate of 0.001, following a polynomial decay strategy, a weight decay of 0.0001, and a batch size of 4 in the experiment.

Furthermore, in step (3), the mask feature f _t and the mask semantic score tensor s _t of the target domain instance t are specifically expressed as:

The mask feature f _t is the target domain backbone network feature multiplied by the instance segmentation mask of the target domain instance t, and the mask semantic score tensor s _t is the target domain semantic score tensor multiplied by the instance segmentation mask of the target domain instance t, s _t ∈ R ^W×H×K , K is the number of instances contained in the image, and W×H is the image size.

Furthermore, in step (6), the leaf nodes of the semantic tree are sampled, specifically:

For the semantic tree T _k constructed for category k, N instance segmentation masks are randomly selected based on the set sampling rate R The annotator quickly determines whether the selected instance segmentation mask is accurate. If the predicted instance segmentation mask _mt is accurate, it is marked with 1, and if it is inaccurate, it is marked with 0.

Furthermore, the present invention is implemented on the Pascal VOC 2012 dataset and the COCO dataset. The Pascal VOC2012 dataset consists of 1464 training images and 1449 verification images with source segmentation results. There are 20 categories in the Pascal VOC2012 dataset, and the indicator for evaluating the quality of the predicted segmentation mask is the mean Intersection over Union (mIoU); there are 80 categories in the COCO dataset, and the evaluation criteria are the average precision of the predicted box AP _box and the average precision of the mask AP _mask .

According to a second aspect of the present specification, a domain adaptive instance segmentation device based on weakly supervised learning is provided, comprising a memory and one or more processors, wherein the memory stores an executable code, and when the processor executes the executable code, it is used to implement the domain adaptive instance segmentation method based on weakly supervised learning as described in the first aspect.

According to a third aspect of this specification, a computer-readable storage medium is provided, on which a program is stored, characterized in that when the program is executed by a processor, the domain adaptive instance segmentation method based on weakly supervised learning as described in the first aspect is implemented.

Compared with the prior art, the present invention has the following beneficial effects:

1. Train the initial instance segmentation model in the source domain, and use it to perform instance segmentation in the target domain. Output the mask features and mask semantic score tensor of each instance in the target domain, build a semantic tree using the hierarchical agglomerative clustering method, and explore the appearance and semantic similarity between the predicted images in a hierarchical manner.

2. Sample the leaf nodes of the semantic tree to quickly determine whether the instance segmentation mask is accurate, use accurate samples to mask correct inaccurate samples, and fine-tune the initial instance segmentation model according to the mask correction results. This solves the problem that in terms of domain adaptation, although the segmentation model can be improved by introducing supervisory signals from the target domain dataset, manual labeling is cumbersome and time-consuming, and self-training contains too much noise in the pseudo-labels.

3. Experimental results on the Pascal VOC 2012 dataset and the COCO dataset show that compared with other advanced methods, the present invention spends limited human resources for label verification and achieves effectiveness close to that of supervised learning methods, which is quite competitive.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

FIG1 is a schematic diagram illustrating a domain adaptation problem provided by an embodiment of the present invention;

FIG2 is a schematic diagram of an overall framework of domain adaptive instance segmentation based on weakly supervised learning provided by an embodiment of the present invention;

FIG3 is a schematic diagram of constructing a semantic tree according to an embodiment of the present invention;

FIG4 is a schematic diagram of segmented output of an embodiment of the present invention on the Pascal VOC 2012 dataset;

FIG5 is a schematic diagram of segmented output of an embodiment of the present invention on the COCO dataset;

FIG6 is a structural block diagram of a domain adaptive instance segmentation device based on weakly supervised learning provided by an embodiment of the present invention.

DETAILED DESCRIPTION

In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

FIG1 is a schematic diagram of the domain adaptation problem. The present invention provides a domain adaptation instance segmentation method based on weakly supervised learning, which solves the problem that, in terms of domain adaptation, although the segmentation model can be improved by introducing supervisory signals from the target domain dataset, manual labeling is cumbersome and time-consuming, and self-training contains too much noise in the pseudo labels.

An embodiment of the present invention provides a domain adaptive instance segmentation method based on weakly supervised learning, as shown in FIG2 , and the method includes the following steps:

1. Train the initial segmentation model in the source domain

An initial instance segmentation model is trained on the source domain, the training dataset includes an image set {X _source } and a corresponding instance mask image set {Y _source }, and the backbone network features and a semantic score tensor are output. The semantic score tensor includes the probability that each pixel belongs to a different instance.

During the training phase, data augmentation includes horizontal/vertical flipping, translation, and scale variance.

The backbone network of the initial instance segmentation model is composed of swin transformer. The initial instance segmentation model is trained using the AdamW optimizer with an initial learning rate of 0.001, following a polynomial decay strategy, a weight decay of 0.0001, and a batch size of 4 in the experiment.

2. Apply the initial segmentation model to the target domain

The initial instance segmentation model trained in the source domain is used to perform instance segmentation on the target domain. The test dataset only includes the image set {X _tatget }, and the backbone network features and semantic score tensor corresponding to each image are output.

3. Extract target domain features

Take the maximum value on the instance dimension of the semantic score tensor obtained in step 2 to obtain the instance segmentation mask of each image in the target domain; multiply the instance segmentation mask of each image in the target domain with the target domain backbone network feature and the target domain semantic score tensor respectively to obtain the mask feature and mask semantic score tensor of each instance in the target domain;

The mask feature f _t and mask semantic score tensor s _t of the target domain instance t are specifically: the mask feature f _t is the target domain backbone network feature multiplied by the instance segmentation mask of the target domain instance t, and the mask semantic score tensor s _t is the target domain semantic score tensor multiplied by the instance segmentation mask of the target domain instance t, s _t ∈R ^W×H×K , K is the number of instances contained in the image, and W×H is the image size.

4. Splicing features

Concatenate the mask feature f t of instance _t obtained in step 3 and the mask semantic score tensor s _t to obtain the enhanced mask feature f _t ⁺ of instance t.

5. Constructing a semantic tree

As shown in Figure 3, hierarchical agglomerative clustering is used to construct the semantic tree corresponding to each category. The enhanced mask feature of each instance belonging to the category is regarded as a leaf node. Each time the two child nodes with the smallest Euclidean distance between the enhanced mask features of the instances are selected for merging to obtain a merged node. The enhanced mask feature and mask semantic score tensor of the merged node are respectively the linear combination of the enhanced mask feature and mask semantic score tensor corresponding to the child nodes; specifically:

The child nodes corresponding to instance t and the child nodes corresponding to instance o are merged to obtain the merged node n _j . The enhanced mask feature f _j ⁺ and mask semantic score tensor s _j of the merged node n _j are linear combinations of the enhanced mask features and mask semantic score tensors corresponding to the child nodes:

s _j = w _t s _t + w _o s _o

Among them, the weights w _t and w _o are related to the size of the child nodes:

Where _Pt and _Po are the number of instances contained in the corresponding child nodes; for leaf nodes, _wt = _wo = 1/2;

By merging nodes multiple times, a semantic tree corresponding to each category is finally constructed; the root node of the semantic tree is represented as n ₀ , and the remaining intermediate nodes are represented as where _Jk is the number of intermediate nodes of category k.

6. Sampling to quickly determine mask accuracy

For the semantic tree T _k constructed for category k, N instance segmentation masks are randomly selected based on the set sampling rate R The annotator quickly determines whether the selected instance segmentation mask is accurate. If the predicted instance segmentation mask _mt is accurate, it is marked with 1, and if it is inaccurate, it is marked with 0.

7. Mask correction

Compare the statistical value of the annotation results of all sampled instances on the semantic tree corresponding to category k with the set threshold: if the statistical value is greater than the threshold, it means that the prediction of category k is accurate, and the inaccurate sampled instances will be masked and corrected using the accurate sampled instances; if the statistical value is less than or equal to the threshold, it means that the prediction of category k is inaccurate, and the corresponding semantic tree is split into two subtrees, and each subtree resamples instances to calculate the statistical value of the annotation results, and then compares it with the designed threshold, and repeats the split-comparison process until the subtree cannot be split or the subtree does not contain any accurate sampled instances;

The calculation formula for the statistical value Q _k of the annotation results of all sampled instances on the semantic tree corresponding to category k is:

Wherein, N is the number of instances sampled for the semantic tree of category k, the instances are numbered as k ₁ ,…, _N , and _t is the judgment result of the instance segmentation mask in step 6.

8. Fine-tune the initial instance segmentation model

The initial instance segmentation model is fine-tuned according to the target domain mask correction results to improve the effectiveness of the instance segmentation model.

The present invention is implemented on the Pascal VOC 2012 dataset and the COCO dataset. The Pascal VOC 2012 dataset consists of 1464 training images and 1449 verification images with source segmentation results. There are 20 categories in the Pascal VOC 2012 dataset, and the indicator for evaluating the quality of the predicted segmentation mask is the mean Intersection over Union (mIoU); there are 80 categories in the COCO dataset, and the evaluation criteria are the average precision of the predicted box AP _box and the average precision of the mask AP _mask .

The applicant's experimental results on the Pascal VOC 2012 dataset and the COCO dataset are shown in Figures 4 and 5. The experimental results show that compared with other advanced methods, the present invention spends limited human resources on label verification and achieves effectiveness close to that of supervised learning methods, which is quite competitive.

Corresponding to the above-mentioned embodiment of the domain adaptive instance segmentation method based on weakly supervised learning, the present invention also provides an embodiment of a domain adaptive instance segmentation device based on weakly supervised learning.

Referring to FIG. 6 , an embodiment of the present invention provides a domain adaptive instance segmentation device based on weakly supervised learning, comprising a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, it is used to implement the domain adaptive instance segmentation method based on weakly supervised learning in the above embodiment.

The embodiment of the domain adaptive instance segmentation device based on weakly supervised learning of the present invention can be applied to any device with data processing capabilities, and the any device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located to read the corresponding computer program instructions in the non-volatile memory into the memory and run it. From the hardware level, as shown in Figure 6, it is a hardware structure diagram of any device with data processing capabilities where the domain adaptive instance segmentation device based on weakly supervised learning of the present invention is located. In addition to the processor, memory, network interface, and non-volatile memory shown in Figure 6, any device with data processing capabilities where the device in the embodiment is located can also include other hardware according to the actual function of the device with data processing capabilities, which will not be repeated.

The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

An embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the domain adaptive instance segmentation method based on weakly supervised learning in the above embodiment is implemented.

The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be an external storage device of any device with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capability. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.

It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "one", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and/or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

It should be understood that although the terms first, second, third, etc. may be used to describe various information in one or more embodiments of this specification, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

The above description is merely a preferred embodiment of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present specification shall be included in the scope of protection of one or more embodiments of the present specification.

Claims (10)

1. The domain self-adaptive instance segmentation method based on weak supervised learning is characterized by comprising the following steps of:

(1) Training an initial instance segmentation model on a source domain, and outputting backbone network characteristics and a semantic score tensor, wherein the semantic score tensor comprises probabilities of different instances to which each pixel belongs;

(2) Performing instance segmentation on a target domain by using an initial instance segmentation model obtained by source domain training, and outputting backbone network characteristics and semantic score tensors corresponding to each image;

(3) Taking the maximum value on the example dimension of the semantic score tensor obtained in the step (2) to obtain an example segmentation mask of each image of the target domain; multiplying the instance segmentation mask of each image of the target domain with the backbone network characteristics of the target domain and the semantic fraction tensor of the target domain respectively to obtain mask characteristics and mask semantic fraction tensor of each instance of the target domain;

(4) Splicing the mask features of the examples obtained in the step (3) with mask semantic score tensors to obtain enhanced mask features of the examples;

(5) Constructing a semantic tree corresponding to each category by using hierarchical condensation clustering, regarding the enhanced mask feature of each instance belonging to the category as a leaf node, and merging two child nodes with the minimum Euclidean distance of the enhanced mask feature among the selected instances each time to obtain a merged node, wherein the enhanced mask feature and the mask semantic score tensor of the merged node are respectively linear combinations of the enhanced mask feature and the mask semantic score tensor corresponding to the child nodes;

(6) For each semantic tree, sampling leaf nodes of the semantic tree based on a set sampling rate, rapidly judging whether an instance segmentation mask is accurate or not, and labeling a judgment result;

(7) Comparing the statistics of the labeling results of all sampling examples on the semantic tree corresponding to the category k with a set threshold value: if the statistical value is larger than the threshold value, the prediction of the category k is accurate, and an inaccurate sampling example can utilize the accurate sampling example to carry out mask correction; if the statistical value is less than or equal to the threshold value, the prediction of the specification class k is inaccurate, splitting the corresponding semantic tree into two subtrees, re-sampling the example calculation labeling result statistical value of each subtree, comparing with the design threshold value, and repeating the splitting-comparing process until the subtree is not detachable or does not contain any accurate sampling example;

(8) And fine-tuning the initial instance segmentation model according to the target domain mask correction result.

2. The domain adaptive instance segmentation method based on weak supervised learning as set forth in claim 1, wherein the step (5) specifically includes:

merging the child node corresponding to the instance t and the child node corresponding to the instance o to obtain a merging node n _j, wherein the enhanced mask feature f _j ⁺ and the mask semantic score tensor s _j of the merging node n _j are linear combinations of the enhanced mask feature and the mask semantic score tensor corresponding to the child node:

f_j ⁺＝w_tf_t ⁺+w_of_o ⁺

s_j＝w_ts_t+w_os_o

Wherein the weights w _t and w _o are related to the size of the child nodes:

Wherein, P _t and P _o are the number of examples contained in the corresponding child nodes respectively; for leaf nodes, w _t＝w_o =1/2;

and finally constructing a semantic tree corresponding to each category by condensing and merging the nodes for multiple times.

3. The method for domain adaptive instance segmentation based on weak supervised learning as set forth in claim 1, wherein in the step (7), a calculation formula of the statistics Q _k of labeling results of all sampling instances on the semantic tree corresponding to the category k is:

wherein N is the number of instances of sampling the semantic tree of the category k, and the number of instances k ₁,...,k_N,l_t is the judgment result of the instance segmentation mask in the step (6).

4. The domain adaptive instance segmentation method based on weak supervised learning as set forth in claim 1, wherein the backbone network of the initial instance segmentation model is composed of swin transformer; in the source domain, the training dataset includes an image set { X _source } and a corresponding instance mask image set { Y _source }; in the target domain, the test dataset includes only the image set { X _tatget }.

5. The domain adaptive instance segmentation method based on weak supervised learning as set forth in claim 1, wherein the initial instance segmentation model performs data enhancement during a training phase, including horizontal/vertical rollover, translation, and scale scaling; the initial example segmentation model was trained using AdamW optimizer with an initial learning rate of 0.001, following polynomial decay strategy, with a weight decay of 0.0001.

6. The method for domain adaptive instance segmentation based on weakly supervised learning as set forth in claim 1, wherein in step (3), the mask feature f _t and the mask semantic score tensor s _t of the target domain instance t are specifically:

the mask feature f _t is an instance partition mask of the target domain backbone network feature multiplied by the target domain instance t, the mask semantic score tensor s _t is an instance partition mask of the target domain semantic score tensor multiplied by the target domain instance t, s _t∈R^W×H×K, K is the number of instances contained in the image, and w×h is the image size.

7. The method for domain adaptive instance segmentation based on weakly supervised learning as set forth in claim 1, wherein in step (6), leaf nodes of the semantic tree are sampled, specifically:

for the semantic tree T _k constructed by category k, N instance segmentation masks are randomly selected based on the set sampling rate R The annotator quickly determines whether the selected instance segmentation mask is accurate, if the predicted instance segmentation mask m _t is accurate, 1 is annotated, and if not, 0 is annotated.

8. The method for domain-adaptive instance segmentation based on weakly supervised learning as recited in claim 1, wherein the Pascal VOC 2012 dataset and the COCO dataset are used as training sets for the instance segmentation model.

9. A weakly supervised learning based domain adaptive instance segmentation apparatus comprising a memory and one or more processors, the memory having executable code stored therein, wherein the processor, when executing the executable code, is configured to implement the weakly supervised learning based domain adaptive instance segmentation method as set forth in any of claims 1-8.

10. A computer-readable storage medium, on which a program is stored, which program, when being executed by a processor, implements a domain-adaptive instance segmentation method based on weakly supervised learning as set forth in any one of claims 1 to 8.