CN112434748A - Interactive data marking method under weak supervision environment - Google Patents

Interactive data marking method under weak supervision environment Download PDF

Info

Publication number
CN112434748A
CN112434748A CN202011392917.2A CN202011392917A CN112434748A CN 112434748 A CN112434748 A CN 112434748A CN 202011392917 A CN202011392917 A CN 202011392917A CN 112434748 A CN112434748 A CN 112434748A
Authority
CN
China
Prior art keywords
data
sample
samples
instruction
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011392917.2A
Other languages
Chinese (zh)
Inventor
董竹奔
刘胜蓝
刘相
孙焘
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dalian University of Technology
Original Assignee
Dalian University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dalian University of Technology filed Critical Dalian University of Technology
Priority to CN202011392917.2A priority Critical patent/CN112434748A/en
Publication of CN112434748A publication Critical patent/CN112434748A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2132—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on discrimination criteria, e.g. discriminant analysis
    • G06F18/21322—Rendering the within-class scatter matrix non-singular
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2137—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on criteria of topology preservation, e.g. multidimensional scaling or self-organising maps
    • G06F18/21375—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on criteria of topology preservation, e.g. multidimensional scaling or self-organising maps involving differential geometry, e.g. embedding of pattern manifold
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/40—Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2132—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on discrimination criteria, e.g. discriminant analysis
    • G06F18/21322—Rendering the within-class scatter matrix non-singular
    • G06F18/21328—Rendering the within-class scatter matrix non-singular involving subspace restrictions, e.g. nullspace techniques

Abstract

The invention relates to the technical field of data interaction, and provides an interactive data marking method under a weak supervision environment, which comprises the following steps: step 100: importing sample data which is arranged in advance into a data module; step 200: importing the sample data in the data module into a dimensionality reduction module and performing dimensionality reduction; step 300: the man-machine interface receives the dimensionality reduction data and displays the dimensionality reduction data through the interactive display interface; step 400: the man-machine interface carries out interaction behavior and finally generates a marking and interaction instruction result; step 500: the human-computer interface judges the instruction of the user; step 600: and the computing module receives the user instruction and the marking result transmitted by the human-computer interface, computes a data screening result, updates the sample data of the dimension reduction module and performs dimension reduction again. The invention can better realize data marking and classification and improve the performance of the model.

Description

Interactive data marking method under weak supervision environment
Technical Field
The invention relates to the technical field of data interaction, in particular to an interactive data marking method under a weak supervision environment.
Background
Interactive learning is a typical application of human-computer interaction, and provides a new idea for solving the marking problem. Namely, the data to be marked is visually displayed to the user by means of human-computer interaction, and the marking efficiency and accuracy are greatly improved by means of visual interaction. Interactive tagging is a typical application of visual interactive learning, which solves the problem of lacking tagging in classification, retrieval, etc. of image and video fields through interactive learning. The interactive marking greatly reduces the time cost of manual marking work, improves the working efficiency of data set marking, and provides possibility for large-scale automatic data set marking.
Marking is a problem that has been difficult to solve in computer vision. On one hand, for a computer, massive information and materials (such as pictures, texts, videos, music and the like) can become valuable sample information after being marked, and unmarked data cannot generate effective value for the computer; on the other hand, it is a huge, complex and laborious work for the user to undertake data tagging. For massive sample data, complete marking requires a huge labor cost. There is also a significant problem in the field of labeling, i.e., some data need to have knowledge of the relevant field to be labeled correctly, which requires experts in the relevant field, which further increases the difficulty of labeling data. How to mark efficiently and accurately is a problem to be solved urgently.
Disclosure of Invention
The invention mainly solves the technical problems of higher difficulty in marking data, huge and complex data marking workload in the prior art, and provides an interactive data marking method in a weak supervision environment so as to better realize data marking and classification and improve the model performance.
The invention provides an interactive data marking method under a weak supervision environment, which comprises the following processes:
step 100: importing sample data which is arranged in advance into a data module, wherein the sample data consists of a sample to be marked and a sample containing a mark;
step 200: importing the sample data in the data module into a dimensionality reduction module and performing dimensionality reduction;
step 300: the man-machine interface receives the dimensionality reduction data and displays the dimensionality reduction data through the interactive display interface;
step 400: the man-machine interface carries out interaction behavior and finally generates a marking and interaction instruction result;
step 500: the man-machine interface judges the instruction of the user, if the content of the instruction is successful, a confirmation instruction is generated, and the confirmation instruction and the data in the marking process are transmitted to the calculation module; if the content of the instruction is dimensionality reduction again, the sample data and the instruction of the user are transmitted into a data module;
step 600: the computing module receives a user instruction and a marking result transmitted by the human-computer interface, computes a data screening result, updates sample data of the dimensionality reduction module and performs dimensionality reduction again;
step 700: repeating the steps 200 to 600, and when the user instruction received by the computing module contains a confirmation instruction, storing and updating the data to the storage module; when the computing module receives an interactive instruction of a user and contains a termination instruction, the loop is terminated;
step 800: and after receiving the termination instruction, the computing module updates the data marked for the last time to the storage module, and the marking process is finished.
Further, the step 200: importing the sample data in the data module into a dimensionality reduction module, and carrying out dimensionality reduction, wherein the dimensionality reduction comprises the following processes:
given X ═ X(1),X(2),…,X(m)M samples, the optimization equation of the principal component analysis dimension reduction method is expressed by formula (1):
Figure BDA0002813340570000031
in formula (1), X ═ X(1)，X(2)，…，X(m)Denotes m samples, XprojectProjection matrix, Var (X), representing the dimensionality reduction of the samplesproject) Representing the element variance in a certain projection direction of the sample, and w represents a projection matrix;
determining the element variance Var (X) by using an optimization equation of a principal component analysis dimension reduction methodproject) The maximum projection matrix w.
Further, step 300: the man-machine interface receives the dimension reduction data, and the dimension reduction data is displayed by the interactive display interface, wherein the method comprises the following steps 301 to 303:
step 301: the marked sample and the unmarked sample in the sample are displayed in the same shape and size;
step 302: the unmarked samples are not colored, the marked samples are colored according to the sample types, the same type of samples are colored in the same color, and the different types of samples are colored in different colors;
step 303: adjusting the size of the sample to be marked, wherein the adjustment ranges from 1 to 4 times its original size.
Further, step 400: the man-machine interface carries out interactive action and finally generates a marking and interactive instruction result, and the method comprises the following steps of 401 to 402:
step 401: samples that are closer together under the same conditions are easily considered to be of the same category; a group of samples arranged in a closed shape is more easily considered to be of the same category; a set of samples arranged with first-order to high-order curvature continuity is easily considered as the same category;
step 402: marking from a point furthest from the center of the data sample; judging whether the marked sample and the unmarked sample have data overlapping condition through the following formula (2), if so, generating a re-visualized interactive instruction, otherwise, not generating;
dist(Wk，Wk+r)＞ζdordist(Wr，Wk+r)＞ζd (2)
wherein, dist (W)m，Wn) Represents the subspace distance, W, of the labeled and unlabeled sampleskRepresents the reduced dimensional subspace of k unlabeled samples, WrReduced dimensional subspace, ζ, representing r labeled samplesdIs a threshold parameter, 0 < ζd＜1，ζd0.8 as default; subspace distance dist (W) of marked and unmarked samplesm，Wn) The calculation is performed as in equation (3) as follows:
Figure BDA0002813340570000041
further, step 600: the calculation module receives a user instruction and a marking result transmitted by a human-computer interface, calculates a data screening result, updates sample data of the dimensionality reduction module, and performs dimensionality reduction again, and comprises the following steps of 601 to 603:
step 601: the calculation module receives the user instruction and the marking result, calculates the screening result of the data, performs secondary dimensionality reduction operation on the sample, and executes the step 602 to calculate if the number of the sample is greater than the dimensionality of the sample; if the number of samples is less than the dimension of the samples, executing step 603 to calculate;
step 602: calculating the dispersion matrix of the marked sample and the unmarked sample respectively by the following formula (4):
Figure BDA0002813340570000042
and then according to the dispersion matrix C of the marked samplesrAnd a dispersion matrix C of unlabeled sampleskComputing the reduced dimensional subspace W of the r marked samplesrAnd a reduced dimensional subspace W of k unlabeled samplesk。
Suppose Sk+r＝Sk∪SrAnd is
Figure BDA0002813340570000043
SkContains k unlabeled samples (S)k＝{x1，…，xk}) and S) of the baserContains r marked samples (S)r＝{xk+1，…，xk+r}); in addition, Ck，Ck+rAre respectively defined as Sk，Sk+rThe dispersion matrix of (2); the dispersion matrix C of such marked sampleskIt can be calculated by equation (4);
step 603: suppose that
Figure BDA0002813340570000044
Maximum d eigenvalues η1≥η2≥…≥ηdThe corresponding feature vector is defined as u1，u2，…，udThen, then
Figure BDA0002813340570000045
The eigenvectors corresponding to the largest d eigenvalues are calculated using the following formula (5):
Figure BDA0002813340570000046
further, step 700: repeating the steps 200 to 600, and when the user instruction received by the computing module contains a confirmation instruction, storing and updating the data to the storage module; when the computing module receives a termination instruction in the interactive instruction of the user, the loop is terminated, and the method comprises steps 701 to 702:
step 701: the sample data in the storage module is stored in a mode that one sample data corresponds to one sample mark;
step 702: after the sample data in the storage module receives the data for the last time and updates, the data is persisted in the disk as the result of the interactive mark.
The interactive data marking method under the weak supervision environment provided by the invention can select a proper subspace in the dimension reduction process based on the strong perception capability of human beings on the prevalence of bent and closed data; a proper dimensionality reduction subspace is found through multiple times of man-machine interaction, so that the marking and classifying functions can be better realized; by searching for the classification boundary, huge calculation overhead generated by a nonlinear dimension reduction method can be avoided, and the performance of the model is effectively ensured. The method is suitable for the fields of artificial auxiliary marking, semi-automatic marking, no-marking model training and the like.
Drawings
FIG. 1 is a block diagram of an application environment of an interactive data tagging method in a weakly supervised environment provided by the present invention;
FIG. 2 is a flow chart of an implementation of the interactive data tagging method in the weakly supervised environment provided by the present invention;
FIG. 3 is a human machine interface selection policy process;
FIG. 4 is a design rule of a pattern in a human interface;
FIG. 5 is a schematic diagram of human perceptual impact labeling;
FIG. 6 is a schematic diagram of a single interaction process;
FIG. 7 is a diagram illustrating a multi-interaction dimension reduction process;
FIG. 8 is a schematic diagram of a human machine interface selection strategy.
Detailed Description
In order to make the technical problems solved, technical solutions adopted and technical effects achieved by the present invention clearer, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some but not all of the relevant aspects of the present invention are shown in the drawings.
FIG. 1 is a block diagram of an application environment of an interactive data tagging method in a weakly supervised environment provided by the present invention; as shown in FIG. 1, the application environment of the present invention includes a data module Q1, a dimensionality reduction module Q2, a human-machine interface Q3, an interaction module Q4, a computation module Q5 and a storage module Q6.
The data module Q1 includes the data feature extraction and its preprocessing, where the samples include scatter, picture, music, video, etc. and the data is composed of a large number of samples to be marked and a small number of samples containing marks; extracting data features refers to calculating a set with higher abstraction degree from a sample, such as sift features, Hog features and depth convolution features of a picture; the preprocessing section performs a suitable scaling, e.g. normalization, of the different sample types so that their samples are in a standard positive distribution.
The dimension reduction module Q2 reads data from the data module Q1, and reduces data dimensions such as Principal Component Analysis (PCA) and t-SNE on the premise of keeping sample discriminant information as much as possible. In order to realize human-computer interaction, high-dimensional data needs to be reduced to a dimension which can be perceived by human, and the dimension is generally two-dimensional or three-dimensional.
The human-machine interface Q3 and the interaction module import visual data from the dimensionality reduction module Q2 to interact with the user, and the interaction process comprises the steps of reading data by the user and returning a result through interaction.
The calculation module Q5 reads the decision after the user interaction from the interaction module Q4 to judge whether the dimensionality reduction meets the separation requirement of human perception or not, namely whether the marking action is successfully completed or not; if the marking is successful, a new round of interaction is started, and if the marking is unsuccessful, the subspace is reselected for dimension reduction, and the steps are repeated until a markable result is obtained.
As shown in fig. 2, an interactive data tagging method in a weakly supervised environment provided by an embodiment of the present invention includes the following processes:
step 100: and importing the sample data which is sorted in advance into a data module (Q1), wherein the sample data consists of a large number of samples to be marked and a small number of samples containing marks.
Step 200: and importing the sample data in the data module into a dimension reduction module (Q2) and performing dimension reduction.
The dimension reduction method in the step can adopt principal component analysis or t-SNE (t-distributed stored geometric neighbor embedding algorithm). Specifically, in step 200, the following method is used to perform dimension reduction on the sample data:
given X ═ X(1)，X(2)，…，X(m)M samples, the optimization equation of the principal component analysis dimension reduction method is expressed by formula (1):
Figure BDA0002813340570000071
in formula (1), X ═ X(1)，X(2)，…，X(m)Denotes m samples, XprojectProjection matrix, Var (X), representing the dimensionality reduction of the samplesproject) The element variance in a certain projection direction of the sample is represented, and w represents the projection matrix.
Determining elements by using an optimization equation of a principal component analysis dimension reduction methodVariance Var (X)project) The maximum projection matrix w.
Step 300: the man-machine interface (Q3) receives the dimension reduction data and displays the dimension reduction data through the interactive display interface.
The visualized sample of step 300 is displayed through a human-machine interface (Q3), and referring to the human-machine interface marking strategy of fig. 3, the method comprises the following sub-steps:
step 301: the labeled samples in the sample are shown in the same shape, size as the unlabeled samples.
Step 302: the unlabeled samples are not colored, while the labeled samples are colored according to the sample types, the same type of samples are colored in the same color, and different types of samples are colored in different colors.
Step 303: adjusting the size of the sample to be marked, wherein the adjustment ranges from 1 to 4 times its original size.
Step 400: the man-machine interface (Q3) carries out interactive action and finally generates marking and interactive instruction results.
Based on the principle of a lattice tower, the step carries out human-computer interaction pattern design; the interaction is based on a human perception and interaction tagging policy, and the interaction is influenced by human perception in step 400 in the following manner:
referring to fig. 5, an example of a pattern of the human-computer interaction interface, step 401: samples that are closer under the same conditions are easily considered by the user as the same category; a group of samples arranged in a closed shape is more easily considered by a user to be the same category; a set of samples arranged with first-order to high-order curvature continuity is easily considered to be of the same class.
The step 400 of interaction behavior is influenced by an interaction tagging policy, which comprises the following steps:
step 402: marking from a point furthest from the center of the data sample; judging whether the marked sample and the unmarked sample have data overlapping condition through the following formula (2), if so, generating a re-visualized interactive instruction, otherwise, not generating;
dist(Wk，Wk+r)＞ζdordist(Wr，Wk+r)＞ζd (2)
wherein, dist (W)m，Wn) Represents the subspace distance, W, of the labeled and unlabeled sampleskRepresents the reduced dimensional subspace of k unlabeled samples, WrReduced dimensional subspace, ζ, representing r labeled samplesdIs a threshold parameter, 0 < ζd＜1，ζd0.8 as default; subspace distance dist (W) of marked and unmarked samplesm，Wn) The calculation is performed as in equation (3) as follows:
Figure BDA0002813340570000081
step 500: the man-machine interface (Q3) judges the instruction of the user, if the content of the instruction is successful, a confirmation instruction is generated, and the confirmation instruction and the data in the current marking process are transmitted to the calculation module (Q5); if the content of the instruction is to be dimension reduced again, the sample data and the user's instruction are passed into the data module (Q1).
Step 600: and after receiving the user command and the marking result transmitted by the human-computer interface (Q3), the computing module (Q4) computes the data screening result, updates the sample data of the dimension reduction module (Q2) and performs dimension reduction again. The method specifically comprises the following steps 601 to 603:
step 601: after receiving the user instruction and the marking result, the calculating module (Q4) calculates the screening result of the data, performs secondary dimensionality reduction operation on the sample, and if the number of the sample is greater than the dimensionality of the sample, executes the step 602 to calculate; if the number of samples is less than the sample dimension, step 603 is performed for calculation.
The user instruction is an instruction for confirming the selection of the frame on the interactive interface by the user. And (3) calculating the screening result of the data: and (3) judging whether the subspace selection has overlapping through a formula (1) and a formula (2), if so, reducing the dimension again, and if not, starting the next interaction period.
Step 602: calculating the dispersion matrix of the marked sample and the unmarked sample respectively by the following formula (4):
Figure BDA0002813340570000091
and then according to the dispersion matrix C of the marked samplesrAnd a dispersion matrix C of unlabeled sampleskComputing the reduced dimensional subspace W of the r marked samplesrAnd a reduced dimensional subspace W of k unlabeled samplesk。
The formula (4) is defined as follows. Suppose Sk+r＝Sk∪SrAnd is
Figure BDA0002813340570000092
SkContains k unlabeled samples (S)k＝{x1，…，xk}) and S) of the baserContains r marked samples (S)r＝{xk+1，…，xk+r}). In addition, Ck，Ck+rAre respectively defined as Sk，Sk+rThe dispersion matrix of (2). The dispersion matrix C of such marked sampleskIt can be calculated by equation (4).
Step 603: suppose that
Figure BDA0002813340570000093
Maximum d eigenvalues η1≥η2≥…≥ηdThe corresponding feature vector is defined as u1，u2，…，udThen, then
Figure BDA0002813340570000094
The eigenvectors corresponding to the largest d eigenvalues are calculated using the following formula (5):
Figure BDA0002813340570000095
step 700: repeating steps 200 to 600. When the user command received by the computing module (Q5) contains a confirmation command, the data is saved and updated to the storage module (Q6); when the computation module (Q5) receives the user's interactive command with a termination command, the loop terminates.
Step 701: the sample data in the storage module (Q6) is stored in a form that one sample data corresponds to one sample mark.
This storage may be partially updated. When the computing module transmits part of the data and the label, the storage module updates the part of the data and the label, and keeps other data not updated.
Step 702: after the sample data in the storage module receives the data for the last time and updates, the data is persisted in the disk as the result of the interactive mark.
Step 800: after the computation module (Q5) receives the termination command, the last marked data is updated to the storage module (Q6), and the marking process is ended.
The following distances illustrate the present embodiment:
referring to fig. 3, a decision process of the human-machine interface Q3 in fig. 1 is shown.
The marking starts from the point furthest away from the center of the data sample. The process of tagging is performed at the direction of the user.
Judging whether data overlapping exists between the marked samples and the unmarked samples through the formula (2), if so, generating an interactive instruction for re-visualization, otherwise, not generating.
0＜ζd＜1，ζd0.8 as a default value. dist (W)m，Wn) The calculation method of (c) is shown in formula (3).
Referring to fig. 4, the design principle for the patterns in the human-machine interface described in steps 301, 302 and 303 is as follows:
principle 1 (corresponding to a): all samples are shown in one and the same shape (circle).
Principle 2 (corresponding to a): the marked samples were randomly colored to a completely different color, and the unmarked samples remained uncolored.
Principle 3 (corresponding to b, c, d): the size of the unlabeled samples is initialized according to the amount of data prevalence. Additionally, the size of the marked sample can be resized at any time during the interaction, with the scaling range between 1-4 times the size of the unmarked sample.
Referring to fig. 5, a specific description of the human perception principle with respect to steps 401, 402 and 403 is provided.
First, a2 consists of a series of consecutive dots. Following the principle of continuity in accordance with 403 human perception, the continuous portion tends to be seen as a whole (A)2). Next, data prevalence A follows the proximity principle according to 401 human perception1Is incorporated into A2. After the first two steps, group A (A)1，A2) Are determined under the rules of the good continuity principle and the proximity principle, and unmarked points in group a may be marked as existing marks. Another part, B1、B2And B3Can be classified into group B for the following reasons. Following the proximity principle according to 401 human perception, B1、B2And B3In close proximity. Furthermore, according to 402 human perception following the principle of closeness, B1、B2And B3The data population of the composition approximates a closed ellipse.
FIG. 6 is a detailed illustration of a single dimensionality reduction interaction of the human-machine interface Q3: the user opens the interactive system and the machine presents the results to the user in a visual scatter manner via the data module Q1 and the dimension reduction module Q2. The user observes the visualization distribution result of the data popularity (a) makes his own decision and uses the circle selection tool (b) to select the right neighborhood. The selected points are then observed in the original feature space (c). Finally, the selection (d) of this round is confirmed and the result is passed to the calculation module 105.
FIG. 7 is a detailed illustration of the multiple interaction dimension reduction process. Is formed by the single interactive dimensionality reduction iteration shown in fig. 6, and the specific process is as follows:
a and b: the user opens the interactive system, and the machine displays the result in front of the user in a visual scatter mode through calculation and dimension reduction.
c: and the user feeds back the decision to the system. However, when the system notices that the number of the samples selected by the user exceeds two, the system passes the criterion condition dist (W)k，Wk+r)＞ξd or dist(Wr，Wk+r)＞ξdJudging that the step is a step of multiple dimensionality reduction.
d: the system performs secondary dimensionality reduction calculation and returns a secondary dimensionality reduction result to the user. Note that the sample is now a two-class dimension reduction result, with no user labeling.
e: and marking the result of the secondary dimension reduction by the user. At the moment, the two types of dimension reduction results already meet the separation requirement of human perception, and marking work can be carried out.
f: the system feeds back the result of the secondary marking to the user.
FIG. 8 is a detailed illustration of the selection policy for the human machine interface as follows:
the reduced dimensional subspace of the selected portion (set C, D) is denoted Wr and the reduced dimensional subspace of the unselected portion (set A, B) is denoted Wk. First we judge dist (W)r，Wk+r)＜ξdIt means that the subspace Wk + r is compatible with the subspace Wr for the selected portion. Next, we judge dist (W)r，Wk+r)＞ξdUnfortunately, the unselected portions of the subspace Wk + r are incompatible with the subspace Wr. In this case, the unselected portions need to be further interactively reduced in dimension. Since the number of samples is significantly larger than the current number of dimensions, the subspace is calculated by equation (3).
And calculating the dispersion matrix of the marked sample and the unmarked sample respectively through formula (4).
Finally, the group A and the group B are mapped in a new subspace under the calculation of secondary dimensionality reduction, and the next selection process is facilitated.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: modifications of the technical solutions described in the embodiments or equivalent replacements of some or all technical features may be made without departing from the scope of the technical solutions of the embodiments of the present invention.

Claims (6)

1. An interactive data marking method under a weak supervision environment is characterized by comprising the following processes:
step 100: importing sample data which is arranged in advance into a data module, wherein the sample data consists of a sample to be marked and a sample containing a mark;
step 200: importing the sample data in the data module into a dimensionality reduction module and performing dimensionality reduction;
step 300: the man-machine interface receives the dimensionality reduction data and displays the dimensionality reduction data through the interactive display interface;
step 400: the man-machine interface carries out interaction behavior and finally generates a marking and interaction instruction result;
step 500: the man-machine interface judges the instruction of the user, if the content of the instruction is successful, a confirmation instruction is generated, and the confirmation instruction and the data in the marking process are transmitted to the calculation module; if the content of the instruction is dimensionality reduction again, the sample data and the instruction of the user are transmitted into a data module;
step 600: the computing module receives a user instruction and a marking result transmitted by the human-computer interface, computes a data screening result, updates sample data of the dimensionality reduction module and performs dimensionality reduction again;
step 700: repeating the steps 200 to 600, and when the user instruction received by the computing module contains a confirmation instruction, storing and updating the data to the storage module; when the computing module receives an interactive instruction of a user and contains a termination instruction, the loop is terminated;
step 800: and after receiving the termination instruction, the computing module updates the data marked for the last time to the storage module, and the marking process is finished.
2. The interactive data tagging method in a weakly supervised environment as recited in claim 1, wherein the step 200: importing the sample data in the data module into a dimensionality reduction module, and carrying out dimensionality reduction, wherein the dimensionality reduction comprises the following processes:
given X ═ X(1),X(2),…,X(m)M samples, the optimization equation of the principal component analysis dimension reduction method is expressed by formula (1):
Figure FDA0002813340560000011
in formula (1), X ═ X(1),X(2),…,X(m)Denotes m samples, XprojectProjection matrix, Var (X), representing the dimensionality reduction of the samplesproject) Representing the element variance in a certain projection direction of the sample, and w represents a projection matrix;
determining the element variance Var (X) by using an optimization equation of a principal component analysis dimension reduction methodproject) The maximum projection matrix w.
3. The interactive data tagging method in a weakly supervised environment as recited in claim 2, wherein step 300: the man-machine interface receives the dimension reduction data, and the dimension reduction data is displayed by the interactive display interface, wherein the method comprises the following steps 301 to 303:
step 301: the marked sample and the unmarked sample in the sample are displayed in the same shape and size;
step 302: the unmarked samples are not colored, the marked samples are colored according to the sample types, the same type of samples are colored in the same color, and the different types of samples are colored in different colors;
step 303: adjusting the size of the sample to be marked, wherein the adjustment ranges from 1 to 4 times its original size.
4. The interactive data tagging method in a weakly supervised environment as recited in claim 3, wherein step 400: the man-machine interface carries out interactive action and finally generates a marking and interactive instruction result, and the method comprises the following steps of 401 to 402:
step 401: samples that are closer together under the same conditions are easily considered to be of the same category; a group of samples arranged in a closed shape is more easily considered to be of the same category; a set of samples arranged with first-order to high-order curvature continuity is easily considered as the same category;
step 402: marking from a point furthest from the center of the data sample; judging whether the marked sample and the unmarked sample have data overlapping condition through the following formula (2), if so, generating a re-visualized interactive instruction, otherwise, not generating;
dist(Wk,Wk+r)>ζdordist(Wr,Wk+r)>ζd (2)
wherein, dist (W)m,Wn) Represents the subspace distance, W, of the labeled and unlabeled sampleskRepresents the reduced dimensional subspace of k unlabeled samples, WrReduced dimensional subspace, ζ, representing r labeled samplesdIs a threshold parameter, 0<ζd<1，ζd0.8 as default; subspace distance dist (W) of marked and unmarked samplesm,Wn) The calculation is performed as in equation (3) as follows:
Figure FDA0002813340560000031
5. the interactive data tagging method under a weakly supervised environment as recited in claim 4, wherein step 600: the calculation module receives a user instruction and a marking result transmitted by a human-computer interface, calculates a data screening result, updates sample data of the dimensionality reduction module, and performs dimensionality reduction again, and comprises the following steps of 601 to 603:
step 601: the calculation module receives the user instruction and the marking result, calculates the screening result of the data, performs secondary dimensionality reduction operation on the sample, and executes the step 602 to calculate if the number of the sample is greater than the dimensionality of the sample; if the number of samples is less than the dimension of the samples, executing step 603 to calculate;
step 602: calculating the dispersion matrix of the marked sample and the unmarked sample respectively by the following formula (4):
Figure FDA0002813340560000032
and then according to the dispersion matrix C of the marked samplesrAnd a dispersion matrix C of unlabeled sampleskComputing the reduced dimensional subspace W of the r marked samplesrAnd a reduced dimensional subspace W of k unlabeled samplesk；
Suppose Sk+r＝Sk∪SrAnd is
Figure FDA0002813340560000033
SkContains k unlabeled samples (S)k＝{x1,…,xk}) and S) of the baserContains r marked samples (S)r＝{xk+1,…,xk+r}); in addition, Ck,Ck+rAre respectively defined as Sk,Sk+rThe dispersion matrix of (2); the dispersion matrix C of such marked sampleskIt can be calculated by equation (4);
step 603: suppose that
Figure FDA0002813340560000034
Maximum d eigenvalues η1≥η2≥…≥ηdThe corresponding feature vector is defined as u1,u2,…,udThen, then
Figure FDA0002813340560000035
The eigenvectors corresponding to the largest d eigenvalues are calculated using the following formula (5):
Figure FDA0002813340560000036
6. the interactive data tagging method in a weakly supervised environment as recited in claim 5, wherein step 700: repeating the steps 200 to 600, and when the user instruction received by the computing module contains a confirmation instruction, storing and updating the data to the storage module; when the computing module receives a termination instruction in the interactive instruction of the user, the loop is terminated, and the method comprises steps 701 to 702:
step 701: the sample data in the storage module is stored in a mode that one sample data corresponds to one sample mark;
step 702: after the sample data in the storage module receives the data for the last time and updates, the data is persisted in the disk as the result of the interactive mark.
CN202011392917.2A 2020-12-03 2020-12-03 Interactive data marking method under weak supervision environment Pending CN112434748A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011392917.2A CN112434748A (en) 2020-12-03 2020-12-03 Interactive data marking method under weak supervision environment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011392917.2A CN112434748A (en) 2020-12-03 2020-12-03 Interactive data marking method under weak supervision environment

Publications (1)

Publication Number Publication Date
CN112434748A true CN112434748A (en) 2021-03-02

Family

ID=74690705

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011392917.2A Pending CN112434748A (en) 2020-12-03 2020-12-03 Interactive data marking method under weak supervision environment

Country Status (1)

Country Link
CN (1) CN112434748A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113095427A (en) * 2021-04-23 2021-07-09 中南大学 High-dimensional data analysis method and face data analysis method based on user guidance

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180356949A1 (en) * 2017-06-12 2018-12-13 Royal Bank Of Canada System and method for adaptive data visualization
CN110287978A (en) * 2018-03-19 2019-09-27 国际商业机器公司 For having the computer implemented method and computer system of the machine learning of supervision

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180356949A1 (en) * 2017-06-12 2018-12-13 Royal Bank Of Canada System and method for adaptive data visualization
CN110287978A (en) * 2018-03-19 2019-09-27 国际商业机器公司 For having the computer implemented method and computer system of the machine learning of supervision

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
SHENGLAN LIU 等: ""Neural Method for Explicit Mapping of Weighted Locally Linear Embedding in Image Retrieval"", 《IEEE》 *
SHENGLAN LIU 等: ""Perceptual Visual Interactive Learning"", 《ARXIV》, pages 3 *
冯林 等: ""基于最长公共子序列的人体运动序列检索"", 《系统仿真学报》, vol. 21, no. 22 *
刘胜蓝: ""余弦度量下的高维数据降维及分类方法研究"", 《中国博士学位论文全文数据库 信息科技辑》 *
李珂: ""基于草图的精细图像检索"", 《中国博士学位论文全文数据库 信息科技辑》 *
高飞 等: ""基于样本类别确定度的半监督分类"", 《北京航空航天大学学报》, vol. 44, no. 9 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113095427A (en) * 2021-04-23 2021-07-09 中南大学 High-dimensional data analysis method and face data analysis method based on user guidance

Similar Documents

Publication Publication Date Title
Tabernik et al. Deep learning for large-scale traffic-sign detection and recognition
CN109460704B (en) Fatigue detection method and system based on deep learning and computer equipment
CN103020971A (en) Method for automatically segmenting target objects from images
CN103208007B (en) Face recognition method based on support vector machine and genetic algorithm
CN102436637B (en) Method and system for automatically segmenting hairs in head images
CN108846359A (en) It is a kind of to divide the gesture identification method blended with machine learning algorithm and its application based on skin-coloured regions
Park et al. Attributed grammars for joint estimation of human attributes, part and pose
CN104751175A (en) Multi-label scene classification method of SAR (Synthetic Aperture Radar) image based on incremental support vector machine
CN112434748A (en) Interactive data marking method under weak supervision environment
CN104573701B (en) A kind of automatic testing method of Tassel of Corn
Marčetić et al. Deformable part-based robust face detection under occlusion by using face decomposition into face components
Tanguay Hidden Markov models for gesture recognition
CN113345052B (en) Classification data multi-view visualization coloring method and system based on similarity significance
CN110659563A (en) Power grid dispatching drawing automatic identification method, system and medium based on improved YOLOv3 network
CN112446417B (en) Spindle-shaped fruit image segmentation method and system based on multilayer superpixel segmentation
CN113269089A (en) Real-time gesture recognition method and system based on deep learning
CN116945258A (en) Die cutting machine control system and method thereof
Wu et al. Rethinking Image Forgery Detection via Contrastive Learning and Unsupervised Clustering
Do et al. Kernel methods and visualization for interval data mining
CN110135363A (en) Based on differentiation dictionary insertion pedestrian image search method, system, equipment and medium
JP5413156B2 (en) Image processing program and image processing apparatus
CN111553195B (en) Three-dimensional face shielding discrimination method based on multi-bitmap tangent plane and multi-scale uLBP
CN108898188A (en) A kind of image data set aid mark system and method
JP5092458B2 (en) Map generating apparatus, map manufacturing method, and program
Mörzinger et al. Classifying Patent Images.

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination