CN111400581A - System, method and apparatus for annotating samples - Google Patents
System, method and apparatus for annotating samples Download PDFInfo
- Publication number
- CN111400581A CN111400581A CN202010174981.7A CN202010174981A CN111400581A CN 111400581 A CN111400581 A CN 111400581A CN 202010174981 A CN202010174981 A CN 202010174981A CN 111400581 A CN111400581 A CN 111400581A
- Authority
- CN
- China
- Prior art keywords
- webpage
- labeling
- annotation
- user
- labeled
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000002372 labelling Methods 0.000 claims abstract description 251
- 230000000977 initiatory effect Effects 0.000 claims abstract description 156
- 230000004044 response Effects 0.000 claims description 47
- 239000003999 initiator Substances 0.000 claims description 19
- 238000012790 confirmation Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 9
- 238000001514 detection method Methods 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 13
- 230000008569 process Effects 0.000 description 9
- 238000004891 communication Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000000644 propagated effect Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Embodiments of the present disclosure disclose systems, methods, and apparatuses for annotating samples. One embodiment of the system comprises: the annotation task initiating terminal is configured to respond to the receiving of an acquisition request of a first user for the annotation task initiating webpage, and acquire the annotation task initiating webpage from the server terminal; presenting an annotation task initiating webpage; the server is configured to acquire marking tool information and a to-be-marked sample set, wherein the marking tool information and the to-be-marked sample set are input by a first user through a marking task initiation webpage; generating a webpage for labeling based on the labeling tool information and the sample set to be labeled; the annotation task execution end is configured to respond to the receiving of the acquisition request of the second user for the webpage for annotation, and acquire the webpage for annotation from the server end; and presenting the webpage for annotation. The embodiment is beneficial to improving the application range of the labeling system and improving the safety of the sample to be labeled.
Description
Technical Field
Embodiments of the present disclosure relate to the field of computer technologies, and in particular, to a system, method, and apparatus for annotating a sample.
Background
In recent years, artificial intelligence technology has been developed, and the amount of training samples required for training algorithms has been increasing. The training sample is data obtained after the marking personnel marks the data collected in advance.
Currently, data annotation is generally performed by using an offline annotation tool. Offline annotation tools are typically in C/S architecture (client-server) mode. The marking personnel need to install client software on the marking equipment, and then acquire the data to be marked from the server side by using the client software and mark the data.
Disclosure of Invention
Embodiments of the present disclosure propose systems, methods, and apparatuses for annotating samples.
In a first aspect, embodiments of the present disclosure provide a system for annotating a sample, the system comprising: the system comprises an annotation task initiating end, a server end and an annotation task executing end, wherein the annotation task initiating end is configured to: the method comprises the steps that in response to receiving an acquisition request of a first user for an annotation task initiation webpage, the annotation task initiation webpage is acquired from a server side; presenting an annotation task initiating webpage; the server is configured to: acquiring marking tool information and a to-be-marked sample set input by a first user through a marking task initiating webpage; generating a webpage for labeling based on the information of the labeling tool and the sample set to be labeled, wherein the webpage for labeling comprises the sample to be labeled in the sample set to be labeled and the labeling tool indicated by the information of the labeling tool, and the labeling tool comprises a label; the annotation task execution end is configured to: acquiring a webpage for marking from a server in response to receiving an acquisition request of a second user for the webpage for marking; and presenting the webpage for annotation.
In some embodiments, the server is further configured to: and obtaining a labeled sample obtained after a second user labels the sample to be labeled in the webpage for labeling by using a labeling tool in the webpage for labeling.
In some embodiments, the server is further configured to: generating a labeled webpage comprising a labeled sample; and the annotation task initiating end is further configured to: acquiring the marked webpage from the server in response to receiving an acquisition request of the first user for the marked webpage; and presenting the marked webpage.
In some embodiments, the server is further configured to: acquiring an initial labeled sample obtained after a second user labels a sample to be labeled in a webpage for labeling by using a labeling tool in the webpage for labeling; generating an initial labeled webpage comprising the initial labeled sample; and the annotation task execution end is further configured to: in response to receiving an acquisition request of a second user for the initially labeled webpage, acquiring the initially labeled webpage from the server; presenting the initially labeled webpage; and the server is further configured to: and in response to the detection of the confirmation operation of the second user on the initial labeled sample in the initial labeled webpage, determining the initial labeled sample confirmed by the second user as a labeled sample.
In some embodiments, the server is further configured to: acquiring initial labeling tool information and sample type information input by a first user through a labeling task initiating webpage; acquiring a sample for previewing from a preset sample set, wherein the sample type of the sample for previewing is the same as the sample type indicated by the sample type information; generating a webpage for previewing based on the initial marking tool information and the sample for previewing; and the annotation task initiating end is further configured to: acquiring a webpage for previewing from a server in response to receiving an acquisition request of a first user for the webpage for previewing; presenting a webpage for previewing, wherein the webpage for previewing comprises an initial marking tool indicated by the initial marking tool information; and the server is further configured to: and in response to detecting the confirmation operation of the first user for the initial marking tool in the webpage for previewing, determining the initial marking tool information as the marking tool information.
In some embodiments, the server is further configured to: acquiring annotation task execution end information input by a first user through an annotation task initiating webpage; responding to an acquisition request aiming at a webpage for labeling sent by a labeling task execution end; determining whether the annotation task execution end sending the acquisition request is the annotation task execution end indicated by the annotation task execution end information; and responding to the fact that the annotation task execution end sending the acquisition request is the annotation task execution end indicated by the annotation task execution end information, and sending the webpage for annotation to the annotation task execution end sending the acquisition request.
In some embodiments, the annotation tool indicated by the annotation tool information includes more than two levels of labels.
In some embodiments, the type of the sample to be labeled in the set of samples to be labeled comprises at least one of: image class, text class, voice class, video class.
In a second aspect, embodiments of the present disclosure provide a method for annotating a sample, the method comprising: in response to receiving an acquisition request which is sent by a first user by using a labeling task initiating terminal and aims at a labeling task initiating webpage, sending the labeling task initiating webpage to the labeling task initiating terminal so that the labeling task initiating terminal can present the labeling task initiating webpage to the first user; acquiring marking tool information and a to-be-marked sample set input by a first user through a marking task initiating webpage; generating a webpage for labeling based on the information of the labeling tool and the sample set to be labeled, wherein the webpage for labeling comprises the sample to be labeled in the sample set to be labeled and the labeling tool indicated by the information of the labeling tool, and the labeling tool comprises a label; and in response to receiving an acquisition request aiming at the webpage for labeling sent by the second user by using the labeling task execution end, sending the webpage for labeling to the labeling task execution end so that the labeling task execution end can present the webpage for labeling to the second user.
In a third aspect, embodiments of the present disclosure provide an apparatus for annotating a sample, the apparatus comprising: the system comprises a first sending unit, a second sending unit and a third sending unit, wherein the first sending unit is configured to respond to the receiving of an acquisition request which is sent by a first user by utilizing an annotation task initiating terminal and aims at an annotation task initiating webpage, and send the annotation task initiating webpage to the annotation task initiating terminal so that the annotation task initiating terminal can present the annotation task initiating webpage to the first user; the system comprises an acquisition unit, a processing unit and a display unit, wherein the acquisition unit is configured to acquire marking tool information and a to-be-marked sample set which are input by a first user through a marking task initiating webpage; the generating unit is configured to generate a webpage for labeling based on the labeling tool information and the to-be-labeled sample set, wherein the webpage for labeling comprises the to-be-labeled sample in the to-be-labeled sample set and a labeling tool indicated by the labeling tool information, and the labeling tool comprises a label; and the second sending unit is configured to respond to the receiving of the acquisition request which is sent by the second user by utilizing the labeling task execution end and aims at the webpage for labeling, and send the webpage for labeling to the labeling task execution end so that the labeling task execution end can present the webpage for labeling to the second user.
In a fourth aspect, an embodiment of the present disclosure provides a server, including: one or more processors; a storage device having one or more programs stored thereon which, when executed by one or more processors, cause the one or more processors to implement the method for labeling a sample described above.
In a fifth aspect, embodiments of the present disclosure provide a computer-readable medium on which a computer program is stored, which when executed by a processor, implements the above-described method for annotating a sample.
According to the system, the method and the device for marking the sample, provided by the embodiment of the disclosure, the marking task initiating end responds to the received request of the first user for the marking task initiating webpage, the marking task initiating webpage is obtained from the server end, and the marking task initiating webpage is presented; then, the server side obtains marking tool information and a to-be-marked sample set which are input by a first user through a marking task initiating webpage, and generates a to-be-marked webpage based on the marking tool information and the to-be-marked sample set, wherein the to-be-marked webpage comprises a to-be-marked sample in the to-be-marked sample set and a marking tool indicated by the marking tool information, and the marking tool comprises a label; then, the annotation task execution end responds to the received acquisition request of the second user for the annotation webpage, acquires the annotation webpage from the server and presents the annotation webpage, so that the annotation task initiating end and the annotation task execution end in the method only need to install browsers for accessing the webpage managed by the server, the initiation of the annotation task and the execution of the annotation task can be realized, compared with a C/S architecture mode in the prior art, the method does not need to limit operating systems of the annotation task initiating end and the annotation task execution end, and is beneficial to improving the application range of the annotation system; in addition, in the labeling process, the labeling task execution end only needs to add labels on the samples to be labeled in the webpage for labeling, and the labeling task execution end does not need to download the samples to be labeled, so that the propagation of the samples to be labeled is controlled, and the safety of the samples to be labeled is improved.
Drawings
Other features, objects and advantages of the disclosure will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;
FIG. 2 is a timing diagram of one embodiment of a system for annotating samples according to the present application;
FIG. 3 is a schematic diagram of one application scenario of a system for annotating a specimen according to the present application;
FIG. 4 is a flow diagram of one embodiment of a method for annotating a sample according to the present application;
FIG. 5 is a schematic structural diagram of one embodiment of an apparatus for labeling samples according to the present application;
FIG. 6 is a schematic block diagram of a computer system suitable for use in implementing a server according to embodiments of the present application.
Detailed Description
The present disclosure is described in further detail below with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that, in the present disclosure, the embodiments and features of the embodiments may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 illustrates an exemplary system architecture 100 to which embodiments of a system for annotating a sample, a method for annotating a sample, or an apparatus for annotating a sample of the present disclosure can be applied.
As shown in FIG. 1, the system architecture 100 can include an annotation task initiating device 101, a network 102, a server 103, a network 104, and an annotation task performing device 105. The network 102 serves as a medium to provide a communication link between the annotation task initiating device 101 and the server 103. The network 104 serves as a medium to provide a communication link between the annotation task execution device 105 and the server 103. The networks 102, 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The annotation task initiator can use the annotation task initiator 101 to interact with the server 103 over the network 102. Similarly, an annotated task performer can interact with the server 103 via the network 104 using the annotated task performing device 105. The annotation task initiating device 101 and the annotation task executing device 105 can have various communication client applications installed thereon, such as a web browser application, a shopping application, a search application, an instant messaging tool, a mailbox client, social platform software, and the like.
The annotation task initiator 101 and the annotation task executor 105 may be hardware or software, and when the annotation task initiator 101 and the annotation task executor 105 are hardware, they may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio L layer III, mpeg Audio layer 3), MP4(Moving Picture Experts Group Audio L layer IV, mpeg Audio layer 4) players, laptop portable computers, desktop computers, and the like.
The server 103 may be a server providing various services, such as a background web server providing support for web pages displayed on the annotation task initiator 101 and the annotation task executor 105. The background web server may analyze and process the received data such as the web page obtaining request, and feed back the processing result (e.g., the web page) to the annotation task initiator 101 and the annotation task executor 105.
It should be noted that the method for labeling samples provided by the embodiments of the present disclosure is generally performed by the server 103, and accordingly, the apparatus for labeling samples is generally disposed in the server 103.
The server may be hardware or software. When the server is hardware, it may be implemented as a distributed server cluster formed by multiple servers, or may be implemented as a single server. When the server is software, it may be implemented as multiple pieces of software or software modules (e.g., multiple pieces of software or software modules used to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.
It should be understood that the numbers of annotation task initiating devices, annotation task performing devices, networks, and servers in FIG. 1 are merely illustrative. Any number of annotation task initiating devices, annotation task executing devices, networks, and servers can be provided, as desired for implementation.
Referring to FIG. 2, a timing diagram 200 of one embodiment of a system for annotating a specimen according to the present application is shown.
The system for labeling samples in the embodiment of the application can comprise a labeling task initiating terminal, a server terminal and a labeling task executing terminal, wherein the labeling task initiating terminal is configured to: the method comprises the steps that in response to receiving an acquisition request of a first user for an annotation task initiation webpage, the annotation task initiation webpage is acquired from a server side; presenting an annotation task initiating webpage; the server is configured to: acquiring marking tool information and a to-be-marked sample set input by a first user through a marking task initiating webpage; generating a webpage for labeling based on the information of the labeling tool and the sample set to be labeled, wherein the webpage for labeling comprises the sample to be labeled in the sample set to be labeled and the labeling tool indicated by the information of the labeling tool, and the labeling tool comprises a label; the annotation task execution end is configured to: acquiring a webpage for marking from a server in response to receiving an acquisition request of a second user for the webpage for marking; presenting a webpage for labeling; and the server is further configured to: and obtaining a labeled sample obtained after a second user labels the sample to be labeled in the webpage for labeling by using a labeling tool in the webpage for labeling.
As shown in fig. 2, in step 201, in response to receiving an acquisition request of a first user for initiating a webpage of an annotation task, an annotation task initiator sends an acquisition request for initiating the webpage of the annotation task to a server.
In this embodiment, the annotation task initiator (e.g., the annotation task initiator 101 shown in fig. 1) may send, in response to receiving, through a wired connection manner or a wireless connection manner, an acquisition request for the annotation task initiation webpage from the first user, to a server (e.g., the server 103 shown in fig. 1). Wherein the first user may be the user who initiated the annotation task. The annotation task may be a task of annotating the sample to be annotated. The annotation task initiation webpage can be a preset webpage for initiating the annotation task. The annotation task initiating webpage can be managed by the server side.
In practice, the first user may send the acquisition request for the annotation task initiation webpage in various ways. For example, the first user may input a website corresponding to the annotation task initiation webpage at the annotation task initiation end; or, the first user may click a link corresponding to the annotation task initiation webpage presented by the annotation task initiation end.
It can be understood that, after receiving the obtaining request, the annotation task initiating terminal may forward the obtaining request to the server terminal, so as to request the server terminal to obtain the annotation task initiating webpage.
In step 202, the server sends the annotation task initiation webpage to the annotation task initiation end in response to receiving the acquisition request for the annotation task initiation webpage sent by the annotation task initiation end.
In this embodiment, the server may send the annotation task initiation webpage to the annotation task initiation end in response to receiving, by a wired connection manner or a wireless connection manner, an acquisition request for the annotation task initiation webpage sent by the annotation task initiation end.
It should be noted that, in practice, after receiving a web page acquisition request sent by another electronic device in communication connection, a server actually sends web page data used for representing the web page to the electronic device, and the electronic device analyzes the web page data, so that the web page can be obtained.
In step 203, the annotation task initiator presents an annotation task initiation webpage.
In this embodiment, after acquiring the annotation task initiation webpage, the annotation task initiation end may present the annotation task initiation webpage to the first user.
In step 204, the server obtains the annotation tool information and the to-be-annotated sample set, which are input by the first user through the annotation task initiating webpage.
In this embodiment, after the annotation task initiating terminal presents the annotation task initiating webpage to the first user, the server may obtain annotation tool information and a to-be-annotated sample set that are input by the first user through the annotation task initiating webpage. The type of the sample to be labeled in the sample set to be labeled can be any.
In some optional implementations of the present embodiment, the type of the sample to be labeled in the sample set to be labeled may include, but is not limited to, at least one of the following: image class, text class, voice class, video class. That is, the annotation system corresponding to the implementation manner can annotate various types of samples to be annotated, including but not limited to images, texts, voices, and videos, which is helpful for improving the application range of the annotation system.
Specifically, the annotation task initiation webpage may include an interface for the first user to input annotation tool information and an interface for the first user to upload a to-be-annotated sample set, and the first user may send the annotation tool information and the to-be-annotated sample set to the server via the interfaces. Wherein the marking tool information can be used to indicate a marking tool. The labeling tools can include various tools for labeling a sample to be labeled.
In particular, the tagging tool may comprise a label. And the labeling task initiated by the first user can be to label the sample set to be labeled by using the label. For example, the sample to be labeled input by the first user is an image, and the input labeling tool information may include "tag one: a landscape; and II, labeling: a character ". The first user has specified two labels, landscape and people respectively. The annotation task initiated by the first user may be a task of annotating the set of images to be annotated with "landscape" and "people".
It should be noted that the labeling tool may include other tools besides the label. By way of example, the annotation tool may include a tool for highlighting the target to be annotated in the image to be annotated, such as a rectangular box.
It will be appreciated that when the annotation tool comprises a tool for highlighting the target to be annotated in the sample to be annotated, the first user-initiated annotation task may be a task of annotating the target to be annotated in the sample to be annotated with a tag.
In some optional implementations of this embodiment, the annotation tool indicated by the annotation tool information may include more than two levels of tags. The labeling system corresponding to the implementation mode can be suitable for labeling labels with more than two levels, so that the diversity of labeling is improved, and more refined labeling is realized.
In some optional implementations of this embodiment, the server may be further configured to: acquiring initial labeling tool information and sample type information input by a first user through a labeling task initiating webpage; obtaining a sample for previewing from a preset sample set; generating a webpage for previewing based on the initial marking tool information and the sample for previewing; and the annotation task initiator may be further configured to: acquiring a webpage for previewing from a server in response to receiving an acquisition request of a first user for the webpage for previewing; presenting a webpage for preview; and the server is further configured to: and in response to detecting the confirmation operation of the first user for the initial marking tool in the webpage for previewing, determining the initial marking tool information as the marking tool information.
Wherein the sample type information may be used to indicate the type of the sample. The sample type of the sample for preview is the same as the sample type indicated by the sample type information, for example, if the sample type indicated by the sample type information is an image class, the sample type of the sample for preview is also an image class.
Specifically, the server may store a program for generating a webpage for preview in advance, and after acquiring the initial marking tool information and the sample for preview, the server may input the initial marking tool information and the sample for preview into the program to acquire the webpage for preview. The webpage for previewing can comprise an initial marking tool indicated by the initial marking tool information and a sample for previewing. Furthermore, the first user may browse the webpage for previewing, and may further mark the sample for previewing with an initial marking tool in the webpage for previewing to check the initial marking tool.
In this implementation manner, if the first user checks the initial marking tool without error, the first user may perform a confirmation operation on the initial marking tool. The confirmation operation may be various operations, and may be, for example, an operation of clicking a confirmation icon on a preview web page.
In the implementation mode, links of previewing the marking tool appointed by the first user and confirming the previewed marking tool are added in the process of initiating the marking task, and the effectiveness of the marking tool appointed by the user is improved.
In step 205, the server generates a webpage for annotation based on the annotation tool information and the sample set to be annotated.
In this embodiment, after the server obtains the labeling tool information and the to-be-labeled sample set, a webpage for labeling may be generated based on the labeling tool information and the to-be-labeled sample set. The webpage for labeling comprises a sample to be labeled in a sample set to be labeled and a labeling tool indicated by information of the labeling tool. The webpage for labeling can be a webpage for labeling the sample to be labeled in the webpage.
Specifically, the server may pre-store a program for generating a webpage for labeling based on the information of the labeling tool and the sample set to be labeled, and then, after acquiring the information of the labeling tool and the sample set to be labeled, the server may input the information of the labeling tool and the sample set to be labeled into the program to obtain the webpage for labeling.
It should be noted that the webpage for labeling may include only one sample to be labeled, or may include a plurality of samples to be labeled, which may be specifically set by a technician.
In step 206, the annotation task execution end sends an acquisition request for the webpage for annotation to the server end in response to receiving the acquisition request for the webpage for annotation by the second user.
In this embodiment, after the server obtains the webpage for annotation, the annotation task execution end (for example, the annotation task execution device 105 shown in fig. 1) may send an acquisition request for the webpage for annotation to the server in response to receiving an acquisition request for the webpage for annotation by the second user through a wired connection manner or a wireless connection manner. Wherein the second user may be a user performing the annotation task.
In practice, the second user may send the acquisition request for the webpage for annotation in various ways. For example, the second user may input a website corresponding to the webpage for annotation at the annotation task execution end; or the second user can click a link corresponding to the webpage for labeling presented by the labeling task execution end.
It can be understood that, after receiving the acquisition request for the webpage for annotation, the annotation task execution end may forward the acquisition request to the server end to request the server end to acquire the webpage for annotation.
In step 207, the server sends the webpage for annotation to the annotation task execution end in response to receiving the acquisition request for the webpage for annotation sent by the annotation task execution end.
In this embodiment, the server may send the webpage for annotation to the annotation task execution end in response to receiving, by a wired connection manner or a wireless connection manner, an acquisition request for the webpage for annotation sent by the annotation task execution end.
In some optional implementations of this embodiment, in the generating of the webpage for annotation, the server may be further configured to: acquiring annotation task execution end information input by a first user through an annotation task initiating webpage; and after generating the webpage for annotation, the server may be further configured to: responding to an acquisition request aiming at a webpage for labeling sent by a labeling task execution end; determining whether the annotation task execution end sending the acquisition request is the annotation task execution end indicated by the annotation task execution end information; and responding to the fact that the annotation task execution end sending the acquisition request is the annotation task execution end indicated by the annotation task execution end information, and sending the webpage for annotation to the annotation task execution end sending the acquisition request.
It is understood that the annotation task execution end information input by the first user may be information of an annotation task execution end that the first user desires to execute the annotation task initiated by the first user. Furthermore, after acquiring the request for acquiring the webpage for annotation, the server may determine whether the annotation task execution end that sent the request is the annotation task execution end that the first user desires, and if so, may send the webpage for annotation to the annotation task execution end that sent the request.
The annotation task in the implementation manner can be executed by the annotation task execution end designated by the first user, which helps to reduce the possibility of data propagation and improve the security of data.
In step 208, the annotation task execution end presents the webpage for annotation.
In this embodiment, after the labeling task execution end obtains the webpage for labeling, the webpage for labeling can be presented to the second user, so that the second user can label the sample to be labeled in the webpage for labeling.
In some optional implementations of this embodiment, the server may be further configured to: and obtaining a labeled sample obtained after a second user labels the sample to be labeled in the webpage for labeling by using a labeling tool in the webpage for labeling.
In this implementation manner, after the labeling task execution end presents the labeling webpage to the second user, the server end may obtain a labeled sample obtained after the second user labels the sample to be labeled in the labeling webpage by using the labeling tool in the labeling webpage. The marked sample can correspond to marking information generated by marking the to-be-marked sample by the second user by using the marking tool.
As an example, the sample to be annotated on the webpage for annotation is the image to be annotated, and the annotation tools are labels "landscape" and "people". The second user may label the image to be labeled with the label "landscape" to obtain a labeled image, where the labeled image may correspond to the labeling information "landscape".
Note that the labeling information corresponding to the labeled sample may be included in the labeled sample (for example, included in the labeled image), or may be stored in association with the labeled sample.
In some optional implementations of this embodiment, the server may be further configured to: generating a labeled webpage comprising a labeled sample; and the annotation task initiator may be further configured to: acquiring the marked webpage from the server in response to receiving an acquisition request of the first user for the marked webpage; and presenting the marked webpage.
Specifically, the server may store a program for generating the labeled webpage in advance, and then, after acquiring the labeled sample, the server may input the labeled sample into the program to obtain the labeled webpage.
According to the implementation mode, the marked webpage is presented to the first user, so that the first user can conveniently check and download the marked sample through the marked webpage, and the completeness of the sample marking process is improved.
In some optional implementations of this embodiment, the server may be further configured to: acquiring an initial labeled sample obtained after a second user labels a sample to be labeled in a webpage for labeling by using a labeling tool in the webpage for labeling; generating an initial labeled webpage comprising the initial labeled sample; and the annotation task execution end may be further configured to: in response to receiving an acquisition request of a second user for the initially labeled webpage, acquiring the initially labeled webpage from the server; presenting the initially labeled webpage; and the server may be further configured to: and in response to the detection of the confirmation operation of the second user on the initial labeled sample in the initial labeled webpage, determining the initial labeled sample confirmed by the second user as a labeled sample.
The confirmation operation may be various operations, for example, an operation of clicking a confirmation icon. Specifically, after the second user acquires the initially labeled web page, the initially labeled sample in the initially labeled web page and the label information corresponding to the initially labeled sample may be checked, and if the check is correct, the confirmation operation may be performed.
It can be understood that, if the second user checks that the labeled information is wrong, the labeled information may also be modified.
According to the implementation mode, a confirmation link for the initial labeled sample can be added before the final labeled sample is generated, and the accuracy of the finally obtained labeled sample is improved.
With continued reference to fig. 3, fig. 3 is a schematic diagram of an application scenario of the system for labeling a sample according to the present embodiment.
In the application scenario of fig. 3, first, a computer (i.e. an annotation task initiator) 301 obtains an annotation task initiation webpage 305 from a server 304 in response to receiving a obtaining request 303 of a first user 302 for the annotation task initiation webpage, and presents the annotation task initiation webpage 305.
Then, the server 304 may obtain the annotating tool information 306 and the set of samples to be annotated 307 input by the annotating task initiation webpage 305 presented by the first user 302 through the computer 301, and generate a webpage for annotation 308 based on the annotating tool information 306 and the set of samples to be annotated 307, wherein the webpage for annotation 308 includes the samples to be annotated in the set of samples to be annotated 307 and the annotating tool indicated by the annotating tool information 307, and the annotating tool includes a label.
Finally, the computer (i.e., the annotation task execution terminal) 309 can retrieve the webpage for annotation 308 from the server 304 and render the webpage for annotation 308 in response to receiving a retrieval request 311 for the webpage for annotation 308 by the second user 310.
Specifically, after the computer 309 presents the webpage for annotation 308 to the second user 310, the server 304 can obtain the annotated sample 312 obtained after the second user 310 annotates the sample to be annotated in the webpage for annotation 308 with the annotation tool in the webpage for annotation 308 presented by the computer 309.
Compared with the C/S architecture mode in the prior art, the method does not need to limit the operating systems of the annotation task initiating end and the annotation task executing end, and is beneficial to improving the application range of the annotation system; in addition, in the labeling process, the labeling task execution end only needs to add labels on the samples to be labeled in the webpage for labeling, and the labeling task execution end does not need to download the samples to be labeled, so that the propagation of the samples to be labeled is controlled, and the safety of the samples to be labeled is improved.
With continued reference to FIG. 4, a flow 400 of one embodiment of a method for annotating a sample in accordance with the present disclosure is illustrated. The method for labeling the sample comprises the following steps:
In this embodiment, an executing subject (for example, a server shown in fig. 1) of the method for annotating a sample may send, in response to receiving, through a wired connection manner or a wireless connection manner, an acquisition request for an annotation task initiation webpage, which is sent by a first user by using an annotation task initiation end (for example, an annotation task initiation device shown in fig. 1), the annotation task initiation webpage to the annotation task initiation end, so that the annotation task initiation end presents the annotation task initiation webpage to the first user. Wherein the first user may be the user who initiated the annotation task. The annotation task may be a task of annotating the sample to be annotated. The annotation task initiation webpage can be a preset webpage for initiating the annotation task. The annotation task initiation web page can be managed by the execution agent.
In this embodiment, after the executing body sends the annotation task initiation webpage to the annotation task initiation end, the annotating tool information and the to-be-annotated sample set input by the first user may be obtained in response to detecting that the first user performs an input operation using the annotation task initiation webpage. The type of the sample to be labeled in the sample set to be labeled can be any. The marking tool information can be used to indicate a marking tool. The labeling tools can include various tools for labeling a sample to be labeled. Specifically, the labeling tool may include a label, and may also include other tools.
And step 403, generating a webpage for labeling based on the labeling tool information and the sample set to be labeled.
In this embodiment, based on the information of the labeling tool and the set of samples to be labeled obtained in step 402, the executing entity may generate a webpage for labeling. The webpage for labeling comprises a sample to be labeled in a sample set to be labeled and a labeling tool indicated by information of the labeling tool.
In this embodiment, after the webpage for annotation is generated, the execution main body may send the webpage for annotation to the annotation task execution end in response to receiving, in a wired connection manner or a wireless connection manner, an acquisition request for the webpage for annotation, which is sent by the second user using the annotation task execution end, so that the annotation task execution end presents the webpage for annotation to the second user. Wherein the second user may be a user performing the annotation task.
It should be noted that the execution subject in the embodiment corresponding to fig. 4 may have the same features as the server in the system for labeling samples, and therefore, the embodiment corresponding to fig. 4 may be combined with the features of the server in the system for labeling samples to form a new technical solution. To avoid repetition, further description is omitted here.
Compared with the C/S architecture mode in the prior art, the method for labeling the sample can acquire the sample to be labeled and the information of a labeling tool from the labeling task initiating end in a webpage form, and acquire the labeled sample corresponding to the sample to be labeled from the labeling task executing end in a webpage form; in addition, in the labeling process, the labeling task execution end only needs to add labels on the samples to be labeled in the webpage for labeling, and the labeling task execution end does not need to download the samples to be labeled, so that the propagation of the samples to be labeled is controlled, and the safety of the samples to be labeled is improved.
With further reference to fig. 5, as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an apparatus for labeling a sample, which corresponds to the embodiment of the method shown in fig. 4, and which can be applied in various electronic devices.
As shown in fig. 5, the apparatus 500 for labeling a sample of the present embodiment includes: a first sending unit 501, an obtaining unit 502, a generating unit 503 and a second sending unit 504. The first sending unit 501 is configured to, in response to receiving an acquisition request, sent by a first user by using an annotation task initiator, for an annotation task initiation webpage, send the annotation task initiation webpage to the annotation task initiator, so that the annotation task initiator presents the annotation task initiation webpage to the first user; the obtaining unit 502 is configured to obtain annotation tool information and a set of samples to be annotated, which are input by a first user through an annotation task initiating webpage; the generating unit 503 is configured to generate a webpage for labeling based on the labeling tool information and the set of samples to be labeled, where the webpage for labeling includes the samples to be labeled in the set of samples to be labeled and the labeling tool indicated by the labeling tool information, and the labeling tool includes a label; the second sending unit 504 is configured to send the webpage for annotation to the annotation task execution end in response to receiving the acquisition request for the webpage for annotation sent by the second user by using the annotation task execution end, so that the annotation task execution end presents the webpage for annotation to the second user.
In this embodiment, the first sending unit 501 of the apparatus 500 for annotating a sample may send, in response to receiving, through a wired connection manner or a wireless connection manner, an acquisition request for an annotation task initiation webpage, which is sent by a first user using an annotation task initiation end (e.g., the annotation task initiation device shown in fig. 1), the annotation task initiation webpage to the annotation task initiation end, so that the annotation task initiation end presents the annotation task initiation webpage to the first user. Wherein the first user may be the user who initiated the annotation task. The annotation task may be a task of annotating the sample to be annotated. The annotation task initiation webpage can be a preset webpage for initiating the annotation task. The annotation task initiation web page can be managed by the apparatus 500.
In this embodiment, after the first sending unit 501 sends the annotation task initiation webpage to the annotation task initiation end, the obtaining unit 502 may obtain the annotation tool information and the to-be-annotated sample set input by the first user in response to detecting the input operation of the first user, which is performed by using the annotation task initiation webpage. The type of the sample to be labeled in the sample set to be labeled can be any. The marking tool information can be used to indicate a marking tool. The labeling tools can include various tools for labeling a sample to be labeled. Specifically, the labeling tool may include a label, and may also include other tools.
In this embodiment, the generating unit 503 may generate a webpage for annotation based on the annotation tool information and the sample set to be annotated obtained by the obtaining unit 502. The webpage for labeling comprises a sample to be labeled in a sample set to be labeled and a labeling tool indicated by information of the labeling tool.
In this embodiment, after the generating unit 503 generates the webpage for annotation, the second sending unit 504 sends the webpage for annotation to the annotation task execution end in response to receiving, by a wired connection manner or a wireless connection manner, an acquisition request for the webpage for annotation, which is sent by the second user by using the annotation task execution end, so that the annotation task execution end presents the webpage for annotation to the second user. Wherein the second user may be a user performing the annotation task.
It will be understood that the elements described in the apparatus 500 correspond to various steps in the method described with reference to fig. 4. Thus, the operations, features and resulting advantages described above with respect to the method are also applicable to the apparatus 500 and the units included therein, and are not described herein again.
Compared with the C/S architecture mode in the prior art, the device 500 provided by the above embodiment of the present disclosure can acquire the sample to be labeled and the information of the labeling tool from the labeling task initiating terminal in the form of a webpage, and acquire the labeled sample corresponding to the sample to be labeled from the labeling task executing terminal in the form of a webpage, and the labeling task initiating terminal and the task executing terminal in the present disclosure only need to install browsers for browsing the webpage, do not need to limit operating systems of the labeling task initiating terminal and the labeling task executing terminal, and are helpful to improve the application range of the labeling method; in addition, in the labeling process, the labeling task execution end only needs to add labels on the samples to be labeled in the webpage for labeling, and the labeling task execution end does not need to download the samples to be labeled, so that the propagation of the samples to be labeled is controlled, and the safety of the samples to be labeled is improved.
Referring now to FIG. 6, a block diagram of an electronic device (e.g., the server of FIG. 1) 600 suitable for use in implementing embodiments of the present disclosure is shown. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 6, electronic device 600 may include a processing means (e.g., central processing unit, graphics processor, etc.) 601 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data necessary for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
In general, input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc., output devices 607 including, for example, a liquid crystal display (L CD), speaker, vibrator, etc., storage devices 608 including, for example, magnetic tape, hard disk, etc., and communication devices 609.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 609, or may be installed from the storage means 608, or may be installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of the embodiments of the present disclosure.
It should be noted that the computer readable medium in the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: in response to receiving an acquisition request which is sent by a first user by using a labeling task initiating terminal and aims at a labeling task initiating webpage, sending the labeling task initiating webpage to the labeling task initiating terminal so that the labeling task initiating terminal can present the labeling task initiating webpage to the first user; acquiring marking tool information and a to-be-marked sample set input by a first user through a marking task initiating webpage; generating a webpage for labeling based on the labeling tool information and the sample set to be labeled, wherein the webpage for labeling comprises the sample to be labeled in the sample set to be labeled and a labeling tool indicated by the labeling tool information, and the labeling tool comprises a label; and in response to receiving an acquisition request aiming at the webpage for labeling sent by the second user by using the labeling task execution end, sending the webpage for labeling to the labeling task execution end so that the labeling task execution end can present the webpage for labeling to the second user.
Computer program code for carrying out operations of the present disclosure may be written in any combination of one or more programming languages, including AN object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of an element does not in some cases constitute a limitation on the element itself, for example, the first sending element may also be described as an "element sending an annotation task initiation web page".
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.
Claims (12)
1. A system for annotating a sample comprising: a labeling task initiating terminal, a service terminal and a labeling task executing terminal, wherein,
the annotation task initiator is configured to: in response to receiving an acquisition request of a first user for an annotated task initiation webpage, acquiring the annotated task initiation webpage from the server; presenting the labeling task initiating webpage;
the server is configured to: acquiring marking tool information and a to-be-marked sample set input by the first user through the marking task initiating webpage; generating a webpage for labeling based on the labeling tool information and the to-be-labeled sample set, wherein the webpage for labeling comprises the to-be-labeled sample in the to-be-labeled sample set and a labeling tool indicated by the labeling tool information, and the labeling tool comprises a label;
the labeling task execution end is configured to: responding to an acquisition request of a second user for the webpage for marking, and acquiring the webpage for marking from the server; and presenting the webpage for marking.
2. The system of claim 1, wherein the server is further configured to: and obtaining a labeled sample obtained after the second user labels the sample to be labeled in the webpage for labeling by using a labeling tool in the webpage for labeling.
3. The system of claim 2, wherein the server is further configured to: generating a labeled webpage comprising the labeled sample; and
the annotation task initiator is further configured to: in response to receiving an acquisition request of a first user for the labeled webpage, acquiring the labeled webpage from the server; and presenting the marked webpage.
4. The system of claim 2, wherein the server is further configured to: acquiring an initial labeled sample obtained after the second user labels the sample to be labeled in the webpage for labeling by using a labeling tool in the webpage for labeling; generating an initial labeled webpage comprising the initial labeled sample; and
the labeling task execution end is further configured to: in response to receiving an acquisition request of the second user for the initially labeled webpage, acquiring the initially labeled webpage from the server; presenting the initially labeled webpage; and
the server is further configured to: and in response to the detection of the second user's confirmation operation on the initial labeled sample in the initial labeled webpage, determining the initial labeled sample confirmed by the second user as a labeled sample.
5. The system of claim 1, wherein the server is further configured to: acquiring initial labeling tool information and sample type information input by the first user through the labeling task initiating webpage; acquiring a sample for previewing from a preset sample set, wherein the sample type of the sample for previewing is the same as the sample type indicated by the sample type information; generating a webpage for previewing based on the initial marking tool information and the sample for previewing; and
the annotation task initiator is further configured to: in response to receiving an acquisition request of the first user for a webpage for previewing, acquiring the webpage for previewing from the server; presenting the webpage for previewing, wherein the webpage for previewing comprises an initial marking tool indicated by the initial marking tool information; and
the server is further configured to: in response to detecting a confirmation operation of the first user for an initial marking tool in the webpage for previewing, determining the initial marking tool information as marking tool information.
6. The system of claim 1, wherein the server is further configured to: acquiring annotation task execution end information input by the first user through the annotation task initiating webpage; responding to an acquisition request aiming at the webpage for labeling sent by a labeling task execution end; determining whether the annotation task execution end sending the acquisition request is the annotation task execution end indicated by the annotation task execution end information; and in response to the fact that the annotation task execution end sending the acquisition request is the annotation task execution end indicated by the annotation task execution end information, sending the webpage for annotation to the annotation task execution end sending the acquisition request.
7. The system of claim 1, wherein the annotation tool indicated by the annotation tool information comprises a label of more than two levels.
8. The system according to one of claims 1 to 7, wherein the type of the sample to be labeled in the set of samples to be labeled comprises at least one of: image class, text class, voice class, video class.
9. A method for annotating a sample, comprising:
in response to receiving an acquisition request which is sent by a first user by using a labeling task initiating terminal and aims at a labeling task initiating webpage, sending the labeling task initiating webpage to the labeling task initiating terminal so that the labeling task initiating terminal can present the labeling task initiating webpage to the first user;
acquiring marking tool information and a to-be-marked sample set input by the first user through the marking task initiating webpage;
generating a webpage for labeling based on the labeling tool information and the to-be-labeled sample set, wherein the webpage for labeling comprises the to-be-labeled sample in the to-be-labeled sample set and a labeling tool indicated by the labeling tool information, and the labeling tool comprises a label;
and in response to receiving an acquisition request aiming at the webpage for labeling sent by a second user by using a labeling task execution end, sending the webpage for labeling to the labeling task execution end so that the labeling task execution end can present the webpage for labeling to the second user.
10. An apparatus for labeling a sample, comprising:
the system comprises a first sending unit, a second sending unit and a third sending unit, wherein the first sending unit is configured to respond to the receiving of an acquisition request which is sent by a first user by utilizing a labeling task initiating terminal and aims at a labeling task initiating webpage, and send the labeling task initiating webpage to the labeling task initiating terminal so that the labeling task initiating terminal can present the labeling task initiating webpage to the first user;
the acquisition unit is configured to acquire marking tool information and a to-be-marked sample set input by the first user through the marking task initiation webpage;
a generating unit configured to generate a webpage for labeling based on the labeling tool information and the to-be-labeled sample set, wherein the webpage for labeling comprises the to-be-labeled sample in the to-be-labeled sample set and a labeling tool indicated by the labeling tool information, and the labeling tool comprises a label;
and the second sending unit is configured to respond to the receiving of an acquisition request which is sent by a second user by utilizing the labeling task execution end and aims at the webpage for labeling, and send the webpage for labeling to the labeling task execution end so that the labeling task execution end can present the webpage for labeling to the second user.
11. A server, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, cause the one or more processors to implement the method as recited in claim 9.
12. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method as claimed in claim 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010174981.7A CN111400581B (en) | 2020-03-13 | 2020-03-13 | System, method and apparatus for labeling samples |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010174981.7A CN111400581B (en) | 2020-03-13 | 2020-03-13 | System, method and apparatus for labeling samples |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111400581A true CN111400581A (en) | 2020-07-10 |
| CN111400581B CN111400581B (en) | 2024-02-06 |
Family
ID=71434212
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010174981.7A Active CN111400581B (en) | 2020-03-13 | 2020-03-13 | System, method and apparatus for labeling samples |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111400581B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113407083A (en) * | 2021-06-24 | 2021-09-17 | 上海商汤科技开发有限公司 | Data labeling method and device, electronic equipment and storage medium |
| CN113822137A (en) * | 2021-07-23 | 2021-12-21 | 腾讯科技（深圳）有限公司 | Data annotation method, device and equipment and computer readable storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101739415A (en) * | 2008-11-25 | 2010-06-16 | 华中师范大学 | Browser-oriented webpage labeling system |
| CN103678510A (en) * | 2013-11-25 | 2014-03-26 | 北京奇虎科技有限公司 | Method and device for providing visualized label for webpage |
| CN105446969A (en) * | 2014-06-05 | 2016-03-30 | 北大方正信息产业集团有限公司 | Webpage generation method, device and server |
| CN106156025A (en) * | 2015-03-25 | 2016-11-23 | 阿里巴巴集团控股有限公司 | The management method of a kind of data mark and device |
| CN107066437A (en) * | 2017-02-22 | 2017-08-18 | 深圳市天朗时代科技有限公司 | The method and device of copyright mark |
| CN109325213A (en) * | 2018-09-30 | 2019-02-12 | 北京字节跳动网络技术有限公司 | Method and apparatus for labeled data |
| CN109446357A (en) * | 2018-10-18 | 2019-03-08 | 杭州快忆科技有限公司 | Data mask method and device |
| CN110209849A (en) * | 2019-06-04 | 2019-09-06 | 北京字节跳动网络技术有限公司 | Method and apparatus for marking key point |
| CN110390067A (en) * | 2019-07-22 | 2019-10-29 | 出门问问(苏州)信息科技有限公司 | A kind of webpage label method, apparatus, storage medium and electronic equipment |
-
2020
- 2020-03-13 CN CN202010174981.7A patent/CN111400581B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101739415A (en) * | 2008-11-25 | 2010-06-16 | 华中师范大学 | Browser-oriented webpage labeling system |
| CN103678510A (en) * | 2013-11-25 | 2014-03-26 | 北京奇虎科技有限公司 | Method and device for providing visualized label for webpage |
| CN105446969A (en) * | 2014-06-05 | 2016-03-30 | 北大方正信息产业集团有限公司 | Webpage generation method, device and server |
| CN106156025A (en) * | 2015-03-25 | 2016-11-23 | 阿里巴巴集团控股有限公司 | The management method of a kind of data mark and device |
| CN107066437A (en) * | 2017-02-22 | 2017-08-18 | 深圳市天朗时代科技有限公司 | The method and device of copyright mark |
| CN109325213A (en) * | 2018-09-30 | 2019-02-12 | 北京字节跳动网络技术有限公司 | Method and apparatus for labeled data |
| CN109446357A (en) * | 2018-10-18 | 2019-03-08 | 杭州快忆科技有限公司 | Data mask method and device |
| CN110209849A (en) * | 2019-06-04 | 2019-09-06 | 北京字节跳动网络技术有限公司 | Method and apparatus for marking key point |
| CN110390067A (en) * | 2019-07-22 | 2019-10-29 | 出门问问(苏州)信息科技有限公司 | A kind of webpage label method, apparatus, storage medium and electronic equipment |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113407083A (en) * | 2021-06-24 | 2021-09-17 | 上海商汤科技开发有限公司 | Data labeling method and device, electronic equipment and storage medium |
| CN113822137A (en) * | 2021-07-23 | 2021-12-21 | 腾讯科技（深圳）有限公司 | Data annotation method, device and equipment and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111400581B (en) | 2024-02-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10635735B2 (en) | Method and apparatus for displaying information | |
| CN110708346B (en) | Information processing system and method | |
| CN109325213B (en) | Method and device for labeling data | |
| CN109302461B (en) | Information display method, information processing method, information display medium, information processing system and computing equipment | |
| US20150120816A1 (en) | Tracking use of content of an online library | |
| CN111800671B (en) | Method and apparatus for aligning paragraphs and video | |
| CN109359194B (en) | Method and apparatus for predicting information categories | |
| CN109446442B (en) | Method and apparatus for processing information | |
| CN110321544B (en) | Method and device for generating information | |
| WO2022218034A1 (en) | Interaction method and apparatus, and electronic device | |
| CN110058854B (en) | Method, terminal device and computer-readable medium for generating application | |
| CN111443909B (en) | Method and device for generating page | |
| CN110780874B (en) | Method and device for generating information | |
| US9760557B2 (en) | Tagging autofill field entries | |
| JP7050857B2 (en) | Summary generation method and equipment | |
| CN109413056B (en) | Method and apparatus for processing information | |
| WO2024099171A1 (en) | Video generation method and apparatus | |
| US11150864B2 (en) | Displaying enhancement items associated with an audio recording | |
| CN111400581B (en) | System, method and apparatus for labeling samples | |
| WO2015119970A1 (en) | Visual tagging to record interactions | |
| CN111813685B (en) | Automatic test method and device | |
| CN112309389A (en) | Information interaction method and device | |
| CN113407229B (en) | Method and device for generating offline scripts | |
| CN113221554A (en) | Text processing method and device, electronic equipment and storage medium | |
| CN112307723A (en) | Method and device for generating code document and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 221, 2nd floor, Block C, 18 Kechuang 11th Street, Daxing Economic and Technological Development Zone, Beijing, 100176 Applicant after: Jingdong Technology Holding Co.,Ltd. Address before: Room 221, 2nd floor, Block C, 18 Kechuang 11th Street, Daxing Economic and Technological Development Zone, Beijing, 100176 Applicant before: Jingdong Digital Technology Holding Co.,Ltd. Address after: Room 221, 2nd floor, Block C, 18 Kechuang 11th Street, Daxing Economic and Technological Development Zone, Beijing, 100176 Applicant after: Jingdong Digital Technology Holding Co.,Ltd. Address before: Room 221, 2nd floor, Block C, 18 Kechuang 11th Street, Daxing Economic and Technological Development Zone, Beijing, 100176 Applicant before: JINGDONG DIGITAL TECHNOLOGY HOLDINGS Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |