CN111400581B - System, method and apparatus for labeling samples - Google Patents
System, method and apparatus for labeling samples Download PDFInfo
- Publication number
- CN111400581B CN111400581B CN202010174981.7A CN202010174981A CN111400581B CN 111400581 B CN111400581 B CN 111400581B CN 202010174981 A CN202010174981 A CN 202010174981A CN 111400581 B CN111400581 B CN 111400581B
- Authority
- CN
- China
- Prior art keywords
- webpage
- labeling
- sample
- marking
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000002372 labelling Methods 0.000 title claims abstract description 449
- 238000000034 method Methods 0.000 title claims abstract description 41
- 230000000977 initiatory effect Effects 0.000 claims abstract description 178
- 230000004044 response Effects 0.000 claims abstract description 37
- 238000012790 confirmation Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 9
- 239000003999 initiator Substances 0.000 claims description 6
- 230000009286 beneficial effect Effects 0.000 abstract description 5
- 238000009877 rendering Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 9
- 238000004891 communication Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Embodiments of the present disclosure disclose systems, methods, and devices for labeling a sample. One embodiment of the system comprises: the labeling task initiating terminal is configured to obtain a labeling task initiating webpage from the server in response to receiving an obtaining request of the first user for the labeling task initiating webpage; presenting a labeling task initiating webpage; the server side is configured to acquire marking tool information and a sample set to be marked, which are input by a first user initiating a webpage through a marking task; generating a webpage for marking based on marking tool information and a sample set to be marked; the labeling task execution end is configured to acquire the labeling webpage from the server in response to receiving an acquisition request of the second user for the labeling webpage; and rendering the labeling web page. The embodiment is beneficial to improving the application range of the labeling system and improving the safety of the sample to be labeled.
Description
Technical Field
Embodiments of the present disclosure relate to the field of computer technology, and more particularly, to systems, methods, and devices for labeling samples.
Background
In recent years, artificial intelligence technology is continuously developed, and the demand of training samples required by training algorithms is gradually increased. The training sample is data obtained after labeling the pre-collected data by a labeling person.
At present, an offline marking tool is generally adopted for data marking. Offline labeling tools are typically in a C/S architecture (client-server) mode. The labeling personnel needs to install client software on the labeling equipment, and further obtains data to be labeled from the server by using the client software and labels the data.
Disclosure of Invention
Embodiments of the present disclosure propose systems, methods, and devices for labeling samples.
In a first aspect, embodiments of the present disclosure provide a system for labeling a sample, the system comprising: the labeling task comprises a labeling task initiating terminal, a service terminal and a labeling task executing terminal, wherein the labeling task initiating terminal is configured to: responding to the receiving of an acquisition request of a first user for the labeling task initiating webpage, and acquiring the labeling task initiating webpage from a server; presenting a labeling task initiating webpage; the server is configured to: acquiring marking tool information and a sample set to be marked, which are input by a first user through a marking task initiating webpage; generating a webpage for marking based on marking tool information and a sample set to be marked, wherein the webpage for marking comprises a sample to be marked in the sample set to be marked and marking tools indicated by the marking tool information, and the marking tools comprise labels; the labeling task execution end is configured to: responding to the receiving of an acquisition request of a second user for the labeling webpage, and acquiring the labeling webpage from the server; and rendering the labeling web page.
In some embodiments, the server is further configured to: and obtaining a marked sample obtained after the second user marks the sample to be marked in the webpage for marking by using the marking tool in the webpage for marking.
In some embodiments, the server is further configured to: generating a marked webpage comprising a marked sample; and the labeling task initiator is further configured to: responding to the receiving of an acquisition request of a first user for the marked webpage, and acquiring the marked webpage from a server; and presenting the marked webpage.
In some embodiments, the server is further configured to: acquiring an initial marked sample obtained after a second user marks a sample to be marked in a webpage for marking by using a marking tool in the webpage for marking; generating an initial marked webpage comprising an initial marked sample; and the labeling task execution end is further configured to: responding to the receiving of an acquisition request of a second user for the initial marked webpage, and acquiring the initial marked webpage from the server; presenting the webpage after the initial annotation; and the server is further configured to: and in response to detecting the confirmation operation of the second user on the initial marked sample in the initial marked webpage, determining the initial marked sample confirmed by the second user as the marked sample.
In some embodiments, the server is further configured to: acquiring initial marking tool information and sample type information which are input by a first user through a marking task initiating webpage; obtaining a sample for previewing from a preset sample set, wherein the sample type of the sample for previewing is the same as the sample type indicated by the sample type information; generating a webpage for previewing based on the initial labeling tool information and the sample for previewing; and the labeling task initiator is further configured to: responding to receiving an acquisition request of a first user for a webpage for previewing, and acquiring the webpage for previewing from a server; presenting a webpage for previewing, wherein the webpage for previewing comprises an initial annotation tool indicated by the initial annotation tool information; and the server is further configured to: in response to detecting a confirmation operation of the first user with respect to the initial marking tool in the webpage for preview, the initial marking tool information is determined as marking tool information.
In some embodiments, the server is further configured to: acquiring annotation task execution end information input by a first user through an annotation task initiating webpage; responding to receiving an acquisition request for the labeling webpage sent by the labeling task execution end; determining whether the labeling task execution end sending the acquisition request is a labeling task execution end indicated by the labeling task execution end information; and the labeling task execution end responding to the acquisition request is a labeling task execution end indicated by the labeling task execution end information, and the labeling webpage is sent to the labeling task execution end sending the acquisition request.
In some embodiments, the marking tool indicated by the marking tool information includes more than two levels of labels.
In some embodiments, the type of sample to be annotated in the set of samples to be annotated comprises at least one of: image class, text class, speech class, video class.
In a second aspect, embodiments of the present disclosure provide a method for labeling a sample, the method comprising: responding to the received acquisition request for the labeling task initiating webpage sent by the first user by using the labeling task initiating terminal, and sending the labeling task initiating webpage to the labeling task initiating terminal so that the labeling task initiating terminal presents the labeling task initiating webpage to the first user; acquiring marking tool information and a sample set to be marked, which are input by a first user through a marking task initiating webpage; generating a webpage for marking based on marking tool information and a sample set to be marked, wherein the webpage for marking comprises a sample to be marked in the sample set to be marked and marking tools indicated by the marking tool information, and the marking tools comprise labels; and in response to receiving an acquisition request for the labeling web page, which is sent by the second user by using the labeling task execution end, sending the labeling web page to the labeling task execution end, so that the labeling task execution end presents the labeling web page to the second user.
In a third aspect, embodiments of the present disclosure provide an apparatus for labeling a sample, the apparatus comprising: the first sending unit is configured to send the labeling task initiating webpage to the labeling task initiating terminal in response to receiving an acquisition request for the labeling task initiating webpage sent by the first user by using the labeling task initiating terminal, so that the labeling task initiating terminal presents the labeling task initiating webpage to the first user; the acquisition unit is configured to acquire marking tool information and a sample set to be marked, which are input by a first user through a marking task initiating webpage; the generating unit is configured to generate a webpage for marking based on marking tool information and a sample set to be marked, wherein the webpage for marking comprises a sample to be marked in the sample set to be marked and marking tools indicated by the marking tool information, and the marking tools comprise labels; the second sending unit is configured to send the labeling webpage to the labeling task execution end in response to receiving an acquisition request for the labeling webpage sent by the second user by using the labeling task execution end, so that the labeling task execution end presents the labeling webpage to the second user.
In a fourth aspect, embodiments of the present disclosure provide a server, comprising: one or more processors; and a storage device having one or more programs stored thereon, which when executed by the one or more processors, cause the one or more processors to implement the method for labeling samples described above.
In a fifth aspect, embodiments of the present disclosure provide a computer-readable medium having stored thereon a computer program which, when executed by a processor, implements the above-described method for labeling a sample.
The system, the method and the device for labeling the sample, which are provided by the embodiment of the disclosure, respond to the receiving of the request of the first user for acquiring the labeling task initiating webpage by the labeling task initiating terminal, acquire the labeling task initiating webpage from the server terminal, and present the labeling task initiating webpage; the method comprises the steps that a server side obtains marking tool information and a sample set to be marked, which are input by a first user through a marking task initiating webpage, and a webpage for marking is generated based on the marking tool information and the sample set to be marked, wherein the webpage for marking comprises a sample to be marked in the sample set to be marked and marking tools indicated by the marking tool information, and the marking tools comprise labels; then, the labeling task execution end responds to the receiving of the request of the second user for the labeling webpage, the labeling webpage is obtained from the server end, and the labeling webpage is presented, so that the labeling task initiation end and the labeling task execution end in the present disclosure only need to install a browser for accessing the webpage managed by the server end, and initiation of the labeling task and execution of the labeling task can be realized. In addition, in the labeling process, the labeling task execution end only needs to add labels on the samples to be labeled in the labeling webpage, and the labeling task execution end does not need to download the samples to be labeled, so that the control of the propagation of the samples to be labeled is facilitated, and the safety of the samples to be labeled is improved.
Drawings
Other features, objects and advantages of the present disclosure will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the following drawings:
FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;
FIG. 2 is a timing diagram of one embodiment of a system for labeling samples according to the present application;
FIG. 3 is a schematic illustration of one application scenario of a system for labeling samples according to the present application;
FIG. 4 is a flow chart of one embodiment of a method for labeling a sample according to the present application;
FIG. 5 is a schematic structural view of one embodiment of a device for labeling a sample according to the present application;
FIG. 6 is a schematic diagram of a computer system suitable for use in implementing embodiments of the present application.
Detailed Description
The present disclosure is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
It should be noted that, without conflict, the embodiments of the present disclosure and features of the embodiments may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
FIG. 1 illustrates an exemplary system architecture 100 in which embodiments of a system for labeling a sample, a method for labeling a sample, or an apparatus for labeling a sample of the present disclosure may be applied.
As shown in fig. 1, the system architecture 100 may include an annotation task initiating device 101, a network 102, a server 103, a network 104, and an annotation task performing device 105. The network 102 is the medium used to provide the communication link between the labeling task initiation device 101 and the server 103. The network 104 is used as a medium to provide a communication link between the labeling task execution device 105 and the server 103. The networks 102, 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The annotation task sponsor may interact with the server 103 via the network 102 using the annotation task sponsor 101. Similarly, the annotation task performer can interact with the server 103 via the network 104 using the annotation task performing device 105. Various communication client applications, such as a web browser application, a shopping class application, a search class application, an instant messaging tool, a mailbox client, social platform software, and the like, may be installed on the annotation task initiating device 101 and the annotation task performing device 105.
The labeling task initiation device 101 and the labeling task execution device 105 may be hardware or software. When the annotation task initiating device 101 and the annotation task performing device 105 are hardware, they may be a variety of electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, dynamic video expert compression standard audio plane 3), MP4 (Moving Picture Experts Group Audio Layer IV, dynamic video expert compression standard audio plane 4) players, laptop and desktop computers, and the like. When the labeling task initiation device 101 and the labeling task execution device 105 are software, they can be installed in the above-listed electronic devices. Which may be implemented as multiple software or software modules (e.g., multiple software or software modules for providing distributed services) or as a single software or software module. The present invention is not particularly limited herein.
The server 103 may be a server providing various services, such as a background web server providing support for web pages displayed on the labeling task initiation device 101 and the labeling task execution device 105. The background web server may analyze and process the received data, such as the web acquisition request, and feed back the processing result (e.g., web page) to the labeling task initiating device 101 and the labeling task executing device 105.
It should be noted that, the method for labeling a sample provided by the embodiment of the present disclosure is generally performed by the server 103, and accordingly, the device for labeling a sample is generally disposed in the server 103.
The server may be hardware or software. When the server is hardware, the server may be implemented as a distributed server cluster formed by a plurality of servers, or may be implemented as a single server. When the server is software, it may be implemented as a plurality of software or software modules (e.g., a plurality of software or software modules for providing distributed services), or as a single software or software module. The present invention is not particularly limited herein.
It should be understood that the number of annotation task initiating devices, annotation task performing devices, networks, and servers in FIG. 1 are merely illustrative. There may be any number of labeling task initiation devices, labeling task execution devices, networks, and servers, as desired for implementation.
Referring to FIG. 2, a timing diagram 200 of one embodiment of a system for labeling samples according to the present application is shown.
The system for labeling a sample in the embodiment of the present application may include a labeling task initiating terminal, a service terminal, and a labeling task executing terminal, where the labeling task initiating terminal is configured to: responding to the receiving of an acquisition request of a first user for the labeling task initiating webpage, and acquiring the labeling task initiating webpage from a server; presenting a labeling task initiating webpage; the server is configured to: acquiring marking tool information and a sample set to be marked, which are input by a first user through a marking task initiating webpage; generating a webpage for marking based on marking tool information and a sample set to be marked, wherein the webpage for marking comprises a sample to be marked in the sample set to be marked and marking tools indicated by the marking tool information, and the marking tools comprise labels; the labeling task execution end is configured to: responding to the receiving of an acquisition request of a second user for the labeling webpage, and acquiring the labeling webpage from the server; presenting a webpage for labeling; and the server is further configured to: and obtaining a marked sample obtained after the second user marks the sample to be marked in the webpage for marking by using the marking tool in the webpage for marking.
As shown in fig. 2, in step 201, the labeling task initiator sends an acquisition request for a labeling task-initiated web page to the server in response to receiving the acquisition request for the labeling task-initiated web page by the first user.
In this embodiment, the labeling task initiating terminal (for example, the labeling task initiating device 101 shown in fig. 1) may send, to the server terminal (for example, the server 103 shown in fig. 1), an acquisition request for a labeling task initiating web page in response to receiving, by a wired connection manner or a wireless connection manner, an acquisition request for a labeling task initiating web page by the first user. Wherein the first user may be a user initiating an annotation task. The labeling task may be a task that labels a sample to be labeled. The labeling task initiation web page may be a preset web page for initiating a labeling task. The labeling task initiation web page can be managed by the server.
In practice, the first user may send the acquisition request for the labeling task-initiated web page in various ways. For example, the first user may input a website corresponding to the labeling task initiating webpage at the labeling task initiating terminal; or the first user can click on a link corresponding to the labeling task initiating webpage presented by the labeling task initiating terminal.
It can be understood that after the labeling task initiating terminal receives the obtaining request, the obtaining request can be forwarded to the server terminal, so as to request the server terminal to obtain the labeling task initiating webpage.
In step 202, the server side sends the labeling task initiating webpage to the labeling task initiating terminal in response to receiving the obtaining request for the labeling task initiating webpage sent by the labeling task initiating terminal.
In this embodiment, the server may send the labeling task initiating webpage to the labeling task initiating terminal in response to receiving, by the wired connection manner or the wireless connection manner, an obtaining request for the labeling task initiating webpage sent by the labeling task initiating terminal.
In practice, after receiving a web page acquisition request sent by other electronic devices in communication connection, the server may actually send web page data representing the web page to the electronic devices, and the electronic devices analyze the web page data to obtain the web page.
In step 203, the labeling task initiation terminal presents a labeling task initiation web page.
In this embodiment, after the labeling task initiating terminal obtains the labeling task initiating webpage, the labeling task initiating webpage may be presented to the first user.
In step 204, the server obtains the labeling tool information and the sample set to be labeled, which are input by the first user initiating the webpage through the labeling task.
In this embodiment, after the labeling task initiating terminal presents the labeling task initiating webpage to the first user, the server may obtain labeling tool information and a sample set to be labeled, which are input by the first user through the labeling task initiating webpage. The type of the sample to be marked in the sample set to be marked can be arbitrary.
In some optional implementations of the present embodiments, the types of the samples to be annotated in the set of samples to be annotated may include, but are not limited to, at least one of: image class, text class, speech class, video class. The labeling system corresponding to the implementation mode can label various types of samples to be labeled, including but not limited to images, texts, voices and videos, which is beneficial to improving the application range of the labeling system.
Specifically, the labeling task initiating webpage may include an interface for the first user to input labeling tool information and an interface for the first user to upload a sample set to be labeled, and further, the first user may send the labeling tool information and the sample set to be labeled to the server through the interfaces. Wherein the marking tool information may be used to indicate the marking tool. The labeling tools may include various tools for labeling a sample to be labeled.
In particular, the labeling tool may comprise a label. And the labeling task initiated by the first user can be to label the sample set to be labeled by using the label. For example, the sample to be marked input by the first user is an image, and the input marking tool information may include "tag one: scenery; label two: character). The first user designates two labels, respectively "landscape" and "people". The labeling task initiated by the first user may be a task of labeling the image set to be labeled by using "scenery" and "person".
Note that the labeling tool may include other tools in addition to the label. As an example, the annotation tool may comprise a tool for highlighting the object to be annotated in the image to be annotated, such as a rectangular box.
It will be appreciated that when the annotation tool comprises a tool for highlighting objects to be annotated in a sample to be annotated, the first user-initiated annotation task may be a task for annotating objects to be annotated in a sample to be annotated with labels.
In some alternative implementations of the present embodiment, the marking tool indicated by the marking tool information may include more than two levels of labels. The labeling system corresponding to the implementation mode can be suitable for labeling labels with more than two levels, which is beneficial to improving the diversity of labeling and realizing more refined labeling.
In some optional implementations of this embodiment, the server may be further configured to: acquiring initial marking tool information and sample type information which are input by a first user through a marking task initiating webpage; obtaining a sample for previewing from a preset sample set; generating a webpage for previewing based on the initial labeling tool information and the sample for previewing; and the labeling task initiation end may be further configured to: responding to receiving an acquisition request of a first user for a webpage for previewing, and acquiring the webpage for previewing from a server; presenting a webpage for preview; and the server is further configured to: in response to detecting a confirmation operation of the first user with respect to the initial marking tool in the webpage for preview, the initial marking tool information is determined as marking tool information.
Wherein the sample type information may be used to indicate the type of sample. The sample type of the sample for preview is the same as the sample type indicated by the sample type information, for example, the sample type indicated by the sample type information is an image class, and the sample type of the sample for preview is also an image class.
Specifically, the server may store a program for generating the preview web page in advance, and after acquiring the initial marking tool information and the preview sample, the server may input the initial marking tool information and the preview sample into the program to obtain the preview web page. The preview web page may include an initial annotation tool indicated by the initial annotation tool information and a preview sample. Further, the first user may browse the preview web page, and may further annotate the preview sample with an initial annotation tool in the preview web page to check the initial annotation tool.
In this implementation, if the first user checks the initial labeling tool without error, the first user may perform a confirmation operation with respect to the initial labeling tool. The confirmation operation may be various operations, for example, an operation of clicking a confirmation icon in the preview web page.
In the process of initiating the labeling task, the link of previewing the labeling tool appointed by the first user and confirming the previewed labeling tool is added, so that the effectiveness of the labeling tool appointed by the user is improved.
In step 205, the server generates a labeling web page based on the labeling tool information and the sample set to be labeled.
In this embodiment, after the server side obtains the labeling tool information and the sample set to be labeled, a web page for labeling may be generated based on the labeling tool information and the sample set to be labeled. The webpage for marking comprises a sample to be marked in the sample set to be marked and marking tools indicated by marking tool information. The labeling web page may be a web page for labeling a sample to be labeled therein.
Specifically, the server may store a program for generating a webpage for labeling based on the labeling tool information and the sample set to be labeled in advance, and further, the server may input the labeling tool information and the sample set to be labeled into the program after acquiring the labeling tool information and the sample set to be labeled, to obtain the webpage for labeling.
It should be noted that the labeling web page may include only one sample to be labeled, or may include a plurality of samples to be labeled, which may be specifically set by a technician.
In step 206, the labeling task execution end sends an acquisition request for the labeling web page to the server end in response to receiving the acquisition request for the labeling web page by the second user.
In this embodiment, after the server obtains the labeling web page, the labeling task execution end (for example, the labeling task execution device 105 shown in fig. 1) may send an obtaining request for the labeling web page to the server in response to receiving, by a wired connection manner or a wireless connection manner, an obtaining request for the labeling web page by the second user. Wherein the second user may be a user performing an annotation task.
In practice, the second user may send the retrieval request for the labeling web page in various ways. For example, the second user may input a website corresponding to the labeling web page at the labeling task execution end; or the second user can click on a link corresponding to the labeling webpage presented by the labeling task execution end.
It can be understood that after the labeling task execution end receives the acquisition request for the labeling web page, the acquisition request can be forwarded to the server end, so as to request the server end to acquire the labeling web page.
In step 207, the server side sends the labeling web page to the labeling task execution side in response to receiving the request for acquiring the labeling web page sent by the labeling task execution side.
In this embodiment, the server may send the labeling web page to the labeling task execution end in response to receiving, by the wired connection manner or the wireless connection manner, an acquisition request for the labeling web page sent by the labeling task execution end.
In some optional implementations of this embodiment, in generating the labeling web page, the server may be further configured to: acquiring annotation task execution end information input by a first user through an annotation task initiating webpage; and after generating the labeling web page, the server may be further configured to: responding to receiving an acquisition request for the labeling webpage sent by the labeling task execution end; determining whether the labeling task execution end sending the acquisition request is a labeling task execution end indicated by the labeling task execution end information; and the labeling task execution end responding to the acquisition request is a labeling task execution end indicated by the labeling task execution end information, and the labeling webpage is sent to the labeling task execution end sending the acquisition request.
It can be understood that the information of the labeling task execution end input by the first user may be information of the labeling task execution end on which the first user desires to execute the labeling task initiated by the first user. After the server side obtains the obtaining request of the webpage for marking, the server side can determine whether the marking task execution end sending the request is the marking task execution end expected by the first user, and if yes, the webpage for marking can be sent to the marking task execution end sending the request.
The labeling task in the implementation manner can be executed by the labeling task execution end designated by the first user, which is helpful for reducing the possibility of data transmission and improving the security of the data.
In step 208, the labeling task execution terminal presents the labeling web page.
In this embodiment, after the labeling task execution end obtains the labeling webpage, the labeling webpage may be presented to the second user, so that the second user may label the sample to be labeled in the labeling webpage.
In some optional implementations of this embodiment, the server may be further configured to: and obtaining a marked sample obtained after the second user marks the sample to be marked in the webpage for marking by using the marking tool in the webpage for marking.
In this implementation manner, after the labeling task execution end presents the labeling webpage to the second user, the server end may obtain a labeled sample obtained after the second user labels the sample to be labeled in the labeling webpage by using the labeling tool in the labeling webpage. The marked sample can correspond to marking information generated by marking the sample to be marked by a second user through a marking tool.
As an example, the sample to be marked on the marking web page is an image to be marked, and the marking tools are labels "landscape" and "people". The second user can label the image to be labeled by using a label landscape to obtain a labeled image, wherein the labeled image can correspond to labeling information landscape.
Note that, the labeling information corresponding to the labeled sample may be included in the labeled sample (for example, included in the labeled image), or may be stored in association with the labeled sample.
In some optional implementations of this embodiment, the server may be further configured to: generating a marked webpage comprising a marked sample; and the labeling task initiation end may be further configured to: responding to the receiving of an acquisition request of a first user for the marked webpage, and acquiring the marked webpage from a server; and presenting the marked webpage.
Specifically, the server may store a program for generating the noted web page in advance, and further, the server may input the noted sample into the program after obtaining the noted sample, to obtain the noted web page.
According to the method, the marked webpage is presented to the first user, so that the first user can conveniently check and download marked samples through the marked webpage, and the integrity of the sample marking flow is improved.
In some optional implementations of this embodiment, the server may be further configured to: acquiring an initial marked sample obtained after a second user marks a sample to be marked in a webpage for marking by using a marking tool in the webpage for marking; generating an initial marked webpage comprising an initial marked sample; and the labeling task execution end may be further configured to: responding to the receiving of an acquisition request of a second user for the initial marked webpage, and acquiring the initial marked webpage from the server; presenting the webpage after the initial annotation; and the server may be further configured to: and in response to detecting the confirmation operation of the second user on the initial marked sample in the initial marked webpage, determining the initial marked sample confirmed by the second user as the marked sample.
The confirmation operation may be various operations, for example, an operation of clicking a confirmation icon. Specifically, after the second user obtains the webpage after initial annotation, the initial post-annotation sample in the webpage after initial annotation and the annotation information corresponding to the initial post-annotation sample can be checked, and if the verification is correct, the confirmation operation can be executed.
It will be appreciated that if the second user finds that the annotation information is incorrect during the verification process, the annotation information may also be modified.
The method can increase the confirmation link of the initial marked sample before the final marked sample is generated, and is beneficial to improving the accuracy of the finally obtained marked sample.
With continued reference to fig. 3, fig. 3 is a schematic diagram of an application scenario of the system for labeling samples according to the present embodiment.
In the application scenario of fig. 3, first, a computer (i.e., an annotation task initiator) 301 obtains an annotation task initiation web page 305 from a server 304 in response to receiving an acquisition request 303 for the annotation task initiation web page by a first user 302, and presents the annotation task initiation web page 305.
Then, the server 304 may obtain the labeling tool information 306 and the set of samples to be labeled 307 input by the first user 302 through the labeling task initiation web page 305 presented by the computer 301, and generate the web page 308 for labeling based on the labeling tool information 306 and the set of samples to be labeled 307, where the web page 308 for labeling includes the samples to be labeled in the set of samples to be labeled 307 and labeling tools indicated by the labeling tool information 307, and the labeling tools include labels.
Finally, the computer (i.e., labeling task execution end) 309 may obtain the labeling web page 308 from the server 304 and render the labeling web page 308 in response to receiving the obtaining request 311 from the second user 310 for the labeling web page 308.
In particular, after the computer 309 presents the labeling web page 308 to the second user 310, the server 304 may obtain a post-labeling sample 312 obtained after the second user 310 labels the sample to be labeled in the labeling web page 308 using the labeling tool in the labeling web page 308 presented by the computer 309.
The marking task initiating terminal and the marking task executing terminal in the system provided by the embodiment of the invention only need to install the browser for accessing the webpage managed by the server, so that the initiation of the marking task and the execution of the marking task can be realized, and compared with the C/S architecture mode in the prior art, the marking task initiating terminal and the marking task executing terminal do not need to be limited by the operating system, and the application range of the marking system is improved; in addition, in the labeling process, the labeling task execution end only needs to add labels on the samples to be labeled in the labeling webpage, and the labeling task execution end does not need to download the samples to be labeled, so that the control of the propagation of the samples to be labeled is facilitated, and the safety of the samples to be labeled is improved.
With continued reference to fig. 4, a flow 400 of one embodiment of a method for labeling a sample according to the present disclosure is shown. The method for labeling a sample comprises the following steps:
step 401, in response to receiving an acquisition request for the labeling task initiating webpage sent by the first user by using the labeling task initiating terminal, sending the labeling task initiating webpage to the labeling task initiating terminal, so that the labeling task initiating terminal presents the labeling task initiating webpage to the first user.
In this embodiment, the execution body (e.g., the server shown in fig. 1) of the method for labeling a sample may send the labeling task initiation web page to the labeling task initiation end in response to receiving, by a wired connection manner or a wireless connection manner, an acquisition request for the labeling task initiation web page sent by the first user using the labeling task initiation end (e.g., the labeling task initiation device shown in fig. 1), so that the labeling task initiation end presents the labeling task initiation web page to the first user. Wherein the first user may be a user initiating an annotation task. The labeling task may be a task that labels a sample to be labeled. The labeling task initiation web page may be a preset web page for initiating a labeling task. The labeling task initiation web page may be managed by the execution body described above.
Step 402, obtaining marking tool information and a sample set to be marked, which are input by a first user through a marking task initiating webpage.
In this embodiment, after the execution body sends the labeling task initiating webpage to the labeling task initiating terminal, the input operation performed by the first user using the labeling task initiating webpage may be detected, and the labeling tool information and the sample set to be labeled input by the first user may be obtained. The type of the sample to be marked in the sample set to be marked can be arbitrary. The marking tool information may be used to indicate the marking tool. The labeling tools may include various tools for labeling a sample to be labeled. In particular, the labeling tool may include a tag, and may also include other tools.
And step 403, generating a webpage for marking based on the marking tool information and the sample set to be marked.
In this embodiment, the execution body may generate the web page for labeling based on the labeling tool information and the sample set to be labeled obtained in step 402. The webpage for marking comprises a sample to be marked in the sample set to be marked and marking tools indicated by marking tool information.
And step 404, in response to receiving the request for acquiring the labeling webpage sent by the second user by using the labeling task execution end, sending the labeling webpage to the labeling task execution end so that the labeling task execution end presents the labeling webpage to the second user.
In this embodiment, after the labeling web page is generated, the executing body may send the labeling web page to the labeling task executing end in response to receiving, by using a wired connection manner or a wireless connection manner, an acquisition request for the labeling web page sent by the second user by using the labeling task executing end, so that the labeling task executing end presents the labeling web page to the second user. Wherein the second user may be a user performing an annotation task.
It should be noted that, the execution body in the embodiment corresponding to fig. 4 may have the same features as the service end in the system for labeling samples, so the embodiment corresponding to fig. 4 may be combined with the features of the service end in the system for labeling samples to form a new technical solution. In order to avoid repetition, the description is omitted here.
The method for labeling the sample provided by the embodiment of the disclosure can acquire the sample to be labeled and the labeling tool information from the labeling task initiating terminal in the form of a webpage, and acquire the labeled sample corresponding to the sample to be labeled from the labeling task executing terminal in the form of a webpage, compared with the C/S architecture mode in the prior art, the labeling task initiating terminal and the task executing terminal in the disclosure only need to install the browser for browsing the webpage, do not need to limit the operating systems of the labeling task initiating terminal and the labeling task executing terminal, and are beneficial to improving the application range of the labeling method; in addition, in the labeling process, the labeling task execution end only needs to add labels on the samples to be labeled in the labeling webpage, and the labeling task execution end does not need to download the samples to be labeled, so that the control of the propagation of the samples to be labeled is facilitated, and the safety of the samples to be labeled is improved.
With further reference to fig. 5, as an implementation of the method shown in the foregoing figures, the present disclosure provides an embodiment of an apparatus for labeling a sample, which corresponds to the method embodiment shown in fig. 4, and which is particularly applicable to various electronic devices.
As shown in fig. 5, the apparatus 500 for labeling a sample of the present embodiment includes: a first transmission unit 501, an acquisition unit 502, a generation unit 503, and a second transmission unit 504. Wherein, the first sending unit 501 is configured to send the labeling task initiating webpage to the labeling task initiating terminal in response to receiving an obtaining request for the labeling task initiating webpage sent by the first user by using the labeling task initiating terminal, so that the labeling task initiating terminal presents the labeling task initiating webpage to the first user; the obtaining unit 502 is configured to obtain labeling tool information and a sample set to be labeled, which are input by a first user initiating a webpage through a labeling task; the generating unit 503 is configured to generate a web page for labeling based on the labeling tool information and the sample set to be labeled, where the web page for labeling includes a sample to be labeled in the sample set to be labeled and a labeling tool indicated by the labeling tool information, and the labeling tool includes a label; the second sending unit 504 is configured to send the labeling web page to the labeling task execution end in response to receiving the acquisition request for the labeling web page sent by the second user with the labeling task execution end, so that the labeling task execution end presents the labeling web page to the second user.
In this embodiment, the first sending unit 501 of the apparatus 500 for labeling samples may send the labeling task initiation web page to the labeling task initiation end in response to receiving, by a wired connection manner or a wireless connection manner, an acquisition request for the labeling task initiation web page sent by the first user using the labeling task initiation end (for example, the labeling task initiation device shown in fig. 1), so that the labeling task initiation end presents the labeling task initiation web page to the first user. Wherein the first user may be a user initiating an annotation task. The labeling task may be a task that labels a sample to be labeled. The labeling task initiation web page may be a preset web page for initiating a labeling task. The annotation task initiating web page may be managed by the apparatus 500.
In this embodiment, after the first sending unit 501 sends the labeling task initiating web page to the labeling task initiating end, the obtaining unit 502 may obtain the labeling tool information and the sample set to be labeled, which are input by the first user, in response to detecting the input operation performed by the first user using the labeling task initiating web page. The type of the sample to be marked in the sample set to be marked can be arbitrary. The marking tool information may be used to indicate the marking tool. The labeling tools may include various tools for labeling a sample to be labeled. In particular, the labeling tool may include a tag, and may also include other tools.
In this embodiment, the generating unit 503 may generate a web page for labeling based on the labeling tool information and the sample set to be labeled obtained by the obtaining unit 502. The webpage for marking comprises a sample to be marked in the sample set to be marked and marking tools indicated by marking tool information.
In this embodiment, after the generating unit 503 generates the labeling web page, the second transmitting unit 504 transmits the labeling web page to the labeling task executing end in response to receiving, by the wired connection or the wireless connection, an acquisition request for the labeling web page, which is transmitted by the second user using the labeling task executing end, so that the labeling task executing end presents the labeling web page to the second user. Wherein the second user may be a user performing an annotation task.
It will be appreciated that the elements described in the apparatus 500 correspond to the various steps in the method described with reference to fig. 4. Thus, the operations, features and resulting benefits described above with respect to the method are equally applicable to the apparatus 500 and the units contained therein, and are not described in detail herein.
The device 500 provided in the foregoing embodiment of the present disclosure may obtain, in a web page form, a sample to be marked and marking tool information from a marking task initiating terminal, and obtain, in a web page form, a sample after marking corresponding to the sample to be marked from a marking task executing terminal, where, compared with a C/S architecture mode in the prior art, the marking task initiating terminal and the task executing terminal in the present disclosure only need to install a browser for browsing a web page, without restricting operating systems of the marking task initiating terminal and the marking task executing terminal, which is helpful for improving an application range of a marking method; in addition, in the labeling process, the labeling task execution end only needs to add labels on the samples to be labeled in the labeling webpage, and the labeling task execution end does not need to download the samples to be labeled, so that the control of the propagation of the samples to be labeled is facilitated, and the safety of the samples to be labeled is improved.
Referring now to fig. 6, a schematic diagram of an electronic device (e.g., server in fig. 1) 600 suitable for use in implementing embodiments of the present disclosure is shown. The electronic device shown in fig. 6 is merely an example and should not be construed to limit the functionality and scope of use of the disclosed embodiments.
As shown in fig. 6, the electronic device 600 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 601, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data required for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
In general, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, and the like; an output device 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, magnetic tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 shows an electronic device 600 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via communication means 609, or from storage means 608, or from ROM 602. The above-described functions defined in the methods of the embodiments of the present disclosure are performed when the computer program is executed by the processing device 601.
It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: responding to the received acquisition request for the labeling task initiating webpage sent by the first user by using the labeling task initiating terminal, and sending the labeling task initiating webpage to the labeling task initiating terminal so that the labeling task initiating terminal presents the labeling task initiating webpage to the first user; acquiring marking tool information and a sample set to be marked, which are input by a first user through a marking task initiating webpage; generating a webpage for marking based on marking tool information and a sample set to be marked, wherein the webpage for marking comprises a sample to be marked in the sample set to be marked and marking tools indicated by the marking tool information, and the marking tools comprise labels; and in response to receiving an acquisition request for the labeling web page, which is sent by the second user by using the labeling task execution end, sending the labeling web page to the labeling task execution end, so that the labeling task execution end presents the labeling web page to the second user.
Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. The name of the unit does not in some case limit the unit itself, for example, the first sending unit may also be described as "a unit that sends a labeling task initiating web page".
The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by persons skilled in the art that the scope of the disclosure referred to in this disclosure is not limited to the specific combinations of features described above, but also covers other embodiments which may be formed by any combination of features described above or equivalents thereof without departing from the spirit of the disclosure. Such as those described above, are mutually substituted with the technical features having similar functions disclosed in the present disclosure (but not limited thereto).
Claims (11)
1. A system for labeling a sample, comprising: the labeling task initiating terminal, the service terminal and the labeling task executing terminal, wherein,
the labeling task initiating terminal is configured to: responding to receiving an acquisition request of a first user for a labeling task initiating webpage, and acquiring the labeling task initiating webpage from the server; presenting the labeling task initiating webpage;
The server is configured to: acquiring marking tool information and a sample set to be marked, which are input by the first user through the marking task initiating webpage; generating a webpage for marking based on the marking tool information and the sample set to be marked, wherein the webpage for marking comprises a sample to be marked in the sample set to be marked and a marking tool indicated by the marking tool information, and the marking tool comprises a label;
the labeling task execution end is configured to: responding to the receiving of an acquisition request of a second user for the labeling webpage, and acquiring the labeling webpage from the server; presenting the webpage for labeling;
wherein the server is further configured to: acquiring initial marking tool information and sample type information which are input by the first user through the marking task initiating webpage; obtaining a sample for previewing from a preset sample set, wherein the sample type of the sample for previewing is the same as the sample type indicated by the sample type information; generating a webpage for previewing based on the initial labeling tool information and the sample for previewing; and the labeling task initiator is further configured to: responding to the receiving of an acquisition request of the first user for a webpage for previewing, and acquiring the webpage for previewing from the server; presenting the webpage for previewing, wherein the webpage for previewing comprises an initial marking tool and a sample for previewing, which are indicated by the initial marking tool information, so that the first user marks the sample for previewing by using the initial marking tool in the webpage for previewing; and the server is further configured to: in response to detecting a confirmation operation of the first user for an initial marking tool in the webpage for previewing, determining the initial marking tool information as marking tool information.
2. The system of claim 1, wherein the server is further configured to: and obtaining a marked sample obtained after the second user marks the sample to be marked in the webpage by using the marking tool in the webpage for marking.
3. The system of claim 2, wherein the server is further configured to: generating a marked webpage comprising the marked sample; and
the labeling task initiator is further configured to: responding to the receiving of an acquisition request of a first user for the marked webpage, and acquiring the marked webpage from the server; and presenting the marked webpage.
4. The system of claim 2, wherein the server is further configured to: acquiring an initial marked sample obtained after the second user marks the sample to be marked in the webpage by using a marking tool in the webpage for marking; generating an initial marked webpage comprising the initial marked sample; and
the labeling task execution end is further configured to: responding to the receiving of the acquisition request of the second user for the initial marked webpage, and acquiring the initial marked webpage from the server; presenting the initial marked webpage; and
The server is further configured to: and in response to detecting the confirmation operation of the second user on the initial marked sample in the initial marked webpage, determining the initial marked sample confirmed by the second user as the marked sample.
5. The system of claim 1, wherein the server is further configured to: acquiring annotation task execution end information input by the first user through the annotation task initiating webpage; responding to receiving an acquisition request for the labeling webpage sent by a labeling task execution end; determining whether the labeling task execution end sending the acquisition request is a labeling task execution end indicated by the labeling task execution end information; and responding to the labeling task execution end which sends the acquisition request is the labeling task execution end indicated by the labeling task execution end information, and sending the webpage for labeling to the labeling task execution end which sends the acquisition request.
6. The system of claim 1, wherein the marking tool indicated by the marking tool information includes more than two levels of labels.
7. The system of one of claims 1 to 6, wherein the type of sample to be annotated in the set of samples to be annotated comprises at least one of: image class, text class, speech class, video class.
8. A method for labeling a sample, comprising:
responding to a received acquisition request sent by a first user by using a labeling task initiating terminal and aiming at a labeling task initiating webpage, and sending the labeling task initiating webpage to the labeling task initiating terminal so that the labeling task initiating terminal presents the labeling task initiating webpage to the first user;
acquiring marking tool information and a sample set to be marked, which are input by the first user through the marking task initiating webpage;
generating a webpage for marking based on the marking tool information and the sample set to be marked, wherein the webpage for marking comprises a sample to be marked in the sample set to be marked and a marking tool indicated by the marking tool information, and the marking tool comprises a label;
responding to the received acquisition request for the labeling webpage sent by the second user by using the labeling task execution end, and sending the labeling webpage to the labeling task execution end so that the labeling task execution end presents the labeling webpage to the second user;
the method further comprises the steps of: acquiring initial marking tool information and sample type information which are input by the first user through the marking task initiating webpage; obtaining a sample for previewing from a preset sample set, wherein the sample type of the sample for previewing is the same as the sample type indicated by the sample type information; generating a webpage for previewing based on the initial labeling tool information and the sample for previewing; in response to receiving an acquisition request of the first user for a webpage for previewing, sending the webpage for previewing to the marking task initiating terminal so that the marking task initiating terminal can present the webpage for previewing, wherein the webpage for previewing comprises an initial marking tool and a sample for previewing, which are indicated by the initial marking tool information, so that the first user marks the sample for previewing by using the initial marking tool in the webpage for previewing; in response to detecting a confirmation operation of the first user for an initial marking tool in the webpage for previewing, determining the initial marking tool information as marking tool information.
9. An apparatus for labeling a sample, comprising:
the first sending unit is configured to send the labeling task initiating webpage to the labeling task initiating terminal in response to receiving an acquisition request for the labeling task initiating webpage sent by a first user by using the labeling task initiating terminal, so that the labeling task initiating terminal presents the labeling task initiating webpage to the first user;
the acquisition unit is configured to acquire marking tool information and a sample set to be marked, which are input by the first user through the marking task initiating webpage;
the generation unit is configured to generate a webpage for marking based on the marking tool information and the sample set to be marked, wherein the webpage for marking comprises a sample to be marked in the sample set to be marked and a marking tool indicated by the marking tool information, and the marking tool comprises a label;
the second sending unit is configured to send the webpage for marking to the marking task execution end in response to receiving an acquisition request for the webpage for marking, which is sent by a second user by using the marking task execution end, so that the marking task execution end presents the webpage for marking to the second user;
The generating unit is further configured to acquire initial marking tool information and sample type information which are input by the first user through the marking task initiating webpage; obtaining a sample for previewing from a preset sample set, wherein the sample type of the sample for previewing is the same as the sample type indicated by the sample type information; generating a webpage for previewing based on the initial labeling tool information and the sample for previewing;
the first sending unit is further configured to send the webpage for preview to the labeling task initiating terminal in response to receiving an acquisition request of the first user for the webpage for preview, so that the labeling task initiating terminal will present the webpage for preview, wherein the webpage for preview comprises an initial labeling tool and a sample for preview indicated by the initial labeling tool information, so that the first user labels the sample for preview by using the initial labeling tool in the webpage for preview;
and a determining unit configured to determine the initial marking tool information as marking tool information in response to detecting a confirmation operation of the first user with respect to the initial marking tool in the web page for preview.
10. A server, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method as recited in claim 8.
11. A computer readable medium having stored thereon a computer program, wherein the program when executed by a processor implements the method as claimed in claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010174981.7A CN111400581B (en) | 2020-03-13 | 2020-03-13 | System, method and apparatus for labeling samples |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010174981.7A CN111400581B (en) | 2020-03-13 | 2020-03-13 | System, method and apparatus for labeling samples |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111400581A CN111400581A (en) | 2020-07-10 |
| CN111400581B true CN111400581B (en) | 2024-02-06 |
Family
ID=71434212
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010174981.7A Active CN111400581B (en) | 2020-03-13 | 2020-03-13 | System, method and apparatus for labeling samples |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111400581B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113407083A (en) * | 2021-06-24 | 2021-09-17 | 上海商汤科技开发有限公司 | Data labeling method and device, electronic equipment and storage medium |
| CN113822137A (en) * | 2021-07-23 | 2021-12-21 | 腾讯科技（深圳）有限公司 | Data annotation method, device and equipment and computer readable storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101739415A (en) * | 2008-11-25 | 2010-06-16 | 华中师范大学 | Browser-oriented webpage labeling system |
| CN103678510A (en) * | 2013-11-25 | 2014-03-26 | 北京奇虎科技有限公司 | Method and device for providing visualized label for webpage |
| CN105446969A (en) * | 2014-06-05 | 2016-03-30 | 北大方正信息产业集团有限公司 | Webpage generation method, device and server |
| CN106156025A (en) * | 2015-03-25 | 2016-11-23 | 阿里巴巴集团控股有限公司 | The management method of a kind of data mark and device |
| CN107066437A (en) * | 2017-02-22 | 2017-08-18 | 深圳市天朗时代科技有限公司 | The method and device of copyright mark |
| CN109325213A (en) * | 2018-09-30 | 2019-02-12 | 北京字节跳动网络技术有限公司 | Method and apparatus for labeled data |
| CN109446357A (en) * | 2018-10-18 | 2019-03-08 | 杭州快忆科技有限公司 | Data mask method and device |
| CN110209849A (en) * | 2019-06-04 | 2019-09-06 | 北京字节跳动网络技术有限公司 | Method and apparatus for marking key point |
| CN110390067A (en) * | 2019-07-22 | 2019-10-29 | 出门问问(苏州)信息科技有限公司 | A kind of webpage label method, apparatus, storage medium and electronic equipment |
-
2020
- 2020-03-13 CN CN202010174981.7A patent/CN111400581B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101739415A (en) * | 2008-11-25 | 2010-06-16 | 华中师范大学 | Browser-oriented webpage labeling system |
| CN103678510A (en) * | 2013-11-25 | 2014-03-26 | 北京奇虎科技有限公司 | Method and device for providing visualized label for webpage |
| CN105446969A (en) * | 2014-06-05 | 2016-03-30 | 北大方正信息产业集团有限公司 | Webpage generation method, device and server |
| CN106156025A (en) * | 2015-03-25 | 2016-11-23 | 阿里巴巴集团控股有限公司 | The management method of a kind of data mark and device |
| CN107066437A (en) * | 2017-02-22 | 2017-08-18 | 深圳市天朗时代科技有限公司 | The method and device of copyright mark |
| CN109325213A (en) * | 2018-09-30 | 2019-02-12 | 北京字节跳动网络技术有限公司 | Method and apparatus for labeled data |
| CN109446357A (en) * | 2018-10-18 | 2019-03-08 | 杭州快忆科技有限公司 | Data mask method and device |
| CN110209849A (en) * | 2019-06-04 | 2019-09-06 | 北京字节跳动网络技术有限公司 | Method and apparatus for marking key point |
| CN110390067A (en) * | 2019-07-22 | 2019-10-29 | 出门问问(苏州)信息科技有限公司 | A kind of webpage label method, apparatus, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111400581A (en) | 2020-07-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107622135B (en) | Method and apparatus for displaying information | |
| US11023716B2 (en) | Method and device for generating stickers | |
| CN109325213B (en) | Method and device for labeling data | |
| US9804862B2 (en) | Translatable texts identification in in-context localization utilizing pseudo-language and an external server | |
| CN109302461B (en) | Information display method, information processing method, information display medium, information processing system and computing equipment | |
| US11758088B2 (en) | Method and apparatus for aligning paragraph and video | |
| CN110020329B (en) | Method, device and system for generating webpage | |
| US10776444B1 (en) | Methods and systems for universal deep linking across web and mobile applications | |
| CN109981787B (en) | Method and device for displaying information | |
| US11106754B1 (en) | Methods and systems for hyperlinking user-specific content on a website or mobile applications | |
| US20210294864A1 (en) | Methods and systems for generating custom content using universal deep linking across web and mobile applications | |
| CN112684968A (en) | Page display method and device, electronic equipment and computer readable medium | |
| US11687715B2 (en) | Summary generation method and apparatus | |
| CN110058854B (en) | Method, terminal device and computer-readable medium for generating application | |
| CN111400581B (en) | System, method and apparatus for labeling samples | |
| CN109413056B (en) | Method and apparatus for processing information | |
| CN110232091A (en) | Mthods, systems and devices for synchrodata | |
| WO2024099171A1 (en) | Video generation method and apparatus | |
| CN109840072B (en) | Information processing method and device | |
| CN110673886B (en) | Method and device for generating thermodynamic diagrams | |
| CN109710866A (en) | Image display method and device in online document | |
| CN111026849B (en) | Data processing method and device | |
| CN112084441A (en) | Information retrieval method and device and electronic equipment | |
| CN113407229B (en) | Method and device for generating offline scripts | |
| CN111930761B (en) | Information interaction method and device and model-view-renderer architecture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 221, 2nd floor, Block C, 18 Kechuang 11th Street, Daxing Economic and Technological Development Zone, Beijing, 100176 Applicant after: Jingdong Technology Holding Co.,Ltd. Address before: Room 221, 2nd floor, Block C, 18 Kechuang 11th Street, Daxing Economic and Technological Development Zone, Beijing, 100176 Applicant before: Jingdong Digital Technology Holding Co.,Ltd. Address after: Room 221, 2nd floor, Block C, 18 Kechuang 11th Street, Daxing Economic and Technological Development Zone, Beijing, 100176 Applicant after: Jingdong Digital Technology Holding Co.,Ltd. Address before: Room 221, 2nd floor, Block C, 18 Kechuang 11th Street, Daxing Economic and Technological Development Zone, Beijing, 100176 Applicant before: JINGDONG DIGITAL TECHNOLOGY HOLDINGS Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |