CN113139109A - Data labeling method, device, server and storage medium - Google Patents
Data labeling method, device, server and storage medium Download PDFInfo
- Publication number
- CN113139109A CN113139109A CN202110425810.1A CN202110425810A CN113139109A CN 113139109 A CN113139109 A CN 113139109A CN 202110425810 A CN202110425810 A CN 202110425810A CN 113139109 A CN113139109 A CN 113139109A
- Authority
- CN
- China
- Prior art keywords
- data
- access mode
- service
- task
- mode
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/906—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The application discloses a data labeling method, a device, a server and a storage medium, wherein the data identification method is applied to the server of a labeling platform, and the method comprises the following steps: receiving a data annotation task sent by a service end; according to the data marking task, determining a corresponding online data access mode as a target access mode and a corresponding online data feedback mode as a target feedback mode; acquiring data to be labeled corresponding to the data labeling task according to the target access mode; marking the data to be marked according to the data marking task to obtain a data marking result; and feeding back the data labeling result to the service end according to the target feedback mode. The method can improve the efficiency of data annotation.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data annotation method, an apparatus, a server, and a storage medium.
Background
Data annotation refers to the act of processing data according to a data annotation task. In the related technology of data annotation, the data access method and delivery method usually adopted are single, and cannot cover multiple data sources of a service party, which may cause data to be accessed and delivered offline, and thus the efficiency of data annotation is low.
Disclosure of Invention
In view of the foregoing problems, the present application provides a data annotation method, apparatus, server and storage medium.
In a first aspect, an embodiment of the present application provides a data annotation method, which is applied to a server of an annotation platform, and the method includes: receiving a data annotation task sent by a service end; according to the data marking task, determining a corresponding online data access mode as a target access mode and a corresponding online data feedback mode as a target feedback mode; acquiring data to be labeled corresponding to the data labeling task according to the target access mode; marking the data to be marked according to the data marking task to obtain a data marking result; and feeding back the data labeling result to the service end according to the target feedback mode.
In a second aspect, an embodiment of the present application provides a data annotation device, which is applied to a server of an annotation platform, where the device includes: the system comprises a task receiving module, a mode determining module, a data acquiring module, a label executing module and a result feedback module, wherein the task receiving module is used for receiving a data label task sent by a service end; the mode determining module is used for determining a corresponding online data access mode as a target access mode and a corresponding online data feedback mode as a target feedback mode according to the data marking task; the data acquisition module is used for acquiring the data to be labeled corresponding to the data labeling task according to the target access mode; the marking execution module is used for marking the data to be marked according to the data marking task to obtain a data marking result; and the result feedback module is used for feeding back the data labeling result to the service end according to the target feedback mode.
In a third aspect, an embodiment of the present application provides a server, including: one or more processors; a memory; one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the data annotation methods provided above in the first aspect.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a program code is stored in the computer-readable storage medium, and the program code may be called by a processor to execute the data annotation method provided in the first aspect.
According to the scheme, a server of a labeling platform receives a data labeling task sent by a service end, determines a corresponding online data access mode as a target access mode and a corresponding online data feedback mode as a target feedback mode according to the data labeling task, acquires data to be labeled corresponding to the data labeling task according to the target access mode, labels the data to be labeled according to the data labeling task to obtain a data labeling result, and finally feeds the data labeling result back to the service end according to the target feedback mode, so that the corresponding online data access mode can be determined according to the data labeling task, the data access mode during data labeling can cover the data access requirement of the service end, meanwhile, the corresponding online data feedback mode can be provided for the labeling task, and the data labeling efficiency is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 shows a schematic diagram of an application scenario provided in an embodiment of the present application.
FIG. 2 shows a flow diagram of a data annotation process according to one embodiment of the present application.
FIG. 3 shows a flow diagram of a data annotation process according to another embodiment of the present application.
Fig. 4 shows an interface schematic diagram provided in an embodiment of the present application.
FIG. 5 shows a flow diagram of a data annotation process according to yet another embodiment of the present application.
Fig. 6 shows a flowchart of step S340 in a data annotation method according to another embodiment of the present application.
FIG. 7 is a flow chart illustrating a method for annotating data according to yet another embodiment of the present application.
FIG. 8 shows a flow diagram of a data annotation process according to yet another embodiment of the present application.
FIG. 9 shows a block diagram of a data annotation device, in accordance with one embodiment of the present application.
Fig. 10 is a block diagram of a server for executing a data annotation method according to an embodiment of the present application.
Fig. 11 is a storage unit according to an embodiment of the present application, configured to store or carry program code for implementing a data annotation method according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.
In the related technology of data annotation, the data annotation is usually performed by means of offline access and offline delivery, but this way has a certain loss in timeliness of data access and delivery; although there may be a partial data labeling scheme performed by using an online data access method, both the access method and the delivery method are fixed and single, and when data provided by a service party is from different data sources, it is not possible to cover multiple data sources of the service party, which may cause that partial data needs to be performed by using offline access and offline delivery methods, so that the efficiency of data labeling is low.
In view of the above problems, the inventor provides a data annotation method, a device, an electronic device, and a storage medium provided in the embodiments of the present application, which can determine a corresponding online data access manner according to a data annotation task, so that the data access manner during data annotation can cover a data access requirement of a service end, and meanwhile, a corresponding online data feedback manner can be provided for the annotation task, thereby improving data annotation efficiency. The specific data labeling method is described in detail in the following embodiments.
Fig. 1 is a schematic diagram of an implementation environment of a data annotation method provided in an embodiment of the present application, and referring to fig. 1, the implementation environment includes a server 100 and a service end 200. The service end 200 is connected to the server 100 through a wireless network or a wired network, and data interaction can be performed between the server 100 and the service end 200.
Optionally, the server 100 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a web service, cloud communication, a middleware service, a domain name service, a security service, a Delivery Network (CDN), a big data and artificial intelligence platform, but is not limited thereto.
Alternatively, the service end 200 may be a mobile terminal (e.g., a smart phone, a tablet computer, a smart watch, a smart band, etc.), a notebook computer, a PC computer, a server, etc., but is not limited thereto. The service end 200 generally refers to one of a plurality of service ends, and the embodiment of the present application is illustrated by the service end 200. Those skilled in the art will appreciate that the number of mobile terminals described above may be greater or fewer. For example, the number of the mobile terminals is only 1, or the number of the mobile terminals is tens or hundreds, or more, and in this case, other terminals are also included in the implementation environment. The number and the device type of the mobile terminals are not limited in the embodiment of the application.
In some embodiments, the service end 200 may be a device used by a user, and the service end 200 may generate a data annotation task according to an operation of the user, or receive a data annotation task input by the user through another device, and then submit the data annotation task to the server 100; after receiving the data annotation task, the server 100 may determine a corresponding online data access manner according to the data annotation task, then obtain data to be annotated corresponding to the data annotation task according to the determined online data access manner, perform annotation corresponding to the data annotation task on the data to be annotated, obtain a data annotation result, and finally feed back the data annotation result to the service end 200 according to a data feedback manner corresponding to the online data access manner, thereby providing a corresponding online data access manner and a data feedback manner for different data annotation tasks, enabling the data access manner during data annotation to cover the data access requirement of the service end, and improving the data annotation efficiency.
Referring to fig. 2, fig. 2 is a schematic flow chart illustrating a data annotation method according to an embodiment of the present application. In a specific embodiment, the data annotation method is applied to the data annotation device 400 shown in fig. 9 and the server 100 (fig. 10) configured with the data annotation device 400. The specific flow of the present embodiment will be described below by taking a server as an example, and it is understood that the server applied in the present embodiment may be a physical server, a cloud server, and the like, and is not limited herein. As will be described in detail with respect to the flow shown in fig. 2, the data annotation method may specifically include the following steps:
step S110: and receiving a data annotation task sent by a service end.
The service end can send a data annotation task to a server of the annotation platform so as to complete the data annotation task required by the user. The annotation platform can be loaded on a server to provide annotation service for data to a service end. The data marking refers to the action of processing data according to a data marking task, and mainly marks corresponding labels on the data.
In some embodiments, when the service end is a user terminal, the data annotation task may be generated according to an operation of a user. Specifically, the service end may generate a data tagging task according to a data type to be tagged, a tagged service scenario, and a tagged label system input by a user. The labeled data type refers to a data type corresponding to the labeled data, such as text, audio, image, video, and the like; the marked service scene is a marking type required by a marking task, such as voice transcription, portrait marking, text classification and the like; the label system refers to a set formed by labels corresponding to the service scene, for example, when the service scene is a text classification, the label system may be a category set to be labeled for the data to be labeled. Optionally, the generated data annotation task may include the data type to be annotated, the annotated service scenario, and the annotated label system. Of course, the data annotation task may also include information such as the source of the data to be annotated.
Optionally, after receiving the data annotation task sent by the service end, the server may determine whether information carried in the data annotation task is complete; if the information carried in the data annotation task is incomplete, an information supplement request can be returned to the service end to indicate the service end to supplement the information which is temporarily absent in the data annotation task; if the information carried in the data annotation task is complete, the server can perform subsequent processing steps for the data annotation task.
In other embodiments, when the service end is a server, the information related to the data annotation task uploaded by the user end, such as the above data type to be annotated, the annotated service scenario, and the annotated tag system, may be received, and then the data annotation task may be generated according to the information related to the data annotation task uploaded by the user end.
Correspondingly, after the service end sends the data annotation task to the server of the annotation platform, the server can receive the data annotation task, and the subsequent server can complete the data annotation required by the service end according to the data annotation task.
Step S120: and determining a corresponding online data access mode as a target access mode and a corresponding online data feedback mode as a target feedback mode according to the data marking task.
In the embodiment of the application, after receiving the data annotation task sent by the service end, the server of the annotation platform can respond to the data annotation task. When the data annotation task is completed, the server can determine an online data access mode corresponding to the data annotation task as a target access mode from a plurality of online data access modes according to the data annotation task; moreover, the online data feedback mode corresponding to the data annotation task can be determined as a target feedback mode from a plurality of online data feedback modes.
In some embodiments, the online data access mode when the server of the annotation platform performs data annotation may include: application Programming Interface (API), service data source access, and message queue access. The server can obtain the data to be pushed by the service end through the API interface; the service data source access refers to that a server acquires data to be marked from a corresponding data source, for example, acquires the data to be marked from a specified database, cloud storage and the like; the message queue access refers to that the server acquires data to be marked pushed by the service end from the corresponding message queue.
In some embodiments, the online data feedback manner of the server of the annotation platform on the data annotation result may include: feedback through the API, feedback through the service data source, and feedback through the message queue. The server feeds back the data labeling result to the service end through a corresponding API interface in an API feedback mode; writing the data annotation result into a corresponding service data source for the server in a service data source feedback mode, wherein the service data source can be a database, cloud storage and the like; and writing the data annotation result into the corresponding message queue by the server in a message queue feedback mode, so that the service end can obtain the data annotation result from the message queue.
In some embodiments, the service end may carry a required data access mode and a required data feedback mode when submitting the data annotation task, and therefore, when determining the corresponding online data access mode and the online data feedback mode according to the data annotation task, the server may determine the corresponding online data access mode and the online data feedback mode according to the data access mode and the data feedback mode carried in the data annotation task. For example, the data tagging task submitted by the service end carries API access and API feedback, and the server may use the corresponding online data access mode as the API access mode and use the corresponding online data access mode as the mode of API feedback.
In other embodiments, in submitting a data task, the server of the annotation platform may also feed back a selectable online data access manner and an online data feedback manner to the service end, so that the service end selects a required online data access manner and an online data feedback manner.
It can be understood that the target access mode may be one or multiple, and when the target access mode includes multiple online data access modes, the server may obtain the data to be labeled through the multiple online data modes; similarly, the target feedback mode may be one or more, and when the target feedback mode includes multiple online data feedback modes, the server may feed back the data annotation result to the service end through the multiple online data feedback modes. In addition, the target access mode may correspond to the target feedback mode, for example, if the target access mode is API access, the target feedback mode is feedback through API; the target access method and the target feedback method may be a free combination of the above online data access method and the online data feedback method.
In the embodiment of the application, after receiving the data annotation task, the server of the annotation platform may further generate an annotation scheme, an annotation template, an annotation flow, an annotation data format, and the like according to the requirement information carried in the data annotation task, such as the type of the annotation data, the scene of the annotation service, and the label system of the annotation, so that after subsequently acquiring the data to be annotated, the data to be annotated is annotated according to the generated annotation scheme, the generated annotation template, the generated annotation flow, the generated annotation data format, and the like.
Step S130: and acquiring the data to be labeled corresponding to the data labeling task according to the target access mode.
In the embodiment of the application, after the server determines the target access mode, the data to be annotated corresponding to the data annotation task can be acquired through the target access mode. If the target access mode is an API access mode, the server can acquire the data to be marked sent by the service end through the API; if the target access mode is a service data source access mode, the server can acquire the data to be marked by the appointed service data source; if the target access mode is a message queue access mode, the server can acquire the data to be labeled pushed by the service end from the message queue.
Step S140: and marking the data to be marked according to the data marking task to obtain a data marking result.
In the embodiment of the application, after the server of the annotation platform acquires the data to be annotated, the data to be annotated can be annotated according to the data annotation task, so that a data annotation result for feeding back to the service end is obtained.
In some embodiments, after receiving the data annotation task, the server may label the data to be annotated according to the labeling scheme, the labeling template, the labeling flow, and the labeling data format if a labeling scheme, a labeling template, a labeling flow, and a labeling data format are generated according to information such as the type of the labeling data, the scene of the labeling service, and the labeling label system carried in the data annotation task.
In some embodiments, the labeling of the data to be labeled may be manual labeling or automatic labeling. The manual marking mode refers to that a marking person manually marks each marking data correspondingly; the automatic labeling mode refers to automatically identifying the label corresponding to each data, and marking the corresponding label on each data to finish the labeling of the data to be labeled.
Illustratively, when the labeling mode is a manual labeling mode, the data to be labeled can be divided into a plurality of parts, and each part is taken as a labeling subtask to be allocated to equipment corresponding to a labeling person, so that the labeling person can label the allocated data; and after the quality inspection personnel finish the quality inspection, returning the data after the quality inspection to the server, so that the server obtains a data labeling result. Of course, the above is merely an example and does not represent a limitation on the way in which specific manual labeling is performed.
Step S150: and feeding back the data labeling result to the service end according to the target feedback mode.
In the embodiment of the application, after the server obtains the data annotation result, the server may feed back the data annotation result to the service end according to the determined target feedback manner. If the target access mode is a mode of feeding back through an API, the server can feed back a data annotation result to the service end through the API; if the target access mode is a service data source access mode, the server can feed back the data annotation result to the specified data source; if the target access mode is a message queue access mode, the server can write the data annotation result into the specified message queue so that the service end can obtain the data annotation result from the message queue.
In the data annotation method provided by the embodiment of the application, the server of the annotation platform receives the data annotation task sent by the service end, according to the data marking task, determining a corresponding on-line data access mode as a target access mode and a corresponding on-line data feedback mode as a target feedback mode, then the data to be labeled corresponding to the data labeling task is obtained according to the target access mode, the data to be labeled is labeled according to the data labeling task to obtain a data labeling result, finally the data labeling result is fed back to the service end according to the target feedback mode, thereby determining the corresponding on-line data access mode according to the data marking task, enabling the data access mode during data marking to cover the data access requirement of the service end, meanwhile, a corresponding online data feedback mode can be provided for the labeling task, and the data labeling efficiency is improved.
Referring to fig. 3, fig. 3 is a schematic flow chart illustrating a data annotation method according to another embodiment of the present application. The data annotation method is applied to the server, and will be described in detail with respect to the flow shown in fig. 3, where the data annotation method specifically includes the following steps:
step S210: and receiving a data annotation task sent by a service end.
In the embodiment of the present application, step S210 may refer to the contents of the foregoing embodiments, which are not described herein again.
Step S220: and determining a selectable online data access mode according to the data annotation task.
In the embodiment of the application, after the server of the annotation platform obtains the data annotation task, the server can determine an optional online data access mode according to the data annotation task. It can be understood that different data labeling tasks have different requirements and may have different requirements for data access modes, so that the server may determine an optional online data access mode according to the data labeling task, so as to be selected by the service end. For example, a scenario for which the data annotation task is directed to perform voice transcription on voice, the service end needs to acquire a transcription result in real time and perform an application scenario according to the transcription result, and in order to ensure real-time performance, an API access mode and a message queue access mode can ensure better timeliness, so that an optional online data access mode can be determined as the API access mode and the message queue access mode. It can be understood that the selectable online data access modes determined according to the data annotation task may be part of all online data access modes, or all online data access modes.
In some embodiments, when the data types corresponding to the data annotation tasks are different, the possible online data access modes are also different. Therefore, the server can determine a data access mode corresponding to the data type according to the data type corresponding to the data annotation task, and the data access mode can be selected as an online data access mode. Optionally, when the data types are different, different data access modes have an influence on the transmission efficiency of the data, and an optional online data access mode can be determined according to the data size corresponding to the data types; if the average data amount corresponding to the data type is greater than a specified threshold, an online data access mode with transmission efficiency meeting specified efficiency conditions can be selected, and the specified efficiency conditions can be that the data transmission speed is greater than a specified speed; if the average data amount corresponding to the data type is not larger than the specified threshold, all online data access modes can be selected as selectable online data access modes. For example, the service end is a user terminal, and when the data type is a video type or an audio type, the user terminal may determine that the data source access mode and the message queue access mode are selectable online data access modes because the transmission efficiency of the user terminal to the data is limited and the data tagging efficiency is affected if the API access mode is used for transmission.
In some embodiments, a data access mode determined by a service end is provided in a possible data annotation task, so that the server can judge whether the data access mode is provided before determining an optional online data access mode according to the data annotation task; if the data annotation task provides a data access mode, determining the data access mode corresponding to the data annotation task as a target access mode according to the data access mode provided in the data annotation task; if the data marking task does not provide a data access mode, a selectable online data access mode can be determined according to the data marking task, so that the service end can select the online data access mode.
In a possible implementation manner, different service ends may have different permissions, and the server may further determine an optional online data access manner according to the permission of the service end corresponding to the data annotation task. For example, the API access mode needs to access an API interface, the message queue access mode needs a server to configure a message queue corresponding to a service end in advance, which may bring processing burden to the server, and the service data source access mode is that the server actively acquires data from a data source, and the server may acquire data to be labeled from the data source according to an actual load condition of the server, so that when the authority of the service end is lower than a preset level, the selectable mode may be determined as the service data source access mode; when the authority of the service end is not lower than the preset level of authority, the selectable modes can be determined to be an API access mode, a service data source access mode and a message queue access mode.
Step S230: and sending the selectable online data access mode to the service end.
In the embodiment of the application, after determining the selectable online data access mode, the server can feed back the online data access mode to the service end, so that the service end can select the online data access mode. For example, referring to fig. 4, after receiving a selectable online data access mode, the service end may output and display the selectable online data access mode, and after a user corresponding to the service end selects a corresponding online data access mode, the service end returns a selection instruction for selecting the online data access mode to the server according to a selection operation of the user.
Step S240: and responding to a selection instruction which is returned by the service end and is used for selecting the online data access mode, and determining the corresponding online data access mode as a target access mode.
In the embodiment of the application, when receiving a selection instruction for selecting an online data access mode returned by the service end, the server may respond to the selection instruction, and determine the online data access mode selected corresponding to the selection instruction as the online data access mode corresponding to the data tagging task. It can be understood that the online data access mode selected corresponding to the selection instruction may include one online data access mode, and may also include multiple online data access modes.
In some embodiments, if the online data access mode selected corresponding to the selection instruction includes an API access mode, a mode in which the service end actively uploads or the service end uploads according to the operation of the user may be returned to the service end, so that the service end selects the data upload mode according to the requirement. The method comprises the steps that a service end actively uploads service data which are transmitted to a server regularly or irregularly by the service end so that the server can obtain data to be marked; and the service end selects corresponding data to upload to the server according to the operation of the user according to the operation uploading mode of the user, so that the server can label the data.
In some embodiments, if the online data access mode selected corresponding to the selection instruction includes a service data source access mode, a data source addition instruction may be further sent to the service end, so that the service end adds a corresponding service data source, and a subsequent server may obtain data to be labeled from the corresponding service data source.
In some embodiments, if the online data access mode selected corresponding to the selection instruction includes a message queue access mode, the server may configure and authorize the specified message queue for the service end, and synchronize the message queue to the service end, so that after the service end pushes the data to be tagged to the message queue, the server pulls the data to be tagged from the message queue.
Step S250: and determining a corresponding online data feedback mode as a target feedback mode according to the data annotation task.
In this embodiment of the application, the server determines the online data feedback mode corresponding to the data tagging task, and may refer to the content of the determined online data access mode, which is not described herein again.
Step S260: and marking the data to be marked according to the data marking task to obtain a data marking result.
Step S270: and feeding back the data labeling result to the service end according to the target feedback mode.
In the embodiment of the present application, step S260 and step S270 may refer to the contents of the foregoing embodiments, and are not described herein again.
According to the data labeling method provided by the embodiment of the application, the server of the labeling platform determines the corresponding online data access mode according to the data labeling task, so that the data access mode during data labeling can cover the data access requirement of a service end, meanwhile, the corresponding online data feedback mode can be provided for the labeling task, and the data labeling efficiency is improved. And when the corresponding online data access mode and the online data feedback mode are determined according to the data marking task, a selectable mode is provided for the service end to select the mode, and the requirements of the service end on the data access mode and the data feedback mode are met.
Referring to fig. 5, fig. 5 is a schematic flow chart illustrating a data annotation method according to another embodiment of the present application. The data annotation method is applied to the server, and will be described in detail with respect to the flow shown in fig. 5, where the data annotation method specifically includes the following steps:
step S310: and receiving a data annotation task sent by a service end.
Step S320: and determining a corresponding online data access mode as a target access mode and a corresponding online data feedback mode as a target feedback mode according to the data marking task.
In the embodiment of the present application, step S310 and step S320 may refer to the contents of the foregoing embodiments, and are not described herein again.
Step S330: and if the target access mode is an API access mode, responding to a verification request sent by the service terminal according to the API access mode, and verifying the verification request.
In the embodiment of the application, when the target access mode is an API access mode, and when the service end sends the data to be labeled to the server through the API, corresponding verification may be performed, so as to improve the security of the data packet labeling process. Specifically, the service end may send a verification request to the server, where the verification request carries a message body containing data to be annotated, and the server may verify the verification request after receiving the verification request.
In a possible implementation manner, the verifying the verification request sent by the service end may include: and checking whether the service end identifier carried in the check request is matched with the preset identifier.
After receiving the data annotation task, the server may allocate a service end identifier, for example, token, to the service end, and synchronize the allocated service end identifier to the service end; when the service end sends a verification request through the API, a service end identifier can be added in the verification request; if the service end identification is not included in the verification request received by the server, or the service end identification in the verification request is not matched with a preset identification (namely the service end identification distributed by the server), determining that the verification is passed; and if the service end identification in the verification request is not matched with the preset identification, determining that the verification is not passed.
In a possible implementation manner, the verifying the verification request sent by the service end may include: and checking whether the verification request carries time stamp information and a signature. The server can check the time stamp and check the signature of the check request, and if the check request does not carry the time stamp information or the signature, the check is determined not to pass; and if the verification request carries the timestamp information and the signature, determining that the verification is passed.
In a possible implementation manner, the verifying the verification request sent by the service end may include: and verifying whether an encryption result obtained after appointed encryption is carried out on the message body carried in the verification request is matched with the signature carried in the verification request, wherein the appointed encryption is to encrypt the message body according to an appointed encryption mode, timestamp information carried in the verification request and a secret key corresponding to the service end identifier. The server can distribute corresponding keys for the service end, and stores the corresponding relation between the keys and the service end identification; after receiving the verification request, the server can perform specified encryption on a message body carried in the verification request by using timestamp information carried in the verification request according to a secret key corresponding to the service end identifier of the service end to obtain an encryption result; if the encryption result is matched with the signature carried in the verification request, the verification is passed; and if the encryption result is not matched with the signature carried in the verification request, the verification is not passed. The specified encryption mode may be symmetric encryption or the like, and the specific encryption mode may not be limited. By the method, the verification request is verified, the condition that the verification request is tampered can be avoided, and the safety of the data annotation process is improved.
In a possible implementation manner, the verifying the verification request sent by the service end may include: and checking whether the size of the message body meets a preset data volume condition. The preset data volume condition may be whether the size of the message body exceeds the size of the preset data volume, and if the size of the message body exceeds the size of the preset data volume, the preset data volume condition is determined not to be met; and if the data size does not exceed the preset data size, determining that the preset data size condition is met. When the size of the message body meets the preset data volume condition, the verification can be determined to pass; otherwise, the verification is determined not to pass. By checking the checking request in the mode, the large data volume transmitted in a single time through the API can be avoided.
In a possible implementation manner, the verifying the verification request sent by the service end may include: and checking whether the format of the message body meets a preset format condition. The server can judge whether the format of the message body is a legal json format or not and carry out necessary field verification, and if the format of the message body is the legal json format and the field verification is passed, the verification request can be determined to be passed; if the format of the message body is not legal json format or the field check fails, the check can be determined to fail. By the method, errors, such as failure in format conversion, can be avoided when the data to be marked are subsequently processed.
In addition, the verification request may be verified in one of the above embodiments, or in a plurality of embodiments.
Exemplary, checking the check request may include all of the above embodiments. Referring to fig. 6, the server may check whether the service identifier carried in the check request matches a preset identifier; if the carried service end identifier is not matched with the preset identifier, determining that the verification is not passed; if the carried service end identifier is matched with the preset identifier, performing verification whether the verification request carries timestamp information and a signature; if the verification request does not carry the time stamp information or the signature, determining that the verification is not passed; if the verification request carries timestamp information and a signature, performing verification to determine whether an encryption result obtained by performing specified encryption on a message body carried in the verification request is matched with the signature carried in the verification request; if the encryption result is not matched with the signature carried in the verification request, determining that the verification is not passed; if the encryption result is matched with the signature carried in the verification request, performing verification to determine whether the size of the message body meets a preset data volume condition; if the size of the message body does not meet the preset data volume condition, determining that the verification does not pass; if the size of the message body meets the preset data volume condition, checking whether the format of the message body meets the preset format condition or not; and if the format of the message body does not meet the preset format condition, determining that the verification is not passed, and if the format of the message body meets the preset format condition, determining that the verification is passed.
Step S340: and if the verification is passed, writing the data to be labeled in the message body carried by the verification request into an appointed message queue, wherein the data in the appointed message queue is used for labeling the server.
In the embodiment of the application, after the verification request is verified, if the verification is passed, the data to be labeled in the message body carried by the verification request can be written into the specified message queue, so that the consumer service running in the server can obtain the corresponding data from the message queue for labeling.
Step S350: and marking the data to be marked according to the data marking task to obtain a data marking result.
Step S360: and feeding back the data labeling result to the service end according to the target feedback mode.
In the embodiment of the present application, step S350 and step S360 can refer to the contents of the foregoing embodiments, and are not described herein again.
According to the data labeling method provided by the embodiment of the application, the server of the labeling platform determines the corresponding online data access mode according to the data labeling task, so that the data access mode during data labeling can cover the data access requirement of a service end, meanwhile, the corresponding online data feedback mode can be provided for the labeling task, and the data labeling efficiency is improved. And when the data access mode is the API access mode, the verification request sent by the service end is verified, and the safety of data marking is effectively ensured.
Referring to fig. 7, fig. 7 is a schematic flowchart illustrating a data annotation method according to still another embodiment of the present application. The data annotation method is applied to the server, and will be described in detail with respect to the flow shown in fig. 7, where the data annotation method specifically includes the following steps:
step S410: and receiving a data annotation task sent by a service end.
Step S420: and determining a corresponding online data access mode as a target access mode and a corresponding online data feedback mode as a target feedback mode according to the data marking task.
In the embodiment of the present application, step S410 and step S420 may refer to the contents of the foregoing embodiments, and are not described herein again.
Step S430: and if the target access mode is a service data source access mode, determining a specified service data source corresponding to the data annotation task according to the data annotation task.
In this embodiment of the application, when the target access manner is a service data source access manner, the server may determine a specified service data source corresponding to the data annotation task. Optionally, the data annotation task may include a service data source provided by the service end, and the server may determine the service data source included in the data annotation task as the specified service data source corresponding to the data annotation task. Optionally, the server may send a data source acquisition request to the service end to request the service end to determine a service data source for acquiring the data to be annotated.
Step S440: and acquiring the service data from the specified service data source as the data to be labeled.
In the embodiment of the application, after determining the designated service data source, the server may obtain the data to be annotated from the designated service data source. Optionally, the server may extract the service data from the data source by means of the data extraction plug-in at a specified frequency (e.g., once per hour, once per day, once per week); for the extracted service data, the service data can be converted into a data format required by the data annotation task according to the requirement of the data annotation task, and the data to be annotated is obtained.
In some embodiments, when the data source is some service party, the service party is in consideration of data protection, and some service data of the service party may not be disclosed to the outside, which results in that data annotation cannot be performed smoothly. Therefore, if the data to be labeled provided by the specified service data source is the service data which is kept secret from the outside, the server can send the access address of the specified service data source to the service end, and the access address is used for the service end to initiate an authentication request to the specified service data source; and when the appointed service data source successfully authenticates the service end, acquiring service data from the appointed service data source as data to be marked. Therefore, the authentication mode can be initiated to the appointed service data source through the service end, when the authentication is passed, the service end can synchronize the authentication result to the server, and the server acquires the data to be labeled from the appointed service data source according to the authentication result, so that the normal operation of data labeling is ensured. When the authentication fails, the server may not perform subsequent actions.
Optionally, the step of specifying the service data source to authenticate the service end may include: determining an access token carried by the authentication request, wherein the access token is used for representing access authority information of a service end; wherein, the access token is assigned by the appointed service data source for the service end in advance. Based on the access token, the access authority and authority valid characteristics of the service end can be determined; wherein, the access authority is used for representing the authority level of the business end for accessing the business data; rights valid features refer to features that define the validity of access rights, such as: the access right of the temporary user has certain timeliness, or has certain requirements on the network security of the user access and use, and the like.
Step S450: and marking the data to be marked according to the data marking task to obtain a data marking result.
Step S460: and feeding back the data labeling result to the service end according to the target feedback mode.
In the embodiment of the present application, step S450 and step S460 may refer to the contents of the foregoing embodiments, and are not described herein again.
According to the data labeling method provided by the embodiment of the application, the server of the labeling platform determines the corresponding online data access mode according to the data labeling task, so that the data access mode during data labeling can cover the data access requirement of a service end, meanwhile, the corresponding online data feedback mode can be provided for the labeling task, and the data labeling efficiency is improved. And when the data access mode is a service data source access mode, if the data to be labeled provided by the specified service data source is the service data which is kept secret from outside, the normal operation of data labeling is ensured by the mode that the service end authenticates the specified service data source.
Referring to fig. 8, fig. 8 is a schematic flow chart illustrating a data annotation method according to yet another embodiment of the present application. The data annotation method is applied to the server, and will be described in detail with respect to the flow shown in fig. 8, where the data annotation method specifically includes the following steps:
step S510: and receiving a data annotation task sent by a service end.
Step S520: and determining a corresponding online data access mode as a target access mode and a corresponding online data feedback mode as a target feedback mode according to the data marking task.
In the embodiment of the present application, step S510 and step S520 may refer to the contents of the foregoing embodiments, and are not described herein again.
Step S530: and monitoring a service message queue corresponding to the service end if the target access mode is a message queue access mode, wherein the service message queue is a message queue configured for the service end in advance.
In this embodiment of the present application, when the target access mode is a message queue access mode, the server may configure and authorize a service message queue for the service end, and then pull the data to be labeled pushed by the service end by monitoring the service message queue.
Step S540: when data to be marked pushed by the service end in the service message queue is monitored, the data to be marked are written into an appointed message queue, and the data in the appointed message queue is used for marking by the server.
In the embodiment of the application, when data to be marked, which is pushed by a service end in a service message queue, is monitored, the service data can be pulled; then converting the pulled business data into a data format required by a data annotation task to obtain data to be annotated; and pulling the data to be labeled to an appointed message queue so as to facilitate the consumer service running in the server to obtain corresponding data from the message queue for labeling.
Step S550: and marking the data to be marked according to the data marking task to obtain a data marking result.
Step S560: and feeding back the data labeling result to the service end according to the target feedback mode.
In the embodiment of the present application, step S550 and step S560 may refer to the contents of the foregoing embodiments, and are not described herein again.
According to the data labeling method provided by the embodiment of the application, the server of the labeling platform determines the corresponding online data access mode according to the data labeling task, so that the data access mode during data labeling can cover the data access requirement of a service end, meanwhile, the corresponding online data feedback mode can be provided for the labeling task, and the data labeling efficiency is improved. And when the data access mode is a message queue access mode, acquiring the data to be marked in a message queue monitoring mode.
Referring to fig. 9, a block diagram of a data annotation device 400 according to an embodiment of the present application is shown. The data annotation device 400 is applied to a server of an annotation platform, and the data annotation device 400 includes: the task receiving module 410, the mode determining module 420, the data obtaining module 430, the annotation executing module 440, and the result feedback module 450. The task receiving module 410 is configured to receive a data annotation task sent by a service end; the mode determining module 420 is configured to determine, according to the data tagging task, a corresponding online data access mode as a target access mode and a corresponding online data feedback mode as a target feedback mode; the data obtaining module 430 is configured to obtain data to be labeled corresponding to the data labeling task according to the target access manner; the labeling execution module 440 is configured to label the data to be labeled according to the data labeling task to obtain a data labeling result; the result feedback module 450 is configured to feed back the data annotation result to the service end according to the target feedback manner.
In some embodiments, the online data access mode comprises one or more of application program interface, API, access, service data source access, and message queue access; the online data feedback mode comprises one or more of feedback through an API (application programming interface), feedback through a service data source and feedback through a message queue.
As a possible implementation manner, if the target access manner is an API access manner, the data obtaining module 430 may be configured to: responding to a verification request sent by the service terminal according to the API access mode, and verifying the verification request; and if the verification is passed, writing the data to be labeled in the message body carried by the verification request into an appointed message queue, wherein the data in the appointed message queue is used for labeling the server.
Optionally, the data obtaining module 430 checks the check request, which may include one or more of the following manners:
checking whether the service end identifier carried in the checking request is matched with a preset identifier or not;
checking whether the check request carries timestamp information and a signature or not;
verifying whether an encryption result obtained after appointed encryption is carried out on a message body carried in the verification request is matched with a signature carried in the verification request, wherein the appointed encryption is to encrypt the message body according to an appointed encryption mode, timestamp information carried in the verification request and a secret key corresponding to the service end identifier;
checking whether the size of the message body meets a preset data volume condition or not; and
and checking whether the format of the message body meets a preset format condition.
As a possible implementation manner, if the target access manner is a service data source access manner, the data obtaining module 430 may be configured to: determining a designated service data source corresponding to the data annotation task according to the data annotation task; and acquiring the service data from the specified service data source as the data to be labeled.
Optionally, the data obtaining module 430 may further be configured to: if the data to be marked provided by the appointed service data source is the service data which is kept secret from outside, sending an access address of the data source to the service end, wherein the access address is used for the service end to initiate an authentication request to the appointed service data source. When the specified service data source successfully authenticates the service terminal, the data obtaining module 430 obtains service data from the specified service data source as data to be labeled.
As a possible implementation manner, if the target access manner is a message queue access manner, the data obtaining module 430 may be configured to: monitoring a service message queue corresponding to the service end, wherein the service message queue is a message queue configured for the service end in advance; when data to be marked pushed by the service end in the service message queue is monitored, the data to be marked are written into an appointed message queue, and the data in the appointed message queue is used for marking by the server.
As one possible implementation, the manner determining module 420 may be configured to: determining a selectable online data access mode according to the data marking task; sending the selectable online data access mode to the service end; and responding to a selection instruction which is returned by the service end and is used for selecting the online data access mode, and determining the corresponding online data access mode as a target access mode.
Optionally, the determining module 420 determines an optional online data access mode according to the data annotation task, and may include: and determining a data access mode corresponding to the data type according to the data type corresponding to the data marking task, and taking the data access mode as a selectable online data access mode.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and modules may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, the coupling between the modules may be electrical, mechanical or other type of coupling.
In addition, functional modules in the embodiments of the present application may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
In summary, in the scheme provided by the application, the server of the annotation platform receives the data annotation task sent by the service end, according to the data marking task, determining a corresponding on-line data access mode as a target access mode and a corresponding on-line data feedback mode as a target feedback mode, then the data to be labeled corresponding to the data labeling task is obtained according to the target access mode, the data to be labeled is labeled according to the data labeling task to obtain a data labeling result, finally the data labeling result is fed back to the service end according to the target feedback mode, thereby determining the corresponding on-line data access mode according to the data marking task, enabling the data access mode during data marking to cover the data access requirement of the service end, meanwhile, a corresponding online data feedback mode can be provided for the labeling task, and the data labeling efficiency is improved.
Referring to fig. 10, a block diagram of a server according to an embodiment of the present application is shown. The server 100 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing a basic cloud computing service, but is not limited thereto. The server 100 in the present application may include one or more of the following components: a processor 110, a memory 120, and one or more applications, wherein the one or more applications may be stored in the memory 120 and configured to be executed by the one or more processors 110, the one or more programs configured to perform a method as described in the aforementioned method embodiments.
Processor 110 may include one or more processing cores. The processor 110 connects various parts within the overall server 100 using various interfaces and lines, performs various functions of the server 100 and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 120, and calling data stored in the memory 120. Alternatively, the processor 110 may be implemented in hardware using at least one of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 110 may integrate one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing display content; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 110, but may be implemented by a communication chip.
The Memory 120 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). The memory 120 may be used to store instructions, programs, code sets, or instruction sets. The memory 120 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing various method embodiments described below, and the like. The storage data area may also store data created by the server 100 in use (such as phone books, audio and video data, chat log data), and the like.
Referring to fig. 11, a block diagram of a computer-readable storage medium according to an embodiment of the present application is shown. The computer-readable medium 800 has stored therein a program code that can be called by a processor to execute the method described in the above-described method embodiments.
The computer-readable storage medium 800 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 800 includes a non-volatile computer-readable storage medium. The computer readable storage medium 800 has storage space for program code 810 to perform any of the method steps of the method described above. The program code can be read from or written to one or more computer program products. The program code 810 may be compressed, for example, in a suitable form.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not necessarily depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (12)
1. A data annotation method is applied to a server of an annotation platform, and comprises the following steps:
receiving a data annotation task sent by a service end;
according to the data marking task, determining a corresponding online data access mode as a target access mode and a corresponding online data feedback mode as a target feedback mode;
acquiring data to be labeled corresponding to the data labeling task according to the target access mode;
marking the data to be marked according to the data marking task to obtain a data marking result;
and feeding back the data labeling result to the service end according to the target feedback mode.
2. The method of claim 1, wherein the online data access mode comprises one or more of an Application Program Interface (API) access, a service data source access, and a message queue access; the online data feedback mode comprises one or more of feedback through an API (application programming interface), feedback through a service data source and feedback through a message queue.
3. The method according to claim 2, wherein if the target access mode is an API access mode, the obtaining the data to be labeled corresponding to the data labeling task according to the target access mode includes:
responding to a verification request sent by the service terminal according to the API access mode, and verifying the verification request;
and if the verification is passed, writing the data to be labeled in the message body carried by the verification request into an appointed message queue, wherein the data in the appointed message queue is used for labeling the server.
4. The method of claim 3, wherein the verifying the verification request comprises one or more of:
checking whether the service end identifier carried in the checking request is matched with a preset identifier or not;
checking whether the check request carries timestamp information and a signature or not;
verifying whether an encryption result obtained after appointed encryption is carried out on a message body carried in the verification request is matched with a signature carried in the verification request, wherein the appointed encryption is to encrypt the message body according to an appointed encryption mode, timestamp information carried in the verification request and a secret key corresponding to the service end identifier;
checking whether the size of the message body meets a preset data volume condition or not; and
and checking whether the format of the message body meets a preset format condition.
5. The method according to claim 2, wherein if the target access mode is a service data source access mode, the obtaining the data to be labeled corresponding to the data labeling task according to the target access mode includes:
determining a designated service data source corresponding to the data annotation task according to the data annotation task;
and acquiring the service data from the specified service data source as the data to be labeled.
6. The method according to claim 5, wherein before said obtaining the service data from the specified service data source as the data to be labeled, the method further comprises:
if the data to be marked provided by the appointed service data source is the service data which is kept secret from the outside, sending an access address of the data source to the service end, wherein the access address is used for the service end to initiate an authentication request to the appointed service data source;
and when the specified service data source successfully authenticates the service end, executing the step of acquiring service data from the specified service data source as data to be marked.
7. The method of claim 2, wherein if the target access mode is a message queue access mode, the obtaining the data to be labeled corresponding to the data labeling task according to the target access mode comprises:
monitoring a service message queue corresponding to the service end, wherein the service message queue is a message queue configured for the service end in advance;
when data to be marked pushed by the service end in the service message queue is monitored, the data to be marked are written into an appointed message queue, and the data in the appointed message queue is used for marking by the server.
8. The method according to any one of claims 1 to 7, wherein the determining, according to the data tagging task, the corresponding online data access mode as a target access mode includes:
determining a selectable online data access mode according to the data marking task;
sending the selectable online data access mode to the service end;
and responding to a selection instruction which is returned by the service end and is used for selecting the online data access mode, and determining the corresponding online data access mode as a target access mode.
9. The method of claim 8, wherein determining the selectable online data access mode according to the data annotation task comprises:
and determining a data access mode corresponding to the data type according to the data type corresponding to the data marking task, and taking the data access mode as a selectable online data access mode.
10. A data annotation device, which is applied to a server of an annotation platform, the device comprises: a task receiving module, a mode determining module, a data acquiring module, a label executing module and a result feedback module, wherein,
the task receiving module is used for receiving a data annotation task sent by a service end;
the mode determining module is used for determining a corresponding online data access mode as a target access mode according to the data marking task;
the data acquisition module is used for determining a corresponding online data access mode as a target access mode and a corresponding online data feedback mode as a target feedback mode according to the data marking task;
the marking execution module is used for marking the data to be marked according to the data marking task to obtain a data marking result;
and the result feedback module is used for feeding back the data labeling result to the service end according to the target feedback mode.
11. A server, comprising:
one or more processors;
a memory;
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method of any of claims 1-9.
12. A computer-readable storage medium, having stored thereon program code that can be invoked by a processor to perform the method according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110425810.1A CN113139109A (en) | 2021-04-20 | 2021-04-20 | Data labeling method, device, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110425810.1A CN113139109A (en) | 2021-04-20 | 2021-04-20 | Data labeling method, device, server and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113139109A true CN113139109A (en) | 2021-07-20 |
Family
ID=76812854
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110425810.1A Pending CN113139109A (en) | 2021-04-20 | 2021-04-20 | Data labeling method, device, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113139109A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113839953A (en) * | 2021-09-27 | 2021-12-24 | 上海商汤科技开发有限公司 | Labeling method and device, electronic equipment and storage medium |
| CN113918713A (en) * | 2021-09-22 | 2022-01-11 | 南京复保科技有限公司 | Data annotation method and device, computer equipment and storage medium |
-
2021
- 2021-04-20 CN CN202110425810.1A patent/CN113139109A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113918713A (en) * | 2021-09-22 | 2022-01-11 | 南京复保科技有限公司 | Data annotation method and device, computer equipment and storage medium |
| CN113839953A (en) * | 2021-09-27 | 2021-12-24 | 上海商汤科技开发有限公司 | Labeling method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111416808A (en) | Cross-block-chain data mutual storage method, device, equipment and storage medium | |
| CN104660635B (en) | Message synchronization method, device and system | |
| CN112131002B (en) | Data management method and device | |
| CN113139109A (en) | Data labeling method, device, server and storage medium | |
| CN112073307B (en) | Mail processing method, mail processing device, electronic equipment and computer readable medium | |
| CN105429959A (en) | Image processing method, client equipment, image verification method and server | |
| CN109194651A (en) | A kind of identity identifying method, device, equipment and storage medium | |
| CN111191200B (en) | Three-party linkage authentication page display method and device and electronic equipment | |
| CN113706091B (en) | Intelligent remote auditing method, device, system, computer equipment and storage medium | |
| CN112966304B (en) | Tamper-proof method and device for flow document, computer equipment and medium | |
| CN111506551A (en) | Conference file extraction method and system and computer equipment | |
| US10681094B2 (en) | Control system, communication control method, and program product | |
| CN114615031A (en) | File storage method and device, electronic equipment and storage medium | |
| CN113946739A (en) | Sensitive data query method, device, equipment and storage medium | |
| CN111327680A (en) | Authentication data synchronization method, device, system, computer equipment and storage medium | |
| US10530729B2 (en) | Video retrieval | |
| CN113420133B (en) | Session processing method, device, equipment and storage medium | |
| CN109979051A (en) | For veritifying the method, apparatus and computer readable storage medium of user identity | |
| CN113489714A (en) | Multi-module-based intelligent message cross processing method and system | |
| KR102461836B1 (en) | Apparatus and method for connecting chatbot | |
| CN101600042A (en) | A kind of management system of managing digital television receiving terminal and method | |
| CN112669000A (en) | Government affair item processing method and device, electronic equipment and storage medium | |
| CN114270389A (en) | Information acquisition method, device, system, electronic equipment and storage medium | |
| US11368501B1 (en) | Apparatus, methods, and systems for formatting documents for multi-device web conferencing | |
| TWI779749B (en) | Method and system for verifying by messaging service |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |