WO2023109631A1 - 数据处理方法、装置、设备、存储介质及程序产品 - Google Patents

数据处理方法、装置、设备、存储介质及程序产品 Download PDF

Info

Publication number
WO2023109631A1
WO2023109631A1 PCT/CN2022/137442 CN2022137442W WO2023109631A1 WO 2023109631 A1 WO2023109631 A1 WO 2023109631A1 CN 2022137442 W CN2022137442 W CN 2022137442W WO 2023109631 A1 WO2023109631 A1 WO 2023109631A1
Authority
WO
WIPO (PCT)
Prior art keywords
initial
labeling
result
updated
candidate
Prior art date
Application number
PCT/CN2022/137442
Other languages
English (en)
French (fr)
Inventor
伍健荣
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Publication of WO2023109631A1 publication Critical patent/WO2023109631A1/zh
Priority to US18/368,680 priority Critical patent/US20240005211A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • the present application relates to the technical field of the Internet, and in particular to a data processing method, device, equipment, storage medium and program product.
  • the current labeling of target objects in images mainly includes pure manual labeling, pure machine labeling, and artificial intelligence-assisted labeling.
  • Pure manual labeling means that there is no model assistance in the labeling process, and the labeler relies on the identification of the target object to label;
  • pure machine labeling means that there is no manual intervention in the labeling process, and the artificial intelligence model prediction results are used as the labeling results;
  • artificial intelligence-assisted labeling is the index
  • the artificial intelligence model predicts the image and generates a prediction result, and the annotator combines the prediction result to complete the annotation of the target object in the image.
  • the labeler In the relevant artificial intelligence-assisted labeling, the labeler is often only the user of the artificial intelligence model and does not participate in the update of the artificial intelligence model, which leads to the failure of the model to be updated in time, and ultimately affects the accuracy of the auxiliary labeling; in addition, the relevant artificial intelligence-assisted labeling method There is no review link for the existing labeling results, so the existing labeling results cannot be updated. If there are existing labeling results with low precision, the labeling results with low precision will continue to be used in subsequent training or use.
  • Embodiments of the present application provide a data processing method, device, device, storage medium, and program product, which help to improve the recognition ability of an image recognition model and improve the accuracy of labeling results.
  • a data processing method which is executed in a computer device, and the method includes:
  • the original image includes a first original image and a second original image
  • the initial auxiliary annotation result includes a first initial auxiliary annotation result of the first original image
  • the initial standard annotation result includes the first initial standard annotation result of the first original image, and the first initial standard annotation result of the second original image 2.
  • the updated image recognition model is determined as a target image recognition model, and the target image recognition model is used for Generate annotation results for the target image.
  • An embodiment of the present application provides a data processing device on the one hand, including:
  • a first acquisition module configured to predict an initial auxiliary annotation result of an original image based on an initial image recognition model, the original image includes a first original image and a second original image, and the initial auxiliary annotation result includes an initial auxiliary annotation result of the first original image
  • the first initial auxiliary annotation result obtain the initial standard annotation result determined by correcting the initial auxiliary annotation result; wherein, the initial standard annotation result includes the first initial standard annotation result of the first original image, and the The second initial standard labeling result of the second original image;
  • the update model module is used to adjust the model parameters in the initial image recognition model according to the first initial standard labeling result and the first initial auxiliary labeling result to generate an updated image recognition model;
  • the second acquisition module is used to predict the updated auxiliary labeling result of the second original image based on the updated image recognition model, and obtain the updated standard labeling result of the second original image; the updated standard labeling result is based on the update of the auxiliary labeling result to the second initial It is obtained by adjusting the standard labeling results;
  • the first determination module is used to determine the updated image recognition model as the target image recognition model when the updated image recognition model meets the model convergence condition according to the updated auxiliary labeling result and the updated standard labeling result; the target image recognition model is used to generate the target image labeling results.
  • One aspect of the present application provides a computer device, including: a processor, a memory, and a network interface;
  • the above-mentioned processor is connected to the above-mentioned memory and the above-mentioned network interface, wherein the above-mentioned network interface is used to provide a data communication function, the above-mentioned memory is used to store a computer program, and the above-mentioned processor is used to call the above-mentioned computer program, so that the computer device executes the embodiment of the present application method in .
  • the embodiments of the present application provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and the computer program is adapted to be loaded by a processor and execute the method in the embodiment of the present application.
  • Embodiments of the present application provide a computer program product or computer program on the one hand, the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; The medium reads the computer instruction, and the processor executes the computer instruction, so that the computer device executes the method in the embodiment of the present application.
  • FIG. 1 is a schematic diagram of a system architecture provided by an embodiment of the present application.
  • Fig. 2 is a schematic flow chart of a data processing method provided by an embodiment of the present application.
  • FIG. 3 is a schematic diagram of a data processing scenario provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a data processing scenario provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a data processing scenario provided by an embodiment of the present application.
  • FIG. 6 is a schematic flow chart of a data processing method provided in an embodiment of the present application.
  • FIG. 7 is a schematic flow chart of a data processing method provided by an embodiment of the present application.
  • FIG. 8 is a schematic flowchart of a data processing method provided in an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of a data processing device provided in an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a computer device provided by an embodiment of the present application.
  • Artificial Intelligence is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
  • artificial intelligence is a comprehensive technique of computer science that attempts to understand the nature of intelligence and produce a new kind of intelligent machine that can respond in a similar way to human intelligence.
  • Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making.
  • Artificial intelligence technology is a comprehensive subject that involves a wide range of fields, including both hardware-level technology and software-level technology.
  • Artificial intelligence basic technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation/interaction systems, and mechatronics.
  • Artificial intelligence software technology mainly includes several major directions such as computer vision technology, speech processing technology, natural language processing technology, machine learning/deep learning, automatic driving, and intelligent transportation.
  • Computer Vision technology (Computer Vision, CV) is a science that studies how to make machines "see”. To put it further, it refers to the use of cameras and computers instead of human eyes to identify and measure the target and other machine vision, and further make graphics Processing, so that the computer processing becomes an image that is more suitable for human observation or sent to the instrument for detection.
  • Computer Vision studies related theories and technologies, trying to build artificial intelligence systems that can obtain information from images or multidimensional data.
  • Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content/behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and maps Construction, automatic driving, intelligent transportation and other technologies, as well as common biometric identification technologies such as face recognition and fingerprint recognition.
  • computer vision technology can be used to identify target objects (such as people, dogs, cats, birds, etc.) in the image, and outline and mark the target objects.
  • Machine Learning is a multi-field interdisciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. Specializes in the study of how computers simulate or implement human learning behaviors to acquire new knowledge or skills, and reorganize existing knowledge structures to continuously improve their performance.
  • Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its application pervades all fields of artificial intelligence.
  • Machine learning and deep learning usually include techniques such as artificial neural network, belief network, reinforcement learning, transfer learning, inductive learning, and teaching learning.
  • both the initial image recognition model and the updated image recognition model are AI models based on machine learning technology, which can be used for image recognition processing.
  • FIG. 1 is a schematic diagram of a system architecture provided by an embodiment of the present application.
  • the system can include a business server 100, an annotation terminal cluster, a first audit terminal 200a, and a second audit terminal 200b; an annotation terminal cluster can include: an annotation terminal 100a, an annotation terminal 100b, ..., an annotation terminal 100c,
  • the above system may include one or more tagging terminals, and the present application does not limit the number of tagging terminals.
  • the above system may include one or more first audit terminals, and may also include one or more second audit terminals.
  • the embodiment of the present application does not limit the number of first audit terminals and second audit terminals.
  • the tagging terminal cluster may include one or more tagging terminals corresponding to the tagging user;
  • the service server 100 may be an initial candidate tagging result provided by the tagging terminal and an update candidate tagging result (equivalent to the candidate tagging result described below) equipment;
  • the first review terminal may be a review terminal for reviewing at least two candidate labeling results;
  • the second review terminal may be a review terminal for reviewing target candidate labeling results.
  • any labeling terminal in the labeling terminal cluster can have a communication connection with the service server 100, for example, there is a communication connection between the labeling terminal 100a and the service server 100; wherein, any labeling terminal in the above-mentioned labeling terminal cluster can communicate with the above-mentioned
  • the audit terminals including the first audit terminal 200a and the second audit terminal 200b
  • there is a communication connection between the marking terminal 100a and the first audit terminal 200a there is a communication connection between the marking terminal 100b and the first audit terminal 200a.
  • any audit terminal may have a communication connection with the business server 100, for example, the first
  • the above-mentioned communication connection is not limited to the connection method, and may be directly or indirectly connected by wired communication, wireless communication, or other methods, which are not limited in this application.
  • each labeling terminal in the labeling terminal cluster shown in Figure 1 can be installed with an application client, and when the application client runs in each labeling terminal, it can communicate with the service server shown in Figure 1 above 100 for data interaction, that is, the communication connection mentioned above.
  • the application client can be a short video application, a video application, a live broadcast application, a social application, an instant messaging application, a game application, a music application, a shopping application, a novel application, a payment application, a browser, etc.
  • the application client of the function can be an independent client, or an embedded sub-client integrated in a certain client (for example, a social client, an educational client, and a multimedia client, etc.), which is not limited here .
  • the service server 100 may be a collection of multiple servers including a background server and a data processing server corresponding to the social application.
  • each tagging terminal can upload its local image to the service server 100 through the application client of the social application, and then the service server 100 can send the image to the review terminal or send it to the cloud server.
  • one marking terminal may be selected as the target marking terminal in the marking terminal cluster shown in FIG. 1 , for example, the marking terminal 100a is used as the target marking terminal.
  • the labeling object ie, the labeling user
  • the labeling terminal 100a can use the initial auxiliary labeling result as a reference label.
  • the labeling operation is performed on the reference labeling result, such as adding a label of a target object, deleting a label of a non-target object, modifying a wrong label of a target object, and confirming a label of a target object, the labeling terminal 100a can generate the original image.
  • Initial candidate labeling results and send the initial candidate labeling results to the service server 100 .
  • the above-mentioned initial auxiliary labeling result is obtained by predicting the image features of the original image based on the initial image recognition model, which includes the initial auxiliary labeling area for the target object in the original image, and the initial auxiliary object label for the initial auxiliary labeling area .
  • the aforementioned initial candidate labeling results include initial candidate labeling regions for labeling target objects, and initial candidate object labels for labeling the initial candidate labeling regions.
  • the service server 100 may obtain an initial standard labeling result based on the initial candidate labeling result.
  • the original image includes a first original image and a second original image
  • the initial standard labeling result includes a first initial standard labeling result of the first original image, and a second initial standard labeling result of the second original image.
  • the initial auxiliary annotation result includes a first initial auxiliary annotation result of the first original image.
  • the business server 100 adjusts the model parameters in the initial image recognition model according to the first initial standard labeling result and the first initial auxiliary labeling result to generate an updated image recognition model, and this process can realize the update of the initial image recognition model; Further, based on the updated image recognition model, the updated auxiliary labeling result of the second original image is predicted, and the updated standard labeling result obtained by adjusting the second initial standard labeling result based on the updated auxiliary labeling result is obtained. This process can realize the marked first 2. Updating of the labeling result of the original image (i.e.
  • the service server 100 will update the image recognition model is determined as the target image recognition model, wherein the target image recognition model is used to generate the target auxiliary labeling result of the target image.
  • the first audit terminal 200a and the second audit terminal 200b please refer to the description in step S103 in the embodiment corresponding to FIG. 2 below, and the description will not be expanded here.
  • the labeling terminal 100a can obtain the initial auxiliary labeling result of the original image through the local initial image recognition model, and then generate the initial standard labeling result based on the initial auxiliary labeling result Similarly, if the above-mentioned updated image recognition model is stored locally in the labeling terminal 100a, then the labeling terminal 100a can obtain the updated auxiliary labeling result of the second original image through the local updated image recognition model, and then generate an update standard based on the updated auxiliary labeling result Mark the results, and the rest of the process is consistent with the above process, so I won’t go into details here, please refer to the above description.
  • both the local initial image recognition model and the updated image recognition model of the labeling terminal 100a can be performed by the service server 100 after the training is completed. sent to the labeling terminal 100a.
  • marking terminal 100a marking terminal 100b, ..., marking terminal 100c, first review terminal 200a, and second review terminal 200b can all be blockchain nodes in the blockchain network
  • the data described in the full text (such as the initial image recognition model, original data, and initial standard labeling results) can be stored
  • the storage method can be that the blockchain node generates a block according to the data, and adds the block to the blockchain for storage The way.
  • Blockchain is a new application mode of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. It is mainly used to organize data in chronological order and encrypt them into ledgers so that they cannot be tampered with or forged. , and data verification, storage and update can be performed at the same time.
  • the blockchain is essentially a decentralized database. Each node in the database stores the same blockchain.
  • the blockchain network can distinguish nodes into core nodes, data nodes, and light nodes. Core nodes, data nodes and light nodes together form blockchain nodes. Among them, the core node is responsible for the consensus of the entire blockchain network, that is to say, the core node is the consensus node in the blockchain network.
  • the process of writing transaction data in the blockchain network to the ledger can be as follows: the data nodes or light nodes in the blockchain network obtain the transaction data, and transmit the transaction data in the blockchain network (that is, the nodes use the baton transfer) until the consensus node receives the transaction data, the consensus node then packs the transaction data into a block, executes a consensus on the block, and writes the transaction data into the ledger after the consensus is completed.
  • the original data and initial standard labeling results are used as an example of transaction data.
  • the business server 100 After the business server 100 (block chain node) passes the consensus on the transaction data, it generates a block according to the transaction data, and stores the block in the block chain network; For the reading of transaction data (that is, original data and initial standard labeling results), the block chain node can obtain the block containing the transaction data in the block chain network, and further, obtain the transaction data in the block .
  • the method provided in the embodiment of the present application can be executed by computer equipment, and the computer equipment includes but is not limited to a marking terminal or a service server.
  • the business server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also provide cloud database, cloud service, cloud computing, cloud function, cloud storage, network service, cloud Cloud servers for basic cloud computing services such as communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.
  • Labeled terminals include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, etc.
  • the marking terminal and the service server may be connected directly or indirectly through wired or wireless means, which is not limited in this embodiment of the present application.
  • FIG. 2 is a schematic flowchart of a data processing method provided by an embodiment of the present application.
  • the data processing method may be executed by a service server (for example, the service server 100 shown in FIG. 1 above), or by a terminal device (for example, the terminal device 200a shown in FIG. 1 above), or by a service server and a terminal Device interactive execution.
  • a service server for example, the service server 100 shown in FIG. 1 above
  • a terminal device for example, the terminal device 200a shown in FIG. 1 above
  • the embodiment of the present application takes the method executed by the service server as an example for description.
  • the data processing method may at least include the following steps S101-S104.
  • Step S101 Predict the initial auxiliary labeling result of the original image based on the initial image recognition model, and obtain the initial standard labeling result determined by correcting the initial auxiliary labeling result;
  • the original image includes the first original image and the second original image;
  • the initial standard labeling result The first initial standard labeling result of the first original image and the second initial standard labeling result of the second original image are included;
  • the initial auxiliary labeling result includes the first initial auxiliary labeling result of the first original image.
  • the operation of step S101 includes: acquiring an original image; the original image includes the target object; inputting the original image into the initial image recognition model, and obtaining the image features of the original image in the initial image recognition model; determining the target according to the image features The initial area identification feature of the object, and the initial object identification feature of the target object; generate the initial auxiliary labeling area for the target object according to the initial area identification feature, and generate the initial auxiliary object label for the initial auxiliary labeling area according to the initial object identification feature; the initial The auxiliary labeling area and the label of the initial auxiliary object are determined as the initial auxiliary labeling result.
  • the initial image recognition model refers to the artificial intelligence model used to identify the target object in the original image.
  • the embodiment of the present application does not limit the model type of the initial image recognition model.
  • the initial image recognition model can be determined according to the actual application scenario, including but not limited to Convolutional Neural Networks (CNN), Fully Convolutional Networks (FCN), Residual Networks (Res-Net), etc.
  • the embodiment of the present application does not limit the number of images of the original image, the number of images includes at least two, and does not limit the image type of the original image, which may be any image type.
  • the embodiment of the present application does not limit the object type of the target object, which can be any object type, such as person, bicycle, table, medical endoscope object, etc., and can be set according to the actual application scene.
  • this embodiment of the present application does not limit the number of target objects.
  • the target object is a person, there may be no target object in the original image, or at least one target object; it should be understood that the target object may include one type or multiple types Objects, for example, target objects may include bicycles, bicycles and people.
  • the original image is a medical image
  • the target object is a medical detection target, that is, a target in the medical image.
  • FIG. 3 is a schematic diagram of a data processing scenario provided by an embodiment of the present application.
  • the service server 30 a may be equivalent to the service server 100 in FIG. 1
  • the marking terminal 30 f may be any marking terminal in the marking terminal cluster in FIG. 1 .
  • the service server 30a may include an image database 30b for storing original images and data associated with the original images, including but not limited to an initial image recognition model 30c and the like.
  • the target object is set to be human.
  • the service server 30a inputs the first original image 301b in the image database 30b to the initial image recognition model 30c, and can obtain the image feature 30d of the first original image 301b in the initial image recognition model 30c; further, the business The server 30a can determine the initial area identification feature of the target object (for example, being a person) according to the image feature 30d, and the initial object identification feature of the target object; generate an initial auxiliary labeling area for the target object according to the initial area identification feature, as shown in Figure 3. Annotate the marked area in the image 30e, and generate an initial auxiliary object label for the initial auxiliary marked area according to the initial object recognition feature.
  • the object label is set to, for example, a person.
  • the service server 100 can display the initial auxiliary annotation image 30e in the figure, which carries the first initial auxiliary annotation result 301e for the first original image 301b.
  • the annotation results in this application includes the labeling area for the target object and the object label for the target object.
  • the business server 30a sends the initial auxiliary annotation image 30e carrying the first initial auxiliary annotation result 301e to the annotation terminal 30f, and the annotation object 301f can correct the first initial auxiliary annotation result 301e, for example, through the annotation installed by the annotation terminal 30f
  • the application software checks the original image 301b and the initial auxiliary annotated image 30f.
  • the annotated object 301f can first confirm whether the original image 301b contains a person.
  • the labeling terminal 30f can determine the first initial auxiliary labeling result 301e as the initial candidate labeling result (because there is only one target object, so here the default initial auxiliary object label is equal to the object label of the target object); if the labeling object 301f is not If the initial auxiliary labeling area is approved, the position and shape of the target object will be marked in the form of polygons. When marking, the labeling object 301f is required to be as close to the edge of the target object as possible, and all the target objects are included in the area.
  • the marked area can be Referred to as the region of interest (Region of Interest, ROI);
  • the labeling object 301f modifies the initial auxiliary labeling area to obtain the initial candidate labeling result, as shown in Figure 3, the labeling terminal 30f can display the initial candidate labeling image 30g, the initial candidate annotation image 30g may display an initial candidate annotation result 301g.
  • the labeling terminal 30f returns the initial candidate labeling image 30g carrying the initial candidate labeling result 301g to the service server 100 .
  • the embodiment of the present application does not limit the number of independently marked labeling objects, which may be one or more.
  • This step takes one labeling object (such as the labeling object 301f in FIG. 3) as an example to illustrate the generation process of the second initial standard labeling result, Please refer to the description in step S103 below for the scene where multiple labeling objects are marked independently.
  • the service server 100 After obtaining the initial candidate labeling result 301g, the service server 100 determines it as the first initial standard labeling result of the first original image 301b, and can combine the first initial standard labeling result and the first original image 301b The association is stored in the image database 30b.
  • the image database 30b may be a database specially used by the business server 30a for storing images, and the above-mentioned image database 30b may be regarded as an electronic filing cabinet—store electronic files (this application may include original images, initial auxiliary labeling results, and initial standard labeling results, etc.), the service server 30a can perform operations such as adding, querying, updating, and deleting original images in the file, initial auxiliary labeling results, and initial standard labeling results.
  • the so-called “database” is a collection of data that is stored together in a certain way, can be shared with multiple users, has as little redundancy as possible, and is independent of the application program.
  • Step S102 according to the first initial standard labeling result and the first initial auxiliary labeling result, adjust the model parameters in the initial image recognition model to generate an updated image recognition model.
  • the embodiments of the present application can be applied to various scenarios such as cloud technology, artificial intelligence, intelligent transportation, and assisted driving.
  • cloud technology artificial intelligence
  • intelligent transportation intelligent transportation
  • assisted driving the field of automatic recognition of medical images
  • detection and classification of medical imaging-oriented artificial intelligence in real time is expected to help clinicians Improve inspection quality and reduce missed diagnosis of lesions.
  • An excellent image recognition model depends on a large amount of representative high-quality labeled data, and the quality of data labeling determines the stability and accuracy of the algorithm model.
  • the embodiment of this application proposes An artificial intelligence-assisted labeling method based on two-way quality control is proposed, which aims to improve the accuracy and efficiency of labeling.
  • FIG. 4 is a schematic diagram of a data processing scenario provided by an embodiment of the present application.
  • the initial auxiliary annotation image 30e includes a first initial auxiliary annotation result 301e
  • the first initial auxiliary annotation result 301e includes a first annotation area for the target object (equivalent to the initial auxiliary annotation area 401a in FIG. 4 ), and the first object label (equivalent to the initial auxiliary object label 401b in FIG. 4 ) for the first labeling region
  • the initial standard labeling image 40c includes a first initial standard labeling result 401c
  • the first initial standard labeling result 401c includes A second labeling area of the object (equivalent to the initial standard labeling area 402a in FIG. 4 ), and a second object label for the second labeling area (equivalent to the initial standard object label 402b in FIG. 4 ).
  • the service server determines the initial area error between the initial auxiliary label area 401a and the initial standard label area 402a, and determines the initial object error between the initial auxiliary object label 401b and the initial standard object label 402b. Further, a weighted summation is performed on the initial region error and the initial object error to obtain the first labeling result error.
  • the business server adjusts the model parameters in the initial image recognition model 30c according to the error of the first annotation result, and generates an updated image recognition model 40d.
  • the embodiment of the present application does not limit the update conditions of the initial image recognition model 30c, and may be a service server responding to a model update instruction for the initial image recognition model 30c.
  • step S202 in the embodiment corresponding to FIG. 6 below.
  • the description will not be expanded for the time being; the update condition of the initial image recognition model 30c can also be that the result error between the initial auxiliary labeling result and the initial standard labeling result in step S101 above reaches the initial loss value threshold.
  • step S302-step S306 in the embodiment corresponding to 7 will not be described here.
  • the embodiment of the present application can determine the first initial standard labeling result and the first initial auxiliary labeling result according to the requirements of the labeling object, so that the initial image recognition model can be personalized and updated.
  • the embodiment of the present application can perform specific training on the model according to individual requirements (for example, specific recognition of medical images by doctors), so as to improve the accuracy of target recognition in personalized scenarios.
  • the target object includes multiple object types, and the multiple object types may include a first target object (such as a malignant tumor) and a second target object (such as a benign tumor), and the prediction accuracy of the initial image recognition model for the first target object, is lower than the prediction accuracy for the second target object, so the initial standard labeling result including the first target object can be used as the first initial standard labeling result, and the initial auxiliary labeling result including the first target object can be used as the first initial auxiliary labeling result , at this time, according to the above-mentioned first initial standard labeling result and the first initial auxiliary labeling result, the model parameters in the initial image recognition model are adjusted to generate an updated image recognition model for the first target object, so as to improve the accuracy of the first target object.
  • the prediction accuracy of a target object It can be seen that, in medical scenarios, the embodiments of the present application can improve the accuracy of hospital detection. It should be noted that updating the image recognition model does not change the prediction accuracy for the second target object.
  • Step S103 Predict the updated auxiliary labeling result of the second original image based on the updated image recognition model, and obtain the updated standard labeling result of the second original image; the updated standard labeling result is obtained by adjusting the second initial standard labeling result based on the updated auxiliary labeling result owned.
  • the updated auxiliary labeling results are sent to at least two labeling terminals.
  • at least two labeling terminals adjust the second initial standard labeling results according to the updated auxiliary labeling results respectively to obtain candidate labeling results of the second original image; obtain candidate labeling results returned by at least two labeling terminals; at least two The candidate labeling results respectively include candidate labeling regions for labeling the target object in the second original image; determine the number of regions corresponding to the candidate labeling regions included in at least two candidate labeling results; determine at least two The initial review and labeling results of candidate labeling results; the updated standard labeling results are obtained according to the initial review and labeling results.
  • the specific process of determining the initial review labeling results of at least two candidate labeling results may include: comparing at least two area numbers; at least two area numbers include area numbers B a ; a is positive Integer, and a is less than or equal to the number of results of at least two candidate labeling results; if there is a number of regions different from the number of regions B a in the remaining number of regions, then at least two candidate labeling results are determined as the initial review labeling results respectively;
  • the number of remaining areas includes the number of areas other than the number of areas B a in at least two areas; if the number of remaining areas is the same as the number of areas B a , then at least two candidate labeling results are obtained, and each of the two candidate labeling results respectively included candidate labeling regions; determining the coincidence degree between the candidate labeling regions respectively included in every two candidate labeling results, and determining the initial review labeling result according to the coincidence degree.
  • the coincidence degree of the two candidate marked regions is, for example, the coincidence degree between the position information of the two candidate marked regions
  • the coincidence degree between the two position information is, for example, the intersection and union of the two position information ratio
  • the position information of the candidate labeled region may represent the position of the candidate labeled region in the image where it is located.
  • At least two candidate labeling results also respectively include candidate object labels for labeling the included candidate labeling regions;
  • the specific process of determining the initial review labeling results according to the coincidence degree may include: if at least one coincidence degree is less than the coincidence degree threshold, then Determine at least two candidate labeling results as the initial review labeling results; if each coincidence degree is equal to or greater than the coincidence degree threshold, divide the same candidate object labels in the at least two candidate labeling results into the same object label group, and get n object label groups; n is a positive integer; determine the initial review labeling result according to n object label groups.
  • the specific process of determining the initial review and labeling results according to the n object label groups may include: counting the number of object labels of the candidate object labels included in the n object label groups respectively, and obtaining from the number of object labels corresponding to the n object label groups respectively The maximum number of object labels; determine the number ratio between the maximum number of object labels and the number of object labels corresponding to at least two candidate labeling results; compare the number ratio with the number ratio threshold, if the number ratio is less than the number ratio threshold, at least two Candidate labeling results are determined as the initial audit labeling results; if the number ratio is equal to or greater than the number ratio threshold, the object label group corresponding to the maximum number of object labels is determined as the target object label group; from the target object label group associated The target candidate labeling result is obtained from the candidate labeling result, and the target candidate labeling result is determined as the initial review labeling result.
  • the specific process of obtaining the updated standard labeling results according to the initial review labeling results may include: if the initial review labeling results are at least two candidate labeling results, sending the initial review labeling results to the first review terminal, so that the first review terminal According to at least two candidate labeling results, the review labeling result sent to the second review terminal is determined; the second review terminal is used to return the update standard labeling result according to the review labeling result; if the initial review labeling result is the target candidate labeling result, the initial review The tagging result is sent to the second review terminal, so that the second review terminal returns an updated standard tagging result according to the target candidate tagging result.
  • the process of predicting the updated auxiliary annotation result of the second original image based on the updated image recognition model please refer to the description of the process of predicting the initial auxiliary annotation result of the original image based on the initial image recognition model in step S101 above, the data processing process of the two The difference is that the updated image recognition model is a model after the initial image recognition model is updated, so details will not be described here.
  • the process for the business server to obtain the updated standard labeling result is basically the same as the process for obtaining the initial standard labeling result, so the process of a labeling terminal adjusting the second initial standard labeling result based on the updated auxiliary labeling result to obtain the updated standard labeling result will not be described here. Please refer to the description in step S101 above.
  • the labeling process can be marked independently by multiple labeling objects, so the business server can send the updated auxiliary labeling results to the labeling terminals corresponding to at least two labeling objects,
  • the labeling terminals corresponding to at least two labeling objects respectively adjust the second initial standard labeling results according to the updated auxiliary labeling results to obtain candidate labeling results of the second original image.
  • FIG. 5 is a schematic diagram of an image processing scene provided by an embodiment of the present application.
  • the number of images of the update candidate annotation images is set to 3, that is, the update candidate annotation image 501a, the update candidate annotation image 502a, and the update candidate annotation image 503a in FIG. 5, when at least two update When the number of candidate labeled images is equal to 2 or other numbers, this embodiment can be referred to.
  • the second original image 501d may include objects such as houses, pedestrians, escalators, and buildings.
  • the target objects are set to include pedestrians and houses.
  • the business server 502d obtains the update candidate annotation image 501a, the update candidate annotation image 502a, and the update candidate annotation image 503a respectively provided by the three annotation objects.
  • the above three update candidate annotation images are all based on the update auxiliary annotation result and the second initial standard annotation result
  • the updated candidate annotation image 501a is an image obtained by adjusting the second initial standard annotation result by the annotation object 101A according to the update auxiliary annotation result
  • the update candidate annotation image 502a is the second initial standard annotation result adjusted by the annotation object 102A according to the update auxiliary annotation result.
  • the image obtained by adjusting the standard annotation results is based on the update auxiliary annotation result and the second initial standard annotation result.
  • the candidate annotation results corresponding to the updated candidate annotation image 501a include the candidate annotation result 501c for the house and the candidate annotation result 501b for the pedestrian, so the candidate annotation results for the updated candidate annotation image 501a include two candidate annotations Region;
  • the candidate labeling result corresponding to the update candidate labeling image 502a includes the candidate labeling result 502c for labeling the house, and the candidate labeling result 502b for marking pedestrians, so the candidate labeling result corresponding to the update candidate labeling image 502a includes 2 candidate labeling regions;
  • the candidate labeling results corresponding to the labeled image 503a include the candidate labeling results 503c labeling houses and the candidate labeling results 503b labeling pedestrians, so the candidate labeling results corresponding to the updated candidate labeling image 503a include two candidate labeling regions.
  • the service server 502d determines the number of regions corresponding to the candidate labeling regions respectively included in the three candidate labeling results (that is, the three updated candidate labeling images). Obviously, in Fig. 5, the number of the three areas is the same, both are 2, at this time, the business server 502d needs to determine each candidate labeling area in each update candidate labeling image, and the candidate labeling area in other update candidate area images The degree of coincidence between them, and then determine the initial review and labeling results according to the degree of coincidence.
  • the three updated candidate labeled images in FIG. 5 have no difference except the candidate labeling results included respectively (because the three updated candidate labeled images are all generated based on the second original image 501d), so the above
  • the upper left corner of the three images (that is, the update candidate annotation image 501a, the update candidate annotation image 502a, and the update candidate annotation image 503a) is the origin of the coordinates, the x-axis is to the right of the origin, and the y-axis is to the bottom of the origin.
  • the generated coordinates are consistent Therefore, the position information corresponding to the target objects in the three images is consistent.
  • the degree of coincidence between the candidate marked regions included in other images See below for understanding.
  • the business server 502d acquires the location information L 501c of the candidate annotation result 501c in the updated candidate annotation image 501a, and the location information L 501b of the candidate annotation result 501b ; obtains the location information of the candidate annotation result 502c in the updated candidate annotation image 502a L 502c , and the location information L 502b of the candidate tagging result 502b ; the business server 502d determines the intersection location information L 501c ⁇ 502c of the location information L 501c of the candidate tagging result 501c and the location information L 502c of the candidate tagging result 502c , and determines the location information L 501c and the union position information L 501c ⁇ 502c of the position information L 502c; the service server 502d determines the intersection position information L 501b ⁇ 502c of the position information L 501b of the candidate labeling result 501b and the position information L 502c of the candidate labeling result 502
  • the following is an example of determining and updating the first coincidence degree of the candidate labeling result 501c in the candidate labeling image 501a (equal to the coincidence degree of the candidate labeling region included in the candidate labeling result 501c), and determining and updating the candidate labeling result 501b in the candidate labeling image 501a
  • the first coincidence degree can refer to the following process.
  • the service server 502d can determine the candidate coincidence degree C (501c, 502c) between the candidate tagging result 501c and the candidate tagging result 502c according to formula (1).
  • ROI 501c may represent the candidate labeling region of the candidate labeling result 501c, and may be determined by the location information L 501c
  • ROI 502c may represent the candidate labeling region of the candidate labeling result 502c, and may be determined by the location information L 502c
  • ROI 501c ⁇ ROI 502c may represent the intersection area of the candidate labeling region of the candidate labeling result 501c and the candidate labeling region of the candidate labeling result 502c, and may be determined by the intersection position information L 501c ⁇ 502c
  • ROI 501c ⁇ ROI 502c may represent the candidate labeling of the candidate labeling result 501c region and the union region of the candidate labeling regions of the candidate labeling result 502c, and may be determined by the union position information L 501c ⁇ 502c .
  • the service server 502d can determine the candidate coincidence degree C (501c, 502b) between the candidate tagging result 501c and the candidate tagging result 502b according to formula (2).
  • ROI 502b can represent the candidate labeling area of the candidate labeling result 502b, and can be determined by the location information L 502b
  • ROI 501c ⁇ ROI 502b can represent the intersection of the candidate labeling area of the candidate labeling result 501c and the candidate labeling area of the candidate labeling result 502b region, and can be determined by the intersection position information L 501c ⁇ 502b
  • ROI 501c ⁇ ROI 502b can represent the union region of the candidate labeling region of the candidate labeling result 501c and the candidate labeling region of the candidate labeling result 502b, and can be determined by the union position information L 501c ⁇ 502b is determined.
  • the business server 502d compares the candidate coincidence degree C (501c, 502c) and the candidate coincidence degree C (501c, 502b), and updates the candidate marked image 501a and the updated candidate marked image 502a. Obviously, the candidate marked result 501c and the candidate marked result 502b have no difference. Therefore, the first coincidence degree of the candidate labeling result 501c is the candidate coincidence degree C (501c, 502c) .
  • the following describes the determination of the second coincidence degree of the candidate annotation result 502b in the updated candidate annotation image 502a as an example.
  • determining the second coincidence degree of the candidate annotation result 502c in the updated candidate annotation image 502a refer to the following process.
  • the service server 502d can determine the candidate coincidence degree C (501b, 502b) between the candidate tagging result 502b and the candidate tagging result 501b according to formula (3).
  • ROI 501b can represent the candidate labeling region of the candidate labeling result 501c, and can be determined by the location information L 501b
  • ROI 501b ⁇ ROI 502b can represent the intersection of the candidate labeling region of the candidate labeling result 501b and the candidate labeling region of the candidate labeling result 502b region, and can be determined by the intersection position information L 501b ⁇ 502b
  • ROI 501b ⁇ ROI 502b can represent the union region of the candidate labeling region of the candidate labeling result 501b and the candidate labeling region of the candidate labeling result 502b, and can be determined by the union position information L 501b ⁇ 502b is determined.
  • the business server 502d compares the candidate coincidence degree C (501b, 502b) and the candidate coincidence degree C (501c, 502b), and updates the candidate marked image 501a and the updated candidate marked image 502a. Obviously, the candidate marked result 501c and the candidate marked result 502b have no difference. Therefore, the second coincidence degree of the candidate labeling result 502b is the candidate coincidence degree C (501b, 502b) .
  • the business server 502d will update the first coincidence degree of each candidate marked region (including the candidate marked result 501c and the candidate marked result 501b) in the candidate marked image 501a, and update each candidate marked region (including the candidate marked result 501b) in the candidate marked image 502a (including the candidate).
  • the second coincidence degree of the labeling result 502c and the candidate labeling result 502b) is determined as the coincidence degree between the candidate labeling regions respectively included in the updated candidate labeling image 501a and the updating candidate labeling image 502a.
  • the service server 502d can display the overlapping area image 50e, wherein the candidate labeling result 501c and the candidate labeling area
  • the black area between the result 502c is the overlapping area of the two
  • the black area between the candidate tagging result 501b and the candidate tagging result 502b is the overlapping area of the two.
  • the business server 502d compares the above coincidence degree with the coincidence degree threshold, and if at least one coincidence degree is less than the coincidence degree threshold, at least two candidate labeling results (that is, the candidate labeling results included in the three updated candidate labeling images) are respectively determined.
  • the business server After the business server determines the initial review and labeling result, it needs to send the initial review and labeling result to the review terminal (including the first review terminal and the second review terminal), so that the review terminal can confirm the result and return the updated standard labeling result. If the initial review and labeling result is at least two candidate labeling results, the business server sends the initial review and labeling result to the first review terminal (equivalent to the first review terminal 200a described in FIG. 1 above), and the first review terminal is in It has the function of arbitration in the whole data processing process.
  • the first review terminal After the first review terminal obtains at least two candidate labeling results, its corresponding arbitration object can view the second original image and at least two candidate labeling results. If the arbitration object confirms that at least two candidate labeling results are not ideal, it can The region labeling and object labeling of the second original image, the process of marking the second original image by the referee is consistent with the process of labeling the first original image by the labeling object, so please refer to the labeling content described in step S101 above . Subsequently, the arbitration object can use the first audit terminal to send the audit mark result remarked by itself as the arbitration result to the second audit terminal (equivalent to the second audit terminal 200b described in FIG. 1 above), so that the second audit terminal The audit object corresponding to the audit terminal audits the arbitration result.
  • the arbitration object approves one of the at least two candidate labeling results
  • the approved candidate labeling result can be directly sent to the second review terminal as the arbitration result, so that the review object can review the arbitration result.
  • the initial review and labeling result is the target candidate labeling result
  • the initial review and labeling result is sent to the second review terminal, and the second review terminal has a review function in the entire image processing process.
  • the review object can review the image through the second review terminal.
  • the review object approves the target candidate tagging result or the arbitration result sent by the first review terminal, it can be stored in the image database associated with the service server (equivalent to the image database 30b in FIG. 3 ). If the review object does not agree with the target candidate labeling result or the arbitration result sent by the first review terminal, the existing labeling data can be discarded, and other labeling objects can be used to perform region labeling and object labeling on the second original image, or the second The original image is re-forwarded to the first review terminal, so that the arbitration object can mark the second original image. Subsequently, the review object reviews the regenerated review labeling results, and the review process is consistent with the above review process, so it will not be described again.
  • this step can use the updated initial image recognition model (that is, update the image recognition model) to perform quality control on the second original image with existing annotation results, so that the existing annotation results can be dynamically updated, thereby improving the target recognition accuracy.
  • updated initial image recognition model that is, update the image recognition model
  • Step S104 when it is determined according to the updated auxiliary labeling result and the updated standard labeling result that the updated image recognition model satisfies the model convergence condition, determine the updated image recognition model as the target image recognition model; the target image recognition model is used to generate the labeling result of the target image.
  • the second original image includes the target object;
  • the updated auxiliary labeling result includes an updated auxiliary labeling area for the target object, and an updated auxiliary object label for the updated auxiliary labeling area;
  • the updated standard labeling result includes an updated standard labeling area for the target object , and the update standard object label for the update standard label area; determine the update area loss value between the update auxiliary label area and the update standard label area; determine the update object loss value between the update auxiliary object label and the update standard object label;
  • the update area loss value and the update object loss value are weighted and summed to obtain the update loss value of the update image recognition model; when the update loss value is greater than or equal to the update loss value threshold, it is determined that the update image recognition model does not meet the model convergence conditions, and continue to The model parameters in the updated image recognition model are adjusted; when the updated loss value is less than the updated loss value threshold, it is determined that the updated image recognition model satisfies the model convergence condition, and the updated image recognition model is determined as the target image recognition model.
  • the original image also includes the third original image
  • the initial standard labeling result also includes the third initial standard labeling result of the third original image
  • the initial auxiliary labeling result also includes the third initial auxiliary labeling result of the third original image
  • the specific process of adjusting the model parameters in the image recognition model may include: determining the adjusted loss value according to the third initial standard labeling result and the third initial auxiliary labeling result; performing weighted summation of the adjusted loss value and the update loss value to obtain the target Loss value; adjust the model parameters in the update image recognition model according to the target loss value.
  • the embodiment of the present application does not limit the number of images respectively corresponding to the first original image, the second original image, and the third original image, and may be any number, which should be set according to actual application scenarios. It can be understood that the first original image and the second original image are different from each other, and the second original image and the third original image are different from each other.
  • the update loss value is less than the update loss value threshold, but the labeling object sends a model continuation update instruction through the labeling terminal, the business server can keep updating the image recognition model, and the process and the update loss value are equal to or greater than the update
  • the subsequent process of the loss value threshold is the same, so it will not be repeated here.
  • the service server can determine the third original image according to the update loss value, and the specific determination process can be as follows: the target object can include at least two target objects, and the at least two target objects can include the first target object; it can be understood that the update loss value can be obtained by an average of the first update loss value for the first target object and the remaining update loss values for the remaining target objects, wherein the remaining target objects include target objects other than the first target object among the at least two target objects; Therefore, the service server can determine the first update loss value and the first loss value ratio between the update loss value, and according to the first loss value ratio and the number of training samples (equal to the number of images of the third original image), from the original image An original image including the first target object and original images including the remaining target objects are acquired, and the above two original images are determined as a third original image.
  • the service server can randomly extract 160 images including the first target object from the original image, similarly, randomly extract the remaining images from the original image, and extract The image including the first target object and the remaining images are determined as the third original image.
  • the computer device may use the first initial standard labeling result and the first initial auxiliary labeling result as a training sample set to update the initial image recognition model, that is, adjust the model parameters to obtain an updated image recognition model,
  • this process can not only realize model update, but also determine the direction of model update according to the training sample set; further, based on the updated image recognition model, predict the updated auxiliary labeling result of the second original image, and obtain the updated auxiliary labeling result,
  • the updated standard labeling result obtained by adjusting the second initial standard labeling result this process can realize the update of the second initial standard labeling result; further, when the updated image recognition model is determined as the target image recognition model, use the target image recognition
  • the model generates object-assisted annotation results for object images.
  • the embodiment of the present application can not only update the initial image recognition model based on the training sample set to improve the recognition ability of the updated image recognition model; it can also update the second initial standard labeling result by updating the image recognition model to improve The accuracy of the standard labeling results is updated, so the two-way update of the image recognition model and the labeling results can be realized by using this application.
  • FIG. 6 is a schematic flowchart of a data processing method provided by an embodiment of the present application.
  • the method can be executed by a service server (for example, the service server 100 shown in the above-mentioned FIG. 1 ), or by a labeling terminal (for example, the labeling terminal 100a shown in the above-mentioned FIG. 1 ), and can also be performed by the service server and the labeling terminal. implement.
  • the method may at least include the following steps.
  • Step S201 Predict the initial auxiliary labeling result of the original image based on the initial image recognition model, and obtain the initial standard labeling result determined by correcting the initial auxiliary labeling result;
  • the original image includes the first original image and the second original image;
  • the initial standard labeling result The first initial standard labeling result of the first original image and the second initial standard labeling result of the second original image are included;
  • the initial auxiliary labeling result includes the first initial auxiliary labeling result of the first original image.
  • step S201 for the specific implementation process of step S201, please refer to step S101 in the embodiment corresponding to FIG. 2 above, which will not be repeated here.
  • Step S202 in response to the model update instruction, determine the first original image as the sample image, determine the first initial standard annotation result as the sample label of the sample image, and determine the first initial auxiliary annotation result as the sample prediction result of the sample image.
  • Step S203 determine the total loss value of the initial image recognition model according to the sample label and the sample prediction result.
  • Step S204 adjust the model parameters in the initial image recognition model according to the total loss value, and determine the adjusted initial image recognition model as the updated image recognition model when the adjusted initial image recognition model satisfies the model convergence condition.
  • the business server In conjunction with steps S202 to S204, the business server currently uses the initial image recognition model to predict the original image in the image database, and generates the initial auxiliary labeling result corresponding to the original image, and obtains the initial standard labeling result determined based on the initial auxiliary labeling result
  • the embodiment of the present application does not describe the process of determining the average labeling result error between the initial standard labeling result and the initial auxiliary labeling result. Please refer to the description of steps S302 to S304 in the embodiment corresponding to FIG. 7 below.
  • the model update instruction carries training sample information
  • the training sample information may include at least two object labels, and the number of training samples corresponding to the at least two object labels, for example, the at least two object labels include the first object label, and the second object label
  • the model update command carries the first number of training samples for the first object label and the second number of training samples for the second object label
  • the service server can obtain the number of labeling results equal to the number of labeling results from the initial standard labeling results
  • the initial auxiliary labeling result of is used as the first initial auxiliary labeling result; further, the service server will determine the first initial standard
  • the embodiment of the present application can not only update the initial image recognition model, but also can determine the update direction by the business object, so the update efficiency can be improved, and the prediction accuracy of the model can also be improved.
  • Step S205 predicting the updated auxiliary annotation result of the second original image based on the updated image recognition model, and obtaining the updated standard annotation result; the updated standard annotation result is obtained by adjusting the second initial standard annotation result based on the updated auxiliary annotation result.
  • Step S206 when it is determined that the updated image recognition model satisfies the model convergence condition according to the updated auxiliary labeling result and the updated standard labeling result, determine the updated image recognition model as the target image recognition model; the target image recognition model is used to generate the target auxiliary labeling of the target image result.
  • step S205-step S206 please refer to step S103-step S104 in the embodiment corresponding to FIG. 2 above, and details will not be described here.
  • the computer device may use the first initial standard labeling result and the first initial auxiliary labeling result as a training sample set to update the initial image recognition model, that is, adjust the model parameters to obtain an updated image recognition model,
  • this process can not only realize model update, but also determine the direction of model update according to the training sample set; further, based on the updated image recognition model, predict the updated auxiliary labeling result of the second original image, and obtain the updated auxiliary labeling result,
  • the updated standard labeling result obtained by adjusting the second initial standard labeling result this process can realize the update of the second initial standard labeling result; further, when the updated image recognition model is determined as the target image recognition model, use the target image recognition
  • the model generates object-assisted annotation results for object images.
  • the embodiment of the present application can not only update the initial image recognition model based on the training sample set to improve the recognition ability of the updated image recognition model; it can also update the second initial standard labeling result by updating the image recognition model to improve The accuracy of the standard labeling results is updated, so the two-way update of the image recognition model and the labeling results can be realized by using this application.
  • FIG. 7 is a schematic flowchart of a data processing method provided in an embodiment of the present application.
  • the method can be executed by a service server (for example, the service server 100 shown in the above-mentioned FIG. 1 ), or by a labeling terminal (for example, the labeling terminal 100a shown in the above-mentioned FIG. 1 ), and can also be performed by the service server and the labeling terminal. implement.
  • the method may at least include the following steps.
  • Step S301 Predict the initial auxiliary labeling result of the original image based on the initial image recognition model, and obtain the initial standard labeling result determined based on the initial auxiliary labeling result;
  • the original image includes the first original image and the second original image;
  • the initial standard labeling result includes the first A first initial standard labeling result of an original image, and a second initial standard labeling result of a second original image;
  • the initial auxiliary labeling result includes the first initial auxiliary labeling result of the first original image.
  • step S301 for the specific implementation process of step S301, please refer to step S101 in the embodiment corresponding to FIG. 2 above, which will not be repeated here.
  • Step S302 determining a first labeling result error between the first initial auxiliary labeling result and the first initial standard labeling result.
  • Step S303 determining a second labeling result error between the second initial auxiliary labeling result and the second initial standard labeling result.
  • Step S304 determining the average labeling result error between the first labeling result error and the second labeling result error
  • Step S305 determining the initial loss value of the initial image recognition model according to the average labeling result error.
  • Step S306 if the initial loss value is greater than or equal to the initial loss value threshold, adjust the model parameters in the initial image recognition model according to the first initial standard labeling result and the first initial auxiliary labeling result to generate an updated image recognition model.
  • Step S307 Predict the updated auxiliary annotation result of the second original image based on the updated image recognition model, and obtain the updated standard annotation result; the updated standard annotation result is obtained by adjusting the second initial standard annotation result based on the updated auxiliary annotation result.
  • Step S308 when it is determined that the updated image recognition model satisfies the model convergence condition according to the updated auxiliary labeling results and the updated standard labeling results, determine the updated image recognition model as the target image recognition model; the target image recognition model is used to generate target auxiliary labeling of the target image result.
  • step S307-step S308 For the specific implementation process of step S307-step S308, please refer to step S103-step S104 in the embodiment corresponding to FIG. 2 above, and details will not be described here.
  • FIG. 8 is a schematic flowchart of a data processing method provided in an embodiment of the present application.
  • the business server inputs the original image into the artificial intelligence-assisted annotation model (equivalent to the above-mentioned initial image recognition model), and obtains the initial auxiliary annotation result of the original image; the business server sends the initial auxiliary annotation result to the corresponding annotation object.
  • the artificial intelligence-assisted annotation model equivalent to the above-mentioned initial image recognition model
  • the labeling terminal so that the labeling object can view the original image and the initial auxiliary labeling result through the labeling terminal, and determine the initial candidate labeling result based on the initial secondary labeling result; the business server obtains the initial candidate labeling result returned by the labeling terminal, and obtains Initial standard labeling results; counting the result errors between the initial auxiliary labeling results and the initial standard labeling results, whether to manually start the model update, if the model update is started, the business server will be based on the first initial standard labeling results and the first initial auxiliary labeling results, Perform a model update on the initial image recognition model. If the model update is not started, check whether the auxiliary labeling effect is up to standard.
  • the auxiliary labeling effect is effective, continue to run the artificial intelligence-assisted labeling model; if the auxiliary labeling If the effect is not up to standard, then based on the first initial standard labeling result and the first initial auxiliary labeling result, the initial image recognition model is updated to obtain an updated artificial intelligence-assisted labeling model (equivalent to the above-mentioned updated image recognition model); business server By updating the image recognition model, re-predicting the marked second original image to obtain an updated auxiliary labeling result; sending the updated auxiliary labeling result to the labeling terminal corresponding to the labeling object, so that the labeling object can view the updated auxiliary labeling result through the labeling terminal, And based on the updated auxiliary labeling results, modify or confirm the second initial standard labeling results to obtain candidate labeling results; the business server obtains the candidate labeling results returned by the labeling terminal, and obtains the updated standard labeling results based on the candidate labeling results; The result error between the labeling result and the updated
  • the computer device may use the first initial standard labeling result and the first initial auxiliary labeling result as a training sample set to update the initial image recognition model, that is, adjust the model parameters to obtain an updated image recognition model,
  • this process can not only realize model update, but also determine the direction of model update according to the training sample set; further, based on the updated image recognition model, predict the updated auxiliary labeling result of the second original image, and obtain the updated auxiliary labeling result,
  • the updated standard labeling result obtained by adjusting the second initial standard labeling result this process can realize the update of the second initial standard labeling result; further, when the updated image recognition model is determined as the target image recognition model, use the target image recognition
  • the model generates object-assisted annotation results for object images.
  • the embodiment of the present application can not only update the initial image recognition model based on the training sample set to improve the recognition ability of the updated image recognition model; it can also update the second initial standard labeling result by updating the image recognition model to improve The accuracy of the standard labeling results is updated, so the two-way update of the image recognition model and the labeling results can be realized by using this application.
  • FIG. 9 is a schematic structural diagram of a data processing device provided by an embodiment of the present application.
  • the above-mentioned data processing device may be a computer program (including program code) running on a computer device, for example, the data processing device is an application software; the device may be used to execute the corresponding steps in the method provided by the embodiment of the present application.
  • the data processing device 1 may include: a first acquisition module 11 , an update model module 12 , a second acquisition module 13 and a first determination module 14 .
  • the first acquisition module 11 is used to predict the initial auxiliary labeling result of the original image based on the initial image recognition model, and obtain the initial standard labeling result determined by correcting the initial auxiliary labeling result;
  • the original image includes the first original image and the second original image ;
  • the initial standard labeling result includes the first initial standard labeling result of the first original image, and the second initial standard labeling result of the second original image;
  • the initial auxiliary labeling result includes the first initial auxiliary labeling result of the first original image;
  • the update model module 12 is used to adjust the model parameters in the initial image recognition model according to the first initial standard labeling result and the first initial auxiliary labeling result to generate an updated image recognition model;
  • the second acquisition module 13 is used to predict the updated auxiliary labeling result of the second original image based on the updated image recognition model, and obtain the updated standard labeling result of the second original image; the updated standard labeling result is based on the update of the auxiliary labeling result to the second initial standard obtained by adjusting the labeling results;
  • the first determination module 14 is configured to determine the updated image recognition model as the target image recognition model when it is determined according to the updated auxiliary labeling result and the updated standard labeling result that the updated image recognition model satisfies the model convergence condition; the target image recognition model is used to generate the target Image annotation results.
  • the specific function implementation manners of the first acquisition module 11, the update model module 12, the second acquisition module 13, and the first determination module 14 can refer to steps S101-step S104 in the above-mentioned embodiment corresponding to FIG. 2 , and will not be repeated here .
  • the data processing device 1 may further include: a second determination module 15 .
  • the second determination module 15 is configured to determine the first original image as the sample image in response to the model update instruction, determine the first initial standard labeling result as the sample label of the sample image, and determine the first initial auxiliary labeling result as the sample image Sample prediction results;
  • the update model module 12 includes: a first determination unit 121 and a second determination unit 122 .
  • the first determination unit 121 is configured to determine the total loss value of the initial image recognition model according to the sample label and the sample prediction result;
  • the second determination unit 122 is configured to adjust the model parameters in the initial image recognition model according to the total loss value, and determine the adjusted initial image recognition model as an update when the adjusted initial image recognition model satisfies the model convergence condition Image recognition model.
  • steps S202-step S204 for the implementation of specific functions of the second determining module 15, the first determining unit 121 and the second determining unit 122, reference may be made to steps S202-step S204 in the embodiment corresponding to FIG. 6 above, which will not be repeated here.
  • the initial auxiliary labeling result also includes a second initial auxiliary labeling result of the second original image
  • the data processing device 1 may further include: a third determination module 16 and an execution step module 17 .
  • the third determining module 16 is configured to determine the first labeling result error between the first initial auxiliary labeling result and the first initial standard labeling result;
  • the third determining module 16 is also used to determine the second labeling result error between the second initial auxiliary labeling result and the second initial standard labeling result;
  • the third determination module 16 is also used to determine the average labeling result error between the first labeling result error and the second labeling result error;
  • the third determination module 16 is also used to determine the initial loss value of the initial image recognition model according to the average labeling result error
  • Executing step module 17 for if the initial loss value is greater than or equal to the initial loss value threshold, perform adjustment of the model parameters in the initial image recognition model according to the first initial standard labeling result and the first initial auxiliary labeling result, and generate an update The steps of an image recognition model.
  • steps S302-step S306 in the embodiment corresponding to FIG. 7 above, which will not be repeated here.
  • the first original image includes the target object;
  • the first initial auxiliary labeling result includes the first labeling area for the target object, and the first object label for the first labeling area;
  • the first initial standard labeling result includes for a second marked region of the target object, and a second object label for the second marked region;
  • the third determining module 16 may include: a third determining unit 161 and a first weighting unit 162 .
  • a third determining unit 161, configured to determine an initial area error between the first marked area and the second marked area
  • the third determining unit 161 is further configured to determine an initial object error between the first object label and the second object label;
  • the first weighting unit 162 is configured to perform a weighted summation of the initial region error and the initial object error to obtain a first labeling result error.
  • step S302 for specific function implementation manners of the third determination unit 161 and the first weighting unit 162, reference may be made to step S302 in the above-mentioned embodiment corresponding to FIG. 7 , which will not be repeated here.
  • the second original image includes the target object;
  • the updated auxiliary labeling result includes an updated auxiliary labeling area for the target object, and an updated auxiliary object label for the updated auxiliary labeling area;
  • the updated standard labeling result includes an update for the target object Standard callout areas, and updated standard object labels for updated standard callout areas;
  • the first determination module 14 may include: a fourth determination unit 141 , a second weighting unit 142 , a fifth determination unit 143 and a sixth determination unit 144 .
  • the fourth determination unit 141 is configured to determine the update area loss value between the updated auxiliary marked area and the updated standard marked area;
  • the fourth determination unit 141 is also configured to determine the updated object loss value between the updated auxiliary object label and the updated standard object label;
  • the second weighting unit 142 is configured to perform weighted summation of the updated region loss value and the updated object loss value to obtain an updated image recognition model updated loss value;
  • the fifth determining unit 143 is configured to determine that the updated image recognition model does not meet the model convergence condition when the updated loss value is greater than or equal to the updated loss value threshold, and continue to adjust the model parameters in the updated image recognition model;
  • the sixth determination unit 144 is configured to determine that the updated image recognition model satisfies the model convergence condition when the updated loss value is smaller than the updated loss value threshold, and determine the updated image recognition model as the target image recognition model.
  • step S104 for the implementation of the specific functions of the fourth determination unit 141 , the second weighting unit 142 , the fifth determination unit 143 and the sixth determination unit 144 , please refer to step S104 in the embodiment corresponding to FIG. 2 above, which will not be repeated here.
  • the original image also includes a third original image;
  • the initial standard labeling result also includes a third initial standard labeling result of the third original image;
  • the initial auxiliary labeling result also includes a third initial auxiliary labeling result of the third original image ;
  • the fifth determination unit 143 may include: a first determination subunit 1431 and an adjustment model subunit 1432 .
  • the first determination subunit 1431 is configured to determine the adjusted loss value according to the third initial standard labeling result and the third initial auxiliary labeling result;
  • the first determination subunit 1431 is further configured to perform weighted summation of the adjusted loss value and the updated loss value to obtain the target loss value;
  • the adjusting model subunit 1432 is used to adjust the model parameters in the updated image recognition model according to the target loss value.
  • step S104 for the specific function implementation manners of the first determination subunit 1431 and the adjustment model subunit 1432, reference may be made to step S104 in the above-mentioned embodiment corresponding to FIG. 2 , which will not be repeated here.
  • the second obtaining module 13 may include: a sending assistance unit 131 , a first obtaining unit 132 , a seventh determining unit 133 , an eighth determining unit 134 , and a second obtaining unit 135 .
  • the sending auxiliary unit 131 is configured to send the updated auxiliary labeling results to the labeling terminals corresponding to at least two labeling objects, so that the labeling terminals corresponding to the at least two labeling objects perform the second initial standard labeling results according to the updated auxiliary labeling results respectively. Adjust to obtain the candidate annotation result of the second original image;
  • the first acquiring unit 132 is configured to acquire candidate labeling results returned by labeling terminals respectively corresponding to at least two labeling objects; the at least two candidate labeling results respectively include candidate labeling regions for labeling target objects in the second original image;
  • the seventh determination unit 133 is configured to determine the number of regions corresponding to the candidate labeling regions respectively included in the at least two candidate labeling results;
  • the eighth determining unit 134 is configured to determine the initial review and marking results of at least two candidate marking results according to the quantity of at least two regions;
  • the second acquiring unit 135 is configured to acquire an updated standard labeling result according to the initial review labeling result.
  • step S103 for the implementation of specific functions of the sending auxiliary unit 131, the first obtaining unit 132, the seventh determining unit 133, the eighth determining unit 134, and the second obtaining unit 135, please refer to step S103 in the above-mentioned embodiment corresponding to FIG. Let me repeat.
  • the eighth determination unit 134 may include: a comparison quantity subunit 1341 , a second determination subunit 1342 , an acquisition area subunit 1343 and a third determination subunit 1344 .
  • the comparison number subunit 1341 is used to compare at least two area numbers; the at least two area numbers include the area number Ba; a is a positive integer, and a is less than or equal to the result number of at least two candidate labeling results;
  • the second determination subunit 1342 is used to determine at least two candidate labeling results as the initial review labeling results respectively if there is a region number different from the region quantity Ba in the remaining region quantity; the remaining region quantity includes at least two region quantities The number of areas in excluding the area number Ba;
  • the obtaining region subunit 1343 is used to obtain the candidate labeling regions included in each of the at least two candidate labeling results, if the number of remaining regions is the same as the number of regions Ba;
  • the third determining subunit 1344 is configured to determine the coincidence degree between the candidate labeling regions included in each two candidate labeling results, and determine the initial review labeling result according to the coincidence degree.
  • step S104 the specific function implementation manners of the comparison quantity subunit 1341, the second determination subunit 1342, the acquisition area subunit 1343, and the third determination subunit 1344 can refer to step S104 in the above-mentioned embodiment corresponding to FIG. 2 , which will not be repeated here. .
  • At least two candidate labeling results further include candidate object labels for labeling the included candidate labeling regions;
  • the third determination subunit 1344 may include: a first review subunit 13441 , a division label subunit 13442 and a second review subunit 13443 .
  • the first review subunit 13441 is configured to determine at least two candidate labeling results as initial review labeling results if at least one coincidence degree is less than the coincidence degree threshold;
  • n is a positive integer
  • the second review subunit 13443 is configured to determine an initial review and labeling result according to the n object label groups.
  • step S103 For the implementation of specific functions of the first review subunit 13441 , the partition label subunit 13442 and the second review subunit 13443 , refer to step S103 in the above-mentioned embodiment corresponding to FIG. 2 , which will not be repeated here.
  • the second review subunit 13443 is specifically used to count the number of object tags of the candidate object tags included in the n object tag groups, and obtain the largest object tag among the number of object tags corresponding to the n object tag groups. quantity;
  • the second review subunit 13443 is also specifically configured to determine the ratio between the maximum number of object labels and the number of object labels corresponding to at least two candidate labeling results;
  • the second review subunit 13443 is also specifically configured to compare the quantity ratio with the quantity ratio threshold, and if the quantity ratio is smaller than the quantity ratio threshold, determine at least two candidate labeling results as the initial review labeling results;
  • the second review subunit 13443 is also specifically configured to determine the object tag group corresponding to the maximum number of object tags as the target object tag group if the number ratio is equal to or greater than the number ratio threshold;
  • the second review subunit 13443 is also specifically configured to obtain target candidate tagging results from candidate tagging results associated with the target object tag group, and determine the target candidate tagging results as the initial review tagging results.
  • step S103 for the specific function implementation manner of the second review subunit 13443, refer to step S103 in the above-mentioned embodiment corresponding to FIG. 2 , which will not be repeated here.
  • the second acquiring unit 135 may include: a first sending subunit 1351 and a second sending subunit 1352 .
  • the first sending subunit 1351 is configured to send the initial review marking result to the first review terminal if the initial review marking result is at least two candidate marking results, so that the first review terminal determines to send To the review mark result of the second review terminal; the second review terminal is used to return the update standard mark result according to the review mark result;
  • the second sending subunit 1352 is configured to send the initial review marking result to the second review terminal if the initial review marking result is the target candidate marking result, so that the second review terminal returns an updated standard marking result according to the target candidate marking result.
  • the specific function implementation manners of the first sending subunit 1351 and the second sending subunit 1352 can refer to step S103 in the above-mentioned embodiment corresponding to FIG. 2 , which will not be repeated here.
  • the first obtaining module 11 may include: a third obtaining unit 111 , a fourth obtaining unit 112 , a ninth determining unit 113 and a generating result unit 114 .
  • the third acquiring unit 111 is configured to acquire an original image; the original image includes a target object;
  • the fourth acquisition unit 112 is configured to input the original image into the initial image recognition model, and acquire the image features of the original image in the initial image recognition model;
  • a ninth determining unit 113 configured to determine an initial area identification feature of the target object and an initial object identification feature of the target object according to the image feature;
  • a generating result unit 114 configured to generate an initial auxiliary labeling area for the target object according to the initial area identification feature, and generate an initial auxiliary object label for the initial auxiliary labeling area according to the initial object identification feature;
  • the generating result unit 114 is further configured to determine the initial auxiliary labeling region and the initial auxiliary object label as the initial auxiliary labeling result.
  • step S101 for the implementation of specific functions of the third obtaining unit 111 , the fourth obtaining unit 112 , the ninth determining unit 113 , and the generating result unit 114 , refer to step S101 in the above-mentioned embodiment corresponding to FIG. 2 , and details are not repeated here.
  • the computer device may use the first initial standard labeling result and the first initial auxiliary labeling result as a training sample set to update the initial image recognition model, that is, adjust the model parameters to obtain an updated image recognition model,
  • this process can not only realize model update, but also determine the direction of model update according to the training sample set; further, based on the updated image recognition model, predict the updated auxiliary labeling result of the second original image, and obtain the updated auxiliary labeling result,
  • the updated standard labeling result obtained by adjusting the second initial standard labeling result this process can realize the update of the second initial standard labeling result; further, when the updated image recognition model is determined as the target image recognition model, use the target image recognition
  • the model generates object-assisted annotation results for object images.
  • the embodiment of the present application can not only update the initial image recognition model based on the training sample set to improve the recognition ability of the updated image recognition model; it can also update the second initial standard labeling result by updating the image recognition model to improve The accuracy of the standard labeling results is updated, so the two-way update of the image recognition model and the labeling results can be realized by using this application.
  • the computer device 1000 may include: at least one processor 1001 , such as a CPU, at least one network interface 1004 , user interface 1003 , memory 1005 , and at least one communication bus 1002 .
  • the communication bus 1002 is used to realize connection and communication between these components.
  • the user interface 1003 may include a display screen (Display) and a keyboard (Keyboard), and the network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).
  • the memory 1005 can be a high-speed RAM memory, or a non-volatile memory, such as at least one disk memory.
  • the memory 1005 may optionally also be at least one storage device located far away from the aforementioned processor 1001 .
  • the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.
  • the network interface 1004 can provide a network communication function;
  • the user interface 1003 is mainly used to provide an input interface for the user; and
  • the processor 1001 can be used to call the device control application stored in the memory 1005 program to achieve:
  • the original image includes the first original image and the second original image
  • the initial standard annotation result includes the first original image
  • the initial auxiliary labeling result includes the first initial auxiliary labeling result of the first original image
  • the model parameters in the initial image recognition model are adjusted to generate an updated image recognition model
  • the updated standard labeling result is obtained by adjusting the second initial standard labeling result based on the updated auxiliary labeling result
  • the updated image recognition model is determined as the target image recognition model; the target image recognition model is used to generate the target auxiliary labeling result of the target image.
  • the computer device 1000 described in the embodiment of the present application can execute the description of the data processing method in the embodiments corresponding to Figure 2, Figure 6, Figure 7 and Figure 8 above, and can also execute the embodiment corresponding to Figure 9 above
  • the description of the data processing device 1 in will not be repeated here.
  • the description of the beneficial effect of adopting the same method will not be repeated here.
  • the embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, the computer-readable storage medium in FIG. 2 , FIG. 6 , and FIG. 7 is implemented.
  • the data processing method provided by each step in FIG. 8 please refer to the implementation methods provided by each step in FIG. 2 , FIG. 6 , FIG. 7 and FIG. 8 , and details will not be repeated here.
  • the description of the beneficial effect of adopting the same method will not be repeated here.
  • the above-mentioned computer-readable storage medium may be the data processing apparatus provided in any one of the foregoing embodiments or an internal storage unit of the above-mentioned computer equipment, such as a hard disk or memory of the computer equipment.
  • the computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (smart media card, SMC), a secure digital (secure digital, SD) card, Flash card (flash card), etc.
  • the computer-readable storage medium may also include both an internal storage unit of the computer device and an external storage device.
  • the computer-readable storage medium is used to store the computer program and other programs and data required by the computer device.
  • the computer-readable storage medium can also be used to temporarily store data that has been output or will be output.
  • the embodiment of the present application also provides a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium.
  • the processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device can execute the data processing in the embodiments corresponding to Figure 2, Figure 6, Figure 7 and Figure 8 above.
  • the description of the method will not be repeated here.
  • the description of the beneficial effect of adopting the same method will not be repeated here.
  • each flow and/or of the method flow charts and/or structural diagrams can be implemented by computer program instructions or blocks, and combinations of processes and/or blocks in flowcharts and/or block diagrams.
  • These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a A device for realizing the functions specified in one or more steps of the flowchart and/or one or more blocks of the structural diagram.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions
  • the device implements the functions specified in one or more blocks of the flowchart and/or one or more blocks of the structural schematic diagram.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby
  • the instructions provide steps for implementing the functions specified in one or more steps of the flowchart and/or one or more blocks in the structural illustration.

Abstract

一种数据处理方法、装置、设备、存储介质及程序产品。该方法包括：基于初始图像识别模型预测原始图像的初始辅助标注结果，获取初始标准标注结果；根据第一初始标准标注结果以及第一初始辅助标注结果，对初始图像识别模型中的模型参数进行调整，生成更新图像识别模型；基于更新图像识别模型预测第二原始图像的更新辅助标注结果，获取更新标准标注结果；当根据更新辅助标注结果以及更新标准标注结果确定更新图像识别模型满足模型收敛条件时，将更新图像识别模型确定为目标图像识别模型。采用本申请，不仅可以提高图像识别模型的识别能力，还可以提高标注结果的精度。本申请实施例可应用于云技术、人工智能、智慧交通、辅助驾驶和医学检测等各种场景。

Description

数据处理方法、装置、设备、存储介质及程序产品
本申请要求于2021年12月13日提交中国专利局、申请号为202111521261.4、申请名称为“一种数据处理方法、设备以及计算机可读存储介质”的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本申请涉及互联网技术领域，尤其涉及一种数据处理方法、装置、设备、存储介质及程序产品。
背景技术
当前对图像中的目标对象的标注，主要包括纯人工标注、纯机器标注以及人工智能辅助标注等。纯人工标注是指标注过程中无模型辅助，依靠标注者对目标对象的识别进行标注；纯机器标注是指标注过程无人工干预，以人工智能模型预测结果为标注结果；人工智能辅助标注是指标注过程中，人工智能模型对图像进行预测，生成预测结果，标注者结合预测结果完成对该图像中的目标对象的标注。
相关人工智能辅助标注中，标注者往往只是人工智能模型的使用者，不参与人工智能模型的更新，从而导致模型无法得到及时的更新，最终影响辅助标注的精度；另外，相关人工智能辅助标注方法对已有标注结果缺乏复核的环节，导致已有标注结果无法更新，若存在精度不高的已有标注结果，则后续训练或使用时，会继续使用该精度不高的标注结果。
发明内容
本申请实施例提供一种数据处理方法、装置、设备、存储介质及程序产品，有助于提高图像识别模型的识别能力，和提高标注结果的精度。
本申请实施例一方面，提供了一种数据处理方法，在计算机设备中执行，所述方法包括：
基于初始图像识别模型预测原始图像的初始辅助标注结果，所述原始图像包括第一原始图像以及第二原始图像，所述初始辅助标注结果包括所述第一原始图像的第一初始辅助标注结果；
获取对所述初始辅助标注结果进行校正所确定的初始标准标注结果；其中，所述初始标准标注结果包括所述第一原始图像的第一初始标准标注结果，以及所述第二原始图像的第二初始标准标注结果；
根据所述第一初始标准标注结果以及所述第一初始辅助标注结果，对所述初始图像识别模型中的模型参数进行调整，生成更新图像识别模型；
基于所述更新图像识别模型预测所述第二原始图像的更新辅助标注结果；
获取所述第二原始图像的更新标准标注结果，所述更新标准标注结果是基于所述更新辅助标注结果对所述第二初始标准标注结果进行调整所得到的；
当根据所述更新辅助标注结果以及所述更新标准标注结果确定所述更新图像识别模型满足模型收敛条件时，将所述更新图像识别模型确定为目标图像识别模型，所述目标图像识别模型用于生成目标图像的标注结果。
本申请实施例一方面提供了一种数据处理装置，包括：
第一获取模块，用于基于初始图像识别模型预测原始图像的初始辅助标注结果，所述原始图像包括第一原始图像以及第二原始图像，所述初始辅助标注结果包括所述第一原始图像的第一初始辅助标注结果；获取对所述初始辅助标注结果进行校正所确定的初始标准标注结果；其中，所述初始标准标注结果包括所述第一原始图像的第一初始标准标注结果，以及所述第二原始图像的第二初始标准标注结果；
更新模型模块，用于根据第一初始标准标注结果以及第一初始辅助标注结果，对初始图像识别模型中的模型参数进行调整，生成更新图像识别模型；
第二获取模块，用于基于更新图像识别模型预测第二原始图像的更新辅助标注结果，获取所述第二原始图像的更新标准标注结果；更新标准标注结果是基于更新辅助标注结果对第二初始标准标注结果进行调整所得到的；
第一确定模块，用于当根据更新辅助标注结果以及更新标准标注结果确定更新图像识别模型满足 模型收敛条件时，将更新图像识别模型确定为目标图像识别模型；目标图像识别模型用于生成目标图像的标注结果。
本申请一方面提供了一种计算机设备，包括：处理器、存储器、网络接口；
上述处理器与上述存储器、上述网络接口相连，其中，上述网络接口用于提供数据通信功能，上述存储器用于存储计算机程序，上述处理器用于调用上述计算机程序，以使得计算机设备执行本申请实施例中的方法。
本申请实施例一方面提供了一种计算机可读存储介质，上述计算机可读存储介质中存储有计算机程序，上述计算机程序适于由处理器加载并执行本申请实施例中的方法。
本申请实施例一方面提供了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中；计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行本申请实施例中的方法。
附图说明
为了更清楚地说明本申请实施例或相关技术中的技术方案，下面将对实施例或相关技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种系统架构示意图；
图2是本申请实施例提供的一种数据处理方法的流程示意图；
图3是本申请实施例提供的一种数据处理的场景示意图；
图4是本申请实施例提供的一种数据处理的场景示意图；
图5是本申请实施例提供的一种数据处理的场景示意图；
图6是本申请实施例提供的一种数据处理方法的流程示意图；
图7是本申请实施例提供的一种数据处理方法的流程示意图；
图8是本申请实施例提供的一种数据处理方法的流程示意图；
图9是本申请实施例提供的一种数据处理装置的结构示意图；
图10是本申请实施例提供的一种计算机设备的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
为了便于理解，首先对部分名词进行以下简单解释：
人工智能(Artificial Intelligence，AI)是利用数字计算机或者数字计算机控制的机器模拟、延伸和扩展人的智能，感知环境、获取知识并使用知识获得最佳结果的理论、方法、技术及应用系统。换句话说，人工智能是计算机科学的一个综合技术，它企图了解智能的实质，并生产出一种新的能以人类智能相似的方式做出反应的智能机器。人工智能也就是研究各种智能机器的设计原理与实现方法，使机器具有感知、推理与决策的功能。
人工智能技术是一门综合学科，涉及领域广泛，既有硬件层面的技术也有软件层面的技术。人工智能基础技术一般包括如传感器、专用人工智能芯片、云计算、分布式存储、大数据处理技术、操作/交互系统、机电一体化等技术。人工智能软件技术主要包括计算机视觉技术、语音处理技术、自然语言处理技术以及机器学习/深度学习、自动驾驶、智慧交通等几大方向。
计算机视觉技术(Computer Vision，CV)是一门研究如何使机器“看”的科学，更进一步的说，就是指用摄影机和电脑代替人眼对目标进行识别和测量等机器视觉，并进一步做图形处理，使电脑处理成为更适合人眼观察或传送给仪器检测的图像。作为一个科学学科，计算机视觉研究相关的理论和技术，试图建立能够从图像或者多维数据中获取信息的人工智能系统。计算机视觉技术通常包括图像处 理、图像识别、图像语义理解、图像检索、OCR、视频处理、视频语义理解、视频内容/行为识别、三维物体重建、3D技术、虚拟现实、增强现实、同步定位与地图构建、自动驾驶、智慧交通等技术，还包括常见的人脸识别、指纹识别等生物特征识别技术。在本申请实施例中，计算机视觉技术可以用于识别图像中的目标对象(例如人、狗、猫、鸟等)，并勾画标注目标对象。
机器学习(Machine Learning，ML)是一门多领域交叉学科，涉及概率论、统计学、逼近论、凸分析、算法复杂度理论等多门学科。专门研究计算机怎样模拟或实现人类的学习行为，以获取新的知识或技能，重新组织已有的知识结构使之不断改善自身的性能。机器学习是人工智能的核心，是使计算机具有智能的根本途径，其应用遍及人工智能的各个领域。机器学习和深度学习通常包括人工神经网络、置信网络、强化学习、迁移学习、归纳学习、式教学习等技术。在本申请实施例中，初始图像识别模型以及更新图像识别模型均是基于机器学习技术的AI模型，可用于对图像进行识别处理。
请参见图1，图1是本申请实施例提供的一种系统架构示意图。如图1所示，该系统可以包括业务服务器100、标注终端集群、第一审核终端200a以及第二审核终端200b；标注终端集群可以包括：标注终端100a、标注终端100b、…、标注终端100c，可以理解的是，上述系统可以包括一个或者多个标注终端，本申请不对标注终端的数量进行限制。上述系统可以包括一个或者多个第一审核终端，也可以包括一个或者多个第二审核终端，本申请实施例不对第一审核终端以及第二审核终端的数量进行限制。
其中，标注终端集群可以包括一个或者多个标注用户对应的标注终端；业务服务器100可以为获取由标注终端所提供的初始候选标注结果以及更新候选标注结果(等同于下文描述的候选标注结果)的设备；第一审核终端可以为审核至少两个候选标注结果的审核终端；第二审核终端可以为审核目标候选标注结果的审核终端。
其中，标注终端集群之间可以存在通信连接，例如标注终端100a与标注终端100b之间存在通信连接，标注终端100a与标注终端100c之间存在通信连接。同时，标注终端集群中的任一标注终端可以与业务服务器100存在通信连接，例如标注终端100a与业务服务器100之间存在通信连接；其中，上述的标注终端集群中的任一标注终端可以与上述的审核终端(包括第一审核终端200a以及第二审核终端200b)之间可以存在通信连接，例如标注终端100a与第一审核终端200a之间存在通信连接，标注终端100b与第一审核终端200a之间存在通信连接，标注终端100b与第二审核终端200b之间存在通信连接。
其中，第一审核终端200a以及第二审核终端200b之间可以存在通信连接；任一审核终端(包括第一审核终端200a以及第二审核终端200b)可以与业务服务器100存在通信连接，例如第一审核终端200a与业务服务器100之间存在通信连接。
其中，上述通信连接不限定连接方式，可以通过有线通信方式进行直接或间接地连接，也可以通过无线通信方式进行直接或间接地连接，还可以通过其它方式，本申请在此不做限制。
应当理解，如图1所示的标注终端集群中的每个标注终端均可以安装有应用客户端，当该应用客户端运行于各标注终端中时，可以分别与上述图1所示的业务服务器100之间进行数据交互，即上述的通信连接。其中，该应用客户端可以为短视频应用、视频应用、直播应用、社交应用、即时通信应用、游戏应用、音乐应用、购物应用、小说应用、支付应用、浏览器等具有标注图像中的目标对象的功能的应用客户端。其中，该应用客户端可以为独立的客户端，也可以为集成在某客户端(例如，社交客户端、教育客户端以及多媒体客户端等)中的嵌入式子客户端，在此不做限定。以社交应用为例，业务服务器100可以为包括社交应用对应的后台服务器、数据处理服务器等多个服务器的集合，因此，每个标注终端均可以通过该社交应用对应的应用客户端与业务服务器100进行数据传输，如每个标注终端均可以通过社交应用的应用客户端将其本地的图像上传至业务服务器100，进而业务服务器100可以将该图像下发给审核终端或传送至云服务器。
可以理解的是，在本申请的具体实施方式中，涉及到用户信息(如本申请中的初始标准标注结果)等相关的数据，当本申请中的实施例运用到具体产品或技术中时，需要获得用户许可或者同意，且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准。
为便于后续理解和说明，本申请实施例可以在图1所示的标注终端集群中选择一个标注终端作为 目标标注终端，例如以标注终端100a作为目标标注终端。当获取到业务服务器100发送的针对原始图像的初始辅助标注结果，并接收到针对原始图像的对象标注指令时，标注终端100a对应的标注对象(即标注用户)可以以初始辅助标注结果为参考标注结果，对该参考标注结果进行标注操作，如新增目标对象的标注、删除非目标对象的标注、修改目标对象的错误标注以及确认目标对象的标注等操作，则标注终端100a可以生成原始图像的初始候选标注结果，并将初始候选标注结果发送至业务服务器100。其中，上述的初始辅助标注结果是基于初始图像识别模型预测原始图像的图像特征所得到的，其包括针对原始图像中的目标对象的初始辅助标注区域，以及针对初始辅助标注区域的初始辅助对象标签。上述的初始候选标注结果包括用于标注目标对象的初始候选标注区域，以及用于标注初始候选标注区域的初始候选对象标签。
进一步，业务服务器100接收到标注终端100a发送的初始候选标注结果后，可以基于初始候选标注结果得到初始标准标注结果。其中，原始图像包括第一原始图像以及第二原始图像，初始标准标注结果包括第一原始图像的第一初始标准标注结果，以及第二原始图像的第二初始标准标注结果。初始辅助标注结果包括第一原始图像的第一初始辅助标注结果。进一步，业务服务器100根据第一初始标准标注结果以及第一初始辅助标注结果，对初始图像识别模型中的模型参数进行调整，生成更新图像识别模型，该过程可以实现对初始图像识别模型的更新；进一步，基于更新图像识别模型预测第二原始图像的更新辅助标注结果，获取基于更新辅助标注结果对第二初始标准标注结果进行调整所得到的更新标准标注结果，该过程可以实现对已标注的第二原始图像的标注结果(即第二初始标准标注结果)的更新；后续，当根据更新辅助标注结果以及更新标准标注结果确定更新图像识别模型满足模型收敛条件时，业务服务器100将更新图像识别模型确定为目标图像识别模型，其中，目标图像识别模型用于生成目标图像的目标辅助标注结果。其中，第一审核终端200a以及第二审核终端200b的功能，请参见下文图2所对应的实施例中步骤S103中的描述，此处暂不展开描述。
可选的，若标注终端100a的本地存储了上述初始图像识别模型，则标注终端100a可以通过本地的初始图像识别模型获取原始图像的初始辅助标注结果，然后基于初始辅助标注结果生成初始标准标注结果；同理，若标注终端100a的本地存储了上述更新图像识别模型，则标注终端100a可以通过本地的更新图像识别模型获取第二原始图像的更新辅助标注结果，然后基于更新辅助标注结果生成更新标准标注结果，其余过程与上述过程一致，故此处不进行赘述，请参见上文描述。
可以理解的是，由于训练初始图像识别模型以及更新图像识别模型均涉及到大量的离线计算，因此标注终端100a本地的初始图像识别模型以及更新图像识别模型，均可以是由业务服务器100训练完成后发送至标注终端100a的。
需要说明的是，上述业务服务器100、标注终端100a、标注终端100b、...、标注终端100c、第一审核终端200a以及第二审核终端200b均可以为区块链网络中的区块链节点，全文叙述的数据(例如初始图像识别模型、原始数据以及初始标准标注结果)可以进行存储，存储方式可以是区块链节点根据数据生成区块，并将区块添加至区块链中进行存储的方式。
区块链是一种分布式数据存储、点对点传输、共识机制以及加密算法等计算机技术的新型应用模式，主要用于对数据按时间顺序进行整理，并加密成账本，使其不可被篡改和伪造，同时可进行数据的验证、存储和更新。区块链本质上是一个去中心化的数据库，该数据库中的每个节点均存储一条相同的区块链，区块链网络可以将节点区分为核心节点、数据节点以及轻节点。核心节点、数据节点以及轻节点共同组成区块链节点。其中核心节点负责区块链全网的共识，也就是说核心节点为区块链网络中的共识节点。对于区块链网络中的交易数据被写入账本的流程可以为，区块链网络中的数据节点或轻节点获取到交易数据，将交易数据在区块链网络中传递(也就是节点以接力棒的方式进行传递)，直到共识节点收到该交易数据，共识节点再将该交易数据打包进区块，对该区块执行共识，待共识完成后将该交易数据写入账本。此处以原始数据以及初始标准标注结果示例交易数据，业务服务器100(区块链节点)在通过对交易数据的共识后，根据交易数据生成区块，将区块存储至区块链网络中；而对于交易数据(即原始数据以及初始标准标注结果)的读取，则可以由区块链节点在区块链网络中，获取到包含该交易数据的区块，进一步，在区块中获取交易数据。
可以理解的是，本申请实施例提供的方法可以由计算机设备执行，计算机设备包括但不限于标注 终端或业务服务器。其中，业务服务器可以是独立的物理服务器，也可以是多个物理服务器构成的服务器集群或者分布式系统，还可以是提供云数据库、云服务、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、CDN、以及大数据和人工智能平台等基础云计算服务的云服务器。标注终端包括但不限于手机、电脑、智能语音交互设备、智能家电、车载终端等。其中，标注终端和业务服务器可以通过有线或无线方式进行直接或间接地连接，本申请实施例在此不做限制。
进一步地，请参见图2，图2是本申请实施例提供的一种数据处理方法的流程示意图。该数据处理方法可以由业务服务器(例如，上述图1所示的业务服务器100)执行，也可以由终端设备(例如，上述图1所示的终端设备200a)执行，还可以由业务服务器和终端设备交互执行。为便于理解，本申请实施例以该方法由业务服务器执行为例进行说明。如图2所示，该数据处理方法至少可以包括以下步骤S101-步骤S104。
步骤S101，基于初始图像识别模型预测原始图像的初始辅助标注结果，获取对初始辅助标注结果进行校正所确定的初始标准标注结果；原始图像包括第一原始图像以及第二原始图像；初始标准标注结果包括第一原始图像的第一初始标准标注结果，以及第二原始图像的第二初始标准标注结果；初始辅助标注结果包括第一原始图像的第一初始辅助标注结果。
在一个实施例中，步骤S101的操作包括：获取原始图像；原始图像包括目标对象；将原始图像输入至初始图像识别模型，在初始图像识别模型中获取原始图像的图像特征；根据图像特征确定目标对象的初始区域识别特征，以及目标对象的初始对象识别特征；根据初始区域识别特征生成针对目标对象的初始辅助标注区域，根据初始对象识别特征生成针对初始辅助标注区域的初始辅助对象标签；将初始辅助标注区域以及初始辅助对象标签确定为初始辅助标注结果。
初始图像识别模型是指用于识别原始图像中的目标对象的人工智能模型，本申请实施例不对初始图像识别模型的模型类型进行限定，可以根据实际应用场景确定初始图像识别模型，包括但不限与卷积神经网络(Convolutional Neural Networks,CNN)，全卷积网络(Fully Convolutional Networks，FCN)，残差网络(Residual Network，Res-Net)等。
本申请实施例不限定原始图像的图像数量，图像数量包括至少两个，且不限定原始图像的图像类型，可以为任意一种图像类型。其中，本申请实施例不对目标对象的对象类型进行限定，可以为任意一种对象类型，例如人、自行车、桌子、医学内镜对象等等，可以根据实际应用场景进行设定。此外，本申请实施例不限定目标对象的数量，例如目标对象为人，原始图像中可以没有目标对象、或至少一个目标对象均可；需要理解的是，目标对象可以包括一种类型或多种类型的对象，例如目标对象可以包括自行车，也可以包括自行车以及人。在一个实施例中，原始图像为医学图像，目标对象为医学检测目标，即医学图像中的目标。
为了便于理解，请一并参见图3，图3是本申请实施例提供的一种数据处理的场景示意图。如图3所示，业务服务器30a可以等同于图1中的业务服务器100，标注终端30f可以为图1中的标注终端集群中的任意一个标注终端。业务服务器30a可以包括图像数据库30b，图像数据库30b用于存储原始图像，以及与原始图像相关联的数据，包括但不限初始图像识别模型30c等。本申请实施例设定目标对象为人。
请再参见图3，业务服务器30a将图像数据库30b中的第一原始图像301b输入至初始图像识别模型30c，可以在初始图像识别模型30c中获取第一原始图像301b的图像特征30d；进一步，业务服务器30a可以根据图像特征30d确定目标对象(例如为人)的初始区域识别特征，以及目标对象的初始对象识别特征；根据初始区域识别特征生成针对目标对象的初始辅助标注区域，如图3中初始辅助标注图像30e中的标注区域，根据初始对象识别特征生成针对初始辅助标注区域的初始辅助对象标签。对象标签例如设定为人。业务服务器100可以显示图中的初始辅助标注图像30e，其携带针对第一原始图像301b的第一初始辅助标注结果301e，需要理解的是，本申请中的标注结果(包括初始辅助标注结果以及初始标准标注结果)包括针对目标对象的标注区域以及针对目标对象的对象标签。
进一步，业务服务器30a将携带第一初始辅助标注结果301e的初始辅助标注图像30e发送至标注终端30f，标注对象301f可以对第一初始辅助标注结果301e进行校正，例如通过标注终端30f所安装的标注应用软件查看原始图像301b以及初始辅助标注图像30f，标注对象301f可以首先确认原始图像 301b是否包含人，如果包含，可以查看第一初始辅助标注结果301e中的初始辅助标注区域，若认可该初始辅助标注区域，则标注终端30f可以将第一初始辅助标注结果301e确定为初始候选标注结果(因为目标对象只有一个，所以此处默认初始辅助对象标签等于目标对象的对象标签)；若标注对象301f不认可初始辅助标注区域，则以多边形的形式标注出目标对象的位置与形状，标注时要求标注对象301f尽可能地贴近目标对象的边缘，并将目标对象全部包含在区域中，被标注的区域可以称作感兴趣的区域(Region of Interest，ROI)；可选的，标注对象301f对初始辅助标注区域进行修改，得到初始候选标注结果，如图3所示，标注终端30f可以显示初始候选标注图像30g，该初始候选标注图像30g可以显示初始候选标注结果301g。
进一步，标注终端30f将携带初始候选标注结果301g的初始候选标注图像30g，返回至业务服务器100。本申请实施例不限定独立标注的标注对象的数量，可以为一个或多个，本步骤以一个标注对象(例如图3中的标注对象301f)为例示意第二初始标准标注结果的生成过程，多个标注对象分别独立标注的场景，请参见下文步骤S103中的描述，两者过程一致，区别仅在于处理的数据不同，故此处不进行赘述。
请再参见图3，业务服务器100获取到初始候选标注结果301g后，将其确定为第一原始图像301b的第一初始标准标注结果，并可以将第一初始标准标注结果以及第一原始图像301b关联存储于图像数据库30b中。
其中，图像数据库30b可以是业务服务器30a专门用于存储图像的数据库，上述图像数据库30b可视为电子化的文件柜——存储电子文件(本申请可以包括原始图像、初始辅助标注结果以及初始标准标注结果等)的处所，业务服务器30a可以对文件中的原始图像、初始辅助标注结果以及初始标准标注结果进行新增、查询、更新、删除等操作。所谓“数据库”是以一定方式储存在一起、能与多个用户共享、具有尽可能小的冗余度、与应用程序彼此独立的数据集合。
图3是以生成第一原始图像301b的第一初始辅助标注结果301e以及第一初始标准标注结果示例描述，可以理解的是，生成剩余原始图像(第二原始图像)的初始辅助标注结果以及初始标准标注结果的过程，与上文描述的过程一致，区别仅在于处理的图像不同，故此处不进行一一赘述。
步骤S102，根据第一初始标准标注结果以及第一初始辅助标注结果，对初始图像识别模型中的模型参数进行调整，生成更新图像识别模型。
本申请实施例可应用于云技术、人工智能、智慧交通、辅助驾驶等各种场景。近年来，随着以深度学习为代表的新一代人工智能技术的突破，医学图像的自动识别领域取得了革命性的进展，面向医学影像的人工智能实时辅助病变的检出与分类有望助力临床医生提高检查质量和减少病变漏诊。
卓越的图像识别模型依赖于海量具有代表性的高质量标注数据，数据标注质量决定了算法模型的稳定性和准确率。但各类不同模态数据以及不同疾病病灶存在明显的个体差异性与复杂性，故需要不断更新已有的图像识别模型，同时，对已标注数据也进行更新，基于此，本申请实施例提出了一种基于双向质控的人工智能辅助标注方法，该方法旨在提高标注的准确度与效率。
请一并参见图4，图4是本申请实施例提供的一种数据处理的场景示意图。如图4所示，初始辅助标注图像30e包括第一初始辅助标注结果301e，第一初始辅助标注结果301e包括针对目标对象的第一标注区域(等同于图4中的初始辅助标注区域401a)，以及针对第一标注区域的第一对象标签(等同于图4中的初始辅助对象标签401b)；初始标准标注图像40c包括第一初始标准标注结果401c，该第一初始标准标注结果401c包括针对目标对象的第二标注区域(等同于图4中的初始标准标注区域402a)，以及针对第二标注区域的第二对象标签(等同于图4中的初始标准对象标签402b)。
业务服务器确定初始辅助标注区域401a以及初始标准标注区域402a之间的初始区域误差，确定初始辅助对象标签401b以及初始标准对象标签402b之间的初始对象误差。进一步，对初始区域误差以及初始对象误差进行加权求和，得到第一标注结果误差。业务服务器根据第一标注结果误差，对初始图像识别模型30c中的模型参数进行调整，生成更新图像识别模型40d。
本申请实施例不限定初始图像识别模型30c的更新条件，可以为业务服务器响应针对初始图像识别模型30c的模型更新指令，该场景请参见下文图6所对应的实施例中步骤S202的描述，此处暂不展开叙述；初始图像识别模型30c的更新条件还可以为，上文步骤S101中的初始辅助标注结果以及初始 标准标注结果之间的结果误差达到初始损失值阈值，该场景请参见下文图7所对应的实施例中步骤S302-步骤S306的描述，此处暂不展开叙述。
综上，本申请实施例可以依据标注对象的需求，确定第一初始标准标注结果以及第一初始辅助标注结果，故可以对初始图像识别模型进行个性化更新。换言之，本申请实施例可以根据个性化需求(例如为医生对医学影像的特定识别)，对模型进行特定训练，以提高在个性化场景中的目标识别的准确性。例如，目标对象包括多个对象类型，多个对象类型可以包括第一目标对象(例如恶性肿瘤)以及第二目标对象(例如良性肿瘤)，且初始图像识别模型针对第一目标对象的预测精度，低于针对第二目标对象的预测精度，故可以将包括第一目标对象的初始标准标注结果作为第一初始标准标注结果，将包括第一目标对象的初始辅助标注结果作为第一初始辅助标注结果，此时，根据上述的第一初始标准标注结果以及第一初始辅助标注结果，对初始图像识别模型中的模型参数进行调整，从而生成针对第一目标对象的更新图像识别模型，以提高对第一目标对象的预测精度。由此可见，在医学场景中，本申请的实施例可以提高医院检测的准确性。需要注意的是，更新图像识别模型不更改针对第二目标对象的预测精度。
步骤S103，基于更新图像识别模型预测第二原始图像的更新辅助标注结果，获取第二原始图像的更新标准标注结果；更新标准标注结果是基于更新辅助标注结果对第二初始标准标注结果进行调整所得到的。
具体的，将更新辅助标注结果发送至至少两个标注终端。这样，至少两个标注终端，分别根据更新辅助标注结果对第二初始标准标注结果进行调整，得到第二原始图像的候选标注结果；获取至少两个标注终端所返回的候选标注结果；至少两个候选标注结果分别包括用于标注第二原始图像中的目标对象的候选标注区域；确定至少两个候选标注结果所分别包括的候选标注区域对应的区域数量；根据至少两个区域数量，确定至少两个候选标注结果的初始审核标注结果；根据初始审核标注结果获取更新标准标注结果。
其中，根据至少两个区域数量，确定至少两个候选标注结果的初始审核标注结果的具体过程可以包括：对至少两个区域数量进行对比；至少两个区域数量包括区域数量B a；a为正整数，且a小于或等于至少两个候选标注结果的结果数量；若剩余区域数量中存在与区域数量B a不相同的区域数量，则将至少两个候选标注结果分别确定为初始审核标注结果；剩余区域数量包括至少两个区域数量中除了区域数量B a之外的区域数量；若剩余区域数量均与区域数量B a相同，则获取至少两个候选标注结果中，每两个候选标注结果所分别包括的候选标注区域；确定每两个候选标注结果所分别包括的候选标注区域之间的重合度，根据重合度确定初始审核标注结果。在一个实施例中，两个候选标注区域的重合度例如是两个候选标注区域的位置信息之间的重合度，两个位置信息之间的重合度例如为两个位置信息的交集与并集的比值。这里候选标注区域的位置信息可以表征该候选标注区域在其所处图像中的位置。
其中，至少两个候选标注结果还分别包括用于标注所包括的候选标注区域的候选对象标签；根据重合度确定初始审核标注结果的具体过程可以包括：若至少一个重合度小于重合度阈值，则将至少两个候选标注结果分别确定为初始审核标注结果；若各个重合度等于或大于重合度阈值，则将至少两个候选标注结果中相同的候选对象标签划分在同一个对象标签组中，得到n个对象标签组；n为正整数；根据n个对象标签组确定初始审核标注结果。
其中，根据n个对象标签组确定初始审核标注结果的具体过程可以包括：统计n个对象标签组分别包括的候选对象标签的对象标签数量，在n个对象标签组分别对应的对象标签数量中获取最大对象标签数量；确定最大对象标签数量与至少两个候选标注结果对应的对象标签数量之间的数量比例；将数量比例与数量比例阈值进行对比，若数量比例小于数量比例阈值，则将至少两个候选标注结果分别确定为初始审核标注结果；若数量比例等于或大于数量比例阈值，则将最大对象标签数量所对应的对象标签组确定为目标对象标签组；从与目标对象标签组相关联的候选标注结果中获取目标候选标注结果，将目标候选标注结果确定为初始审核标注结果。
其中，根据初始审核标注结果获取更新标准标注结果的具体过程可以包括：若初始审核标注结果为至少两个候选标注结果，则将初始审核标注结果发送至第一审核终端，以使第一审核终端根据至少 两个候选标注结果确定发送至第二审核终端的审核标注结果；第二审核终端用于根据审核标注结果返回更新标准标注结果；若初始审核标注结果为目标候选标注结果，则将初始审核标注结果发送至第二审核终端，以使第二审核终端根据目标候选标注结果返回更新标准标注结果。
其中，基于更新图像识别模型预测第二原始图像的更新辅助标注结果的过程描述，请参见上文步骤S101中基于初始图像识别模型预测原始图像的初始辅助标注结果的过程描述，两者数据处理过程一致，区别仅在于更新图像识别模型是初始图像识别模型更新之后的模型，故此处不进行赘述。
业务服务器获取更新标准标注结果的过程与获取初始标准标注结果的过程基本一致，故此处不赘述一个标注终端基于更新辅助标注结果对第二初始标准标注结果进行调整，得到更新标准标注结果的过程，请参见上文步骤S101中描述。
可选的，为保证数据标注质量，降低标注对象间个体差异性，标注过程可以是多个标注对象独立标注，故业务服务器可以将更新辅助标注结果发送至至少两个标注对象对应的标注终端，以使至少两个标注对象对应的标注终端，分别根据更新辅助标注结果对第二初始标准标注结果进行调整，得到第二原始图像的候选标注结果。
参见图5，图5是本申请实施例提供的一种图像处理的场景示意图。如图5所示，本申请实施例设定更新候选标注图像的图像数量为3，即图5中的更新候选标注图像501a、更新候选标注图像502a以及更新候选标注图像503a，当至少两张更新候选标注图像的图像数量等于2或者其他数量时，可以参照本实施例。如图5所示，第二原始图像501d可以包括房屋、行人、扶梯以及大厦等物体，本实施例设定目标对象包括行人以及房屋。业务服务器502d获取3个标注对象所分别提供的更新候选标注图像501a、更新候选标注图像502a以及更新候选标注图像503a，上述3张更新候选标注图像均基于更新辅助标注结果以及第二初始标准标注结果生成，例如更新候选标注图像501a是由标注对象101A根据更新辅助标注结果对第二初始标准标注结果进行调整所得的图像，更新候选标注图像502a是由标注对象102A根据更新辅助标注结果对第二初始标准标注结果进行调整所得的图像。
如图5所示，更新候选标注图像501a对应的候选标注结果包括标注房屋的候选标注结果501c，以及标注行人的候选标注结果501b，故更新候选标注图像501a对应的候选标注结果包括2个候选标注区域；更新候选标注图像502a对应的候选标注结果包括标注房屋的候选标注结果502c，以及标注行人的候选标注结果502b，故更新候选标注图像502a对应的候选标注结果包括2个候选标注区域；更新候选标注图像503a对应的候选标注结果包括标注房屋的候选标注结果503c，以及标注行人的候选标注结果503b，故更新候选标注图像503a对应的候选标注结果包括2个候选标注区域。
请再参见图5，业务服务器502d确定3个候选标注结果(即3张更新候选标注图像)所分别包括的候选标注区域对应的区域数量。明显地，在图5中，3个区域数量相同，均为2，此时业务服务器502d需要确定每张更新候选标注图像中的每个候选标注区域，与其他更新候选区域图像中的候选标注区域之间的重合度，然后根据重合度确定初始审核标注结果。
可以理解的是，图5中的3张更新候选标注图像除了分别包括的候选标注结果存在差异，其他并无差异(因为3张更新候选标注图像均基于第二原始图像501d生成)，故以上述3张图像(即更新候选标注图像501a、更新候选标注图像502a以及更新候选标注图像503a)的左上角为坐标原点，原点往右为x轴，原点往下为y轴所分别生成的坐标是一致的，因此，3张图像中的目标对象分别对应的位置信息是一致的。为了便于叙述，以确定更新候选标注图像501a中的候选标注区域以及更新候选标注图像502a中的候选标注区域之间的重合度示例叙述，其他图像所分别包括的候选标注区域之间的重合度，可以参见下文理解。
根据上述的坐标，业务服务器502d获取更新候选标注图像501a中候选标注结果501c的位置信息L 501c，以及候选标注结果501b的位置信息L 501b；获取更新候选标注图像502a中候选标注结果502c的位置信息L 502c，以及候选标注结果502b的位置信息L 502b；业务服务器502d确定候选标注结果501c的位置信息L 501c以及候选标注结果502c的位置信息L 502c的交集位置信息L 501c∩502c，确定位置信息L 501c以及位置信息L 502c的并集位置信息L 501c∪502c；业务服务器502d确定候选标注结果501b的位置信息L 501b以及候选标注结果502c的位置信息L 502c的交集位置信息L 501b∩502c，确定位置信息L 501b以及位置信息L 502c的并集位置信息L 501b∪502c；业务服务器502d确定候选标注结果501c的位置信息L 501c以及 候选标注结果502b的位置信息L 502b的交集位置信息L 501c∩502b，确定位置信息L 501c以及位置信息L 502b的并集位置信息L 501c∪502b；业务服务器502d确定候选标注结果501c的位置信息L 501b以及候选标注结果502b的位置信息L 502b的交集位置信息L 501b∩502b，确定位置信息L 501b以及位置信息L 502b的并集位置信息L 501b∪502b。
下面以确定更新候选标注图像501a中候选标注结果501c的第一重合度(等同于候选标注结果501c所包括的候选标注区域的重合度)为例叙述，确定更新候选标注图像501a中候选标注结果501b的第一重合度可以参见下面的过程。
业务服务器502d可以根据公式(1)确定候选标注结果501c以及候选标注结果502c之间的候选重合度C (501c,502c)。
Figure PCTCN2022137442-appb-000001
其中，ROI 501c可以表示候选标注结果501c的候选标注区域，且可以由位置信息L 501c确定，ROI 502c可以表示候选标注结果502c的候选标注区域，且可以由位置信息L 502c确定，ROI 501c∩ROI 502c可以表示候选标注结果501c的候选标注区域以及候选标注结果502c的候选标注区域的交集区域，且可以由交集位置信息L 501c∩502c确定，ROI 501c∪ROI 502c可以表示候选标注结果501c的候选标注区域以及候选标注结果502c的候选标注区域的并集区域，且可以由并集位置信息L 501c∪502c确定。
业务服务器502d可以根据公式(2)确定候选标注结果501c以及候选标注结果502b之间的候选重合度C (501c,502b)。
Figure PCTCN2022137442-appb-000002
其中，ROI 502b可以表示候选标注结果502b的候选标注区域，且可以由位置信息L 502b确定，ROI 501c∩ROI 502b可以表示候选标注结果501c的候选标注区域以及候选标注结果502b的候选标注区域的交集区域，且可以由交集位置信息L 501c∩502b确定，ROI 501c∪ROI 502b可以表示候选标注结果501c的候选标注区域以及候选标注结果502b的候选标注区域的并集区域，且可以由并集位置信息L 501c∪502b确定。
业务服务器502d将候选重合度C (501c,502c)以及候选重合度C (501c,502b)进行对比，针对更新候选标注图像501a以及更新候选标注图像502a，显然候选标注结果501c与候选标注结果502b无交集区域，故候选标注结果501c的第一重合度为候选重合度C (501c,502c)。
下面以确定更新候选标注图像502a中候选标注结果502b的第二重合度为例叙述，确定更新候选标注图像502a中候选标注结果502c的第二重合度可以参见下面的过程。
业务服务器502d可以根据公式(3)确定候选标注结果502b以及候选标注结果501b之间的候选重合度C (501b,502b)。
Figure PCTCN2022137442-appb-000003
其中，ROI 501b可以表示候选标注结果501c的候选标注区域，且可以由位置信息L 501b确定，ROI 501b∩ROI 502b可以表示候选标注结果501b的候选标注区域以及候选标注结果502b的候选标注区域 的交集区域，且可以由交集位置信息L 501b∩502b确定，ROI 501b∪ROI 502b可以表示候选标注结果501b的候选标注区域以及候选标注结果502b的候选标注区域的并集区域，且可以由并集位置信息L 501b∪502b确定。
业务服务器502d将候选重合度C (501b,502b)以及候选重合度C (501c,502b)进行对比，针对更新候选标注图像501a以及更新候选标注图像502a，显然候选标注结果501c与候选标注结果502b无交集区域，故候选标注结果502b的第二重合度为候选重合度C (501b,502b)。
业务服务器502d将更新候选标注图像501a中的每个候选标注区域(包括候选标注结果501c以及候选标注结果501b)的第一重合度，和更新候选标注图像502a中的每个候选标注区域(包括候选标注结果502c以及候选标注结果502b)的第二重合度，确定为更新候选标注图像501a以及更新候选标注图像502a所分别包括的候选标注区域之间的重合度。
请再参见图5，根据更新候选标注图像501a以及更新候选标注图像502a所分别包括的候选标注区域之间的重合区域，业务服务器502d可以显示重合区域图像50e，其中，候选标注结果501c与候选标注结果502c之间的黑色区域为该两者的重合区域，候选标注结果501b与候选标注结果502b之间的黑色区域为该两者的重合区域。
业务服务器502d将上述重合度与重合度阈值进行对比，若至少一个重合度小于重合度阈值，则将至少两个候选标注结果(即3张更新候选标注图像所分别包括的候选标注结果)分别确定为初始审核标注结果；若上述各个重合度均大于或等于重合度阈值，则从更新候选标注图像501a所包括的候选标注结果中获取候选对象标签(包括候选标注结果501c以及候选标注结果501b分别包括的候选对象标签)，从更新候选标注图像502a所包括的候选标注结果中获取候选对象标签(包括候选标注结果502c以及候选标注结果502b分别包括的候选对象标签)，从更新候选标注图像503a所包括的候选标注结果中获取候选对象标签(包括候选标注结果503c以及候选标注结果503b分别包括的候选对象标签)，业务服务器502d将3张更新候选标注图像分别包括的候选对象标签中，相同的候选对象标签划分在同一个对象标签组中，得到n个对象标签组，统计n个对象标签组分别包括的候选对象标签的对象标签数量，在n个对象标签组分别对应的对象标签数量中获取最大对象标签数量；确定最大对象标签数量与至少两个候选标注结果对应的对象标签数量之间的数量比例；将数量比例与数量比例阈值进行对比，若数量比例小于数量比例阈值，则将3张更新候选标注图像分别对应的候选标注结果，均确定为初始审核标注结果；若数量比例等于或大于数量比例阈值，则将最大对象标签数量所对应的对象标签组确定为目标对象标签组；从与目标对象标签组相关联的候选标注结果中获取目标候选标注结果，将目标候选标注结果确定为初始审核标注结果。
业务服务器确定初始审核标注结果后，需要将初始审核标注结果发送至审核终端(包括第一审核终端以及第二审核终端)，以使审核终端对该结果进行确认，并返回更新标准标注结果，若初始审核标注结果为至少两个候选标注结果，则业务服务器将初始审核标注结果发送至第一审核终端(等同于上文图1中所述的第一审核终端200a)，上述第一审核终端在整个数据处理过程中具备仲裁的功能。
第一审核终端获取到至少两个候选标注结果后，其对应的仲裁对象可以查看第二原始图像以及至少两个候选标注结果，若仲裁对象确认至少两个候选标注结果均不理想，则可以对第二原始图像进行区域标注以及对象标注，仲裁对象对第二原始图像进行标注的过程，与标注对象对第一原始图像进行标注的过程一致，故请参见上文步骤S101中所描述的标注内容。后续，仲裁对象可以通过第一审核终端将自身重新标注的审核标注结果作为仲裁结果，发送至第二审核终端(等同于上文图1中所述的第二审核终端200b)，以使第二审核终端对应的审核对象对仲裁结果进行审核。
若仲裁对象认可至少两个候选标注结果中的某一个候选标注结果，则可以直接将认可的候选标注结果作为仲裁结果发送至第二审核终端，以使审核对象对仲裁结果进行审核。
若初始审核标注结果为目标候选标注结果，则将初始审核标注结果发送至第二审核终端，上述第二审核终端在整个图像处理过程中具备审核的功能。当第二审核终端获取到目标候选标注结果后，审核对象可以通过第二审核终端审核该图像。
若审核对象认可目标候选标注结果或第一审核终端发送的仲裁结果，则可以将其保存在与业务服务器相关联的图像数据库(等同于图3中的图像数据库30b)中。若审核对象不认可目标候选标注结果或第一审核终端发送的仲裁结果，则可以丢弃现有的标注数据，让其他的标注对象对第二原始图像进行区域标注以及对象标注，或者，将第二原始图像重新转发至第一审核终端，以使仲裁对象对第二原始图像进行标注。后续，审核对象对重新生成的审核标注结果进行审核处理，该审核过程与上述审核过程一致，故不再进行赘述。
综上所述，本步骤可以通过更新后的初始图像识别模型(即更新图像识别模型)，对已有标注结果的第二原始图像进行质控，让已有标注结果实现动态更新，从而提高目标识别准确性。
步骤S104，当根据更新辅助标注结果以及更新标准标注结果确定更新图像识别模型满足模型收敛条件时，将更新图像识别模型确定为目标图像识别模型；目标图像识别模型用于生成目标图像的标注结果。
具体的，第二原始图像包括目标对象；更新辅助标注结果包括针对目标对象的更新辅助标注区域，以及针对更新辅助标注区域的更新辅助对象标签；更新标准标注结果包括针对目标对象的更新标准标注区域，以及针对更新标准标注区域的更新标准对象标签；确定更新辅助标注区域以及更新标准标注区域之间的更新区域损失值；确定更新辅助对象标签以及更新标准对象标签之间的更新对象损失值；对更新区域损失值以及更新对象损失值进行加权求和，得到更新图像识别模型的更新损失值；当更新损失值大于或等于更新损失值阈值时，确定更新图像识别模型不满足模型收敛条件，继续对更新图像识别模型中的模型参数进行调整；当更新损失值小于更新损失值阈值时，确定更新图像识别模型满足模型收敛条件，将更新图像识别模型确定为目标图像识别模型。
其中，原始图像还包括第三原始图像；初始标准标注结果还包括第三原始图像的第三初始标准标注结果；初始辅助标注结果还包括第三原始图像的第三初始辅助标注结果；继续对更新图像识别模型中的模型参数进行调整的具体过程可以包括：根据第三初始标准标注结果以及第三初始辅助标注结果，确定调整损失值；对调整损失值以及更新损失值进行加权求和，得到目标损失值；根据目标损失值对更新图像识别模型中的模型参数进行调整。
本申请实施例不对第一原始图像、第二原始图像以及第三原始图像分别对应的图像数量进行限定，可以为任意数量，应当根据实际应用场景进行设定。可以理解的是，第一原始图像以及第二原始图像互不相同，第二原始图像以及第三原始图像互不相同。可选的，若更新损失值小于更新损失值阈值，但标注对象通过标注终端发送模型继续更新指令，则业务服务器可以保持对更新图像识别模型的更新处理，其过程与更新损失值等于或大于更新损失值阈值的后续过程一致，故此处不进行赘述。
业务服务器可以根据更新损失值确定第三原始图像，具体确定过程可以如下：目标对象可以包括至少两个目标对象，至少两个目标对象可以包括第一目标对象；可以理解的是，更新损失值可以由针对第一目标对象的第一更新损失值以及针对剩余目标对象的剩余更新损失值的平均值得到，其中，剩余目标对象包括至少两个目标对象中除了第一目标对象之外的目标对象；故业务服务器可以确定第一更新损失值与更新损失值之间的第一损失值比例，并依据该第一损失值比例以及训练样本数量(等于第三原始图像的图像数量)，从原始图像中获取包括该第一目标对象的原始图像，以及包括剩余目标对象的原始图像，将上述两种原始图像确定为第三原始图像。例如训练样本数量等于200，第一损失值比例为0.8，则业务服务器可以随机从原始图像中抽取160张包括第一目标对象的图像，同理，随机从原始图像中抽取剩余的图像，将抽取的包括第一目标对象的图像以及剩余的图像确定为第三原始图像。
在本申请实施例中，计算机设备可以将第一初始标准标注结果以及第一初始辅助标注结果作为训练样本集，对初始图像识别模型进行更新，即对模型参数进行调整，得到更新图像识别模型，可以理解的是，该过程不仅可以实现模型更新，还可以根据训练样本集确定模型更新的方向；进一步，基于更新图像识别模型预测第二原始图像的更新辅助标注结果，获取基于更新辅助标注结果，对第二初始标准标注结果进行调整所得到的更新标准标注结果，该过程可以实现第二初始标准标注结果的更新；进一步，当将更新图像识别模型确定为目标图像识别模型时，利用目标图像识别模型生成目标图像的目标辅助标注结果。上述可知，本申请实施例不仅可以依据训练样本集对初始图像识别模型进行更新， 以提高更新图像识别模型的识别能力；还可以通过更新图像识别模型对第二初始标准标注结果进行更新，以提高更新标准标注结果的精度，故采用本申请可以实现图像识别模型以及标注结果的双向更新。
请参见图6，图6是本申请实施例提供的一种数据处理方法的流程示意图。该方法可以由业务服务器(例如，上述图1所示的业务服务器100)执行，也可以由标注终端(例如，上述图1所示的标注终端100a)执行，还可以由业务服务器和标注终端交互执行。如图6所示，该方法至少可以包括以下步骤。
步骤S201，基于初始图像识别模型预测原始图像的初始辅助标注结果，获取对初始辅助标注结果进行校正所确定的初始标准标注结果；原始图像包括第一原始图像以及第二原始图像；初始标准标注结果包括第一原始图像的第一初始标准标注结果，以及第二原始图像的第二初始标准标注结果；初始辅助标注结果包括第一原始图像的第一初始辅助标注结果。
其中，步骤S201的具体实现过程，请参见上文图2所对应的实施例中的步骤S101，此处不进行赘述。
步骤S202，响应模型更新指令，将第一原始图像确定为样本图像，将第一初始标准标注结果确定为样本图像的样本标签，将第一初始辅助标注结果确定为样本图像的样本预测结果。
步骤S203，根据样本标签以及样本预测结果，确定初始图像识别模型的总损失值。
步骤S204，根据总损失值，对初始图像识别模型中的模型参数进行调整，当调整后的初始图像识别模型满足模型收敛条件时，将调整后的初始图像识别模型确定为更新图像识别模型。
结合步骤S202至步骤S204叙述，业务服务器当前使用初始图像识别模型对图像数据库中的原始图像进行预测，并生成原始图像对应的初始辅助标注结果，获取基于初始辅助标注结果所确定的初始标准标注结果，本申请实施例暂不对初始标准标注结果以及初始辅助标注结果之间的平均标注结果误差的确定过程，展开描述，请参见下文图7所对应的实施例中的步骤S302至步骤S304的描述。
此时，基于初始标准标注结果以及初始辅助标注结果之间的平均标注结果误差所生成的初始损失值小于初始损失值阈值，当获取到模型更新指令时，业务服务器响应该模型更新指令；可选的，该模型更新指令携带训练样本信息，该训练样本信息可以包括至少两个对象标签，以及至少两个对象标签分别对应的训练样本数量，例如至少两个对象标签包括第一对象标签，以及第二对象标签，模型更新指令携带针对第一对象标签的第一训练样本数量，以及针对第二对象标签的第二训练样本数量，则业务服务器可以从初始标准标注结果中，获取标注结果数量等于第一训练样本数量且包括第一对象标签的初始标准标注结果，将获取的初始标准标注结果确定为第一初始标准标注结果；业务服务器从初始辅助标注结果中获取与第一初始标准标注结果相对应的初始辅助标注结果，作为第一初始辅助标注结果；进一步，业务服务器将将第一初始标准标注结果确定为样本图像的样本标签，将第一初始辅助标注结果确定为样本图像的样本预测结果，确定样本标签以及样本预测结果之间的误差，将该误差确定为初始图像识别模型的总损失值，利用该总损失值对初始图像识别模型中的模型参数进行调整，当调整后的初始图像识别模型满足模型收敛条件时，将调整后的初始图像识别模型确定为更新图像识别模型。
上述可知，本申请实施例不仅可以对初始图像识别模型进行更新，还可以由业务对象确定更新方向，故可以提高更新效率，也可以提高模型的预测准确度。
步骤S205，基于更新图像识别模型预测第二原始图像的更新辅助标注结果，获取更新标准标注结果；更新标准标注结果是基于更新辅助标注结果对第二初始标准标注结果进行调整所得到的。
步骤S206，当根据更新辅助标注结果以及更新标准标注结果确定更新图像识别模型满足模型收敛条件时，将更新图像识别模型确定为目标图像识别模型；目标图像识别模型用于生成目标图像的目标辅助标注结果。
步骤S205-步骤S206的具体实现过程，请参见上文图2所对应的实施例中的步骤S103-步骤S104，此处不进行赘述。
在本申请实施例中，计算机设备可以将第一初始标准标注结果以及第一初始辅助标注结果作为训练样本集，对初始图像识别模型进行更新，即对模型参数进行调整，得到更新图像识别模型，可以理解的是，该过程不仅可以实现模型更新，还可以根据训练样本集确定模型更新的方向；进一步，基于 更新图像识别模型预测第二原始图像的更新辅助标注结果，获取基于更新辅助标注结果，对第二初始标准标注结果进行调整所得到的更新标准标注结果，该过程可以实现第二初始标准标注结果的更新；进一步，当将更新图像识别模型确定为目标图像识别模型时，利用目标图像识别模型生成目标图像的目标辅助标注结果。上述可知，本申请实施例不仅可以依据训练样本集对初始图像识别模型进行更新，以提高更新图像识别模型的识别能力；还可以通过更新图像识别模型对第二初始标准标注结果进行更新，以提高更新标准标注结果的精度，故采用本申请可以实现图像识别模型以及标注结果的双向更新。
请参见图7，图7是本申请实施例提供的一种数据处理方法的流程示意图。该方法可以由业务服务器(例如，上述图1所示的业务服务器100)执行，也可以由标注终端(例如，上述图1所示的标注终端100a)执行，还可以由业务服务器和标注终端交互执行。如图7所示，该方法至少可以包括以下步骤。
步骤S301，基于初始图像识别模型预测原始图像的初始辅助标注结果，获取基于初始辅助标注结果所确定的初始标准标注结果；原始图像包括第一原始图像以及第二原始图像；初始标准标注结果包括第一原始图像的第一初始标准标注结果，以及第二原始图像的第二初始标准标注结果；初始辅助标注结果包括第一原始图像的第一初始辅助标注结果。
其中，步骤S301的具体实现过程，请参见上文图2所对应的实施例中的步骤S101，此处不进行赘述。
步骤S302，确定第一初始辅助标注结果以及第一初始标准标注结果之间的第一标注结果误差。
步骤S303，确定第二初始辅助标注结果以及第二初始标准标注结果之间的第二标注结果误差。
步骤S304，确定第一标注结果误差以及第二标注结果误差之间的平均标注结果误差
步骤S305，根据平均标注结果误差确定初始图像识别模型的初始损失值。
步骤S306，若初始损失值大于或等于初始损失值阈值，则根据第一初始标准标注结果以及第一初始辅助标注结果，对初始图像识别模型中的模型参数进行调整，生成更新图像识别模型。
步骤S307，基于更新图像识别模型预测第二原始图像的更新辅助标注结果，获取更新标准标注结果；更新标准标注结果是基于更新辅助标注结果对第二初始标准标注结果进行调整所得到的。
步骤S308，当根据更新辅助标注结果以及更新标准标注结果确定更新图像识别模型满足模型收敛条件时，将更新图像识别模型确定为目标图像识别模型；目标图像识别模型用于生成目标图像的目标辅助标注结果。
步骤S307-步骤S308的具体实现过程，请参见上文图2所对应的实施例中的步骤S103-步骤S104，此处不进行赘述。
结合图2、图6以及图7，请一并参见图8，图8是本申请实施例提供的一种数据处理方法的流程示意图。如图8所示，业务服务器将原始图像输入至人工智能辅助标注模型(等同于上述的初始图像识别模型)，得到原始图像的初始辅助标注结果；业务服务器将初始辅助标注结果发送至标注对象对应的标注终端，以使标注对象通过标注终端查看原始图像以及初始辅助标注结果，并基于初始辅助标注结果确定初始候选标注结果；业务服务器获取标注终端返回的初始候选标注结果，基于初始候选标注结果得到初始标准标注结果；统计初始辅助标注结果以及初始标准标注结果之间的结果误差，人工是否启动模型更新，若启动模型更新，则业务服务器基于第一初始标准标注结果以及第一初始辅助标注结果，对初始图像识别模型进行模型更新，若不启动模型更新，则检测辅助标注效果是否达标，具体参见上文图6中的描述；若辅助标注效果，则继续运行人工智能辅助标注模型；若辅助标注效果不达标，则基于第一初始标准标注结果以及第一初始辅助标注结果，对初始图像识别模型进行模型更新，得到更新的人工智能辅助标注模型(等同于上述的更新图像识别模型)；业务服务器通过更新图像识别模型，对已标注的第二原始图像重新预测，得到更新辅助标注结果；将更新辅助标注结果发送至标注对象对应的标注终端，以使标注对象通过标注终端查看更新辅助标注结果，并基于更新辅助标注结果，对第二初始标准标注结果进行更改或确认，得到候选标注结果；业务服务器获取标注终端返回的候选标注结果，基于候选标注结果得到更新标准标注结果；业务服务器统计更新辅助标注结果以及更新标准标注结果之间的结果误差，基于结果误差，将更新图像识别模型确定为目标图像识别模型。
在本申请实施例中，计算机设备可以将第一初始标准标注结果以及第一初始辅助标注结果作为训 练样本集，对初始图像识别模型进行更新，即对模型参数进行调整，得到更新图像识别模型，可以理解的是，该过程不仅可以实现模型更新，还可以根据训练样本集确定模型更新的方向；进一步，基于更新图像识别模型预测第二原始图像的更新辅助标注结果，获取基于更新辅助标注结果，对第二初始标准标注结果进行调整所得到的更新标准标注结果，该过程可以实现第二初始标准标注结果的更新；进一步，当将更新图像识别模型确定为目标图像识别模型时，利用目标图像识别模型生成目标图像的目标辅助标注结果。上述可知，本申请实施例不仅可以依据训练样本集对初始图像识别模型进行更新，以提高更新图像识别模型的识别能力；还可以通过更新图像识别模型对第二初始标准标注结果进行更新，以提高更新标准标注结果的精度，故采用本申请可以实现图像识别模型以及标注结果的双向更新。
进一步地，请参见图9，图9是本申请实施例提供的一种数据处理装置的结构示意图。上述数据处理装置可以是运行于计算机设备中的一个计算机程序(包括程序代码)，例如该数据处理装置为一个应用软件；该装置可以用于执行本申请实施例提供的方法中的相应步骤。如图9所示，该数据处理装置1可以包括：第一获取模块11、更新模型模块12、第二获取模块13以及第一确定模块14。
第一获取模块11，用于基于初始图像识别模型预测原始图像的初始辅助标注结果，获取对初始辅助标注结果进行校正所确定的初始标准标注结果；原始图像包括第一原始图像以及第二原始图像；初始标准标注结果包括第一原始图像的第一初始标准标注结果，以及第二原始图像的第二初始标准标注结果；初始辅助标注结果包括第一原始图像的第一初始辅助标注结果；
更新模型模块12，用于根据第一初始标准标注结果以及第一初始辅助标注结果，对初始图像识别模型中的模型参数进行调整，生成更新图像识别模型；
第二获取模块13，用于基于更新图像识别模型预测第二原始图像的更新辅助标注结果，获取第二原始图像的更新标准标注结果；更新标准标注结果是基于更新辅助标注结果对第二初始标准标注结果进行调整所得到的；
第一确定模块14，用于当根据更新辅助标注结果以及更新标准标注结果确定更新图像识别模型满足模型收敛条件时，将更新图像识别模型确定为目标图像识别模型；目标图像识别模型用于生成目标图像的标注结果。
其中，第一获取模块11、更新模型模块12、第二获取模块13以及第一确定模块14的具体功能实现方式可以参见上述图2对应实施例中的步骤S101-步骤S104，这里不再进行赘述。
再请参见图9，数据处理装置1还可以包括：第二确定模块15。
第二确定模块15，用于响应模型更新指令，将第一原始图像确定为样本图像，将第一初始标准标注结果确定为样本图像的样本标签，将第一初始辅助标注结果确定为样本图像的样本预测结果；
则更新模型模块12，包括：第一确定单元121以及第二确定单元122。
第一确定单元121，用于根据样本标签以及样本预测结果，确定初始图像识别模型的总损失值；
第二确定单元122，用于根据总损失值，对初始图像识别模型中的模型参数进行调整，当调整后的初始图像识别模型满足模型收敛条件时，将调整后的初始图像识别模型确定为更新图像识别模型。
其中，第二确定模块15、第一确定单元121以及第二确定单元122的具体功能实现方式可以参见上述图6对应实施例中的步骤S202-步骤S204，这里不再进行赘述。
再请参见图9，初始辅助标注结果还包括第二原始图像的第二初始辅助标注结果；
数据处理装置1还可以包括：第三确定模块16以及执行步骤模块17。
第三确定模块16，用于确定第一初始辅助标注结果以及第一初始标准标注结果之间的第一标注结果误差；
第三确定模块16，还用于确定第二初始辅助标注结果以及第二初始标准标注结果之间的第二标注结果误差；
第三确定模块16，还用于确定第一标注结果误差以及第二标注结果误差之间的平均标注结果误差；
第三确定模块16，还用于根据平均标注结果误差确定初始图像识别模型的初始损失值；
执行步骤模块17，用于若初始损失值大于或等于初始损失值阈值，则执行根据第一初始标准标注结果以及第一初始辅助标注结果，对初始图像识别模型中的模型参数进行调整，生成更新图像识别模型的步骤。
其中，第三确定模块16以及执行步骤模块17的具体功能实现方式可以参见上述图7对应实施例中的步骤S302-步骤S306，这里不再进行赘述。
再请参见图9，第一原始图像包括目标对象；第一初始辅助标注结果包括针对目标对象的第一标注区域，以及针对第一标注区域的第一对象标签；第一初始标准标注结果包括针对目标对象的第二标注区域，以及针对第二标注区域的第二对象标签；
第三确定模块16可以包括：第三确定单元161以及第一加权单元162。
第三确定单元161，用于确定第一标注区域以及第二标注区域之间的初始区域误差；
第三确定单元161，还用于确定第一对象标签以及第二对象标签之间的初始对象误差；
第一加权单元162，用于对初始区域误差以及初始对象误差进行加权求和，得到第一标注结果误差。
其中，第三确定单元161以及第一加权单元162的具体功能实现方式可以参见上述图7对应实施例中的步骤S302，这里不再进行赘述。
再请参见图9，第二原始图像包括目标对象；更新辅助标注结果包括针对目标对象的更新辅助标注区域，以及针对更新辅助标注区域的更新辅助对象标签；更新标准标注结果包括针对目标对象的更新标准标注区域，以及针对更新标准标注区域的更新标准对象标签；
第一确定模块14可以包括：第四确定单元141、第二加权单元142、第五确定单元143以及第六确定单元144。
第四确定单元141，用于确定更新辅助标注区域以及更新标准标注区域之间的更新区域损失值；
第四确定单元141，还用于确定更新辅助对象标签以及更新标准对象标签之间的更新对象损失值；
第二加权单元142，用于对更新区域损失值以及更新对象损失值进行加权求和，得到更新图像识别模型的更新损失值；
第五确定单元143，用于当更新损失值大于或等于更新损失值阈值时，确定更新图像识别模型不满足模型收敛条件，继续对更新图像识别模型中的模型参数进行调整；
第六确定单元144，用于当更新损失值小于更新损失值阈值时，确定更新图像识别模型满足模型收敛条件，将更新图像识别模型确定为目标图像识别模型。
其中，第四确定单元141、第二加权单元142、第五确定单元143以及第六确定单元144的具体功能实现方式可以参见上述图2对应实施例中的步骤S104，这里不再进行赘述。
再请参见图9，原始图像还包括第三原始图像；初始标准标注结果还包括第三原始图像的第三初始标准标注结果；初始辅助标注结果还包括第三原始图像的第三初始辅助标注结果；
第五确定单元143可以包括：第一确定子单元1431以及调整模型子单元1432。
第一确定子单元1431，用于根据第三初始标准标注结果以及第三初始辅助标注结果，确定调整损失值；
第一确定子单元1431，还用于对调整损失值以及更新损失值进行加权求和，得到目标损失值；
调整模型子单元1432，用于根据目标损失值对更新图像识别模型中的模型参数进行调整。
其中，第一确定子单元1431以及调整模型子单元1432的具体功能实现方式可以参见上述图2对应实施例中的步骤S104，这里不再进行赘述。
再请参见图9，第二获取模块13可以包括：发送辅助单元131、第一获取单元132、第七确定单元133、第八确定单元134以及第二获取单元135。
发送辅助单元131，用于将更新辅助标注结果发送至至少两个标注对象对应的标注终端，以使至少两个标注对象对应的标注终端，分别根据更新辅助标注结果对第二初始标准标注结果进行调整，得到第二原始图像的候选标注结果；
第一获取单元132，用于获取至少两个标注对象分别对应的标注终端所返回的候选标注结果；至少两个候选标注结果分别包括用于标注第二原始图像中的目标对象的候选标注区域；
第七确定单元133，用于确定至少两个候选标注结果所分别包括的候选标注区域对应的区域数量；
第八确定单元134，用于根据至少两个区域数量，确定至少两个候选标注结果的初始审核标注结果；
第二获取单元135，用于根据初始审核标注结果获取更新标准标注结果。
其中，发送辅助单元131、第一获取单元132、第七确定单元133、第八确定单元134以及第二获取单元135的具体功能实现方式可以参见上述图2对应实施例中的步骤S103，这里不再进行赘述。
再请参见图9，第八确定单元134可以包括：对比数量子单元1341、第二确定子单元1342、获取区域子单元1343以及第三确定子单元1344。
对比数量子单元1341，用于对至少两个区域数量进行对比；至少两个区域数量包括区域数量Ba；a为正整数，且a小于或等于至少两个候选标注结果的结果数量；
第二确定子单元1342，用于若剩余区域数量中存在与区域数量Ba不相同的区域数量，则将至少两个候选标注结果分别确定为初始审核标注结果；剩余区域数量包括至少两个区域数量中除了区域数量Ba之外的区域数量；
获取区域子单元1343，用于若剩余区域数量均与区域数量Ba相同，则获取至少两个候选标注结果中，每两个候选标注结果所分别包括的候选标注区域；
第三确定子单元1344，用于确定每两个候选标注结果所分别包括的候选标注区域之间的重合度，根据重合度确定初始审核标注结果。
其中，对比数量子单元1341、第二确定子单元1342、获取区域子单元1343以及第三确定子单元1344的具体功能实现方式可以参见上述图2对应实施例中的步骤S104，这里不再进行赘述。
再请参见图9，至少两个候选标注结果还分别包括用于标注所包括的候选标注区域的候选对象标签；
第三确定子单元1344可以包括：第一审核子单元13441、划分标签子单元13442以及第二审核子单元13443。
第一审核子单元13441，用于若至少一个重合度小于重合度阈值，则将至少两个候选标注结果分别确定为初始审核标注结果；
划分标签子单元13442，用于若各个重合度等于或大于重合度阈值，则将至少两个候选标注结果中相同的候选对象标签划分在同一个对象标签组中，得到n个对象标签组；n为正整数；
第二审核子单元13443，用于根据n个对象标签组确定初始审核标注结果。
其中，第一审核子单元13441、划分标签子单元13442以及第二审核子单元13443的具体功能实现方式可以参见上述图2对应实施例中的步骤S103，这里不再进行赘述。
再请参见图9，第二审核子单元13443，具体用于统计n个对象标签组分别包括的候选对象标签的对象标签数量，在n个对象标签组分别对应的对象标签数量中获取最大对象标签数量；
第二审核子单元13443，还具体用于确定最大对象标签数量与至少两个候选标注结果对应的对象标签数量之间的数量比例；
第二审核子单元13443，还具体用于将数量比例与数量比例阈值进行对比，若数量比例小于数量比例阈值，则将至少两个候选标注结果分别确定为初始审核标注结果；
第二审核子单元13443，还具体用于若数量比例等于或大于数量比例阈值，则将最大对象标签数量所对应的对象标签组确定为目标对象标签组；
第二审核子单元13443，还具体用于从与目标对象标签组相关联的候选标注结果中获取目标候选标注结果，将目标候选标注结果确定为初始审核标注结果。
其中，第二审核子单元13443的具体功能实现方式可以参见上述图2对应实施例中的步骤S103，这里不再进行赘述。
再请参见图9，第二获取单元135可以包括：第一发送子单元1351以及第二发送子单元1352。
第一发送子单元1351，用于若初始审核标注结果为至少两个候选标注结果，则将初始审核标注结果发送至第一审核终端，以使第一审核终端根据至少两个候选标注结果确定发送至第二审核终端的审核标注结果；第二审核终端用于根据审核标注结果返回更新标准标注结果；
第二发送子单元1352，用于若初始审核标注结果为目标候选标注结果，则将初始审核标注结果发送至第二审核终端，以使第二审核终端根据目标候选标注结果返回更新标准标注结果。
其中，第一发送子单元1351以及第二发送子单元1352的具体功能实现方式可以参见上述图2对 应实施例中的步骤S103，这里不再进行赘述。
再请参见图9，第一获取模块11可以包括：第三获取单元111、第四获取单元112、第九确定单元113以及生成结果单元114。
第三获取单元111，用于获取原始图像；原始图像包括目标对象；
第四获取单元112，用于将原始图像输入至初始图像识别模型，在初始图像识别模型中获取原始图像的图像特征；
第九确定单元113，用于根据图像特征确定目标对象的初始区域识别特征，以及目标对象的初始对象识别特征；
生成结果单元114，用于根据初始区域识别特征生成针对目标对象的初始辅助标注区域，根据初始对象识别特征生成针对初始辅助标注区域的初始辅助对象标签；
生成结果单元114，还用于将初始辅助标注区域以及初始辅助对象标签确定为初始辅助标注结果。
其中，第三获取单元111、第四获取单元112、第九确定单元113以及生成结果单元114的具体功能实现方式可以参见上述图2对应实施例中的步骤S101，这里不再进行赘述。
在本申请实施例中，计算机设备可以将第一初始标准标注结果以及第一初始辅助标注结果作为训练样本集，对初始图像识别模型进行更新，即对模型参数进行调整，得到更新图像识别模型，可以理解的是，该过程不仅可以实现模型更新，还可以根据训练样本集确定模型更新的方向；进一步，基于更新图像识别模型预测第二原始图像的更新辅助标注结果，获取基于更新辅助标注结果，对第二初始标准标注结果进行调整所得到的更新标准标注结果，该过程可以实现第二初始标准标注结果的更新；进一步，当将更新图像识别模型确定为目标图像识别模型时，利用目标图像识别模型生成目标图像的目标辅助标注结果。上述可知，本申请实施例不仅可以依据训练样本集对初始图像识别模型进行更新，以提高更新图像识别模型的识别能力；还可以通过更新图像识别模型对第二初始标准标注结果进行更新，以提高更新标准标注结果的精度，故采用本申请可以实现图像识别模型以及标注结果的双向更新。
进一步地，请参见图10，图10是本申请实施例提供的一种计算机设备的结构示意图。如图10所示，该计算机设备1000可以包括：至少一个处理器1001，例如CPU，至少一个网络接口1004，用户接口1003，存储器1005，至少一个通信总线1002。其中，通信总线1002用于实现这些组件之间的连接通信。其中，用户接口1003可以包括显示屏(Display)、键盘(Keyboard)，网络接口1004可选地可以包括标准的有线接口、无线接口(如WI-FI接口)。存储器1005可以是高速RAM存储器，也可以是非不稳定的存储器(non-volatile memory)，例如至少一个磁盘存储器。存储器1005可选地还可以是至少一个位于远离前述处理器1001的存储装置。如图10所示，作为一种计算机存储介质的存储器1005可以包括操作系统、网络通信模块、用户接口模块以及设备控制应用程序。
在图10所示的计算机设备1000中，网络接口1004可提供网络通讯功能；而用户接口1003主要用于为用户提供输入的接口；而处理器1001可以用于调用存储器1005中存储的设备控制应用程序，以实现：
基于初始图像识别模型预测原始图像的初始辅助标注结果，获取基于初始辅助标注结果所确定的初始标准标注结果；原始图像包括第一原始图像以及第二原始图像；初始标准标注结果包括第一原始图像的第一初始标准标注结果，以及第二原始图像的第二初始标准标注结果；初始辅助标注结果包括第一原始图像的第一初始辅助标注结果；
根据第一初始标准标注结果以及第一初始辅助标注结果，对初始图像识别模型中的模型参数进行调整，生成更新图像识别模型；
基于更新图像识别模型预测第二原始图像的更新辅助标注结果，获取更新标准标注结果；更新标准标注结果是基于更新辅助标注结果对第二初始标准标注结果进行调整所得到的；
当根据更新辅助标注结果以及更新标准标注结果确定更新图像识别模型满足模型收敛条件时，将更新图像识别模型确定为目标图像识别模型；目标图像识别模型用于生成目标图像的目标辅助标注结果。
应当理解，本申请实施例中所描述的计算机设备1000可执行前文图2、图6、图7以及图8所对应实施例中对数据处理方法的描述，也可执行前文图9所对应实施例中对数据处理装置1的描述，在 此不再赘述。另外，对采用相同方法的有益效果描述，也不再进行赘述。
本申请实施例还提供一种计算机可读存储介质，该计算机可读存储介质存储有计算机程序，该计算机程序包括程序指令，该程序指令被处理器执行时实现图2、图6、图7以及图8中各个步骤所提供的数据处理方法，具体可参见上述图2、图6、图7以及图8各个步骤所提供的实现方式，在此不再赘述。另外，对采用相同方法的有益效果描述，也不再进行赘述。
上述计算机可读存储介质可以是前述任一实施例提供的数据处理装置或者上述计算机设备的内部存储单元，例如计算机设备的硬盘或内存。该计算机可读存储介质也可以是该计算机设备的外部存储设备，例如该计算机设备上配备的插接式硬盘，智能存储卡(smart media card，SMC)，安全数字(secure digital，SD)卡，闪存卡(flash card)等。进一步地，该计算机可读存储介质还可以既包括该计算机设备的内部存储单元也包括外部存储设备。该计算机可读存储介质用于存储该计算机程序以及该计算机设备所需的其他程序和数据。该计算机可读存储介质还可以用于暂时地存储已经输出或者将要输出的数据。
本申请实施例还提供了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备可执行前文图2、图6、图7以及图8所对应实施例中对数据处理方法的描述，在此不再赘述。另外，对采用相同方法的有益效果描述，也不再进行赘述。
本申请实施例的说明书和权利要求书及附图中的术语“第一”、“第二”等是用于区别不同对象，而非用于描述特定顺序。此外，术语“包括”以及它们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、装置、产品或设备没有限定于已列出的步骤或模块，而是可选地还包括没有列出的步骤或模块，或可选地还包括对于这些过程、方法、装置、产品或设备固有的其他步骤单元。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本申请实施例提供的方法及相关装置是参照本申请实施例提供的方法流程图和/或结构示意图来描述的，具体可由计算机程序指令实现方法流程图和/或结构示意图的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。这些计算机程序指令可提供到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或结构示意图一个方框或多个方框中指定的功能的装置。这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或结构示意图一个方框或多个方框中指定的功能。这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或结构示意一个方框或多个方框中指定的功能的步骤。
以上所揭露的仅为本申请较佳实施例而已，当然不能以此来限定本申请之权利范围，因此依本申请权利要求所作的等同变化，仍属本申请所涵盖的范围。

Claims (16)

  1. 一种数据处理方法，其特征在于，在计算机设备中执行，所述方法包括：
    基于初始图像识别模型预测原始图像的初始辅助标注结果，所述原始图像包括第一原始图像以及第二原始图像，所述初始辅助标注结果包括所述第一原始图像的第一初始辅助标注结果；
    获取对所述初始辅助标注结果进行校正所确定的初始标准标注结果，其中，所述初始标准标注结果包括所述第一原始图像的第一初始标准标注结果，以及所述第二原始图像的第二初始标准标注结果；
    根据所述第一初始标准标注结果以及所述第一初始辅助标注结果，对所述初始图像识别模型中的模型参数进行调整，生成更新图像识别模型；
    基于所述更新图像识别模型预测所述第二原始图像的更新辅助标注结果；
    获取所述第二原始图像的更新标准标注结果，所述更新标准标注结果是基于所述更新辅助标注结果对所述第二初始标准标注结果进行调整所得到的；
    当根据所述更新辅助标注结果以及所述更新标准标注结果确定所述更新图像识别模型满足模型收敛条件时，将所述更新图像识别模型确定为目标图像识别模型，所述目标图像识别模型用于生成目标图像的标注结果。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    响应模型更新指令，将所述第一原始图像确定为样本图像，将所述第一初始标准标注结果确定为所述样本图像的样本标签，将所述第一初始辅助标注结果确定为所述样本图像的样本预测结果；
    所述根据所述第一初始标准标注结果以及所述第一初始辅助标注结果，对所述初始图像识别模型中的模型参数进行调整，生成更新图像识别模型，包括：
    根据所述样本标签以及所述样本预测结果，确定所述初始图像识别模型的总损失值；
    根据所述总损失值，对所述初始图像识别模型中的模型参数进行调整，当调整后的初始图像识别模型满足模型收敛条件时，将调整后的初始图像识别模型确定为所述更新图像识别模型。
  3. 根据权利要求1所述的方法，其特征在于，所述初始辅助标注结果还包括所述第二原始图像的第二初始辅助标注结果；
    所述方法还包括：
    确定所述第一初始辅助标注结果以及所述第一初始标准标注结果之间的第一标注结果误差；
    确定所述第二初始辅助标注结果以及所述第二初始标准标注结果之间的第二标注结果误差；
    确定所述第一标注结果误差以及所述第二标注结果误差之间的平均标注结果误差；
    根据所述平均标注结果误差确定所述初始图像识别模型的初始损失值；
    若所述初始损失值大于或等于初始损失值阈值，则执行所述根据所述第一初始标准标注结果以及所述第一初始辅助标注结果，对所述初始图像识别模型中的模型参数进行调整，生成更新图像识别模型的步骤。
  4. 根据权利要求3所述的方法，其特征在于，所述第一初始辅助标注结果包括针对目标对象的第一标注区域，以及针对所述第一标注区域的第一对象标签；所述第一初始标准标注结果包括针对所述目标对象的第二标注区域，以及针对所述第二标注区域的第二对象标签；
    所述确定所述第一初始辅助标注结果以及所述第一初始标准标注结果之间的第一标注结果误差，包括：
    确定所述第一标注区域以及所述第二标注区域之间的初始区域误差；
    确定所述第一对象标签以及所述第二对象标签之间的初始对象误差；
    对所述初始区域误差以及所述初始对象误差进行加权求和，得到所述第一标注结果误差。
  5. 根据权利要求1所述的方法，其特征在于，所述更新辅助标注结果包括针对目标对象的更新辅助标注区域，以及针对所述更新辅助标注区域的更新辅助对象标签；所述更新标准标注结果包括针对 所述目标对象的更新标准标注区域，以及针对所述更新标准标注区域的更新标准对象标签；
    所述当根据所述更新辅助标注结果以及所述更新标准标注结果确定所述更新图像识别模型满足模型收敛条件时，将所述更新图像识别模型确定为目标图像识别模型，包括：
    确定所述更新辅助标注区域以及所述更新标准标注区域之间的更新区域损失值；
    确定所述更新辅助对象标签以及所述更新标准对象标签之间的更新对象损失值；
    对所述更新区域损失值以及所述更新对象损失值进行加权求和，得到所述更新图像识别模型的更新损失值；
    当所述更新损失值大于或等于更新损失值阈值时，确定所述更新图像识别模型不满足模型收敛条件，继续对所述更新图像识别模型中的模型参数进行调整；
    当所述更新损失值小于所述更新损失值阈值时，确定所述更新图像识别模型满足模型收敛条件，将所述更新图像识别模型确定为所述目标图像识别模型。
  6. 根据权利要求5所述的方法，其特征在于，所述原始图像还包括第三原始图像；所述初始标准标注结果还包括所述第三原始图像的第三初始标准标注结果；所述初始辅助标注结果还包括所述第三原始图像的第三初始辅助标注结果；
    所述继续对所述更新图像识别模型中的模型参数进行调整，包括：
    根据所述第三初始标准标注结果以及所述第三初始辅助标注结果，确定调整损失值；
    对所述调整损失值以及所述更新损失值进行加权求和，得到目标损失值；
    根据所述目标损失值对所述更新图像识别模型中的模型参数进行调整。
  7. 根据权利要求1所述的方法，其特征在于，所述获取所述第二原始图像的更新标准标注结果，包括：
    将所述更新辅助标注结果发送至至少两个标注终端，以使所述至少标注终端，分别根据所述更新辅助标注结果对所述第二初始标准标注结果进行调整，得到所述第二原始图像的候选标注结果；
    获取所述至少两个标注终端所返回的候选标注结果；至少两个候选标注结果分别包括用于标注所述第二原始图像中的目标对象的候选标注区域；
    确定所述至少两个候选标注结果所分别包括的候选标注区域对应的区域数量；
    根据至少两个区域数量，确定所述至少两个候选标注结果的初始审核标注结果；
    根据所述初始审核标注结果获取所述更新标准标注结果。
  8. 根据权利要求7所述的方法，其特征在于，所述根据至少两个区域数量，确定所述至少两个候选标注结果的初始审核标注结果，包括：
    对所述至少两个区域数量进行对比；所述至少两个区域数量包括区域数量B a；a为正整数，且a小于或等于所述至少两个候选标注结果的结果数量；
    若剩余区域数量中存在与所述区域数量B a不相同的区域数量，则将所述至少两个候选标注结果分别确定为所述初始审核标注结果；所述剩余区域数量包括所述至少两个区域数量中除了所述区域数量B a之外的区域数量；
    若所述剩余区域数量均与所述区域数量B a相同，则获取所述至少两个候选标注结果中，每两个候选标注结果所分别包括的候选标注区域；
    确定所述每两个候选标注结果所分别包括的候选标注区域之间的重合度，根据所述重合度确定所述初始审核标注结果。
  9. 根据权利要求8所述的方法，其特征在于，所述至少两个候选标注结果还分别包括用于标注所包括的候选标注区域的候选对象标签；
    所述根据所述重合度确定所述初始审核标注结果，包括：
    若至少一个所述重合度小于重合度阈值，则将所述至少两个候选标注结果分别确定为所述初始审核标注结果；
    若各个所述重合度等于或大于所述重合度阈值，则将所述至少两个候选标注结果中相同的候选对象标签划分在同一个对象标签组中，得到n个对象标签组；n为正整数；
    根据所述n个对象标签组确定所述初始审核标注结果。
  10. 根据权利要求9所述的方法，其特征在于，所述根据所述n个对象标签组确定所述初始审核标注结果，包括：
    统计所述n个对象标签组分别包括的候选对象标签的对象标签数量，在所述n个对象标签组分别对应的对象标签数量中获取最大对象标签数量；
    确定所述最大对象标签数量与所述至少两个候选标注结果对应的对象标签数量之间的数量比例；
    将所述数量比例与数量比例阈值进行对比，若所述数量比例小于所述数量比例阈值，则将所述至少两个候选标注结果分别确定为所述初始审核标注结果；
    若所述数量比例等于或大于所述数量比例阈值，则将所述最大对象标签数量所对应的对象标签组确定为目标对象标签组；
    从与所述目标对象标签组相关联的候选标注结果中获取目标候选标注结果，将所述目标候选标注结果确定为所述初始审核标注结果。
  11. 根据权利要求10所述的方法，其特征在于，所述根据所述初始审核标注结果获取所述更新标准标注结果，包括：
    若所述初始审核标注结果为所述至少两个候选标注结果，则将所述初始审核标注结果发送至第一审核终端，以使所述第一审核终端根据所述至少两个候选标注结果确定发送至第二审核终端的审核标注结果；所述第二审核终端用于根据所述审核标注结果返回所述更新标准标注结果；
    若所述初始审核标注结果为所述目标候选标注结果，则将所述初始审核标注结果发送至所述第二审核终端，以使所述第二审核终端根据所述目标候选标注结果返回所述更新标准标注结果。
  12. 根据权利要求1-11任一项所述的方法，其特征在于，所述基于初始图像识别模型预测原始图像的初始辅助标注结果，包括：
    获取所述原始图像，所述原始图像包括目标对象；
    将所述原始图像输入至所述初始图像识别模型，在所述初始图像识别模型中获取所述原始图像的图像特征；
    根据所述图像特征确定所述目标对象的初始区域识别特征，以及所述目标对象的初始对象识别特征；
    根据所述初始区域识别特征生成针对所述目标对象的初始辅助标注区域，根据所述初始对象识别特征生成针对所述初始辅助标注区域的初始辅助对象标签；
    将所述初始辅助标注区域以及所述初始辅助对象标签确定为所述初始辅助标注结果。
  13. 一种数据处理装置，其特征在于，包括：
    第一获取模块，用于基于初始图像识别模型预测原始图像的初始辅助标注结果，所述原始图像包括第一原始图像以及第二原始图像，所述初始辅助标注结果包括所述第一原始图像的第一初始辅助标注结果；获取对所述初始辅助标注结果进行校正所确定的初始标准标注结果；其中，所述初始标准标注结果包括所述第一原始图像的第一初始标准标注结果，以及所述第二原始图像的第二初始标准标注结果；
    更新模型模块，用于根据第一初始标准标注结果以及第一初始辅助标注结果，对初始图像识别模型中的模型参数进行调整，生成更新图像识别模型；
    第二获取模块，用于基于更新图像识别模型预测第二原始图像的更新辅助标注结果，获取所述第二原始图像的更新标准标注结果；更新标准标注结果是基于更新辅助标注结果对第二初始标准标注结果进行调整所得到的；
    第一确定模块，用于当根据更新辅助标注结果以及更新标准标注结果确定更新图像识别模型满足 模型收敛条件时，将更新图像识别模型确定为目标图像识别模型；目标图像识别模型用于生成目标图像的标注结果。
  14. 一种计算机设备，其特征在于，包括：处理器、存储器以及网络接口；所述处理器与所述存储器、所述网络接口相连，其中，所述网络接口用于提供数据通信功能，所述存储器用于存储计算机程序，所述处理器用于调用所述计算机程序，以使得所述计算机设备执行权利要求1至12任一项所述的方法。
  15. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有计算机程序，所述计算机程序适于由处理器加载并执行，以使得具有所述处理器的计算机设备执行权利要求1-12任一项所述的方法。
  16. 一种计算机程序产品，其特征在于，计算机程序产品包括计算机指令，所述计算机指令存储在计算机可读存储介质中，所述计算机指令适于由处理器读取并执行，以使得具有所述处理器的计算机设备执行如权利要求1-12任一项的方法。
PCT/CN2022/137442 2021-12-13 2022-12-08 数据处理方法、装置、设备、存储介质及程序产品 WO2023109631A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/368,680 US20240005211A1 (en) 2021-12-13 2023-09-15 Data processing method and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111521261.4 2021-12-13
CN202111521261.4A CN114219971A (zh) 2021-12-13 2021-12-13 一种数据处理方法、设备以及计算机可读存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/368,680 Continuation US20240005211A1 (en) 2021-12-13 2023-09-15 Data processing method and apparatus

Publications (1)

Publication Number Publication Date
WO2023109631A1 true WO2023109631A1 (zh) 2023-06-22

Family

ID=80701453

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/137442 WO2023109631A1 (zh) 2021-12-13 2022-12-08 数据处理方法、装置、设备、存储介质及程序产品

Country Status (3)

Country Link
US (1) US20240005211A1 (zh)
CN (1) CN114219971A (zh)
WO (1) WO2023109631A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113168527A (zh) * 2018-11-06 2021-07-23 谷歌有限责任公司 用于从实体文档提取信息的系统和方法
CN114219971A (zh) * 2021-12-13 2022-03-22 腾讯科技（深圳）有限公司 一种数据处理方法、设备以及计算机可读存储介质
CN117785964A (zh) * 2024-02-28 2024-03-29 宜宾市万事通网络信息服务有限公司 应用于网络服务的数据处理方法及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109446369A (zh) * 2018-09-28 2019-03-08 武汉中海庭数据技术有限公司 图像半自动标注的交互方法及系统
CN111507480A (zh) * 2020-04-16 2020-08-07 光际科技(上海)有限公司 标注方法、装置、设备及存储介质
CN112989087A (zh) * 2021-01-26 2021-06-18 腾讯科技（深圳）有限公司 一种图像处理方法、设备以及计算机可读存储介质
US20210271809A1 (en) * 2018-07-05 2021-09-02 The Fourth Paradigm (Beijing) Tech Co Ltd Machine learning process implementation method and apparatus, device, and storage medium
CN114219971A (zh) * 2021-12-13 2022-03-22 腾讯科技（深圳）有限公司 一种数据处理方法、设备以及计算机可读存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210271809A1 (en) * 2018-07-05 2021-09-02 The Fourth Paradigm (Beijing) Tech Co Ltd Machine learning process implementation method and apparatus, device, and storage medium
CN109446369A (zh) * 2018-09-28 2019-03-08 武汉中海庭数据技术有限公司 图像半自动标注的交互方法及系统
CN111507480A (zh) * 2020-04-16 2020-08-07 光际科技(上海)有限公司 标注方法、装置、设备及存储介质
CN112989087A (zh) * 2021-01-26 2021-06-18 腾讯科技（深圳）有限公司 一种图像处理方法、设备以及计算机可读存储介质
CN114219971A (zh) * 2021-12-13 2022-03-22 腾讯科技（深圳）有限公司 一种数据处理方法、设备以及计算机可读存储介质

Also Published As

Publication number Publication date
US20240005211A1 (en) 2024-01-04
CN114219971A (zh) 2022-03-22

Similar Documents

Publication Publication Date Title
WO2023109631A1 (zh) 数据处理方法、装置、设备、存储介质及程序产品
CN110276366A (zh) 使用弱监督模型来检测对象
WO2021143103A1 (zh) 视频数据处理方法、装置、设备及计算机可读存储介质
CN112989087B (zh) 一种图像处理方法、设备以及计算机可读存储介质
CN110163111A (zh) 基于人脸识别的叫号方法、装置、电子设备及存储介质
WO2023015935A1 (zh) 一种体检项目推荐方法、装置、设备及介质
CN111783863A (zh) 一种图像处理方法、装置、设备以及计算机可读存储介质
CN115050064A (zh) 人脸活体检测方法、装置、设备及介质
Gao et al. A hierarchical recurrent approach to predict scene graphs from a visual‐attention‐oriented perspective
CN112214588A (zh) 多意图识别方法、装置、电子设备及存储介质
WO2022105118A1 (zh) 基于图像的健康状态识别方法、装置、设备及存储介质
CN114780701A (zh) 自动问答匹配方法、装置、计算机设备及存储介质
WO2022105546A1 (zh) 一种基于区块链的大数据交易方法、装置、介质及设备
KR102145858B1 (ko) 문서 이미지로부터 인식된 용어를 표준화하기 위한 방법
CN116468043A (zh) 嵌套实体识别方法、装置、设备及存储介质
CN116757207A (zh) 基于人工智能的icd自动编码方法及相关设备
CN116205723A (zh) 基于人工智能的面签风险检测方法及相关设备
WO2007077694A1 (ja) 鼻紋情報生成装置及び鼻紋情報照合装置及び鼻紋情報生成プログラム及び鼻紋情報生成方法
CN112862840B (zh) 图像分割方法、装置、设备及介质
CN113657086B (zh) 词语处理方法、装置、设备及存储介质
CN115880702A (zh) 数据处理方法、装置、设备、程序产品及存储介质
WO2021114626A1 (zh) 一种病历数据的质量检测方法和相关装置
CN115705678A (zh) 一种图像数据处理方法、计算机设备以及介质
WO2021147404A1 (zh) 依存关系分类方法及相关设备
Zhou et al. EACE: A document-level event argument extraction model with argument constraint enhancement

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22906379

Country of ref document: EP

Kind code of ref document: A1