CN117456156A

CN117456156A - Picture labeling method, system and medium based on crowdsourcing

Info

Publication number: CN117456156A
Application number: CN202311512013.2A
Authority: CN
Inventors: 张霖涛
Original assignee: Basebit Shanghai Information Technology Co ltd
Current assignee: Basebit Shanghai Information Technology Co ltd
Priority date: 2023-11-14
Filing date: 2023-11-14
Publication date: 2024-01-26

Abstract

The embodiment of the application provides a crowdsourcing-based picture labeling method, a crowdsourcing-based picture labeling system and a crowdsourcing-based picture labeling medium. The method comprises the following steps: the method comprises the steps of obtaining image characteristics to be filled according to image matching professional crowdsourcing groups, classifying the image to obtain the marking rate, screening and detecting the image characteristics to obtain the real image characteristic quantity and characteristic marking rate, screening effective personnel meeting average requirements according to the effective marking rate of each crowdsourcing personnel if the image characteristics are compared through threshold values, extracting historical marking rate, processing to obtain professional marking effective correction coefficients, judging the total marking effectiveness through threshold value comparison, repairing the image and checking the effect if the image is effective; therefore, effective personnel are screened according to the circle marking effect of the professional crowdsourcing personnel, the circle marking effectiveness of the effective personnel is judged, then the picture is repaired and the repair effect is checked, the crowdsourcing marking effectiveness is checked through the picture marking effect of the crowdsourcing personnel, the picture repair effect is checked, and the crowdsourcing marking and repair technology of the picture is realized.

Description

Picture labeling method, system and medium based on crowdsourcing

Technical Field

The application relates to the technical field of picture annotation, in particular to a crowdsourcing-based picture annotation method, system and medium.

Background

The picture marking is a common marking type, mainly, unprocessed pictures are identified and marked in circles, so that the picture information characteristics are further corrected to obtain accurate picture information, a large number of professional pictures cannot be identified and marked through a computer or limited personnel, workload backlog or overlarge deviation is easily caused, therefore, a user group marking method for marking pictures by adopting professional crowdsourcing personnel is an effective way capable of meeting the marking of a large number of picture information, and the quality, profession, content and background information of different pictures have large differences, and the reliability of crowdsourcing marking is difficult to be checked by a traditional system identification and distribution method, and the repair effect of the crowdsourcing marked pictures is difficult to judge, so that the current situation that crowdsourcing picture marking and repair effect errors are large is caused, and how to verify and obtain effective crowdsourcing picture marking quality and realize high-quality picture marking repair is a current vacant technology.

In view of the above problems, an effective technical solution is currently needed.

Disclosure of Invention

An object of the embodiment of the application is to provide a picture marking method, a system and a medium based on crowdsourcing, which can screen effective crowdsourcing personnel according to a marking effect checking result of the ring of the professional crowdsourcing personnel, judge the marking effectiveness of the ring of the effective crowdsourcing personnel, repair and check the repairing effect by repairing the picture, and realize marking effectiveness of the crowdsourcing personnel and checking the repairing effect of the picture by the picture marking effect of the crowdsourcing personnel, thereby realizing crowdsourcing marking and repairing means of the picture.

The embodiment of the application also provides a crowdsourcing-based picture labeling method, which comprises the following steps:

obtaining a picture to be marked, carrying out information pickup and identification on the picture to obtain picture integrity information and picture category information, carrying out integral quality grade division on the picture through a preset crowdsourcing platform to obtain picture quality grade, and carrying out attribute classification on the picture to obtain picture attribute category information;

performing picture pairing processing according to the picture attribute type information to obtain an adaptive professional crowdsourcing group and a corresponding professional labeling difficulty coefficient, performing identification processing on the picture through the professional crowdsourcing group to obtain a plurality of picture features to be filled and corresponding a plurality of picture feature vacancy rates, and processing to obtain a picture vacancy rate average index;

respectively carrying out image feature classification circle marking on the images by a plurality of crowd-sourced personnel of the professional crowd-sourced group to obtain image feature circle marking results, wherein the image feature circle marking results comprise corresponding circle marking information obtained by classification circle marking, and obtaining the circle marking rates of corresponding circle marking frequency, and respectively screening key correct image features, high abnormal image features and key error image features through threshold comparison according to the circle marking rates;

Respectively verifying and identifying the key correct picture features and the key incorrect picture features to obtain real feature quantity and corresponding feature labeling rate data, respectively carrying out threshold comparison with a preset feature labeling rate threshold to obtain a corresponding labeling threshold comparison result, if the results meet the preset requirements, identifying the high-abnormality picture features to obtain real abnormal feature quantity and corresponding abnormal feature labeling rate data, and carrying out threshold comparison with a preset abnormal feature labeling rate threshold to obtain a third labeling threshold comparison result;

if the third labeling threshold comparison result meets the preset threshold comparison requirement, acquiring effective labeling rate data of each classification feature and average effective labeling rate data of each classification feature of the picture feature ring-injection labeling result of each crowdsourcing personnel of the professional crowdsourcing group, screening effective crowdsourcing labeling personnel meeting the average effective labeling rate data of each classification feature, and extracting corresponding historical effective ring-injection labeling rate data;

processing according to the historical effective circle annotation rate data of each effective crowd-sourced label person and combining the picture quality grade, the professional annotation difficulty coefficient and the picture vacancy rate average index to obtain a professional circle annotation effective correction coefficient, and comparing with a preset professional picture characteristic annotation effect detection threshold value to judge the total circle annotation effectiveness of each effective crowd-sourced label person;

And if the total circle annotation validity is valid, extracting the image corresponding to the image features to be filled and the key error image features of the effective crowdsourcing label personnel, filling and correcting the image to obtain a repaired image, comparing the repaired image with a standardized processing image to obtain an image repair result, and updating records of the effective crowdsourcing label personnel.

Optionally, in the crowdsourcing-based picture labeling method according to the embodiment of the present application, the obtaining a picture to be labeled, performing information pickup and identification on the picture to obtain picture integrity information and picture category information, performing integral quality grading on the picture through a preset crowdsourcing platform to obtain picture quality grades, and performing attribute classification on the picture to obtain picture attribute category information includes:

acquiring a picture to be marked;

carrying out information pickup identification on the picture according to a preset picture identification model to obtain picture integrity information and picture category information;

carrying out quality grade division on the picture according to the picture integrity grade division through a preset crowdsourcing platform according to the picture integrity information to obtain a picture quality grade;

Carrying out attribute classification on the picture according to the picture category information and a preset attribute classification method to obtain picture attribute category information;

the picture attribute type information comprises picture element content information, picture object field information and picture scene information.

Optionally, in the crowdsourcing-based image labeling method according to the embodiment of the present application, performing image pairing processing according to the image attribute type information to obtain an adapted professional crowdsourcing group, and corresponding professional labeling difficulty coefficient, performing identification processing on an image by the professional crowdsourcing group to obtain a plurality of image features to be filled and corresponding plurality of image feature void ratios, and processing to obtain an image void ratio average index, including:

performing professional matching processing on the picture information through the preset crowdsourcing platform according to the picture element content information, the picture object field information and the picture scene information to obtain professional crowdsourcing groups correspondingly adapted to the pictures and corresponding professional labeling difficulty coefficients;

the pictures are identified through the professional crowdsourcing group, and a plurality of picture features to be filled and corresponding to a plurality of picture feature vacancy rates are obtained, wherein the picture features to be filled are supplemented by a plurality of crowdsourcing personnel of the professional crowdsourcing group;

And obtaining the average index of the picture vacancy rate according to the picture characteristic vacancy rate.

Optionally, in the crowdsourcing-based image labeling method according to the embodiment of the present application, the performing, by the plurality of crowdsourcing personnel of the professional crowdsourcing group, image feature classification and marking on the image to obtain an image feature marking result, including classifying each piece of marking information obtained by marking, obtaining each circle rate of the corresponding circle mark frequency, and respectively screening the key correct image feature, the high anomaly image feature and the key error image feature according to the each circle rate through threshold comparison, including:

respectively carrying out image feature classification marking on the images by a plurality of crowdsourcing personnel of the professional crowdsourcing group to obtain image feature marking results;

the picture feature circle marking result comprises correct picture feature circle marking information obtained by the first circle marking, abnormal picture feature circle marking information obtained by the second circle marking and error picture feature circle marking information obtained by the third circle marking;

respectively carrying out first circle marking, second circle marking and third circle marking according to each picture feature in the picture to obtain a first circle marking rate corresponding to each correct picture feature, a second circle marking rate corresponding to each abnormal picture feature and a third circle marking rate corresponding to each error picture feature;

Marking the correct picture characteristics meeting the requirement of a preset first round of injection threshold value in the correct picture characteristics corresponding to the first round of injection rate as key correct picture characteristics;

marking the abnormal picture characteristics meeting the requirement of a preset second circle filling threshold value in the abnormal picture characteristics corresponding to the second circle filling rate as high abnormal picture characteristics;

and marking the error picture characteristics meeting the requirement of a preset third round of injection threshold value in the error picture characteristics corresponding to the third round of injection rate as key error picture characteristics.

Optionally, in the crowdsourcing-based image labeling method according to the embodiment of the present application, the verifying and identifying each key correct image feature and each key incorrect image feature to obtain real feature quantity and corresponding feature labeling rate data, and performing threshold comparison with a preset feature labeling rate threshold to obtain a corresponding labeling threshold comparison result, if the results all meet the preset requirements, identifying each high anomaly image feature to obtain real anomaly feature quantity and corresponding anomaly feature labeling rate data, and performing threshold comparison with a preset anomaly feature labeling rate threshold to obtain a third labeling threshold comparison result, including:

Respectively verifying and identifying the key correct picture features through a preset feature verification and identification model to obtain the number of true correct features and corresponding correct feature labeling rate data;

respectively verifying and identifying the key error picture features through a preset feature verification and identification model to obtain the number of real error features and corresponding error feature marking rate data;

threshold comparison is carried out according to the correct feature labeling rate data and a preset correct feature labeling rate threshold value, and a first labeling threshold value comparison result is obtained;

performing threshold comparison according to the error feature labeling rate data and a preset error feature labeling rate threshold value to obtain a second labeling threshold value comparison result;

if the first labeling threshold comparison result and the second labeling threshold comparison result both meet the preset threshold comparison requirement, identifying each high-abnormality picture feature through a preset picture identification model to obtain the number of real abnormality features and corresponding abnormality feature labeling rate data;

and comparing the threshold value according to the abnormal feature labeling rate data with a preset abnormal feature labeling rate threshold value to obtain a third labeling threshold value comparison result.

Optionally, in the crowdsourcing-based image labeling method according to the embodiment of the present application, if the third labeling threshold comparison result meets a preset threshold comparison requirement, obtaining effective labeling rate data of each classification feature and average effective labeling rate data of each classification feature of each crowdsourcing person of the professional crowdsourcing group for image feature ring labeling result, screening effective crowdsourcing labeling persons meeting the average effective labeling rate data of each classification feature, and extracting corresponding historical effective ring labeling rate data, including:

If the third labeling threshold comparison result meets a preset threshold comparison requirement, acquiring correct feature effective labeling rate data, error feature effective labeling rate data and abnormal feature effective labeling rate data of picture feature ring-annotating labeling results of all crowdsourcing personnel of the professional crowdsourcing group;

acquiring correct feature average effective labeling rate data, error feature average effective labeling rate data and abnormal feature average effective labeling rate data of all crowdsourcing personnel;

screening the effective crowdsourcing labeling personnel which meet the average effective labeling rate data of the correct features, the average effective labeling rate data of the error features and the average effective labeling rate data of the abnormal features in the crowdsourcing personnel;

and extracting corresponding historical effective circle annotation rate data of each effective crowdsourcing annotation personnel.

Optionally, in the crowd-sourcing-based image labeling method according to the embodiment of the present application, the processing according to the historical effective crowd-sourcing labeling rate data of each effective crowd-sourcing labeling person and the image quality level, the professional labeling difficulty coefficient and the image vacancy rate average index to obtain the professional crowd-sourcing labeling effective correction coefficient, and comparing with a preset professional image feature labeling effect detection threshold to determine total crowd-sourcing effectiveness of each effective crowd-sourcing labeling person, including:

Processing according to the historical effective ring annotation rate data corresponding to each effective crowdsourcing annotation person and combining the picture quality grade, the professional annotation difficulty coefficient and the picture vacancy rate average index to obtain a professional ring annotation effective correction coefficient;

and comparing the threshold value according to the professional circle annotating effective correction coefficient with a preset professional picture characteristic annotating effect detection threshold value, and judging the total circle annotating effectiveness of each effective crowd-sourced annotating personnel according to a threshold value comparison result.

Optionally, in the crowdsourcing-based image labeling method according to the embodiment of the present application, if the total circle annotation validity is valid, extracting the image corresponding to the plurality of image features to be filled and the plurality of key error image features of the valid crowdsourcing labeling personnel, filling and correcting the image to obtain a repaired image, comparing the repaired image with a standardized processing image to obtain an image repair result, and updating records of each valid crowdsourcing labeling personnel, including:

if the total circle annotation validity is valid, extracting the corresponding plurality of picture features to be filled and a plurality of key error picture features of the valid crowdsourcing labeling personnel;

Filling and correcting the picture according to the picture features to be filled and the key error picture features to obtain a repaired picture;

comparing the repaired picture with the standardized picture to obtain a picture repairing effect result;

and updating the picture repairing effect records of the effective crowdsourcing labels according to the picture repairing effect results.

In a second aspect, an embodiment of the present application provides a crowd sourcing-based picture annotation system, including: the system comprises a memory and a processor, wherein the memory comprises a program based on a crowdsourcing picture marking method, and the program based on the crowdsourcing picture marking method realizes the following steps when being executed by the processor:

In a third aspect, an embodiment of the present application further provides a computer readable storage medium, where the computer readable storage medium includes a crowd-sourced based picture labeling method program, where the crowd-sourced based picture labeling method program, when executed by a processor, implements the steps of the crowd-sourced based picture labeling method according to any one of the preceding claims.

As can be seen from the foregoing, according to the crowd-sourced picture labeling method, system and medium provided by the embodiments of the present application, a plurality of picture features and average indexes of blank rates are obtained by obtaining a picture quality grade and an attribute category of a picture to be labeled and obtaining an adaptive professional crowd-sourced group for identifying the picture, and the picture features are classified, and are marked to obtain each marking information and each marking rate, and picture feature screening is performed, the screened picture features are classified to obtain the number of true correct/incorrect/abnormal picture features and feature marking rate data, and the next step is performed through the results of threshold comparison, if all pass, the effective crowd-sourced marking personnel of each classified feature are obtained, and the effective crowd-sourced marking personnel meeting the average effective marking rate data of each classified feature are screened, and the corresponding historical effective marking rate data are extracted, and then the professional marking effective correction coefficient is obtained by combining the picture quality grade, the professional marking difficulty coefficient and the average index of the picture blank rates, and then the total marking effectiveness is judged through threshold comparison, and if the effective marking is performed, and the picture is filled and corrected and the repaired after the picture is checked and updated; and the effective crowdsourcing personnel are screened according to the ring-filling marking effect test result of the professional crowdsourcing personnel, the ring-filling marking effectiveness of the effective crowdsourcing personnel is judged, and the picture is repaired and the repairing effect is tested by judging, so that the crowdsourcing personnel marking effectiveness is tested and the picture repairing effect is tested through the picture marking effect of the crowdsourcing personnel, and the crowdsourcing marking and repairing means of the picture are realized.

Additional features and advantages of the application will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the embodiments of the application. The objects and other advantages of the present application may be realized and attained by the structure particularly pointed out in the written description and drawings.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a flowchart of a crowd-sourcing-based picture labeling method provided in an embodiment of the present application;

fig. 2 is a flowchart of obtaining quality level and attribute category information of a picture according to a crowdsourcing-based picture labeling method provided in an embodiment of the present application;

fig. 3 is a flowchart of obtaining a professional labeling difficulty coefficient and an average index of a picture vacancy rate according to the crowd-sourced picture labeling method provided in the embodiment of the present application;

Fig. 4 is a flowchart of obtaining and screening key correct picture features, high abnormal picture features and key error picture features according to a crowdsourcing-based picture labeling method provided in an embodiment of the present application;

fig. 5 is a flowchart of obtaining various labeling threshold comparison results according to a crowdsourcing-based image labeling method provided in an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. The components of the embodiments of the present application, which are generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, as provided in the accompanying drawings, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, are intended to be within the scope of the present application.

It should be noted that like reference numerals and letters refer to like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only to distinguish the description, and are not to be construed as indicating or implying relative importance.

Referring to fig. 1, fig. 1 is a flowchart of a crowd-sourced based picture tagging method in some embodiments of the present application. The crowdsourcing-based picture marking method is used in terminal equipment, such as computers, mobile phone terminals and the like. The image labeling method based on crowdsourcing comprises the following steps:

s11, obtaining a picture to be marked, carrying out information pickup identification on the picture to obtain picture integrity information and picture category information, carrying out integral quality grade division on the picture through a preset crowdsourcing platform to obtain picture quality grade, and carrying out attribute classification on the picture to obtain picture attribute category information;

s12, performing picture pairing processing according to the picture attribute type information to obtain an adaptive professional crowdsourcing group and a corresponding professional labeling difficulty coefficient, performing recognition processing on the picture through the professional crowdsourcing group to obtain a plurality of picture features to be filled and corresponding a plurality of picture feature vacancy rates, and processing to obtain a picture vacancy rate average index;

s13, respectively carrying out image feature classification circle marking on the images by a plurality of crowdsourcing personnel of the professional crowdsourcing group to obtain image feature circle marking results, wherein the image feature circle marking results comprise corresponding circle marking information obtained by classification circle marking, and obtaining the circle marking rates of corresponding circle marking frequency, and respectively screening key correct image features, high abnormal image features and key error image features through threshold comparison according to the circle marking rates;

S14, respectively verifying and identifying the key correct picture features and the key incorrect picture features to obtain real feature quantity and corresponding feature labeling rate data, respectively carrying out threshold comparison with a preset feature labeling rate threshold to obtain a corresponding labeling threshold comparison result, if the results meet the preset requirements, identifying the high-abnormality picture features to obtain real abnormal feature quantity and corresponding abnormal feature labeling rate data, and carrying out threshold comparison with a preset abnormal feature labeling rate threshold to obtain a third labeling threshold comparison result;

s15, if the third labeling threshold comparison result meets a preset threshold comparison requirement, acquiring effective labeling rate data of each classification feature and average effective labeling rate data of each classification feature of picture feature ring-injection labeling results of all crowdsourcing personnel of the professional crowdsourcing group, screening effective crowdsourcing labeling personnel meeting the average effective labeling rate data of each classification feature, and extracting corresponding historical effective ring-injection labeling rate data;

s16, processing according to the historical effective circle annotation rate data of each effective crowd-sourced annotators and combining the picture quality grade, the professional annotation difficulty coefficient and the picture vacancy rate average index to obtain a professional circle annotation effective correction coefficient, and comparing with a preset professional picture feature annotation effect detection threshold value to judge the total circle annotation effectiveness of each effective crowd-sourced annotator;

And S17, if the total circle annotation validity is valid, extracting the image characteristics corresponding to the images to be filled and the key error image characteristics of the effective crowd-sourced labeling personnel, filling and correcting the image to obtain a repaired image, comparing the repaired image with a standardized processing image to obtain an image repair effect result, and updating the record of each effective crowd-sourced labeling personnel.

In order to realize the verification of the validity of crowdsourcing personnel marking through the image marking effect of the crowdsourcing personnel and the verification of the effectiveness of the crowdsourcing personnel on the image repairing, the reliability of the crowdsourcing personnel image marking and the validity of the repaired image after the image marking are improved, the technology firstly verifies the valid circle filling rate of the correct/wrong/abnormal image characteristics after the preliminary screening in the image characteristics of the professional crowdsourcing personnel so as to verify the accuracy and the validity of the image circle filling marking of batch crowdsourcing personnel, if the crowdsourcing personnel pass the verification of the marking validity, the high-efficiency crowdsourcing personnel exceeding the average image marking level are screened out, and the further verification of the validity of the total circle filling is carried out by combining the professional difficulty condition, the characteristic vacancy condition and the image quality condition of the target marking image, if the quality and the attribute category of the picture to be marked are obtained, the picture identification is carried out by the adaptive professional crowdsourcing group to obtain a plurality of picture features to be filled and the average indexes of the void ratios, the picture feature classification and the ring annotating marking are carried out to obtain the ring marking information and the ring annotating ratio, the picture feature screening is carried out, the number of true correct/error/abnormal picture feature numbers and the feature marking ratio data are obtained for the screened picture feature classification, and the next step is carried out through the results of respectively threshold comparison, and if the marking rate data of the classification characteristics of the crowdsourcing staff are all passed, the effective crowdsourcing marking staff meeting the average effective marking rate data of the classification characteristics are screened, the corresponding historical effective marking rate data is extracted, the image quality grade, the professional marking difficulty coefficient and the image vacancy rate average index are combined for processing to obtain the professional marking effective correction coefficient, the threshold value is used for comparing and judging the validity of the total marking, if the marking is valid, the image is filled and corrected, the repair success result is checked and the record is updated.

Referring to fig. 2, fig. 2 is a flowchart of obtaining quality level and attribute category information of a picture according to a crowd-sourced picture tagging method in some embodiments of the present application. According to the embodiment of the invention, the pictures to be marked are obtained, the pictures are picked up and identified to obtain the picture integrity information and the picture category information, the pictures are subjected to integral quality grade division through a preset crowdsourcing platform to obtain the picture quality grade, and the pictures are subjected to attribute classification to obtain the picture attribute category information, specifically:

s21, obtaining a picture to be marked;

s22, carrying out information pickup identification on the picture according to a preset picture identification model to obtain picture integrity information and picture category information;

s23, carrying out quality grade division on the picture according to the picture integrity grade division through a preset crowdsourcing platform according to the picture integrity information to obtain a picture quality grade;

s24, carrying out attribute classification on the picture according to the picture category information and a preset attribute classification method to obtain picture attribute category information;

s25, the picture attribute type information comprises picture element content information, picture object field information and picture scene information.

The method comprises the steps of firstly obtaining a picture to be marked, carrying out information pickup identification on the picture according to a preset picture identification model to obtain picture integrity information and picture category information, grading the picture according to the preset picture integrity by a preset third party crowdsourcing platform to obtain picture quality grade, indirectly determining the effectiveness and the effect of the picture marking by the picture quality grade, and carrying out attribute classification on the picture according to a preset attribute classification method to obtain picture attribute category information, namely classifying and classifying the content, scene, field and background information of the picture so as to facilitate professional crowdsourcing distribution on the picture, wherein the picture attribute category information comprises element content of the picture, the field of a picture display object and scene information of the picture.

Referring to fig. 3, fig. 3 is a flowchart of obtaining a professional labeling difficulty coefficient and an average index of a picture vacancy rate according to a crowd-sourced picture labeling method in some embodiments of the present application. According to the embodiment of the invention, the matching processing of the pictures is performed according to the picture attribute type information to obtain the adapted professional crowdsourcing group and the corresponding professional labeling difficulty coefficient, the identification processing is performed on the pictures through the professional crowdsourcing group to obtain a plurality of picture features to be filled and corresponding a plurality of picture feature vacancy rates, and the processing is performed to obtain the average index of the picture vacancy rates, specifically:

S31, performing professional matching processing on the picture information through the preset crowdsourcing platform according to the picture element content information, the picture object field information and the picture scene information to obtain professional crowdsourcing groups correspondingly adapted to the pictures and corresponding professional labeling difficulty coefficients;

s32, identifying the picture through the professional crowdsourcing group, and acquiring a plurality of picture features to be filled and corresponding a plurality of picture feature void ratios, which are complemented by a plurality of crowdsourcing personnel of the professional crowdsourcing group;

s33, obtaining an average index of the picture vacancy rate according to the picture characteristic vacancy rate.

The method comprises the steps of identifying picture marking characteristics more efficiently and accurately, identifying and repairing pictures, performing professional matching processing on picture information through a preset crowdsourcing platform according to content information of picture elements, field information of picture objects and scene information of the pictures, obtaining professional crowdsourcing groups correspondingly adapted to the pictures, and corresponding professional marking difficulty coefficients, namely, adaptively selecting the corresponding professional crowdsourcing groups according to the picture information, identifying and processing the pictures according to a plurality of persons of the professional crowdsourcing groups to obtain picture characteristics to be filled with the filling defects in the pictures, processing the picture characteristics obtained by the plurality of persons to obtain picture characteristic vacancy rates, and processing the picture characteristic vacancy rates of all the persons to obtain picture vacancy rate average indexes, wherein a calculation formula of the picture vacancy rate average indexes is as follows:

；

Wherein,for the average index of the picture vacancy rate, +.>The picture characteristic vacancy rate of a single person is that r is the number of people and +.>Is a preset characteristic coefficient (the characteristic coefficient is obtained through searching a preset crowdsourcing platform database).

Referring to fig. 4, fig. 4 is a flowchart of obtaining and screening key correct picture features, high anomaly picture features and key error picture features according to a crowdsourcing-based picture labeling method in some embodiments of the present application. According to the embodiment of the invention, the image feature classification circle annotation is performed on the image by the crowd-sourcing staff of the professional crowd-sourcing group to obtain an image feature circle annotation result, wherein the image feature circle annotation result comprises corresponding circle annotation information obtained by classification circle annotation, each circle annotation rate corresponding to circle annotation frequency is obtained, and key correct image features, high abnormal image features and key error image features are respectively screened through threshold comparison according to each circle annotation rate, specifically:

s41, respectively carrying out image feature classification circle annotation on the images by a plurality of crowdsourcing personnel of the professional crowdsourcing group to obtain image feature circle annotation results;

s42, the picture feature circle marking result comprises correct picture feature circle marking information obtained by the first circle marking, abnormal picture feature circle marking information obtained by the second circle marking and error picture feature circle marking information obtained by the third circle marking;

S43, respectively carrying out first circle marking, second circle marking and third circle marking according to each picture feature in the picture to obtain a first circle marking rate corresponding to each correct picture feature, a second circle marking rate corresponding to each abnormal picture feature and a third circle marking rate corresponding to each error picture feature;

s44, marking the correct picture features meeting the requirement of a preset first round of injection threshold value in the correct picture features corresponding to the first round of injection rate as key correct picture features;

s45, marking the abnormal picture characteristics meeting the requirement of a preset second circle filling threshold value in the abnormal picture characteristics corresponding to the second circle filling rate as high abnormal picture characteristics;

s46, marking the error picture characteristics meeting the requirement of a preset third round of injection threshold value in the error picture characteristics corresponding to the third round of injection rate as key error picture characteristics.

Wherein, because the classification of correct, wrong and unacknowledged fuzzy abnormal characteristic information exists for the picture marking, three types of picture characteristics need to be classified and marked for checking the validity of professional crowdsourcing, the marking is to find target characteristics and to display the target characteristics, the marking is to annotate the characteristic information according to the information, the marking results of the marking of the picture characteristic classification marking are respectively carried out on the pictures by a plurality of crowdsourcing personnel of the professional crowdsourcing group, wherein the marking results comprise a first marking of the correct picture characteristic marking information, a second marking of the abnormal picture characteristic marking information and a third marking of the wrong picture characteristic marking information, and meanwhile, according to the frequency of marked marks of each feature marked by all professional crowdsourcing personnel in the picture, obtaining a first marking rate of each correct picture feature marked by the mark, a second marking rate of each abnormal picture feature and a third marking rate of each error picture feature, screening out picture features of high-frequency marking, marking the picture features as screened picture features, namely, finding out picture features meeting the requirement of the corresponding marking threshold in each correct/abnormal/error picture feature corresponding to the first/second/third marking rate through comparison of preset thresholds, and marking the picture features as key correct picture features/high-abnormality picture features/key error picture features respectively, thereby carrying out high-frequency screening on all kinds of picture features marked by the mark so as to improve the accuracy of picture marking.

Referring to fig. 5, fig. 5 is a flowchart of obtaining various labeling threshold comparison results according to a crowd-sourced picture labeling method in some embodiments of the present application. According to the embodiment of the invention, the key correct picture features and the key incorrect picture features are respectively verified and identified to obtain real feature quantity and corresponding feature labeling rate data, and are respectively subjected to threshold comparison with a preset feature labeling rate threshold to obtain a corresponding labeling threshold comparison result, if the results all meet the preset requirements, the high-anomaly picture features are identified to obtain real anomaly feature quantity and corresponding anomaly feature labeling rate data, and are subjected to threshold comparison with the preset anomaly feature labeling rate threshold to obtain a third labeling threshold comparison result, specifically:

s51, respectively verifying and identifying the key correct picture features through a preset feature verification and identification model to obtain the number of true correct features and corresponding correct feature labeling rate data;

s52, respectively verifying and identifying the key error picture features through a preset feature verification and identification model to obtain the number of real error features and corresponding error feature marking rate data;

S53, carrying out threshold comparison according to the correct feature labeling rate data and a preset correct feature labeling rate threshold value to obtain a first labeling threshold value comparison result;

s54, comparing the threshold value according to the error feature labeling rate data with a preset error feature labeling rate threshold value to obtain a second labeling threshold value comparison result;

s55, if the first labeling threshold comparison result and the second labeling threshold comparison result both meet the preset threshold comparison requirement, identifying each high-anomaly picture feature through a preset picture identification model to obtain the number of real anomaly features and corresponding anomaly feature labeling rate data;

s56, comparing the threshold value according to the abnormal feature labeling rate data with a preset abnormal feature labeling rate threshold value, and obtaining a third labeling threshold value comparison result.

The screened key correct picture features/high abnormal picture features/key error picture features are further required to be verified for marking authenticity, namely authenticity of each key correct picture feature and key error picture feature is verified and identified through a preset feature verification and identification model, the occupation ratio of the number of effective true marking features to the number of corresponding category ring marking features is counted, the real quality of the image ring marking effect of a crowded person can be verified through the occupation ratio of the effective marking, the effect of the picture marking is primarily judged through the first marking threshold value comparison result and the second marking threshold value comparison result of the correct/error feature marking rate data and the respective threshold value comparison of the corresponding preset correct/error feature marking rate threshold value, and if the effect is met, the abnormal feature marking rate data is subjected to threshold value comparison to obtain a third marking threshold value comparison result.

According to the embodiment of the invention, if the third labeling threshold comparison result meets a preset threshold comparison requirement, obtaining effective labeling rate data of each classification feature and average effective labeling rate data of each classification feature of the picture feature ring-note labeling result of each crowdsourcing personnel of the professional crowdsourcing group, screening effective crowdsourcing labeling personnel meeting the average effective labeling rate data of each classification feature, and extracting corresponding historical effective ring-note labeling rate data, wherein the steps are as follows:

If the comparison result of the third labeling threshold value also meets the preset requirement, the fact that the picture marking effect of the professional crowdsourcing personnel meets the reliability requirement is indicated, then the part of high-precision labeling personnel with more accurate marking performance in the professional crowdsourcing personnel is further screened to serve as target personnel for repairing pictures in the next step, the effective marking rate data of each classification characteristic of each crowdsourcing personnel, namely marking effect data, are obtained, the effective crowdsourcing labeling personnel meeting the average effective marking rate data of each classification characteristic are screened, namely, the high-precision labeling personnel with marking performance exceeding the average line level are selected, and the historical effective marking rate data corresponding to the selected personnel, namely the historical marking performance data of selected labeling personnel are extracted.

According to the embodiment of the invention, the historical effective marking rate data of each effective crowdsourcing marking person is combined with the picture quality grade, the professional marking difficulty coefficient and the picture vacancy rate average index to obtain the professional marking effective correction coefficient, and then the professional marking effective correction coefficient is compared with a preset professional picture characteristic marking effect detection threshold value to judge the total marking effectiveness of each effective crowdsourcing marking person, specifically:

The method comprises the steps of further verifying the predicted effect of picture repair by a screened effective crowdsourcing marker, calculating according to a historical effective filling marking rate data set of the marker and combining a picture quality grade, a professional marking difficulty coefficient and a picture vacancy rate average index to obtain a professional filling marking effective correction coefficient, namely comprehensively correcting and calculating according to the historical filling marking achievement data of the effective crowdsourcing marker and combining the picture quality, marking difficulty and vacancy conditions to obtain a predicted result correction coefficient of the filling marking of the marker, comparing with a preset professional picture characteristic marking effect detection threshold value, and judging the overall filling effectiveness of the effective crowdsourcing marker according to the result, wherein the calculation formula of the professional filling marking effective correction coefficient is as follows:

；

Wherein,for professional circle annotating markInjecting effective correction factors, < > in->Historical effective circle annotation rate data of the ith effective crowdsourcing annotation personnel, wherein n is the number of the effective crowdsourcing annotation personnel and is +.>Marking difficulty coefficient for profession, and->For picture quality level, ++>For the average index of the picture vacancy rate, +.>、/>、/>Is a preset characteristic coefficient (the characteristic coefficient is obtained through searching a preset crowdsourcing platform database).

According to the embodiment of the invention, if the total circle annotation validity is valid, the image is filled and corrected by extracting the image characteristics to be filled and the key error image characteristics corresponding to the effective crowd-sourced labeling personnel, so as to obtain a repaired image, and the repaired image is compared with a standardized processing image to obtain an image repair effect result and update the records of the effective crowd-sourced labeling personnel, specifically:

The method comprises the steps of detecting the validity of the overall ring filling effect of an effective crowdsourcing labeling person, indicating that the picture labeling and repairing capability of the effective crowdsourcing labeling person passes identification, extracting a plurality of corresponding picture features to be filled and a plurality of key error picture features of the effective crowdsourcing labeling person, filling and repairing the picture to obtain a repaired picture, detecting the repairing effect of the repaired picture, and updating the picture repairing effect record of the person according to the repairing effect result.

The invention also discloses a crowdsourcing-based picture marking system, which comprises a memory and a processor, wherein the memory comprises a crowdsourcing-based picture marking method program, and the crowdsourcing-based picture marking method program realizes the following steps when the processor executes sign abnormal correction data:

According to the embodiment of the invention, the pictures to be marked are obtained, the pictures are picked up and identified to obtain the picture integrity information and the picture category information, the pictures are subjected to integral quality grade division through a preset crowdsourcing platform to obtain the picture quality grade, and the pictures are subjected to attribute classification to obtain the picture attribute category information, specifically:

acquiring a picture to be marked;

According to the embodiment of the invention, the matching processing of the pictures is performed according to the picture attribute type information to obtain the adapted professional crowdsourcing group and the corresponding professional labeling difficulty coefficient, the identification processing is performed on the pictures through the professional crowdsourcing group to obtain a plurality of picture features to be filled and corresponding a plurality of picture feature vacancy rates, and the processing is performed to obtain the average index of the picture vacancy rates, specifically:

；

According to the embodiment of the invention, the image feature classification circle annotation is performed on the image by the crowd-sourcing staff of the professional crowd-sourcing group to obtain an image feature circle annotation result, wherein the image feature circle annotation result comprises corresponding circle annotation information obtained by classification circle annotation, each circle annotation rate corresponding to circle annotation frequency is obtained, and key correct image features, high abnormal image features and key error image features are respectively screened through threshold comparison according to each circle annotation rate, specifically:

According to the embodiment of the invention, the key correct picture features and the key incorrect picture features are respectively verified and identified to obtain real feature quantity and corresponding feature labeling rate data, and are respectively subjected to threshold comparison with a preset feature labeling rate threshold to obtain a corresponding labeling threshold comparison result, if the results all meet the preset requirements, the high-anomaly picture features are identified to obtain real anomaly feature quantity and corresponding anomaly feature labeling rate data, and are subjected to threshold comparison with the preset anomaly feature labeling rate threshold to obtain a third labeling threshold comparison result, specifically:

；

Wherein,marking effective correction coefficients for professional circles, < >>Historical effective circle annotation rate data of the ith effective crowdsourcing annotation personnel, wherein n is the number of the effective crowdsourcing annotation personnel and is +.>Marking difficulty coefficient for profession, and->For picture quality level, ++>For the average index of the picture vacancy rate, +.>、/>、/>Is a preset characteristic coefficient (the characteristic coefficient is obtained through searching a preset crowdsourcing platform database).

The method comprises the steps of detecting the validity of the overall ring injection effect of an effective crowdsourcing labeling person, indicating that the picture labeling and repairing capability of the effective crowdsourcing labeling person passes identification, extracting a plurality of corresponding picture features to be filled and a plurality of key error picture features of the effective crowdsourcing labeling person, filling and repairing pictures to obtain repaired pictures, detecting the repairing effect of the repaired pictures, and updating the picture repairing effect record of the person according to the repairing effect result.

A third aspect of the present invention provides a readable storage medium, where the readable storage medium includes a crowd-sourced picture marking method program, where the crowd-sourced picture marking method program, when executed by a processor, implements the steps of the crowd-sourced picture marking method according to any one of the above.

The invention discloses a crowdsourcing-based picture marking method, system and medium, which are characterized in that picture quality grade and attribute category of pictures to be marked are obtained, a plurality of picture features to be filled and average indexes of blank rate are obtained by identifying pictures by a suitable professional crowdsourcing group, the picture features are classified, marked and marked to obtain each circle marking information and circle marking rate, picture feature screening is carried out, the screened picture features are detected to obtain real correct/incorrect/abnormal picture feature quantity and feature marking rate data, the next step is carried out through the result of threshold comparison respectively, if the picture features pass through the result, the effective marking rate data of each classification feature of each crowdsourcing person is obtained, the effective crowdsourcing marking person meeting the average effective marking rate data of each classification feature is screened, the corresponding historical effective circle marking rate data is extracted, then the professional circle marking effective correction coefficient is obtained by combining the picture quality grade, the professional marking difficulty coefficient and the average index of the picture blank rate, the total circle marking effectiveness is judged through threshold comparison, and if the picture is valid, the filling correction is carried out on the picture, and the repair result is checked and updated; and the effective crowdsourcing personnel are screened according to the ring-filling marking effect test result of the professional crowdsourcing personnel, the ring-filling marking effectiveness of the effective crowdsourcing personnel is judged, and the picture is repaired and the repairing effect is tested by judging, so that the crowdsourcing personnel marking effectiveness is tested and the picture repairing effect is tested through the picture marking effect of the crowdsourcing personnel, and the crowdsourcing marking and repairing means of the picture are realized.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above described device embodiments are only illustrative, e.g. the division of the units is only one logical function division, and there may be other divisions in practice, such as: multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. In addition, the various components shown or discussed may be coupled or directly coupled or communicatively coupled to each other via some interface, whether indirectly coupled or communicatively coupled to devices or units, whether electrically, mechanically, or otherwise.

The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units; can be located in one place or distributed to a plurality of network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated in one processing unit, or each unit may be separately used as one unit, or two or more units may be integrated in one unit; the integrated units may be implemented in hardware or in hardware plus software functional units.

Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the above method embodiments may be implemented by hardware related to program instructions, and the foregoing program may be stored in a readable storage medium, where the program, when executed, performs steps including the above method embodiments; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk or an optical disk, or the like, which can store program codes.

Alternatively, the above-described integrated units of the present invention may be stored in a readable storage medium if implemented in the form of software functional modules and sold or used as separate products. Based on such understanding, the technical solution of the embodiments of the present invention may be embodied in essence or a part contributing to the prior art in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, ROM, RAM, magnetic or optical disk, or other medium capable of storing program code.

Claims

1. The image labeling method based on crowdsourcing is characterized by comprising the following steps of:

2. The crowd-sourced picture labeling method according to claim 1, wherein the obtaining the picture to be labeled, performing information pickup and identification on the picture to obtain picture integrity information and picture category information, performing integral quality grading on the picture through a preset crowd-sourced platform to obtain picture quality grades, and performing attribute classification on the picture to obtain picture attribute category information comprises:

acquiring a picture to be marked;

3. The crowd-sourced based picture labeling method of claim 2, wherein the performing picture pairing processing according to the picture attribute category information to obtain an adapted professional crowd-sourced group and corresponding professional labeling difficulty coefficients, performing recognition processing on pictures by the professional crowd-sourced group to obtain a plurality of to-be-filled picture features and corresponding plurality of picture feature void ratios, and performing processing to obtain a picture void ratio average index, comprises:

4. The crowd-sourced picture marking method according to claim 3, wherein the step of respectively performing picture feature classification and marking on the pictures by the crowd-sourced personnel of the professional crowd-sourced group to obtain picture feature marking results comprises the steps of classifying each piece of marking information obtained by marking, obtaining each circle of marking frequency, and respectively screening key correct picture features, high abnormal picture features and key error picture features by threshold comparison according to each circle of marking frequency, and comprises the following steps:

5. The crowd-sourced based picture marking method according to claim 4, wherein the verifying and identifying each key correct picture feature and each key incorrect picture feature to obtain real feature quantity and corresponding feature marking rate data, and performing threshold comparison with a preset feature marking rate threshold to obtain a corresponding marking threshold comparison result, and if the results all meet preset requirements, identifying each high-anomaly picture feature to obtain real anomaly feature quantity and corresponding anomaly feature marking rate data, and performing threshold comparison with a preset anomaly feature marking rate threshold to obtain a third marking threshold comparison result, includes:

6. The crowd-sourced picture marking method according to claim 5, wherein if the third marking threshold comparison result meets a preset threshold comparison requirement, acquiring effective marking rate data of each classification feature and average effective marking rate data of each classification feature of the picture feature ring annotation result of each crowd-sourced person of the professional crowd-sourced group, screening effective crowd-sourced marking persons meeting the average effective marking rate data of each classification feature, and extracting corresponding historical effective ring annotation rate data, comprises:

7. The crowd-sourced based picture marking method according to claim 6, wherein the processing according to the historical effective crowd-sourced marking rate data of each effective crowd-sourced marking person and the picture quality grade, the professional marking difficulty coefficient and the picture vacancy rate average index to obtain the professional crowd-sourced marking effective correction coefficient, and comparing with a preset professional picture feature marking effect detection threshold value to judge the total crowd-sourced marking effectiveness of each effective crowd-sourced marking person, comprises:

8. The crowd-sourced based picture marking method of claim 7, wherein extracting the plurality of to-be-filled picture features and the plurality of key error picture features of the effective crowd-sourced marking personnel if the total coil is valid, filling and correcting the picture to obtain a repaired picture, comparing the repaired picture with a standardized processing picture to obtain a picture repairing effect result, and updating records of each effective crowd-sourced marking personnel, comprises:

9. A crowd-sourcing-based picture annotation system, comprising: the system comprises a memory and a processor, wherein the memory comprises a program based on a crowdsourcing picture marking method, and the program based on the crowdsourcing picture marking method realizes the following steps when being executed by the processor:

10. A computer readable storage medium, wherein a crowdsourcing based picture annotation method program is included in the computer readable storage medium, which when executed by a processor, implements the steps of the crowdsourcing based picture annotation method of any of claims 1 to 8.