CN112422312A - Crowdsourcing-based industrial Internet system log processing method - Google Patents

Crowdsourcing-based industrial Internet system log processing method Download PDF

Info

Publication number
CN112422312A
CN112422312A CN202011047873.XA CN202011047873A CN112422312A CN 112422312 A CN112422312 A CN 112422312A CN 202011047873 A CN202011047873 A CN 202011047873A CN 112422312 A CN112422312 A CN 112422312A
Authority
CN
China
Prior art keywords
task
log
answer
crowdsourcing
answers
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011047873.XA
Other languages
Chinese (zh)
Other versions
CN112422312B (en
Inventor
卓小军
石宇
张天江
孙崇曦
于宪徵
乔少杰
徐天承
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sichuan Ninegate Technology Co ltd
Original Assignee
Sichuan Ninegate Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sichuan Ninegate Technology Co ltd filed Critical Sichuan Ninegate Technology Co ltd
Priority to CN202011047873.XA priority Critical patent/CN112422312B/en
Publication of CN112422312A publication Critical patent/CN112422312A/en
Application granted granted Critical
Publication of CN112422312B publication Critical patent/CN112422312B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06—Management of faults, events, alarms or notifications
    • H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/50—Network service management, e.g. ensuring proper service fulfilment according to agreements
    • H04L41/5003—Managing SLA; Interaction between SLA and QoS
    • H04L41/5009—Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention discloses a crowdsourcing-based industrial Internet system log processing method, which comprises the following steps of: the method comprises the steps that a task requester collects system logs and abnormal information and publishes crowdsourcing tasks, each task is divided into three parts, and after crowdsourcing workers return task answers, quality of the crowdsourcing workers is graded according to a task answer set and the task answers are graded; obtaining answer credibility through crowdsourcing worker quality scores and task answer quality scores; and sequencing the credibility of the answers from large to small, taking out the task answers corresponding to the credibility of the first N answers, and returning the task answers to the task requester to complete the log processing of the industrial Internet system. The invention carries out modeling scoring on the answers returned by the workers, can accurately predict the answer quality, solves the problem of answer preference and can improve the efficiency of follow-up system maintainers.

Description

Crowdsourcing-based industrial Internet system log processing method
Technical Field
The invention belongs to the field of industrial internet comprehensive management systems, and particularly relates to a crowdsourcing-based industrial internet system log processing method.
Background
The development of the smart city can be divided into the construction of information infrastructure and the construction of a digital city, and the information infrastructure and the digital city are organically integrated through the Internet of things. Smart cities emphasize the orchestration and coordination of city management and the integration and sharing of city information. The industrial internet is used as a global industrial system and is connected and fused with advanced computing, analyzing and sensing technologies and the internet, and plays an important role in the process of fusing information infrastructure and digital cities by the internet of things. The industrial internet system has the characteristics of a traditional information system, the log system is one of important components of the industrial internet system, in the aspect of safety audit, the performance of the system can be optimized by debugging the information recorded by the log system, and the stability and reliability of the system are guaranteed by adjusting the behavior of the system according to the log information; meanwhile, the log service also provides convenience for system abnormity and error correction.
At present, it is difficult to realize the correct analysis and treatment of the log file by the computer itself through programming (the log with the exception can be roughly selected, but the reason of the exception is difficult to determine). The traditional industrial Internet system log management method is that a specially-assigned person is hired to check abnormal logs regularly to ensure the safety and stability of the system, or the system logs are checked after the system fails, and the reason of the system failure is traced. Often, such methods require a high level of system-related knowledge from the individual employed, which may require multiple experts to be employed for log-screening maintenance, and may require a significant amount of manpower and material resources. Moreover, if the method of employing experts is adopted, since the experts have difficulty in processing abnormal logs all day long, it is difficult to ensure that the system can be immediately solved when a fault occurs.
The problem of exception log processing can be solved by using a crowdsourcing technology, but several challenges related to the crowdsourcing technology in the prior art are as follows: 1. how to decompose a complex task into small-granularity tasks which can be completed by individuals so that the complex task can be processed by a crowdsourcing network; 2. how to issue complex tasks to a crowdsourcing platform in a simple and easy way, so that crowdsourcing workers can easily know the tasks; 3. how to screen the answers of the tasks and improve the completion quality of the tasks.
Disclosure of Invention
Aiming at the defects in the prior art, the problem in the prior art is solved by the crowdsourcing-based industrial Internet system log processing method.
In order to achieve the purpose of the invention, the invention adopts the technical scheme that: a log processing method of an industrial Internet system based on crowdsourcing comprises the following steps:
s1, uploading task information to a crowdsourcing platform through a task requester, and issuing a task to obtain a task group X;
s2, decomposing the tasks in the task group X into three parts, waiting for crowdsourcing workers to answer the decomposed tasks, and transmitting the obtained task answers to a crowdsourcing platform;
s3, scoring crowdsourcing workers through an abnormal log selection part and an abnormal log sorting part in the task answers, and obtaining quality scores of the crowdsourcing workers according to the two scores;
s4, scoring the abnormal log selection part and the abnormal log sorting part in the task answers, obtaining the quality score of the task answers according to the two scores, and calculating the answer credibility through the quality score of crowdsourcing workers and the quality score of the task answers;
and S5, sequencing the answer credibility from big to small, taking out the task answers corresponding to the first N answer credibility, and returning the task answers to the task requester to complete the log processing of the industrial Internet system.
Further, the tasks in step S1 include a maintenance task and an abnormal task, where the maintenance task is a periodic system log maintenance detection task, and the abnormal task is a task for tracing a system fault reason through a log file when the system is abnormal;
the task information in the step S1 includes detailed description of the task, task reward, task-related file, task return answer format, and task deadline.
Further, the step S2 is to decompose the task in the task group X into three parts, specifically: the first part is an abnormal log selection part which screens out a set number of abnormal log files from the log files of the tasks in the task group X; the second part is an abnormal log sorting part which sorts the screened abnormal log files according to the importance of the files; and the third part is an abnormal log analysis part which analyzes the reason of the system abnormality according to the sorted abnormal log files.
Further, the step S3 includes the following sub-steps:
s31, according to the task answers, constructing a Gaussian distribution model x of the task corresponding to the abnormal log filej～N(μj,ηj) Wherein x isjDenotes the jth task, xj～N(μj,ηj) Denotes the jth task xjObedience parameter mujAnd ηjGaussian distribution of (u)jMathematical expectation, η, representing a Gaussian distribution modeljStandard deviation of gaussian distribution model, J1, 2.. and J, J represents total number of tasks;
s32 passing through Gaussian distribution model xj～N(μj,ηj) Obtaining a log number vector cijProbability value of occurrence cdf (c)ij,N(μj,ηj))；
S33, judging the mathematical expectation mu of the Gaussian distribution modeljWhether or not it is greater than the log number vector cijIf so, the probability value cdf (c)ij,N(μj,ηj) Excellence as a task answer, otherwise 1-cdf (c)ij,N(μj,ηj) Excellent degree as a task answer;
s34, obtaining scores A of crowdsourcing workers on abnormal log selection parts through the excellence degree of task answers;
s35, aggregating the task answers to obtain an aggregated log sorting answer
Figure BDA0002708559960000031
Wherein t represents that the log sorting answer is a correct answer obtained by aggregating all answers;
s36, sorting partial answer p according to abnormal log in task answerijComputing aggregated log ranking answers
Figure BDA0002708559960000032
And exception log sorting partial answers pijDegree of approximation of
Figure BDA0002708559960000033
S37 passing the degree of approximation
Figure BDA0002708559960000034
Acquiring a score B of a crowdsourcing worker on an abnormal log sorting part;
s38, obtaining the quality score xi of crowdsourcing workers according to the score A and the score Bw。
Further, the mathematical expectation μ of the gaussian distribution model in the step S31jAnd standard deviation ηjThe method specifically comprises the following steps:
Figure BDA0002708559960000041
wherein, cijRepresenting crowdsourcing workers wiFor task xjA log number vector returned with respect to the abnormal log selection part, I ═ 1,2jRepresentation represents all crowdsourced workers to task xjLog number vector set for partial answer to exception log selection, | CjI represents CjThe superscript T represents transposition;
probability value cdf (c) in said step S32ij,N(μj,ηj) Specifically, the following are:
Figure BDA0002708559960000042
where cdf () represents the cumulative distribution function, | ηj| represents ηjD represents a differential, cij (n)Representing a log number vector cijIs given as 1,2, q, q denotes a log number vector cijIntegral may represent multiple integral symbols, exp represents an exponential function with e as the base,
Figure BDA0002708559960000043
representing a vector c with a journal numberijIntegrating the 1 to q dimensions;
the quality score a in step S34 is specifically:
Figure BDA0002708559960000044
wherein the content of the first and second substances,
Figure BDA0002708559960000045
a collection of answers representing the exception log selection portion of task j,
Figure BDA0002708559960000046
to represent
Figure BDA0002708559960000047
The number of elements of (c).
Further, the degree of approximation in the step S36
Figure BDA0002708559960000048
The method specifically comprises the following steps:
Figure BDA0002708559960000051
wherein p isijRepresenting crowdsourcing workers wiPair crowdsourcing task xjThe exception log sorting section answer for the exception log sorting section,
Figure BDA0002708559960000052
representation collection
Figure BDA0002708559960000053
The number of elements in (1);
the quality score B in step S37 is specifically:
Figure BDA0002708559960000054
wherein cs () represents a similarity calculation function,
Figure BDA0002708559960000055
represents pijAnd
Figure BDA0002708559960000056
the degree of similarity of (a) to (b),
Figure BDA0002708559960000057
representing a task xjThe answer set of the exception log sorting portion,
Figure BDA0002708559960000058
representation collection
Figure BDA0002708559960000059
The number of elements in (1);
credibility score ξ of crowdsourcing workers in the step S38wThe method specifically comprises the following steps:
ξw＝τA+(1-τ)B
where τ represents the weight of the anomaly log selection section.
Further, the step S4 includes the following sub-steps:
s41, mathematical expectation μ according to the Gaussian distribution model in step S31jAnd standard deviation ηjObtaining the log number vector c of the abnormal log selection partijScore of (3) C;
s42, approximating the degree
Figure BDA00027085599600000510
Score D as part of the anomaly log ordering;
s43, obtaining quality score xi of task answer through score C and score Dt；
S44, quality score xi by crowdsourcing workerwAnd quality score ξ of task answerstAnd obtaining the credibility E of the answer.
Further, the score C in step S41 specifically is:
Figure BDA00027085599600000511
wherein pdf () represents the probability density function, pdf (c)ij,N(μj,ηj) Is represented by c)ijIn N (mu)j,ηj) The probability density of (1);
answer score xi in said step S43tThe method specifically comprises the following steps:
ξt＝τC+(1-τ)D
the answer confidence E in step S44 is specifically:
Figure BDA0002708559960000061
the invention has the beneficial effects that:
(1) the invention reduces the system cost by separating tasks, screening answers and solving problems in a crowdsourcing mode, wherein the tasks are processed in a traditional employment and contract mode.
(2) According to the task issuing and answer submitting mode of the crowdsourcing platform and the different log task modes required by the industrial Internet system, the log task is divided into a plurality of parts with different weights, the crowdsourcing worker can conveniently answer, and the answer quality can be conveniently evaluated.
(3) According to the method, each log is taken as one dimension of the vector, the step of selecting the abnormal log by the worker is graded by combining the multidimensional Gaussian distribution model, and compared with the mode of coding a plurality of logs and establishing one-dimensional Gaussian distribution, the quality grade of the part of answers of the worker can be obtained more accurately, so that the answer reliability is more reliable.
(4) According to the invention, modeling scoring is carried out on the answers returned by the workers, the quality scores of the workers and the quality scores of the answers are calculated by combining the models, the quality of the answers can be accurately estimated, the problem of answer preference is solved, and the efficiency of subsequent system maintainers can be improved.
Drawings
Fig. 1 is a flowchart of a method for processing logs in an industrial internet system based on crowdsourcing according to the present invention.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate the understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and it will be apparent to those skilled in the art that various changes may be made without departing from the spirit and scope of the invention as defined and defined in the appended claims, and all matters produced by the invention using the inventive concept are protected.
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
As shown in fig. 1, a method for processing logs in an industrial internet system based on crowdsourcing includes the following steps:
s1, uploading task information to a crowdsourcing platform through a task requester, and issuing a task to obtain a task group X;
s2, decomposing the tasks in the task group X into three parts, waiting for crowdsourcing workers to answer the decomposed tasks, and transmitting the obtained task answers to a crowdsourcing platform;
s3, scoring crowdsourcing workers through an abnormal log selection part and an abnormal log sorting part in the task answers, and obtaining quality scores of the crowdsourcing workers according to the two scores;
s4, scoring the abnormal log selection part and the abnormal log sorting part in the task answers, obtaining the quality score of the task answers according to the two scores, and calculating the answer credibility through the quality score of crowdsourcing workers and the quality score of the task answers;
and S5, sequencing the answer credibility from big to small, taking out the task answers corresponding to the first N answer credibility, and returning the task answers to the task requester to complete the log processing of the industrial Internet system.
The tasks in the step S1 include a maintenance task and an abnormal task, where the maintenance task is a periodic system log maintenance detection task, and the abnormal task is a task for tracing a system fault reason through a log file when the system is abnormal;
the task information in the step S1 includes detailed description of the task, task reward, task-related file, task return answer format, and task deadline.
In this embodiment, the task-related file is a log file set, and the log file set is specifically shown in table 1.
TABLE 1
Figure BDA0002708559960000071
Figure BDA0002708559960000081
In step S2, the task in the task group X is decomposed into three parts: the first part is an abnormal log selection part which screens out a set number of abnormal log files from the log files of the tasks in the task group X; the second part is an abnormal log sorting part which sorts the screened abnormal log files according to the importance of the files; and the third part is an abnormal log analysis part which analyzes the reason of the system abnormality according to the sorted abnormal log files.
In this embodiment, task group X contains task X1、x2And x3Specifically, the results are shown in Table 2.
TABLE 2
Abnormal state Exception logs actually associated therewith
x1 E1010001 O1O5O4O3
x2 E1010005 O6O5O1O3
x3 T0000001 O7O3O6O10
Crowd-sourcing worker to task x1、x2And x3The task decomposition conditions of (1) are shown in table 3.
TABLE 3
Figure BDA0002708559960000082
Figure BDA0002708559960000091
The step S3 includes the following sub-steps:
s31, according to the task answers, constructing a Gaussian distribution model x of the task corresponding to the abnormal log filej～N(μj,ηj) Wherein x isjDenotes the jth task, xj～N(μj,ηj) Denotes the jth task xjObedience parameter mujAnd ηjGaussian distribution of (u)jMathematical expectation, η, representing a Gaussian distribution modeljStandard deviation of gaussian distribution model, J1, 2.. and J, J represents total number of tasks;
s32 passing through Gaussian distribution model xj～N(μj,ηj) Obtaining a log number vector cijProbability value of occurrence cdf (c)ij,N(μj,ηj))；
S33, judging the mathematical expectation mu of the Gaussian distribution modeljWhether or not it is greater than the log number vector cijIf so, the probability value cdf (c)ij,N(μj,ηj) Excellence as a task answer, otherwise 1-cdf (c)ij,N(μj,ηj) Excellent degree as a task answer;
s34, obtaining scores A of crowdsourcing workers on abnormal log selection parts through the excellence degree of task answers;
s35, aggregating the task answers to obtain an aggregated log sorting answer
Figure BDA0002708559960000092
Wherein t represents that the log sorting answer is a correct answer obtained by aggregating all answers;
s36, sorting partial answer p according to abnormal log in task answerijComputing aggregated log ranking answers
Figure BDA0002708559960000093
And exception log sorting partial answers pijDegree of approximation of
Figure BDA0002708559960000094
S37, passing approximation programDegree of rotation
Figure BDA0002708559960000095
Acquiring a score B of a crowdsourcing worker on an abnormal log sorting part;
s38, obtaining the quality score xi of crowdsourcing workers according to the score A and the score Bw。
Mathematical expectation μ of the gaussian distribution model in said step S31jAnd standard deviation ηjThe method specifically comprises the following steps:
Figure BDA0002708559960000101
wherein, cijRepresenting crowdsourcing workers wiFor task xjA log number vector returned with respect to the abnormal log selection part, I ═ 1,2jRepresentation represents all crowdsourced workers to task xjLog number vector set for partial answer to exception log selection, | CjI represents CjThe superscript T represents transposition;
probability value cdf (c) in said step S32ij,N(μj,ηj) Specifically, the following are:
Figure BDA0002708559960000102
where cdf () represents the cumulative distribution function, | ηj| represents ηjD represents a differential, cij (n)Representing a log number vector cijIs given as 1,2, q, q denotes a log number vector cijIntegral may represent multiple integral symbols, exp represents an exponential function with e as the base,
Figure BDA0002708559960000103
representing a vector c with a journal numberijIntegrating the 1 to q dimensions;
the quality score a in step S34 is specifically:
Figure BDA0002708559960000104
wherein the content of the first and second substances,
Figure BDA0002708559960000105
a collection of answers representing the exception log selection portion of task j,
Figure BDA0002708559960000106
to represent
Figure BDA0002708559960000107
The number of elements of (c).
The degree of approximation in said step S36
Figure BDA0002708559960000108
The method specifically comprises the following steps:
Figure BDA0002708559960000109
wherein p isijRepresenting crowdsourcing workers wiPair crowdsourcing task xjThe exception log sorting section answer for the exception log sorting section,
Figure BDA0002708559960000111
representation collection
Figure BDA0002708559960000112
The number of elements in (1);
the quality score B in step S37 is specifically:
Figure BDA0002708559960000113
wherein cs () represents a similarity calculation function,
Figure BDA0002708559960000114
represents pijAnd
Figure BDA0002708559960000115
the degree of similarity of (a) to (b),
Figure BDA0002708559960000116
representing a task xjThe answer set of the exception log sorting portion,
Figure BDA0002708559960000117
representation collection
Figure BDA0002708559960000118
The number of elements in (1);
credibility score ξ of crowdsourcing workers in the step S38wThe method specifically comprises the following steps:
ξw＝τA+(1-τ)B
where τ represents the weight of the anomaly log selection section.
The step S4 includes the following sub-steps:
s41, mathematical expectation μ according to the Gaussian distribution model in step S31jAnd standard deviation ηjObtaining the log number vector c of the abnormal log selection partijScore of (3) C;
s42, approximating the degree
Figure BDA0002708559960000119
Score D as part of the anomaly log ordering;
s43, obtaining quality score xi of task answer through score C and score Dt；
S44, quality score xi by crowdsourcing workerwAnd quality score ξ of task answerstAnd obtaining the credibility E of the answer.
The score C in step S41 is specifically:
Figure BDA00027085599600001110
wherein pdf () represents the probability density functionNumber, pdf (c)ij,N(μj,ηj) Is represented by c)ijIn N (mu)j,ηj) The probability density of (1);
answer score xi in said step S43tThe method specifically comprises the following steps:
ξt＝τC+(1-τ)D
the answer confidence E in step S44 is specifically:
Figure BDA0002708559960000121

Claims (8)

1. a log processing method of an industrial Internet system based on crowdsourcing is characterized by comprising the following steps:
s1, uploading task information to a crowdsourcing platform through a task requester, and issuing a task to obtain a task group X;
s2, decomposing the tasks in the task group X into three parts, waiting for crowdsourcing workers to answer the decomposed tasks, and transmitting the obtained task answers to a crowdsourcing platform;
s3, scoring crowdsourcing workers through an abnormal log selection part and an abnormal log sorting part in the task answers, and obtaining quality scores of the crowdsourcing workers according to the two scores;
s4, scoring the abnormal log selection part and the abnormal log sorting part in the task answers, obtaining the quality score of the task answers according to the two scores, and calculating the answer credibility through the quality score of crowdsourcing workers and the quality score of the task answers;
and S5, sequencing the answer credibility from big to small, taking out the task answers corresponding to the first N answer credibility, and returning the task answers to the task requester to complete the log processing of the industrial Internet system.
2. The method for processing logs in the crowdsourcing-based industrial internet system according to claim 1, wherein the tasks in the step S1 include a maintenance task and an exception task, the maintenance task is a periodic system log maintenance detection task, and the exception task is a task for tracing back a system fault cause through a log file when an exception occurs in the system;
the task information in the step S1 includes detailed description of the task, task reward, task-related file, task return answer format, and task deadline.
3. The method for processing logs in the crowdsourcing-based industrial internet system according to claim 1, wherein the step S2 is to decompose the tasks in the task group X into three parts: the first part is an abnormal log selection part which screens out a set number of abnormal log files from the log files of the tasks in the task group X; the second part is an abnormal log sorting part which sorts the screened abnormal log files according to the importance of the files; and the third part is an abnormal log analysis part which analyzes the reason of the system abnormality according to the sorted abnormal log files.
4. The crowdsourcing-based industrial internet system log processing method according to claim 3, wherein the step S3 comprises the substeps of:
s31, according to the task answers, constructing a Gaussian distribution model x of the task corresponding to the abnormal log filej～N(μj,ηj) Wherein x isjDenotes the jth task, xj～N(μj,ηj) Denotes the jth task xjObedience parameter mujAnd ηjGaussian distribution of (u)jMathematical expectation, η, representing a Gaussian distribution modeljStandard deviation of gaussian distribution model, J1, 2.. and J, J represents total number of tasks;
s32 passing through Gaussian distribution model xj～N(μj,ηj) Obtaining a log number vector cijProbability value of occurrence cdf (c)ij,N(μj,ηj))；
S33, judging the mathematical expectation mu of the Gaussian distribution modeljWhether or not it is greater than the log number vector cijIf yes, the probability value is determinedcdf(cij,N(μj,ηj) Excellence as a task answer, otherwise 1-cdf (c)ij,N(μj,ηj) Excellent degree as a task answer;
s34, obtaining scores A of crowdsourcing workers on abnormal log selection parts through the excellence degree of task answers;
s35, aggregating the task answers to obtain an aggregated log sorting answer
Figure FDA0002708559950000021
Wherein t represents that the log sorting answer is a correct answer obtained by aggregation;
s36, sorting partial answer p according to abnormal log in task answerijComputing aggregated log ranking answers
Figure FDA0002708559950000022
And exception log sorting partial answers pijDegree of approximation of
Figure FDA0002708559950000023
S37 passing the degree of approximation
Figure FDA0002708559950000024
Acquiring a score B of a crowdsourcing worker on an abnormal log sorting part;
s38, obtaining the quality score xi of crowdsourcing workers according to the score A and the score Bw。
5. The crowd-sourced industrial internet system log processing method based on claim 4, wherein the mathematical expectation μ of the gaussian distribution model in the step S31jAnd standard deviation ηjThe method specifically comprises the following steps:
Figure FDA0002708559950000031
wherein, cijRepresenting crowdsourcing workers wiFor task xjA log number vector returned with respect to the abnormal log selection part, I ═ 1,2jRepresentation represents all crowdsourced workers to task xjLog number vector set for partial answer to exception log selection, | CjI represents CjThe superscript T represents transposition;
probability value cdf (c) in said step S32ij,N(μj,ηj) Specifically, the following are:
Figure FDA0002708559950000032
where cdf () represents the cumulative distribution function, | ηj| represents ηjD represents a differential, cij (n)Representing a log number vector cijIs given as 1,2, q, q denotes a log number vector cijIntegral may represent multiple integral symbols, exp represents an exponential function with e as the base,
Figure FDA0002708559950000033
Figure FDA0002708559950000034
representing a vector c with a journal numberijIntegrating the 1 to q dimensions;
the quality score a in step S34 is specifically:
Figure FDA0002708559950000035
wherein the content of the first and second substances,
Figure FDA0002708559950000036
a collection of answers representing the exception log selection portion of task j,
Figure FDA0002708559950000037
to represent
Figure FDA0002708559950000038
The number of elements of (c).
6. The crowd-sourced industrial internet system log processing method according to claim 5, wherein the degree of approximation in the step S36 is
Figure FDA0002708559950000039
The method specifically comprises the following steps:
Figure FDA00027085599500000310
wherein p isijRepresenting crowdsourcing workers wiPair crowdsourcing task xjThe exception log sorting section answer for the exception log sorting section,
Figure FDA0002708559950000041
representation collection
Figure FDA0002708559950000042
The number of elements in (1);
the quality score B in step S37 is specifically:
Figure FDA0002708559950000043
wherein cs () represents a similarity calculation function,
Figure FDA0002708559950000044
represents pijAnd
Figure FDA0002708559950000045
the degree of similarity of (a) to (b),
Figure FDA0002708559950000046
representing a task xjThe answer set of the exception log sorting portion,
Figure FDA0002708559950000047
representation collection
Figure FDA0002708559950000048
The number of elements in (1);
credibility score ξ of crowdsourcing workers in the step S38wThe method specifically comprises the following steps:
ξw＝τA+(1-τ)B
where τ represents the weight of the anomaly log selection section.
7. The crowdsourcing-based industrial internet system log processing method according to claim 6, wherein the step S4 comprises the following substeps:
s41, mathematical expectation μ according to the Gaussian distribution model in step S31jAnd standard deviation ηjObtaining the log number vector c of the abnormal log selection partijScore of (3) C;
s42, approximating the degree
Figure FDA0002708559950000049
Score D as part of the anomaly log ordering;
s43, obtaining quality score xi of task answer through score C and score Dt；
S44, quality score xi by crowdsourcing workerwAnd quality score ξ of task answerstAnd obtaining the credibility E of the answer.
8. The method for processing logs of the crowdsourcing-based industrial internet system according to claim 7, wherein the specific calculation method of the score C in the step S41 is as follows:
Figure FDA00027085599500000410
wherein pdf () represents the probability density function, pdf (c)ij,N(μj,ηj) Is represented by c)ijIn N (mu)j,ηj) The probability density of (1);
answer score xi in said step S43tThe method specifically comprises the following steps:
ξt＝τC+(1-τ)D
the answer confidence E in step S44 is specifically:
Figure FDA0002708559950000051
CN202011047873.XA 2020-09-29 2020-09-29 Crowdsourcing-based industrial Internet system log processing method Active CN112422312B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011047873.XA CN112422312B (en) 2020-09-29 2020-09-29 Crowdsourcing-based industrial Internet system log processing method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011047873.XA CN112422312B (en) 2020-09-29 2020-09-29 Crowdsourcing-based industrial Internet system log processing method

Publications (2)

Publication Number Publication Date
CN112422312A true CN112422312A (en) 2021-02-26
CN112422312B CN112422312B (en) 2022-08-05

Family

ID=74854291

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011047873.XA Active CN112422312B (en) 2020-09-29 2020-09-29 Crowdsourcing-based industrial Internet system log processing method

Country Status (1)

Country Link
CN (1) CN112422312B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150309988A1 (en) * 2014-04-29 2015-10-29 International Business Machines Corporation Evaluating Crowd Sourced Information Using Crowd Sourced Metadata
CN109582581A (en) * 2018-11-30 2019-04-05 平安科技（深圳）有限公司 A kind of result based on crowdsourcing task determines method and relevant device
CN110310028A (en) * 2019-06-25 2019-10-08 阿里巴巴集团控股有限公司 Method and apparatus for crowdsourcing

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150309988A1 (en) * 2014-04-29 2015-10-29 International Business Machines Corporation Evaluating Crowd Sourced Information Using Crowd Sourced Metadata
CN109582581A (en) * 2018-11-30 2019-04-05 平安科技（深圳）有限公司 A kind of result based on crowdsourcing task determines method and relevant device
CN110310028A (en) * 2019-06-25 2019-10-08 阿里巴巴集团控股有限公司 Method and apparatus for crowdsourcing

Also Published As

Publication number Publication date
CN112422312B (en) 2022-08-05

Similar Documents

Publication Publication Date Title
Debrah et al. Artificial intelligence in green building
US10846640B2 (en) Architecture, engineering and construction (AEC) risk analysis system and method
CN113537807B (en) Intelligent wind control method and equipment for enterprises
CN113807747A (en) Enterprise budget management maturity evaluation system
CN116955092B (en) Multimedia system monitoring method and system based on data analysis
CN114048436A (en) Construction method and construction device for forecasting enterprise financial data model
Banjongkan et al. A Study of Job Failure Prediction at Job Submit-State and Job Start-State in High-Performance Computing System: Using Decision Tree Algorithms [J]
WO2021103401A1 (en) Data object classification method and apparatus, computer device and storage medium
Iren et al. Cost of quality in crowdsourcing
CN114429265A (en) Enterprise portrait service construction method, device and equipment based on grid technology
CN112422312B (en) Crowdsourcing-based industrial Internet system log processing method
CN117495094A (en) Comprehensive evaluation and early warning method and system for safety risk of industrial chain
CN115204501A (en) Enterprise evaluation method and device, computer equipment and storage medium
CN115292167A (en) Life cycle prediction model construction method, device, equipment and readable storage medium
CN114510405A (en) Index data evaluation method, index data evaluation device, index data evaluation apparatus, storage medium, and program product
Fernández-Diego et al. Sensitivity of results to different data quality meta-data criteria in the sample selection of projects from the ISBSG dataset
Zhang et al. Does socio-technical congruence have an effect on continuous integration build failures? An empirical study on 10 GitHub projects
CN113610225A (en) Quality evaluation model training method and device, electronic equipment and storage medium
CN112395343A (en) DSG-based field change data acquisition and extraction method
CN111475319A (en) Hard disk screening method and device based on machine learning
CN112508400B (en) Self-generation method of crowdsourcing collaborative iteration task
CN116542380B (en) Power plant supply chain carbon footprint optimization method and device based on natural language
CN117592943A (en) Science and technology service data collaboration system based on Internet
CN112907124B (en) Data link abnormity evaluation method and device, electronic equipment and storage medium
WO2022227213A1 (en) Industry recommendation method and apparatus, computer device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant