CN116049679A - Big data acquisition strategy updating method and system for AI training sample service - Google Patents

Big data acquisition strategy updating method and system for AI training sample service Download PDF

Info

Publication number
CN116049679A
CN116049679A CN202310216287.0A CN202310216287A CN116049679A CN 116049679 A CN116049679 A CN 116049679A CN 202310216287 A CN202310216287 A CN 202310216287A CN 116049679 A CN116049679 A CN 116049679A
Authority
CN
China
Prior art keywords
target
acquisition
big data
noise
denoising
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN202310216287.0A
Other languages
Chinese (zh)
Inventor
张兴东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to CN202310216287.0A priority Critical patent/CN116049679A/en
Publication of CN116049679A publication Critical patent/CN116049679A/en
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00Machine learning
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The embodiment of the invention provides a big data acquisition strategy updating method and a big data acquisition strategy updating system for an AI training sample service, which are used for outputting denoising optimization basic data of an AI application service server aiming at an AI training sample service process based on acquisition noise point information acquired from a plurality of acquisition initial targets to a plurality of acquisition termination targets in an AI application service server, carrying out strategy updating reference on big data denoising strategy sequences of big data denoising applications in the AI application service server based on the denoising optimization basic data to obtain target big data denoising strategies of the big data denoising applications of the AI application service server corresponding to all noise sources, and carrying out big data acquisition strategy updating on the AI training sample service process according to the target big data denoising strategies.

Description

Big data acquisition strategy updating method and system for AI training sample service
The application is a divisional application of China application with the application number 202210571942X, the application date of 2022, the month of 05 and the 25 th, and the invention and creation name of an AI analysis output method for serving big data denoising optimization and an artificial intelligence system.
Technical Field
The invention relates to the technical field of big data, in particular to a big data acquisition strategy updating method and system for an AI training sample service.
Background
Big data processing is the extraction of massive and complex data value, and the most valuable is predictive analysis, namely, data scientists can be helped to better understand data through data visualization, statistical pattern recognition, data description and other data mining forms, predictive decisions can be obtained according to data mining results, or the method can be used in the AI training field at present, and AI training is carried out by collecting effective big data, so that a predictive decision model for predicting specific labels (such as user interest analysis, network behavior analysis, emotion semantic analysis and the like) is obtained. Therefore, the accuracy of big data acquisition plays a very critical role in the accuracy of subsequent data mining, and how to improve the decision accuracy of acquiring noise point information in the big data acquisition process, thereby improving the effectiveness of denoising the subsequent big data is a technical problem to be solved urgently at present.
Disclosure of Invention
In order to at least overcome the above-mentioned shortcomings in the prior art, the present invention aims to provide a method and a system for updating big data acquisition strategy for AI training sample service.
In a first aspect, an embodiment of the present invention provides an AI analysis output method for serving big data denoising optimization, applied to an artificial intelligence system, the method including:
acquiring big data acquisition activity data of an AI training sample service process of the big data acquisition server on a target training sample data service source and historical priori noise clue data on the target training sample data service source, wherein a model deployment application relation is preconfigured between the target training sample data service source and a target acquisition noise point decision model, the target training sample data service source is a training sample data service source in an AI application service server, and the big data acquisition activity data of the target training sample data service source represents activity cooperative relation information between a target big data acquisition activity cluster corresponding to the target training sample data service source and target big data acquisition activity in the target big data acquisition activity cluster;
loading big data acquisition activity data of the target training sample data service source and historical priori noise clue data on the target training sample data service source to the target acquisition noise point decision model, and determining target acquisition noise point information generated by the target acquisition noise point decision model;
And outputting acquisition noise point information from an initial target acquisition to a termination target acquisition of the target acquisition in the AI application service server based on the target acquisition noise point information, wherein an acquisition path from the initial target acquisition to the termination target acquisition of the target acquisition is optimized by using the target training sample data service source, and the acquisition noise point information is used for denoising big data.
In a second aspect, the embodiment of the invention further provides an AI analysis output system for serving big data denoising optimization, wherein the AI analysis output system for serving big data denoising optimization comprises an artificial intelligence system and a plurality of big data acquisition servers in communication connection with the artificial intelligence system;
the artificial intelligence system is used for:
acquiring big data acquisition activity data of an AI training sample service process of the big data acquisition server on a target training sample data service source and historical priori noise clue data on the target training sample data service source, wherein a model deployment application relation is preconfigured between the target training sample data service source and a target acquisition noise point decision model, the target training sample data service source is a training sample data service source in an AI application service server, and the big data acquisition activity data of the target training sample data service source represents activity cooperative relation information between a target big data acquisition activity cluster corresponding to the target training sample data service source and target big data acquisition activity in the target big data acquisition activity cluster;
Loading big data acquisition activity data of the target training sample data service source and historical priori noise clue data on the target training sample data service source to the target acquisition noise point decision model, and determining target acquisition noise point information generated by the target acquisition noise point decision model;
and outputting acquisition noise point information from an initial target acquisition to a termination target acquisition of the target acquisition in the AI application service server based on the target acquisition noise point information, wherein an acquisition path from the initial target acquisition to the termination target acquisition of the target acquisition is optimized by using the target training sample data service source, and the acquisition noise point information is used for denoising big data.
According to the embodiment scheme of any aspect, according to the big data acquisition activity data of the target training sample data service source and the historical priori noise clue data on the target training sample data service source, the big data acquisition activity data and the historical priori noise clue data are loaded to the target acquisition noise point decision model, target acquisition noise point information generated by the target acquisition noise point decision model is determined, based on the target acquisition noise point information, acquisition noise point information from initial target acquisition to termination target acquisition of target acquisition in the AI application service server is output, namely, according to the big data acquisition activity data of the target training sample data service source and the historical priori noise clue data of the target training sample data service source, the big data acquisition activity data and the historical priori noise clue data of the target training sample data service source are loaded to the acquisition noise point decision model, and then acquisition noise point information is determined, so that the decision accuracy of the acquisition noise point information is improved, and the big data denoising effectiveness is improved.
Drawings
FIG. 1 is a schematic flow chart of an AI analysis output method for large data denoising optimization according to an embodiment of the present invention;
FIG. 2 is a schematic block diagram of functional components of an artificial intelligence system for implementing the above-described AI analysis output method for large data denoising optimization according to an embodiment of the invention.
Detailed Description
The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented, for example, in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "includes" and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.
The architecture of the AI analysis output system 10 serving big data denoising optimization provided by an embodiment of the present invention is described below, and the AI analysis output system 10 serving big data denoising optimization may include an artificial intelligence system 100 and a big data acquisition server 200 communicatively connected with the artificial intelligence system 100. The artificial intelligence system 100 and the big data acquisition server 200 in the AI analysis output system 10 for big data denoising optimization may be used in conjunction with the AI analysis output method for big data denoising optimization described in the following method embodiments, and the detailed description of the method embodiments may be referred to for the execution steps of the artificial intelligence system 100 and the big data acquisition server 200.
The AI analysis output method for the big data denoising optimization according to the present embodiment may be executed by the artificial intelligence system 100, and is described in detail below with reference to fig. 1.
And a Process110, acquiring big data acquisition activity data of a target training sample data service source and historical priori noise clue data on the target training sample data service source, wherein the target training sample data service source and a target acquisition noise point decision model are preconfigured with a model deployment application relationship, the target training sample data service source is a training sample data service source in an AI application service server, and the big data acquisition activity data of the target training sample data service source represents activity coordination relationship information between a target big data acquisition activity cluster corresponding to the target training sample data service source and a target big data acquisition activity in the target big data acquisition activity cluster.
And the Process120 loads the big data acquisition activity data of the target training sample data service source and the historical priori noise clue data on the target training sample data service source to the target acquisition noise point decision model to determine target acquisition noise point information generated by the target acquisition noise point decision model.
And the Process130 outputs acquisition noise point information from the initial target acquisition to the termination target acquisition of the target acquisition in the AI application service server based on the target acquisition noise point information, and an acquisition path from the initial target acquisition to the termination target acquisition of the target acquisition passes through the target training sample data service source.
For some possible design considerations, the historical prior noise clue data on the target training sample data traffic source may include, but is not limited to, noise clue information that has been previously present and has been confirmed by the relevant developer for the target training sample data traffic source, which may include noise traffic field information, noise feature vector information, and the like.
For some possible design ideas, the pre-configuring the corresponding relationship between the target training sample data service source and the target collected noise point decision model may include, but is not limited to, configuring the collected noise point decision model for each training sample data service source in the AI application service server, or configuring the collected noise point decision model for a preset portion of the training sample data service sources in the AI application service server, where each training sample data service source corresponds to one collected noise point decision model.
For some possible design ideas, the activity coordination relationship information between the big data collection activities in the target big data collection activity cluster may include, but is not limited to, big data collection activities having an activity coordination relationship on the same big data collection activity, and the like.
For some possible design ideas, the collected noise point decision model may include, but is not limited to, model parameter layer tuning and selection of an initial collected noise point decision model based on an example training sample data service source, an example historical prior noise clue data and an example collected noise point information, and the collected noise point decision model may include, but is not limited to, a decision tree network model.
For some possible design ideas, the target acquisition noise point information may include, but is not limited to, acquisition noise point information of each big data acquisition activity in the target training sample data service source, may also include, but is not limited to, acquisition noise point information from any one big data acquisition activity to any other big data acquisition activity in the target training sample data service source, and may also include, but is not limited to, acquisition noise point information between different training sample data service sources.
For some possible design considerations, the acquisition initiation target may refer to a service field for functioning as a big data acquisition guide, and the acquisition termination target may refer to a service field for functioning as a big data acquisition termination.
According to the technical scheme, the big data acquisition activity data of the target training sample data service source and the historical priori noise clue data on the target training sample data service source are acquired, the big data acquisition activity data of the target training sample data service source and the historical priori noise clue data on the target training sample data service source are loaded to the target acquisition noise point decision model, the target acquisition noise point information generated by the target acquisition noise point decision model is determined, the acquisition noise point information from the initial target acquisition to the termination target acquisition of the target acquisition in the AI application service server is output based on the target acquisition noise point information, and the big data acquisition activity data of the target training sample data service source and the historical priori noise clue data of the target training sample data service source are acquired from the AI application service server and are loaded to the acquisition noise point decision model, so that the acquisition noise point information is determined, and the decision accuracy of the acquisition noise point information is improved, and the effectiveness of big data denoising is improved.
For some possible design ideas, for the Process120, the embodiment of the present invention provides an AI analysis output method for serving big data denoising optimization, which includes the following steps.
The Process210 outputs noise clue associated variables of target acquisition activity members included in the big data acquisition activity data and noise clue associated variables of target cooperative relationship categories included in the big data acquisition activity data based on the acquired big data acquisition activity data of the target training sample data service source and historical priori noise clue data on the target training sample data service source, wherein the target acquisition activity members and the target big data acquisition activity have characteristic mapping relationships, the target acquisition activity members represent a corresponding big data acquisition activity in the target big data acquisition activity cluster, the target cooperative relationship categories are connected with at least one pair of target acquisition activity members, two target big data acquisition activities corresponding to at least one pair of target acquisition activity members are represented to have cooperative relationships, and the target big data acquisition activity is the big data acquisition activity in the target big data acquisition activity cluster, and represents part of training sample data service sources in the target training sample data service source;
A Process220 for outputting noise transfer characteristics of the target acquisition activity member based on the noise cue correlation variable of the target acquisition activity member and the noise cue correlation variable of the target cooperative relationship class;
and a Process230 for outputting the target acquisition noise point information based on the noise transmission characteristics of the target acquisition active member.
For some possible design considerations, the noise cue correlation variable for the target acquisition activity member may include, but is not limited to, historical prior noise cue data for the target acquisition activity member, and the noise cue correlation variable for the target cooperative relationship class characterizes cooperative relationship labels among the plurality of target acquisition activity members.
For some possible design ideas, the noise transmission characteristics of the target acquisition activity member are obtained by the acquisition noise point decision model, and the noise transmission characteristics of the target acquisition activity member include information of cooperative acquisition activity members from the target acquisition activity member, and may also include, but are not limited to, information of cooperative acquisition activity members of different noise transmission decision nodes.
For some possible design considerations, this may include, but is not limited to, using iterative updating of noise transfer characteristics of acquisition activity members of the target set in accordance with the acquisition noise point decision model.
According to the technical scheme, big data acquisition activity data based on the acquired target training sample data service source and historical priori noise clue data on the target training sample data service source are adopted, noise clue related variables of target acquisition activity members included in the big data acquisition activity data and noise clue related variables of target cooperative relation categories included in the big data acquisition activity data are output, the target acquisition activity members and the target big data acquisition activity have characteristic mapping relation, the target acquisition activity members represent one big data acquisition activity corresponding to the target big data acquisition activity cluster, the target cooperative relation categories are connected with two target acquisition activity members, the two target big data acquisition activity corresponding to the two target acquisition activity members represent the big data acquisition activity in the target big data acquisition activity cluster, part of training sample data service sources in the target training sample data service source are represented, the noise transfer characteristics of the target acquisition activity members are output based on the noise clue related variables of the target acquisition activity members and the noise clue related variables of the target cooperative relation categories, the noise transfer characteristics of the target acquisition activity members are output, the noise transfer characteristics of the target acquisition activity members are information are represented, the noise point information of the target acquisition activity members are determined to be in a more cooperative relation with each other according to the target acquisition activity set, the information is more relevant to the target acquisition activity set, and the noise acquisition activity set can be more in a cooperative relation with the process of the target acquisition activity set, and the information is more in a better than the cooperative relation set of the local information acquisition activity set is obtained, and the decision capability is improved, and the accuracy of the information of the acquisition noise points of the big data acquisition activity is optimized and determined.
For some possible design ideas, the outputting the noise transfer characteristic of the target collection activity member based on the noise cue association variable of the target collection activity member and the noise cue association variable of the target cooperative relationship class includes:
and outputting the noise transmission characteristics of the traversing collecting active member on the next noise transmission decision node based on the noise clue related variable of the traversing collecting active member, the noise clue related variable of the collecting active member category cooperated with the traversing collecting active member, the noise clue related variable of the cooperated collecting active member of the traversing collecting active member, and the noise transmission characteristics of the cooperated collecting active member of the traversing collecting active member on the current noise transmission decision node, wherein the target collecting active member comprises the traversing collecting active member and the cooperated collecting active member of the traversing collecting active member.
For some possible design ideas, taking traversing collection activity members as collection activity members y, taking a current noise transmission decision node as a t noise transmission decision node and taking a next noise transmission decision node as a t+1 noise transmission decision node as an example, the noise transmission characteristics of the output collection activity members y on the t+1 noise transmission decision node can include, but are not limited to, the following:
Assuming that acquisition activity member 5 is acquisition activity member y, the updated function of its noise transfer characteristics includes, but is not limited to, the following equation:
e5=f(p5,p(3,5),p(5,6),e3,e6,p3,p6)
in the above formula, p5 represents a noise cue association variable of the acquisition activity member 5, p (3, 5) represents a noise cue association variable of the acquisition activity member category of the acquisition activity member 5 and the cooperative acquisition activity member 3 thereof, p (5, 6) represents a noise cue association variable of the acquisition activity member category of the acquisition activity member 5 and the cooperative acquisition activity member 6 thereof, p3, p6 respectively represents a noise cue association variable of the cooperative acquisition activity member 3 and the cooperative acquisition activity member 6, e3 and e6 respectively represent a noise transmission characteristic of the cooperative acquisition activity member 3 of the acquisition activity member 5 at a t-th noise transmission decision node and a noise transmission characteristic of the cooperative acquisition activity member 6 at a t-noise transmission decision node.
The method comprises the steps that a current noise transmission decision node is used as partial input to generate a next noise transmission decision node to acquire noise transmission characteristics of an active member y according to the noise transmission characteristics of the active member, the noise transmission characteristics of each active member change very little, characteristic flow in the whole training process tends to be stable, and thus, each active member acquires information of the active member.
According to the technical scheme, the noise clue related variable based on the traversing collecting active member, the noise clue related variable of the collecting active member category cooperated with the traversing collecting active member, the noise clue related variable of the cooperated collecting active member of the traversing collecting active member, the noise transfer characteristic of the cooperated collecting active member of the traversing collecting active member on the current noise transfer decision node are adopted, the noise transfer characteristic of the traversing collecting active member on the next noise transfer decision node is output, the noise clue related variable of the cooperated collecting active member of the collecting active member y and the noise transfer characteristic of the last noise transfer decision node of the cooperated collecting active member are input into a collecting noise point decision model, and the influence of the cooperated collecting active member of each collecting active member on the collecting active member is used as an input characteristic according to the noise transfer characteristic of each collecting active member in the collecting noise point decision model, so that the generated target collecting noise point information precision is higher.
For some possible design ideas, the outputting, based on the target acquisition noise point information, acquisition noise point information from a target acquisition initial target acquisition to a target acquisition termination target in the AI application service server includes:
When the target acquisition initial target is the acquisition initial target of the target training sample data service source and the target acquisition termination target is the acquisition termination target of the target training sample data service source, outputting acquisition noise point information from the acquisition initial target of the target training sample data service source to the acquisition termination target of the target training sample data service source as the target acquisition noise point information; and/or
When the target acquisition initial target is the acquisition initial target of the target big data acquisition activity and the target acquisition termination target is the acquisition termination target of the target big data acquisition activity, outputting acquisition noise point information from the acquisition initial target of the target big data acquisition activity to the acquisition termination target of the target big data acquisition activity as the acquisition noise point information of the target big data acquisition activity; and/or
When the target acquisition initial target is the acquisition initial targets of the large data acquisition activities and the target acquisition termination target is the acquisition termination target of the large data acquisition activities, acquiring noise point information from the acquisition initial targets of the large data acquisition activities to the acquisition termination targets of the large data acquisition activities is output as an acquisition noise point information sequence of the large data acquisition activities.
For some possible design ideas, the target acquisition noise point information includes a plurality of unit acquisition noise point information, the plurality of unit acquisition noise point information corresponds to a plurality of big data acquisition activities in the target big data acquisition activity cluster, and the plurality of big data acquisition activities are continuous big data acquisition activities may include, but are not limited to, outputting acquisition noise point information from an initial acquisition target of the plurality of big data acquisition activities to an acquisition termination target of the plurality of big data acquisition activities as an acquisition noise point information sequence equal to the plurality of big data acquisition activities, determining the acquisition noise point information of the big data acquisition activities according to a fused form based on different actual needs, for example, when the acquisition noise point information of the big data acquisition activities to be determined is the initial acquisition target of the target to the target acquisition termination target, acquiring the big data acquisition activities included from the initial acquisition target of the target to the target acquisition termination target, determining the acquisition noise point information of each big data acquisition activity included, and finally outputting the acquisition noise point information of the big data acquisition activities as the target acquisition noise point information sequence corresponding to each big data acquisition activity.
For some possible design ideas, the embodiment of the invention also provides an AI analysis output method for serving big data denoising optimization, which can comprise the following steps.
And a Process310, based on the sample big data acquisition activity data of the sample training sample data service source, the sample history priori noise clue data on the sample training sample data service source and the sample acquisition noise point information of the sample training sample data service source, performing tuning and selection of a model parameter layer on a sample acquisition noise point decision model, determining the target acquisition noise point decision model, and outputting the tuning and selection of the sample acquisition noise point decision model termination model parameter layer when the noise point decision cost value between the sample acquisition noise point information generated by the sample acquisition noise point decision model and the sample acquisition noise point information matches a target cost value condition, and outputting the tuning and selection of the termination model parameter layer and the sample acquisition noise point decision model as the target acquisition noise point decision model.
For some possible design ideas, the distribution mode of the sample big data collection activity data and the target training sample data service source is the same, that is, the training sample data service source in the AI application service server adopts historical prior noise clue data to construct a plurality of sample superfine training sample data service sources to form the sample training sample data service source.
For some possible design ideas, the sample prior noise clue data of the sample history and the sample collection noise point information of the target training sample data service source can be obtained from a database, or marked manually, so as to realize the optimization and selection of a model parameter layer of the sample collection noise point decision model and determine the target collection noise point decision model.
For some possible design ideas, the noise point decision cost value between the example collection noise point information and the example collection noise point information generated by the example collection noise point decision model matches the target cost value condition, which may include, but is not limited to, that a loss function value between the example collection noise point information and the example collection noise point information is less than or equal to a predetermined loss function value, or that a ratio between the example collection noise point information and the example collection noise point information is less than or equal to a predetermined loss function value, the noise point decision cost value is considered to be converged, that is, the example collection noise point decision model is output as the target collection noise point decision model.
For some possible design ideas, linear clustering based on multiple noise point decision cost values (may be weighted appropriately in a specific application process) may be included, but not limited to, to improve the generalization ability of the collected noise point decision model.
According to the technical scheme, the model parameter layer adjustment and selection are carried out on the sample acquisition noise point decision model by adopting sample big data acquisition activity data based on the sample training sample data service source, sample prior noise clue data on the sample training sample data service source and sample acquisition noise point information of the sample training sample data service source, the target acquisition noise point decision model is determined, when the noise point decision cost value between the sample acquisition noise point information generated by the sample acquisition noise point decision model and the sample acquisition noise point information is matched with the target cost value condition, model adjustment on the sample acquisition noise point decision model is stopped, the sample acquisition noise point decision model when training is stopped is output as the target acquisition noise point decision model, so that the acquisition noise point decision model is obtained, and then the acquisition noise point information is determined, so that the decision accuracy of the acquisition noise point information is improved, and the effectiveness of big data denoising is improved.
For some possible design considerations, the method further comprises: and calculating characteristic loss of the target acquisition noise point information and the sample acquisition noise point information of the target training sample data service source so as to perform tuning and selection of a modulus parameter layer on the target acquisition noise point decision model, and performing tuning and selection of the modulus parameter layer on the target acquisition noise point decision model when the noise point decision cost value between the target acquisition noise point information and the sample acquisition noise point information is matched with a target cost value condition.
For some possible design ideas, the method may further include, but is not limited to, performing feature loss calculation on the target acquisition noise point information and the sample acquisition noise point information to update part of model weight parameters in the target acquisition noise point decision model, so as to realize tuning and selection of a modulus parameter layer of the target acquisition noise point decision model.
For some possible design considerations, the method further comprises:
acquiring big data acquisition activity data of a derivative training sample data service source and historical priori noise clue data on the derivative training sample data service source, wherein a model deployment application relation is preconfigured between the derivative training sample data service source and a derivative acquisition noise point decision model, and the big data acquisition activity data of the derivative training sample data service source represents activity cooperative relation information between a derivative big data acquisition activity cluster corresponding to the derivative training sample data service source and big data acquisition activities in the derivative big data acquisition activity cluster;
loading big data acquisition activity data of the derivative training sample data service source and historical priori noise clue data on the derivative training sample data service source to the derivative acquisition noise point decision model, and outputting derivative acquisition noise point information generated by the derivative acquisition noise point decision model;
The outputting, based on the target acquisition noise point information, acquisition noise point information from the initial target acquisition to the termination target acquisition of the target acquisition in the AI application service server, includes: and outputting acquisition noise point information from an initial target acquisition to a target acquisition termination target in the AI application service server based on the target acquisition noise point information and the derivative acquisition noise point information, wherein an acquisition path from the initial target acquisition to the target acquisition termination target is transmitted through the target training sample data service source, and the acquisition noise point information serves big data denoising optimization and the derivative training sample data service source.
For some possible design considerations, the derived training sample data traffic source may include, but is not limited to, each training sample data traffic source in the set of training sample data traffic sources similar to the target training sample data traffic source, may include, but is not limited to, manual allocation of actual traffic scenarios, or may be based on, but is not limited to, employing historical prior noise clue data to construct a plurality of superfine training sample data traffic sources as the set of training sample data traffic sources, where the set of training sample data traffic sources is composed of a plurality of adjacent superfine training sample data traffic sources (training sample data traffic sources other than the target training sample data traffic source).
For some possible design ideas, for the big data collection activity data of each training sample data service source, assuming that m1, m2, m3, m4 and m5 are collection activity members in a certain training sample data service source in a group of training sample data service sources, that is, the collection activity members are big data collection activities in the training sample data service source, and the activity cooperative relation information and the influence exerted by each other between different collection activity members in the training sample data service source are represented according to the collection activity member category between the collection activity members, for example, the collection activity member m1 only has activity cooperative relation information with the collection activity member m5, then the collection activity member m1 and the collection activity member m5 have mutual influence, and the collection activity member m5 has activity cooperative relation information with the collection activity member m2, the collection activity member m3 and the collection activity member m4 respectively, so that the collection activity member m5 can have mutual influence with the collection activity member m1, the collection activity member m2, the collection activity member m3 and the collection activity member m 4.
For some possible design ideas, it is assumed that in each training sample data service source in the set of training sample data service sources, a set of training sample data service sources may include, but is not limited to, training sample data service source 1, training sample data service source 2, …, training sample data service source m-1, and training sample data service source m, and each training sample data service source is correspondingly configured with a corresponding acquisition noise point decision model in the set of acquisition noise point decision models, so as to input the big data acquisition activity data and the historical prior noise clue data of the training sample data service source 1, the training sample data service source 2, …, the training sample data service source m-1, and the training sample data service source m into the corresponding acquisition noise point decision models respectively, and determine the set of acquisition noise point information.
For another example, it is assumed that the set of collected noise point decision models may include, but is not limited to, M1, M2, … Mn, and the large data collected activity data and the historical prior noise cue data of the training sample data service source 1, the training sample data service source 2, …, and the training sample data service source M are respectively input into the corresponding M1, M2, … Mn to obtain a set of collected noise point information T1, T2, …, tm generated by the corresponding set of collected noise point decision models.
According to the technical scheme, big data acquisition activity data of a derivative training sample data service source and historical prior noise clue data on the derivative training sample data service source are acquired, a model deployment application relation is preconfigured between the derivative training sample data service source and a derivative acquisition noise point decision model, the big data acquisition activity data of the derivative training sample data service source represents activity cooperative relation information between a derivative big data acquisition activity cluster corresponding to the derivative training sample data service source and big data acquisition activity in the derivative big data acquisition activity cluster, the big data acquisition activity data of the derivative training sample data service source and the historical prior noise clue data on the derivative training sample data service source are loaded to the derivative acquisition noise point decision model, derivative acquisition noise point information generated by the derivative acquisition noise point decision model is output, and based on the target acquisition noise point information, acquisition noise point information from a target acquisition initial target acquisition to a target acquisition termination target in an AI application service server is output, and the method comprises the steps of: based on the target acquisition noise point information and the derivative acquisition noise point information, acquiring noise point information from an initial target acquisition to a target acquisition termination target in the AI application service server is output, and an acquisition path from the initial target acquisition to the target acquisition termination target passes through the target training sample data service source, wherein the acquisition noise point information serves the mode of big data denoising optimization and the derivative training sample data service source, so that acquisition noise point information from the initial target acquisition to the target acquisition termination target is determined according to a group of acquisition noise point information, and accuracy of determining the acquisition noise point information is improved.
For some possible design ideas, the loading the big data acquisition activity data of the derived training sample data service source and the historical prior noise clue data on the derived training sample data service source into the derived acquisition noise point decision model, and outputting derived acquisition noise point information generated by the derived acquisition noise point decision model includes:
outputting noise clue related variables of derivative acquisition activity members included in the big data acquisition activity data of the derivative training sample data service source and noise clue related variables of derivative synergistic relationship categories included in the big data acquisition activity data of the derivative training sample data service source based on the big data acquisition activity data of the derivative training sample data service source and historical priori noise clue data on the derivative training sample data service source, wherein the derivative acquisition activity members have a one-to-one correspondence with the derivative big data acquisition activities in the derivative training sample data service source, the derivative acquisition activity members represent one big data acquisition activity corresponding to one big data acquisition activity in the derivative big data acquisition activity cluster, the derivative synergistic relationship categories connect the two derivative acquisition activity members, represent that a synergistic relationship exists between the two derivative big data acquisition activities corresponding to the derivative acquisition activity members, and the derivative big data acquisition activity is the big data acquisition activity in the derivative big data acquisition activity cluster and represents part of the training sample data service source in the derivative training sample data service source;
Outputting the noise transmission characteristics of the derived collection activity members based on the noise clue related variables of the derived collection activity members and the noise clue related variables of the derived cooperative relationship categories;
and outputting the derived acquisition noise point information based on the noise transmission characteristics of the derived acquisition active members.
For some possible design ideas, the noise clue related variables of the derived collection activity members may include, but are not limited to, historical prior noise clue data of the collection activity members, and the noise clue related variables of the derived cooperative relationship class characterize session bias business relationships among collection activity members in the derived collection activity members, influence factors imposed by each other among the collection activity members, and the like.
For some possible design ideas, the noise transfer characteristic of each collection activity member is obtained by the collection noise point decision model, and the noise transfer characteristic of each collection activity member includes information of a cooperative collection activity member from each collection activity member, and may also include, but is not limited to, information of a cooperative collection activity member of different noise transfer decision nodes.
For some possible design considerations, this may include, but is not limited to, iteratively updating the noise transfer characteristics of the derived acquisition activity members in accordance with the acquisition noise point decision model.
For some possible design ideas, the outputting the noise transfer characteristic of each of the derived collection activity members based on the noise cue correlation variable of the derived collection activity members and the noise cue correlation variable of the derived cooperative relationship class includes:
and outputting the noise transmission characteristics of the traversing collecting active member on the next noise transmission decision node based on the noise clue related variable of the traversing collecting active member, the noise clue related variable of the collecting active member category cooperated with the traversing collecting active member, the noise clue related variable of the cooperated collecting active member of the traversing collecting active member, and the noise transmission characteristics of the cooperated collecting active member of the traversing collecting active member on the current noise transmission decision node, wherein the target collecting active member comprises the traversing collecting active member and the cooperated collecting active member of the traversing collecting active member.
For some possible design ideas, taking the above traversal collection activity member as a collection activity member u, where the collection activity member u includes, but is not limited to, a collection activity member j3, and the collaborative collection activity member includes, as an example, a collection activity member j1, j2, and j4, where j1, j2, and j4 are all adjacent collection activity members of j3, the influence of different noise transmission decision nodes of different collaborative collection activity members on the collection activity member can be used to obtain an output result of the collection activity member after passing through the collection noise point decision model.
For example, the following functions may be used to obtain the output result:
oy=g(hy,jy)
where g is a local output function, g may also be expressed by a neural network, where the capability of the cooperative collection active member to affect the collection active member is expressed by a cooperation between different noise transfer decision nodes, for example, at the T1 noise transfer decision node, the state of the collection active member j3 accepts the noise transfer characteristics from the last noise transfer decision node of the collection active member j1, the collection active member j2, the collection active member j4, because the collection active member j1, the collection active member j2, the collection active member j4 are all adjacent to the collection active member j3 until the Tn noise transfer decision node, the noise transfer characteristics of the collection active members converge, and each collection active member adds the neural network g to obtain the output o1, o2, o3, o4 (corresponding to the target collection noise point information or one collection noise point information in the set of collection noise point information) of the corresponding collection active member.
According to the technical scheme, the noise clue related variable based on the traversing collecting active member, the noise clue related variable of the collecting active member category cooperated with the traversing collecting active member, the noise clue related variable of the cooperated collecting active member of the traversing collecting active member, the noise transfer characteristic of the cooperated collecting active member of the traversing collecting active member on the current noise transfer decision node are adopted, the noise transfer characteristic of the traversing collecting active member on the next noise transfer decision node is output, the target collecting active member comprises the modes of the traversing collecting active member and the cooperated collecting active member of the traversing collecting active member, the noise clue related variable of the cooperated collecting active member u and the noise transfer characteristic of the last noise transfer decision node of the cooperated collecting active member are input into a collecting noise point decision model, and the influence of the cooperated collecting active member of each collecting active member on the collecting active member is also used as input according to the noise transfer characteristic of each collecting active member in the collecting noise point decision model, so that the generated noise point information is higher, and the accuracy of the noise point information of the collecting active member is greatly improved, and the accuracy of the data is ensured.
For some possible design ideas, the outputting, based on the set of collected noise point information, collected noise point information from a target collection initial target collection to a target collection termination target in the AI application service server includes:
outputting acquisition noise point information from the acquisition initial target of the derived training sample data traffic source in the set of training sample data traffic sources to the acquisition termination target of the derived training sample data traffic source as equal to corresponding one of the set of acquisition noise point information when the target acquisition initial target includes the acquisition initial target of the derived training sample data traffic source and the target acquisition termination target includes the acquisition termination target of the derived training sample data traffic source; and/or
Outputting acquisition noise point information acquired from the initial target acquisition to the termination target acquisition in the AI application service server as a set of corresponding part of acquisition noise point information in the set of acquisition noise point information when the initial target acquisition is the initial acquisition target of the set of training sample data service sources or the initial acquisition target of the set of training sample data service sources and a part of continuous training sample data service source in the target training sample data service sources is the termination target acquisition of the part of continuous training sample data service sources, wherein the part of continuous training sample data service sources and the part of acquisition noise point information have a one-to-one correspondence, and each piece of acquisition noise point information in the part of acquisition noise point information is the acquisition noise point information of a corresponding training sample data service source in the part of continuous training sample data service sources; and/or
And outputting acquisition noise point information acquired from the initial target acquisition in the AI application service server to the target acquisition termination target as a set of acquisition noise point information when the initial target acquisition is the initial acquisition target of the set of training sample data service sources, the target acquisition termination target is the acquisition termination target of the set of training sample data service sources, and the set of training sample data service sources are continuous training sample data service sources in the AI application service server.
For some possible design ideas, outputting the acquisition noise point information from the initial target acquisition of the derived training sample data service source to the end target acquisition of the derived training sample data service source to be equal to the corresponding one of the set of acquisition noise point information may include, but is not limited to, determining the acquisition noise point information corresponding to different training sample data service sources according to different training sample data service sources, respectively, and in a subsequent processing process, obtaining the corresponding acquisition noise point information from the initial target acquisition to the end target acquisition of the target acquisition in the AI application service server according to the acquisition noise point information combined with the different training sample data service sources based on the difference between the initial target acquisition and the end target acquisition of the target acquisition.
For some possible design ideas, the outputting the collection noise point information from the initial target collection in the AI application service server to the target collection termination target as the collection noise point information of the corresponding part of the collection noise point information in the group of collection noise point information may include, but is not limited to, obtaining the collection noise point information of different big data collection activities, and obtaining the collection noise point information of the big data collection activities from the initial target collection to the target collection termination target.
For some possible design ideas, the outputting the collection noise point information from the initial target collection in the AI application service server to the target collection termination target as the collection noise point information of the corresponding part of the collection noise point information in the group of collection noise point information may include, but is not limited to, obtaining the collection noise point information of different big data collection activities, and obtaining the collection noise point information of the big data collection activities from the initial target collection to the target collection termination target.
For some possible design ideas, the outputting, based on the set of collected noise point information, collected noise point information from a target collection initial target collection to a target collection termination target in the AI application service server includes:
and when the target acquisition initial target is the acquisition initial target of the group of training sample data service sources or the group of training sample data service sources and part of continuous big data acquisition activities in the target training sample data service sources, and the target acquisition termination target is the acquisition termination target in the part of continuous big data acquisition activities, outputting acquisition noise point information acquired from the target acquisition initial target to the target acquisition termination target in the AI application service server as a set of part of acquisition noise point information in the group of acquisition noise point information, wherein the part of continuous big data acquisition activities have a one-to-one correspondence with part of unit acquisition noise point information in the group of acquisition noise point information, and each unit acquisition noise point information in the part of unit acquisition noise point information is acquisition noise point information of a corresponding one of the part of continuous big data acquisition activities.
For some possible design ideas, the embodiment of the invention also provides a big data collection strategy updating method based on big data noise analysis, which comprises the following steps.
And a Process410 for outputting denoising optimization basic data of the AI application service server for the AI training sample service Process based on acquisition noise point information acquired from a plurality of acquisition initial targets to a plurality of acquisition termination targets in the AI application service server.
Illustratively, the artificial intelligence system may provide a denoising optimization service for AI training sample service processes corresponding to respective noise sources according to the AI application service server. The denoising optimization basic data comprises a noise source path of a noise source of an AI application service server, big data denoising nodes of big data denoising application of the AI application service server, noise point communication information between the noise source paths and denoising corresponding relations between the noise source paths and the big data denoising nodes, the noise point communication information represents noise characteristic point association information between the noise source of the AI application service server, and the denoising corresponding relations represent priori association relation information of the noise source of the AI application service server corresponding to the big data denoising application of the AI application service server. The noise point communication information comprises communication characteristic information between noise source paths, and the noise point communication information can be represented according to directed edge connecting lines between the noise source paths. The noise source path may be a noise source object in the denoising optimization base data. The big data denoising node may refer to each big data denoising member in the denoising optimization basic data, each big data denoising member may represent a big data denoising application or a code of the big data denoising application, and a denoising correspondence relationship between a noise source path and the big data denoising node may represent configuration data of a denoising correspondence rule of a corresponding noise source corresponding to a corresponding big data denoising node (big data denoising member/big data denoising application). The denoising corresponding relation between the noise source path and the big data denoising node can be expressed according to a directional edge connecting mode.
And the Process420 performs strategy updating reference on the big data denoising strategy sequence of each big data denoising application in the AI application service server based on the denoising optimization basic data to obtain a target big data denoising strategy of each big data denoising application of each noise source corresponding to the AI application service server.
And the Process430 updates the big data acquisition strategy of the AI training sample service Process according to the target big data denoising strategy.
For the Process410, for some possible design ideas, the obtained exemplary denoising optimization basic data may be first tuned and selected in a model parameter layer, and then the denoising optimization basic data is obtained based on the denoising optimization basic data after training, and exemplary embodiments include the following descriptions (1) - (7).
(1) Obtaining example denoising optimization basic data, wherein the example denoising optimization basic data comprises example noise source paths of noise sources of an example AI application service server, example big data denoising nodes of big data denoising application of the example AI application service server, example noise point communication information between the example noise source paths and example denoising corresponding relations between the example noise source paths and the example big data denoising nodes, the example noise point communication information represents noise characteristic point association information between the noise sources of the example AI application service server, and the example denoising corresponding relations represent priori association relation information of the noise sources of the example AI application service server corresponding to big data denoising application of the example AI application service server.
(2) And acquiring a priori trust big data denoising strategy sequence of the noise source of each example AI application service server.
(3) Outputting each example noise source path as an example noise coding feature and each big data denoising node as an example big data denoising feature according to the pre-specified feature extraction network.
(4) And outputting the sample priori trust big data denoising strategy sequence of the noise source of each sample AI application service server as the sample priori big data denoising trust characteristic according to the AI characteristic screening network. For example, the big data denoising characteristics of the target big data denoising nodes corresponding to each priori trust big data denoising strategy in the priori trust big data denoising strategy sequence can be obtained first; then, outputting the big data denoising characteristics of each target big data denoising node as target priori big data denoising trust characteristics according to the AI characteristic screening network; then, a preset denoising execution behavior cluster of each target big data denoising node is obtained, and a network is screened according to AI characteristics based on the preset denoising execution behavior clusters to generate corresponding derived denoising label field information; and finally, clustering the target priori big data denoising trust characteristics of the plurality of target big data denoising nodes corresponding to the priori trust big data denoising strategy sequences into the priori big data denoising trust characteristics based on the derived denoising label field information corresponding to each target big data denoising node.
(5) Based on the example noise coding feature of each example noise source path and the example big data denoising feature of each example big data denoising node, acquiring target feature substitution value of the pre-specified feature extraction network and target cost weight information corresponding to the target feature extraction cost value.
(6) Based on the sample priori big data denoising trust characteristics of each sample priori trust big data denoising strategy sequence and the sample big data denoising characteristics of each sample big data denoising node, acquiring derivative characteristic substitution values of the AI characteristic screening network and derivative cost weight information corresponding to the derivative characteristic extraction cost values.
(7) And carrying out weighted fusion on the target feature extraction substitution value and the derivative feature extraction substitution value based on the target cost weight information and the derivative cost weight information to obtain weighted feature extraction substitution value, determining a training optimization basis based on the weighted feature extraction substitution value, carrying out iterative training on the pre-designated feature extraction network and the AI feature screening network based on the training optimization basis, and determining trained denoising optimization basic data.
According to the above, the obtaining of the denoising optimization basic data of the AI application service server based on the trained denoising optimization basic data may be achieved in the following manner.
Firstly, acquiring noise characteristic point association information of noise sources of each AI application service server in the AI application service servers, and determining a noise characteristic point association information sequence.
And then, acquiring the priori trust big data denoising strategy of the noise source of each AI application service server corresponding to big data denoising application in the AI application service server, and determining the priori trust big data denoising strategy sequence. Illustratively, when acquiring the denoising optimization basic data of the AI application service servers, noise feature point association information of a noise source of each AI application service server in the AI application service servers may be acquired, and a noise feature point association information sequence may be determined. In addition, the prior trust big data denoising strategy of the noise source of each AI application service server corresponding to the big data denoising application in the AI application service server is further obtained, and a prior trust big data denoising strategy sequence is determined, for example, the prior trust big data denoising strategy sequence can be output as set= { (Opj, tj) |p e P }, ouj represents the prior trust big data denoising strategy executed by the noise source P of the AI application service server, tj represents the execution time of the prior trust big data denoising strategy, and P represents all the noise sources in the AI application service server.
And finally, inputting the noise characteristic point association information sequence and the priori trust big data denoising strategy sequence into the trained denoising optimization basic data to generate the denoising optimization basic data of the artificial intelligent system aiming at the AI application service server.
Including four noise source paths (actually much larger than four, for convenience of example only herein), noise source path a of noise source a, noise source path b of noise source b, noise source path c of noise source c, and noise source path d of noise source d of the AI application service server, respectively. Further, it is assumed that four big data denoising application objects are also included, which are respectively a big data denoising node a of the big data denoising application a, a big data denoising node B of the big data denoising application B, a big data denoising node C of the big data denoising application C, and a big data denoising node D of the big data denoising application D of the corresponding AI application service server. The noise point connection information between the noise source paths represents noise characteristic point association information between the noise sources of the corresponding AI application service server. The denoising corresponding relation between the noise source path and the big data denoising node represents priori association relation information of the corresponding noise source corresponding to big data denoising application.
For some possible design ideas, in the above Process420, based on the denoising optimization basic data, performing policy update referencing on a big data denoising policy sequence of each big data denoising application in an AI application service server to obtain a target big data denoising policy of each big data denoising application of each noise source corresponding to the AI application service server, and an exemplary embodiment includes the following description of processes 421-S225.
And a Process421 outputting a noise source path of a noise source of the AI application service server as a noise coding feature according to a pre-specified feature extraction network, and outputting each big data denoising node as a big data denoising feature.
For example, a pre-designated feature extraction network for completing training may be preset in the artificial intelligence system, and the pre-designated feature extraction network is configured to perform feature learning of denoising optimization basic data on a noise source path and a big data denoising application object, and convert the noise source path and the big data denoising application feature into corresponding big data denoising trust features. Similar to feature learning of the deep neural network, the aim is to keep the feature information in the denoising optimization basic data as far as possible. Correspondingly, according to the preassigned feature extraction network of the matching network convergence requirement, the noise source path of the noise source of the AI application service server in the noise removal optimization basic data is subjected to noise removal optimization basic data feature learning, and the noise source path is output as an updated coding feature and recorded as a noise coding feature. In addition, the large data denoising nodes applied to the denoising optimization basic data in the AI application service server can be subjected to denoising optimization basic data feature learning according to the pre-designated feature extraction network, and the large data denoising nodes are output as an updated coding feature and recorded as large data denoising features.
And the Process422 acquires the priori trust big data denoising strategy sequence of each noise source of the AI application service server, and outputs the priori trust big data denoising strategy sequence as the priori big data denoising trust characteristic according to the AI characteristic screening network.
And a Process423, based on the priori big data denoising trust characteristics of the priori trust big data denoising policy sequence, the noise coding characteristics of the noise source paths and the big data denoising characteristics of each big data denoising node, performing policy update reference on the big data denoising policy sequence of each big data denoising application of each noise source of the AI application service server corresponding to the AI application service server according to a preset target policy update reference template to obtain a target big data denoising policy. For example, the specific implementation steps of the target big data denoising strategy can be as follows: and for each target noise source, extracting an associated coding feature which is associated with the target noise coding feature of the target noise source from the noise coding feature, extracting a target trust feature which is matched with the associated coding feature from the priori big data denoising trust feature, acquiring an aggregate feature of the target trust feature and the big data denoising feature of each big data denoising node, and carrying out policy updating reference on a big data denoising policy sequence applied by each big data denoising of the AI application service server according to the aggregate feature to generate the target big data denoising policy.
And a Process424, based on the priori big data denoising trust characteristics of the priori trusted big data denoising policy sequence and the big data denoising characteristics of each big data denoising node, performing policy updating reference on the big data denoising policy sequence of each noise source corresponding to each big data denoising application according to a preset derived policy updating reference template to obtain a derived big data denoising policy. For example, the specific implementation steps of generating the derived big data denoising strategy can be as follows: and extracting all big data denoising trust characteristics matched with the target big data denoising characteristics from the priori big data denoising trust characteristics aiming at the target big data denoising characteristics with denoising corresponding relation with any target noise source, and carrying out strategy updating reference on big data denoising strategy sequences of each big data denoising application of the AI application service server according to all the big data denoising trust characteristics to generate derivative big data denoising strategies corresponding to the target noise source.
And (2) processing 425, determining a target big data denoising strategy of each big data denoising application of the noise source corresponding to the AI application service server based on the target big data denoising strategy and the derivative big data denoising strategy. For example, the target big data denoising strategy and each big data denoising strategy member in the derived big data denoising strategy may be sequenced according to the denoising effect index to generate a final target big data denoising strategy, or the target big data denoising strategy and the derived big data denoising strategy may be directly aggregated to obtain the final target big data denoising strategy, which is not particularly limited.
FIG. 2 illustrates a hardware architecture diagram of an artificial intelligence system 100 for implementing the AI analysis output system serving big data denoising optimization as described above, according to an embodiment of the invention, and as shown in FIG. 2, the artificial intelligence system 100 may include a processor 110, a machine-readable storage medium 120, a bus 130, and a communication unit 140.
The processor 110 may perform various suitable actions and processes based on programs stored in the machine-readable storage medium 120, such as the program instructions associated with the AI analysis output method for servicing big data denoising optimization as described in the foregoing embodiments. The processor 110, the machine-readable storage medium 120, and the communication unit 140 communicate signals over the bus 130.
In particular, the processes described in the above exemplary flowcharts may be implemented as computer software programs, in accordance with embodiments of the present invention. For example, embodiments of the present invention include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication unit 140, which, when executed by the processor 110, performs the above-described functions defined in the method of the embodiment of the invention.
Still another embodiment of the present invention provides a computer readable storage medium having stored therein computer-executable instructions that when executed by a processor are configured to implement the AI analysis output method for serving big data denoising optimization as described in any of the above embodiments.
The computer readable medium of the present invention may be a computer readable signal medium, a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (LAM), a read-only memory (LOM), an erasable programmable read-only memory (EPLOM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-LOM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, LM (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.
The computer-readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above-described embodiments.
Yet another embodiment of the present invention provides a computer program product comprising a computer program which, when executed by a processor, implements an AI analysis output method serving big data denoising optimization as described in any of the above embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.

Claims (10)

1. A method for updating big data collection policies for AI training sample services, the method performed by an artificial intelligence system, the method comprising:
based on collected noise point information collected from a plurality of collected initial targets to a plurality of collected termination targets in the AI application service server, outputting denoising optimization basic data of the AI application service server aiming at the AI training sample service process, wherein the denoising optimization basic data comprises noise source paths of noise sources of the AI application service server, big data denoising nodes of big data denoising application of the AI application service server, noise point connection information among the noise source paths, and denoising corresponding relations between the noise source paths and the big data denoising nodes, the noise point connection information represents noise characteristic point association information among the noise sources of the AI application service server, the denoising corresponding relation represents priori association relation information of the noise sources of the AI application service server corresponding to big data denoising application of the AI application service server, the noise point connection information comprises communication characteristic information among the noise source paths, the noise point connection information is represented according to directed edges among the noise source paths, the noise source paths are noise source objects in the denoising optimization basic data, the big data denoising nodes refer to noise source feature point association information among the noise source paths, the big data denoising nodes represent the big data corresponding to the big data denoising nodes, and the big data node association information represents the big data node association information among the big data denoising nodes, and the big data node association information represents the big data node association relation among the big data nodes;
Based on the denoising optimization basic data, performing strategy updating reference on big data denoising strategy sequences of all big data denoising applications in an AI application service server to obtain target big data denoising strategies of all big data denoising applications of all noise sources corresponding to the AI application service server;
and updating the big data acquisition strategy of the AI training sample service process according to the target big data denoising strategy.
2. The big data collection policy update method for AI training sample services of claim 1, further comprising:
acquiring big data acquisition activity data of an AI training sample service process of the big data acquisition server on a target training sample data service source and historical priori noise clue data on the target training sample data service source, wherein a model deployment application relation is preconfigured between the target training sample data service source and a target acquisition noise point decision model, the target training sample data service source is a training sample data service source in an AI application service server, and the big data acquisition activity data of the target training sample data service source represents activity cooperative relation information between a target big data acquisition activity cluster corresponding to the target training sample data service source and target big data acquisition activity in the target big data acquisition activity cluster;
Loading big data acquisition activity data of the target training sample data service source and historical priori noise clue data on the target training sample data service source to the target acquisition noise point decision model, and determining target acquisition noise point information generated by the target acquisition noise point decision model;
and outputting acquisition noise point information from an initial target acquisition to a termination target acquisition of the target acquisition in the AI application service server based on the target acquisition noise point information, wherein an acquisition path from the initial target acquisition to the termination target acquisition of the target acquisition is optimized by using the target training sample data service source, and the acquisition noise point information is used for denoising big data.
3. The big data collection policy update method for AI training sample services of claim 2, further comprising:
acquiring big data acquisition activity data of a derivative training sample data service source and historical priori noise clue data on the derivative training sample data service source, wherein a model deployment application relation is preconfigured between the derivative training sample data service source and a derivative acquisition noise point decision model, and the big data acquisition activity data of the derivative training sample data service source represents activity cooperative relation information between a derivative big data acquisition activity cluster corresponding to the derivative training sample data service source and big data acquisition activities in the derivative big data acquisition activity cluster;
Loading big data acquisition activity data of the derivative training sample data service source and historical priori noise clue data on the derivative training sample data service source to the derivative acquisition noise point decision model, and outputting derivative acquisition noise point information generated by the derivative acquisition noise point decision model;
the outputting, based on the target acquisition noise point information, acquisition noise point information from the initial target acquisition to the termination target acquisition of the target acquisition in the AI application service server, includes:
and outputting acquisition noise point information from an initial target acquisition to a target acquisition termination target in the AI application service server based on the target acquisition noise point information and the derivative acquisition noise point information, wherein an acquisition path from the initial target acquisition to the target acquisition termination target is transmitted through the target training sample data service source, and the acquisition noise point information serves big data denoising optimization and the derivative training sample data service source.
4. The method for updating big data collection policy for AI training sample service according to claim 2, wherein performing policy update referencing on big data denoising policy sequences of big data denoising applications in AI application service servers based on the denoising optimization basic data to obtain target big data denoising policies of the noise sources corresponding to the big data denoising applications of the AI application service servers, comprises:
Outputting a noise source path of a noise source of the AI application service server as a noise coding feature according to a pre-designated feature extraction network, and outputting each big data denoising node as a big data denoising feature;
acquiring a priori trust big data denoising strategy sequence of each noise source of the AI application service server, and outputting the priori trust big data denoising strategy sequence as a priori big data denoising trust characteristic according to an AI characteristic screening network;
based on the priori big data denoising trust characteristics of the priori trust big data denoising policy sequence, the noise coding characteristics of the noise source paths and the big data denoising characteristics of each big data denoising node, performing policy updating reference on the big data denoising policy sequence of each noise source of the AI application service server corresponding to each big data denoising application of the AI application service server according to a preset target policy updating reference template to obtain a target big data denoising policy;
based on the priori big data denoising trust characteristics of the priori trusted big data denoising policy sequence and the big data denoising characteristics of each big data denoising node, performing policy update reference on the big data denoising policy sequence of each noise source corresponding to each big data denoising application according to a preset derived policy update reference template to obtain a derived big data denoising policy;
Determining a target big data denoising strategy of each big data denoising application of the noise source corresponding to the AI application service server based on the target big data denoising strategy and the derived big data denoising strategy;
the performing policy update reference on the big data denoising policy sequence of each big data denoising application of each noise source of the AI application service server corresponding to the AI application service server according to a preset target policy update reference template to obtain a target big data denoising policy, wherein the policy update reference comprises:
for each target noise source, extracting an associated coding feature associated with a target noise coding feature of the target noise source from the noise coding features, extracting a target trust feature matched with the associated coding feature from the priori big data denoising trust feature, acquiring an aggregate feature of the target trust feature and big data denoising features of each big data denoising node, and carrying out policy updating reference on a big data denoising policy sequence applied by each big data denoising of the AI application service server according to the aggregate feature to generate the target big data denoising policy;
The performing policy update reference on the big data denoising policy sequence of each noise source corresponding to each big data denoising application according to a preset derived policy update reference template based on the priori big data denoising trust characteristics of the priori trust big data denoising policy sequence and the big data denoising characteristics of each big data denoising node, and the method comprises the following steps:
for target big data denoising features with denoising corresponding relations with any target noise source, extracting all big data denoising trust features matched with the target big data denoising features from the priori big data denoising trust features, and carrying out strategy updating reference on big data denoising strategy sequences of each big data denoising application of the AI application service server according to all the big data denoising trust features to generate derivative big data denoising strategies corresponding to the target noise source;
the determining, based on the target big data denoising policy and the derived big data denoising policy, the target big data denoising policy of each big data denoising application of the noise source corresponding to the AI application service server specifically includes:
and sequencing all big data denoising strategy members in the target big data denoising strategy and the derivative big data denoising strategy according to the denoising effect index to generate a final target big data denoising strategy, or directly aggregating the target big data denoising strategy and the derivative big data denoising strategy to obtain the target big data denoising strategy.
5. The method for updating big data collection policy for AI training sample services according to any of claims 1-4, wherein loading big data collection activity data of the target training sample data traffic source and historical prior noise clue data on the target training sample data traffic source to the target collection noise point decision model determines target collection noise point information generated by the target collection noise point decision model comprises:
outputting noise clue related variables of target acquisition activity members included in the big data acquisition activity data and noise clue related variables of target cooperative relationship categories included in the big data acquisition activity data based on the obtained big data acquisition activity data of the target training sample data service source and historical priori noise clue data on the target training sample data service source, wherein the target acquisition activity members and the target big data acquisition activity have characteristic mapping relationships, the target acquisition activity members represent a big data acquisition activity corresponding to one big data acquisition activity in the target big data acquisition activity cluster, the target cooperative relationship categories are connected with at least one pair of target acquisition activity members, represent that cooperative relationships exist between two target big data acquisition activities corresponding to at least one pair of target acquisition activity members, and represent part of training sample data service sources in the target training sample data service source;
Outputting the noise transfer characteristics of the target acquisition activity members based on the noise clue related variables of the target acquisition activity members and the noise clue related variables of the target cooperative relation categories;
and outputting the target acquisition noise point information based on the noise transmission characteristics of the target acquisition active members.
6. The method for updating a big data collection policy for AI training sample services of claim 5, wherein outputting the noise transfer characteristics of the target collection activity member based on the noise cue correlation variable of the target collection activity member and the noise cue correlation variable of the target cooperative relationship class comprises:
and outputting the noise transmission characteristics of the traversing collecting active member on the next noise transmission decision node corresponding to the current noise transmission decision node based on the noise clue related variable of the traversing collecting active member, the noise clue related variable of the collecting active member category cooperated with the traversing collecting active member, the noise clue related variable of the cooperated collecting active member of the traversing collecting active member, and the noise transmission characteristics of the cooperated collecting active member of the traversing collecting active member on the current noise transmission decision node, wherein the target collecting active member comprises the traversing collecting active member and the cooperated collecting active member of the traversing collecting active member.
7. The big data collection policy update method for AI training sample services of claim 1, further comprising:
based on the sample big data acquisition activity data of the sample training sample data service source, the sample history priori noise clue data on the sample training sample data service source and the sample acquisition noise point information of the sample training sample data service source, performing model parameter layer tuning and selection on the sample acquisition noise point decision model configured with initialization model parameter information, determining the target acquisition noise point decision model, and outputting the sample acquisition noise point decision model to terminate tuning and selection of the model parameter layer and outputting the sample acquisition noise point decision model to be the target acquisition noise point decision model when the noise point decision cost value between the sample acquisition noise point information generated by the sample acquisition noise point decision model and the sample acquisition noise point information matches a target cost value condition.
8. The big data collection strategy updating method for AI training sample service of claim 1, wherein outputting collection noise point information from a target collection initial target collection to a target collection termination target in the AI application service server based on the target collection noise point information comprises:
When the target acquisition initial target is the acquisition initial target of the target training sample data service source and the target acquisition termination target is the acquisition termination target of the target training sample data service source, outputting acquisition noise point information from the acquisition initial target of the target training sample data service source to the acquisition termination target of the target training sample data service source as the target acquisition noise point information;
and/or when the target acquisition initial target is the acquisition initial target of the target big data acquisition activity and the target acquisition termination target is the acquisition termination target of the target big data acquisition activity, outputting acquisition noise point information from the acquisition initial target of the target big data acquisition activity to the acquisition termination target of the target big data acquisition activity as the acquisition noise point information of the target big data acquisition activity;
and/or when the target acquisition initial target is the acquisition initial target of the large data acquisition activities, and the target acquisition termination target is the acquisition termination target of the large data acquisition activities, outputting acquisition noise point information from the acquisition initial target of the large data acquisition activities to the acquisition termination target of the large data acquisition activities into an acquisition noise point information sequence of the large data acquisition activities.
9. The big data collection policy update method for AI training sample services of claim 1, further comprising:
and calculating characteristic loss of the target acquisition noise point information and the sample acquisition noise point information of the target training sample data service source so as to perform tuning and selection of a modulus parameter layer on the target acquisition noise point decision model, and performing tuning and selection of the modulus parameter layer on the target acquisition noise point decision model when the noise point decision cost value between the target acquisition noise point information and the sample acquisition noise point information is matched with a target cost value condition.
10. An artificial intelligence system, characterized in that it comprises a processor and a memory for storing a computer program capable of running on the processor, said processor being adapted to execute the big data acquisition strategy updating method for AI training sample services of any of claims 1-9 when said computer program is run.
CN202310216287.0A 2022-05-25 2022-05-25 Big data acquisition strategy updating method and system for AI training sample service Withdrawn CN116049679A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310216287.0A CN116049679A (en) 2022-05-25 2022-05-25 Big data acquisition strategy updating method and system for AI training sample service

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210571942.XA CN114840513B (en) 2022-05-25 2022-05-25 AI analysis output method and artificial intelligence system for denoising optimization of big data
CN202310216287.0A CN116049679A (en) 2022-05-25 2022-05-25 Big data acquisition strategy updating method and system for AI training sample service

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN202210571942.XA Division CN114840513B (en) 2022-05-25 2022-05-25 AI analysis output method and artificial intelligence system for denoising optimization of big data

Publications (1)

Publication Number Publication Date
CN116049679A true CN116049679A (en) 2023-05-02

Family

ID=82572256

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202210571942.XA Active CN114840513B (en) 2022-05-25 2022-05-25 AI analysis output method and artificial intelligence system for denoising optimization of big data
CN202310216287.0A Withdrawn CN116049679A (en) 2022-05-25 2022-05-25 Big data acquisition strategy updating method and system for AI training sample service

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN202210571942.XA Active CN114840513B (en) 2022-05-25 2022-05-25 AI analysis output method and artificial intelligence system for denoising optimization of big data

Country Status (1)

Country Link
CN (2) CN114840513B (en)

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018236674A1 (en) * 2017-06-23 2018-12-27 Bonsai Al, Inc. For hiearchical decomposition deep reinforcement learning for an artificial intelligence model
CN110324170B (en) * 2018-03-30 2021-07-09 华为技术有限公司 Data analysis equipment, multi-model co-decision system and method
CN113326427A (en) * 2020-11-17 2021-08-31 崔海燕 Service push configuration updating method based on big data positioning and cloud computing center
CN112711578B (en) * 2020-12-30 2021-09-21 深圳市全景网络有限公司 Big data denoising method for cloud computing service and cloud computing financial server
CN112988845B (en) * 2021-04-01 2021-11-16 湖南机械之家信息科技有限公司 Data information processing method and information service platform in big data service scene

Also Published As

Publication number Publication date
CN114840513A (en) 2022-08-02
CN114840513B (en) 2023-07-14

Similar Documents

Publication Publication Date Title
Sonnerat et al. Learning a large neighborhood search algorithm for mixed integer programs
Gee et al. Online diversity assessment in evolutionary multiobjective optimization: A geometrical perspective
CN114697128B (en) Big data denoising method and big data acquisition system through artificial intelligence decision
CN111797327B (en) Social network modeling method and device
CN114880314B (en) Big data cleaning decision-making method applying artificial intelligence strategy and AI processing system
US20230289618A1 (en) Performing knowledge graph embedding using a prediction model
CN114691665A (en) Big data analysis-based acquisition noise point mining method and big data acquisition system
CN114840513B (en) AI analysis output method and artificial intelligence system for denoising optimization of big data
Nguyen et al. A genetic programming approach for evolving variable selectors in constraint programming
CN112541556A (en) Model construction optimization method, device, medium, and computer program product
CN114896502B (en) User demand decision method applying AI and big data analysis and Internet system
CN115422486B (en) Cloud service online page optimization method based on artificial intelligence and big data system
CN114168966B (en) Big data analysis-based security protection upgrade mining method and information security system
Miranda et al. A novel context-free grammar for the generation of pso algorithms
CN114678114A (en) Big data mining evaluation method and big data mining system applied to intelligent medical treatment
CN115829717A (en) Wind control decision rule optimization method, system, terminal and storage medium
Castro et al. A Gaussian artificial immune system for multi-objective optimization in continuous domains
CN114978765A (en) Big data processing method serving information attack defense and AI attack defense system
CN116186016A (en) Training data cleaning method and system for AI training task
CN112149623B (en) Self-adaptive multi-sensor information fusion system, method and storage medium
Chandra Multi-objective cooperative neuro-evolution of recurrent neural networks for time series prediction
CN111027709B (en) Information recommendation method and device, server and storage medium
Zhao et al. AutoOpt: A general framework for automatically designing metaheuristic optimization algorithms with diverse structures
Achieng et al. Replacing Humans with Machines: Threats and Opportunities
Valerio et al. Optimising cost vs accuracy of decentralised analytics in Fog computing environments

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication
WW01 Invention patent application withdrawn after publication

Application publication date: 20230502